Skip to main content

make_partitioned_execution_plan

Function make_partitioned_execution_plan 

Source
pub fn make_partitioned_execution_plan(
    schema: Arc<Schema>,
    reader_factory: Arc<ReaderFactory>,
    state: &dyn Session,
    projection: Option<&Vec<usize>>,
    filters: &[Expr],
    limit: Option<usize>,
    partitions: Arc<Vec<Partition>>,
    scan_ordering: &ScanOrdering,
) -> Result<Arc<dyn ExecutionPlan>, DataFusionError>
Expand description

Creates a partitioned execution plan for scanning Parquet files.

scan_ordering declares how the scan’s already-satisfied ordering (see View::get_scan_output_ordering), if any, is realized:

  • Unordered: no ordering is declared to DataFusion.
  • Concatenated { columns, bounds }: the file group is sorted by the leading column’s bound (read per bounds) and checked for non-overlap (erroring if violated), per-file min/max statistics are attached so DataFusion accepts the declared ordering, and the ordering is attached to the resulting FileScanConfig so EnsureRequirements can elide a redundant Sort node.
  • PerFile { columns }: gated by Partition::certifies_sort_order – if every non-empty partition’s recorded sort_order certifies columns, each non-empty partition becomes its own single-file group (no overlap check, no per-file statistics – DataFusion’s multi-file-group ordering validation only needs those to prove cross-file order within a group, and single-file groups pass trivially), all declaring the same columns ordering for a downstream SortPreservingMergeExec. If any non-empty partition fails to certify, this degrades to the same plan shape as Unordered.