Expand description
ExecutionPlan based on a set of parquet files
Enums§
- Ordering
Bounds - Which pair of bounds on
Partitiona declared ordering’s leading column is checked against. - Scan
Ordering - How a partition scan’s declared output ordering is realized.
Functions§
- assert_
ordering_ satisfied - Bails unless
plan’s output ordering satisfies the declaredcolumns, defensively re-verifying that asort_orderguarantee about to be recorded on a fresh partition is truthful.labelis a short noun phrase used in the intermediate error-context messages (e.g."per-file merge"or"extract-query");subjectnames the query for the bail message;reasonsupplies the full, call-site-specific trailing text of the bail message (what was declared, and any guidance for diagnosing a mismatch). Shared by the two paths that record asort_orderguarantee:QueryMerger::execute_sorted_mergeandsql_partition_spec::plan_sorted_extract.QueryMerger::execute_concatenated_mergedoes not call this – its ordering is a structural property of the sorted, non-overlapping file group rather than a query-plan sort DataFusion could get wrong. - assert_
single_ partition - Bails unless
planis a single-partition physical plan.subjectnames the query for the error message (e.g.format!("merge query {:?}", query)orformat!("extract query for {view}"));reasonsupplies the full, call-site-specific trailing sentence(s) explaining what a non-single-partition plan means here – executing such a plan would coalesce partitions and silently destroy a declared ordering before it is safe to record or execute. Shared by the query-execution paths that must verify this before executing:QueryMerger::execute_concatenated_merge(only when its ordering is declared),QueryMerger::execute_sorted_merge, andsql_partition_spec::plan_sorted_extract. - make_
lex_ ordering - Builds the
LexOrderingdeclaring the already-satisfied output ordering of the scan, matching DataFusion’s defaultORDER BYsemantics (ASC NULLS LAST unlessdescending). - make_
partitioned_ execution_ plan - Creates a partitioned execution plan for scanning Parquet files.
- sort_
exprs - Builds the
DataFrame::sortexpressions forcolumns, pairing each with!descendingforascso the logical sort matches thenulls_first: c.descendingphysical orderingmake_lex_orderingdeclares below – the two must stay in lockstep orassert_ordering_satisfiedstops matching the sort that was actually applied.