Skip to main content

regenerate_partition_range

Function regenerate_partition_range 

Source
pub fn regenerate_partition_range(
    existing_partitions_all_views: Arc<PartitionCache>,
    lakehouse: Arc<LakehouseContext>,
    view: Arc<dyn View>,
    insert_range: TimeRange,
    partition_time_delta: TimeDelta,
    logger: Arc<dyn Logger>,
) -> impl Future<Output = Result<(), Error>>
Expand description

Regenerates the partition(s) covering insert_range directly from source data, bypassing the “already up to date” freshness check materialize_partition_range would otherwise stop at. See tasks/blocks_view_ordered_merges_plan.md’s Design §3.

Invariant callers must uphold: (begin, end, delta) must exactly cover the boundaries of the partition(s) being regenerated – a misaligned range/delta means the new partition’s range does not fully contain the old one, so retire_partitions never retires it, leaving silent duplicate rows. This is enforced by validating that delta exactly tiles (begin, end) before any partition is written, and (per bucket) by verify_force_regeneration_alignment.

Both checks read a snapshot and are advisory UX: the authoritative guard is the lakehouse_partitions_no_overlap exclusion constraint, which makes the insert fail loudly if a conflicting partition was committed concurrently (e.g. by the maintenance daemon merging buckets after the snapshot was taken).