Skip to main content

find_up_to_date_partitions

Function find_up_to_date_partitions 

Source
pub fn find_up_to_date_partitions(
    pool: &Pool<Postgres>,
    view_meta: ViewMetadata,
    block_order: BlockOrder,
    specs: &[SourceDataBlocksInMemory],
) -> impl Future<Output = Result<Vec<bool>, Error>>
Expand description

One candidates fetch over [specs.first().min, specs.last().max] (specs have ascending, non-overlapping insert ranges), then the matcher per spec, run to a fixpoint. An empty specs returns Ok(vec![]) without issuing any fetch – reachable at every call site, since generate_*_jit_partitions returns no specs when the range holds no blocks.

The first pass runs spec_is_up_to_date for every spec against the full candidate set. After that, only a worklist of specs whose verdict might have changed is re-evaluated: because specs have ascending, non-overlapping insert ranges, a candidate row can be entirely contained in at most two (necessarily adjacent) specs’ ranges, so when spec i newly becomes stale, the only other specs whose verdict can possibly change as a result are i’s immediate neighbors in the sorted order (index i - 1 and i + 1) – no non-adjacent spec’s verdict can ever be affected. A re-evaluated spec j is checked against a candidate set that drops any row entirely contained in the insert range of some other, currently-stale spec i (i != j) – such a row is a RetireMatch::Containment match for spec i and will be gone once i’s write runs this call, so it must not count towards j’s freshness. Spec j’s own range is always excluded from that drop: a row j itself would retire is not evidence about whether j’s own data is current, so it must stay in the set used to (re-)evaluate j. A spec, once found stale, is never recomputed again (so verdicts are monotone – up to date to stale only – by construction, with no separate clamping step), and only newly-stale specs push their neighbors onto the worklist. This bounds total work to at most specs.len() initial spec_is_up_to_date calls plus at most two more per spec (once per neighbor, if ever re-triggered) – O(specs), not O(specs^2) or O(specs^3). Returns up-to-date flags parallel to specs.