Skip to main content

fetch_process_blocks

Function fetch_process_blocks 

Source
pub async fn fetch_process_blocks(
    lakehouse: Arc<LakehouseContext>,
    blocks_view: &BlocksView,
    partitions: &PartitionCache,
    range: &TimeRange,
    process: Arc<ProcessMetadata>,
    stream_tag: &str,
    stream_metadata: &HashMap<Uuid, (Arc<StreamMetadata>, String)>,
) -> Result<Vec<Arc<PartitionSourceBlock>>, Error>
Expand description

Fetches and parses one batch window’s worth of process-scoped blocks (process_batch_sql), looking up each block’s stream metadata in stream_metadata (built once per call to generate_process_jit_partitions by fetch_stream_metadata_map) rather than rebuilding it per row – this is the lean projection’s fetch-and-parse half.

A stream_id missing from stream_metadata is a hard error: the metadata pre-query and this query share the same identity predicate and insert range, so it cannot happen unless something is wrong.