Struct ExportLogView 

pub struct ExportLogView { /* private fields */ }

A view for exporting log data.

Implementations

impl ExportLogView

pub async fn new( runtime: Arc<RuntimeEnv>, view_set_name: Arc<String>, count_src_query: Arc<String>, extract_query: Arc<String>, exporter: Arc<dyn RecordBatchTransformer>, lake: Arc<DataLakeConnection>, view_factory: Arc<ViewFactory>, session_configurator: Arc<dyn SessionConfigurator>, update_group: Option<i32>, max_partition_delta_from_source: TimeDelta, max_partition_delta_from_merge: TimeDelta, ) -> Result<ExportLogView, Error>

Trait Implementations

impl Debug for ExportLogView

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl View for ExportLogView

fn get_view_set_name(&self) -> Arc<String>

name of the table from the user’s perspective

fn get_view_instance_id(&self) -> Arc<String>

get_view_instance_id can be a process_id, a stream_id or ‘global’.

fn make_batch_partition_spec<'life0, 'async_trait>( &'life0 self, lakehouse: Arc<LakehouseContext>, existing_partitions: Arc<PartitionCache>, insert_range: TimeRange, ) -> Pin<Box<dyn Future<Output = Result<Arc<dyn PartitionSpec>, Error>> + Send + 'async_trait>>

where 'life0: 'async_trait, ExportLogView: 'async_trait,

make_batch_partition_spec determines what should be found in an up to date partition. The resulting PartitionSpec can be used to validate existing partitions are create a new one.

fn get_file_schema_hash(&self) -> Vec<u8>

get_file_schema_hash returns a hash (can be a version number, version string, etc.) that allows to identify out of date partitions.

fn get_file_schema(&self) -> Arc<Schema>

get_file_schema returns the schema of the partition file in object storage

fn jit_update<'life0, 'async_trait>( &'life0 self, _lakehouse: Arc<LakehouseContext>, _query_range: Option<TimeRange>, ) -> Pin<Box<dyn Future<Output = Result<(), Error>> + Send + 'async_trait>>

where 'life0: 'async_trait, ExportLogView: 'async_trait,

jit_update creates or updates process-specific partitions before a query

fn make_time_filter( &self, begin: DateTime<Utc>, end: DateTime<Utc>, ) -> Result<Vec<Expr>, Error>

make_time_filter returns a set of expressions that will filter out the rows of the partition outside the time range requested.

fn get_time_bounds(&self) -> Arc<dyn DataFrameTimeBounds>

fn get_update_group(&self) -> Option<i32>

tells the daemon which view should be materialized and in what order

fn get_max_partition_time_delta( &self, strategy: &PartitionCreationStrategy, ) -> TimeDelta

allow the view to subdivide the requested partition

fn register_table<'life0, 'life1, 'async_trait>( &'life0 self, ctx: &'life1 SessionContext, table: MaterializedView, ) -> Pin<Box<dyn Future<Output = Result<(), Error>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, Self: 'async_trait,

register the table in the SessionContext

fn merge_partitions<'life0, 'async_trait>( &'life0 self, lakehouse: Arc<LakehouseContext>, partitions_to_merge: Arc<Vec<Partition>>, partitions_all_views: Arc<PartitionCache>, insert_range: TimeRange, ) -> Pin<Box<dyn Future<Output = Result<MergeQueryResult, Error>> + Send + 'async_trait>>

where 'life0: 'async_trait, Self: 'async_trait,

fn get_merged_partition_sort_order( &self, _partitions_to_merge: &[Partition], ) -> Option<Vec<String>>

Returns the sort guarantee a merge of partitions_to_merge will actually produce, to be recorded as the resulting partition’s Partition::sort_order (see merge::create_merged_partition).

fn get_scan_output_ordering(&self) -> Vec<ScanSortColumn>

Declares an ordering the view’s partition scan already emits, letting DataFusion elide redundant Sort nodes for queries that ORDER BY these columns.