Skip to main content

daemon

Function daemon 

Source
pub async fn daemon<F>(
    lakehouse: Arc<LakehouseContext>,
    views_to_update: Vec<Arc<dyn View>>,
    retention_days: i32,
    shutdown: F,
    grace: Duration,
) -> Result<()>
where F: Future<Output = ()> + Send + 'static,
Expand description

Starts the maintenance daemon, which runs various scheduled tasks.

This function initializes and spawns several CronTasks for daily, hourly, minute, and second-based maintenance operations, such as data materialization and cleanup, plus a once-a-minute collector that samples the metadata Postgres’s pg_stat_* views for self-observability. All runner loops react to shutdown: they stop scheduling and drain in-flight tasks. A deadline arm forces return after grace even if tasks haven’t drained.

§Arguments

  • lakehouse - The lakehouse context with shared metadata cache.
  • views_to_update - A vector of views that need to be updated by the daemon.
  • retention_days - Delete lake data older than this many days (retention horizon).
  • shutdown - Future that completes when the process should begin shutting down.
  • grace - Maximum time to wait for in-flight tasks after the shutdown signal.