Skip to main content

Module sql_migration

Module sql_migration 

Source

Constants§

LATEST_DATA_LAKE_SCHEMA_VERSION
The latest schema version for the data lake.

Functions§

execute_migration
Executes the database migration.
read_data_lake_schema_version
Reads the current schema version from the database.
upgrade_data_lake_schema_v2
Upgrades the data lake schema to version 2.
upgrade_data_lake_schema_v3
Upgrades the data lake schema to version 3. Drops old non-unique indexes (superseded by the unique indexes created before this transaction).
upgrade_data_lake_schema_v4
Upgrades the data lake schema to version 4. Adds the format column to streams so OTLP and native blocks can be distinguished.
upgrade_data_lake_schema_v5
Upgrades the data lake schema to version 5. Adds ingestion_api_keys and analytics_api_keys, the DB-backed API key store: a key_id UUID primary key (a non-secret handle for DELETE/GET, distinct from the key_hash lookup value), a SHA-256 key_hash with a unique index for O(1) validation lookups, and a created_at/created_by/last_used_at/ revoked_at/revoked_by audit trail. The two tables are identical in shape but deliberately separate (never a shared scopes column) so an ingestion (write) key can never double as an analytics (read) credential.
upgrade_data_lake_schema_v6
Upgrades the data lake schema to version 6. Adds the audience column to ingestion_api_keys: the write audience a key is immutably bound to. Backfilled to 'public' before SET NOT NULL – every pre-existing row predates audience stamping, and public is the accurate description of its current, unstamped-and-visible-to-everyone state, not a new grant. No DEFAULT on the column: that would let a not-yet-upgraded analytics-web-srv keep inserting rows that silently take public, defeating the column’s fail-closed property. analytics_api_keys is untouched – its read-side mirror is read_audiences, a set-valued grant in the opposite direction.
upgrade_data_lake_schema_v7
Upgrades the data lake schema to version 7. Adds audience_grants: the DB-backed audience grant store, a 1:1 stand-in for the long-term model’s group_read_grants/group_mint_grants tables. One table with an axis column ('read'/'mint') rather than two, mirroring today’s single env-var grant map ({prefix}_AUDIENCE_GRANTS); (audience, axis, selector) is the row’s own natural key, so there is no surrogate grant_id the way ingestion_api_keys.key_id has one (a grant carries no secret to keep unlinkable from its own identity). No revoked_at/revoked_by either – grants are hard-DELETEd, since a removed grant has no ongoing artifact whose provenance a caller might later need. The two CHECK constraints mirror the same validation AudienceGrants::from_rows (rust/auth/src/policy.rs) re-runs independently in Rust, so a row inserted by any means other than the admin API (a manual psql fix, a future migration) still can’t produce an unparseable or unreadable grant silently.
upgrade_data_lake_schema_v8
Upgrades the data lake schema to version 8.
upgrade_data_lake_schema_v9
Upgrades the data lake schema to version 9.
upgrade_data_lake_schema_v10
Upgrades the data lake schema to version 10.
upgrade_data_lake_schema_v11
Upgrades the data lake schema to version 11. Adds a nullable allowed_cidrs TEXT[] column to both ingestion_api_keys and analytics_api_keys: the CIDR ranges/bare IPs a key may be presented from. No backfill needed (unlike audience, which had to become NOT NULL): NULL/absent means “no restriction”, which is exactly the correct value for every pre-existing row.
warn_if_data_lake_schema_stale
Warns when the connected database is behind LATEST_DATA_LAKE_SCHEMA_VERSION.