Authentication¶

Micromegas supports unified authentication across all services using both API keys and OpenID Connect (OIDC). This page covers identity only — what a caller may then read and write is Authorization.

Overview¶

Both the analytics server (flight-sql-srv) and ingestion server (telemetry-ingestion-srv) support two authentication methods:

OIDC (OpenID Connect) — for human users and service accounts via federated identity providers (Google, Azure AD, Okta, Auth0, etc.)
API Keys — bearer token authentication

Both methods can be enabled simultaneously. When multiple providers are configured, they are tried in order until one succeeds (OIDC first, then the DB-backed key store).

Authentication Methods¶

OIDC Authentication (Recommended)¶

OIDC provides federated authentication with automatic token refresh and support for multiple identity providers.

Benefits:

Standards-based authentication (OAuth 2.0 / OpenID Connect)
Centralized user management via identity provider
Automatic token refresh with no manual intervention
Support for multiple identity providers simultaneously
Full audit trail with user identity (email, subject)
Token revocation support via identity provider

Supported Identity Providers:

Google OAuth
Microsoft Azure AD
Okta
Auth0
Any standards-compliant OIDC provider

API Keys¶

Bearer token authentication. For ingestion and flight-sql, the only source is:

DB-backed keys — ingestion_api_keys / analytics_api_keys rows, validated by hash lookup. Minted, listed, and revoked over HTTP without a redeploy. See API Keys for the full reference.

object-cache-srv is the one exception: it has no DB connection by design, so it authenticates against a permanent env-var keyring instead — see Object Cache.

Benefits:

Simple to configure
Fast validation — cached hash lookup for DB-backed keys
No external identity provider dependency

Limitations:

No automatic expiration — this design adds revocation, not expiry; a DB-backed key with no revoked_at is valid indefinitely
Rotation is a manual operator action either way
No per-request user identity beyond the key's own name; DB-backed keys additionally record created_by/revoked_by

Server Configuration¶

OIDC Configuration¶

export MICROMEGAS_OIDC_CONFIG='{
  "issuers": [
    {
      "issuer": "https://accounts.google.com",
      "audience": "your-app-id.apps.googleusercontent.com"
    },
    {
      "issuer": "https://login.microsoftonline.com/{tenant}/v2.0",
      "audience": "api://your-api-id"
    }
  ],
  "jwks_refresh_interval_secs": 3600,
  "token_cache_size": 1000,
  "token_cache_ttl_secs": 300
}'

Configuration Fields:

Field	Description	Default
`issuers`	Array of OIDC issuer configurations	Required
`issuers[].issuer`	OIDC issuer URL	Required
`issuers[].audience`	Expected audience claim (client_id)	Required
`jwks_refresh_interval_secs`	JWKS cache TTL in seconds	3600
`token_cache_size`	Maximum validated tokens to cache	1000
`token_cache_ttl_secs`	Token cache TTL in seconds	300

Admin-ness — partition management and other admin SQL functions — is no longer an env var. It comes from membership in the reserved admins local group, managed with micromegas-groups or the Groups admin page. See Groups for the model, the v10 upgrade path, and the MICROMEGAS_ADMINS-family removal.

API Key Configuration¶

DB-backed keys. Mint an ingestion key with POST /api/ingestion-api-keys, or an analytics key with POST /api/analytics-api-keys — both on analytics-web-srv (OIDC + admin required). See API Keys for the full route reference and the mmk_-prefixed key shape.

object-cache-srv's env keyring. Not read by ingestion or flight-sql — it is object-cache-srv's permanent auth path (no DB access by design). See Object Cache for the MICROMEGAS_API_KEYS JSON shape it uses.

Disable Authentication (Development Only)¶

# Analytics server
flight-sql-srv --disable-auth

# Ingestion server
telemetry-ingestion-srv --disable-auth

Security Warning

Never disable authentication in production environments. This flag is intended only for local development and testing.

Authorization¶

Authentication establishes who is calling; authorization decides what they may read and write. Configuring any authentication method activates the authorization layer: queries are narrowed to the audiences the caller is granted, and a few lakehouse-management functions are gated on admin-ness.

That model — audience labels, the grant map, write-time stamping, self-service key mint — is documented separately in Authorization.

Client Configuration¶

Python Client with OIDC¶

The Python client supports automatic browser-based login with token persistence and refresh.

Interactive Use (Jupyter, Scripts)¶

from micromegas.auth import OidcAuthProvider
from micromegas.flightsql.client import FlightSQLClient

# First use: Opens browser for authentication
auth = OidcAuthProvider.login(
    issuer="https://accounts.google.com",
    client_id="your-app-id.apps.googleusercontent.com",
    client_secret="your-client-secret",  # Optional for some providers
    token_file="~/.micromegas/tokens.json"  # Persists tokens
)

# Create authenticated client
client = FlightSQLClient(
    "grpc+tls://analytics.example.com:50051",
    auth_provider=auth
)

# Run queries - tokens auto-refresh before expiration
df = client.query("SELECT * FROM processes LIMIT 10")

Parameters:

Parameter	Description	Default
`issuer`	OIDC issuer URL	Required
`client_id`	OAuth client ID	Required
`client_secret`	OAuth client secret	Optional (for public clients)
`token_file`	Path to save tokens	`~/.micromegas/tokens.json`
`audience`	API audience/identifier	Optional (required by Auth0)
`scope`	OAuth scopes to request	`openid email profile offline_access`

Subsequent Use (Token Reuse)¶

from micromegas.auth import OidcAuthProvider
from micromegas.flightsql.client import FlightSQLClient

# Load existing tokens - no browser interaction needed
auth = OidcAuthProvider.from_file(
    "~/.micromegas/tokens.json",
    client_secret="your-client-secret"  # Optional
)

client = FlightSQLClient(
    "grpc+tls://analytics.example.com:50051",
    auth_provider=auth
)

# Tokens automatically refresh when needed
import datetime
now = datetime.datetime.now(datetime.timezone.utc)
begin = now - datetime.timedelta(hours=1)
df = client.query("SELECT * FROM log_entries LIMIT 1000", begin, now)

Token Management¶

# Clear saved tokens (logout)
import os
from pathlib import Path

token_file = Path.home() / ".micromegas" / "tokens.json"
if token_file.exists():
    token_file.unlink()
    print("Logged out - tokens cleared")

CLI Tools with OIDC¶

CLI tools automatically support OIDC when environment variables are set:

# Configure OIDC
export MICROMEGAS_OIDC_ISSUER="https://accounts.google.com"
export MICROMEGAS_OIDC_CLIENT_ID="your-app-id.apps.googleusercontent.com"
export MICROMEGAS_OIDC_CLIENT_SECRET="your-client-secret"  # Optional
export MICROMEGAS_ANALYTICS_URI="grpc+tls://analytics.example.com:50051"

# First use: Opens browser for authentication
micromegas-query "SELECT process_id, exe, start_time FROM processes" --begin 1h

# Subsequent uses: No browser interaction, uses cached tokens
micromegas-query "SELECT time, level, msg FROM log_entries WHERE process_id = '<process_id>'" --begin 1h

# Logout (clear saved tokens)
micromegas-logout

Environment Variables:

Variable	Description	Required
`MICROMEGAS_OIDC_ISSUER`	OIDC issuer URL	Yes
`MICROMEGAS_OIDC_CLIENT_ID`	OAuth client ID	Yes
`MICROMEGAS_OIDC_CLIENT_SECRET`	OAuth client secret	No*
`MICROMEGAS_OIDC_AUDIENCE`	API audience/identifier	No (for Auth0, Azure API)
`MICROMEGAS_OIDC_SCOPE`	OAuth scopes to request	No (default: openid email profile offline_access)
`MICROMEGAS_PROFILE`	Named connection profile to select from `~/.micromegas/config.json`'s `profiles` map	No (see Named profiles)
`MICROMEGAS_ANALYTICS_URI`	Analytics server URI	No (default: grpc://localhost:50051)

*Required for some providers (e.g., Google) even with PKCE

Python Client with API Keys¶

Mint a static analytics API key (see Minting an analytics key over HTTP), store it in a file readable only by you (chmod 0600), and load it with StaticTokenAuthProvider:

from micromegas.flightsql.client import FlightSQLClient
from micromegas.auth import StaticTokenAuthProvider

auth = StaticTokenAuthProvider.from_file("~/.micromegas/local.key")
client = FlightSQLClient(
    "grpc://localhost:50051",
    auth_provider=auth
)

df = client.query("SELECT * FROM processes LIMIT 10")

The key travels verbatim as the bearer token, with no refresh and no OIDC flow. A named profile (~/.micromegas/config.json) can also source one via the api_key_file key — see Static Analytics API Keys.

Deprecated API

The headers parameter is deprecated. Use auth_provider with StaticTokenAuthProvider (a static API key) or OidcAuthProvider (OIDC) instead.

Ingestion Service Authentication¶

The telemetry ingestion service (telemetry-ingestion-srv) uses the same authentication infrastructure as the analytics service.

Server Configuration¶

# Start ingestion server with authentication
export MICROMEGAS_OIDC_CONFIG='{"issuers": [...]}'
telemetry-ingestion-srv

# Or disable auth for development
telemetry-ingestion-srv --disable-auth

The OTLP/HTTP routes (/ingestion/otlp/v1/{logs,metrics,traces}) share this authentication chain. OTel SDKs attach the bearer token via OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer <key>". See OTLP Ingestion for SDK-side configuration.

Rust Client Authentication¶

Rust applications sending telemetry can use either API keys or OIDC client credentials.

Automatic Configuration (Recommended)¶

Applications using #[micromegas_main] automatically configure authentication from environment variables:

use micromegas::micromegas_main;
use micromegas::tracing::prelude::*;

#[micromegas_main(interop_max_level = "info")]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    info!("Application starting");
    // Telemetry automatically authenticated based on environment variables
    Ok(())
}

Environment Variables:

Variable	Authentication Method	Required
`MICROMEGAS_INGESTION_API_KEY`	API key (simple)	For API key auth
`MICROMEGAS_OIDC_TOKEN_ENDPOINT`	OIDC client credentials	For OIDC auth
`MICROMEGAS_OIDC_CLIENT_ID`	OIDC client credentials	For OIDC auth
`MICROMEGAS_OIDC_CLIENT_SECRET`	OIDC client credentials	For OIDC auth
`MICROMEGAS_TELEMETRY_URL`	Ingestion server URL	Yes (e.g., http://localhost:9000)

Example (API Key):

export MICROMEGAS_INGESTION_API_KEY=secret-key-123
export MICROMEGAS_TELEMETRY_URL=http://localhost:9000
cargo run

Example (OIDC Client Credentials):

export MICROMEGAS_OIDC_TOKEN_ENDPOINT=https://accounts.google.com/o/oauth2/token
export MICROMEGAS_OIDC_CLIENT_ID=my-service@project.iam.gserviceaccount.com
export MICROMEGAS_OIDC_CLIENT_SECRET=secret-from-secret-manager
export MICROMEGAS_TELEMETRY_URL=http://localhost:9000
cargo run

Manual Configuration¶

For applications not using #[micromegas_main], configure authentication manually:

API Key Authentication (Simple)¶

use micromegas_telemetry_sink::http_event_sink::{HttpEventSink, HttpSinkConfig};
use micromegas_telemetry_sink::api_key_decorator::ApiKeyRequestDecorator;
use std::sync::Arc;

// From environment variable
std::env::set_var("MICROMEGAS_INGESTION_API_KEY", "secret-key-123");
let decorator = ApiKeyRequestDecorator::from_env().unwrap();

// Configure HttpEventSink with authentication
let sink = HttpEventSink::new(
    "http://localhost:9000",
    HttpSinkConfig::default(),
    Box::new(move || Arc::new(decorator.clone())),
);

OIDC Client Credentials (Production)¶

use micromegas_telemetry_sink::http_event_sink::{HttpEventSink, HttpSinkConfig};
use micromegas_telemetry_sink::oidc_client_credentials_decorator::OidcClientCredentialsDecorator;
use std::sync::Arc;

// Configure OIDC client credentials
std::env::set_var("MICROMEGAS_OIDC_TOKEN_ENDPOINT",
    "https://accounts.google.com/o/oauth2/token");
std::env::set_var("MICROMEGAS_OIDC_CLIENT_ID",
    "my-service@project.iam.gserviceaccount.com");
std::env::set_var("MICROMEGAS_OIDC_CLIENT_SECRET",
    "secret-from-secret-manager");

let decorator = OidcClientCredentialsDecorator::from_env().unwrap();

let sink = HttpEventSink::new(
    "http://localhost:9000",
    HttpSinkConfig::default(),
    Box::new(move || Arc::new(decorator.clone())),
);

Authentication Methods Comparison:

Method	Use Case	Token Lifetime	Complexity
API Key	Development, testing	No expiration	Low
Client Credentials	Production services	~1 hour (auto-refresh)	Medium

Health Endpoint¶

The /health endpoint remains public for monitoring and liveness checks, even when authentication is enabled.

# Health check always works without authentication
curl http://localhost:9000/health

Setting Up OIDC Providers¶

Google OAuth Setup¶

Go to Google Cloud Console
Create a new project or select existing
Navigate to APIs & Services → OAuth consent screen
Select "External" user type
Fill in app name and contact emails
Add test users (yourself and team members)
Navigate to APIs & Services → Credentials
Click "+ CREATE CREDENTIALS" → "OAuth client ID"
Application type: "Desktop app" (for CLI/local use)
Click "Create"
Copy both credentials:
Client ID (ends with .apps.googleusercontent.com)
Client Secret
Add authorized redirect URIs:
http://localhost:48080/callback

Server Configuration:

export MICROMEGAS_OIDC_CONFIG='{
  "issuers": [
    {
      "issuer": "https://accounts.google.com",
      "audience": "123-abc.apps.googleusercontent.com"
    }
  ]
}'

Client Configuration:

export MICROMEGAS_OIDC_ISSUER="https://accounts.google.com"
export MICROMEGAS_OIDC_CLIENT_ID="123-abc.apps.googleusercontent.com"
export MICROMEGAS_OIDC_CLIENT_SECRET="GOCSPX-..."

Azure AD Setup¶

Go to Azure Portal
Navigate to Azure Active Directory → App registrations
Click "New registration"
Name: "Micromegas Analytics"
Supported account types: Choose based on your needs
Redirect URI: "Public client/native" - http://localhost:48080/callback
Note the Application (client) ID
Navigate to Authentication
Under "Advanced settings", set "Allow public client flows" to Yes
This enables PKCE without requiring a client secret
Navigate to API permissions (optional)
Add permissions if needed for your organization

Server Configuration:

export MICROMEGAS_OIDC_CONFIG='{
  "issuers": [
    {
      "issuer": "https://login.microsoftonline.com/{tenant-id}/v2.0",
      "audience": "{application-id}"
    }
  ]
}'

Client Configuration:

export MICROMEGAS_OIDC_ISSUER="https://login.microsoftonline.com/{tenant-id}/v2.0"
export MICROMEGAS_OIDC_CLIENT_ID="{application-id}"
# No MICROMEGAS_OIDC_CLIENT_SECRET needed - Azure AD supports public clients with PKCE

Auth0 Setup¶

Go to Auth0 Dashboard
Create application:
Applications → Create Application
Name: "Micromegas Analytics"
Application type: "Native" (for CLI/desktop)
Configure application:
Allowed Callback URLs: http://localhost:48080/callback
Allowed Web Origins: http://localhost:48080
Note the Domain and Client ID
For Native apps, client secret is optional (true public client)

Server Configuration:

export MICROMEGAS_OIDC_CONFIG='{
  "issuers": [
    {
      "issuer": "https://your-tenant.auth0.com/",
      "audience": "your-client-id"
    }
  ]
}'

Client Configuration:

export MICROMEGAS_OIDC_ISSUER="https://your-tenant.auth0.com/"
export MICROMEGAS_OIDC_CLIENT_ID="your-client-id"
# No client_secret needed for Native apps

Security Considerations¶

Token Storage¶

Tokens are stored at ~/.micromegas/tokens.json with secure file permissions (0600 - owner read/write only). If ~/.micromegas/config.json has a profiles map, the cache moves to a per-profile ~/.micromegas/tokens-<profile>.json instead — see Named profiles.

Token File Contents:

Access token (JWT)
Refresh token
ID token
Expiration time
Issuer and client ID

Token File Security

Never commit token files to version control or share them. Tokens provide full access to your analytics data.

Token Refresh¶

The Python client automatically refreshes tokens when they approach expiration (5-minute buffer). This ensures: - No mid-query authentication failures - Transparent token management - Thread-safe concurrent query support

Token Revocation¶

To revoke access:

User accounts: Disable the user in your identity provider (Google, Azure AD, etc.)
Service accounts: Disable or delete the service account in your identity provider
Immediate revocation: Restart the analytics server to clear the token validation cache

Revocation Timing:

New tokens will be rejected immediately after disabling the account
Existing cached tokens remain valid for up to 5 minutes (configurable via token_cache_ttl_secs)
Total revocation time: Cache TTL (5 min) + Token lifetime (typically 60 min) = ~65 minutes worst case

For faster revocation, use shorter token cache TTL or restart the analytics server.

Admin Privileges¶

Admin status comes from membership in the reserved admins local group (see Groups), managed with micromegas-groups or the Groups admin page. Only add trusted principals to it.

Admin Capabilities:

Partition management functions
Schema migration operations
Administrative SQL functions
Administering audience grants and group/membership rows (create/delete a grant, manage groups)

What admin is not: admin membership confers no implicit read or mint on any audience. Minting an ingestion key, importing one, or reading an audience's data is always a grant row naming the caller — for an admin exactly as for anyone else. An admin who needs data access grants it to themselves in one create_grant call; that self-grant is then a durable, visible, revocable row, not an invisible property of admins membership.

An API-key caller normally carries no email, so it can never match a user:/group: member of admins and is never admin — except while admins still holds its seeded wildcard (*) member, in which case every authenticated caller, API keys included, is admin until an operator adds a user: member and removes * (see Groups's upgrade path). See Authorization → Admin-gated lakehouse functions for the nine gated SQL functions this governs, and Admin SQL Functions for their reference.

HTTPS/TLS¶

Always use TLS for production deployments:

# Production: Use grpc+tls
client = FlightSQLClient(
    "grpc+tls://analytics.example.com:50051",
    auth_provider=auth
)

# Development only: Plain grpc
client = FlightSQLClient(
    "grpc://localhost:50051",
    auth_provider=auth
)

Configure your load balancer or reverse proxy to handle TLS termination.

PKCE (Proof Key for Code Exchange)¶

The Python client uses PKCE for all OIDC flows, providing security for public clients (desktop apps, CLIs) that cannot securely store client secrets.

How PKCE Works:

Client generates random code_verifier
Client creates code_challenge (SHA256 hash of verifier)
Authorization request includes code_challenge
Token exchange includes original code_verifier
Identity provider validates the verifier matches the challenge

This prevents authorization code interception attacks even if the client secret is compromised or unavailable.

Troubleshooting¶

Authentication Failures¶

Symptom: "Invalid token" or "Authentication failed" errors

Solutions:

Check server logs: tail -f /tmp/analytics.log | grep -i auth
Verify OIDC configuration matches between server and client
Ensure Client ID and Issuer URL are correct
Check token expiration: cat ~/.micromegas/tokens.json | jq .token.expires_at (or tokens-<profile>.json if a profiles map is in use)
Clear tokens and re-authenticate: micromegas-logout (clears every cached token file; use --profile <name> to narrow to one)

Token Refresh Failures¶

Symptom: Browser opens on every CLI invocation

Solutions:

Check if refresh token is present: cat ~/.micromegas/tokens.json | jq .token.refresh_token (or tokens-<profile>.json if a profiles map is in use)
Verify client secret matches (if required by provider)
Check token file permissions: ls -la ~/.micromegas/tokens.json (should be 600; or tokens-<profile>.json if a profiles map is in use)
Re-authenticate: micromegas-logout then retry (clears every cached token file; use --profile <name> to narrow to one)

Server Configuration Issues¶

Symptom: Server fails to start or rejects all authentication

Solutions:

Validate OIDC config JSON syntax: echo $MICROMEGAS_OIDC_CONFIG | jq .
Check server can reach identity provider: curl https://accounts.google.com/.well-known/openid-configuration
Verify audience matches client ID exactly
Check server logs for OIDC discovery errors

Multi-Provider Issues¶

Symptom: Only one identity provider works

Solutions:

Verify all issuers are in the configuration array
Check each issuer URL is correct and accessible
Ensure audience (client_id) matches for each provider
Review server logs for OIDC discovery failures per issuer

Migrating Machine Credentials off the Env Keyring¶

OIDC is the destination for human/service-account identities, not machine credentials — service keys migrate to DB-backed ingestion_api_keys/ analytics_api_keys instead. See API Keys → Migrating from the env keyring for the exact procedure and commands; object-cache-srv is the one service that keeps MICROMEGAS_API_KEYS permanently (see Object Cache).

Best Practices¶

Use OIDC for all new deployments - Better security and user management
Enable admin privileges sparingly - Only for users who need administrative access
Use short token cache TTL in high-security environments (60-300 seconds)
Monitor authentication logs - Track failed auth attempts and unusual patterns
Rotate client secrets regularly - Update in identity provider and redistribute
Use separate OAuth clients for different environments (dev, staging, prod)
Document your identity provider setup - Makes onboarding new team members easier

Authentication¶

Overview¶

Authentication Methods¶

OIDC Authentication (Recommended)¶

API Keys¶

Server Configuration¶

OIDC Configuration¶

API Key Configuration¶

Disable Authentication (Development Only)¶

Authorization¶

Client Configuration¶

Python Client with OIDC¶

Interactive Use (Jupyter, Scripts)¶

Subsequent Use (Token Reuse)¶

Token Management¶

CLI Tools with OIDC¶

Python Client with API Keys¶

Ingestion Service Authentication¶

Server Configuration¶

Rust Client Authentication¶

Automatic Configuration (Recommended)¶

Manual Configuration¶

API Key Authentication (Simple)¶

OIDC Client Credentials (Production)¶

Health Endpoint¶

Setting Up OIDC Providers¶

Google OAuth Setup¶

Azure AD Setup¶

Auth0 Setup¶

Security Considerations¶

Token Storage¶

Token Refresh¶

Token Revocation¶

Admin Privileges¶

HTTPS/TLS¶

PKCE (Proof Key for Code Exchange)¶

Troubleshooting¶

Authentication Failures¶

Token Refresh Failures¶

Server Configuration Issues¶

Multi-Provider Issues¶

Migrating Machine Credentials off the Env Keyring¶

Best Practices¶

Reference¶