Configuration

All knobs are read through airflow_dq.config.settings() (src/airflow_dq/config.py) — environment variables, cached once per process. The repo's .env.example is a commented template. Set these on every process that runs airflow-dq code: the API server (plugin/UI) and every worker (check tasks).

Environment variables

Storage

Env var	Default	Purpose
DQ_RESULTS_DSN	compose demo DSN	SQLAlchemy DSN of the DQ results/registry store. airflow-dq owns the dq schema there and migrates it automatically. Must be Postgres-compatible in practice; never point it at Airflow's metadata DB.
DQ_DWH_DSN	compose demo DSN	Fallback warehouse DSN for postgres* backends when no Airflow Connection matches (see backend resolution).
DQ_DUCKDB_PATH	:memory:	DuckDB database path for duckdb* backends. :memory: is a throwaway per-process DB — checks against it scan an empty database.
DQ_CONTRACTS_DIR	contracts	Directory of versioned contract YAML (<id>/<id>.v<N>.yaml). Relative paths resolve against each process's CWD — use an absolute path in deployments, and make sure API server and workers see the same directory (shared volume or baked into the image).

Warning

The defaults for DQ_RESULTS_DSN / DQ_DWH_DSN are the docker-compose demo credentials (postgresql+psycopg2://dq:dq@dq-postgres:5432/dq). Always set both explicitly outside the demo stack.

Experimental warehouse DSN fallbacks

Used only when no Airflow Connection with conn_id == backend exists:

Env var	Example
DQ_SNOWFLAKE_DSN	snowflake://user:pass@account/db/schema
DQ_BIGQUERY_DSN	bigquery://project/dataset
DQ_DATABRICKS_DSN	databricks://token:...@host?http_path=...
DQ_TRINO_DSN	trino://user@host:8080/catalog/schema

Install the matching extra (pip install "airflow-dq[snowflake]" etc.). These adapters are experimental: SQL generation is unit-tested per dialect, live connections are not.

Plugin / API

Env var	Default	Purpose
DQ_API_TOKEN	unset	Legacy shared secret for machine clients (CI, curl). Unset = Airflow auth-manager JWT/cookie auth only. Grants read and write.
DQ_UI_DIST	unset	Override for the built React bundle location. Rarely needed: wheel installs auto-resolve the packaged airflow_dq/ui_dist, source checkouts ui/dist.
DQ_BASE_URL	empty	Absolute base URL for clickable deep links in alerts, e.g. https://airflow.example.com. Empty = relative links (not clickable in Slack).

Execution

Env var	Default	Purpose
DQ_STATEMENT_TIMEOUT_S	300	Session statement timeout applied (best-effort, per dialect) to check/preview/profile queries.
DQ_SAMPLE_LIMIT	10	Max bad rows captured as the failure sample per failing check.
DQ_QUARANTINE_LIMIT	1000	Max rows copied into dq.dq_quarantine per failing check.

Alerting

Env var	Default	Purpose
DQ_ALERT_WEBHOOK	unset	Default alert channel for fail/error results — a channel URL. Unset (and no route match) = alerting off.
DQ_ALERT_ROUTES	unset	JSON owner→channel map, optionally severity-aware. See below.
DQ_SMTP_HOST	unset	SMTP host for mailto: routes.
DQ_SMTP_PORT	587	SMTP port.
DQ_SMTP_FROM	unset	From address for email alerts.
DQ_SMTP_USER / DQ_SMTP_PASSWORD	unset	SMTP credentials (optional).

DQ_ALERT_WEBHOOK and DQ_ALERT_ROUTES are deliberately re-read from the environment on every alert (not cached), so routing changes apply without a process restart.

Lineage, contracts git flow, retention

Env var	Default	Purpose
DQ_OL_NAMESPACE	airflow-dq	Explicit OpenLineage namespace override. Default behavior derives the namespace from the executor's resolved connection per the OL naming spec (e.g. postgres://host:5432), falling back to this value.
DQ_CONTRACTS_GIT	false	Enable the git-based "propose" save flow: the editor's Propose as PR commits new versions on a dq/contract-<id>-v<N> branch instead of writing to the tree. Requires DQ_CONTRACTS_DIR to be inside a git checkout.
DQ_CONTRACTS_GIT_REMOTE	origin	Remote used by the propose flow.
DQ_RESULTS_RETENTION_DAYS	0	Days to keep raw check results / metric anomalies (0 = forever).

Backend resolution (Airflow Connections)

The backend string you pass to build_contract_checks(...) is resolved to a real connection in this order:

1. An Airflow Connection whose conn_id equals the backend string — resolved worker-side, so your secrets backend applies. The connection's conn_type picks the adapter and is mapped to a SQLAlchemy URL: postgres, duckdb, snowflake, bigquery/google_cloud_platform, databricks, trino.
2. Env-DSN fallback by name prefix — when no such Connection exists (or its conn_type is unsupported): backends starting with postgres → DQ_DWH_DSN, duckdb → DQ_DUCKDB_PATH, snowflake/bigquery/databricks/trino → the matching DQ_<PREFIX>_DSN.

So backend="postgres_warehouse" uses the Airflow Connection postgres_warehouse if it exists, else DQ_DWH_DSN (because of the postgres prefix). A backend with a prefix that matches no adapter and no Connection raises at runtime.

Engines are pooled per DSN per process (pool_pre_ping, pool size 5 + 5 overflow) and get the statement timeout applied per dialect. Third-party executors can be registered at runtime (airflow_dq.executors.base.register_executor) or via the airflow_dq.executors entry-point group.

Tip

In production, prefer real Airflow Connections: credentials live in your secrets backend, and per-team backends stop being "whatever DQ_DWH_DSN points at". Keep the env fallbacks for demos and single-warehouse setups. The dashboard's editor defaults to backend postgres_dwh.

Authentication

Airflow 3 does not auto-protect plugin FastAPI routes, so airflow-dq secures them explicitly (see plugin/auth.py). Every route except GET /dq/api/health requires one of, in order:

1. Airflow auth-manager JWT — Authorization: Bearer <jwt> validated against the running auth manager (or directly against Airflow's JWT signing key). This is what the embedded UI and any logged-in browser session use.
2. Legacy shared secret — the same Bearer header compared to DQ_API_TOKEN. Intended for machine clients (CI, curl); grants read and write.
3. Airflow session cookie — the _token cookie (set by the GET /auth/token/login flow), validated as in (1).

The plugin's own UI (embedded or at /dq/app/) authenticates via (1): it reads the JWT the Airflow UI keeps in localStorage — same origin, same storage — and sends it as a Bearer header. If you see 401s in the embedded views, log into the Airflow UI in the same browser. This is why the dashboard "just works" same-origin without any token in the page.

Write routes (contract save, preview, quarantine resolve, dbt import) additionally consult the auth manager's is_authorized_custom_view(method="PUT", resource_name="DQ Contracts") when available — a viewer-role JWT gets 403 on writes. If nothing is configured (no Airflow auth stack importable and no DQ_API_TOKEN), the API fails closed with 503.

Getting a JWT for scripting against a stock Airflow 3.1:

JWT=$(curl -s -X POST http://localhost:8080/auth/token \
  -H "Content-Type: application/json" \
  -d '{"username": "admin", "password": "<password>"}' | python -c 'import sys,json; print(json.load(sys.stdin)["access_token"])')
curl -s -H "Authorization: Bearer $JWT" http://localhost:8080/dq/api/overview

Alert channels and routing

Channels are selected by the URL scheme of the route:

Scheme	Channel
https://... (anything not below)	JSON POST webhook, Slack-compatible payload
mailto:you@example.com	email via SMTP (DQ_SMTP_*)
pagerduty://<routing_key>	PagerDuty Events API v2 (severity derived from check status)
opsgenie://<api_key>	Opsgenie alerts API

Routing: DQ_ALERT_ROUTES maps a contract owner to a channel, either flat or severity-aware (fail = data failures, error = infra errors); DQ_ALERT_WEBHOOK is the fallback for unrouted owners:

# one channel for everything:
DQ_ALERT_WEBHOOK=https://hooks.slack.com/services/T000/B000/XXXX

# per-owner, severity-aware, mixed channels:
DQ_ALERT_ROUTES='{
  "growth@example.com": {"fail": "https://hooks.slack.com/services/...",
                          "error": "pagerduty://YOUR_ROUTING_KEY"},
  "finance@example.com": "mailto:finance-oncall@example.com"
}'

One grouped alert per (run, contract) — the first failing check claims the group, gathers all failed/error results of the run, and sends a single message; retries and sibling failures don't re-fire. Delivery is best-effort (5s timeout, never breaks a task). Set DQ_BASE_URL to make the run deep-link clickable.

Schema migrations

The results store's dq schema (results, quarantine, anomalies, alert groups, contract registry) is migrated automatically: versioned SQL files ship in the package and are applied on first use per process, recorded in dq.dq_schema_version — each migration in one transaction. A fresh, empty database needs no manual bootstrap; existing installs pick up new migrations on upgrade. make db-migrate (repo checkout) applies them explicitly if you prefer migrations at deploy time.