airflow-dq — Current-State Feature Inventory & Production Assessment¶

Date: 2026-07-11 · Scope: entire repo at working tree (incl. the uncommitted Airflow-native UI restyle) · Method: five parallel subsystem audits over the full source + targeted re-verification of contested claims.

Purpose: the factual baseline for the upcoming refactor/roadmap research — what exists today, how it actually behaves, and where it bends or breaks in a productive system. This document deliberately does not propose the roadmap; §8 only seeds the questions the research phase should answer.

1. System overview¶

airflow-dq is an Airflow 3.1-native data-quality framework: data contracts (ODCS-dialect YAML) → compiled checks → one mapped Airflow task per check → results store → embedded dashboard + low-code editor, with three failure policies (fail / warn / quarantine), OpenLineage emission, and webhook alerting.

Moving parts and data flow:

contracts/*.yaml ──ContractStore──▶ ContractCompiler ──▶ CheckSpecs (XCom)
                                                            │ .expand()
                                                            ▼
Airflow DAG ──build_contract_checks()──▶ DataContractCheckOperator (one task/check)
                                             │  get_executor(backend)
                                             ▼
                              DuckDB / Postgres executor ──SQL──▶ warehouse
                                             │
              ┌──────────────┬───────────────┼────────────────┬─────────────┐
              ▼              ▼               ▼                ▼             ▼
        dq_check_results  dq_quarantine  OpenLineage      webhook      task XCom
        (critical path!)  (best-effort)  (best-effort)  (best-effort)
                                             ▲
FastAPI plugin (/dq) ── serves React UI, results/trends API, authoring API
  (columns · preview · profile · save · diff)      UI: Dashboard + Contract Editor

Key numbers: 12 rule types (11 UI-authorable), ~2,000 LOC Python, 39 unit tests (DuckDB-only for execution; zero coverage for the API/store/operator/quarantine surface), single docker-compose demo stack where one Postgres doubles as warehouse and DQ metadata store — a coupling several features silently depend on.

2. Feature inventory by subsystem¶

Format per feature: what it does → how it behaves in production (the load-bearing caveats). File references point at entry points.

2A. Contracts & authoring (`contracts/`, `authoring.py`)¶

Feature	What it does	Production reality
ODCS-dialect contract model (`contracts/models.py`)	Pydantic model: id, int version, dataset, owner, schema (columns + required/unique), quality rules (12 types), severity blocking/warning	A dialect, not spec-ODCS v3: int version (not semver), custom `dataset`/`owner` fields, quality placement differs, no servers/team/SLA/customProperties. Spec-compliant contracts parse but are silently stripped on any editor round-trip. Rule params are untyped — typos surface (or no-op) only at run time. `status` is parsed but consumed nowhere (no lifecycle)
YAML-on-disk versioned store (`contracts/store.py`)	One file per version (`<id>/<id>.v<N>.yaml`), refs `id@vN`, root from `DQ_CONTRACTS_DIR` (default relative `contracts`)	Filesystem is the only storage: API server + every worker must share the same directory (RWX volume or baked image). No caching (re-parse per run/request), no locking (concurrent saves = last-write-wins), no ACLs/tenancy. Lexicographic sort misorders v10 before v2. Ref-regex vs `safe_identifier` mismatch: hyphenated ids load but can't be saved; dotted ids save but can't be listed/loaded. Registry-DB sync is an explicit TODO (`store.py:49`)
Version pinning in DAGs	`build_contract_checks(contract="orders@v3")` — explicit pins, no "latest"	Good safety property, but upgrades are grep-and-edit across DAG repos; deleted contract = runtime FileNotFoundError on the worker, not parse-time
Mutable "versions" (`authoring.py:save_contract`)	POST save writes/overwrites `<id>.v<N>.yaml`, version number is client-supplied	Published versions are silently overwritable — the pinning guarantee isn't enforced. No monotonicity check, no authorship, no audit, no git integration, non-atomic write (crash = truncated YAML that fails every pinned DAG)
Breaking-change detector (`contracts/versioning.py`)	Diffs two versions; breaking = column removed / type-family changed / new-required / uniqueness added; info = rest. Shown on every save	Schema-centric blind spots: rule param changes, severity flips (warning→blocking), and threshold tightening are invisible (rules diffed only on `(rule, property)`); adding a blocking rule counts as "info". Advisory only — computed after the save, persisted nowhere, never blocks
Column introspection (`authoring.py:list_columns`)	Live `information_schema` read to populate editor dropdowns	Filters by `table_name` only — same-named tables across schemas merge; schema-qualified names return empty. Missing table = silent empty list
Table profiling → draft (`authoring.py:profile_table`)	One click: count/distinct per column, min/max for numerics, low-cardinality values → suggested contract	Severe cost: `count(distinct)` for every column + per-column scans, synchronous in the API request, no sampling/timeout. Suggested valueRange always fails on its own boundary rows (observed min/max + strict `<`/`>` predicate). Copies real data values into YAML (governance concern). `float()` cast breaks on exotic numerics
Live preview (`authoring.py:preview_check`)	"Test" runs one check against the live table pre-save; errors return as `{status:'error'}`	Same execution path as DAG runs (tolerances/scopes/sample included) — genuinely good. But synchronous unbounded warehouse scan per click, no timeout/rate limit/cancel; any token holder can run guarded-read SQL against any table the shared DSN reaches (incl. `pg_sleep`-style DoS)
YAML generation (`authoring.py:build_contract_doc`)	Draft → ODCS-dialect YAML (single schema object), live preview endpoint	Only id/dataset validated at save; bogus column names/params persist and explode later at pipeline time. Comments/order destroyed on round-trip. Hardcoded `status: active`

2B. Rule library & execution engine (`compiler/`, `executors/`)¶

Compilation (compiler/compiler.py): contract → flat CheckSpecs: 1 blocking schema check per dataset + derived notNull/unique from column flags (always severity warning) + explicit rules; dedup by check_name, last-wins — two same-type rules on one column, or two unnamed expression rules, silently collapse. Specs ride XCom as dicts; from_dict(**d) breaks on unknown keys (mixed-version worker fleets during rolling upgrades).

The 12 rules — generated SQL in executors/sql.py, evaluation in _evaluate.py:

Rule	Semantics	Production caveats
notNull	`col IS NULL` count+sample	full scan per run without `where` scope
unique	`count(*) − count(DISTINCT col)`	single-column only; count and sample use different math — NULL duplicates are counted but never sampled/quarantined (violates the "can never disagree" claim in `sql.py:3-5`)
valueRange	exclusive bounds, OR-joined	NULLs silently pass; boundary values fail (author surprise); no bounds → runtime `NotImplementedError` crash
acceptedValues	`NOT IN (literals)`	no list-size cap; case-sensitive; empty list crashes
pattern	regex, the only dialect branch (`!~` vs `regexp_matches`)	PG POSIX vs DuckDB RE2 — same contract can behave differently per backend; malformed regex = task crash
referentialIntegrity	`NOT IN (SELECT ref)`	ref side fully materialized, never scoped; single-column FK only
freshness	rows where `asOfDate − col ≥ N days`; operator injects DAG `logical_date` (backfill-safe, D15)	Not a max-age check: every historical row older than N days counts as failed — permanently red on any table with history unless scoped; DATE columns only (TIMESTAMP = runtime SQL error on PG); `unit` param parsed but ignored (days only); asset-triggered runs have no logical_date → silently falls back to CURRENT_DATE
rowCount	`count(*) > min`, synthetic 0/1 fail	lower bound only; tolerances on it are semantically meaningless but not prevented
expression	user predicate → `failing_predicate` (count/sample/quarantine/tolerances for free — the D11 design win)	guard = 500-char keyword blocklist, not a parser: read-anything subselects allowed (cross-team exfiltration on shared DSN), false-positives on blocked words inside string literals
anomaly	z-score in pure SQL: `ABS(x−avg) > kσ`, baseline deliberately unscoped	≈3 full scans+aggregates with quarantine; baseline includes the anomalies themselves (large bad batch self-masks); n=1 → stddev NULL → silently zero failures
reconciliation	aggregate compare vs another table (rowCount/sum/avg/min/max, tolerancePercent) — verified implemented in `_evaluate.py:54-89`	asymmetric Δ% (normalized by the "against" side); NULL aggregates coerce to 0; binary only — graded warn/fail tolerances don't apply; both tables must share one connection
schema (drift)	information_schema vs contract: missing required columns + coarse type-family mismatch	`table_name`-only filter (multi-schema false results; qualified names report all-missing); optional columns and extra columns never flagged; unknown types → family "other" → comparison skipped (jsonb/arrays/enums drift freely)

Cross-rule machinery: - Tolerances (D8): warn/failIfGreaterThan (abs) + …PercentGreaterThan; any threshold switches to graded mode and severity is ignored — a blocking rule with only warn-thresholds can never fail (silent un-gating). No warn≤fail validation. Schema + reconciliation can't be toleranced. - where scoping (D15): shared identically by count/sample/quarantine — but a static string: no {{ ds }}/logical-date templating, so true incremental scoping isn't expressible from a stored contract. - Bad-row sampling (D10): SELECT * of up to 10 rows (hardcoded) with the same predicate → sample_json. Full rows incl. PII flow into metadata DB, API responses, and task XCom; no masking/column projection/byte cap. - Quarantine: INSERT INTO dq.dq_quarantine … LIMIT 1000 (hardcoded) over the warehouse connection — only works when warehouse == metadata DB (true only in the demo compose). Triggers on rows_failed > 0 even when tolerances graded the status pass. Third predicate evaluation (no snapshot consistency). Append-only: retries re-quarantine; no TTL, no reader/UI, no replay tooling. check_name interpolated unescaped into the INSERT. - Injection guards: identifier regex + numeric coercion + literal escaping are solid for closed rules; free predicates are blocklist-guarded only ("demo-grade" per its own docstring). Quoted/case-sensitive identifiers unusable. - Executor protocol (D2): CheckExecutor Protocol declares run() only, but reality requires run()+quarantine()+query() — a to-spec third-party backend half-works. Backend selection is string-prefix → env var (DQ_DWH_DSN/DQ_DUCKDB_PATH): the alias looks like an Airflow conn_id but isn't; one warehouse per deployment, no Airflow Connections/secrets integration, hardcoded demo-credential defaults, Databricks branch commented out. - Connections: Postgres executor creates a fresh SQLAlchemy engine per call, never disposed — connection storms at mapped-task concurrency; no statement timeout anywhere. DuckDB default :memory: = vacuous empty DB when env unset; file mode = single-writer lock conflicts under parallel mapped checks. - Error handling: any executor/SQL error crashes the task in every on_fail mode — the error status in CheckResult/schema is never produced; dashboards show gaps, warn-mode monitors still go red on infra blips, and a contract typo becomes N runtime task failures.

2C. Airflow integration (`operators/`, `dags/`, `lineage.py`, `alerting.py`)¶

Feature	What it does	Production reality
`build_contract_checks()` (`factory.py`) — the whole public API	load-specs task + `.partial().expand()` mapped operator; `bronze >> validate >> gold`	Only 3 params (contract/backend/on_fail): no retries/pool/queue/timeout pass-through, no check subsetting, executor kwargs never forwarded. `on_fail` unvalidated — a typo silently degrades to warn-like behavior. Same ref twice in a DAG = task_id collision
One mapped task per check (D1)	per-check grid visibility, logs, retries	M checks × N runs × (fresh engine per check) connection storms; >1024 checks breaks at Airflow's max_map_length unhandled; map indexes are positional (reordering rules shuffles them); wide tables inflate the schema-check spec through XCom
on_fail="fail"	status `fail` raises, blocks downstream	only blocking-severity rules can reach `fail` — schema-derived notNull/unique (always warning) never gate, even in fail mode: quiet under-gating
on_fail="warn"	record-only; used by monitor	literally "absence of the raise": any unrecognized string behaves like warn. Alerts still fire → gate+monitor double-alerts per failure
on_fail="quarantine"	copy bad rows, continue	see 2B quarantine caveats; DuckDB variant is count-only
Results write (`results/store.py` via operator)	one row per check into `dq_check_results`	The only non-best-effort sink: results-DB downtime fails every check task in every mode (quarantine/alerts/lineage are all try/except-wrapped, results write isn't). Append-only, no idempotency key: task retries/clears double-count results, quarantine rows, and alerts
XCom usage	specs in, `result.__dict__` out	result XComs (incl. PII sample_json) duplicate data into Airflow's metadata DB and have no consumer; no `do_xcom_push=False`
OpenLineage (D16)	`DataQualityAssertionsDatasetFacet` + metrics facet per check; rigorously best-effort	dataset identity = bare table name + one global namespace env — collides across teams and won't attach to datasets other OL integrations emit (doesn't match postgres://… naming or the Asset URI). Failures degrade to silent no-lineage (zero logging)
Alerting (`alerting.py`)	Slack-style webhook POST per failing check; owner→URL routing via `DQ_ALERT_ROUTES`, fallback `DQ_ALERT_WEBHOOK`	No dedup/grouping/throttling (20 failing checks = 20 POSTs, × 2 with gate+monitor, re-fired on retry). Only `status=='fail'` triggers (warns and infra errors alert nobody). Relative deep-link (not clickable in Slack). 5s blocking send inside the task; HTTP status ignored
Asset-based monitor (D12) (`dq_monitor.py`, `dq_assets.py`)	`schedule=[orders_asset]` — DQ re-runs whenever a producer updates the table	asset fires on producer success (pre-validation, whether or not data changed). Monitor pins v3 while a v5 producer updates the same asset — structural version skew. Asset runs have no logical_date (freshness silently shifts semantics). Check failures invisible in the grid (warn mode) — dashboard-only. One hand-written monitor per asset; the D12 "central sweep DAG" isn't built
Demo DAGs	medallion pipeline + byte-copied v5 variant	placeholders (no real data movement); the copy-paste v5 DAG (stale docstring included) is the current upgrade story demonstrated in-tree
Airflow-3-only surface	`airflow.sdk` imports, graceful degradation for bare unit tests	hard 3.1 floor, no version guard (Airflow 2 fails opaquely at parse); `_PIP_ADDITIONAL_REQUIREMENTS` install-at-boot is demo-only

2D. Results store, plugin API & deployment (`results/`, `plugin/`, compose, packaging, CI)¶

Feature	What it does	Production reality
`dq` schema (`results/schema.sql`)	4 tables + 3 indexes, applied via docker initdb on first volume creation	No migration story at all (no Alembic, no version stamp — schema changes = manual psql or volume wipe). `dq_contracts`/`dq_contract_versions` are dead tables (no reader or writer anywhere; `is_breaking`/`diff_summary`/`checksum` unused). No retention/partitioning: results+quarantine grow forever with inline TEXT samples. No unique constraint against retry-duplicated rows; `sample_json` is TEXT, not JSONB
Results read API	`/api/results` (latest 500, hardcoded), `/api/trends` (last 60 runs, grouped per run, chronological)	no pagination/time-window/dataset filter anywhere; unfiltered sort has no supporting index; trend aggregation scans full history per dashboard load; malformed refs → unhandled 500
FastAPI plugin mount (`plugin/__init__.py`)	`/dq` sub-app + "Data Quality" nav iframe via `external_views`; auto-registered by pip entry point	plugin activates fleet-wide on install (no disable flag). Sync routes run on the Airflow API server's shared threadpool — a slow profile/preview degrades the whole Airflow UI. Airflow does not auth-protect `fastapi_apps` (D7)
Bearer-token auth (`plugin/auth.py`)	all JSON routes require `Bearer $DQ_API_TOKEN`; 503 fail-closed when unset	single static shared token: no identity, no viewer-vs-author roles, no audit, no rotation story, non-constant-time compare. Auth attached per-route (new routes ship open by default). `/dq/docs` + `/dq/openapi.json` are unauthenticated
Token delivery to UI (`app.py:serve_ui`)	injects `window.__DQ_CONFIG__={token}` into index.html	The defining security hole: `/dq/app/` itself is unauthenticated — any client that can reach the Airflow API server receives the full-privilege token in cleartext. The entire auth model reduces to network reachability
UI serving	static assets + index route, 503 "run make ui-build" fallback	`parents[4]` repo-relative dist path only works for editable installs — the wheel ships no UI (`DQ_UI_DIST` per node, undocumented); assets mount decided at import time (build-after-start = 404 until restart)
docker-compose demo	airflow-postgres + dq-postgres (warehouse and metadata) + single standalone Airflow 3.1.0	explicitly demo-only; the warehouse==metadata coupling is what quarantine depends on. `DQ_DB_` env anchor is dead config* (code reads `DQ_RESULTS_DSN`/`DQ_DWH_DSN`, never set — works only because hardcoded defaults match compose hostnames); `DQ_ALERT_*` from `.env` never reaches the container
Packaging (`pyproject.toml`)	hatchling wheel, plugin entry point, extras dev/databricks/lineage	unpinned `apache-airflow>=3.1` (no constraints file); `duckdb`+`psycopg2-binary` unconditional; `lineage` extra pins the legacy `openlineage-airflow` package (Airflow 3 uses the provider) — likely wires nothing; no py.typed, no release workflow
CI (`ci.yml`)	ruff + mypy + pytest-cov (py3.10) + UI build + integration placeholder	integration job is an `echo TODO` that passes green; coverage gates nothing; zero tests for store/API/auth/operator/quarantine — the entire serving surface is manually tested only
Seed data	7-row dirty `orders` table exercising every failure mode	rows dated at volume-init time — demo outcomes drift as the volume ages; no reseed target

2E. React UI (`ui/src`)¶

Feature	What it does	Production reality
Shell	2-tab SPA (Dashboard / Contract Editor), Airflow-native look (verbatim theme copy), light/dark sync via shared `theme` localStorage key	no router/URL state: no shareable "look at this failing check" links; unsaved editor state silently lost on tab switch. Theme is fork-by-copy — every Airflow upgrade is a manual re-copy event
Embedded vs standalone	iframe-sandbox-aware: run deep-links are text-only inside Airflow, clickable at `/dq/app/`	the main triage affordance (jump to run) is unavailable exactly where users are (embedded); no copy-URL fallback
Contract picker	dropdown of all refs + owner display	flat, unsearchable list (unusable at hundreds of contracts); hardcoded `orders@v3` default; list-fetch failures silently swallowed; never refreshes after an editor save
KPI row	active checks, pass-rate, failing count, freshness traffic light — computed client-side from "latest per check_name"	gate and monitor runs overwrite each other in the dedup (a passing monitor can mask a failing gate); the server's 500-row cap silently drops older checks; thresholds (60 %/100 %) hardcoded
Trend chart + anomaly flags	pass-rate per run; red points where rows_failed > mean+2σ (client-side, D14 level 2)	baseline includes the anomalies themselves (sustained incidents self-normalize); window = last 60 runs regardless of age; flags recomputed per page load — not persisted, not alertable; gate+monitor runs mixed into one series. (Ordering verified correct: server returns chronological)
Dimension chart	stacked pass/warn/fail per DQ dimension	`error`-status results are silently excluded; no drill-down
Results table + bad-row samples	expandable rows show the offending rows, NULLs highlighted; deep-link per run	sample PII delivered to every viewer, no masking; observed/expected hidden in tooltips; no sort/filter/pagination; no quarantine browser anywhere — `dq_quarantine` is write-only in the whole product
Editor: load/introspect/profile	load existing → edit → re-version; live columns; profile-to-draft	version not auto-bumped (easy accidental overwrite); loaded tolerances don't hydrate the warn%/fail% inputs (invisible divergence); Profile destructively replaces all blocks without confirmation; backend hardcoded to `postgres_dwh` throughout the UI — the pluggable-executor story is not reachable from the editor
Editor: rule forms + SQL block + live Test	11 rule forms, CodeMirror SQL predicate, per-block Test with real results	free-text where enums belong (reconciliation metric, FK refs); no client-side validation before Test/Save; commas can't appear in acceptedValues; `schema` rule not authorable; absolute (non-%) tolerances not exposed
Save + breaking-change banner	save → path + YAML + red/green diff banner	banner is advisory after the save already happened; no confirm/approval step; the arbitrary-version diff endpoint exists server-side but the UI never calls it
api.ts client	9-endpoint fetch wrapper, token from injected config	no timeouts/aborts/retries (hung requests hang UI states forever); server error bodies discarded (users see "422 Unprocessable Entity" with no detail)

3. Cross-cutting production concerns (the themes that recur everywhere)¶

Security & multi-tenancy — the dominant gap. One static token, served in cleartext to any unauthenticated visitor of /dq/app/; every dashboard viewer is therefore a full author + can run guarded-read SQL against the single shared warehouse DSN; bad-row samples/quarantine/XCom copy raw production rows (PII) into three different stores with no masking, TTL, or access control; no per-user identity anywhere; D7's "validated against Airflow's auth manager" is an acknowledged TODO.
Reliability asymmetry & missing error state. Quarantine, alerting, lineage: rigorously best-effort. Results write and check execution: hard failures. The DQ metadata DB sits in the critical path of every gated pipeline; executor exceptions crash tasks in all modes and the schema's error status is dead — infra failures are indistinguishable from "no data" on the dashboard.
Idempotency. All three sinks (results, quarantine, webhook) are append/fire-and-forget with no run-scoped keys — Airflow retries and clears (the norm in production) duplicate observability data, quarantined rows, and alerts.
Scale & cost. One check ≈ 1–3 full-table scans (count/sample/quarantine; anomaly adds aggregate subqueries); no where-clause templating for true incremental checking; fresh SQLAlchemy engine per operation with no pooling/timeout; no retention on any table; hardcoded API caps (500/60) with no pagination; profile/preview run synchronously inside the Airflow API server.
State topology & split-brain. Contracts = local filesystem of whichever process; registry DB tables = dead; quarantine = requires warehouse==metadata-DB; wheel ships no UI. Every multi-node or split-warehouse deployment breaks a different subsystem silently.
Identity fragmentation. The same table has three uncorrelated identities — physical name in specs/lineage (orders), Asset URI (dq://warehouse/orders), OL namespace env var — none schema-qualified; cross-team name collisions and lineage that doesn't attach to other producers' datasets.
Version/upgrade story. Contract "versions" are mutable files; CheckSpec round-trip breaks on unknown keys (rolling upgrades); no DB migrations; UI theme is fork-by-copy per Airflow release; unpinned Airflow dependency; docs one release behind code.

4. DESIGN.md scorecard (intent vs code)¶

Decision	Status	Delta
D1 dynamic task mapping	✅	map-length cap unhandled; positional indexes unstable across versions
D2 pluggable CheckExecutor	⚠️ partial	Protocol understates real interface (run+quarantine+query); prefix-string→env-var config, no Airflow Connections; Databricks = comment
D3 ODCS standard	⚠️ dialect	subset with silent stripping of spec-compliant contracts
D4 separate metadata schema	⚠️ partial	schema exists; registry tables dead; quarantine contradicts the separation (needs warehouse==metadata DB)
D5 wedge (authoring+codegen+dashboard)	✅	built end-to-end
D6 fail/warn/quarantine	✅	quarantine: same-DB coupling, no dedup/TTL/reader, fires on tolerated passes
D7 explicit auth	❌ effectively	token handed out unauthenticated at `/dq/app/`; auth-manager integration is a TODO
D8 tolerances	✅	thresholds silently override severity (blocking + warn-only-thresholds can never fail); no warn≤fail validation; UI exposes only the % pair
D9 schema-drift check	✅	not schema-qualified; optional/extra columns and "other"-family types invisible
D10 actionable failures (samples)	✅	PII exposure; hardcoded 10; unique-rule count/sample divergence
D11 low-code authoring + live preview	✅	sync unbounded scans; blocklist ≠ sandbox (both acknowledged trade-offs)
D12 gate + monitor duality	⚠️ partial	works; version skew, double alerts, logical_date loss on asset runs; "sweep DAG" third option not built
D13 profiling + breaking-change detection	✅	valueRange boundary bug; diff blind to params/severity; diff never persisted; no version-timeline
D14 anomaly, two levels	✅	z-score self-masking baseline; metric-level flags client-side only, not persisted/alertable; IsolationForest hook absent
D15 incremental + backfill-safe	⚠️ partial	`where` is a static string (no templating — the incremental half is not truly expressible); freshness is row-level not max-age, DATE-only, and silently changes semantics on asset runs
D16 OpenLineage, never critical-path	✅	identity mismatch with OL naming conventions; silent degradation; packaging extra pins the legacy integration

5. Docs-vs-code inconsistencies (fix before publishing anything)¶

PROJECT_DESCRIPTION.md + medium-article.md describe the old runtime theme-mirroring (MutationObserver); the code now uses a verbatim Airflow theme copy + next-themes localStorage sync.
Both docs claim OpenLineage is "not yet built / roadmap" — it is implemented (lineage.py, 5 tests); README/DESIGN say so. Internal contradiction.
PROJECT_DESCRIPTION.md: "React + TypeScript + Recharts" — charts are chart.js now; "29 unit tests" — 39; check table lists 10 rule types — 12 exist.
README.md: status badge still "🟡 Skeleton"; claims "integration test is a stub" — no integration test file exists (CI job is an echo).
ui/README.md said "~6 MVP check types" vs 11 in the form (fixed in the restyle commit-to-be).
Article shows freshness unit: day — parsed but ignored by SQL generation, not exposed in the editor.
Stale TODO comments: app.py:145 (save endpoint "missing" — it exists), compiler docstring ("6 MVP rule types").

6. Dead & latent code/config¶

dq_contracts + dq_contract_versions tables incl. is_breaking/diff_summary/checksum — zero readers/writers (store.py:49 TODO).
CheckResult.status='error' + schema enum value — never produced.
freshness params['unit'] — parsed, ignored.
get_executor(**kwargs) — never passed by the operator.
Databricks executor — commented block; databricks extra installs a connector nothing imports.
DQ_DB_* compose/env-example vars + AIRFLOW_UID — consumed nowhere; DQ_ALERT_* documented but not forwarded into the container.
GET /api/contracts/diff — implemented, never called by the UI.
Mapped-task result XComs — no consumer.
lineage extra pins legacy openlineage-airflow (Airflow 3 wants the provider package).

7. Verification notes¶

Claims in the subsystem audits were spot-checked; two were corrected here: - "reconciliation is unimplemented / raises NotImplementedError" — false: fully implemented at _evaluate.py:54-89 with tests; the caveats in §2B stand. - "trend chart x-axis renders newest-first" — false: store.py returns reversed(rows) (chronological); confirmed visually in the running UI.

Confirmed by direct read: store.py:49 registry TODO; unauthenticated serve_ui token injection (app.py:129-143); results write outside any try/except (check_operator.py:66); trends SQL aliases max(executed_at) AS executed_at (valid).

8. Seeds for the research phase (hypotheses to validate — NOT the roadmap)¶

Refactor candidates by theme (from §3, roughly by blast radius): auth & RBAC (Airflow auth-manager integration, kill the token-in-HTML path) · connection management (Airflow Connections/secrets backend, pooled engines, statement timeouts) · reliability (best-effort or buffered results write, produce error status, idempotency keys on all sinks, alert dedup) · contract integrity (immutable versions, server-side numbering, atomic writes, registry-DB sync or git-backed store, param/severity-aware diffing) · quarantine decoupling from the warehouse==metadata assumption · incremental checking (templated scopes) · freshness semantics (max-age + timestamp support) · per-adapter SQL layer before backend #3 · packaging (ship UI in wheel, constraints file, migrations) · docs regeneration.

Feature-gap areas to research against the market (Soda, Great Expectations, dbt tests, Elementary, Databricks DQX, Monte Carlo/Sifflet-class observability): warehouse coverage (Snowflake/BigQuery/Databricks/Trino) · richer rule pack (composite uniqueness, cross-column conditionals, distribution/seasonality anomaly, volume ranges) · quarantine triage UI + replay · notification channels (email/PagerDuty/Opsgenie) with grouping/escalation · contract approval workflows (PR-based, CODEOWNERS) · SLA/scorecard & multi-contract overview dashboards · results retention/rollups · dbt-test import · OL-spec-compliant dataset naming + column-level lineage · public read API/pagination for BI consumption · multi-warehouse routing per contract.

Each seed should be validated in the research phase for: user demand, differentiation vs incumbents, and fit with the framework's wedge (in-Airflow authoring + gate/monitor duality + breaking-change detection).