Integration

DataHub decides what is true

Comgu does not have its own opinion about which commerce value is correct. It asks the catalog, and if the catalog cannot answer, it stops.

Open the live catalog — judge / northstar-2026. It is our own DataHub Core instance, not a screenshot: browse the lineage, the owners and the structured properties every claim on this page depends on.

What is read, and what it decides

DataHub	Decides	If removed
`get_lineage`	The blast radius — which surfaces project from the change	No assets found; every rule skips
`comgu.authority`	Which asset is the source of truth	Engine halts rather than guessing
`comgu.criticality`, `comgu.customer_facing`	How severely a failure is graded	Same failure, different severity
Ownership aspects	Who a correction routes to	Unowned surface becomes its own finding
Assertions	Corroborating quality evidence	Finding stands; evidence is thinner

The dependency is testable

Two tests in the suite exist to prove this is not decoration.

def test_engine_refuses_to_guess_without_datahub_authority():
    ctx = golden_context()
    for asset in ctx.assets_by_urn.values():
        asset.authority = "projection"        # strip the marker
    report = run_rules(ctx)
    assert report.findings == []
    assert "authoritative" in report.context_error

def test_engine_finds_nothing_without_lineage():
    ctx = golden_context()
    ctx.blast_radius.assets = []              # strip the graph
    report = run_rules(ctx)
    assert all(r.status == RuleStatus.SKIPPED for r in report.results)

You can run the same experiment against the live instance: delete the lineage edges in DataHub, trigger a run, and watch the blast radius come back empty. Restore them and the six findings return.

What is written back

After approval and a passing validation, the resolution goes back into the catalog so the next person — or the next agent — inherits it.

A Decision document

save_document records what was wrong, what was done, the pull request, and a note for whoever changes this next: these surfaces hold pinned values that will not follow the catalog automatically.

Structured properties

comgu.last_run, comgu.last_validation_at, comgu.pull_request_url and comgu.incident_status, written onto the affected assets themselves.

A remediation tag

comgu:remediated, so the catalog shows at a glance which surfaces have been through this.

Read back, then reported

Every write is read again. A write that cannot be verified is reported as unverified — never assumed to have succeeded.

Two things we found in DataHub Core

get_lineage defaults to one hop. Accepting the default silently understates a blast radius — a feed three hops downstream is just as customer-visible. Comgu sets the hop count explicitly.

ES_BULK_REFRESH_POLICY=WAIT_UNTIL stalls the indexer. Every OpenSearch bulk write blocks until the next refresh — about 3 seconds regardless of batch size. Under load the MAE consumer exceeds max.poll.interval.ms, is evicted mid-batch, replays against already-written documents, and stalls without committing an offset. The catalog silently under-indexes. Diagnosis and fix are in the repository.