Linkage loop — raising corpus health to publishable, as an agent¶

DEE-51. A procedure for an agent to iteratively raise a brain's linkage quality until it clears the publish gate, and — just as important — to recognise when the number in front of it is wrong.

This is written for an agent driving the MCP tools. It is not a description of the metrics; it is the order of operations, the decision rules, and the failure modes, in the order you hit them.

Raising linkage is not the same as improving an answer, and the two have been measured diverging: calliope went 68 → 95 here while the reference question's verbatim share stayed at 8–11%. If your goal is a better answer to a specific question, start from QUESTION_LOOP.md — it scores against a rubric and routes work here only when linkage is what is actually wrong.

TL;DR¶

loop:
  1. corpus_health          → read tier + population, not just the score
  2. deterministic tier     → run to a FIXED POINT (actions chain; one pass is not enough)
  3. fold_aliases           → after anything that creates keys
  4. re-measure             → did it converge? did the score move as projected?
  5. classify what is left  → by groundedness (G0–G3), then route by the metric's `tier`
until: no metric is FAIL, the deterministic tier converges in 0 rounds,
       and every remaining escalation is G2 or G3

What you are optimising¶

certify blocks publication when the linkage report's status is fail — that is, when any metric is fail. It does not read the 0–100 score. So:

The goal is "no FAIL metric", not "a high score." A corpus can score 94 and be unpublishable because one count metric (alias_groups) is non-zero.
alias_groups and dangling_endpoints are counts, not ratios. They contribute nothing to the score and can still block you. alias_groups in particular goes fail at 1.
Each ratio metric has its own floor (corpus_health returns them under thresholds, with the env var that moves them). Do not assume 90%; quote_reachability's floor is 70% and provision_node_ratio's is 95%.

The loop¶

1. Read the metrics as a worklist, not a verdict¶

corpus_health(brain_id, ref) returns, per metric:

field	use
`tier`	who owns moving it — `deterministic`, `judgement`, `coverage`, `extraction`, `none`
`population`	`{kind, ref, impact}` — the addressable things a route would act on
`thresholds`	the floor in force, and the env var that sets it
`lens`	whether the index was built from this corpus

Unfixable items are already out of the denominators. A range or list — 12 CFR 1026.19(e) and (f), 15 U.S.C. 1691 et seq. — denotes no single provision, so refusing it a key is correct rather than a defect, and it is excluded from provision_node_ratio. dangling_endpoints likewise splits into absent_provision and prose_concept because they route to different tiers. So a residual is real work, not an artefact you can dismiss: if a metric will not move, the reason is somewhere else (DEE-51 AC2).

The most common mistake is reading "44% against a 90% floor" as a wall. It is a worklist. The population names the documents, entities or provisions to act on, and tier says which route owns them. Every large win in the first run of this loop came from reading the population; every stall came from reading only the ratio.

2. Run the deterministic tier to a fixed point¶

This already runs on every approve. review_service.approve calls the fixed-point runner between committing the merge and rebuilding the lens, and reports the delta on the change set as tier1_repair (DEE-51 AC1). Running it by hand is for a corpus that predates that hook, one that has drifted, or after you change anything that affects keying — not for routine ingestion. If you run it and it reports changed: false, that is the expected answer, not a failure.

One pass is not enough, and this is the single most important step.

repair resolves citation locators. That is what turns a quote into something materialize can build a node from. On a real corpus materialize_provisions offered nothing at all until repair had run, and then offered 179 nodes worth 9 health points. An agent that runs "the deterministic repair" once collects none of that.

Three actions chain, not two. resolve_grounded_citations runs last in each round, and that ordering is deliberate: it joins a citation to a node the corpus already holds, so it has more to work with once materialize has created this round's nodes. Running it first refuses citations whose target appears moments later.

from app.service import corpus_repair
out = corpus_repair.run_to_fixed_point(brain_id, actor="<you>")
assert out["converged"], out["rounds"]   # see Failure modes

Gap: there is no MCP tool for the fixed-point runner. corpus_quality_sweep applies one pass of deterministic actions. Until that is closed, call the service, or call apply_curation_action in a loop until list_curation_actions offers no deterministic kind.

3. Fold aliases after anything that creates keys¶

New keys make duplicates visible that were invisible before. Widening citation_key, enabling CITATION_KEY_V2, adding an Act mapping — each one makes two differently-spelled entities resolve to one provision, and alias_groups jumps from 0 to N.

apply_curation_action(brain_id, "fold_aliases", actor=…)   # repeat until applied == False

This is deterministic and unconditional: two names sharing a citation key denote one provision by definition. Run it after every keying change, and again at the end.

4. Re-measure, and check two things beyond the score¶

Did the tier converge? converged: false is a bug, not a budget. See Failure modes.
Did the projection match? corpus_quality_sweep reports projection: {projected_delta, actual_delta, matched}. A projection that is consistently wrong means the offers are mispriced and the loop is ranking work in the wrong order.

5. Classify what is left by its evidence, before routing it¶

tier says which route owns a metric. Groundedness says whether the evidence to decide a given citation exists at all, which is the question that decides whether anything — a repair, a model, or you — is allowed to answer it.

class	evidence	disposition
G0 furniture	the page title, the part's authority note, publisher chrome — nothing is cited	excluded from the ratio, and counted
G1 quote-named	the quote names exactly one provision the corpus holds	joined deterministically
G2 document-context	two or more candidates, or a locator whose meaning depends on the document	needs a decider — Tier 2, not built
G3 external	resolving it needs facts the corpus does not hold	refuse

from app.service import groundedness
v = groundedness.classify(citation, held_keys=keys, doc_title=doc)
v.grounding, v.candidates, v.resolves_to

Group the G2/G3 residue by (provenance.source_doc_pk, locator) before reporting it. That pair is the unit a rule is scoped to, and it collapses a flat list into something a person can act on — on the reference corpus, 17 citations became 12 families. Group by document id, never by title: source_id is a display string, and matching on it is the defect DEE-100 removed.

The rule that makes this safe, and the one to carry into anything you build on it:

The harness supplies the candidate set. A chooser may only pick from it, or refuse.

FCRA section 603(f) is 15 U.S.C. 1681a(f). An off-by-one reading makes it 1681(f) — findings instead of definitions, a different statute — and it resolves the same four citations, mis-resolves the same zero, and projects the same health delta. No score separates them. So the question is never asked: candidates is empty for G3, and a key that is not in it is not an answer that can be returned. Anything you add here should preserve that property rather than add a confidence threshold, which measures the wrong thing.

6. Route what is left, by tier¶

tier	what it means	what to do
`deterministic`	a repair moves it, no human, no model	apply it; if no action is offered, the fix is in the parser, not the corpus — see below
`judgement`	a curator must decide what a document is	`declare_unstructured` / `declare_fragments_ok` — requires `approved_by`; do not self-approve
`coverage`	the corpus lacks a document	`acquire_provision` (Scout); costs a fetch, judgement-tier by design
`extraction`	the extractor emitted something that is not a locator	no curation action moves this. Report it; do not route it to HITL
`none`	passing, or nothing actionable	skip

Order matters. On the reference corpus, judgement-tier declarations were worth +25 points and acquiring all 41 coverage targets was worth +4.6. Measure before spending fetches:

of N unjoinable citations, how many have a resolvable key with no node?
   → only those are addressable by acquisition

7. Stop conditions¶

Stop and report when either:

No metric is fail, the deterministic tier converges in 0 rounds, and fold_aliases offers nothing. You are publishable. Record the draft sha.
The only remaining work is extraction-tier, or deterministic-tier with no action offered, or every remaining escalation classifies G2 or G3. All three mean the next fix is code, not corpus — hand back with the population and what it needs. A G3 residual specifically means the answers are not in this corpus: it needs acquisition or an externally verified mapping, and neither is something to infer.

Failure modes — what a symptom actually means¶

These cost real time on the first run. Recognise them by their signature.

`converged: false` from the fixed-point runner¶

Something is being recreated every round. Almost always a round-trip failure: an action creates a node whose name does not key back to the key it was built from, so the next pass does not recognise it and creates it again.

On the reference corpus this produced 67 copies of each of five appendix nodes, and alias_groups read them as one well-formed provision the whole time — the score said nothing was wrong. Check:

from app.service.corpus_repair import _display_name
from app.service.entity_resolution import citation_key
assert citation_key(_display_name(key)) == key

Never raise max_rounds to make it converge. That hides the churn and grows the corpus faster.

The score moves the wrong way after a correct action¶

Check what the metric actually counts before concluding the action was wrong. Removing superseded 12 CFR Part 202 from the reference corpus lowered the score, because the superseded copy was better linked than the current one. The health score does not know which of two versions is in force, and will reward keeping repealed law.

A score drop is not proof of a mistake, and a score rise is not proof of correctness.

A projected delta that does not match the actual¶

The projection is computed on a candidate snapshot. If that snapshot is not what gets committed — commit_to_draft(normalize=True) derives hierarchy edges on write — the projection describes a corpus that never exists. This mispriced materialize_provisions at −1 when it was worth +9, so the sweep ranked the biggest available win below do-nothing actions.

A curator's declaration that changes nothing¶

Check that the declaration matched. unstructured_titles returns SourceDoc.title verbatim, and scraped titles arrive wrapped in newlines ("\n Federal Register\n :: \n …"). If the matching does not collapse whitespace the same way _doc_of does, the declaration silently applies to nothing — and the panel shows the judgement recorded with the score unmoved, which reads as "the declaration was wrong" rather than "the declaration never applied." Four of 32 declarations failed this way, costing 187 citations and ~8 points.

A classifier that resolves more than you expected¶

Two traps, both silent, both found building this.

Order of evidence. An authority note names provisions by construction — "Authority: 12 U.S.C. 5512, 5581; 15 U.S.C. 1691b" names three. Classify on the candidates first and every authority note in the corpus reads as outstanding linkage work with a confident target. Furniture has to be decided before candidates, because what a thing is outranks what it mentions.

A blind candidate builder looks exactly like an empty one. appendix_key anchors to the whole string, so a table-of-contents line naming an appendix and then saying what it is yielded nothing — 13 citations classified as document furniture while naming a provision the corpus held. When a class comes back larger than you expect, check whether the parser can see the references before concluding there are none. The failure is silent in the direction that loses work.

A projected delta measured against a different corpus¶

The other cause of the same symptom, and the one to check when the snapshot is what gets committed. Every preview in corpus_repair scores with corpus_health.health(snapshot) — no declarations applied. The panel scores with them. On calliope that is the difference between a baseline of 77 and one of 94, so an offer's projected_delta is a delta on a corpus configuration that is not the one you are looking at. The sign is reliable; the magnitude is not. Measure the real delta with health(snap, unstructured=…, fragments_ok=…) on both sides before reporting a number.

The instruments themselves¶

On the first run of this loop, three of the four largest wins were bugs in the measurement, not in the corpus. A loop cannot fix its own instruments: if the score is wrong, it optimises the wrong thing confidently. When a metric refuses to move despite correct-looking work, suspect the metric.

What this loop does not do yet¶

The procedure above is Tiers 1 and 3 of DEE-51 — deterministic repair, and the human judgements that ride the existing escalation machinery. Tier 2 is not built. G0 and G1 are; they are the half of the groundedness ladder that needs no model, and they were built first precisely because they produce the candidate builder and the verifier that Tier 2 depends on.

Tier 2 is where an LLM reads a cluster of unresolved locators and proposes a rule — "in documents of this family, locators of shape X denote provision Y" — never an edit. The distinction is the whole design: a rule is inspectable before acceptance, testable against the whole corpus (how many does it resolve? does it mis-resolve any already-correct one?), reversible as a unit, and reusable on the next document of that shape. An individual edit is none of those.

You are standing at Tier 2's boundary when: the deterministic tier converges in 0 rounds and the escalations that remain classify G2 — the evidence is there but it takes judgement to read, typically a quote naming two held provisions where only one is the subject. That is the one place a reasoning call does work a parser cannot, and it is bounded work: it chooses among candidates the harness built, it does not produce a key.

A G3 residual is a different hand-back. The evidence is not in the corpus at all, so no amount of reasoning over it helps — that needs acquisition, or a mapping verified against the statute's own source credit. Do not let a G3 population become a Tier-2 backlog; it will produce confident wrong answers, which is worse than the gap.

One prerequisite remains. The previews score without declarations, so the projected delta a proposal carries is measured against the wrong baseline — see the failure mode above.

Citation provenance is now populated (DEE-100): every citation carries its document's source_doc_pk, so the G2 population groups by document with no string comparison. On a corpus written before that fix, run the one-time backfill first — a title match done once, under review, rather than implicitly on every read:

bb corpus backfill-provenance --brain-id <brain>            # dry run; read the unmatched list
bb corpus backfill-provenance --brain-id <brain> --apply

An unmatched citation is safe to leave — every consumer still falls back to source_id. A wrongly matched one is not, because nothing downstream re-checks it, which is why the backfill reports what it could not identify instead of guessing.

Also unbuilt, and named in the ticket as "the part that must not be dropped for scope": the alias write-back. An accepted rule should persist as (doc_origin, locator_pattern) -> provision so the same defect never re-escalates and the next document of that family ingests clean. Without it the loop is a treadmill — the same citations re-escalate on every re-ingest. LocatorResolution.rule has carried a field for exactly this since P1a with nothing consuming it.

Things that need a human¶

Do not self-approve these. The gate exists because getting them wrong is invisible afterwards.

judgement-tier actions. Declaring a document unstructured changes what the score counts as work. Wrong either way, and neither shows up in the score.
Acquisition. Fetching third-party content into a legal corpus is a decision, not a repair.
Encoding a citation mapping. If you add an Act→U.S. Code mapping or similar, verify each entry against the statute's own source credit (uscode.house.gov prints Pub. L. 90-321, title VIII, §814 on the section page) and verify the range against the subchapter listing. ECOA's subchapter reads 1691, 1691a … 1691c, 1691c-1, 1691c-2, 1691d …, so offset arithmetic past §704 lands one or two provisions off. Refuse outside what you verified — a wrong key merges two statutes onto one node and every metric then reports the corpus as healthier for it.
Publishing. The loop raises linkage. certify is a separate human act with its own safety gate.

Verify the corpus after running the test suite¶

Corpus state is a product artefact and the suite has been able to write to it (DEE-99). Before trusting a measurement taken after a test run:

git -C ./data/git/<brain>.git log --format='%h %s' refs/heads/draft | head

Commits from a test actor (curator@example.com) or unexplained approve change set entries mean the corpus drifted underneath you. Re-fold and re-measure.

Worked example¶

The reference run, calliope 68 → 94, publishable, no LLM in the loop:

step	action	effect
1	deterministic tier to fixed point	+9, 179 nodes, joinability 7.7% → 26.5%
2	`quote_reachability` honour declarations (bug fix)	—
3	declare 32 secondary documents unstructured	68 → 75
4	declaration whitespace matching (bug fix)	79 → 87, reachability out of FAIL
5	enable `CITATION_KEY_V2` after reviewing its merge list	`provision_node_ratio` FAIL → WARN
6	CFR prefix `12 CFR § 249.20(a)` (parser fix)	88 → 92
7	Act→U.S. Code table, verified	`provision_node_ratio` 85% → 90.7%
8	appendix keying + fold	joinability → 91.4%
9	audit the documents no declaration matched; declare 3	joinability out of FAIL
10	classify escalations; join 76 G1, exclude 14 G0	94 → 95, joinability 91.4% → 95.4%

Note steps 2, 4 and 6: three of the nine were fixes to the measuring apparatus. Expect that.

Step 10 is worth reading for its ratio rather than its points. It moved the score by 1 and joinability by 4 points, because most of what it joined sits in documents already declared unstructured — genuinely resolved, correctly uncounted. Score movement is a poor proxy for work done, in both directions.

Reference¶

app/service/corpus_health.py — metrics, tiers, populations, thresholds
app/service/corpus_repair.py — the deterministic tier, run_to_fixed_point, fold_aliases
app/service/groundedness.py — the G0–G3 ladder, the candidate builder, and why the candidate set rather than a confidence score is what makes a resolution safe
app/service/curation_actions.py — offers, KIND_TIERS, the routing in routed_actions
app/mcp/server.py — the tool surface, and app/mcp/_autonomy.py for what an agent may take unattended
MCP (operator) — identity, scopes, autonomy classes