The evidence closeout — a report that can say "nothing" and mean it
Implements §4.3 of the Challenge 45 synthesis. Delivery detail is in CHANGELOG.md; this records the reasoning.
1. What was wrong
Section titled “1. What was wrong”The question a compliance reader most needs answered is a negative one: which controls have no evidence? Three shipped surfaces claimed to answer it. None could.
| Surface | What it did | What it reported |
|---|---|---|
orphan-controls-no-evidence Bases view | Filtered control notes on length(evidence) == 0 | No recipe has ever emitted an evidence property, so the filter matched every control. The view said “nothing has evidence” no matter what the vault contained. |
| The GRC guide dashboard | evidence_count: file.backlinks.length, and a “Controls with no evidence” view on file.backlinks.isEmpty() | Hierarchy links, index links, and crosswalks are all backlinks, so nearly every control looked covered. The two errors point in opposite directions, which is worse than either alone. |
orphan-controls.json | Queried subject_id / predicate_id against :hasEvidence | Those are crosswalk-edge fields. Junction notes carry subject / predicate / object, and nothing emits :hasEvidence, so it matched nothing. |
All three produced confident, well-formatted output. That is the actual severity: a coverage report is trusted precisely because it looks authoritative, and none of these failed loudly.
2. Why it was not a naming bug
Section titled “2. Why it was not a naming bug”The tempting reading is that someone picked the wrong property name. Renaming would not have helped.
A Bases filter selects among notes that exist. “Controls with no evidence” is a claim about a relationship that is absent. There is no note for the filter to keep, so no property name produces the row. An empty result and a clean bill of health are indistinguishable.
This is why the fix is a relocation, not a correction. The gap query moved to the SQL index, where a LEFT JOIN can emit a row for a concept with no match. Bases keeps the questions it can answer honestly: reports about the evidence that is there.
3. What counts as evidence
Section titled “3. What counts as evidence”Deliberately strict, because the failure mode is someone concluding a control is satisfied when it is not. A junction counts only when all five hold:
| Rule | Why |
|---|---|
Predicate is exactly has_evidence | Other relationships are legitimate but are not a claim that evidence satisfies a control. |
Resolves to a concept by subject_curie | Matching by wikilink text is how a renamed note silently detaches its evidence. Identity, never text. |
status is approved | Proposed and in-review evidence is not evidence. |
coverage is full or partial | none and n/a are explicit statements that this artifact does not cover the control. |
| Not expired or stale | Evidence has a shelf life. |
full coverage reports as covered; partial reports as partial, which is deliberately not covered. Collapsing them is how a half-satisfied control gets reported as satisfied.
Junctions with no review dates at all are admitted. Refusing them would zero out coverage for every team that has not yet adopted review dates, reporting a data-entry gap as a compliance gap.
4. Strictness needs an escape hatch
Section titled “4. Strictness needs an escape hatch”A strict rule with no explanation is unusable. A team whose junctions all point the wrong way would see “0 controls covered” with no way to discover they are one direction-flip from correct.
So every junction that exists but does not count is diagnosable, with exactly one reason: predicate-not-canonical, no-subject-identity, subject-not-a-known-concept, not-approved, coverage-not-asserted, expired, stale. The summary carries an excluded_junctions count — the number that tells a reader how much data was set aside before the percentages above it were computed.
Excluded junctions are never silently reinterpreted. A near-miss predicate is a data problem for a human to fix; guessing at intent is how a coverage number stops meaning anything.
5. Direction is load-bearing
Section titled “5. Direction is load-bearing”The published GRC example read:
Read literally that says the policy has a property, so the control gets nothing. Every vault built from that example would report zero coverage. The corrected contract is subject = control, object = evidence, predicate has_evidence — “AC-2 has evidence: the MFA policy.”
The recipe now fixes the predicate rather than templating it from a source column. A per-row predicate would let a spreadsheet silently emit junctions that look right and never count.
6. What shipped
Section titled “6. What shipped”src/tier2/evidence-coverage.ts— the anti-join, per-concept tallies, the aggregate summary, and the exclusion diagnostics. One shared SQL fragment defines validity, so a report and its own explanation cannot drift apart.- The false Bases template is withdrawn, with a test asserting it stays null and a comment explaining that no template can be correct under that id.
evidence-junctions-by-controlreplaces it — the positive report Bases can answer honestly.- The evidence recipe is corrected (direction, fixed predicate, identity guidance) and registered, so it is reachable from the import wizard.
spec/recipe.schema.jsongainstier2as anoutput.target(additive, non-breaking).- The GRC guide and the metadata-ecosystem page no longer teach backlink counting.
7. The workflow half, shipped the same day
Section titled “7. The workflow half, shipped the same day”The correctness work above left three gaps, all now closed:
| Was open | Now |
|---|---|
| No UI for creating an evidence link | Link evidence to a control command. It asks for “the control” and “the evidence”, never for a subject and an object, so the direction that decides whether coverage works cannot be entered backwards. The predicate is never asked about. |
| Nothing called the queries from a command | Evidence coverage report command. Writes a note listing gaps, partial coverage, and set-aside links, regenerated wholesale each run. |
| Projection status not surfaced | Projection now stamps when it ran and how completely. Every report quotes it, and warns when the index is partial, errored, or unstamped. |
What the round-trip caught
Section titled “What the round-trip caught”The end-to-end test — build a link, parse it as real YAML, project it, query coverage — found that generated links carried no curie. Projection rejects a junction note without one, so every link the new command created would have been written to the vault, looked correct, and reached no report at all.
Every unit test on both sides passed while that was true. It is the same shape as the original bug: two internally-coherent halves disagreeing about a contract at the seam. The round-trip is now the regression guard.
Screenshotting the modal against a real vault caught a second one: it read the metadata cache during startup indexing, found nothing, and told the user their imported vault contained no controls. It now distinguishes “still indexing” from “nothing imported”.
8. Still open
Section titled “8. Still open”- Challenge 44 validation has not run.
- No bulk review workflow: links are approved one note at a time.
- The report is point-in-time. Nothing warns that an open report has gone stale behind you.
The honest status is still the one Ch 45 set: the evidence loop is now complete and tested end to end, but audit-defensible coverage waits on Ch 44 validating it against a real corpus. What changed today is that a manually reconciled pilot no longer requires hand-writing frontmatter against a contract the docs got wrong.