Skip to content
🚧 Early alpha — building the foundation. See the roadmap →

The evidence closeout — a report that can say "nothing" and mean it

Created Updated

Implements §4.3 of the Challenge 45 synthesis. Delivery detail is in CHANGELOG.md; this records the reasoning.

The question a compliance reader most needs answered is a negative one: which controls have no evidence? Three shipped surfaces claimed to answer it. None could.

SurfaceWhat it didWhat it reported
orphan-controls-no-evidence Bases viewFiltered control notes on length(evidence) == 0No recipe has ever emitted an evidence property, so the filter matched every control. The view said “nothing has evidence” no matter what the vault contained.
The GRC guide dashboardevidence_count: file.backlinks.length, and a “Controls with no evidence” view on file.backlinks.isEmpty()Hierarchy links, index links, and crosswalks are all backlinks, so nearly every control looked covered. The two errors point in opposite directions, which is worse than either alone.
orphan-controls.jsonQueried subject_id / predicate_id against :hasEvidenceThose are crosswalk-edge fields. Junction notes carry subject / predicate / object, and nothing emits :hasEvidence, so it matched nothing.

All three produced confident, well-formatted output. That is the actual severity: a coverage report is trusted precisely because it looks authoritative, and none of these failed loudly.

The tempting reading is that someone picked the wrong property name. Renaming would not have helped.

A Bases filter selects among notes that exist. “Controls with no evidence” is a claim about a relationship that is absent. There is no note for the filter to keep, so no property name produces the row. An empty result and a clean bill of health are indistinguishable.

This is why the fix is a relocation, not a correction. The gap query moved to the SQL index, where a LEFT JOIN can emit a row for a concept with no match. Bases keeps the questions it can answer honestly: reports about the evidence that is there.

Deliberately strict, because the failure mode is someone concluding a control is satisfied when it is not. A junction counts only when all five hold:

RuleWhy
Predicate is exactly has_evidenceOther relationships are legitimate but are not a claim that evidence satisfies a control.
Resolves to a concept by subject_curieMatching by wikilink text is how a renamed note silently detaches its evidence. Identity, never text.
status is approvedProposed and in-review evidence is not evidence.
coverage is full or partialnone and n/a are explicit statements that this artifact does not cover the control.
Not expired or staleEvidence has a shelf life.

full coverage reports as covered; partial reports as partial, which is deliberately not covered. Collapsing them is how a half-satisfied control gets reported as satisfied.

Junctions with no review dates at all are admitted. Refusing them would zero out coverage for every team that has not yet adopted review dates, reporting a data-entry gap as a compliance gap.

A strict rule with no explanation is unusable. A team whose junctions all point the wrong way would see “0 controls covered” with no way to discover they are one direction-flip from correct.

So every junction that exists but does not count is diagnosable, with exactly one reason: predicate-not-canonical, no-subject-identity, subject-not-a-known-concept, not-approved, coverage-not-asserted, expired, stale. The summary carries an excluded_junctions count — the number that tells a reader how much data was set aside before the percentages above it were computed.

Excluded junctions are never silently reinterpreted. A near-miss predicate is a data problem for a human to fix; guessing at intent is how a coverage number stops meaning anything.

The published GRC example read:

subject: "[[MFA-Policy]]"
predicate: implements
object: "[[AC-2]]"

Read literally that says the policy has a property, so the control gets nothing. Every vault built from that example would report zero coverage. The corrected contract is subject = control, object = evidence, predicate has_evidence — “AC-2 has evidence: the MFA policy.”

The recipe now fixes the predicate rather than templating it from a source column. A per-row predicate would let a spreadsheet silently emit junctions that look right and never count.

  • src/tier2/evidence-coverage.ts — the anti-join, per-concept tallies, the aggregate summary, and the exclusion diagnostics. One shared SQL fragment defines validity, so a report and its own explanation cannot drift apart.
  • The false Bases template is withdrawn, with a test asserting it stays null and a comment explaining that no template can be correct under that id.
  • evidence-junctions-by-control replaces it — the positive report Bases can answer honestly.
  • The evidence recipe is corrected (direction, fixed predicate, identity guidance) and registered, so it is reachable from the import wizard.
  • spec/recipe.schema.json gains tier2 as an output.target (additive, non-breaking).
  • The GRC guide and the metadata-ecosystem page no longer teach backlink counting.

7. The workflow half, shipped the same day

Section titled “7. The workflow half, shipped the same day”

The correctness work above left three gaps, all now closed:

Was openNow
No UI for creating an evidence linkLink evidence to a control command. It asks for “the control” and “the evidence”, never for a subject and an object, so the direction that decides whether coverage works cannot be entered backwards. The predicate is never asked about.
Nothing called the queries from a commandEvidence coverage report command. Writes a note listing gaps, partial coverage, and set-aside links, regenerated wholesale each run.
Projection status not surfacedProjection now stamps when it ran and how completely. Every report quotes it, and warns when the index is partial, errored, or unstamped.

The end-to-end test — build a link, parse it as real YAML, project it, query coverage — found that generated links carried no curie. Projection rejects a junction note without one, so every link the new command created would have been written to the vault, looked correct, and reached no report at all.

Every unit test on both sides passed while that was true. It is the same shape as the original bug: two internally-coherent halves disagreeing about a contract at the seam. The round-trip is now the regression guard.

Screenshotting the modal against a real vault caught a second one: it read the metadata cache during startup indexing, found nothing, and told the user their imported vault contained no controls. It now distinguishes “still indexing” from “nothing imported”.

  • Challenge 44 validation has not run.
  • No bulk review workflow: links are approved one note at a time.
  • The report is point-in-time. Nothing warns that an open report has gone stale behind you.

The honest status is still the one Ch 45 set: the evidence loop is now complete and tested end to end, but audit-defensible coverage waits on Ch 44 validating it against a real corpus. What changed today is that a manually reconciled pilot no longer requires hand-writing frontmatter against a contract the docs got wrong.