v0.1 schema spec — governing contracts and representations
§1 Why this page exists
Section titled “§1 Why this page exists”The Foundation phase intro states: “The three interconnected schemas (_crosswalker metadata, ImportRecipe, StewardshipProfile) must be designed together — getting any wrong is expensive to fix later.” The v0.1 stack-pivot makes this concrete by committing to a buildable Tier 1 + Tier 2 sidecar architecture.
This page is the unified v0.1 schema reference. It distinguishes governing Schema Contracts, the artifacts they govern, and derived representations:
- Tier 1 Schema Contract —
spec/tier1.schema.jsongoverns canonical Tier 1 member-record representations, including_crosswalkerprovenance, concept records, crosswalk assertions, and junction/evidence assertions. - Recipe Schema Contract —
spec/recipe.schema.jsongoverns ImportRecipe, an intensional Recipe subtype that drives generation but remains outside Tier 1 knowledge. - Tier 1 record shapes —
_crosswalkermetadata and the junction-note/crosswalk/concept discriminators are member representations within concrete Knowledge Sets. v0.1 does not claim a generalizedKnowledgeSetaggregate schema. - Tier 2 sidecar SQL representation — a qualified Tier 1 → Tier 2 projection into queryable SQLite. Tier 2 is deletable and recoverable from Tier 1.
Artifact roles and authority is the governing conceptual model: Schema Contracts govern; Knowledge Sets are extensional; Recipes are intensional; future Execution Records and Package Manifests are separate administrative envelopes.
The earlier-written config-schema-design and constraint-enforcement pages stay as background design rationale; this page is the consolidated v0.1 build target.
Where this spec is implemented (forward-links to milestones)
Section titled “Where this spec is implemented (forward-links to milestones)”Each schema section in this spec is implemented by specific v0.1 implementation milestones:
| Spec section | Implementing milestone(s) | Status |
|---|---|---|
§3 _crosswalker metadata block | v0.1.3 — Generation engine integration (provenance writer); v0.1.1 — Type system + validation (AJV) | ✅ Done |
| §4 ImportRecipe + render() | v0.1.2 — render() v1 (pure function); v0.1.4 — Junction notes + crosswalk edges (kind dispatch); v0.1.4.5 — Streaming refactor (AsyncIterable rows) | ✅ Done |
| §5 Junction note 13-field schema | v0.1.4 (kind: junction-note dispatch); Ch 07 resolution | ✅ Done |
| §7 Tier 2 sidecar SQL schema | v0.1.5 — Tier 2 sqlite-wasm sidecar (substrate, projector, query API, and closure cache) | ✅ Done |
| §7 Recursive-CTE closure cache | v0.1.5 Phase 3 — see Ch 18 deliverable for the algorithmic patterns and engineering scale model | ✅ Done |
| §8 Cross-schema invariants | All v0.1 milestones | Enforced via test harness |
Higher-level system view: see the system architecture page for the 6-layer view (import / storage / projection / query / export / audit) showing how each schema fits the broader pipeline.
Naming history: FrameworkConfig → ImportRecipe
Section titled “Naming history: FrameworkConfig → ImportRecipe”The type now called ImportRecipe was originally proposed as FrameworkConfig in the config-schema-design page (2026-04). On 2026-05-03 the name was changed for two reasons:
- General-ontology positioning. “Framework” baked the GRC use case into a type that’s actually general — any structured ontology (compliance frameworks, biomedical taxonomies, library classifications, custom domain hierarchies) can be imported via this recipe. “Ontology” is the broader term that already pervades the project’s vocabulary (ontology lifecycle, ontology evolution, ontology diff primitives).
- Recipe vs config. “Config” sounds like settings; this artifact is actually a reusable transformation recipe applied to a source. A recipe is shareable, version-controlled, and replayable — semantics that “config” doesn’t carry.
Companion field renames in _crosswalker metadata and junction-note schemas:
framework_id→ontology_idframework_version→ontology_versionframework(junction-note field) →ontologyconfig_id(in_crosswalker) →recipe_id
Folder convention (Frameworks/...) is user-controlled via recipe.output.base_path — examples in this doc still use Frameworks/NIST-800-53-r5/ because NIST 800-53 is a framework and that’s the natural choice; non-GRC users would set base_path to Ontologies/, Standards/, Domain/X/, etc.
Historical decision logs (zz-log/) and research deliverables (zz-research/) are preserved verbatim with the original FrameworkConfig / framework_id naming — they are dated decision records, not living spec.
§2 The schemas at a glance
Section titled “§2 The schemas at a glance”Cross-contract and representation invariants (load-bearing):
- The Recipe Schema Contract governs ImportRecipe; the Tier 1 Schema Contract governs generated member records. Validity in one contract does not imply validity in the other.
- ImportRecipe remains adjacent intensional policy, not Tier 1 ontology/mapping content.
- Every field in the Tier 2 SQL representation must be derivable from Tier 1 frontmatter.
- Every Tier 1 frontmatter field that needs to be queryable in Tier 2 must be a flat scalar or a wikilink (no nested objects, no inline expressions). Bases-queryable constraint.
- Managed ownership is declared: Crosswalker updates only recipe-managed top-level fields plus
_crosswalker;user_preservefields remain untouched. - Junction notes are generated member records, but users can edit them; v0.1 generation is non-destructive.
- ImportRecipe may be bundled, imported, instantiated from a template, or edited through the workbench. Its canonical specification is preserved separately from implementation-only
ImportMapping,RecipeDocument,SavedConfig, and draft state.
§3 Tier 1 frontmatter contract (canonical Markdown records)
Section titled “§3 Tier 1 frontmatter contract (canonical Markdown records)”Tier 1 (the canonical Markdown representation) is governed by spec/tier1.schema.json. A recipe may manage declared top-level fields, while user_preserve fields remain user-owned. Every generated record also carries _crosswalker provenance (where the record came from).
_crosswalker provenance block
Section titled “_crosswalker provenance block”| Field | Required | Meaning |
|---|---|---|
spec_version | Yes | Tier 1 contract version used to write the record |
source_ref | Yes | Stable or human-readable source reference. Five properties, none of them individually required, plus additionalProperties: false (source: spec/tier1.schema.json § $defs.provenance_block.properties.source_ref): file, url, curie, version, and source_hash, the fingerprint of the source bytes the record was produced from. What validation actually enforces is minProperties: 1 — any single one of the five satisfies it, version or source_hash alone included. The schema’s own description text states a stricter intent, “At least one of file/url/curie required”, but that sentence is descriptive and the current contract does not enforce it; do not rely on it when validating third-party records. See the source-byte subsection |
produced_at | Yes | ISO 8601 production timestamp |
producer | No | Producer name and version |
recipe | No | Reference to the recipe used for this output. Three optional properties, none required (source: spec/tier1.schema.json § $defs.provenance_block.properties.recipe): id, the recipe identifier; hash, the long-standing effective-target receipt, still typed as a plain string with no pattern — deliberately left permissive so existing records keep validating; and recipe_document_digest, added 2026-09-06, strictly ^sha256-[a-f0-9]{64}$ via $defs.recipe_document_digest. Written by generation since 2026-10-02 (Run again slice) whenever a recipe id is written (source: src/generation/provenance.ts § buildProvenance) — see the digest subsection |
import_set | No | Durable ownership stamp. Nine properties, two required when present (source: spec/tier1.schema.json § $defs.provenance_block.properties.import_set): id (required) matches ^iset-[a-z0-9]{6}$; scheme (required) is fixed at mint, closed enum endpoint-v1 or set-qualified-v1; parent_set (optional) matches the same ID pattern and records the framework import set whose run produced this crosswalk edge set; destination (optional) records its last written folder; ontology (optional) pins the CURIE prefix; derivation (optional) pins the identity rule, closed enum filename-stem-v1 or declared-facts-v1; nest_identity (optional) pins nested-level identity modes; outline_identity (optional, Tier 1 SchemaVer 1.9.0) pins the ordered outline depth columns and id template; mapping_form (optional) pins how the set stores its mapping rows, closed enum notes or table, absent means notes. See the two subsections below |
concept_cid | No | Content identifier for the declared concept-identity scope; layout and mapping-only defaults do not enter a concept’s scope |
review_cid | No | The same identity record hashed with the review-normalization rule applied to every string value, so cosmetically-different content (citation markers, curly quotes, collapsed whitespace, markdown link destinations) hashes identically here and differently in concept_cid. This normalized whole-row hash is the only changed/not-changed gate. Absent means the producer computed none, which is not a claim that the content is unchanged |
review_groups | No | Three recipe-driven explanation hashes, all required when the block is present: wording, scope, and housekeeping (source: spec/tier1.schema.json § $defs.review_group_cids). They classify a changed review_cid; they never replace or narrow the whole-row gate |
layout_values | No | Added 2026-10-27 (SchemaVer 1.8.0). The value each folder level of the note’s path rendered from its source row, in path order: an array of { level, value, identity? } (source: spec/tier1.schema.json § $defs.provenance_block.properties.layout_values). level is the layout entry’s declared level name, value is the folder name that level rendered, and identity appears only on an implied concept level. Written by generation on every concept note with folder levels (source: src/generation/provenance.ts § buildProvenance). CSV export reads it to rebuild folder-level source columns from the note itself rather than from a path the user may have renamed; notes written before 1.8.0 fall back to the folder hub, the heading, then the path |
Source-byte digest: what source_hash records
Section titled “Source-byte digest: what source_hash records”source_ref.source_hash (a fingerprint of the bytes a record was produced from) answers exactly one question: which source bytes made this note? Its schema definition has not changed and is not new — a plain string, deliberately unpatterned, described as “Hash of the source file at import time. Format: sha256-{hex}” (source: spec/tier1.schema.json § $defs.provenance_block.properties.source_ref.properties.source_hash). It has always been optional and permissive so that external producers writing valid Tier 1 keep validating; no schema version, $id or pattern moved when the runtime producer described below was added.
What produces it today. Two producers, one of them new:
| Producer | Emits | Notes |
|---|---|---|
| The external fixture CLI | Yes, long-standing | Pre-existing raw-byte provenance; unchanged |
Native .xlsx import in the plugin | Yes, new 2026-09-07 | sha256- plus 64 lowercase hex characters over the complete pre-parse workbook payload, computed before any decode (computeSourceByteDigest() in src/generation/hash.ts) |
| Native CSV, JSON and SSSOM import | No | Unchanged. These do not synthesize a digest from decoded text or parsed rows, because a fingerprint of decoded output answers a different question |
The XLSX parse reads its file once and keeps those bytes privately for the life of the join handle it returns, so a declared source.joins secondary sheet decodes from the same snapshot rather than a fresh read that could return different bytes. The guarantee is scoped to one parse, not to a wizard session: the earlier sheet-name listing is a separate discovery read that takes no snapshot and computes no digest, so what the digest identifies is the bytes behind one import’s rows and joins. Rationale and the worked example are in the XLSX source snapshot log.
What the field does and does not assert.
- Absent means unrecorded, never unchanged. A native runtime import without a recorded byte digest omits
source_hash; existing notes and output from external producers may already carry it. A mixed vault is expected, and nothing backfills the field — existing skip, no-op and ownership-preservation rules stay authoritative over any urge to rewrite a note just to populate provenance. - It identifies bytes, not selectors. Sheet choice, header row and filters are not part of it; two imports of one workbook with different settings produce the same digest.
- It is not a signature and not a replay record. It proves nothing about who produced the record, does not store the source, does not diagnose drift against a file selected later, and is not the source/run binding that v0.1.7 Track 2 still owes.
- The engine does not verify a caller-supplied value.
buildProvenance()insrc/generation/provenance.tsemitssource_hashwhen a truthy value is supplied and omits it otherwise; that function is unchanged. A programmatic producer supplying its own digest is responsible for it identifying the original source bytes from which the accompanying rows were parsed, not a serialization of those rows.
Recipe-document digest: two reference fields
Section titled “Recipe-document digest: two reference fields”A recipe-document digest (a fingerprint naming one complete recipe revision) answers a question the existing hashes cannot: which exact revision of this recipe is this? It is stored in exactly two places, both optional, both new on 2026-09-06:
| Schema | Location | Means |
|---|---|---|
spec/recipe.schema.json (SchemaVer 1.10.0) | metadata.based_on.recipe_document_digest | The complete ancestor recipe named by based_on.recipe |
spec/tier1.schema.json (SchemaVer 1.7.0) | _crosswalker.recipe.recipe_document_digest, inside $defs.provenance_block.properties.recipe | The complete recipe referenced by that note’s provenance |
Both accept only ^sha256-[a-f0-9]{64}$ through a local $defs.recipe_document_digest in each schema. Both are optional, so recipes and notes written before this addition still validate, and neither schema $id changed.
Normalized input, and what that includes. The digest is computed by computeRecipeDocumentDigest(recipe) in src/generation/hash.ts, which is sha256- plus the SHA-256 of canonicalStringify() over the whole recipe object. It neither validates nor normalizes what it is given: its precondition is a validated canonical recipe already put through normalizeRecipe() (src/import/recipe-document.ts § normalizeRecipe). Any external producer computing this digest must therefore do both steps first — passing raw recipe JSON straight in yields a digest for a different object than the one Crosswalker would identify. Keeping normalization on the caller also keeps the module graph acyclic — recipe-document.ts imports hash.ts, never the reverse.
Because normalization runs first, the digest covers the defaults normalizeRecipe() supplies: spec_version, target.linkStyle, the folder variadic defaults (segment, drop_last, max_depth, on_overflow), and the also_emit.body defaults (position, format, omit_if_empty, and heading_depth for section entries) — sourced from that function. The practical consequence for external producers: an omitted default and an explicitly written equivalent default identify the same revision and digest identically. This is a digest of a normalized canonical recipe, not of arbitrary raw recipe JSON, and object key order and input-file whitespace do not affect it while array order does.
How it differs from the existing recipe hash. computeRecipeHash() hashes a deliberately reduced slice: layout, also_emit, enrichment, auto_heading, target.crosswalks, source.nest, source.outline, source.where, and source.joins (source: src/generation/hash.ts § recipeHashCanonicalInput) because it answers “did anything that changes output behavior change?” The document digest covers the whole recipe, so a changed recipe id, title, or ancestry reference changes the digest while leaving the effective-target hash fixed. Both meanings, and every value already written under them, are unchanged by this addition.
No self-field and no stripping. Digests live on references to other documents, so a recipe never carries its own digest: an own-digest property on recipe metadata is rejected by the closed metadata schema. Nothing is excluded before hashing either — an ancestor digest string already present in a recipe is ordinary document content and participates in that recipe’s digest, as does any similarly named key inside an opaque block.
What emits it today. Only the existing fork path: when patchRecipeDocument() customizes a recognized recipe and writes metadata.based_on, it records the digest of the normalized original before the descendant’s edits (src/import/recipe-document.ts § patchRecipeDocument). Fresh documents still get no ancestry, and a no-op patch mints nothing. The Tier 1 field is now written. Since the Run again slice (2026-10-02), generation stamps _crosswalker.recipe.recipe_document_digest on every note whose provenance carries a recipe id. Source retention is still not built.
What is pinned to the set, and why
Section titled “What is pinned to the set, and why”scheme, destination, ontology and derivation are all recorded on the set rather than recomputed per run. They exist because the same question - where does this import already live, and under what names? - was previously answered by looking at the vault, and looking at the vault answers a different question: it describes an address, not an owner.
| Property | Required | Pinned when | What a run does with it |
|---|---|---|---|
id | Yes | Minted once, crypto-random and meaningless | Names the set. A refresh reuses it; nothing derives it from a recipe, a destination, or a source, because all three legitimately change |
scheme | Yes | Mint | Fixes crosswalk-assertion identity for the life of the set. A refresh may not change it |
parent_set | No | When a framework run produces crosswalk edge notes | The producing framework set’s id, recorded on edge notes only. A later framework refresh reuses the edge set only when it alone records this producer and its column recipe id. Legacy unstamped sets are not adopted (source: src/generation/crosswalk-edge-pass.ts and spec/tier1.schema.json § $defs.provenance_block.properties.import_set.properties.parent_set) |
destination | No | Re-stamped every run | A hint about where the set lives, so a refresh does not have to infer a root from a derived default. Never authority: nothing updates it when a user renames the folder in the file explorer, so a reader re-validates it against the set’s actual note paths before writing there |
ontology | No | Mint; a legacy set is pinned on its first refresh to the prefix its own notes already carry | The CURIE prefix a refresh mints under. A run uses the pin in preference to the ontology its own recipe implies |
derivation | No | Mint | Which rule turns a source row into a CURIE local part. A run uses the set’s pin, never this version’s current rule |
mapping_form | No | Mint; a refresh keeps it | How the set stores its mapping rows: notes (one crosswalk-edge note per mapping) or table (one .mapping-table.tsv file whose header carries the whole _crosswalker block once). Absent means notes, so notes-form sets omit it. Switching form is a conversion, never an edit; a set seen both as notes and as a table is refused (source: spec/tier1.schema.json § $defs.provenance_block.properties.import_set.properties.mapping_form, src/generation/provenance.ts § buildProvenance, src/generation/import-set.ts § agreedMappingForm) |
ontology prevents a specific and total failure: a refresh that recomputes the ontology from its own recipe can land on a different answer than the one the set’s notes carry - a renamed saved config, a differently named export file - and every CURIE it then writes matches none of the notes it owns. The run writes a second copy of the whole import beside the first and reports every original as an orphan. Absent on notes written before the pin existed, which is why a legacy set is pinned from its own notes rather than from the run.
derivation prevents the same failure from the opposite direction, and it is why the identity rule could be corrected at all. The rule shipped since v0.1.0 builds a concept’s CURIE local part out of its leaf filename stem, passed through the filesystem sanitizer, so AC 2, AC-2 and AC/2 all collapse onto one identity and even a declared curie column gets rewritten. Correcting that rule in place would give every note in every existing vault a new CURIE, and the next refresh would recognise none of them. So the rule is pinned instead: a set stamped filename-stem-v1, or carrying no stamp at all, keeps the legacy derivation byte-exact forever, and only a newly minted set derives under declared-facts-v1 (source: src/generation/import-set.ts § derivationOf).
declared-facts-v1 consults the source’s own identity columns first, in the order curie, id, subject_id, control_id, code, and falls back to the filename stem only when none is present. A declared curie outside the CURIE charset is refused for that row by name rather than rewritten. Any sanitization that does happen is injective: a value already inside the charset passes through unchanged, and any other value becomes <safe form>..<short digest of the exact raw value>, so two distinct source values can never produce one CURIE (source: src/generation/curie.ts § injectiveCurieLocalPart).
Independently of any pin, a single run refuses to produce one CURIE twice. The second row to claim an identity is reported by name alongside the row that claimed it first, and nothing is written for it. That guard changes no identity, so it applies to legacy sets as well as new ones (source: src/generation/generation-engine.ts § duplicateCurieMessage).
Neither optional property is an ownership decision. Ownership is carried only by id, and the set a run writes under is chosen by the caller, never inferred by the engine from what happens to sit in the destination folder (source: src/generation/import-set.ts § resolveImportSet).
Crosswalk assertion identity
Section titled “Crosswalk assertion identity”Crosswalk assertion identity (the stable name of one mapping note) follows the owning set’s fixed scheme. Both values and forms below are defined by spec/tier1.schema.json § $defs.provenance_block.properties.import_set.properties.scheme:
endpoint-v1:cw-<sanitized-subject>-<sanitized-object>. This is the original form and remains byte-identical on refresh.set-qualified-v1:cwset-<import-set-id>-<sanitized-subject>-<sanitized-object>. Thecwset-marker makes the form visibly distinct; the minted set id lets equal endpoint pairs in different import sets coexist. Row order and mapping-set metadata do not enter either form.
Endpoint sanitization follows the set’s derivation pin for the same reason the concept rule does: under filename-stem-v1 the original many-to-one form is reproduced byte-for-byte, and only a set minted under declared-facts-v1 sanitizes endpoints injectively (source: src/import/sssom-importer.ts § sssomEdgeCurie).
A refresh always reuses the stored scheme. Choosing coexistence mints a new import set whose assertions use set-qualified-v1; no existing note is migrated, moved, or re-identified. See the import-set ownership decision.
Concept-note fields
Section titled “Concept-note fields”A CURIE (a compact stable identifier such as nist:AC-2) identifies the concept independently of its vault path.
| Field | Required | Meaning |
|---|---|---|
curie | Yes | Stable concept identity |
_crosswalker | Yes | Provenance block above |
title | No | Display title |
aliases | No | Alternate names |
tags | No | Obsidian tag paths |
parent | No | Clickable parent wikilink, or wikilink array for display polyhierarchy |
parent_curie | No | Stable identity of the one canonical parent projected to Tier 2; it is never inferred from parent |
children | No | Optional child wikilinks |
Junction-note fields
Section titled “Junction-note fields”A junction note (one evidence relationship represented as its own note) keeps endpoint identity separate from endpoint address.
| Field | Required | Meaning |
|---|---|---|
curie | Yes | Stable junction-note identity |
subject | Yes | Clickable subject wikilink |
predicate | Yes | Relationship type |
object | Yes | Clickable object wikilink |
_crosswalker | Yes | Provenance block above |
kind | No | When present, fixed to junction-note |
subject_curie | No | Stable subject identity; never inferred from subject |
object_curie | No | Stable object identity; never inferred from object |
coverage | No | full, partial, none, or n/a |
reviewer, review_date, status, confidence | No | Review and lifecycle facts |
scope, expires_at, notes, tags | No | Scope, expiry, annotation, and Obsidian tags |
reviewed_against | No | The review baseline: required { curie, review_cid } plus optional complete review_groups (source: spec/tier1.schema.json § $defs.junction_note_frontmatter.properties.reviewed_against). The subject and whole-row fingerprint record what the approver read; the group hashes explain a later difference. Written only when a human approves. Absent is the named state unrecorded: the link still counts toward coverage, and the report names it rather than implying the subject is unchanged |
Crosswalk-edge fields
Section titled “Crosswalk-edge fields”A mapping set (one published collection of mapping assertions) gives repeated assertions and separate releases independent provenance. An assertion occurrence (one row from one mapping-set release) receives its own deterministic curie; the schema does not require every producer to use one particular CURIE algorithm.
| Field | Required | Meaning |
|---|---|---|
curie | Yes | Stable identity of this assertion occurrence |
subject_id | Yes | Subject concept CURIE |
predicate_id | Yes | One of the 8 relationship predicates defined by spec/tier1.schema.json § $defs.crosswalk_edge_frontmatter.properties.predicate_id (6 set-theoretic plus the release-lineage pair; see §6) |
object_id | Yes | Object concept CURIE |
_crosswalker | Yes | Provenance block above |
kind | No | When present, fixed to crosswalk-edge |
mapping_set_id | No | Non-blank stable identifier for the mapping-set release; writers trim surrounding whitespace, then preserve case and content |
predicate_modifier | No | Predicate modifier (a flag changing assertion meaning). The only valid value is NOT; absence means a positive assertion |
match_type, match_confidence | No | SSSOM-shaped match classification and confidence |
mapping_justification, mapping_provider, mapping_date | No | Why, from where, and when the mapping was asserted |
creator_id, review_status, tags | No | Creator, review state, and Obsidian tags |
Mapping-set release record fields
Section titled “Mapping-set release record fields”Added 2026-10-03 (SchemaVer 1.7.0, additive). A MappingSet (one published crosswalk release) gets one release record: its identity, version, participants, provenance, license, assertion count, and two digests. It is stored once per set in the set’s own storage form: one note with kind: mapping-set in the notes form, or the mapping_set header key of the set’s .mapping-table.tsv (the same object without _crosswalker) in the table form. The record is bound to its set by _crosswalker.import_set.id, never by file name or folder (source: spec/tier1.schema.json § $defs.mapping_set_frontmatter, src/mappings/mapping-set.ts).
| Field | Required | Meaning |
|---|---|---|
kind | Yes | Fixed to mapping-set |
mapping_set_id | Yes | The release identity: the publisher’s declared id, else minted once as urn:crosswalker:mapping-set: plus 10 base32 characters and kept. Never derived from file bytes, path, folder, tag, ontology pair, recipe id, or repeated row values |
id_origin | Yes | declared (the source stated the id) or minted (Crosswalker minted it) |
assertion_count | Yes | Mapping rows in the set, integer at least 0 |
membership_digest | Yes | sha256-<hex> over the sorted lines subject_id TAB predicate_id TAB object_id TAB predicate_modifier (STRM predicate, empty modifier as empty string). Answers “same claims?” |
content_digest | Yes | The same lines plus mapping_justification, confidence, mapping_provider. Answers “same claims, same evidence?” Review columns, labels, notes, extra fields, row ids and ordinals enter neither digest, so a review never changes a set’s integrity |
_crosswalker | Yes | Provenance block above; its import_set.id binds the record to its set |
mapping_set_version, mapping_set_title, mapping_set_description | No | As the source declares them |
subject_source, subject_source_version, object_source, object_source_version | No | Declared participants. Absent when the source declares none; prefixes observed in rows are shown in the UI labelled “observed from rows” and never written here |
mapping_provider, mapping_date, creator_id (array), license | No | As the source declares them |
Interchange. The SSSOM exporter writes the record on the 12 standard SSSOM header keys it holds (source: src/export/sssom-exporter.ts § recordHeaderLines) and nothing non-standard; digests are not exported. A re-import recomputes them, and equality is the round-trip proof (tests/mapping-set-roundtrip.test.ts). A set without a record (imported before 2026-10-03) exports a header inferred from its rows, and the export says so.
Refresh. A refresh keeps the release identity the set already holds. A file declaring a different mapping_set_id or mapping_set_version is refused before anything is written, with the action to import it as a new set.
Additive compatibility
Section titled “Additive compatibility”All six 2026-08-21 additions are optional at their containing-record boundary: parent_curie, subject_curie, object_curie, mapping_set_id, predicate_modifier, and _crosswalker.import_set. When import_set is present, both its id and scheme are required (source: spec/tier1.schema.json § $defs.provenance_block.properties.import_set). Legacy Tier 1 records without an ownership stamp therefore continue to validate. The 2026-08-31 addition of _crosswalker.import_set.derivation is optional for the same reason and carries a further guarantee: its absence is a meaningful value, not a missing one, and means filename-stem-v1. Producers clear a declared managed field when its rendered source becomes empty, but never clear a user_preserve field.
§4 Recipe Schema Contract and ImportRecipe specification
Section titled “§4 Recipe Schema Contract and ImportRecipe specification”ImportRecipe is a concrete Recipe subtype: a portable, machine-evaluable declaration of intended source-to-Tier-1 behavior. It is governed by the Recipe Schema Contract and sits adjacent to generation; it is not embedded inside the Tier 1 Knowledge Set it helps produce.
| Recipe role | Meaning in v0.1 |
|---|---|
| Specification | The canonical ImportRecipe JSON governed by spec/recipe.schema.json. |
| Template | Parameterized/default policy, such as a Shape Preset, used to instantiate or initialize a specification/mapping. |
| Binding | A Recipe reference plus concrete source identity and selected parameters. |
| Execution | One application of a bound Recipe. Verified run facts belong in a later separate Execution Record, not in mutable Recipe intent. |
Current additive fields include:
- optional display metadata and structured
metadata.based_onancestry, which since 2026-09-06 may also carryrecipe_document_digest, a strict digest of the complete normalized ancestor recipe (see the digest subsection for its input contract and how it differs from the effective-target hash); - optional
source.versionas compatibility metadata, never stronger than verified run provenance; - ordered
target.also_emit.bodyprojections (append | section×text | code | quote | list); - typed managed scalar coercion through the closed
numbertemplate filter; - omission-safe identity templates through first-position
optionaland empty-safecurie-prefix(prefix)controls; - deterministic normalized equality and canonical JSON serialization;
- four validation layers: JSON Schema, recipe semantics, source compatibility, and editable-model portability.
An ImportRecipe is a portable, deterministic instruction artifact. Bundled recipes currently drive recognition and generation; persistent user-authored recipe-file lifecycle is the next UI/storage slice.
TypeScript shape
Section titled “TypeScript shape”Example: NIST 800-53 r5 ImportRecipe
Section titled “Example: NIST 800-53 r5 ImportRecipe”SavedConfig, ImportMapping, and ImportRecipe boundary
Section titled “SavedConfig, ImportMapping, and ImportRecipe boundary”The earlier v1→v2 migration sketch is superseded by the canonical-fidelity boundary implemented in 2026-07-21.
| Object | Current role | Authority |
|---|---|---|
| SavedConfig | Legacy fingerprint-bound wizard convenience state | Not a canonical portable Recipe and not auto-migrated into one |
| ImportMapping | Editable source-bound workbench projection over Recipe-owned fields | Implementation/editor state; never the preservation authority |
| RecipeDocument | In-memory preservation/editor transaction boundary around original + editable projection | Preserves the original canonical ImportRecipe while edits patch owned regions |
| ImportRecipe | Portable Recipe specification governed by the Recipe Schema Contract | Canonical instruction artifact |
The workbench must patch a validated original rather than rebuild a Recipe from SavedConfig or ImportMapping. See the portable-recipe fidelity synthesis.
§4.5 Recipe query: block (added v0.1.6 — SchemaVer 1.1.0; additive)
Section titled “§4.5 Recipe query: block (added v0.1.6 — SchemaVer 1.1.0; additive)”Per Ch 31 schema design + Ch 36 compositional language stack, v0.1.6 added an optional query: block to the recipe schema. Recipes can now declare WHAT to query (axes, edges, aggregation) without writing SQL — the engine compiles the declared block to SQL recursive CTEs against the Tier 2 sqlite-wasm cache.
Backward compatibility: existing recipes WITHOUT query: continue to validate. The bump is purely additive (SchemaVer ADDITION); SchemaVer URI did NOT bump (https://crosswalker.dev/spec/recipe.schema.json stays).
Eight Layer A query verbs (per Ch 29 adversarial validation): filter / traverse / bind / project / aggregate / anti-join / set-op / diff. Closure folded into parameterized traverse(depth=*, transitive=true). Pivot demoted to Layer B (presentation, not value-producing).
Six view shapes (per Ch 30 view shape taxonomy): table / list / pivot / graph / hierarchy / timeline. v0.1 first-class catalog: Pivot (custom Bases view crosswalkerPivot) + Table/List/Cards (Bases-native consumed) + Hierarchy (graduates v0.1.7-v0.1.8). Graph + Timeline schema-declared; renderers ship v0.2+.
Top-level query block shape:
Pivot primitives example (the v0.1.6 launch-market Coverage Matrix shape):
Query parameters ($defs/QueryParam; source: spec/recipe.schema.json):
import-set (added 2026-10-27, SchemaVer 1.19.0, additive) lets a shipped query target one imported set the user chooses instead of a demo folder. Its value is the chosen set’s minted import set id, and the template filters on note._crosswalker.import_set.id, never on a folder path. kind says which sets are offered: framework (concept notes) or mapping (relationship notes: crosswalk edges, junctions, release records). Sets are classified by identity (a producing parent set, or relationship kind values on their notes), and a table-form mapping set or a set with no notes is not offered, because a Bases view cannot see it (source: src/views/import-set-options.ts). The parameter has no default: one candidate is preselected, otherwise the picker’s Apply stays disabled until the user chooses. Templates use {{#name}}...{{/name}} and inverted {{^name}}...{{/name}} sections, so a saved query whose params lack the set parameter regenerates byte-identically to before.
Two discriminator styles ship (per Ch 31a + Ch 31b):
| Style | Discriminator | Default? | Trade-off |
|---|---|---|---|
| A | oneOf + const | ✅ default | ”Must match exactly one schema” errors |
| B | if/then/else | advanced | Focused per-shape errors; better IDE autocomplete |
Settings → “Recipe schema → Recipe query block schema style” picks the active validator. Both styles produce identical validity verdicts; differ in error-message UX. Implementation detail: validator compiles both styles at init; buildStyleBSchema() deep-clones the schema and patches query_block.allOf[0] to reference ShapeDispatchB (strips $id so AJV compiles as anonymous variant).
Allowed string-expression language: JSONata only (per Ch 36 compositional language stack). No inline SQL, SPARQL, or Crosswalker-invented DSL inside the query: block.
Reference recipes shipped to recipes/v0-1/:
| File | Shape | Notes |
|---|---|---|
coverage-matrix.json | pivot | Launch-market NIST CSF × ISO 27001 |
crosswalk-density.json | table | Aggregates per framework pair |
orphan-controls.json | list | Demonstrates anti-join verb |
hierarchy-view.json | hierarchy | Renderer ships v0.1.7-v0.1.8 |
list-view.json | list | Minimal Bases-native list |
Implementing milestone: v0.1.6 — Bases query layer ships the schema (Phase 1 ✅) + the crosswalkerPivot view that consumes pivot-shaped queries (Phase 3) + recipe-picker UX that emits embedded \“base` blocks (Phase 4) + opt-in materialization (Phase 5).
§4.6 Layout-entry variadic block (added 2026-07-05 — SchemaVer 1.2.0; additive)
Section titled “§4.6 Layout-entry variadic block (added 2026-07-05 — SchemaVer 1.2.0; additive)”A fixed layout lists its folder levels ahead of time — one layout_entry per level. Ragged ids carry a different number of pieces per row (ATT&CK T1055 vs T1055.011; any parent/child id family), so no fixed list fits every row — the result was a flat dump. A folder layout_entry can now carry an optional variadic block that explodes the entry’s rendered scalar after templating (templates stay scalar — one value in, one value out) into a variable number of folder levels.
Backward compatibility: purely additive (SchemaVer ADDITION). Recipes WITHOUT variadic validate and render byte-identically. The spec URI did NOT bump (https://crosswalker.dev/spec/recipe.schema.json stays). variadic is schema-constrained (layout_entry allOf if/then) to mechanism: "folder" only — heading/tag variants are deferred.
Fields:
| Field | Type | Default | Meaning |
|---|---|---|---|
delimiter | string (required) | — | What the rendered scalar is split on (e.g. . for ATT&CK ids). |
segment | prefix | part | prefix | Folder-name shape. prefix: cumulative prefixes joined by the delimiter (X.Y.Z → X, X.Y) — globally-unique, CSF-convention. part: raw pieces (X.Y.Z → X, Y). |
drop_last | boolean | true | Drop the leaf piece before building folders — the file entry names it with the full id anyway. |
max_depth | integer ≥ 1 | 6 | Safety cap on folder levels produced. |
on_overflow | truncate | error | truncate | Past max_depth: truncate keeps the first max_depth + records a variadic-overflow-truncated deviation note (no data loss — full id is in the filename); error fails the render. |
Semantics (exact): render template → scalar → split on delimiter (empty pieces dropped, each → a folder-level-skipped note) → if drop_last, drop the final piece → build segments per segment mode → cap at max_depth → append each as a folder level (same path as fixed folders). Zero segments (a top-level id) is normal, not a deviation. Deterministic: segments derive only from the row’s own value.
Example (ATT&CK technique nesting — T1055 and T1055.011 from one recipe):
T1055 → Techniques/T1055.md (no parent folder); T1055.011 → Techniques/T1055/T1055.011.md.
Scope: recipe-only in this round — the import wizard doesn’t yet propose or edit variadic (planned follow-up: detect the ragged-id signature — a delimiter appearing in ~20–79% of rows — and offer variable-depth nesting). The full folder-note convention (T1055 becoming the T1055/ container as T1055/T1055.md) is a separate batch-scoped post-pass still under design.
§4.7 also_emit.frontmatter.managed_links (added 2026-07-10 — SchemaVer 1.3.0; additive)
Section titled “§4.7 also_emit.frontmatter.managed_links (added 2026-07-10 — SchemaVer 1.3.0; additive)”Single-value links live in managed as a scalar "[[…]]" string (parent: "[[T1078]]"). A multi-value link column — a cell holding several ids (Related Controls: "AC-2, AC-3, PM-9") — needs a YAML array of wikilinks so each id is an indexable graph edge, not one dead [[AC-2, AC-3, PM-9]] string. managed_links is that region.
Each key maps to { template, split? }. render() renders template to a scalar, splits it on split (default comma + semicolon), wraps each non-empty trimmed piece in [[…]], and emits the key as an array. An empty cell omits the key entirely (an empty managed array would still clobber a user value on re-import). Regenerated wholesale on re-import (managed, not unioned).
Backward compatibility: purely additive (SchemaVer ADDITION). Recipes WITHOUT managed_links validate unchanged. Spec URI unchanged.
Related Controls: "AC-2, AC-3, PM-9" → related: ["[[AC-2]]", "[[AC-3]]", "[[PM-9]]"].
§4.8 target.enrichment block (Pass 1.5; added 2026-07-10 — SchemaVer 1.3.0; additive)
Section titled “§4.8 target.enrichment block (Pass 1.5; added 2026-07-10 — SchemaVer 1.3.0; additive)”render() is per-row pure (Pass 1, hashable). Four artifacts need cross-row knowledge, so an optional batch-scope enrichment pass runs after every row renders (src/generation/enrich.ts): children lists on parents, materialized facet hub notes, honest edge stats, and (reserved) folder-note placement. It never touches render(); determinism holds at batch level — every derived list is sorted by curie, hubs by path, so input order never leaks.
Backward compatibility: purely additive. Recipes WITHOUT enrichment emit nothing new and validate unchanged. Spec URI unchanged.
Fields:
| Field | Type | Default | Meaning |
|---|---|---|---|
children_lists | boolean | false | Each parent note gains a managed children array of wikilinks to its direct children (derived from the same parent relation the parent link uses), sorted by child curie. Fully managed — regenerated wholesale on re-import (user extras inside the array are not preserved; user_preserve covers separate keys). A row note that also hosts a level folder (level_hubs: notes, for example an SP 800-53 control with enhancements) gets the array only when this flag is explicitly true (source: src/generation/enrich.ts). |
facet_notes | none | tags-only | notes | none | none: nothing beyond per-note facet tags. tags-only: facet tags only, no hub notes. notes: one synthetic hub note per facet value with a managed members list (sorted by curie), kind: facet, the facet tag, and an H1 body; user prose below the H1 survives re-import. Hubs materialize only for values with ≥2 members. |
parent_note | sibling | folder-note | sibling | Where a concept that is also a parent lives. sibling: X.md beside X/. folder-note: X/X.md — the batch enrichment pass relocates evidence-based (only parents whose children actually nest under their folder), resolves re-import write targets by curie so no duplicates arise, and relocates back symmetrically when set to sibling again. Streamed sources fall back to sibling with a deviation note (eager batch required). |
hub_note_folder | string | (output root) | Optional vault-relative folder for materialized facet hub notes. |
level_hubs | none | notes | none | Added SchemaVer 1.4.0 (§4.9). Hierarchy hub/MOC index notes — one per folder level. |
waypoint_marker | boolean | false | Added SchemaVer 1.4.0 (§4.9). Opt-in %% Waypoint %% marker on generated folder-note/hub notes. |
Stats: GenerationResult.edgeCount = parent links + children entries + facet-hub member entries + level-hub child-link entries — the review screen’s “K graph edges” line (a run shipping 0 edges is a visible warning, not a silent dead graph).
Preset defaults (stamped by instantiate onto ImportMapping.enrichment, then serialized to target.enrichment): browsable-framework → children_lists: true, facet_notes: "notes", level_hubs: "notes". The hyperconnected default IS the default.
Scope: recipe + preset land here; the workbench Connections-card UI knobs and the review-screen edge count are wired (§4.9 for the level-hubs/Waypoint-marker round). Folder-note relocation (step 2 of the design) is implemented in src/generation/enrich.ts.
§4.9 target.enrichment.level_hubs + waypoint_marker (added 2026-07-11 — SchemaVer 1.4.0; additive)
Section titled “§4.9 target.enrichment.level_hubs + waypoint_marker (added 2026-07-11 — SchemaVer 1.4.0; additive)”Closes the ICSB emitter-controls gap audit’s #1-priority gap: Crosswalker had facet hub notes (value-grouping) but no hierarchy-level index/MOC concept, unlike the ICSB pipeline’s hubs.* (Home + per-level MOCs, derived from the canonical model). Delivery detail (folder-note-as-default, per-import root folders, the review screen’s exact-count plan) is in CHANGELOG.md “Frameworks arrive furnished”, not restated here.
level_hubs: 'notes' — every FOLDER level in the generated structure gets an index note, derived from the batch’s own rendered note paths (never a filesystem scan):
- Hosted: when a folder’s own concept note already exists in the batch (sibling
X.mdbesideX/, or folder-noteX/X.md— basename is the only identity that matters, since links in this codebase are basename-only), that note gains a managed body “Contents” section listing the folder’s direct children (subfolders’ hub identities + sibling notes, sorted by curie, as[[wikilinks]]). - Synthetic: a folder with no such note (a pure structural grouping folder, or the per-import root) gets a brand-new hub note (
kind: 'hub', curie<ontology>:hub/<folder-path-slug>) at<folder>/<folder>.md. - Root hub: the per-import root folder (
GenerationOptions.basePath) gets a hub too, named after the folder — the import’s “home” note. - Re-import safe: the managed section is delimited by HTML-comment markers and stripped-and-reappended on regeneration (same discipline as
facet_notes’s H1 split) — user prose elsewhere in the note survives. - Default
'notes'on thebrowsable-frameworkpreset (the connectedness mandate — shape-workbench synthesis §3).
waypoint_marker: true — additionally appends the %% Waypoint %% trigger comment to every folder-note/hub note this import generates, so the Waypoint community plugin can also track notes a user later adds to that folder by hand. Idempotent (never duplicates or strips a %% Begin Waypoint %% block Waypoint has already expanded). Additive, not a replacement — Crosswalker’s own generated section stays the primary, always-on mechanism (Waypoint’s directory-scan approach can’t reconstruct the curated, edge-derived grouping Crosswalker’s own model produces). Default false; the workbench only offers the toggle when a Waypoint-style plugin is detected enabled.
Backward compatibility: purely additive (SchemaVer ADDITION). Recipes without these keys validate and render unchanged. Spec URI did NOT bump.
§4.10 source.where row predicate (added 2026-08-27 — SchemaVer 1.9.0; additive)
Section titled “§4.10 source.where row predicate (added 2026-08-27 — SchemaVer 1.9.0; additive)”The first declaration on source that shapes the collection rather than a value. Per the Ch 46 transform-shapes framing: value-shaping (literal keys, per-item mapping, substrings) stays in templates, which already ship it; deciding which rows exist, and what a row is extends the source contract instead.
| Property | Value |
|---|---|
| Type | string, minLength: 1, maxLength: 512 (source: spec/recipe.schema.json § source.where) |
| Language | JSONata, restricted to a closed subset (below) |
| Evaluated against | one source row, exactly as the parser produced it. Spreadsheet cells are strings, so numeric intent must be explicit: $number(count) > 5, never count > 5 |
| Position in the pipeline | where → identity → CURIE mint → concept_cid → render() → Tier 1 validate → write. Spec-owned, never recipe-owned |
| Recipe hash | participates, because it changes which notes exist. A recipe omitting it hashes byte-identically to its pre-1.9.0 self |
| Absent | every row becomes a note, exactly as before |
Permitted subset (source: src/source/expression.ts § PERMITTED_FUNCTIONS and the AST walk): field references (bare, backtick-quoted, dotted), string/number/true/false/null literals, array literals, = != < <= > >=, in, and, or, &, grouping parentheses, and 8 functions — $not $exists $trim $lowercase $uppercase $string $number $contains. SchemaVer 1.16.0 (2026-09-23) permits $contains(value, 'literal') in source.where, secondary-row predicates, and join keys, only with exactly two arguments and a string-literal second argument; a missing or non-string first argument evaluates to false (source: spec/recipe.schema.json § source.where.description and src/source/expression.ts). This lets a stack run omit identifiers containing ( without introducing regex or computed patterns. No new key is required; recipes not using this function validate and hash unchanged.
Rejected at preflight, by name: regular expressions and every regex-taking function, lambdas and the higher-order functions that need them, $eval / $lookup / $merge / $$, the ~> chain, ^(...) sort, {...} object constructors, ** descendants, conditionals, multi-expression blocks, and :=. The allowed $contains accepts only a literal second argument; regex patterns and dynamic patterns remain banned. Regex is the load-bearing ban: JavaScript regex and Python re disagree, and a producer in another language must be able to evaluate the same predicate and get the same rows.
The loudness contract is the reason this declaration is worth a schema change rather than a preprocessing step. Three guards, because one is measurably not enough:
| Guard | Catches | When |
|---|---|---|
| Result must be a strict boolean | where: "Subcategory" (returns ''), where: "Typo" (returns nothing), any non-predicate | per row |
| Every referenced name exists in the collection | Typo != '' — the case the boolean check cannot see | preflight, before any write |
| Admitting zero rows from a non-empty source | a correct column compared against a literal that never occurs | end of stream, before any write has happened |
The middle guard exists because of a measured fact: JSONata’s comparison operators absorb an undefined operand, so a typo’d column name returns a perfectly good boolean false for every row. A return-type check passes it, the import admits nothing, and nothing is reported. The rule the guard encodes is the same one the join design states for keys: absence of a field on a row is data; absence of a field from the collection is a defect. Heterogeneous JSON is why the check is a union over the collection rather than a per-row requirement.
A per-row failure aborts the run. It never skips the row: a skip that logs a warning is still a vault that quietly lost rows.
Cross-runtime behaviour is pinned as data, not prose: spec/conformance/source-expressions.json records what every permitted construct evaluates to, including the undefined-absorption case above, so a Python producer can be checked against the same file the plugin runs.
Backward compatibility: purely additive (SchemaVer ADDITION). Recipes without the key validate, render, and hash unchanged. Spec URI did NOT bump.
§4.11 source.joins keyed lookup enrichment (added 2026-08-27 — SchemaVer 1.9.0; additive)
Section titled “§4.11 source.joins keyed lookup enrichment (added 2026-08-27 — SchemaVer 1.9.0; additive)”The second collection-shaping declaration, and the one that decides what a row is. Keyed lookup enrichment only: one primary stream plus N independent, bounded lookup indexes. Not relational algebra.
The object key is the alias, constrained to ^[a-z][a-z0-9_]*$ so it is addressable as a bare template segment with no quoting (source: spec/recipe.schema.json § source.joins.propertyNames).
| Field | Contract |
|---|---|
from.sheet + optional from.header_row | another sheet of the same workbook. XOR with iterator |
from.iterator | a sibling array of the same JSON document, same closed iterator grammar as the primary |
from.where | optional predicate over the secondary rows, with exactly the §4.10 contract: same subset, same three guards, same errors. Fixed depth — there is no nested joins |
on.primary / on.secondary | key expressions, same restricted subset (including literal-only $contains), normalized identically with a trimmed string coercion. Composite keys use & |
cardinality | required, no default. "first" does not exist |
select | optional field subset for one; required and exactly one field for many |
Both secondary forms live inside the same source bytes. The source block’s own scope excludes where the bytes come from, so a join naming a second file would break that rule and single-file portability with it. A CSV source is one collection and has nothing to name: declaring a join over one is refused with the format reason.
Cardinality is an assertion about the data, not a preference (source: src/source/joins.ts):
| zero matches | one match | two or more | |
|---|---|---|---|
"one" | alias absent from the row | alias = the matched row, or the select subset of it | error naming row, alias, key and count |
"many" | alias absent from the row | list of length 1 | list, in secondary-collection order |
Zero matches leaves the alias absent rather than empty in both modes, so the existing |optional filter handles a miss identically either way. A single match under "many" stays a list, because output shape must never depend on how many rows happened to match.
"many" binds a list of scalars, never a list of objects, and that is a deliberate consequence of the frozen template grammar: path traversal does not lift over arrays (source: src/render/template.ts), so a list of objects would require extending the grammar. Instead, a many list lands directly in also_emit.frontmatter.managed as a YAML array, in managed_links, or in a body projection with format: "list".
Collision policy: never shadow, never merge, never auto-rename. A joined row gains exactly one key — the alias — and its value is the match. Nothing enters the primary row’s own namespace, so a secondary column can never overwrite a primary one. An alias equal to an existing column, or to a reserved render-scope name, is refused at preflight. The consequence is the point: source.joins requires zero template-grammar change, because the alias is an ordinary nested value reached by the dotted traversal and the ['literal key'] quoting that already shipped in SchemaVer 1.7.0 — {mitigation.['Mitigation Name']} composes at any depth.
Loudness, mirroring §4.10: a key expression that yields nothing, an empty string after trimming, or a non-scalar is an error naming the row. No match is not an error — absence of a relation is data, absence of the key is a defect. A secondary collection that is empty, or that from.where narrows to nothing, is an error: an index that can never match is a defect.
Order in the pipeline: where → joins → identity → CURIE mint → concept_cid → render(). where runs first so it stays a pure function of the source row, so a discarded row costs no lookup, and so inner-join semantics cannot arrive through where: "$exists(alias)" — at the time where runs, no alias exists, and the reference preflight rejects the name. A joined value therefore participates in concept_cid, which is correct: the alias is part of what the concept asserts.
Streaming and cost: the primary collection is still consumed row by row; only the secondary is materialized. Measured against ATT&CK’s relationships at 18,570 rows, the index builds in ~25 ms, and a 200,000-row primary import against it never holds more than one primary row at a time. Retained memory is a function of the secondary only.
Two-hop is out of scope, permanently. ATT&CK’s mitigation prose lives two hops away (technique → relationships → mitigations), and hop 2’s key depends on hop 1’s result set — a join composing with a join, which is the relational algebra this design excludes by name. A technique note gains mitigation identifiers; names and prose are producer work. An honest half-close beats a silent one.
Backward compatibility: purely additive (SchemaVer ADDITION). Recipes without the key validate, render, and hash unchanged. Spec URI did NOT bump.
§4.12 Delimiter-set template filters part / prefix (added 2026-09-15 — SchemaVer 1.11.0; additive)
Section titled “§4.12 Delimiter-set template filters part / prefix (added 2026-09-15 — SchemaVer 1.11.0; additive)”$defs/template only; no structural key changes (source: spec/recipe.schema.json $comment history line). Three filter forms join the FILTERS list (source: src/render/template.ts FILTERS), all splitting on any single character of a delimiter set instead of one fixed delimiter:
| Filter | Result | GV.OC-01.01 with set .- |
|---|---|---|
part(delimiters,index) | the 0-based non-empty trimmed piece; an index past the last piece is empty with render note part-index-missing | index 2 → 01 |
part(delimiters) | the list of non-empty trimmed pieces, like split(delimiter) | → GV, OC, 01, 01 |
prefix(delimiters,index) | the original value truncated right after the indexed piece, original delimiters kept | index 2 → GV.OC-01 |
Why a set: mixed-delimiter identifiers (CSF 2.0 GV.OC-01.01, CRI Profile ids) carry a hierarchy in one column with more than one separator, and split(d,n) can only see one of them. prefix is what a cumulative folder path needs: each level’s name is the id truncated at that level, delimiters intact, so GV, GV.OC, GV.OC-01 nest without a join rule.
Where they are legal: anywhere a template scalar with split(X,0) is legal today, which in practice means the structural templates on folder and naming surfaces (layout folder/file/tag/heading/link templates and LevelRule-derived names). They are $defs/template filters, so they are not confined to one mechanism; they are simply not list-projection filters like join (source: src/render/template.ts FILTERS and src/import/mapping/serialize.ts buildName, which emits them for level names). The delimiter set itself comes from the mapping model’s LevelRule.delimiters (source: src/import/mapping/types.ts; the earlier note in this section cites it).
Lexing follows the SchemaVer 1.7.0 rules: the delimiter set is one filter argument, so a set containing , ) | or \ is written \, \) \| \\. The index is taken greedily off the decoded argument, so part(\,;,0) is a set of , and ; with index 0.
On the mapping side (source: src/import/mapping/types.ts LevelRule), a level carries delimiters (the set) beside the legacy single delimiter; a level with only delimiter still serializes to split(d,n) byte-for-byte, so existing recipe hashes do not move. naming: 'prefix' is now meaningful on fixed levels, not only the variadic tail.
Backward compatibility: purely additive (SchemaVer ADDITION). Recipes without these filters validate, render, and hash unchanged. Spec URI did NOT bump.
§4.13 target.crosswalks[]: a crosswalk column role (added 2026-09-21, SchemaVer 1.12.0; additive)
Section titled “§4.13 target.crosswalks[]: a crosswalk column role (added 2026-09-21, SchemaVer 1.12.0; additive)”Crosswalker turns more of what is latent in a source into vault structure, and the import wizard is where the user decides, column by column, what becomes real. A crosswalk column role (plain: a source column whose cells contain identifiers from another framework) reifies those references as first-class crosswalk-edge notes instead of leaving them as text or treating them as links inside the source ontology.
Each target.crosswalks[] entry has eight fields (source: spec/recipe.schema.json § $defs/crosswalk_entry):
| Field | Contract |
|---|---|
column | Required non-empty source-column name. |
to_ontology | Required non-empty CURIE prefix for the target ontology, for example nist-csf-2. |
predicate | One of five allowed values: is_equivalent_to, is_broader_than, is_narrower_than, is_approximate_to, or intersects_with; default is_approximate_to. See §6 for their meanings. |
split | Cell delimiters; default [",", "\n", ";"]. |
drop | Atoms treated as empty; default ["None", "N/A", "-"]. |
qualifier | Either keep-as-justification or strip; default keep-as-justification. |
mapping_set_id | Optional. Legacy alias of mapping_set.id, same meaning: the declared release id. Since SchemaVer 1.18.0 an undeclared id is assigned once at first import and kept, no longer built from the recipe id (see §4.18). |
mapping_set | Optional declared release metadata for the column’s mapping set; see §4.18. |
For each declared entry, the engine derives edge rows and writes Tier 1 crosswalk_edge_frontmatter notes under _crosswalker/mappings/<source>-to-<target> (source: src/generation/crosswalk-edge-pass.ts). The notes have their own newly minted import set under the sssom identity prefix. Nothing is written into the concept note. The edge pass uses the same SSSOM identity and generation path as a standalone mapping import, then invokes Tier 2 projection and closure precomputation when those handles are available.
Edge endpoint projection (SchemaVer 1.15.0 convention): The bundled edge recipe declares optional managed subject_note and object_note scalar wikilinks and an optional managed body line. The plugin’s crosswalk-column pass and SSSOM importer resolve each subject_id/object_id CURIE against _crosswalker-stamped concept identities at write time, then project folder-qualified note links without deriving a path from an identifier. The CURIE fields remain queryable. If either concept is missing, keep the edge and the resolved link, omit the missing link, and report its cause and an import or repair action; refresh the mapping import explicitly after importing the missing framework. SchemaVer 1.15.0 adds no grammar or required keys: Tier 1 crosswalk frontmatter already allows domain-specific properties (spec/tier1.schema.json $defs/crosswalk_edge_frontmatter.additionalProperties: true), and recipes without the managed keys validate and hash byte-identically. The bundled crosswalk-edge.json recipe’s target hash alone changes deliberately because its target shape changed.
This is separate from target.graph_edges because the two keys describe different output shapes. graph_edges encodes intra-ontology relations as frontmatter on the concept note; a crosswalk column emits separate mapping notes. The SchemaVer 1.12.0 amendment makes the distinction explicit in graph_edge.via: crosswalksTo (intra-vault convenience only; cross-ontology references belong in target.crosswalks) (source: spec/recipe.schema.json § $defs/graph_edge.properties.via).
A trailing parenthetical qualifier is kept verbatim as mapping_justification when qualifier is keep-as-justification, and is removed when the value is strip. Crosswalker does not parse the qualifier for meaning in this contract (source: deriveCrosswalkEdgeRows() in src/generation/crosswalk-edge-pass.ts).
CRI worked example. The NIST CSF v2 Mapping column contains CSF identifiers that are byte-identical to CRI’s own Profile identifiers. A wikilink would therefore resolve to a CRI note and manufacture a false intra-ontology relation. recipes/import/cri-profile-v2-2.json now declares the column as a crosswalk to nist-csf-2, uses is_approximate_to, and keeps qualifiers such as (CRI Modified) as justification. Its effective recipe hash moved because the recipe now produces additional edge notes; that output-changing declaration correctly participates in the hash.
The wizard offers this role through detection kind crosswalk-column in src/import/detection.ts, then shows a Crosswalks card where the user names the framework and chooses the predicate. The framework control lists registry ontologies plus Other, and the predicate control exposes the five schema values above (source: src/import/workbench.ts). An unnamed target blocks Generate with: Column "<column>" is marked as a crosswalk but no framework is named. Pick the framework on the Crosswalks card, or turn the card off. (source: src/import/recipe-document.ts).
target.crosswalks participates in computeRecipeHash() through recipeHashCanonicalInput() because it changes which notes exist. An absent key stays absent rather than becoming null, so every recipe without target.crosswalks keeps its pre-1.12.0 canonical input and hash byte for byte (source: src/generation/hash.ts).
Backward compatibility: purely additive (SchemaVer ADDITION). Recipes without target.crosswalks validate, generate, and hash unchanged. Spec URI did NOT bump.
§4.14 source.detect: offer-only source recognition hints (added 2026-09-21, SchemaVer 1.12.0; additive)
Section titled “§4.14 source.detect: offer-only source recognition hints (added 2026-09-21, SchemaVer 1.12.0; additive)”Crosswalker can carry hints about where a source’s table begins, while the import wizard remains the control surface where the user confirms the source binding and decides what becomes real. source.detect is offer-only: hints may rank or explain a suggestion, but they never select a source, worksheet, or header row on their own.
The block has five optional fields (source: spec/recipe.schema.json § source.properties.detect):
| Field | Contract |
|---|---|
sheet | Preferred worksheet name, matched exactly and case-sensitively after trimming. |
sheet_aliases | Unique alternative worksheet names, matched by the same rule as sheet. |
header_row | Zero-based header-row index on the preferred sheet. |
filename | Unique case-insensitive filename substrings that may raise the rank of a candidate already matched by columns, but never admit one by themselves. |
notes | Up to 512 characters of user-facing guidance about the source and download choice. |
Nothing reads source.detect yet. The wizard’s current sheet and header-row suggestion calls scoreFilePeeks() through src/import/workbook-suggestion.ts, so it scores observed columns, structural depth, signature length, and declaration order rather than consulting these hints. The hints are the next consumer boundary, not a claim of current behavior.
source.detect is excluded from computeRecipeHash() because it changes no generated output. It is included in computeRecipeDocumentDigest() with the rest of the normalized recipe document (source: the source.detect description in spec/recipe.schema.json and the whole-document digest contract in §3 above).
Backward compatibility: purely additive (SchemaVer ADDITION). Recipes without source.detect validate and identify exactly as before. Spec URI did NOT bump.
§4.15 source.nest[]: nested record levels (added 2026-09-21, SchemaVer 1.13.0; additive)
Section titled “§4.15 source.nest[]: nested record levels (added 2026-09-21, SchemaVer 1.13.0; additive)”Deeper reification (plain: making implied source structure into first-class vault objects), and a good user experience for controlling that reification during import, now meet at source depth: a source whose records own lists of sub-records now imports as nested notes, and the wizard is where you decide which levels become notes and how each is named. The gap was a record that owned lists of sub-records but could previously become only one flat collection of rows.
Each source.nest[] entry has seven fields (source: spec/recipe.schema.json § $defs/nest_level):
| Field | Contract |
|---|---|
level | Required non-empty level name. It must appear in source.levels and be used by target.layout. |
id | Required template for this level’s stable identifier, evaluated against the record. |
children | On a JSON-backed level, the field name containing the next level’s array; on a workbook or sibling-array level, a $defs/source_join_from locator for the secondary collection. The last level has no children. |
parent_key | For join-sourced children, the child field that names its parent’s id. It is required on the child level for that shape and ignored for JSON-field children. |
carry | Optional unique field names copied into child lineage at _cw.ancestors.<level>.<field>. The id field is carried by default. |
leaf | On a non-leaf level, folder-note writes that level’s row at <folder>/<folder>.md, while none produces folders only and no note for that row. Every non-leaf level needs one of these values unless target.layout has a file entry for that level. The last level may declare none (its rows are not emitted, which is how the wizard’s Depth select leaves deeper levels out of an import); folder-note on the last level is refused with The last nest level "<level>" is the note itself; folder-note applies only to levels that have children. (source: src/import/recipe-document.ts). |
identity | global uses this level’s id as the CURIE local part; path uses the escaped lineage path when ids repeat under different parents. The default is global. |
The schema states the producer contract verbatim (source: spec/recipe.schema.json § source.properties.nest.description):
Under the schema-as-primitive commitment, an external Tier 1 producer must reproduce those five rules to emit the same nested notes and hash-match the plugin. A producer that chooses different rules is expressing a different recipe, not an invalid producer.
Placement. render() applies only layout entries at or above the emitted row’s declared level. A non-leaf row without its own file entry uses its declared leaf: folder-note derives a file from the final folder segment, and none emits no note for that row (source: src/render/index.ts). Rows without _cw still apply the full layout exactly as before.
Identity. Identity is declared per level, pinned when the import set is minted, and never inferred from the current run. A plain / is not a safe boundary because / is already legal inside the CURIE local-part charset, so ['a/b', 'c'] and ['a', 'b/c'] would otherwise collide. pathIdentityLocalPart() first escapes every declared id and the reserved token marker, replaces a slash inside a piece with --2f--, then joins the escaped pieces with / (source: src/generation/curie.ts § pathIdentityLocalPart). Before the first folder is created, the engine computes every emitted CURIE and refuses the first duplicate with the exact message:
Source: src/generation/generation-engine.ts § nestedIdentityCollision.
Join-sourced children. A workbook sheet or sibling JSON array uses the same source_join_from locator and resolveSecondaryRows() path as source.joins; the secondary collection is read once per level and grouped by the child level’s parent_key through the same normalized key rule (source: src/source/nest.ts, src/source/joins.ts). Rows naming no present parent are counted and warned once as <n> <level> records name a parent that is not in the source and were not imported. A CSV is one collection, so join-sourced nesting is refused rather than pretending a second collection exists.
OSCAL-mini worked example. This three-level shape makes the required non-leaf leaves explicit while keeping the §3.2 source and layout pattern:
For group ac, control ac-1, and part ac-1_smt, the three row addresses are ac/ac.md, ac/ac-1/ac-1.md, and ac/ac-1/ac-1_smt.md (source: src/render/index.ts, tests/render-nest.test.ts).
Wizard control surface. Detection kind nested-records recognizes eager JSON record arrays recursively to four levels when at least 80% of sampled parent rows share an object-array field (source: src/import/detection.ts). The JSON picker reports the observed chain once, on the top-level candidate, then offers Nested: <levels> (<n> notes) and Only <list> (<n> notes); the flat choice remains selected until the user chooses nesting (source: src/import/import-wizard.ts). The mapping card leads with a Depth select that sets leaf: folder-note on every level above the note and leaf: none on every level below it (source: setFolderDepth in src/import/mapping/view-model.ts). Nested workbench rows expose Own note with Yes, as a folder note or No, folder only, and Identified by with Its own identifier or Its path from the top level. Merge and Split are not offered on nested rows because the source already declares the record-type boundaries, and merging two record types into one name would create a collision rather than project children into a parent’s body (source: src/import/workbench.ts).
Hash and source namespace. source.nest participates in computeRecipeHash() because it changes which notes exist. Absence stays absent instead of becoming null, so every recipe without nest keeps its previous canonical input and hash byte for byte (source: src/generation/hash.ts § recipeHashCanonicalInput). A source column literally named _cw is refused with Column "_cw" is reserved for nested-record lineage. Rename it in the source and import again. because _cw belongs to the producer contract (source: src/source/joins.ts § assertNoReservedSourceColumn).
Limits. JSON-field nesting requires an eager source; a streamed source with nest is refused with an action to import without streaming (source: src/source/index.ts). A join-backed nested import previews the primary row count until generation reads the secondary collection and computes the exact emitted count. Wizard detection stops at four levels, so deeper nesting is outside this build’s stated scope. Recursive shapes are walked only to the depth explicitly declared in source.nest (source: src/import/detection.ts, src/source/nest.ts). Folding a nested level into its parent note as sections was deferred here and is now specified in §4.16.
Backward compatibility: purely additive (SchemaVer ADDITION). Recipes without source.nest validate, generate, and hash unchanged. Spec URI did NOT bump.
§4.16 nest_level.leaf: section and level-scoped body projections (added 2026-09-22, SchemaVer 1.14.0; additive)
Section titled “§4.16 nest_level.leaf: section and level-scoped body projections (added 2026-09-22, SchemaVer 1.14.0; additive)”Deeper reification, and user control over it during import, now reach inside the note: a nested level’s records can become sections (plain: headings with text inside their parent’s note) instead of notes of their own or nothing at all. This is body projection of a nested level. It is the fourth answer to “where does this level go” (folder, note, left out, section), and it uses the existing heading layout mechanism: the closed grammar of five mechanisms is unchanged (source: spec/recipe.schema.json § $defs/mechanism).
Fields. Two optional additions, nothing else (source: spec/recipe.schema.json):
| Path | Contract |
|---|---|
$defs/nest_level.leaf | Enum gains section, so it reads folder-note, none, section. A section level’s records are not emitted as rows; each is attached to its nearest emitted ancestor’s row under _cw.sections in document order and rendered as a heading region inside that ancestor’s managed body. Heading text and depth come from that level’s layout heading entry (template, level_depth); content comes from body projections declared with level: <this level>. Legal on any level whose parent is a note or a section level; never on level 0. |
$defs/body_entry (append variant) | New optional level (non-empty string). Restricts the projection to the records of one nest level that declares leaf: section; it renders once per attached record, against that record’s scope, under its heading. Absent: the projection applies to every emitted row as before. The section variant of body_entry does not take level. |
The producer contract in source.properties.nest.description gains rule (6), stated in the schema as:
The schema’s contract text also carries rule (5b), on blanking carried fields of an ancestor that source.where excluded, which the five-rule quote in §4.15 predates.
Identity: a section is content, not a concept. A section record gets no note, no CURIE, no _crosswalker provenance, no Tier 2 row, and no orphan tracking (source: src/generation/generation-engine.ts). Crosswalker never treats [[note#Heading]] as an identity, because that is a path; Obsidian users may still link to the heading. A section never claims a CURIE, so adding or removing one changes only its host’s body and cannot create an orphan. Its fields do count toward the host’s wording review group, so a changed part marks its control note for wording review (source: src/generation/hash.ts). Section levels are pinned at import-set mint like every nest level (source: tests/import-set.test.ts).
Body layout. Everything lands inside the one managed body region; there is no second region and no nesting of regions (source: src/generation/managed-body.ts). Order inside the region: the host’s H1, the host’s own unscoped body projections in declared order, then sections depth-first in source order. There is no sorting and no interleaving. Every attached record gets a heading even when all of its content is empty, so the note is honest about how many records exist; omit_if_empty governs content only. Heading depth is the heading entry’s level_depth; the workbench assigns 2 to the first section level below the note and one deeper per nested section level, which reads ##, ### under the automatic H1 (source: src/import/mapping/view-model.ts, src/render/index.ts § renderAttachedSections). Unscoped body projections never apply to section records, and level-scoped projections never apply to emitted rows.
Worked example (synthetic). Group g1, control c1 with parts p1 (name: statement) and p2 (name: guidance):
g1/c1.md then carries this body region, and no part note is written (source: tests/nest-generation.test.ts):
Refresh. On replace, only the managed region is rebuilt; prose outside it survives byte for byte, and prose typed inside it (under a section heading) is overwritten, as the region contract always said (source: src/generation/existing-note.ts). A refresh that changes nothing but produced_at leaves the note untouched. leaf is a shape choice, not an identity rule, so a refresh may flip it:
| Flip | Result |
|---|---|
| section to note | That level’s rows are emitted and mint CURIEs under the identity scheme pinned at mint; the host region is rebuilt without the sections; zero orphans. |
| note to section | The run claims no CURIE at that level, so the notes written earlier are reported by the existing orphan pass, kept on disk, and never deleted. The wizard adds: Some may be <levels> notes that this recipe now folds into their parent notes as sections. with every section level named in nest order (source: src/import/import-wizard.ts). |
none to section, or back | Host body rebuilt; no identity effect. |
Diagnostics. Nine codes: eight blocking and one warning (source: src/import/recipe-document.ts):
| Code | Severity | Condition |
|---|---|---|
nest-section-on-root | blocking | leaf: section on the first nest level |
nest-section-parent-not-note | blocking | the level above has leaf: none |
nest-section-no-heading-entry | blocking | target.layout has no heading entry for the level |
nest-section-has-file-entry | blocking | the level also has a file or folder layout entry |
nest-section-heading-kind | blocking | the level’s heading entry declares a kind other than concept |
nest-section-depth-order | blocking | a section level under a section level does not have a strictly deeper level_depth |
nest-section-child-is-note | blocking | a level below a section level has leaf: folder-note or a file entry |
body-level-not-section | blocking | a body entry’s level names a level that does not declare leaf: section |
nest-section-no-body | warning | a section level has no body projection with its level; headings render with no content |
level_depth above 6 is refused by the schema itself (source: spec/recipe.schema.json § $defs/level_depth).
Wizard control surface. The Depth row gains Below the note, shown only on nested sources with a level below the note, with Left out or Sections inside each <note level> note. With sections chosen, the hint names each placement (for example each part becomes a section inside its control note) and adds Sections are rebuilt on every refresh. Write your own notes below the managed region. Each nested row below the note has a Placement select (Left out, or Sections inside each <parent> note) and, when set to sections, a Section text select choosing the field projected under each heading. The counts read <n> notes, <m> sections, and the sample preview opens on the first note-level row so the section headings are visible (source: src/import/workbench.ts).
Hash. No hash code changed: leaf sits inside source.nest, the heading entry inside target.layout, and level inside also_emit.body, all already in recipeHashCanonicalInput() (source: src/generation/hash.ts).
Out of scope for this version. source.where over section records, sub-headings inside a projected section, user-chosen section order, folding levels of a non-nested (level-column) source, and section-level frontmatter, tags, links, or crosswalks. See the body projection decision log.
Backward compatibility: purely additive (SchemaVer ADDITION). Recipes without leaf: section or a body level validate, generate, and hash unchanged. Spec URI did NOT bump.
§4.17 Implied concept levels (added 2026-09-23, SchemaVer 1.17.0; additive)
Section titled “§4.17 Implied concept levels (added 2026-09-23, SchemaVer 1.17.0; additive)”An implied concept is a first-class concept whose identity appears in another row’s hierarchy rather than in a row of its own. A folder-level implied_concept: true creates one concept note per distinct rendered folder value; { identity: "{id}" } instead derives the local identity from that template. This is reification of a level, not a replacement for existing row notes. Source: spec/recipe.schema.json $defs/layout_entry.implied_concept, src/render/index.ts, src/generation/enrich.ts.
Only a non-leaf, non-variadic folder entry accepts the key. The concept CURIE is the import set’s pinned ontology prefix plus the injectively escaped rendered identity, never a directory name read back from the vault. A row anywhere in the import with the same CURIE wins. Rows describing one folder must agree on its identity; otherwise the folder note is refused by name and row notes still import. Existing recipes lacking the key keep their canonical hash and output. The flagged layout participates in the recipe hash. The bundled nested NIST 800-53 recipe opts its family folder in; ordinary flat recipes do not (source: recipes/import/nist-800-53-nested.json).
At <folder>/<label>.md, an implied note records curie, title, implied_level, the implied_levels / implied_values rendered chain, sorted managed children, a managed Contents region, and _crosswalker import-set provenance. It has no kind: hub. With no target.enrichment block it still runs enrichment, but does not create ordinary hubs, a root home note, or hosted Contents on row notes. parent / parent_curie on nested notes use the parent’s resolved concept identity (never a path). For an implied-level import, target.enrichment.parent_links: false disables those row links; children_lists: false disables the generated children list. Both are otherwise on for an implied level. Engine-derived parent fields carry a top-level _crosswalker_managed_keys ownership list (only the keys actually patched, outside the closed _crosswalker provenance block). Replace removes previously engine-owned parent fields if their relation is no longer generated; recipe-declared parent fields and unstamped user fields remain untouched. Unpatched output has no ownership list (source: src/generation/generation-engine.ts recordedEngineParentKeys). The wizard exposes a checkbox for each folder level under Arrange levels; the recipe stores that exact choice.
Refresh finds an owned implied note by CURIE, then a pre-feature hub by its recorded value chain or alias. Replace adopts a hub while preserving user prose and clearing kind and hub_* keys; Skip leaves a legacy hub byte-identical and explains that Replace is needed. A held-only implied note keeps its recorded provenance and is not rewritten when unchanged. An absent held-only note is not recreated under Skip. An unused note with a complete recorded chain is an orphan by CURIE; a half-recorded chain is named but not judged. Sources: src/generation/generation-engine.ts identity resolution, tests/implied-concept-engine.test.ts synthetic acceptance cases.
§4.18 Crosswalk column mapping_set block (added 2026-10-03, SchemaVer 1.18.0; additive)
Section titled “§4.18 Crosswalk column mapping_set block (added 2026-10-03, SchemaVer 1.18.0; additive)”A crosswalk column (see §4.13) produces one link set, which is one MappingSet (one crosswalk release). Since SchemaVer 1.18.0 that link set gets the same release record a mapping file import gets, and the recipe may declare the release’s facts in an optional mapping_set block on the column. The block has 12 optional keys and no others (source: spec/recipe.schema.json § $defs/crosswalk_mapping_set, additionalProperties: false):
| Key | Record field it fills | Contract |
|---|---|---|
id | mapping_set_id | Non-empty string. The declared release id. mapping_set_id on the column is its legacy alias. |
version | mapping_set_version | Non-empty string. |
title | mapping_set_title | Non-empty string. |
description | mapping_set_description | Non-empty string. |
license | license | Non-empty string. |
provider | mapping_provider | Non-empty string. |
date | mapping_date | Non-empty string. |
creator_id | creator_id | Non-empty list of non-empty strings. |
subject_source | subject_source | Non-empty string: the ontology the column’s rows come from, as declared. |
subject_source_version | subject_source_version | Non-empty string. |
object_source | object_source | Non-empty string: the ontology the column’s ids point to, as declared. |
object_source_version | object_source_version | Non-empty string. |
Identity. The release id of a column’s link set is, in order: the declared mapping_set.id; else the legacy mapping_set_id; else the id the link set already holds (its record, or, for a link set written before records existed, the id every one of its links carries); else an id minted at the link set’s first run (urn:crosswalker:mapping-set: plus 10 base32 characters) and kept by every later refresh. It is never built from the recipe id, the ontology pair, the file or the rows (source: crosswalkReleaseDeclaration() and the id resolution in src/generation/crosswalk-edge-pass.ts). When the column declares no id, the record’s id_origin is minted, including for an older link set that keeps its old <source>-to-<target>-<recipe id> id: the recipe declared nothing, so Crosswalker assigned it. Those older link sets keep their id on refresh, gain a record, and their links’ managed content does not change.
Refusals. A column whose mapping_set.id and mapping_set_id differ is refused. A refresh whose declared id, or declared version, differs from the release the link set holds is refused before anything is written; a different release belongs in a new link set. Both messages name the action.
Record. After the column’s links are written, the engine builds the record from the declared keys plus the mapping count and the membership and content digests recomputed over the links it derived, and writes it as one kind: mapping-set note in the link set’s folder, carrying the link set’s own _crosswalker block. Crosswalk columns always write one note per link, so the record is always a note, never a table header. A refresh that changes nothing leaves the record note untouched (source: writeReleaseRecordForNoteSet() in src/mappings/mapping-set.ts).
Workbench. The workbench has no control for the block. It carries the block (and the column’s drop list) through every edit unchanged, so an untouched round trip returns the recipe byte for byte (source: src/import/mapping/serialize.ts; portable recipe fidelity).
Known gap. A column’s link set is found by a recipe id that includes the parent recipe’s id, so renaming a recipe still starts a new link set even though the release id no longer depends on the recipe.
Backward compatibility: purely additive (SchemaVer ADDITION). Recipes without the block validate unchanged and hash byte-identically; the block participates in the recipe hash through target.crosswalks when present. Spec URI did NOT bump.
§4.19 Composable filename template filters (added 2026-10-31, SchemaVer 1.20.0; additive)
Section titled “§4.19 Composable filename template filters (added 2026-10-31, SchemaVer 1.20.0; additive)”Naming remains part of the recipe, not a mapping-specific function. These general filters work on any available scope field and map over list items through the existing filter engine (source: spec/recipe.schema.json $defs/template; src/render/template.ts FILTERS).
| Filter | Exact behavior |
|---|---|
hash(N) | First N lowercase hex characters of SHA-256 of the exact UTF-8 value. N is an integer from 4 to 64; invalid arguments fail rendering. The suffix is always present, so another row’s arrival cannot rename this row |
default(text) | Replace an empty or whitespace-only value with the argument. Preserve a nonempty value exactly. Missing variables still fail unless optional is first |
name-safe | Replace each run of whitespace, control characters, filesystem-reserved characters and Obsidian link-breaking characters with one space; remove leading/trailing dots and whitespace. Keep readable punctuation such as parentheses and internal dots; percent and ampersand remain intact |
name-safe handles backslash, slash, colon, asterisk, question mark, double quote, angle brackets, pipe, #, ^, [ and ] (source: src/render/template.ts FILTERS['name-safe']). It also removes invisible formatting characters (zero-width characters, bidirectional controls, byte-order mark) and lone surrogate halves, so a name cannot display differently from its text. It is separate from fs-safe, whose historical underscore replacement stays unchanged. Existing truncate(N) keeps its UTF-16 slicing, including its existing surrogate-pair boundary behavior; placing name-safe after truncate(N), as the default template does, removes any half character a cut leaves behind. default(text) maps over list items like the other filters, so an empty list stays an empty list. Following it with trim(.) removes newly exposed outer dots and whitespace, including a suffix such as . ..
The new standalone mapping default is an ordinary file template (source: src/import/sssom-importer.ts READABLE_SSSOM_FILE_TEMPLATE):
The regex’s first capture preserves everything after the first colon, or the whole value if no colon exists. Endpoint stems are capped at 60 UTF-16 code units each; the hash covers the existing CURIE local part, not either display stem. The filename is not the assertion’s identity and does not encode predicate or direction. See readable mapping names for retained-recipe behavior and Windows path-length limits.
Backward compatibility: additive filters and documentation only at the schema boundary; the stable schema URI is unchanged. Existing recipes validate, render and hash identically unless their author explicitly changes their template. The readable default applies only to new standalone sets, never by replacing an existing set’s retained original.
§4.20 Tag template list fan-out and facet membership source (added 2026-11-02; list fan-out SchemaVer 1.21.0, facet_source SchemaVer 1.22.0; additive)
Section titled “§4.20 Tag template list fan-out and facet membership source (added 2026-11-02; list fan-out SchemaVer 1.21.0, facet_source SchemaVer 1.22.0; additive)”A tag template may now hold one list-valued field and emits one tag per value (source: spec/recipe.schema.json target.also_emit.tags description; src/render/template.ts renderTagTemplate). This is how a technique with several tactics gets one tag per tactic.
| Rule | Exact behavior |
|---|---|
| One list per template | One list-valued interpolation, for example mitre-attack/tactic/{tactics|optional|split(,)|tagsafe}, emits one tag per non-empty value. Filters after the list-producing filter run per value |
| Two lists | A recipe validation error: “A tag template may hold one list-valued field. Split it into two tags.” |
| Order and duplicates | Tags keep source order, then template order. A tag already emitted by an earlier template or value is not emitted again |
| Empty values | An empty value or an optional-absent field emits no tag. A tag whose final path segment renders empty is dropped. An empty middle segment is left as written |
| Other sinks | Aliases, file names and other text templates still refuse a list unless it is joined |
target.enrichment.facet_source (optional; source: spec/recipe.schema.json $defs/enrichment; src/generation/generation-engine.ts facetMembershipsForRecipe) says where facet hub membership comes from:
| Value | Membership |
|---|---|
(omitted) or caller | Unchanged default. Memberships supplied by the caller (the wizard or a preset) win; without them, declared facet notes derive from the tag templates, otherwise from rendered tags |
tag_template | Always derive raw display values from also_emit.tags, including list splitting and filters, so the wizard, a stack and cw produce identical facet hubs |
The linked recommended defaults declare facet_source: tag_template (source: recipes/import/nist-csf-2-linked.json, nist-800-53-linked.json, mitre-attack-technique-linked.json, and mitre-d3fend-linked.json).
Backward compatibility: additive at the schema boundary; the stable schema URI is unchanged. Existing recipes validate unchanged and keep their hash, because neither the field nor a list-valued tag existed before. Two output changes reach existing recipes: a tag repeated across templates is now emitted once, and a tag whose final segment renders empty is no longer emitted. Of the shipped recipes, only mitre-d3fend and mitre-attack-mitigation produce different tags on re-import.
§4.21 source.outline: row-ordered sparse-depth hierarchy (added 2026-11-02; SchemaVer 1.23.0; additive)
Section titled “§4.21 source.outline: row-ordered sparse-depth hierarchy (added 2026-11-02; SchemaVer 1.23.0; additive)”An outline (a table whose filled column says the row’s depth) becomes explicit parent and child links without duplicating its hierarchy as folders. The source’s order supplies outline containment, not an OWL subclass assertion. This is one of the ways sources encode hierarchy.
The declaration has two required fields (source: spec/recipe.schema.json source.properties.outline):
| Field | Contract |
|---|---|
levels | Non-empty, unique column names ordered from shallowest to deepest. These are depth columns, independent of source.levels and the target layout |
id | A template evaluated on the original row. It explicitly declares the row’s identity, including a domain-specific or case-sensitive column such as ID |
Stage order and streaming. Stage [0] chooses either source.nest or source.outline; the schema refuses both together. Outline walks all parsed rows before the source.where row predicate, then joins. The source stage retains only the active ancestor stack, bounded by declared depth; a streamed parser remains streamed. Like nested imports, the generation routes currently collect admitted shaped rows for whole-source, zero-write validation before generation. _cw enters the key universe before predicate preflight (source: src/source/index.ts, src/source/outline.ts).
Lineage. Each row receives the nested-record lineage shape: _cw.level is the filled column name, _cw.path contains ancestor ids followed by self, _cw.parent is the immediate ancestor id or empty at the root, and _cw.ancestors carries rendered ids by level. As with nest, an unfiltered single-field id template carries that field name; a filtered/composite template carries id. Both wizard and cw generation use the outline’s declared id, never a filename guess. Ownership planning first shapes candidate rows with candidateSourceRows, then candidateRunIdentities uses that same shared resolver. Before creating folders or notes, both write routes finish the source stage and check all admitted identities for collisions; a bad row anywhere refuses the run with no writes (source: src/generation/generation-engine.ts deriveRowCurieRaw, candidateSourceRows, and candidateRunIdentities).
Filtering and errors. If where excludes a parent, admitted descendants remain: excluded ancestor fields are blanked and _cw.parent becomes the nearest admitted ancestor, or empty if none remains. Zero filled depth columns, multiple filled columns, a jump without a preceding parent depth, an empty rendered identity, or a reserved _cw source column produce a row-numbered cause and corrective action. Whitespace-only cells are empty. The workbench names data rows from 1 after the header when reporting a malformed row (source: src/source/outline.ts, src/import/workbench.ts, tests/source-outline.test.ts).
Set identity pin. Tier 1 SchemaVer 1.9.0 adds optional _crosswalker.import_set.outline_identity: { id, levels }. Both values are fixed at mint and carried through provenance discovery and refresh. Changing or removing either declaration refuses the refresh; restore the original choices or create a new set. A set with no recorded outline pin cannot silently acquire one. An older identity derivation also requires a new set (source: spec/tier1.schema.json, src/generation/import-set-block.ts, src/generation/import-set.ts, src/generation/provenance.ts).
Navigation recipe. recipes/import/mitre-d3fend-linked.json keeps flat files, templates parent and parent_curie from _cw.parent, adds sorted children lists, and groups tactic hubs from tag templates. Roots omit both parent properties. mitre-d3fend-flat remains available as the previous default, with unchanged bytes and hash. Tier 2 consumers must not interpret this outline parentage as rdfs:subClassOf. The recipe’s hardcoded parent CURIE prefix does not support a second, set-qualified D3FEND set; this known limit is recorded in its $comment.
Hash and editing. source_outline participates in recipeHashCanonicalInput(); absent stays absent, never null, preserving old recipe pins. A workbench edit patches the canonical RecipeDocument and preserves source.outline; its eager preview runs the same source stage before counting parent links. Expanded rows are cached by parsed-data identity and serialized source, and remain visible while replacement rows are prepared. A streamed source shows an actionable preview limitation instead of silently rendering blank (source: src/generation/hash.ts, src/import/workbench.ts, src/import/recipe-document.ts, tests/d3fend-outline-linked.test.ts). The stable schema URI does not change.
§5 Historical junction-note design sketch
Section titled “§5 Historical junction-note design sketch”Per Ch 07 resolution: evidence links are edge-as-note reified — one markdown file per evidence→control relationship.
TypeScript shape
Section titled “TypeScript shape”YAML on-disk shape
Section titled “YAML on-disk shape”Filename convention
Section titled “Filename convention”Junctions/{ontology}/{control_id}--{evidence_slug}.md — e.g., Junctions/nist-800-53-r5/AC-2--MFA-Policy.md.
The composite filename (control + evidence) makes diff-on-rename easy and prevents collision when one piece of evidence implements multiple controls.
Mandatory vs optional fields (rule)
Section titled “Mandatory vs optional fields (rule)”- Mandatory (5):
link_type,evidence,control,ontology,status. Plugin generation refuses to create a junction note missing any of these. Bases queries assume their presence. - Optional (8):
confidence,evidence_type,method,reviewer,review_date,responsible,collected,expires. Pluginrenders missing asnull/empty. - Computed:
freshnessderived fromreview_date+expiresat query time (Tier 1: in-memory; Tier 2: SQL view).
OSCAL by-component mapping
Section titled “OSCAL by-component mapping”The 13-field schema is structurally isomorphic to OSCAL’s by-component assembly per Ch 07 deliverable. The OSCAL export pipeline is straightforward field-rename. See reference/registry/oscal for the mapping table once the OSCAL mental-model doc lands.
§6 Crosswalk assertion vocabulary and SSSOM provenance
Section titled “§6 Crosswalk assertion vocabulary and SSSOM provenance”A crosswalk assertion (one stated relationship between two concepts) stores its relationship in predicate_id and any explicit negation separately in predicate_modifier. The 8 allowed predicates are sourced from spec/tier1.schema.json § $defs.crosswalk_edge_frontmatter.properties.predicate_id:
predicate_id | Plain meaning | Group |
|---|---|---|
is_equivalent_to | The concepts have the same meaning for the mapping purpose | Set-theoretic |
is_broader_than | The subject covers more than the object | Set-theoretic |
is_narrower_than | The subject covers less than the object | Set-theoretic |
is_approximate_to | The concepts are similar but not equivalent | Set-theoretic (Crosswalker extension) |
intersects_with | The concepts overlap | Set-theoretic |
no_relationship | The source explicitly asserts no relationship | Set-theoretic |
superseded_by | The subject was replaced by the object in a later release | Release lineage |
supersedes | The subject replaces the object, which is from an earlier release | Release lineage |
predicate_modifier is absent for a positive assertion or exactly NOT for explicit negation. Negation is not encoded by inventing an _NOT predicate name. A negative equivalence therefore remains predicate_id: is_equivalent_to plus predicate_modifier: NOT.
Release lineage is an ordinary crosswalk, not a subsystem (added 2026-08-28, Ch 43). A version transition is a mapping between two release ontologies: a rename is one edge, a split is N edges sharing subject_id, a merge is N edges sharing object_id. Both directions are named because the effective-edge traversal in src/tier2/queries.ts materialises each stored assertion’s inverse into query rows, so the inverse spelling has to be a value that exists. Both are transitive: a successor walk has to cross several releases (source: src/tier2/predicate-characteristics.ts).
The lineage pair is deliberately not set-theoretic, and the two set-theoretic exporters refuse rather than translate it. superseded_by has no OLIR relationship type and no SKOS mapping property, so a lineage edge is excluded from both the STRM/OLIR TSV and the SSSOM TSV export and reported as lineage-predicate-not-representable-in-strm (source: src/export/strm-tsv-exporter.ts, src/export/sssom-exporter.ts). Coercing it would have published a withdrawal record as the claim that two controls intersect.
SSSOM (the Simple Standard for Sharing Ontological Mappings) provenance is carried by mapping_set_id, match_type, match_confidence, mapping_justification, mapping_provider, mapping_date, creator_id, and review_status. Direct SSSOM-style queries and export preserve mapping-set provenance and negation. STRM export omits negated assertions because STRM cannot represent that modifier without changing meaning.
§7 Tier 2 sidecar SQL schema (sqlite-wasm projection)
Section titled “§7 Tier 2 sidecar SQL schema (sqlite-wasm projection)”The Tier 2 sidecar is a deletable, recoverable projection of Tier 1 frontmatter into SQLite tables. It exists to enable performant queries (transitive closure, multi-ontology joins, coverage matrices, perspective views) that are awkward over markdown frontmatter.
Recovery property (load-bearing): if .crosswalker.sqlite is missing, corrupted, or stale, the projector rebuilds it from canonical Tier 1 on next vault load. This is what makes Tier 2 risk-free to bundle in v0.1.
Current schema summary
Section titled “Current schema summary”src/tier2/schema.sql is the canonical DDL. Tier 2 currently ships as tier2-sqlite-v6. The table below cites every stored column by table so additions are reviewable without copying a second SQL source that can drift.
| Table/view | Current columns | Purpose |
|---|---|---|
schema_meta | key, value | Schema and projection metadata |
ontologies | id, name, version, base_path, upstream_url, recipe_id, imported_at, control_count | One represented ontology summary |
concepts | ontology_id, curie, vault_path, source_hash, import_set_id, title, review_cid, review_wording_cid, review_scope_cid, review_housekeeping_cid, parent_curie, status, imported_at, modified_at | Canonical concept projection with explicit parent identity and current review fingerprints |
mappings | import_set_id, mapping_set_id, subject_id, predicate_id, predicate_modifier, object_id, match_type, match_confidence, mapping_justification, mapping_provider, mapping_date, creator_id, review_status, source_path, source_hash | One mapping occurrence with release provenance and explicit negation |
junction_notes | vault_path, curie, subject, subject_curie, predicate, object, object_curie, coverage, reviewer, review_date, status, confidence, scope, expires_at, notes, reviewed_against_curie, reviewed_against_cid, reviewed_wording_cid, reviewed_scope_cid, reviewed_housekeeping_cid, import_set_id, source_hash, modified_at | Evidence relationship with separate addresses and identities. The subject/whole-row pair projects together; group hashes project only as one complete three-value block (source: src/tier2/projector.ts § upsertJunctionNote) |
junction_notes_with_freshness | All junction columns plus computed freshness, subject_baseline, and change_kind | Review-age and content-drift view. review_cid alone gates subject_baseline = changed; only then does change_kind compare the three group hashes in priority order wording > scope > housekeeping. A changed legacy baseline without complete group hashes defaults conservatively to wording, so it cannot enter the housekeeping dismissal path (source: src/tier2/schema.sql § junction_notes_with_freshness) |
closure_cache | subject_id, predicate_id, object_id, shortest_depth, computed_at | Cached transitive answers |
closure_cache_state | subject_id, predicate_id, computed_max_depth, computed_at | Proof that a closure partition was computed through a depth, including empty results |
concept_embeddings | Reserved virtual-table shape | Optional future vector projection; not authoritative |
The mapping primary key includes mapping_set_id, the subject/predicate/modifier/object assertion, and source_path. Separate releases and duplicate source occurrences therefore do not overwrite one another. Endpoint indexes cover mapping subjects/objects and junction subject_curie/object_curie.
Projection rules (Tier 1 → Tier 2)
Section titled “Projection rules (Tier 1 → Tier 2)”- Read only notes with valid
_crosswalkerprovenance; canonical Markdown remains the authority. - Project
_crosswalker.import_set.idto nullableimport_set_idon concepts, mappings, and junction notes. Legacy unstamped records projectNULL. - Project concept
parent_curiedirectly. Never derive it from theparentwikilink. - Project junction
subject_curieandobject_curiedirectly. Never derive them from wikilink text. - Normalize mapping-set identifiers and predicate modifiers through the shared provenance helpers, then store each mapping occurrence with its source path.
- Reconcile an ontology’s displayed version deterministically across projected concepts using binary lexical order, so file traversal order cannot change the result.
- Exclude
predicate_modifier = 'NOT'from effective graph edges, including symmetric and inverse expansion. DirectcrosswalkBetween()queries retain negated rows and their provenance. - Invalidate mapping-dependent closure rows and depth-coverage state together whenever mappings change.
- Prune unseen rows only during a complete
fullprojection. Filters, projection errors, or unreadable-but-present frontmatter suppress pruning. - Project
review_groupsonly when all three schema-defined hashes are present. Compare them only after the whole-rowreview_ciddiffers; applywording>scope>housekeepingso one attestation receives exactly one change kind.
Schema-version migration
Section titled “Schema-version migration”Tier 2 now ships tier2-sqlite-v6. Because Tier 2 is derived, opening any other schema version drops and recreates the sidecar, then reprojects canonical Tier 1 Markdown. v6 adds current and recorded wording/scope/housekeeping hashes and the computed change_kind; v5 introduced the whole-row review_cid gate and baseline pair. It preserves the v4 ownership projection and the v3 mapping-occurrence, negation, endpoint-identity, and depth-safe closure-cache model.
§8 Cross-schema invariants & constraints
Section titled “§8 Cross-schema invariants & constraints”These are the rules the projector and import paths both respect:
- Contracts stay distinct: the Recipe Schema Contract governs ImportRecipe intent; the Tier 1 Schema Contract governs canonical member records.
- Tier 1 is canonical local managed authority: every Tier 2 column has a Tier 1 source. No authoritative managed knowledge lives only in Tier 2.
- Knowledge Set is conceptual, not a universal envelope: ontology, mapping-set, and evidence-set aggregates may be represented through Tier 1 members without one generalized aggregate schema.
- Managed ownership is declared: a producer may update or clear only recipe-declared managed fields and
_crosswalkerprovenance.user_preservefields remain untouched. - Identity is not address:
parent_curie,subject_curie, andobject_curieare explicit stable identities; wikilinks remain clickable addresses and are never reverse-parsed into identity. - Mandatory junction fields:
curie,subject,predicate,object, and_crosswalker(source:spec/tier1.schema.json § $defs.junction_note_frontmatter.required). - Crosswalk vocabulary and negation are separate:
predicate_idhas 8 allowed relationship values, whilepredicate_modifieris absent orNOT(source:spec/tier1.schema.json § $defs.crosswalk_edge_frontmatter.properties). - Mapping occurrences preserve release provenance:
mapping_set_iddistinguishes published mapping collections; deterministic assertion CURIEs distinguish repeated occurrences without changing concept identity. - Graph traversal uses only positive assertions: direct queries may return negated assertions, but closure, symmetric expansion, and inverse expansion exclude
NOT. - Version-governed representations rebuild safely: ImportRecipe, Tier 1 records, and Tier 2 SQL carry independent versions appropriate to their roles. Tier 2 can be discarded and regenerated.
§9 Build sequence
Section titled “§9 Build sequence”The original v0.1 implementation plan decomposed the governing contracts and representations in this order:
_crosswalkermetadata v2 — extend existingsrc/generation/generation-engine.tsto write the v2 fields. Lowest-risk: this is just a frontmatter shape change.- Junction note 13-field schema — implement junction-note generation as a new code path in the generation engine. Triggered by ImportRecipe with crosswalks that target evidence documents.
- ImportRecipe v2 — refactor
src/types/config.tsto the v2 shape. Migration script for existing v1 saved configs. - Tier 2 sidecar SQL projector — new module in
packages/core/. Auto-runs on vault load. Lazy closure cache + sqlite-vec embedding integration.
Each step has a corresponding test:
- Round-trip test: write Tier 1 frontmatter, project to Tier 2 SQL, query, confirm semantic equivalence to source data.
- Idempotency test: project twice; second projection is a no-op.
- Recovery test: delete the .sqlite, reload, confirm reprojection produces byte-identical state.
§10 Open sub-decisions (deferred from this spec)
Section titled “§10 Open sub-decisions (deferred from this spec)”Items flagged in Ch 07’s “remaining open sub-decisions” and elsewhere that this spec does not lock in:
- UUID enterprise resilience scope — Ch 09 settled UUIDv7 + sha256 + CURIEs; the enterprise scope (cross-tenant ID stability) needs a separate design pass.
- Multi-editor conflict resolution — when two users edit the same junction note simultaneously, what does the merge look like? Out of v0.1 scope; v1.0+ collaboration story.
- Legacy inline-field migration script for v0 vaults — for users who have notes with the superseded
key:: valuesyntax. Out of v0.1 scope. - Tested vault scale threshold documentation — Ch 18 gives a model; need empirical confirmation in real vaults of varying size.
- OSCAL ↔ Crosswalker mental-model documentation —
reference/registry/oscal/page extension. Listed in roadmap Foundation tasks. StewardshipProfilev2 schema — separate from this spec but referenced viaImportRecipe.stewardship_profile_id. Per the 05-01 commitments log, the schema design pass is deferred until after v0.1 schemas land.
§11 Related
Section titled “§11 Related”- Concept model:
- Artifact roles and authority — Schema Contract, Knowledge Set, Recipe, administrative envelope, representation, and carrier distinctions
- Terminology — ImportRecipe, MappingSet, MappingAssertion, ImportMapping, SavedConfig, live view, and materialized snapshot
- System architecture — representation tiers and logical layers
- Ontology lifecycle — Acquire, Shape, Connect, Understand, Share, Maintain
- Decision logs:
- v0.1 stack-pivot log — the stack commitment this spec implements
- Evidence-link edge model synthesis (Ch 07) — junction note schema source
- 04-10 Foundation research synthesis — STRM + SSSOM crosswalk edge semantics
- 05-01 Foundation commitments + meta-schema lifecycle — “schemas are versioned, migration-aware”
- Research deliverables:
- Ch 09 deliverable: identifier strategy
- Ch 18 deliverable: Tier 2-Lite scale ceiling — informs Tier 2 SQL design
- Earlier design pages (background, partly superseded by this unified spec):
- Config schema design — original ImportRecipe draft
- Constraint enforcement & metadata tiers — original
_crosswalkermetadata tiers
- External references:
- STRM (NIST IR 8477) — predicate vocabulary
- SSSOM spec — internal validation envelope
- OSCAL — junction-note isomorphism (
by-component) - Datalog glossary — context for query layers above this schema