🚧 Early alpha — building the foundation. See the roadmap →

System architecture — components, tiers, and data flow

Updated Nov 13, 2026

Status last verified: 2026-09-04 — verified against src/ module layout (three tiers: source/, generation output as Tier 1 Markdown, tier2/ for the derived projection; layers spanning import/, generation/, tier2/, export/, ui/) and the artifact roles and authority page. Tier 2 sidecar rows (Layer 3 components and the tier2/ codebase map) re-verified 2026-10-03 against src/tier2/sidecar.ts, db.ts, worker-db.ts and worker-source.ts; the rest of the page keeps its 2026-09-04 verification.

This page is the detailed reference for how Crosswalker’s pieces fit together — the component-to-tier matrix, the read/write flows, the codebase map. It consolidates information that was previously scattered across the ETL concept, the embedded substrates concept, the v0.1 schema spec, and per-milestone pages.

TL;DR

Crosswalker has three representation/storage tiers and six logical layers that move data between them. These are independent of artifact role and lifecycle capability. For the convention view of the same stack — the named DSL at each layer, built or adopted, as one map — see Conventional systems in the stack.

Dimension	What it classifies	Crosswalker examples
Artifact role	Meaning and authority	Schema Contract; Knowledge Set; Recipe; later Execution Record and Package Manifest
Representation tier	Where knowledge is represented	Tier 0 source; Tier 1 canonical managed Markdown; Tier 2 derived SQLite projection/cache
Logical layer	System responsibility	Import, storage, projection, query, export, audit
Lifecycle capability	Recurring outcome	Acquire, Shape, Connect, Understand, Share, Maintain

No generalized KnowledgeSet schema is claimed in v0.1. Tier 1 is the canonical representation/authority tier in which concrete Knowledge Sets are materialized; it is not itself a root artifact class.

                         CROSSWALKER IN ONE DIAGRAM

   ┌─ Tier 0 — source data ──────┐      ┌─ ImportRecipe ───────────────┐
   │ CSV, JSON, JSONL, XLSX      │      │ intensional policy          │
   │ upstream input              │      │ governed by Recipe Schema   │
   └──────────────┬──────────────┘      └──────────────┬──────────────┘
                  └──────────────────────┬──────────────┘
                                         │
                                 (1) IMPORT layer
                                  parse · render · validate
                                         │
                                         ▼
   ┌──────────────────────────────────────────────────────────────────┐
   │ Tier 1 — canonical Crosswalker-managed Markdown representation  │
   │ records governed by the Tier 1 Schema Contract                  │
   │ materializes concepts and assertions from Knowledge Sets        │
   │                                                                  │
   │ AUTHORITATIVE LOCAL MANAGED STATE. git-tracked; Obsidian reads  │
   │ it; Bases live views query it; exports derive from it.           │
   └───────────────────────────────┬──────────────────────────────────┘
                                   │
                         (3) TIER 1 → TIER 2 PROJECTION
                                   │
                                   ▼
   ┌──────────────────────────────────────────────────────────────────┐
   │ Tier 2 — SQLite sidecar (derived; deletable)                     │
   │ .crosswalker.sqlite: ontologies, concepts, mappings,             │
   │ junction_notes, closure_cache                                    │
   │                                                                  │
   │ CACHE/DERIVATIVE. Delete it → reproject from Tier 1.             │
   └──────────────────────────────────────────────────────────────────┘

Three representation tiers, simply:

Tier 0 = upstream source data in whatever representation it arrives
Tier 1 = Crosswalker’s canonical managed Markdown representation and local authority tier for Knowledge Sets
Tier 2 = a deletable SQLite projection/cache derived from Tier 1 for fast queries

Six logical layers, simply:

#	Layer	What it does	Where it operates
1	Import	Read source → write canonical Markdown	T0 → T1
2	Storage	Hold canonical state on disk	T1 (also T2 once projected)
3	Projection	Read T1 → write T2 (the projector)	T1 → T2
4	Query	Answer user questions over T1 + T2	T1, T2 → user
5	Export	Read T1 (or T2) → write external formats	T1, T2 → external
6	Audit	Track who-changed-what + integrity	git, signed manifests

The rest of this page expands each layer with diagrams, components, and code references.

The simplified picture

Three rules that make the whole architecture tractable:

Tier 1 is canonical local managed authority. Tier 0 is upstream; Tier 2, views, snapshots, and exports are derived. No authoritative Crosswalker-managed knowledge lives only in Tier 2.
Tier 2 is deletable. If .crosswalker.sqlite is missing, corrupted, or stale, the projector rebuilds it from T1 on next vault load. This is what makes Tier 2 risk-free to bundle (Ch 24 §2).
Schema Contracts govern producer boundaries. The Tier 1 Schema Contract makes conforming Tier 1 emitters first-class; the separate Recipe Schema Contract governs ImportRecipe specifications. The engine is replaceable (schema-as-primitive commitment).

These rules preserve two independent authorities: extensional Knowledge Sets represented canonically in Tier 1, and intensional Recipes represented canonically under the Recipe Schema. Neither should be reconstructed from a smaller implementation view model.

The three representation/storage tiers

Tier	What it is	Lifetime	Authority	Governing contract
Tier 0	Upstream source data in whatever shape it ships (CSV, JSON, JSONL, XLSX, OSCAL, etc.)	Whenever the user or publisher has it	Upstream authority may exist, but Tier 0 is not Crosswalker’s managed authority	n/a (source-specific contracts may apply)
Tier 1	Canonical Crosswalker-managed Markdown representation of concepts and assertions	Permanent (git-tracked)	Authoritative local managed knowledge	Tier 1 Schema Contract in `spec/tier1.schema.json`
Tier 2	`.crosswalker.sqlite` sidecar — a qualified Tier 1 → Tier 2 SQL projection	Recreatable on demand	Derived; never independently authoritative	v0.1 schema spec §7

What “canonical” means: every field in Tier 2 has to be derivable from Tier 1 frontmatter. Every queryable Tier 1 frontmatter field has to be a flat scalar or wikilink (no nested objects, no inline expressions) so it’s Bases-queryable directly without going through T2.

What’s NOT a tier:

ParsedData, ConceptIdentity, and Address are implementation/runtime values, not tiers or domain artifacts (see ETL concept “What ParsedData is”).
ImportRecipe is an intensional Recipe artifact adjacent to generation, governed by spec/recipe.schema.json; it is not inside Tier 1 and is not ontology content.
A MappingSet is a Knowledge Set subtype; each MappingAssertion may be represented by a Tier 1 member record, but the tier is not the set.
The closure cache is not a separate tier; it is a materialized cache inside Tier 2.
Live Bases views and materialized snapshots are derivative renditions, not new tiers.

Layer 1 — Import (T0 → T1)

One-liner: apply an ImportRecipe to structured source records, then validate and write Tier 1 records representing Knowledge Set members.

                       IMPORT LAYER

  Source data            ┌─ ImportRecipe ───────────────┐
  (T0)                   │ intensional specification    │
   │                     │ governed by Recipe Schema    │
   ▼                     │ adjacent to generation       │
  Parser                 │ (not inside Tier 1)          │
  (PapaParse,            │ source-to-vault policy      │
   xlsx, JSON)           └───────────┬───────────┘
   │                                 │
   ▼                                 │
  ParsedData                         │
  (in-memory iter or                 │
   AsyncIterable for                 │
   streaming sources)                │
   │                                 │
   └──────────┬──────────────────────┘
              │
              ▼  per row
   ┌─────────────────────────────────────────┐
   │  ConceptIdentity {curie, scope}         │
   │  + Recipe                               │
   │              │                          │
   │              ▼                          │
   │  render() ◄── pure function (Ch 22)     │
   │              │                          │
   │              ▼                          │
   │  Address (path + frontmatter +          │
   │           tags + aliases +              │
   │           wikilinkTarget)               │
   │              │                          │
   │              ▼                          │
   │  + buildProvenance() (_crosswalker)     │
   │              │                          │
   │              ▼                          │
   │  validateTier1Frontmatter() ◄── PRE-    │
   │              │           WRITE GATE     │
   │              ▼                          │
   │  mergeFrontmatter() if existing file    │
   │              │  (managed/user_preserve) │
   │              ▼                          │
   │  app.vault.create() / .modify()         │
   └──────────────┬──────────────────────────┘
                  │
                  ▼
              Tier 1 file

Components in this layer

Component	Code	What it does
Source parsers	`src/import/parsers/csv-parser.ts` (+ JSONL + JSON-with-iterator-path planned for v0.2)	T0 file → `ParsedData` (eager array OR `AsyncIterable<Row>`)
Streaming foundation	`src/import/parsers/csv-parser.ts:parseCSVFileStream`	PapaParse `step` callback → AsyncIterable with backpressure (v0.1.4.5)
render()	`src/render/index.ts`	Pure function `(Recipe, ConceptIdentity) → Address` (v0.1.2)
Template engine	`src/render/template.ts`	R2RML-style `{var\|filter}` interpolation; closed 7-filter set; JSONata sub-language reserved for v0.3+
Mechanisms	`src/render/mechanisms/{folder,file,heading,tag,wikilink}.ts`	The 5 closed hierarchy primitives — folder/file/heading wired in v0.1; tag/wikilink reserved for v0.2
Kind dispatch	`src/render/index.ts` (Tier1Kind)	`kind: concept \| junction-note \| crosswalk-edge` discriminator (v0.1.4)
Generation engine	`src/generation/generation-engine.ts:generateNotes / generateFromRecipe`	Per-row loop calling render() + provenance + validation + merge → vault.create/modify
Frontmatter merge	`src/generation/frontmatter-merge.ts`	managed (recipe-owned, overwritten) vs user_preserve (recipe-untouched) per Ch 22 §8.4
Provenance writer	`src/generation/provenance.ts`	`_crosswalker` block per Tier 1 schema $defs/provenance_block
Validator (pre-write gate)	`src/validation/validator.ts:validateTier1Frontmatter`	AJV against `spec/tier1.schema.json`. Strict mode rejects bad rows pre-write — STRM predicate enforcement happens here
Legacy-recipe shim	`src/generation/legacy-recipe-shim.ts`	Translates v0.1.0 column-role configs → Ch 22 layout Recipe (Phase-0 compat per Ch 22 §10.7)

Two-mode architecture

The import layer supports two architectural modes (decided 2026-05-05):

Mode 1 — bundled projector: external producer (or wizard) hands the bundled engine structured rows + recipe → engine emits Tier 1
Mode 2 — direct emission: external producer (AI agent, MCP server, marketplace bundle) writes Tier 1 Markdown directly, bypassing the bundled engine

Both are first-class. ChunkyCSV / JSONaut / dbt are natural Mode 1 feeders. AI agents are natural Mode 2 emitters. See ETL concept “two-mode architecture” for the full picture.

Milestones that ship this layer

v0.1.1 (validation), v0.1.2 (render), v0.1.3 (engine), v0.1.4 (kind dispatch + STRM), v0.1.4.5 (streaming foundation).

Layer 2 — Storage (Tier 1 = canonical)

One-liner: hold the truth as Markdown files on disk.

                       STORAGE LAYER

  vault root/
  ├── Frameworks/                         ◄── concept-note files
  │   ├── NIST 800-53 r5/
  │   │   ├── AC/
  │   │   │   ├── AC-2.md            ─┐
  │   │   │   └── AU-1.md            │ kind: concept (default)
  │   ├── NIST CSF 2.0/              │ frontmatter: curie, title,
  │   │   ├── PR.AC-01.md           ─┘ parent, _crosswalker, ...
  │
  ├── Crosswalks/                         ◄── crosswalk-edge files
  │   ├── CSF-to-800-53/
  │   │   ├── cw-pr-ac-01-ac-2.md   ─┐
  │   │   └── cw-id-am-01-cm-8.md   ─┘ kind: crosswalk-edge
  │                                    frontmatter: subject_id,
  │                                                 predicate_id,
  │                                                 object_id, ...
  ├── Evidence/
  │   └── Junctions/                      ◄── junction-note files
  │       └── jn-ac-2-mfa-policy.md   ─── kind: junction-note
  │                                       frontmatter: subject,
  │                                                    predicate,
  │                                                    object, ...
  │
  ├── .crosswalker.sqlite                 ◄── Tier 2 sidecar
  │                                          (derived; deletable)
  │
  └── _crosswalker_manifest.json          ◄── Audit (Tier 1
                                              floor, v0.1.8)

Properties of Tier 1 storage

Representation, not root artifact: Tier 1 files materialize member records from one or more Knowledge Sets under a canonical local managed authority tier
File-based: every concept, evidence link, and crosswalk assertion is represented by its own .md file
YAML frontmatter is the data: per Ch 22 §1.3, frontmatter is flat-scalar / wikilink only, not nested objects
git-tracked: every change is a commit (v0.1.8 audit trail)
Obsidian-native: files appear in the file explorer, search, graph view, Bases queries, etc., natively
Survives plugin uninstall: even with no plugins enabled, the vault is still readable Markdown
Three Tier 1 member-record shapes discriminated by kind — concept / junction-note / crosswalk-edge (Tier 1 Schema Contract). They express different assertion shapes; v0.1 does not claim a generalized KnowledgeSet aggregate schema.

Why files instead of a database

See concepts/file-based-graph-database for the full philosophical case. Short version: files are diffable, mergeable, exportable, archivable, attestable, and survive every tool/plugin in the stack getting replaced.

Layer 3 — Tier 1 → Tier 2 projection

One-liner: perform the qualified Tier 1 → Tier 2 projection by reading canonical Markdown records and writing reproducible SQLite rows.

Crosswalker always qualifies projection by scope. This layer is not the same as a field projection, read/view projection, or editable UI projection.

                     PROJECTION LAYER

  Tier 1                                      Tier 2
  ─────────────────                           ───────────────────
  Frameworks/                                 ┌─ ontologies ────┐
  NIST/AC-2.md            ─────►              │ id │name │ver  │
   curie: nist:AC-2                           └─────────────────┘
   title: ...
   parent: nist:AC                            ┌─ concepts ──────┐
   _crosswalker: {...}                        │ ontology│curie│ │
                                              │ ─path───┼──── │
                                              │ vault_path:    │
  Crosswalks/                                 │ Frameworks/... │
  cw-csf-pr-ac-01-                            └─────────────────┘
  nist-ac-2.md            ─────►
   kind: crosswalk-edge                       ┌─ mappings ──────┐
   subject_id: csf:PR.AC-01                   │ subj│pred│obj │ │
   predicate_id: is_eq...                     │ id  │ id │ id │ │
   object_id: nist:AC-2                       └─────────────────┘
   match_type: ...
                                              ┌─ junction_notes ─┐
  Evidence/                                   │ vault_path│curie│ │
  jn-ac-2-mfa.md          ─────►              │ subject│pred│obj│ │
   kind: junction-note                        └──────────────────┘
   subject: AC-2
   predicate: covers                          ┌─ closure_cache ──┐
   object: MFA-Policy                         │ subj│pred│obj │  │
                                              │ shortest_depth│  │  ◄── populated lazily
                                              └──────────────────┘     on first transitive
                                                                       query
                                              ┌─ schema_meta ────┐
                                              │ key │ value     │
                                              │ schema_version:  │
                                              │ tier2-sqlite-v2  │
                                              └──────────────────┘

Components in this layer

Component	Code	What it does
Sidecar lifecycle	`src/tier2/sidecar.ts:openSidecar/clearSidecar`	Open `.crosswalker.sqlite` in a dedicated Worker on the OPFS sahpool VFS (plugin-private, per-vault pool directory); fall back to a main-thread `:memory:` database when the Worker or OPFS is unavailable, reported as `info().persistent === false` and in Settings (“Search index: …”). Before 2026-10-03 sahpool was installed on the main thread, where Chromium has no `createSyncAccessHandle`, so the index never persisted on any host
Database seam	`src/tier2/db.ts` (`Tier2Db`, `LocalDb`)	One async `exec` / `execBatch` / `close` / `info` interface; `LocalDb` wraps a main-thread `oo1.DB` (Jest and the runtime fallback). `execBatch` runs its statements inside one SAVEPOINT
Worker host	`src/tier2/worker-db.ts` (main-thread side), `src/tier2/worker-source.ts` (the Worker program, inlined into `main.js`)	Worker built from a Blob URL so the plugin still ships three files; one init message carries the sqlite module text and wasm bytes; request/response queue with ordered replies. The import engine and projector stay on the main thread (Ch 23 §9.5, narrowed 2026-10-03 to the engine, not a pure sqlite host)
Schema migrations	`src/tier2/migrations.ts`	drop-and-recreate on version mismatch (correct because T2 is purely a T1 projection)
Schema DDL	`src/tier2/schema.sql`	The full Tier 2 SQL schema per spec §7
Projector	`src/tier2/projector.ts` (Phase 2 of v0.1.5)	Walk vault `.md` files via streaming foundation; dispatch by `kind`; idempotent upsert
Closure cache	`src/tier2/queries.ts` (Phase 3 of v0.1.5)	Recursive-CTE materialization on first transitive query; mtime-based invalidation

Recovery property (load-bearing)

If .crosswalker.sqlite is missing, corrupted, or stale, the projector reprojects from canonical Tier 1 on next vault load. This is what makes Tier 2 risk-free to bundle. Per Ch 24 §2 — the modularity rule that lets us swap substrates (libSQL/Limbo/server) without data migration.

Substrate

@sqlite.org/sqlite-wasm (canonical, foundation-governed). sqlite-vec deferred — see the WASM-A path notes in v0.1.5 and the calendar-anchored 2026-11-06 revisit checkpoint in Ch 24 §5 Q4.

Milestones that ship this layer

v0.1.5 (the entire projection layer).

Layer 4 — Query (T1 + T2 → user)

One-liner: let users ask questions over their vault — Bases provides live views over T1; SQL handles joined or transitive reads over the derived T2 projection; materialized snapshots persist selected point-in-time results separately.

                        QUERY LAYER

   User question                  Routing decision
   ─────────────                  ────────────────
   "Show me all NIST          ─►  Bases over Tier 1 frontmatter
    AC-family controls"           (file.hasTag('framework/nist/ac'))

   "What evidence covers      ─►  Bases over Tier 1 frontmatter
    AC-2?"                        (junction-note where subject == AC-2)

   "Which CSF                 ─►  Bases (simple case) OR SQL (cross-
    subcategories map to          ontology join)
    NIST AC-family?"

   "Show all controls         ─►  SQL recursive CTE over mappings
    NIST CSF Identify             table (Tier 2; closure_cache
    maps to via any chain         materialized on first call)
    of crosswalks"

   "Coverage matrix:          ─►  SQL join over concepts ×
    every control × every         mappings × junction_notes
    framework that maps           (Tier 2)
    to it × evidence count"

Components in this layer

Component	Code	What it does
Bases (upstream Obsidian)	n/a (built-in to Obsidian)	Flat YAML-frontmatter queries directly over Tier 1 — no plugin needed
SQL query helpers	`src/tier2/queries.ts` (v0.1.5 Phase 3)	Typed `getControlsByOntology`, `findCoverageGaps`, `crosswalkBetween`, `closureFromConcept`
Bases query templates emitted in concept-note bodies	Body-template rendering in `src/render/body.ts` + `src/render/template.ts` (v0.1.6); no separate templates directory	Recipe-author authors a body template; engine emits a working Bases query in each concept note

When to use which

Question shape	Tier	Why
Flat key/value over a single note kind	Tier 1 via Bases	Native; no plugin; survives Tier 2 corruption
Tag-prefix or single-frontmatter-field filter	Tier 1 via Bases	Same
Multi-ontology joins	Tier 2 via SQL	Bases doesn’t do joins; SQL does
Transitive closure (chain of crosswalks)	Tier 2 via SQL recursive CTE	Depth-covered cache partitions safely reuse equal or shallower queries; deeper queries extend coverage
Coverage matrix (controls × evidence × frameworks)	Tier 2 via SQL	Multi-table join; awkward in Bases
Vector / semantic similarity	Tier 2 via sqlite-vec	Deferred to a future milestone (see Ch 24 §5 Q4)

The choice is the recipe author’s (or query author’s). Tier 1 is the default for simple queries because it survives Tier 2 corruption. Tier 2 is for when SQL’s expressivity is needed.

A live view evaluates current data when opened or refreshed; Bases views belong here. A materialized snapshot persists query results at a point in time and can become stale. Snapshots are useful derivative evidence and publishing fallbacks, but they do not become canonical Knowledge Sets merely because they are stored in the vault.

Three-layer query engine architecture

The query layer separates concerns across three orthogonal layers (settled 2026-05-08; see synthesis log):

Layer	Vocabulary	Concept page
A — Query primitives	filter / traverse / bind / project / aggregate / anti-join / set-op / diff	query-primitives
B — View shapes	table / list / pivot / graph / hierarchy / timeline	view-shapes
C — Recipes / reports	”Coverage Matrix”, “Crosswalk Density”, “Ontology Overlap”, etc.	recipe registry

Query verbs × mechanisms

The canonical Layer A vocabulary is the 8 query verbs — filter, traverse, bind, project, aggregate, anti-join, set-op, diff (locked by Ch 29, 2026-05-09; closure folded into traverse(depth=*), pivot demoted to a Layer B view shape).

Each verb executes on one of four mechanisms: Bases-native, Tier 2 SQL helpers, codeblock processor (v0.1.7), or materialized snapshot (v0.1.8).

query-primitives holds the single verb-to-mechanism table; it is not duplicated here.

Crosswalker recipes (Layer C) name a primitive composition + a view shape. The engine picks the right mechanism for each primitive based on what’s expressible at that layer. v0.1.6 ships the crosswalkerPivot custom Bases view (registerBasesView) for the pivot shape; v0.1.7 ships the crosswalker-query codeblock processor; v0.1.8 ships materialized snapshot output.

Milestones that ship this layer

v0.1.6 (Bases templates + crosswalkerPivot custom view + recipe loader); v0.1.7 (codeblock processor); v0.1.8 (materialized snapshot output).

Layer 5 — Export (T1 → external; or T2 → external)

One-liner: serialize Tier 1 (or Tier 2) data into external interop formats.

                        EXPORT LAYER

   Tier 1 / Tier 2  ──►  Exporter  ──►  External format
   ────────────────      ────────       ────────────────

   crosswalk-edge        STRM          NIST IR 8278A r1
   notes (T1)            TSV           OLIR template
   OR mappings (T2)      exporter      (Excel-friendly TSV)
                                        round-trip determ.

   junction-note         OSCAL JSON    NIST OSCAL
   notes (T1)            profile       Control Mapping Model
   OR junction_notes     exporter      (.json profile)
   (T2)

   crosswalk-edge        SSSOM TSV     SSSOM (Mapping Commons)
   notes (T1)            exporter      academic interop
   OR mappings (T2)      (opt-in)      (TSV with metadata
                                        header; off by default
                                        per Ch 19 — zero GRC
                                        adoption)

Components in this layer

Component	Code	What it does
STRM TSV exporter	`src/export/strm-tsv-exporter.ts` (v0.1.7)	Reads T2 mappings; emits NIST IR 8278A r1 OLIR template format
OSCAL JSON exporter	`src/export/oscal-profile-exporter.ts` (v0.1.7)	Reads T1 junction notes (or T2 junction_notes); emits OSCAL Control Mapping profile
SSSOM TSV exporter	`src/export/sssom-exporter.ts` (v0.1.7)	Reads T2 mappings; emits SSSOM-shaped TSV; opt-in toggle

Round-trip determinism

The export layer commits to byte-identical round-trip: import a STRM TSV → export the resulting Tier 1 vault → STRM TSV bytes match. This is enforced by tests in v0.1.7. Importing your own export and re-exporting must be a fixed point.

Milestones that ship this layer

v0.1.7 (the entire export layer).

Layer 6 — Audit (cross-cutting)

One-liner: track who-changed-what, when, with what integrity guarantees.

Audit implementation objects are not automatically the later administrative artifact contracts. A future Execution Record will separately report one bound Recipe run; a future Package Manifest will separately resolve a distributed release. Per-note _crosswalker provenance, git commits, and the planned v0.1 audit manifest are narrower records, not those complete envelopes.

                         AUDIT LAYER

   Tier 1 = git commits (T1 audit floor)
            ─────────────────────────
            Every generation pass produces one commit
            Commit message includes recipe id + recipe hash +
            concept count + summary diff
            git log + git blame are the audit trail

                   +
            ────────────────────────────────────────────
            Optional Ed25519-signed audit manifests
            for shared mapping bundles
            (not yet a Package Manifest contract)

                   +
            ────────────────────────────────────────────
            On-demand FRE 902(13) certification PDF
            (US litigation evidence-submission template)

   Tier 2/Tier 3 audit profiles (deferred to v1.0+):
            ─────────────────────────────────────────
            T2: OpenTimestamps / RFC 3161 trusted timestamps,
                S3 Object Lock WORM mirror
            T3: Sigstore Rekor v2 + in-toto attestations,
                eIDAS QTSA + W3C VC
            All opt-in compliance-export mode (per Ch 13/15)

Components in this layer

Nothing in this layer has landed yet — the modules below are planned for v0.1.8 and no audit source directory exists in the repo today.

Component	Code	What it does
git commit hook	Planned module (v0.1.8)	One commit per generation pass; deterministic message
Ed25519 sign helper	Planned module (v0.1.8)	`@noble/curves`; signs release manifests
Audit manifest writer	Planned module (v0.1.8)	Planned `_crosswalker_manifest.json` integrity record; not a settled Package Manifest contract
FRE 902(13) PDF template	Planned module (v0.1.8)	`pdf-lib`-rendered certification template

Milestones that ship this layer

v0.1.8 (Tier 1 audit floor: git + Ed25519 + FRE 902(13)).

Component-to-tier matrix

A single table of every load-bearing logical component, where it lives, and what it reads/writes:

Component	Layer	Reads	Writes	Code
Source parser	Import	T0 file	`ParsedData` (in-memory iter)	`src/import/parsers/*.ts`
ImportRecipe specification	Import-adjacent Recipe artifact	source declaration + target policy	Governs generation intent; remains outside Tier 1 knowledge	`spec/recipe.schema.json` + generated recipe types
`render()`	Import	ImportRecipe + ConceptIdentity	`Address` (in-memory)	`src/render/index.ts`
Generation engine	Import	`ParsedData` + ImportRecipe	Tier 1 `.md` member records	`src/generation/generation-engine.ts`
Validators	Import	ImportRecipe or Tier 1 frontmatter	(returns valid/invalid under the corresponding Schema Contract)	`src/validation/validator.ts`
Frontmatter merge	Import	Existing frontmatter + new managed	Merged frontmatter	`src/generation/frontmatter-merge.ts`
Provenance writer	Import	source ref + recipe id	`_crosswalker` block	`src/generation/provenance.ts`
Tier 1 vault	Storage	(passive)	(held on disk)	n/a — files
Sidecar lifecycle	Projection	(init only)	Opens `.crosswalker.sqlite`; applies migrations	`src/tier2/sidecar.ts`
Projector	Projection	Tier 1 `.md` frontmatter	`ontologies`, `concepts`, `mappings`, `junction_notes` rows	`src/tier2/projector.ts` (v0.1.5 Phase 2)
Closure cache populator	Projection	`mappings` table	`closure_cache` table	`src/tier2/queries.ts:closureFromConcept`
Bases queries	Query	Tier 1 frontmatter	(returns user data)	n/a (upstream Obsidian)
SQL query helpers	Query	Tier 2 tables	(returns user data)	`src/tier2/queries.ts`
Body-query templates	Query	Recipe template	Tier 1 note body (Bases query string)	`src/render/body.ts` + `src/render/template.ts` (v0.1.6)
STRM TSV exporter	Export	T2 mappings (or T1 crosswalk-edge notes)	`.tsv` file	`src/export/strm-tsv-exporter.ts` (v0.1.7)
OSCAL JSON exporter	Export	T1 junction notes (or T2)	`.json` file	`src/export/oscal-profile-exporter.ts` (v0.1.7)
SSSOM TSV exporter	Export	T2 mappings	`.tsv` file	`src/export/sssom-exporter.ts` (v0.1.7)
git commit hook	Audit	(write event)	git commit	planned module (v0.1.8)
Ed25519 signer	Audit	Release bundle	Signature on `_crosswalker_manifest.json`	planned module (v0.1.8)
FRE 902(13) PDF	Audit	Vault hash + signed manifest	`.pdf` file	planned module (v0.1.8)

Read/write data flow

Which components read/write which storage tier:

Tier	Written by	Read by
T0 (source)	the user (or external producer)	Source parser
T1 (canonical Markdown)	Generation engine; external Mode 2 producers	Bases queries; projector; exporters; git audit
T2 (SQLite sidecar)	Sidecar lifecycle (open + migrations); projector; closure cache populator	SQL query helpers; exporters (when joining over T2 tables)
External (TSV / JSON / PDF)	Exporters; FRE 902(13) PDF generator	(downstream tools — Excel, OSCAL validators, courts)

Notice that every derived knowledge rendition ultimately traces back to Tier 1: T0 plus an ImportRecipe produces T1, T2 derives from T1, exports come from T1 (or T2 which derives from T1), and audit hashes T1. No authoritative managed knowledge lives only in T2.

The ImportRecipe is intentionally absent from the tier table: it is an adjacent intensional artifact governed by the Recipe Schema Contract. A MappingSet or ontology is extensional knowledge represented through Tier 1 member records. A future Execution Record and Package Manifest will be separate administrative envelopes rather than new tiers.

Where to look in the codebase

src/
├── main.ts                        ── Plugin entry point. Wires everything.
│
├── render/                        ── render() — Layer 1 single coupling point
│   ├── index.ts                      Pure (Recipe, ConceptIdentity) → Address
│   ├── template.ts                   {var|filter} interpolation engine
│   ├── types.ts                      Address, ConceptIdentity, SourceScope
│   └── mechanisms/                   The 5 closed primitives
│       ├── folder.ts                 wired
│       ├── file.ts                   wired
│       ├── heading.ts                wired
│       ├── tag.ts                    deferred to v0.2 (stub)
│       └── wikilink.ts               deferred to v0.2 (stub)
│
├── import/                        ── Layer 1 (source → render input)
│   ├── parsers/
│   │   └── csv-parser.ts             PapaParse + streaming (v0.1.4.5)
│   └── import-wizard.ts              The wizard UI (column-role mapping)
│
├── generation/                    ── Layer 1 (render output → Tier 1)
│   ├── generation-engine.ts          Per-row loop; calls render(); writes vault
│   ├── frontmatter-merge.ts          managed/user_preserve semantics
│   ├── provenance.ts                 _crosswalker block writer
│   └── legacy-recipe-shim.ts         v0.1.0 column-role → Ch 22 layout
│
├── validation/                    ── Layer 1 pre-write gate
│   └── validator.ts                  AJV against spec/tier1.schema.json
│
├── tier2/                         ── Layer 3 (projection)
│   ├── sidecar.ts                    sidecar lifecycle; Worker first, in-memory fallback
│   ├── db.ts                         Tier2Db seam + LocalDb (main-thread fallback, tests)
│   ├── worker-db.ts                  main-thread client for the sqlite Worker
│   ├── worker-source.ts              Worker program (OPFS sahpool), inlined into main.js
│   ├── schema.sql                    Tier 2 DDL (canonical .sql doc)
│   ├── migrations.ts                 schema_version + drop-and-recreate
│   ├── projector.ts                  Tier 1 → Tier 2 (v0.1.5 Phase 2)
│   └── queries.ts                    Typed query API + closure cache (v0.1.5 Phase 3)
│
├── export/                        ── Layer 5 (T1/T2 → external)
│   ├── strm-tsv.ts                   v0.1.7
│   ├── oscal-profile.ts              v0.1.7
│   └── sssom-tsv.ts                  v0.1.7
│
├── audit/                         ── Layer 6 (cross-cutting)
│   ├── git-commit.ts                 v0.1.8
│   ├── sign.ts                       v0.1.8
│   ├── manifest.ts                   v0.1.8
│   └── fre-902.ts                    v0.1.8
│
├── settings/                      ── Plugin settings (cross-layer)
└── types/
    └── config.ts                     ParsedData, ImportRecipe, etc.

spec/
├── tier1.schema.json              ── Tier 1 frontmatter contract
└── recipe.schema.json             ── Recipe shape contract

recipes/starter/                   ── Mode 1 starter recipes
├── nist-csf-to-800-53-crosswalk.json
├── iso27001-to-800-53-crosswalk.json
├── evidence-junction-notes.json
└── README.md                         multi-framework recipe-pattern story

Where to look in the docs

If you want…	Read this
Quickest way to grasp the system	This page (TL;DR section)
The Tier 1 Schema Contract and member-record representations	v0.1 schema spec
The Recipe Schema Contract and ImportRecipe specification shape	`spec/recipe.schema.json` + Ch 22 synthesis
Artifact roles versus tiers, layers, derivatives, and carriers	Artifact roles and authority
Why the architecture is shaped this way (the philosophy)	What makes Crosswalker unique + Vision
Why Tier 1 is files (not a database)	File-based graph database concept
Why Tier 2 is sqlite-wasm (and not Turso/libSQL/Limbo)	Ch 24 substrate synthesis
The closed 5-mechanism recipe grammar	Hierarchy primitives concept + Ch 22 synthesis
The two-mode architecture (bundled vs direct)	ETL two-mode section + decision log
Definitions of every term used here	Terminology
The full implementation roadmap	Milestones hub
The 6 architectural commitments locked 2026-05-04	v0.1 import-engine design log
Why no JSONaut/ChunkyCSV port + GUI scope line	Transform-engine-depth log

How v0.1 implementation maps to this architecture

v0.1.1 ── Import layer types + validation foundation
            └── AJV + spec/*.schema.json wired into runtime

v0.1.2 ── Import layer: render() v1
            └── (Recipe, ConceptIdentity) → Address (folder/file/heading)

v0.1.3 ── Import layer: generation engine integration
            └── render() wired through engine; provenance + merge

v0.1.4 ── Import layer: kind dispatch (3 Tier 1 shapes)
            └── concept / junction-note / crosswalk-edge + STRM gate

v0.1.4.5 ─ Import layer: streaming refactor
            └── ParsedData.rows accepts AsyncIterable<Row>
            └── Engine iterates via for-await; no full-vault-in-RAM

────── Layer 1 + 2 (storage) complete ──────

v0.1.5 ── Projection layer: Tier 2 sqlite-wasm sidecar     ◄── WE ARE HERE
            ├── Phase 1 (DONE): substrate scaffolding
            ├── Phase 2: projector (Tier 1 → Tier 2 tables)
            ├── Phase 3: query API + closure cache
            ├── Phase 4: plugin integration
            ├── Phase 5: tests
            └── Phase 6: delivery log + status flip

v0.1.6 ── Query layer: Bases templates + SQL helpers
            └── Concept-note bodies emit working Bases queries

v0.1.7 ── Export layer: STRM TSV + OSCAL + SSSOM
            └── Round-trip determinism enforced

v0.1.8 ── Audit layer: git + Ed25519 + FRE 902(13)
            └── Tier 1 audit floor

v0.1-RC ── Ship: bundle size, mobile, community plugin submission

Phase 2 of v0.1.5 (the projector) is the next unit of work — it’s the bridge between “Tier 1 exists” and “fast queries are possible.”

Concept pages (this page links to and from):

Artifact roles and authority — Schema Contract, Knowledge Set, Recipe, administrative envelopes, derivatives, and carriers
Ontology lifecycle — Acquire, Shape, Connect, Understand, Share, Maintain
ETL and import — the import layer details + two-mode architecture
Hierarchy primitives — the 5 closed mechanisms render() dispatches over
Embedded vs server substrates — why Tier 2 is embedded
File-based graph database — why Tier 1 is files
Terminology — definitions for every term
What makes Crosswalker unique — architectural differentiators
Ontology evolution — how T1 changes are tracked
Metadata ecosystem — Bases query layer context
Consistency models — T1 ↔ T2 consistency semantics

Agent context:

v0.1 schema spec — Tier 1 + recipe + Tier 2 SQL DDL
Vision — why this architecture exists
Tradeoffs

Foundation decisions (synthesis logs — six load-bearing commitments settled 2026-05-04):

v0.1 import-engine design log — the canonical commitments page
Ch 22 — target-structure expressivity — recipe grammar + render()
Ch 23 — bundle/engine/language — TS in-plugin engine + JSONata
Ch 24 — Tier 2 substrate — sqlite-wasm + sqlite-vec + 5 migration triggers + 2026-11-06 vec revisit

Recent design decisions (2026-05-05):

Two-mode architecture — bundled projector + direct emission
Transform-engine depth + input formats — JSONata sub-language; v0.2 JSONL
ETL pipeline clarification — what ParsedData is/isn’t

Implementation milestones:

Milestone hub
v0.1.1 (validation) · v0.1.2 (render) · v0.1.3 (engine) · v0.1.4 (kind dispatch) · v0.1.4.5 (streaming)
v0.1.5 (projection) · v0.1.6 (query) · v0.1.7 (export) · v0.1.8 (audit) · v0.1-RC (ship)

Spec files:

spec/tier1.schema.json — governing Tier 1 Schema Contract for canonical member-record representations
spec/recipe.schema.json — governing Recipe Schema Contract for ImportRecipe specifications

Registry references (frameworks the architecture is generic over):

STRM · SSSOM · OSCAL · OLIR
NIST · ISO · MITRE · CIS · CRI · FFIEC · SKOS

System architecture — components, tiers, and data flow

TL;DR

The simplified picture

The three representation/storage tiers

Layer 1 — Import (T0 → T1)

Components in this layer

Two-mode architecture

Milestones that ship this layer

Layer 2 — Storage (Tier 1 = canonical)

Properties of Tier 1 storage

Why files instead of a database

Layer 3 — Tier 1 → Tier 2 projection

Components in this layer

Recovery property (load-bearing)

Substrate

Milestones that ship this layer

Layer 4 — Query (T1 + T2 → user)

Components in this layer

When to use which

Three-layer query engine architecture

Query verbs × mechanisms

Milestones that ship this layer

Layer 5 — Export (T1 → external; or T2 → external)

Components in this layer

Round-trip determinism

Milestones that ship this layer

Layer 6 — Audit (cross-cutting)

Components in this layer

Milestones that ship this layer

Component-to-tier matrix

Read/write data flow

Where to look in the codebase

Where to look in the docs

How v0.1 implementation maps to this architecture

Related