Skip to content
Crosswalker
🚧 Early alpha — building the foundation. See the roadmap →

Consistency models

Updated

Why consistency matters for Crosswalker

Section titled “Why consistency matters for Crosswalker”

Every Crosswalker import creates hundreds of interconnected files. When a framework updates and you re-import, every file potentially needs updating — but file systems have no transaction support. Understanding formal consistency models explains why certain operations are safe, which can fail partway through, and how to design for recovery.

ACID analysis for file-based systems

Section titled “ACID analysis for file-based systems”

ACID guarantees reliable database transactions. Obsidian vaults provide only one of four:

PropertyDefinitionObsidian Vault Status
AtomicityAll operations succeed or all failPartial — a script can crash mid-import, leaving half-updated files
ConsistencyDatabase moves between valid statesNo — orphaned links, stale crosswalks, missing properties can accumulate
IsolationConcurrent operations don’t interfereNo — user can edit files while a script runs
DurabilityCommitted changes survive failuresYes — filesystem persistence, plus Obsidian Sync or git

Implication: Crosswalker cannot assume any import operation is atomic. Every operation should be designed to be idempotent (safe to re-run) and resumable (can pick up where it left off).

File-based systems get 1 out of 4 ACID guarantees

CAP theorem positioning

Section titled “CAP theorem positioning”

CAP theorem (Brewer, 2000): A distributed system can provide at most two of three guarantees:

GuaranteeDefinition
ConsistencyEvery read receives the most recent write
AvailabilityEvery request receives a response
Partition toleranceSystem operates despite network failures

Obsidian vaults are AP systems — they prioritize availability and partition tolerance:

  • Available: Users can always read and edit files, even offline
  • Partition tolerant: Vaults work across devices with sync, surviving network interruptions
  • Not consistent: Files can get out of sync between devices, or between import runs

This is the correct trade-off for a personal/team knowledge base. Strong consistency would require locking files during imports — unacceptable for a tool people use throughout the day.

BASE model

Section titled “BASE model”

The alternative to ACID for distributed systems:

PropertyDefinitionCrosswalker Application
Basically AvailableSystem is always usableVault is always readable/editable, even during imports
Soft stateState may change without inputObsidian cache updates, sync propagates changes asynchronously
Eventually consistentGiven enough time, all copies convergeAfter re-import, all files reflect the new framework version

Crosswalker operates under manual eventual consistency — convergence requires explicit action (running the import), not automatic propagation:

Framework updated → User triggers re-import → Files updated → Consistency achieved
                          ↑
                  (Manual trigger, not automatic)

CouchDB MVCC analogy

Section titled “CouchDB MVCC analogy”

Multi-Version Concurrency Control (MVCC): Maintaining multiple versions of data to handle concurrent access without locking.

CouchDB’s approach maps well to Crosswalker’s version-tagged imports:

CouchDBCrosswalker
Document revision (_rev: "3-a7b2f9c1")_crosswalker.source_hash on each note
Conflict detection (divergent revisions)Re-import detects changed vs. unchanged notes
Conflict resolution (user picks winner)User chooses overwrite vs. keep vs. merge
Eventual consistency via replicationRe-import propagates framework changes

The key insight: version-tagged folders (importing NIST 800-53 Rev 5 alongside Rev 4) is essentially MVCC at the folder level — multiple versions coexist until the user is ready to migrate.

Event sourcing pattern

Section titled “Event sourcing pattern”

Event sourcing stores state changes as an immutable sequence of events, reconstructing current state by replaying them.

Applied to Crosswalker’s _crosswalker metadata:

# Minimal (current): just the latest state
_crosswalker:
  source_file: nist-800-53.csv
  import_date: 2026-04-02
  framework_version: "Rev 5"

# Event-sourced (future): full change history
_crosswalker:
  history:
    - event: imported
      date: 2026-04-02
      source: nist-800-53-rev5.csv
    - event: re-imported
      date: 2026-10-15
      source: nist-800-53-rev5-update1.csv
      changes: [description_updated, related_controls_added]

This enables point-in-time queries (“what did this control look like when we imported it?”) and rollback (“revert to the pre-update state”). See ontology evolution for the temporal modeling foundations.

Implications for Crosswalker design

Section titled “Implications for Crosswalker design”
  1. Design for idempotency: Every import operation should produce the same result whether run once or ten times
  2. Expect partial failures: If an import crashes at note 347 of 1000, the next run should detect and resume
  3. Track provenance: _crosswalker metadata is the compensation for lacking ACID — it tells you what state you’re in
  4. Version, don’t overwrite: SCD Type 2 (keep history) is safer than Type 1 (overwrite) because it preserves the ability to diff and rollback
  5. Use Bases for detection: Obsidian Bases can surface notes with stale import_date or missing source_hash — lazy consistency checking via tabular views

The replay promise

Section titled “The replay promise”

Idempotency above is a design goal. For running a saved recipe again on the same source file, Crosswalker makes one specific, tested promise (decision D3 in the Gate 1 ratification):

Running the same recipe on the same source file into the same import set leaves every note’s managed content unchanged.

Managed content means what Crosswalker owns in a note: every frontmatter property it writes and the text inside each managed body region (between <!-- crosswalker:NAME:start --> and its :end marker). The comparison leaves out exactly the things a re-run is allowed to change:

Left outWhy
_crosswalker.produced_atThe time of the run
_crosswalker.producer.versionThe plugin build that ran
Properties the recipe marks as user-owned (user_preserve)Your edits, never overwritten
Body text outside managed regionsYour prose, never overwritten

What is not promised: byte-identical files. A note you have edited is no longer byte-identical to what was generated, so that promise would be false in any real vault. Byte-identity is not offered as an opt-in check either.

The promise has one definition in code, managedContentEquivalent in src/generation/managed-equivalence.ts, and the unit tests and the real-Obsidian Run again test both use it rather than their own comparison. If the source file or the recipe changed since the last run, the promise does not apply, and the wizard says so on its first step before anything is written. See the Run again synthesis.

The promise is proved on full public frameworks. Saving the NIST CSF 2.0 and NIST SP 800-53 recipes, reopening them from the library, and running them twice on the same file into the same set leaves every note’s managed content unchanged: 185 notes for CSF 2.0 and 1,189 for SP 800-53 (source: tests/replay-proof-nist.test.ts and tests/e2e/replay-proof-nist.spec.ts, both checked 2026-10-03). The proofs run through code and through the wizard in real Obsidian.

Any change to the file’s bytes counts as a change, for every note. Each note records the hash of the exact source bytes it came from (_crosswalker.source_ref.source_hash), and that hash is managed content. Editing one cell, or re-saving the file in a program that rewrites it, changes the hash on every note in the set, not only on the notes whose rows changed. In the SP 800-53 proof, editing one cell in the file left one note with changed content and the other 1,188 differing only in the recorded source hash. This is deliberate: a note should never claim to come from bytes it did not come from. It is also why the wizard says whether the picked file matches the last run before anything is written.

Resources

Section titled “Resources”

Distributed systems theory

Section titled “Distributed systems theory”
  • Brewer, E. (2000) — “Towards Robust Distributed Systems” (CAP theorem keynote)
  • CAP Twelve Years Later — Eric Brewer’s retrospective
  • Kleppmann, M. (2017) — “Designing Data-Intensive Applications” (O’Reilly) — chapters 5, 7, 9

Event sourcing

Section titled “Event sourcing”

CouchDB / MVCC

Section titled “CouchDB / MVCC”
Section titled “Related pages”