Skip to content
Crosswalker
🚧 Early alpha — building the foundation. See the roadmap →

Which workbook bytes made this note — the XLSX source snapshot

Created Updated

1. Thirty seconds

Section titled “1. Thirty seconds”

Crosswalker imports compliance frameworks and other structured ontologies (NIST, CIS, MITRE, and any spreadsheet-shaped source) into an Obsidian vault as ordinary Markdown notes, each carrying a record of where it came from. That record is the note’s provenance block.

What changed: when the source is a spreadsheet (.xlsx), Crosswalker now takes one snapshot of the file’s bytes, fingerprints that snapshot, and writes the fingerprint onto every note it produces from it. Everything the import reads afterwards — including a second worksheet pulled in by a declared join — is decoded from that same snapshot, never from a fresh read of the file on disk.

Why it was worth doing: two separate reads of the same file can return different bytes, and the old code re-read the file for joined worksheets. A note could therefore be assembled from two different revisions of a workbook, with nothing recorded to say so.

What it is not: it does not keep a copy of your file, does not detect drift for you later, and does not replay an import. Those remain separate, gated work — see section 6.

2. Two minutes: the two-sheet example

Section titled “2. Two minutes: the two-sheet example”

Say you import controls.xlsx. It has two worksheets:

SheetRole
Controlsthe primary collection — one row per control, the rows that become notes
Ownersa secondary collection, pulled in by a declared source.joins so each control note can name its owner

While that import is running, someone hits save in Excel. Here is the difference.

BeforeNow
Reading the primary sheetread the file, decode Controlsread the file once, keep those bytes privately, decode Controls from a copy of them
Reading the joined sheetread the file again, decode Ownersdecode Owners from another copy of the same kept bytes
If the file changed in betweenthe note mixes revision A’s controls with revision B’s owners, silentlyimpossible — both come from revision A
What the note recordsnothing about which bytes were used_crosswalker.source_ref.source_hash, a fingerprint of revision A’s exact bytes

Two details make the guarantee real rather than nominal:

  • The fingerprint is computed before anything decodes the file. Hashing after a decode step would fingerprint whatever the decoder left behind, not what was read.
  • The decoder never receives the kept bytes themselves, only a disposable copy. A spreadsheet library is free to write into the buffer it is handed; protecting the first read while letting a later read work from a mutated buffer would have looked correct and been wrong.

Note that the file changing on disk is not exotic. Prior real-Obsidian work on this project established that a re-read can return different content even when the file’s reported size and modification time are unchanged — so “check the size first” was never an available shortcut.

The scope of the guarantee, stated precisely. It is per parse, not per wizard session. One parseXLSXFile() call covers its primary sheet and every joined sheet it later serves. Earlier in the wizard, listing a workbook’s sheet names so you can pick one is a separate discovery read that takes no snapshot and computes no fingerprint. So a workbook edited between choosing a sheet and running the import is not something this change detects — what it guarantees is that the rows and joins of a single import, and the fingerprint written beside them, all describe one set of bytes.

3. Why an organization cares

Section titled “3. Why an organization cares”

An audit question is rarely “does this note look right?” It is “which version of the source does this assertion rest on?”

Before this change, the honest answer for a spreadsheet import was: the file named in the note, at some point during the run, probably. Now the note carries a fingerprint that pins the answer to exact bytes. Two consequences follow that a compliance team actually uses:

  • Two notes claiming the same source can be compared. Same fingerprint means the same bytes produced both. Different fingerprints mean different inputs, and that is visible without anyone remembering which export was open that afternoon.
  • A mixed import is no longer possible from a mid-run save. The failure it prevents is the quiet kind: no error, no empty result, just a control row from one revision paired with an owner from another.

The limit worth stating in the same breath: an absent fingerprint means nothing was recorded, not that the source is unchanged. Native runtime imports without a recorded byte digest omit the fingerprint; older notes and output from external producers may already carry one, including notes from the fixture and crosswalk command-line tools. Reading absence as “unchanged” would be exactly the confident-wrong-answer failure mode this project keeps ruling out.

4. What was accepted, rejected and deferred

Section titled “4. What was accepted, rejected and deferred”
Decision
AcceptedFingerprint the complete pre-parse bytes of a native spreadsheet read, once per parse, before any decode
AcceptedEvery worksheet the same parse serves — primary and joined — decodes from that one kept snapshot
AcceptedHand the decoder a defensive copy each time, so a mutating decoder cannot change what a later join sees
AcceptedWrite the value into the existing _crosswalker.source_ref.source_hash field, whose meaning has said “hash of the source file at import time” since it was defined
AcceptedCarry it through every live provenance writer, not just the first one — concept rows via both generation entrances, enrichment, hubs, and facet hubs (source: the buildProvenance() call sites in src/generation/generation-engine.ts)
RejectedAny new schema field. spec/tier1.schema.json is untouched and no schema version moved; this is a runtime producer for a contract that already existed
RejectedInventing a digest for CSV, JSON or SSSOM imports out of decoded text or parsed rows. A fingerprint of decoded output answers a different question than a fingerprint of source bytes, and conflating them would make the field mean two things
RejectedForcing a rewrite to backfill the field onto existing notes. Existing skip, no-op, kept-host and user-owned-frontmatter preservation stay authoritative; this change adds no write solely to populate the new field
RejectedTightening validation on the older permissive source_hash string, which external producers already write against
DeferredEverything the fingerprint enables but does not do: retained source copies, drift diagnosis at replay time, a binding that records parser selectors, and replay execution. All of that sits behind Track 2 of the v0.1.7 milestone and the still-open Gate 2 decisions

5. The memory trade-off, taken deliberately

Section titled “5. The memory trade-off, taken deliberately”

Keeping the bytes costs memory, and the owner’s standing direction settled the trade: correctness and a usable workflow first; performance work is additive and comes later.

The honest shape of the cost:

  • A spreadsheet parse now retains its captured payload — the snapshot’s actual byte length — for as long as the parsed result’s join handle is reachable. Not an assumed file size; the bytes that were actually read.
  • Hashing and each defensive copy add transient allocations on top of that.
  • What is not retained is the decoded workbook. A joined sheet is decoded on demand from the kept bytes and then released, rather than the whole parsed workbook being held open.
  • No measurement of peak memory, corpus scale, or mobile behavior was taken, so no claim about any of them is made here.

One boundary matters more than the number. This is an import-session parsing buffer, not stored source retention. It is not written to disk, not put in settings, not placed in a draft or a cache, and not exposed as an enumerable property on the parsed result. The Gate 1 D4 decision requires explicit opt-in before Crosswalker keeps a copy of anyone’s source file, and that decision drew the same line itself: reading a file to parse it, and the in-memory buffers that requires, is ordinary operation. Nothing here implies consent to a retained copy, and nothing here creates one.

6. What this does NOT do

Section titled “6. What this does NOT do”
  • It does not store your source file, and creates no path by which a copy could be recovered. A fingerprint proves a file has not changed; it cannot bring the file back.
  • It does not diagnose drift. Nothing compares a stored fingerprint against a file you select later — that is Track 2 work.
  • It does not identify decode selectors. Which sheet, which header row, and which filters were applied are not part of the fingerprint; two imports of the same workbook with different settings fingerprint identically.
  • It does not authenticate anything. It is a content digest, not a signature, and placing a digest and rows on one object is not a proof to an arbitrary caller that they belong together.
  • It does not complete replay, add storage, add a recipe or source library, or add UI.
  • It does not close the v0.1.7 milestone or satisfy any Track 2 checkbox, and it does not reopen or alter any Gate 1 answer.
  • It does not change CSV, JSON or SSSOM imports, the external fixture generator (which has had its own raw-byte provenance all along), or the meaning or stored value of any other hash.
Section titled “7. Related”