Skip to content
Crosswalker
🚧 Early alpha — building the foundation. See the roadmap →

The search index never persisted: sqlite moves into a worker

Created Updated

1. The finding

Section titled “1. The finding”
RunHostResult
OpenClast plugin probe, 2026-10-03Real Obsidian 1.12.7 bundle in a Chromium tab over an OPFS shimPlugin loads with no shim gap; sidecar logs “OPFS unavailable; falling back to in-memory sidecar Error: Missing required OPFS APIs”; vfs: unix-none
tests/e2e/tier2-vfs-probe.spec.ts, 2026-10-03Real desktop Obsidian (Electron) via wdioIdentical: vfs: unix-none, filename: "", createSyncAccessHandle absent on the main thread

The opfs-sahpool storage mode needs FileSystemFileHandle.prototype.createSyncAccessHandle, which Chromium exposes only inside dedicated workers. The sidecar installed it on the main thread, so the fallback to :memory: ran on every host, every launch, and nothing surfaced it (the flag was module-private). The CHANGELOG entry of 2026-09-13 already said persistence “was not observed in either run”; this is the first run that observed it, and the answer was no.

2. Rulings

Section titled “2. Rulings”
#RulingWhy
W1One database seam (Tier2Db: exec, execBatch, close, info) with two implementations: a dedicated-worker host (default) and the main-thread wrapper (tests and fallback). Projector, queries, evidence coverage, migrations and the two views take the seam, never the raw sqlite objectEvery SQL call already went through one exec shape and every caller already awaited the open; the worker is the only place the storage API exists
W2The worker is built at runtime from text inlined in main.js (a Blob URL module worker); the sqlite module text and wasm bytes are posted to it in one init message. The plugin still ships only main.js, manifest.json, styles.cssOpenClast’s in-tab installer accepts only those files (plus data.json); its plugin store caps 10 MiB unpacked, and the production bundle is about 4.9 MiB (source: bun run build output on 2026-10-03, main.js 5,114,861 bytes)
W3The storage pool lives in a plugin-private, per-vault OPFS folder; never the SharedArrayBuffer-based opfs modeAvoids contending with the host’s own OPFS handles; no host in scope serves COOP/COEP
W4Statements inside a loop travel as one batch per message, executed as one SAVEPOINT in the worker; projection flushes once per yield window; each closure cache replace and each mapping table is one batchThousands of message round trips would turn a fast loop into a slow one; a SAVEPOINT split across messages risks a partial commit
W5The fallback stays and becomes visible: Settings shows “Search index: stored on this device” or “Search index: in memory, rebuilt each time Obsidian starts” (and “starts the first time a search needs it” before first open); the copied troubleshooting details carry the same factCache lag is not absence applies to storage mode too: a silent mode is how this went unnoticed for five months
W6Close and delete are ordered through one message queue. The worker stays alive across a close and is terminated at unload (500 ms grace); terminating on every close made the next open race the old worker’s file handlesFound in implementation; the ordering precondition from the clear/reset command is preserved
W7The 2026-05-04 commitment “run the engine on the main thread; do not depend on Web Workers” is narrowed, not reversed: it governs the import engine, which needs the Obsidian API and the DOM. A pure sqlite host touches neitherHonest reconciliation; see §3
W8Measure projection before and after on the SP 800-53 set and the local CRI workbook (aggregates only); regression cap 1.5x; report the later-launch savingResource discipline: measure, then cap, record the number
RejectedThe SharedArrayBuffer-based opfs mode (needs COOP/COEP no host in scope serves); terminating the worker on every close (the next open raced the old worker’s file handles); a loose status line outside the setting cardRecorded here so they are findable
W9Diagnostics report browser on a host that is neither Electron nor the mobile app; clipboard actions fall back to pre-selected text when the clipboard is blockedFound in the browser-native audit; both matter under OpenClast

3. Narrowing the Ch 23 commitment

Section titled “3. Narrowing the Ch 23 commitment”

Bundle engine language synthesis §9.5 adopted “run engine on main thread with cooperative yielding; don’t depend on Web Workers (currently unreliable in Obsidian)”. That finding was about the import engine, which reads the vault through Obsidian’s API and renders previews. It still holds for that workload. The query index is a different workload: no Obsidian API, no DOM, one wasm module and a storage pool. The worker requirement here is not a preference but the only way the storage API exists. src/tier2/sidecar.ts cites this narrowing in its header.

4. What the OpenClast collaboration established

Section titled “4. What the OpenClast collaboration established”

The owner’s direction on 2026-10-03 was that Crosswalker must run inside OpenClast within about a month and that the two sessions collaborate as a shared context channel. What the channel produced today:

FactWhere it is recorded
Crosswalker runs unmodified under browser-native: 33 commands and 2 views register, the import wizard renders, sqlite loads through a Blob URL, no CSP, require or SharedArrayBuffer errorsOpenClast experimental/browser-native/tools/plugin-compat/class-map.json entry crosswalker, verdict LOADS
Its one open Problem in OpenClast’s class model, “OPFS sync access handles used from the main thread”, is author-side; Solution owner = the plugin (this change). Hosting precondition “Blob-URL dedicated Workers allowed, no CSP worker-src restriction” is satisfiedOpenClast docs-site/.../zz-research/2026-09-15-plugin-api-class-map.md, section “crosswalker row re-classified”
Store caps: 10 MiB unpacked, 16 MiB uploadOpenClast orchestrator/src/routing/browser-native-plugin-store.ts:135,139
The split both sides hold: unmodified under browser-native, or a Crosswalker-side native assumption to remove; shim gaps go to OpenClast as reproductions, never as plugin-specific shimsOpenClast design principle 10 and DECISION-065

The repo gains bun run e2e:openclast, and the local portagenty workspace (gitignored) an openclast-smoke session, so the host check is one command rather than a hand-assembled run.

5. What this does not do

Section titled “5. What this does not do”
  • Does not reopen the sidecar automatically if the worker crashes mid-session; a reset or reload is needed. Follow-up.
  • Does not skip the startup projection when nothing changed; persistence makes that possible, it does not do it.
  • Does not fix two pre-existing red e2e specs found while running the converted suites: one sssom-import assertion (subject_note undefined) and three first-run cases, all failing identically on main. They need their own ruling.
  • Does not keep the index across a vault rename (the per-vault folder key includes the vault name); it rebuilds once from notes.
  • Does not batch the schema migration script (its leading PRAGMA is ignored inside a transaction).
  • Does not change the sqlite-vec deferral; the 2026-11-06 revisit stands, now with the browser-native constraint added.

6. What this unblocks

Section titled “6. What this unblocks”
  • The index is usable the moment Obsidian opens: the stored index answers immediately and only the routine startup refresh runs, where before an empty in-memory index forced a second rebuild (numbers in the CHANGELOG). The startup projection pass itself still runs every launch; skipping it when no note changed is the next step and sits with the open closure-invalidation question on the v0.1.5 page.
  • The OpenClast row can flip from “Worker not yet exercised” to exercised once the host rerun is green.
  • The TeachingStack consumer’s read-only queries now hit a persisted index rather than a rebuilt one.
Section titled “7. Related”