perf+persistence: the durable reconcile mirror (cache + disk) #47

christian · 2026-07-09T03:19:04Z

christian commented

2026-07-09 03:19:04 +00:00

The mirror layer — cached reads and durable persistence. Two commits:

1. In-memory reconcile cache

A full reconcile is ~2N gitea calls (deps + timelines per issue); every agent tool call was doing a fresh one. Now one shared snapshot cache backs the UI reconcile and the agent: tool calls reuse a snapshot within a 30s TTL; the explicit UI reconcile forces fresh; writes invalidate so the board reflects them immediately.

2. Persist to disk — instant boot + offline reads

snapshot-store.ts: the snapshot is written to app userData as JSON on every successful reconcile (never throws; corrupt/absent → "no cache").
Instant boot (gitea:boot): the app shows the persisted snapshot immediately, then a fresh reconcile supersedes it (stale-while-revalidate) — no blank board while ~2N calls run.
Offline (gitea:reconcile fallback): if gitea is unreachable, reads serve the persisted snapshot (stale: true) instead of erroring.
useBacklog does boot → show → reconcile → replace; a reconcile error keeps the shown snapshot.

Why JSON, not SQLite (yet)

At this scale (~34 issues, a 37KB snapshot) the whole thing fits in memory and the scheduler/forecast operate on arrays — indexed SQL buys nothing yet, and it avoids a native better-sqlite3 + electron-rebuild (Electron 34 ships Node 20, no node:sqlite) or a WASM dependency. Real SQL tables become worthwhile only when the mirror needs indexed queries over larger data; that's the clean next step from here.

Verified

Desktop typecheck clean · 14 fixture e2e green · live Reginald answers correctly from the cache.

Persistence live-verified: the snapshot persists (34 issues / 44 deps / 34 timelines / 5 milestones to disk); a second launch with gitea pointed at an unreachable host renders the full real board — NOW/NEXT/LATER + the Monte Carlo cone — entirely from the cache (new live-persistence e2e, screenshot attached in the run).

🤖 Generated with Claude Code

The mirror layer — cached reads *and* durable persistence. Two commits: ## 1. In-memory reconcile cache A full reconcile is **~2N gitea calls** (deps + timelines per issue); every agent tool call was doing a fresh one. Now one shared snapshot cache backs the UI reconcile and the agent: tool calls reuse a snapshot within a **30s TTL**; the explicit UI reconcile forces fresh; **writes invalidate** so the board reflects them immediately. ## 2. Persist to disk — instant boot + offline reads - `snapshot-store.ts`: the snapshot is written to app userData as JSON on every successful reconcile (never throws; corrupt/absent → "no cache"). - **Instant boot** (`gitea:boot`): the app shows the persisted snapshot immediately, then a fresh reconcile supersedes it (stale-while-revalidate) — no blank board while ~2N calls run. - **Offline** (`gitea:reconcile` fallback): if gitea is unreachable, reads serve the persisted snapshot (`stale: true`) instead of erroring. - `useBacklog` does boot → show → reconcile → replace; a reconcile error keeps the shown snapshot. ## Why JSON, not SQLite (yet) At this scale (~34 issues, a 37KB snapshot) the whole thing fits in memory and the scheduler/forecast operate on arrays — indexed SQL buys **nothing** yet, and it avoids a native `better-sqlite3` + electron-rebuild (Electron 34 ships Node 20, no `node:sqlite`) or a WASM dependency. Real SQL tables become worthwhile only when the mirror needs indexed queries over larger data; that's the clean next step from here. ## Verified Desktop typecheck clean · 14 fixture e2e green · live Reginald answers correctly from the cache. **Persistence live-verified**: the snapshot persists (34 issues / 44 deps / 34 timelines / 5 milestones to disk); a second launch with gitea pointed at an unreachable host renders the **full real board — NOW/NEXT/LATER + the Monte Carlo cone — entirely from the cache** (new `live-persistence` e2e, screenshot attached in the run). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

christian added 1 commit 2026-07-09 03:19:04 +00:00

perf: main-process reconcile cache — stop refetching the repo on every tool call fca36f9075

A full reconcile is ~2N gitea calls (deps + timelines per issue). Every agent
tool call (query_project) was doing a fresh one; the UI reconcile and the agent
didn't share anything. Now a single in-memory snapshot cache backs both.

- gitea.ts: getSnapshot(client, { maxAgeMs }) — reads within the window reuse the
  cache; maxAgeMs:0 forces fresh. invalidateSnapshot() drops it. The explicit UI
  reconcile forces fresh (and warms the cache); agent tool calls tolerate a 30s
  TTL to stay responsive; applyChange + createIssues invalidate so the board and
  forecast reflect the write immediately.
- model.ts: query_project reads getSnapshot (30s TTL) instead of reconciling live.

This is the SQLite mirror's cache semantics in memory — rebuildable, the durable
truth stays in gitea (purity split, D4). Persistent SQLite (offline + instant
boot) is a separate slice: Electron 34's Node 20 has no node:sqlite, so it needs
better-sqlite3 + electron-rebuild or sql.js/WASM — deferred as its own decision.

Verified: desktop typecheck clean, 14 fixture e2e green, live Reginald still
answers correctly from the cache (writes invalidate → board stays correct).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

christian added 1 commit 2026-07-09 03:43:46 +00:00

feat: persist the reconcile cache to disk — instant boot + offline reads 2f6636684e

Extends the in-memory cache into a durable mirror. The reconcile snapshot is
written to disk on every successful reconcile; on boot the app shows it instantly
(stale-while-revalidate) instead of a blank board, and if gitea is unreachable,
reads fall back to it (offline). Rebuildable — the durable truth stays in gitea.

- snapshot-store.ts: load/save the snapshot as JSON in app userData (never throws;
  corrupt/absent → "no cache"). At this scale (~34 issues, 37KB) the whole snapshot
  fits in memory, so a JSON file beats indexed SQL — no query benefit yet, no
  native-module (better-sqlite3/electron-rebuild) or WASM dependency. That's the
  next step if the mirror ever needs indexed queries over larger data.
- gitea.ts: getSnapshot persists on a fresh pull; bootSnapshot() returns the
  persisted snapshot (without seeding the cache — agents still reconcile fresh);
  gitea:boot serves it; gitea:reconcile falls back to it on failure (stale:true).
- useBacklog: stale-while-revalidate — boot instantly, then a fresh reconcile
  supersedes; a reconcile error keeps the shown snapshot instead of erroring.

Verified: desktop typecheck clean, 14 fixture e2e green. Live: the snapshot
persists (34 issues / 44 deps / 34 timelines / 5 milestones written to disk); a
second launch with gitea unreachable renders the full real board — NOW/NEXT/LATER
+ the Monte Carlo cone — entirely from the cache (new live-persistence e2e).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

christian changed title from ~~perf: main-process reconcile cache (the mirror's cache layer)~~ to perf+persistence: the durable reconcile mirror (cache + disk)

2026-07-09 03:44:07 +00:00

christian merged commit 403aef9d16 into main

2026-07-09 03:53:29 +00:00

christian referenced this issue from a commit

2026-07-09 03:53:30 +00:00

Merge pull request 'perf+persistence: the durable reconcile mirror (cache + disk)' (#47) from infra/reconcile-cache into main

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: christian/commitea#47