perf(history): parse each blob once and grep main's tip once per request #309

Merged

dominik.polakovics merged 2 commits from afk/289 into main

2026-10-16 17:49:57 +02:00

dominik.polakovics commented

2026-10-16 17:43:34 +02:00

Owner

Closes #289

What

Two internal speed-ups to GET /api/element/history, with no wire change.

gitrepo (perf(gitrepo): …)
- FileBlob gains ID, the blob's git object id. readBlobs keeps it from the cat-file --batch header it already parsed, so there is no extra git call.
- Every producer carries the id: blobsAt, holdingNeedle, historyEdits, NeedleFiles and DraftEdits.
- NeedleHistory returns a fourth value, start: the kept files holding the needle at the window's start. It is built from the holdingFiles lookup and the batch the function already runs, and equals NeedleFiles(S, …) by construction. A test pins that equality on every page, for windows 0–5 and three Keep filters.
server (perf(history): …)
- locateElement is now a method on a request-scoped elementLocator. It caches each blob's located answer by FileBlob.ID, so a distinct blob is parsed at most once per request.
- Every site goes through it: the version loop, main's tip, editAuthors (merge Edits and draft edits), pendingAuthors (the draft tip) and historyWorkingTree. Working-tree files have no id and are parsed uncached.
- Located shortcodes are shared between sites. None of their readers (DiffElement, ElementTitle, identify, JSON) mutates them.
- A first page takes main's version at the tip from NeedleHistory's start, so it runs one git grep at the tip and one cat-file --batch. Continuation pages still call NeedleFiles(tip, …).

grepHolding/holdingFiles is still the only "files holding the needle at a commit" lookup. elementLocator.locateElement is still the only parse-and-locate. No "same as" comments.

Evidence

Byte-identical responses. A throwaway harness dumped 384 responses from the pre-change tree and replayed them on this branch: diff -r is empty. It covers:
- the perf site: first and continuation pages, mid-window cursors, bad cursors and neighbours;
- WIP and branch mode, merges with edits, pending rows and their authors;
- languages, the 404 and 400 errors, the unavailable cases and mailmap.
New tests:
- TestElementHistoryParsesEachBlobOnce: each blob id is parsed exactly once per request, and the cache must be hit at least once. Runs over version pages, a merge's Edits, DraftEdits and the draft tip.
- TestElementHistoryGrepsTipOnce (counts via GIT_TRACE): a first page runs 1 grep at the tip and 1 batch. A continuation page runs a grep at the tip and one at the window start.
- TestFileBlobIDs, TestFileBlobIDsOfEdits and TestNeedleHistoryStartIsNeedleFiles in gitrepo.
- The existing history tests' assertions are unchanged. Their NeedleHistory call sites only gain a _.
go build ./..., go vet ./..., gofmt -l and go test ./... are all clean.

Before/after, same machine

Alternating runs of base, new, base, new (go 1.26.8, git 2.47.3):

	before (run 1 / 2)	after (run 1 / 2)
`TestElementHistoryCostBound` median	196.0 / 193.6 ms	141.7 / 153.1 ms
`BenchmarkElementHistory` (10x)	193.5 / 192.6 ms/op	144.6 / 151.0 ms/op
`NeedleFiles` on the request path	20.1 / 22.3 ms	not called on a first page
`NeedleHistory`, 200 versions	135.6 / 127.2 ms	118.4 / 130.7 ms
Rest of the median request	37.5 / 41.2 ms	20.6 / 19.7 ms

That is about 45 ms (roughly 23%) off a default-window request, slightly more than the 30–40 ms PR #287 estimated.

Docs and skills

No changes. ADR 0023 quotes only the pickaxe time (about 60 ms) and "well under a quarter of a second", and both are still true. docs/architecture.md, docs/api.md, scaffold/site/AGENTS.md and the scaffolded skills document nothing this change moves.

🤖 Generated with Claude Code

Closes #289 ## What Two internal speed-ups to `GET /api/element/history`, with no wire change. - **gitrepo** (`perf(gitrepo): …`) - `FileBlob` gains `ID`, the blob's git object id. `readBlobs` keeps it from the `cat-file --batch` header it already parsed, so there is no extra git call. - Every producer carries the id: `blobsAt`, `holdingNeedle`, `historyEdits`, `NeedleFiles` and `DraftEdits`. - `NeedleHistory` returns a fourth value, `start`: the kept files holding the needle at the window's start. It is built from the `holdingFiles` lookup and the batch the function already runs, and equals `NeedleFiles(S, …)` by construction. A test pins that equality on every page, for windows 0–5 and three `Keep` filters. - **server** (`perf(history): …`) - `locateElement` is now a method on a request-scoped `elementLocator`. It caches each blob's located answer by `FileBlob.ID`, so a distinct blob is parsed at most once per request. - Every site goes through it: the version loop, main's tip, `editAuthors` (merge `Edits` and draft edits), `pendingAuthors` (the draft tip) and `historyWorkingTree`. Working-tree files have no id and are parsed uncached. - Located shortcodes are shared between sites. None of their readers (`DiffElement`, `ElementTitle`, `identify`, JSON) mutates them. - A first page takes main's version at the tip from NeedleHistory's `start`, so it runs one `git grep` at the tip and one `cat-file --batch`. Continuation pages still call `NeedleFiles(tip, …)`. `grepHolding`/`holdingFiles` is still the only "files holding the needle at a commit" lookup. `elementLocator.locateElement` is still the only parse-and-locate. No "same as" comments. ## Evidence - **Byte-identical responses.** A throwaway harness dumped 384 responses from the pre-change tree and replayed them on this branch: `diff -r` is empty. It covers: - the perf site: first and continuation pages, mid-window cursors, bad cursors and neighbours; - WIP and branch mode, merges with edits, pending rows and their authors; - languages, the 404 and 400 errors, the unavailable cases and mailmap. - **New tests:** - `TestElementHistoryParsesEachBlobOnce`: each blob id is parsed exactly once per request, and the cache must be hit at least once. Runs over version pages, a merge's `Edits`, `DraftEdits` and the draft tip. - `TestElementHistoryGrepsTipOnce` (counts via `GIT_TRACE`): a first page runs 1 grep at the tip and 1 batch. A continuation page runs a grep at the tip and one at the window start. - `TestFileBlobIDs`, `TestFileBlobIDsOfEdits` and `TestNeedleHistoryStartIsNeedleFiles` in gitrepo. - The existing history tests' assertions are unchanged. Their `NeedleHistory` call sites only gain a `_`. - `go build ./...`, `go vet ./...`, `gofmt -l` and `go test ./...` are all clean. ## Before/after, same machine Alternating runs of base, new, base, new (go 1.26.8, git 2.47.3): | | before (run 1 / 2) | after (run 1 / 2) | | --- | --- | --- | | `TestElementHistoryCostBound` median | 196.0 / 193.6 ms | 141.7 / 153.1 ms | | `BenchmarkElementHistory` (10x) | 193.5 / 192.6 ms/op | 144.6 / 151.0 ms/op | | `NeedleFiles` on the request path | 20.1 / 22.3 ms | not called on a first page | | `NeedleHistory`, 200 versions | 135.6 / 127.2 ms | 118.4 / 130.7 ms | | Rest of the median request | 37.5 / 41.2 ms | 20.6 / 19.7 ms | That is about 45 ms (roughly 23%) off a default-window request, slightly more than the 30–40 ms PR #287 estimated. ## Docs and skills No changes. ADR 0023 quotes only the pickaxe time (about 60 ms) and "well under a quarter of a second", and both are still true. `docs/architecture.md`, `docs/api.md`, `scaffold/site/AGENTS.md` and the scaffolded skills document nothing this change moves. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

dominik.polakovics added 2 commits

2026-10-16 17:43:34 +02:00

perf(gitrepo): carry blob ids and answer the window start's holding files 4b8b1dcbb5

FileBlob gains the blob's git object id, kept from the cat-file --batch
header readBlobs already parsed. NeedleHistory additionally returns the
kept files holding the needle at the window's start, exactly
NeedleFiles' answer there, from the grep and batch it already runs.

Refs #289

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

perf(history): parse each blob once and grep main's tip once per request

ci / build-and-test (pull_request) Successful in 3m25s

Details

ci / release (pull_request) Has been skipped

Details

742be863cc

handleElementHistory locates the element through one request-scoped
elementLocator that remembers, per git blob id, what locating the
element in it answered, so a blob shared by neighbouring versions, the
tip, a merge's edits or the draft is parsed once. A first page takes
main's version at the tip from NeedleHistory's window start instead of a
second NeedleFiles grep; continuation pages still call NeedleFiles.

Closes #289

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

dominik.polakovics commented

2026-10-16 17:48:08 +02:00

Author

Owner

[autoland] verdict: pass

PASS — signal: labctl pr checks 309 = success (ci / build-and-test green; release skipped). Title is Conventional Commits, body carries Closes #289, diff stays within the issue (internal/gitrepo + internal/server history only, no wire change). Rule owners kept: elementLocator.locateElement is the sole parse-and-locate; holdingFiles/grepHolding the sole holding-files grep; NeedleHistory's start = blobsAt(blobs, at, holdingFiles(at)), the same path NeedleFiles takes, pinned by TestNeedleHistoryStartIsNeedleFiles. Shared located shortcodes are only read (DiffElement/sourceText build new nodes). Existing tests changed only at NeedleHistory call sites. gofmt -l clean. Docs/skills untouched, correctly: no documented behaviour moves.

[autoland] verdict: pass PASS — signal: labctl pr checks 309 = success (ci / build-and-test green; release skipped). Title is Conventional Commits, body carries Closes #289, diff stays within the issue (internal/gitrepo + internal/server history only, no wire change). Rule owners kept: elementLocator.locateElement is the sole parse-and-locate; holdingFiles/grepHolding the sole holding-files grep; NeedleHistory's start = blobsAt(blobs, at, holdingFiles(at)), the same path NeedleFiles takes, pinned by TestNeedleHistoryStartIsNeedleFiles. Shared located shortcodes are only read (DiffElement/sourceText build new nodes). Existing tests changed only at NeedleHistory call sites. gofmt -l clean. Docs/skills untouched, correctly: no documented behaviour moves.

dominik.polakovics merged commit b37acb486d into main

2026-10-16 17:49:57 +02:00

dominik.polakovics deleted branch afk/289

2026-10-16 17:49:57 +02:00

dominik.polakovics referenced this pull request from a commit

2026-10-16 17:49:57 +02:00

Merge pull request 'perf(history): parse each blob once and grep main's tip once per request' (#309) from afk/289 into main