feat(history): GET /api/element/history — the publish history of one content element, derived from main #283

New issue

Closed

opened 2026-10-14 13:01:01 +02:00 by dominik.polakovics · 0 comments

dominik.polakovics commented

2026-10-14 13:01:01 +02:00

Owner

Summary

Add a read-only, TYPO3-style publish history for one content element: a new GET /api/element/history that lists every publish to main that changed the element's own content, each with a field-level diff labelled by the element's form, plus a top row for the element's unpublished working-tree change. The history is derived from git on demand; nothing is written at save or publish time. This is the server half; the editor's sheet and menu entries are a separate issue that depends on this one. The visual reference is docs/mockups/element-history/ (issue #282; artifact https://claude.ai/artifact/22NW7Pjqc1QUSUEJCxv1qT, private to the owner).

Current behavior

Every mutation snapshots the whole tree (refs/deckle/wip in WIP mode, a draft-branch commit in branch mode; internal/gitrepo, ADR 0003 and 0011). Undo is single-step and page-scoped. There is no way to see what earlier publishes changed in an element.
main's first-parent chain is one commit per publish (ADR 0005, 0013). In branch mode the draft branch is rebased and collapsed after a publish, so only main holds durable history. main is read through the main comparison ref (Repo.mainRef(): refs/heads/main in WIP mode, the remote-tracking main in branch mode).
The pending panel (internal/server/pending.go) already compares a page's elements on main with the working tree: collectElements, elementFingerprint (name, tag form, params minus the uid, body text, folded item/text children; nested role-element children excluded) and diffElements (match by uid, uid-less elements by fingerprint) produce the added/changed/removed counts of a pendingChange. That is the only owner of "what changed in an element" today, and it yields counts, not fields.
Every editable shortcode carries a server-generated 12-character uid in the reserved uid param (internal/elemops/uid.go), stable across moves, reorders and transfers (a transfer re-stamps only when the destination already holds the uid). The engine's one spelling of a string param is uid="<uid>" (content.ParamValueString); links to an element use the different form #c-<uid> (internal/linkrefs). A connected translation keeps the source's uids in its own file beside the page (index.de.md), ADR 0016.
Forms are server-described (ADR 0015, POST /api/form): every field has a label and a widget name; web/src/widgets/names.json pins the widget vocabulary.

Desired behavior

The route

GET /api/element/history?uid=<uid>&lang=<code>&before=<commit> answers the publish history of the element with that uid in that language's file (the default language when lang is absent).

before is the cursor from a previous answer: history continues from the publish before it. Absent: from main's tip.
Each request scans at most 1,000 first-parent commits of main and answers next (the commit to pass as before) when the chain continues beyond the window, absent when it is exhausted.
The response carries, in this order:
- the element's identity: uid, element name, the form's label for the element, the page it lives on now and its language (when the working tree holds it);
- pending: one entry for the working tree against main, present only when the element differs there (added, changed or removed), computed with the same comparison as the pending panel;
- entries: newest first, one per publish in the window in which the element's own content changed, or in which it first appeared or disappeared;
- next.
An entry carries the publish commit id, its committer time (RFC 3339), the commit author name as git has it, a kind of created (first publish carrying the uid in this language's file, or the uid returning after a publish without it), changed or removed, and fields: the field-level diff against the element's previous version (for created, every field against nothing; for removed, every field against nothing the other way).
A field change names the field (param name, body, the items collection name, or enabled), its form label, its widget, and its old and new values in the widget's shape. For text-like widgets (text, multiline, the body, rich text) it additionally carries a word-level diff as a sequence of equal/deleted/inserted runs, so the client renders inline without diffing. An items collection reports its rows matched by uid as kept, added, removed or changed, a changed row with its own field changes. enabled reports the reserved disabled state as a boolean field labelled by the Enabled field's switch settings.
A move between pages, a page rename or move, or a bundle conversion is followed so the history continues across files, but produces no entry of its own; a publish that moved and edited the element is a plain changed entry. A pure reorder within the page produces no entry (the fingerprint ignores position). Nested role-element children are not part of their parent's history; they have their own.
Degradation follows the pending panel's: with no repository or no main, the route answers 200 with empty entries, no pending, and unavailable with its coded form. Errors: 400 request.missingParam {name: "uid"}, 404 editor.unknownLang, and a 404 when neither the working tree nor the scanned window holds the uid in that language. New codes follow internal/apimsg (one owner per code, exported to web/src/i18n/servermessages.json; the editor's parity test needs the en.json/de.json entries in the same change).
The route is read-only and never takes the mutation lock; it reads everything at the main tip resolved once at the start of the request, so a publish landing mid-request cannot mix two states.

The reader

Residency. Which files held the element in which publishes comes from git's pickaxe over main's first-parent chain, restricted to the content root, with the fixed string uid="<uid>", listing the files per commit (git log --first-parent -S'uid="<uid>"' --name-only <range> -- content/). Each hit is the element entering or leaving a file; without --pickaxe-all git lists only the files where the count changed. The window is a commit range (<tip>~1000..<tip> or the cursor), never -n, which does not bound the scan. Use -S with a fixed string, not -G: twice as fast and the spelling is canonical.
Language. The uid appears in every connected translation's file; the file's language tag (the path rule in internal/pagepath) selects the requested language's file.
Versions. Within each residency segment, the publishes that changed that file are listed, each version read in one batched git cat-file --batch, parsed with internal/content, the element located by uid with internal/elemops, and consecutive versions whose fingerprint is equal collapsed to one.
Cost. Measured on a synthetic 300-page site with 15 elements per page and 2,000 publishes touching 3 pages each: pickaxe over the whole chain 110 ms (63 ms for a 1,000-commit window), per-file log 15 ms, batched read 5 ms, parse of one version 0.08 ms. One request stays well under a quarter of a second at that size; the window keeps the cost bounded as the site ages.

Decisions

History is derived from main, never a second store — the snapshots and main already hold every version; a journal would be a second evaluation of "what changed" (CLAUDE.md, one owner per rule) and would be blind to hand edits and host reconciles. Record this as a new ADR.
Publish granularity, main's first-parent chain only — in branch mode per-save history is rewritten and collapsed at every publish, so it cannot be a durable history; main is the one chain both modes share and never rewrite.
Read on demand, bounded window with a cursor — zero cost on save and publish; 1,000 publishes per request measured at about 60 ms; a count-based bound would not bound the scan.
The pending panel's comparison becomes the one owner of the element diff, moved to internal/elemops and extended from counts to fields; pendingChange.elements is computed from the new owner's result. Adding a sibling comparison would be the #230 class of defect.
The server computes the diff and labels it; the client only renders — the client and the theme never re-derive a rule (CLAUDE.md); labels and widgets come from the same form description the editor already shows.
Moves are followed, never listed — a later restore can only restore content, not a location, so a move row would promise what cannot be undone.
A pending top row — the editor sees what the next Publish will change next to what earlier publishes did, from the comparison the pending panel already makes.
Author as git has it — honest today (" editors" in hosted; a developer's own name for a hand commit), and it becomes the signed-in person once the host forwards identity, with no API change.
Per uid and language — undo, discard and the pending panel treat a translation as its own file; the history does the same.
Entries are created, changed, removed — a reorder is silent; enable/disable is the enabled field inside a changed entry.

Key interfaces

internal/elemops — the new owner of the element comparison: a function that, given an element's previous and current shortcode (either may be nil) and the schemas, returns the structured field diff described above, and the fingerprint the pending panel uses today. internal/server/pending.go's collectElements, elementFingerprint and diffElements call or move into it; the counts stay what they are.
internal/gitrepo — the residency scan and the version reads over the main comparison ref (mainRef()), pinned to one resolved tip; a new file beside log.go/show.go, documented in the package comment like the other operations. Site-relative paths in and out, SitePrefix applied internally as everywhere in the package.
internal/server — the handler, registered in the route table and gated like GET /api/publish/pending (quiet degradation, 200 with unavailable); the form description's labels and widgets for the element's schema (forms.go); the word-level diff for text-like widgets.
internal/apimsg — the new codes.
Docs: docs/api.md gets a ### GET /api/element/history heading (the route test in internal/server enforces registered routes ⇔ headings); docs/architecture.md "Git backbone" gets a paragraph linking it; a new docs/adr/0023-… records the derivation decision and is listed in docs/adr/README.md; docs/agents/docs-map.md if the area is new there. No change to scaffold/site/AGENTS.md, the theme README or any skill: no sidecar, grammar or procedure changes.
Starting hints: internal/server/pending.go (comparison), internal/server/git.go (how git-backed routes degrade), internal/gitrepo/log.go and show.go (reading commits and blobs), internal/gitrepo/helpers_test.go (building repositories in tests), internal/server/forms.go (labels and widgets).

Acceptance criteria

GET /api/element/history is registered, documented in docs/api.md with request, response and every code, and the route and docs drift tests pass
On a page with a published element edited and published three times, the route answers three changed entries newest first, each with the right fields, old and new values and labels, and a created entry for the first publish
An element transferred to another page keeps one continuous history across the move; the move itself produces no entry; a move plus edit in one publish is one changed entry
A page rename or move does not break or add to the element's history
A publish that only reordered elements produces no entry; a disable/enable produces a changed entry whose only field is enabled
Inserting, moving or removing a nested element produces no entry on the parent
With unpublished edits, pending is present with the working-tree diff; without, it is absent; the pending panel's counts are unchanged for every existing pending test
A connected translation's history (lang=de) lists the publishes that changed index.de.md's element and none that changed only the source
A 1,000-publish window answers next when older history exists; passing it as before continues without gaps or duplicates; the last window answers no next
The same element's history read in branch mode (remote-tracking main) equals the WIP-mode reading of the same main
No repository or no main: 200 with unavailable and its code; missing uid: 400; unknown language: 404; unknown uid: 404
The route never takes the mutation lock and performs no write to the repository or the working tree
The comparison lives in internal/elemops only; internal/server/pending.go holds no fingerprint or matching of its own
ADR added and listed; docs/architecture.md links it; web/src/i18n/servermessages.json, en.json and de.json carry the new codes and the i18n parity test passes
On a repository the size of the measured one (300 pages, 2,000 publishes), one request with the default window completes in under 250 ms on the test machine

How to verify

go test ./... and cd web && npm test (the parity test needs the catalog entries; go test needs web/dist, see the Makefile)
A gitrepo test that builds a repository with a scripted publish history (edits, a transfer, a rename, a reorder, a translation) with the package's test helpers and checks the entries against the script
A server test over testdata/site with a scripted main, asserting the wire shape of each entry kind, the pending row, the cursor and the degradations
A benchmark or a timed test on a generated repository of the measured size, so the cost bound is observed rather than assumed

Assumptions to check

That git's pickaxe without --pickaxe-all lists only the files whose match count changed, on the git version the hosted image ships; the measurement used git 2.47
That content.ParamValueString is the only spelling a uid ever takes in a stored file, including hand-written sites that harness check accepts
That the form description (forms.go) can be built for an element's schema without a page context, for a removed element whose page no longer holds it
That a transfer re-stamps the uid only when the destination already holds it; if it re-stamps more often, the history breaks at a move and the docs must say so

Out of scope

Restoring an earlier version (a later increment: a forward save through POST /api/page/save)
The editor sheet and the menu entries (the editor issue, blocked by this one)
A page-level history of every element, or of page properties
Showing moves as entries
Forwarding the signed-in user's identity from the host into the git author
Caching across requests, retention or pruning of history
Changing how publish, snapshots, undo or the draft branch work

Blocked by

None - can start immediately

## Summary Add a read-only, TYPO3-style publish history for one content element: a new `GET /api/element/history` that lists every publish to `main` that changed the element's own content, each with a field-level diff labelled by the element's form, plus a top row for the element's unpublished working-tree change. The history is derived from git on demand; nothing is written at save or publish time. This is the server half; the editor's sheet and menu entries are a separate issue that depends on this one. The visual reference is `docs/mockups/element-history/` (issue #282; artifact https://claude.ai/artifact/22NW7Pjqc1QUSUEJCxv1qT, private to the owner). ## Current behavior - Every mutation snapshots the whole tree (`refs/deckle/wip` in WIP mode, a draft-branch commit in branch mode; `internal/gitrepo`, ADR 0003 and 0011). Undo is single-step and page-scoped. There is no way to see what earlier publishes changed in an element. - `main`'s first-parent chain is one commit per publish (ADR 0005, 0013). In branch mode the draft branch is rebased and collapsed after a publish, so only `main` holds durable history. `main` is read through the main comparison ref (`Repo.mainRef()`: `refs/heads/main` in WIP mode, the remote-tracking main in branch mode). - The pending panel (`internal/server/pending.go`) already compares a page's elements on `main` with the working tree: `collectElements`, `elementFingerprint` (name, tag form, params minus the uid, body text, folded item/text children; nested role-`element` children excluded) and `diffElements` (match by uid, uid-less elements by fingerprint) produce the added/changed/removed counts of a `pendingChange`. That is the only owner of "what changed in an element" today, and it yields counts, not fields. - Every editable shortcode carries a server-generated 12-character uid in the reserved `uid` param (`internal/elemops/uid.go`), stable across moves, reorders and transfers (a transfer re-stamps only when the destination already holds the uid). The engine's one spelling of a string param is `uid="<uid>"` (`content.ParamValueString`); links to an element use the different form `#c-<uid>` (`internal/linkrefs`). A connected translation keeps the source's uids in its own file beside the page (`index.de.md`), ADR 0016. - Forms are server-described (ADR 0015, `POST /api/form`): every field has a label and a widget name; `web/src/widgets/names.json` pins the widget vocabulary. ## Desired behavior ### The route `GET /api/element/history?uid=<uid>&lang=<code>&before=<commit>` answers the publish history of the element with that uid in that language's file (the default language when `lang` is absent). - `before` is the cursor from a previous answer: history continues from the publish before it. Absent: from `main`'s tip. - Each request scans at most 1,000 first-parent commits of `main` and answers `next` (the commit to pass as `before`) when the chain continues beyond the window, absent when it is exhausted. - The response carries, in this order: - the element's identity: uid, element name, the form's label for the element, the page it lives on now and its language (when the working tree holds it); - `pending`: one entry for the working tree against `main`, present only when the element differs there (added, changed or removed), computed with the same comparison as the pending panel; - `entries`: newest first, one per publish in the window in which the element's own content changed, or in which it first appeared or disappeared; - `next`. - An entry carries the publish commit id, its committer time (RFC 3339), the commit author name as git has it, a `kind` of `created` (first publish carrying the uid in this language's file, or the uid returning after a publish without it), `changed` or `removed`, and `fields`: the field-level diff against the element's previous version (for `created`, every field against nothing; for `removed`, every field against nothing the other way). - A field change names the field (param name, `body`, the items collection name, or `enabled`), its form label, its widget, and its old and new values in the widget's shape. For text-like widgets (`text`, `multiline`, the body, rich text) it additionally carries a word-level diff as a sequence of equal/deleted/inserted runs, so the client renders inline without diffing. An items collection reports its rows matched by uid as kept, added, removed or changed, a changed row with its own field changes. `enabled` reports the reserved disabled state as a boolean field labelled by the Enabled field's `switch` settings. - A move between pages, a page rename or move, or a bundle conversion is followed so the history continues across files, but produces no entry of its own; a publish that moved and edited the element is a plain `changed` entry. A pure reorder within the page produces no entry (the fingerprint ignores position). Nested role-`element` children are not part of their parent's history; they have their own. - Degradation follows the pending panel's: with no repository or no `main`, the route answers `200` with empty `entries`, no `pending`, and `unavailable` with its coded form. Errors: `400` `request.missingParam {name: "uid"}`, `404` `editor.unknownLang`, and a `404` when neither the working tree nor the scanned window holds the uid in that language. New codes follow `internal/apimsg` (one owner per code, exported to `web/src/i18n/servermessages.json`; the editor's parity test needs the `en.json`/`de.json` entries in the same change). - The route is read-only and never takes the mutation lock; it reads everything at the `main` tip resolved once at the start of the request, so a publish landing mid-request cannot mix two states. ### The reader - **Residency.** Which files held the element in which publishes comes from git's pickaxe over `main`'s first-parent chain, restricted to the content root, with the fixed string `uid="<uid>"`, listing the files per commit (`git log --first-parent -S'uid="<uid>"' --name-only <range> -- content/`). Each hit is the element entering or leaving a file; without `--pickaxe-all` git lists only the files where the count changed. The window is a commit range (`<tip>~1000..<tip>` or the cursor), never `-n`, which does not bound the scan. Use `-S` with a fixed string, not `-G`: twice as fast and the spelling is canonical. - **Language.** The uid appears in every connected translation's file; the file's language tag (the path rule in `internal/pagepath`) selects the requested language's file. - **Versions.** Within each residency segment, the publishes that changed that file are listed, each version read in one batched `git cat-file --batch`, parsed with `internal/content`, the element located by uid with `internal/elemops`, and consecutive versions whose fingerprint is equal collapsed to one. - **Cost.** Measured on a synthetic 300-page site with 15 elements per page and 2,000 publishes touching 3 pages each: pickaxe over the whole chain 110 ms (63 ms for a 1,000-commit window), per-file log 15 ms, batched read 5 ms, parse of one version 0.08 ms. One request stays well under a quarter of a second at that size; the window keeps the cost bounded as the site ages. ## Decisions - **History is derived from `main`, never a second store** — the snapshots and `main` already hold every version; a journal would be a second evaluation of "what changed" (CLAUDE.md, one owner per rule) and would be blind to hand edits and host reconciles. Record this as a new ADR. - **Publish granularity, `main`'s first-parent chain only** — in branch mode per-save history is rewritten and collapsed at every publish, so it cannot be a durable history; `main` is the one chain both modes share and never rewrite. - **Read on demand, bounded window with a cursor** — zero cost on save and publish; 1,000 publishes per request measured at about 60 ms; a count-based bound would not bound the scan. - **The pending panel's comparison becomes the one owner of the element diff**, moved to `internal/elemops` and extended from counts to fields; `pendingChange.elements` is computed from the new owner's result. Adding a sibling comparison would be the #230 class of defect. - **The server computes the diff and labels it; the client only renders** — the client and the theme never re-derive a rule (CLAUDE.md); labels and widgets come from the same form description the editor already shows. - **Moves are followed, never listed** — a later restore can only restore content, not a location, so a move row would promise what cannot be undone. - **A pending top row** — the editor sees what the next Publish will change next to what earlier publishes did, from the comparison the pending panel already makes. - **Author as git has it** — honest today ("<site> editors" in hosted; a developer's own name for a hand commit), and it becomes the signed-in person once the host forwards identity, with no API change. - **Per uid and language** — undo, discard and the pending panel treat a translation as its own file; the history does the same. - **Entries are `created`, `changed`, `removed`** — a reorder is silent; enable/disable is the `enabled` field inside a `changed` entry. ## Key interfaces - `internal/elemops` — the new owner of the element comparison: a function that, given an element's previous and current shortcode (either may be nil) and the schemas, returns the structured field diff described above, and the fingerprint the pending panel uses today. `internal/server/pending.go`'s `collectElements`, `elementFingerprint` and `diffElements` call or move into it; the counts stay what they are. - `internal/gitrepo` — the residency scan and the version reads over the main comparison ref (`mainRef()`), pinned to one resolved tip; a new file beside `log.go`/`show.go`, documented in the package comment like the other operations. Site-relative paths in and out, `SitePrefix` applied internally as everywhere in the package. - `internal/server` — the handler, registered in the route table and gated like `GET /api/publish/pending` (quiet degradation, `200` with `unavailable`); the form description's labels and widgets for the element's schema (`forms.go`); the word-level diff for text-like widgets. - `internal/apimsg` — the new codes. - Docs: `docs/api.md` gets a `### GET /api/element/history` heading (the route test in `internal/server` enforces registered routes ⇔ headings); `docs/architecture.md` "Git backbone" gets a paragraph linking it; a new `docs/adr/0023-…` records the derivation decision and is listed in `docs/adr/README.md`; `docs/agents/docs-map.md` if the area is new there. No change to `scaffold/site/AGENTS.md`, the theme README or any skill: no sidecar, grammar or procedure changes. - Starting hints: `internal/server/pending.go` (comparison), `internal/server/git.go` (how git-backed routes degrade), `internal/gitrepo/log.go` and `show.go` (reading commits and blobs), `internal/gitrepo/helpers_test.go` (building repositories in tests), `internal/server/forms.go` (labels and widgets). ## Acceptance criteria - [ ] `GET /api/element/history` is registered, documented in `docs/api.md` with request, response and every code, and the route and docs drift tests pass - [ ] On a page with a published element edited and published three times, the route answers three `changed` entries newest first, each with the right fields, old and new values and labels, and a `created` entry for the first publish - [ ] An element transferred to another page keeps one continuous history across the move; the move itself produces no entry; a move plus edit in one publish is one `changed` entry - [ ] A page rename or move does not break or add to the element's history - [ ] A publish that only reordered elements produces no entry; a disable/enable produces a `changed` entry whose only field is `enabled` - [ ] Inserting, moving or removing a nested element produces no entry on the parent - [ ] With unpublished edits, `pending` is present with the working-tree diff; without, it is absent; the pending panel's counts are unchanged for every existing pending test - [ ] A connected translation's history (`lang=de`) lists the publishes that changed `index.de.md`'s element and none that changed only the source - [ ] A 1,000-publish window answers `next` when older history exists; passing it as `before` continues without gaps or duplicates; the last window answers no `next` - [ ] The same element's history read in branch mode (remote-tracking main) equals the WIP-mode reading of the same `main` - [ ] No repository or no `main`: `200` with `unavailable` and its code; missing `uid`: `400`; unknown language: `404`; unknown uid: `404` - [ ] The route never takes the mutation lock and performs no write to the repository or the working tree - [ ] The comparison lives in `internal/elemops` only; `internal/server/pending.go` holds no fingerprint or matching of its own - [ ] ADR added and listed; `docs/architecture.md` links it; `web/src/i18n/servermessages.json`, `en.json` and `de.json` carry the new codes and the i18n parity test passes - [ ] On a repository the size of the measured one (300 pages, 2,000 publishes), one request with the default window completes in under 250 ms on the test machine ## How to verify - `go test ./...` and `cd web && npm test` (the parity test needs the catalog entries; `go test` needs `web/dist`, see the Makefile) - A `gitrepo` test that builds a repository with a scripted publish history (edits, a transfer, a rename, a reorder, a translation) with the package's test helpers and checks the entries against the script - A server test over `testdata/site` with a scripted `main`, asserting the wire shape of each entry kind, the pending row, the cursor and the degradations - A benchmark or a timed test on a generated repository of the measured size, so the cost bound is observed rather than assumed ## Assumptions to check - That git's pickaxe without `--pickaxe-all` lists only the files whose match count changed, on the git version the hosted image ships; the measurement used git 2.47 - That `content.ParamValueString` is the only spelling a uid ever takes in a stored file, including hand-written sites that `harness check` accepts - That the form description (`forms.go`) can be built for an element's schema without a page context, for a `removed` element whose page no longer holds it - That a transfer re-stamps the uid only when the destination already holds it; if it re-stamps more often, the history breaks at a move and the docs must say so ## Out of scope - Restoring an earlier version (a later increment: a forward save through `POST /api/page/save`) - The editor sheet and the menu entries (the editor issue, blocked by this one) - A page-level history of every element, or of page properties - Showing moves as entries - Forwarding the signed-in user's identity from the host into the git author - Caching across requests, retention or pruning of history - Changing how publish, snapshots, undo or the draft branch work ## Blocked by None - can start immediately

dominik.polakovics added the

enhancement

ready-for-agent

labels

2026-10-14 13:01:01 +02:00

dominik.polakovics referenced this issue

2026-10-14 13:01:53 +02:00

feat(editor): the element History sheet and its two menu entries #284

dominik.polakovics referenced this issue

2026-10-14 13:02:07 +02:00

docs(mockups): commit the element history mockup #285

dominik.polakovics referenced this issue from a commit