feat: shortcode round-trip engine — .md ⇄ element tree with canonical serialization #19

Merged

dominik.polakovics merged 2 commits from afk/2 into main

2026-07-12 02:02:58 +02:00

dominik.polakovics commented

2026-07-11 15:47:12 +02:00

Owner

Closes #2

What this does

Adds internal/content, a standalone package (no deps on server/supervisor) that parses Hugo content files into a typed element tree and serializes the tree back to one canonical, human-readable form — plus a debug endpoint exposing the parsed tree as JSON.

Design decisions

Lexer: Hugo's own parser/pageparser (v0.161.1, matching the CI/hugo binary), no hand-rolled tokenization. go mod tidy'd to the minimal require set.
Pairing is syntactic. Hugo decides block-vs-inline by inspecting the shortcode template's .Inner usage; this engine has no templates, so a close tag pops to the nearest matching open (which becomes a block), unmatched opens are inline, and canonical inline form is the explicit self-close {{< button label="x" />}} — unambiguous to both this parser and Hugo regardless of template.
Canonical layout is flat: tags on their own lines at column 0. Indenting nested tags/bodies was rejected deliberately — ≥4 spaces of indentation turns rendered shortcode output into markdown code blocks.
Field model: flat named scalar params (string|bool|int64|float64, Hugo ValTyped semantics), sorted by name, unique; positional params are a parse error. One body per element.
Never silently mangle: Serialize errors on anything that would not survive Hugo's lexer — newlines in param values (naming element and param), unescaped {{< in text, unrepresentable backslash/backtick combos, front-matter fence hazards, shortcodes nested in .inline bodies. Comment-escaped shortcodes ({{</* … */>}}) round-trip verbatim.

Round-trip guarantees (all tested)

serialize(parse(x)) byte-identical for canonical x — 13-file corpus incl. every planned v1 shape (grid>column nesting, accordion items, hero with inline buttons mid-text)
parse(serialize(tree)) structurally identical — 720-tree seeded property test + Go native fuzz target (FuzzParseRoundTrip, 185s/13.7M execs clean; the generator caught one real bug — the .inline nesting guard — before it shipped)
Non-canonical hand-written input re-emits canonically — 9 input/golden pairs (sloppy spacing, unsorted params, unclosed blocks, mixed delims, …)

Debug surface

GET /api/debug/tree?page=posts/first.md → {"page":…,"tree":…} with kind-discriminated nodes; 400/404/405/422 (parse errors include the line) with JSON error bodies; path-traversal guarded via filepath.IsLocal.

Verification

go build ./..., go vet ./..., go test ./... (content, server, supervisor) and the frontend vitest suite all green locally; CI runs the same steps.

🤖 Generated with Claude Code

https://claude.ai/code/session_012RD9ScCJbiaeV1uvbkcFfs

Closes #2 ## What this does Adds `internal/content`, a standalone package (no deps on server/supervisor) that parses Hugo content files into a typed element tree and serializes the tree back to one canonical, human-readable form — plus a debug endpoint exposing the parsed tree as JSON. ## Design decisions - **Lexer**: Hugo's own `parser/pageparser` (v0.161.1, matching the CI/hugo binary), no hand-rolled tokenization. `go mod tidy`'d to the minimal require set. - **Pairing is syntactic.** Hugo decides block-vs-inline by inspecting the shortcode template's `.Inner` usage; this engine has no templates, so a close tag pops to the nearest matching open (which becomes a block), unmatched opens are inline, and canonical inline form is the explicit self-close `{{< button label="x" />}}` — unambiguous to both this parser and Hugo regardless of template. - **Canonical layout is flat**: tags on their own lines at column 0. Indenting nested tags/bodies was rejected deliberately — ≥4 spaces of indentation turns rendered shortcode output into markdown code blocks. - **Field model**: flat named scalar params (`string|bool|int64|float64`, Hugo `ValTyped` semantics), sorted by name, unique; positional params are a parse error. One body per element. - **Never silently mangle**: Serialize errors on anything that would not survive Hugo's lexer — newlines in param values (naming element and param), unescaped `{{<` in text, unrepresentable backslash/backtick combos, front-matter fence hazards, shortcodes nested in `.inline` bodies. Comment-escaped shortcodes (`{{</* … */>}}`) round-trip verbatim. ## Round-trip guarantees (all tested) - `serialize(parse(x))` byte-identical for canonical `x` — 13-file corpus incl. every planned v1 shape (grid>column nesting, accordion items, hero with inline buttons mid-text) - `parse(serialize(tree))` structurally identical — 720-tree seeded property test + Go native fuzz target (`FuzzParseRoundTrip`, 185s/13.7M execs clean; the generator caught one real bug — the `.inline` nesting guard — before it shipped) - Non-canonical hand-written input re-emits canonically — 9 input/golden pairs (sloppy spacing, unsorted params, unclosed blocks, mixed delims, …) ## Debug surface `GET /api/debug/tree?page=posts/first.md` → `{"page":…,"tree":…}` with `kind`-discriminated nodes; 400/404/405/422 (parse errors include the line) with JSON error bodies; path-traversal guarded via `filepath.IsLocal`. ## Verification `go build ./...`, `go vet ./...`, `go test ./...` (content, server, supervisor) and the frontend vitest suite all green locally; CI runs the same steps. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_012RD9ScCJbiaeV1uvbkcFfs

dominik.polakovics added 2 commits

2026-07-11 15:47:12 +02:00

feat(content): add shortcode round-trip engine (.md ⇄ element tree) b3083610a6

Parse Hugo content files (front matter + body) into a typed element tree
using Hugo's vendored parser/pageparser as the lexer, and serialize the
tree back to a single canonical, human-readable form.

- Typed nodes: front matter (opaque, format-preserving), block shortcode,
  inline (self-closing) shortcode, text run; JSON marshaling with kind
  discriminators for the debug surface.
- Pairing resolved syntactically (Hugo consults templates, we cannot):
  a close tag pops to the nearest matching open; unmatched opens are
  inline; explicit '/>' self-close is canonical for inline elements.
- Flat named scalar params (string|bool|int64|float64, Hugo ValTyped
  semantics), sorted by name; positional params rejected at parse.
- Canonical layout: tags on their own lines at column 0 (indentation
  would trip markdown's 4-space code-block rule), single-space tags,
  double-quoted strings with backtick-raw fallback, bare scalars.
- serialize(parse(x)) is byte-identical for canonical x (corpus tests);
  parse(serialize(tree)) is structurally identical for valid trees
  (720-tree seeded property test + native fuzz target, 185s+ clean).
- Serialize errors instead of silently mangling: newlines in param
  values, unescaped '{{<' in text, unrepresentable backslash/backtick
  combos, front-matter fence hazards, nested shortcodes in '.inline'
  bodies.
- Corpus covers the planned v1 element shapes: grid>column nesting,
  accordion items, hero with inline buttons mid-text, plus front matter
  formats, markup shortcodes, comment-escaped shortcodes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012RD9ScCJbiaeV1uvbkcFfs

feat(server): add /api/debug/tree returning a page's parsed element tree

ci / build-and-test (pull_request) Successful in 2m31s

Details

b6d1b5ef77

GET /api/debug/tree?page=<content-relative-path> parses the page with
internal/content and returns {page, tree} as JSON. Guards: GET only
(405), required page param (400), filepath.IsLocal traversal check
(400), missing file (404), parse failure (422 with line info). The
handler now takes the site's content directory at construction.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012RD9ScCJbiaeV1uvbkcFfs

dominik.polakovics referenced this pull request

2026-07-12 02:01:39 +02:00

Shortcode round-trip engine: .md ⇄ element tree with canonical serialization #2

dominik.polakovics merged commit dba0c9100a into main

2026-07-12 02:02:58 +02:00

dominik.polakovics referenced this pull request from a commit

2026-07-12 02:02:58 +02:00

Merge pull request 'feat: shortcode round-trip engine — .md ⇄ element tree with canonical serialization' (#19) from afk/2 into main