Multilingual sites: engine, content model and harness lang commands #190
Labels
No labels
bug
enhancement
needs-triage
ready-for-agent
ready-for-human
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
Cloonar/deckle#190
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Make the engine and the
harnessCLI multilingual: a site declares several languages, a translation is a file beside its default-language page (_index.de.md), connected translations follow their source page's element structure, addresses live under/<code>/, andharness lang add/harness lang removemanage the language list. This issue lands the whole content model and API so that the theme issue and the editor issue can build on it; after it, translations are usable by hand and through the API, and the existing editor keeps working for the default language.Current behavior
The page identity on the wire, in the editor URL (
/:module/:lang/*path) and in the stores is already (path, lang), butsiteconfigrequires exactly one language andpagetree.ContentRootmaps it tocontent/itself; any other language is a 404.hugo.tomlhas no[languages]table. Permalinks have no language prefix. Badges, pending changes,publish-allandelement/transferare keyed or scoped by one language. Nothing knows about translations: no translation files, no connected mode, no change tracking. The full status quo is in the PRD below (sections 3 and 4 describe what changes).Desired behavior
Everything in PRD sections 3 (configuration), 4 (content model), 5 (addresses), 6 (links: the engine side — language-neutral values, no rewriting for translation), 7 (operations: the commands, the server endpoints behind Translate page / Translate element / Disconnect / Delete translation, propagation, categories and module entries), 10 (publishing and checks) and 11 (Hugo version), plus the documentation in section 12 that belongs to the engine and CLI. In short:
harness.yamllanguagesbecomes a list of{code, label}entries; the old string-list form keeps loading. The first entry is the default language.harness lang add <code> [--label] [--json]andharness lang remove <code> [--purge] [--json]as specified, including the one-time flip ofdefaultContentLanguageInSubdirwith aliases for every old address, the[languages.<code>]tables with their params block, the language's draft home page, and the Hugo ≥ 0.167 guard.content/<dir>/_index.<code>.md;ContentRootstayscontent/, the file name selects the language; every lang-scoped endpoint resolves a non-default language to the translation files; the tree of a non-default language is the default-language tree with translated titles and a per-node translation state (translated,missing,outdated, plus the existing enabled/badge state) so the editor can draw it.uid; the connected structure invariant is enforced on write and repaired on open; general-tab params and bodies are translatable, appearance-tab params synced;l10nSourcehash per translated element;l10n: connectedfront matter;l10nandl10nSourcereserved.GET /api/siteserves{code, label}entries;GET /api/treegains the translation state per node; page responses carry the translation'sl10nmode and each element'schangedmark and synced params./<code>/once the site has two languages, per-segmentslugoverrides from each ancestor's translation; aliases on slug change and on source rename.langparameter is dropped from publish-all, pending and discard-all; pending rows carrylang; badges are keyed per file.harness checkfindings of PRD section 10.DefaultHugoVersionmoves to the newest stable release ≥ 0.167.setup-site,migrate-site,verify-site,edit-content,add-element,add-page-type,create-featureskills updated as section 12 lists for the engine side (the theme and editor issues update what they change).Decisions
Settled in the grilling session; implement, do not reopen. Reasons are in the PRD next to each.
content/<code>/directory — one tree by construction, no migration of existing sites, categories and modules translate the same way.tabalready separates content from look.l10nSourcehash cleared by save — a save is the review./<code>/for every language, root redirects, but only once a second language exists — single-language sites keep their addresses; the flip writes aliases once.fallbackkey.harness lang add/removeare the only way to change languages; the editor has no settings screen.urlon every translation.Key interfaces
siteconfig.Config.Languages— from[]stringto a list of{Code, Label};validateLanguagesloses the "exactly one" rule, keeps the tag regex, accepts the legacy string list.pagetree.ContentRoot,pagetree.Build,Node— language resolves to file names; a node gains the translation state;Tree.Langstays.pagetree/permalink.go— language prefix and per-segment slug overrides; readsdefaultContentLanguageInSubdir.contentengine: the canonical hash of an element's translatable values;l10nSourceandl10nas reserved names inelement.ReservedParam*and the page-type loader's tool-managed keys.elemops,pageops,servermutations — propagation into connected translations under the existing global mutation lock; the new endpoints;element/transferacross languages of the target page.publish.go,git.go(badges),pendingChangeswire shape — per-file keys, nolangscope.checkcmd— the new findings.langcommands besideaddcmd/removecmd;hugo.tomledits through the existing config-edit machinery used by feature bundles.initcmd.DefaultHugoVersion.internal/siteconfig,internal/pagetree,internal/pageops,internal/elemops,internal/server,internal/checkcmd,internal/addcmd(for the config-edit pattern),cmd/harness/main.go.Acceptance criteria
languages: [en](legacy form) loads and behaves exactly as before: same addresses, same tree, same API answers.harness lang add de --label Deutschon such a site writes both config files as the PRD specifies, flipsdefaultContentLanguageInSubdir, writes the old address of every page into itsaliases, createscontent/_index.de.mdas a draft, andhugobuilds the site with/en/…and/de/addresses and a redirecting root; running it again is a no-op with exit 0 (or exit 1 "already present", documented either way).harness lang addrefuses with exit 1 and nothing written when the effective Hugo version is below 0.167, naming the version.harness lang remove derefuses while any*.de.mdexists, listing them;--purgedeletes them and the config entries; removing the default language exits 1.GET /api/tree?lang=dereturns the default-language tree with translated titles and a translation state per node;GET /api/page?lang=deon a translated page returns the translation; on an untranslated page it returns a documented "missing" answer the editor can render._index.de.mdwithl10n: connected,draft: true, every source element with the same uid, prefixed text, synced appearance params andl10nSource; subpages when asked; free and empty modes as specified.l10nSource; the page response marks a translated element whose hash differs.l10nis treated as free and never written to by propagation./de/leistungen/webdesign/for a translated child under a Services translation withslug: leistungen; changing the slug writes the old address intoaliases; renaming the source writes aliases into translations without a slug.content/categories/<slug>/_index.de.mdtitle; a news item translation listed by the module forlang=de).lang; badges are per file.harness checkreports each finding of PRD section 10 on fixtures that trigger it, and passes on the scaffold site and ontestdata/.harness initscaffolds the newlanguagesshape and the new default Hugo version; AGENTS.md, README.md and the listed skills describe the behaviour;go test ./...passes, including the AGENTS.md and skills drift tests.How to verify
go test ./...with new table tests forsiteconfig,pagetree(tree per language, permalinks), the hash, the invariant repair, propagation, thelangcommands andcheckcmdfindings; fixtures undertestdata/gain a two-language site.harness init, runharness lang add de, create translations through the API withcurl, thenharness checkandhugo(the harness downloads the pinned version): the builtpublic/has/en/,/de/, a redirecting root and the alias pages.Assumptions to check
slugto the pages beneath it (release notes say so; verify on the pinned version with a two-level fixture before relying on it for permalinks).index.htmlroot redirect withdefaultContentLanguageInSubdir = trueis emitted for the default language on the pinned version./services/on a page now served at/en/services/produce a redirect page at/services/(Hugo alias semantics with language subdirs).StampUIDsrepair is per file, so shared uids across translation files are not rewritten (confirm inelemops).deckle-hosted) builds with its own nix-shell Hugo; its pin is not changed here, so a site with two languages may fail to verify there until that repository bumps — note it in the PR.Out of scope
i18n/, the language switch, hreflang,ce/href.html, the render hook (#191).Blocked by
None - can start immediately
PRD: multilingual sites, TYPO3-shaped
Mockup (private, shared on request): https://claude.ai/artifact/6RYbRG7y4yMyeGxYbFsh1n — sources in
docs/mockups/multilingual/(uncommitted at the time of writing; the editor issue commits them). This PRD was settled in a grilling session on 2026-10-10 and is the contract for the three issues that implement it: engine and CLI, theme, editor.1. Goal
A deckle site can have several languages. The editor translates pages and their elements the way TYPO3 does: one page tree, a translation is a record beside the original page, and a translated page either follows the structure of its original (connected) or is a free copy. Every rule below is phrased in terms of the default language, never a specific language: the default language is whatever the site configures first.
2. Terms
languagesinharness.yaml, the same asdefaultContentLanguageinhugo.toml.content/<path>/_index.<code>.md, beside the default-language_index.md.l10n: connected.uidas its source element.3. Configuration
harness.yaml:codeis the existing BCP-47-ish lowercase tag (siteconfig.languageRe),labelthe name shown in the editor and the theme's language switch;labeldefaults to the code. The formerlanguages: [en]string-list form keeps loading as a one-entry list with label = code, so no existing site breaks.hugo.tomlholds the matching Hugo config:defaultContentLanguage= the first code; a[languages.<code>]table per language withlanguageName,weight, and, for every non-default language,titleand a[languages.<code>.params]block (description,address) scaffolded commented-out like the root[params]block.harness lang addwrites these;harness checkreports a mismatch between the two files (language set, order of the default,defaultContentLanguageInSubdir) as an error.defaultContentLanguageInSubdir:falsewhile the site has one language (URLs unchanged:/services/);truefrom the moment a second language exists. Hugo then serves every language under/<code>/and writes a rootindex.htmlthat redirects to the default language. This flip happens exactly once, inharness lang add, which at the same time writes the old unprefixed URL of every page into that page'saliasesso published links keep working as redirects.4. Content model
content/<dir>/_index.<code>.md(orindex.<code>.md/<name>.<code>.mdfor the legacy page shapes the engine still reads). Hugo links it to the default-language file by path. There is nocontent/<code>/directory and notranslationKey.harness checkreports a translation whose source is missing as an error; the editor never creates one.title,draft(Enabled is per language), optionalslug(section 5),aliases,description,menuTitle, andl10n: connectedfor a connected translation.type,weight,backendLayout,backendLayoutNextLevel,hidden,hideChildren,footer,childOrder-relevantdateand every other page-type field are read from the source file; the engine does not write them into translations and the theme reads them from the default-language page.l10njoinstype,draft,weight,aliases,backendLayout,backendLayoutNextLevelas a tool-managed front-matter key (rejected in page-typedefaults, not declarable as a field).uidas its source element. Uid uniqueness is therefore per language: the key of an element is (language, uid).StampUIDsrepair stays per file. Anchors#c-<uid>resolve in every language.tabisgeneral(the default) and the element body are per language — image, link, page and categories params included. Params withtab: appearanceare synced: every save of the source element copies them into each connected translated element, and the translation form shows them read-only. The reserveddisabledparam is per language. An items collection keeps the source structure (one translated item per source item, by uid); the item's own fields are translatable.l10nSource="<hash>": a hash over the canonical form of the source element's translatable values (its general-tab params and body, in sidecar order, after the engine's standard value spelling). Creating a translated element and saving one both write the current hash. On open, a hash that differs from the source's current value marks the element "changed in since translated"; a missing param counts as unknown and shows no mark.l10nSourceis reserved likeuidanddisabled(a sidecar declaring it fails to load;harness sidecarnever emits it). Free translations carry nol10nSourceand no marks._index.de.mdwithoutl10nis a free translation; one withl10n: connectedis repaired on open as described. AGENTS.md "Editing content by hand" documents the file shape.5. Addresses
/<code>/;/redirects to the default language (Hugo's own redirect page).slugwhere one is set, under/<code>/.slugon a translation is optional; empty means "same segment as the source". Hugo ≥ 0.167 applies a section page'sslugto the pages beneath it, which is what makes/de/leistungen/webdesign/follow fromslug: leistungenon the Services translation. The engine's permalink computation (pagetree/permalink.go) mirrors this: language prefix, per-segment slug override from each ancestor's translation.aliases. Renaming the source page (directory rename) keeps every translation's address when the translation has its own slug; translations without one get their old address written into their ownaliases, as the source does today./<code>/; its title is translatable, its address is not.6. Links
/services/means "the Services page in the current language". The theme resolves them at render time: a markdown link render hook (layouts/_default/_markup/render-link.html) and one partial,ce/href.html, that every shortcode uses for alinkparam. Resolution: an internal pretty URL (with optional#c-<uid>fragment or bundle sub-path) is looked up as a page path; if that page has a translation in the current language, the translation's address is emitted; otherwise the default-language page's address (strict: the untranslated page does not exist in this language, so the link goes to the default language). External,mailto:,tel:and media-library links are emitted as stored.pageparams already behave this way:site.GetPagewith the stored content path resolves to the current language's page. They stay as they are./api/link/resolveshow the target in the current language when it exists, else the default-language one with a note.7. Operations
harness lang add <code> [--label <label>] [--json]— adds a language: validates the code and that it is not present; appends it toharness.yamllanguages; writes the[languages.<code>]table and its params block intohugo.toml(and the default language's table if the file had none); on the first non-default language flipsdefaultContentLanguageInSubdirtotrueand writes the old unprefixed address intoaliasesof every page; creates the language's homecontent/_index.<code>.mdas a draft with the site title; refuses (exit 1, nothing written) when the pinned or default Hugo version is below 0.167, naming the version; reports what it changed. Exit codes and--jsonfollow the feature commands' convention.harness lang remove <code> [--purge] [--json]— removes a non-default language: refuses while any*.<code>.mdexists undercontent/unless--purge, listing them; with--purgedeletes those files; removes the language from both config files; when one language remains flipsdefaultContentLanguageInSubdirback tofalseand leaves the aliases in place (they now redirect to the unprefixed address, which is harmless). The default language cannot be removed (exit 1).Translate page (editor, from the page menu of a source page or from an untranslated page in a non-default language): creates the translation. Modes: connected (default) —
l10n: connected, every element of the source copied with the same uid, translatable text params and bodies prefixed with[Translate to <label>:],l10nSourceset, appearance params copied; free — a copy withoutl10nand withoutl10nSource, no prefix; empty — the file with the title only (l10n: connected, no elements, so every source element shows as untranslated). The translation is created withdraft: true. "Also create the N subpages" (default on) applies the same mode to every descendant that has no translation yet. The dialog is in the editor issue; the server endpoint that does the work is in the engine issue.Translate element (connected translation, per untranslated element): creates the translated element from the source with the prefix rule above and opens its form. "Translate all" does it for every untranslated element of the page.
Propagation (connected translations, engine): every structural operation on a source page is applied to each of its connected translations in the same mutation: insert — nothing (the new uid is untranslated); move within the page and across slots — mirrored; reorder — mirrored; delete — the translated element is deleted too, and the confirmation names how many translations are affected; duplicate — the copy's new uid is untranslated everywhere; enable/disable — not propagated; cut to another page (
element/transfer) — each translated element moves with it into that language's translation of the target page, and where the target has no translation in a language the translated element is dropped, which the confirmation names; save of the source element — appearance params copied into each translated element. Insert, move, delete and duplicate are refused on a connected translation (the server answers 409 with the reason; the editor hides the controls). A free translation accepts every element operation like a source page.Disconnect (page menu of a connected translation): removes
l10nand everyl10nSource; the page becomes free. No reconnect in v1.Delete translation (page menu of a translation): deletes only that file. Deleting a source page deletes its translations with it (they cannot exist alone) and the confirmation says so. Moving a source page moves its translations with it (same directory).
Rename (source page): as today, plus the alias rule of section 5. Renaming a translation changes its title; its slug is edited in Properties.
Categories: a category's translation
content/categories/<slug>/_index.<code>.mdgives it a title in that language under the same slug; stored category values stay slugs. Created from the Categories module with the language switch. Renaming or deleting a category applies to all its files.Module entries (news items and any module page type): translated like every page. A module lists the selected language; "New" creates in the default language only; an entry's menu offers "Translate to …". The list's status column shows the translation state in a non-default language.
8. Editor
languagesorder, each with a status dot for the current page (default · translated · changed since translated · unpublished changes · not translated), plus a view-mode toggle Columns / Languages. The language is the existing/:module/:lang/URL segment; switching keeps the page.GET /api/sitealready serves the language list; the client starts reading it.docs/mockups/multilingual/is the visual reference; the editor's own tokens (web/src/styles.css) apply, not the mockup's.9. Theme
<html lang>already followssite.Language.Lang.i18n/<code>.tomlper shipped language (enat least,deas the second reference) for every string the theme and the feature bundles hard-code today: skip link, "Menu", nav and footer aria labels, 404 texts, pager Previous/Next, "All news →", lightbox and consent labels, and the date format. Strings go through Hugo'sTfunction; a missing translation falls back to the default language's string (Hugo's behaviour), never to an empty string..AllTranslations), labelled withlanguageName; hidden on a single-language site.<link rel="alternate" hreflang>for every translation of the page plusx-defaultfor the default language.ce/href.htmland the markdown link render hook of section 6; every shipped shortcode with alinkparam (button, card, img, cta…) and the theme's own links go through it. The theme README's "Authoring a layout" contract documents it.[languages.<code>] titleand[languages.<code>.params] description / address; the footer and meta description readsite.Paramswhich Hugo already merges per language.10. Publishing and checks
/api/publish-all,/api/publish/pendingand/api/publish/discard-alllose theirlangparameter; pending rows carrylang.harness checkadditionally reports: a translation without a source (error); a connected translation whose structure violates the invariant (warning — the server repairs it on open); a[languages]table disagreeing withharness.yaml(error);defaultContentLanguageInSubdirnot matching the language count (error); a shortcode declaringl10nSourceor a page type declaringl10n(load error, as for every reserved name).11. Hugo version
DefaultHugoVersion(initcmd) moves to the newest stable Hugo release at or above 0.167 at implementation time; the pin is exact as today. Existing sites keep theirharness.yamlpin until the developer raises it;harness lang addrefuses to add a second language while the effective version is below 0.167 and says why (section-slug inheritance). The deckle-hosted nix-shell Hugo is a sibling change in that repository and is not part of these issues.12. Documentation and skills (per CLAUDE.md, in the same change as the code)
scaffold/site/AGENTS.md: Layout (translation files,i18n/), theharness langcommands, theharness.yamllanguages shape, the reserved namesl10nandl10nSource, page types (tool-managedl10n), "Editing content by hand" (translation file shape, connected invariant, prefix, hash), the check findings. The drift test (internal/element/agentsdoc_test.go) must keep passing.scaffold/themes/deckle/README.md:ce/href.html, the render hook,i18n/, the language switch, hreflang.setup-site(languages at setup,harness lang add),migrate-site(translations of an existing Hugo multilingual site: only the suffix layout is supported, directory layouts must be converted),verify-site(the new check findings),edit-content(hand-written translations),add-element(the href partial for link params, reserved params),redo-theme(i18n, switch, hreflang, showcase page per language),add-page-type(l10nreserved),create-feature(bundles shipi18n/strings). The skills drift test (scaffold/skills_test.go) must keep passing.13. Out of scope