feat(l10n): multilingual sites — content model, translation API and harness lang #193
No reviewers
Labels
No labels
bug
enhancement
needs-triage
ready-for-agent
ready-for-human
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
Cloonar/deckle!193
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "afk/190"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Closes #190
What
The engine and CLI side of multilingual sites:
harness.yamllanguagesis a list of{code, label}entries; the first entry is the default. The legacylanguages: [en]form still loads.harness initwrites the new shape and pins Hugo 0.167.0.content/<dir>/_index.<code>.md(alsoindex.<code>.md/<name>.<code>.md) beside its source; there is nocontent/<code>/directory. As in Hugo, only a configured non-default language makes a file a translation:node.js.md,jquery.min.mdorfaq.en.mdstay ordinary pages.l10n: connected) keep the source's uids in the same structure. The newinternal/l10npackage enforces this on write and repairs it on open.l10nSourceholds a hash of the source's translatable values and drives the "changed since translated" marks./<code>/, with per-segmentslugoverrides from each ancestor's translation. Tree nodes get a newaddressfield;permalinkstays the language-neutral link value.GET /api/pageanswersmissing: truefor an untranslated page./api/tree?lang=returns translated titles plus atranslationstate per node (translated / missing / outdated).langparameter of publish-all, pending and discard-all is ignored. Pending rows carrylang, and badges and undo/discard work per file.harness lang add|remove.removerefuses while translations exist unless--purge; the default language cannot be removed.lang addfor an existing language is a no-op with exit 0 (statusalready-present).harness check. Reports a translation without a source (error), a connected translation whose structure has drifted or that has elements without a uid (warnings), a page named with the default language's code on a multilingual site (warning), andhugo.tomllanguages, default-language or subdir settings that disagree withharness.yaml(errors).scaffold/site/AGENTS.md: new sections Languages,harness lang, Addresses, Translations, Language checks, and Translations by hand.harness lang, Multilingual sites, and the wire shapes.Where this differs from the PRD (checked on Hugo 0.167.0)
[languages.<code>] label, notlanguageName.languageNameis deprecated since Hugo 0.158 and prints a warning on every build. The theme issue (#191) should read.Language.Label./x/is emitted at/en/x/. So the engine writes aliases without a language prefix, and the one-time flip writes/../services/, which Hugo path-cleans to/services/. I confirmed in the build output that the old unprefixed URLs redirect. Home is never aliased: Hugo's own root redirect covers/, and an alias/is a build error.urlfront matter is not language-prefixed by Hugo, so the address uses it as written.pt-BR) are published lowercase (/pt-br/), and the address follows that.Known limits and follow-ups
type,weight, layouts and page-type fields per file, while translations deliberately hold only title, slug, aliases, description, menuTitle, draft and l10n. The theme has to read those from the default-language page, as the PRD says.teaserare therefore not translatable, which is the PRD's decision./de/categories/) are built by Hugo even without a translation.harness serveafterharness lang add/remove; it readsharness.yamlonly at start. This is documented.Review fixes (second round)
pagepath.SplitLangtakes the site's languages, and every caller uses it, so a one-language site behaves as before.move-refstakes an optionaltoParentUid. A language whose target translation lacks the destination parent is listed asdropped, for a same-page move into a container too.harness checkwarns about a connected translation with elements that have no uid, because opening the page would delete them.disablePathToLoweris set. A sectionslugapplies on one-language sites too. The flip keeps an existingAliaseskey.l10nSource.hugo.binaryfirst, ashugobin.Resolvedoes.address, the checks).For #191 (theme): the language switch should use
.Language.Label, notlanguageName. ReadingbackendLayout,weight,hiddenandfooterfrom the default-language page (PRD §4) has to be in that issue's scope.Verification
go build ./...,go vet ./...and gofmt are clean, andgo test ./...passes. That includes new table tests for siteconfig, pagepath, pagetree (trees and addresses per language), l10n (hash, repair, translate), langcmd, checkcmd and the server endpoints, plus the AGENTS.md and skills drift tests.cd web && npm testpasses (2418 tests);web/is unchanged.languages: [en]site. The only differences are the treeaddressfield, the{code, label}entries of/api/site, and the new element delete-plan endpoint.node.js.md,asp.net.md,jquery.min.md,release.new.md,faq.en.mdandnotes.de.mdare pages in the tree and open through/api/page. Rename and the flip alias them, they get pending rows of their own, andharness checkis clean. Tests cover each of these.addressmatches the build:/about-us/forcontent/About Us/,/services/web/under a section withslug: services, and/tools/node.js/. Afterharness lang add de, every old URL is an alias redirect, a hand-writtenAliases:key included.harness init, thenharness lang add de --label Deutsch, translations made through the API with curl, thenharness check(clean) andhugo0.167.0.public/has/en/,/de/and/de/leistungen/<child>/.index.htmlredirects to/en/, and the old unprefixed addresses are alias redirect pages.harness lang removeworks in all three paths: blocked,--purge, and the default language refused.🤖 Generated with Claude Code
A site declares several languages in harness.yaml (`{code, label}` list, the legacy string list still loads; the first entry is the default). A translation is a file beside its default-language page (`_index.de.md`); there is no per-language content directory. - siteconfig: Languages become {Code, Label}; the "exactly one" rule goes. - pagepath: SplitLang / TranslationPath; every default-language view (tree, pageops, pending, backend layouts, removecmd) ignores translation files, and plain pages carry their configured translations on move/rename/delete/convert. - pagetree: per-language trees (translated titles, enabled, translation state translated/missing/outdated, l10n mode) and a new `address` field: `/<code>/` prefix once two languages exist, per-segment slug overrides from each ancestor's translation, root-relative `url` verbatim. - l10n (new): connected-translation invariant and Repair, the l10nSource hash over translatable values, synced appearance params, translate element/document with the "[Translate to <label>:] " prefix, disconnect. `l10nSource` and `l10n` are reserved names. - server: lang-scoped endpoints resolve to translation files; repair on open; change marks; source saves sync appearance params and propagate structure; insert/move/delete/duplicate/transfer propagate into connected translations and answer 409 on them; new endpoints translate page (connected/free/empty, subpages), translate-plan, translate element/all, disconnect, element delete-plan; delete translation, rename/enabled/properties per translation with alias rules; categories, module entries and link resolve per language; site-wide publish, pending rows with lang, per-file badges, per-file undo/discard. - harness lang add/remove: both config files, the one-time defaultContentLanguageInSubdir flip with `/../<old>/` aliases (Hugo resolves aliases relative to the language root), the draft home translation, Hugo >= 0.167 guard, --purge. - harness check: translation without source, unconfigured language suffix, connected structure drift, hugo.toml languages / default / subdir mismatches; `translations` count. - initcmd: DefaultHugoVersion 0.167.0, new harness.yaml shape. - Docs: AGENTS.md (Languages, Addresses, Translations, Language checks, Translations by hand), README (harness lang, Multilingual sites, wire shapes) and the setup-site, migrate-site, verify-site, edit-content, add-element, add-page-type, create-feature skills. Closes #190 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>[autoland] verdict: reject
Lander verdict: FAIL
What I relied on
ci / build-and-testpassed (go vet,go test,go build,npm testwith Hugo 0.167.0). I did not re-run it.Conventions are fine
Closes #190is present.Blocking: acceptance criterion 1 is broken. That criterion is "a site with
languages: [en]… behaves exactly as before: same addresses, same tree, same API answers".B1. Page files with a dotted name stop being pages, even on a one-language site
pagepath.SplitLang(internal/pagepath/pagepath.go:60) treats any<stem>.<2–3 letters>.mdas a translation and never checks the configured languages. Hugo only reads a configured language from a file name. Withlanguages: [en], Hugo 0.167 builds/tools/node.js/,/jquery.min/and/notes.faq/as normal pages. Five of the seven review passes reproduced this independently, againstorigin/main:internal/pagetree/build.go:198dropsnode.js.md,asp.net.md,jquery.min.md,release.new.mdandfaq.en.mdfromGET /api/tree.GET /api/page?path=tools/node.js.mdnow answers 400 ("…is "tools/node.md" in "js"").server.go:639/701(refuseTranslationPath) refuses it, and so doespageops/paths.go:186/209./tools/node.js/alias.relocate.go:624(affectedPages) builds its list from the tree and no longer sees the file, so the published URL 404s.harness check: a site that passed on main now has 2 errors ("language "js" is not configured", "translation without a source"). The cause ischeckcmd/check.go:417.harness lang addwrites no alias for such pages (langcmd/content.go:72), so/jquery.min/404s after the flip.pending.go:299-304lists{path:"release.md", lang:"new"}, and the editor routes that row to an unknown-language 404.git.go:190/739(badges/discard) andpublish.go:473key these files as translations.The PR is also inconsistent with itself.
pageops.pageTranslationsanddelete.go:277already filter bylangs[lang]. Fix: classify a file as a translation only when its tag is a configured non-default language, in every caller listed above. The check's "unconfigured language suffix" finding can still look at the name alone if you want it to. Add a one-language test with dotted file names. Then update theAGENTS.md"Translations" paragraph (around line 904) that currently documents the limitation.Concerns to fix in the same run (each reproduced unless marked)
internal/server/translation_ops.go:363-398(transferPlanFor), cutting into a container that is untranslated in the target's connected translation still lists the language undermoved. ThenreceiveTranslated/l10n.Place(:267) drops the translated text. The doc comment admits it, but the confirmation the acceptance criteria rely on misreports the loss. A same-page move into an untranslated container also deletes the translated element (through Repair), and no plan or count mentions it.internal/l10n/hash.go:91-97,132-147, an absent param hashes asname=\x00. The form writes a default back, such aswide=falsefor an optional bool, so a save that changed only an appearance param flips the hash. Adding any optional general-tab field to a sidecar does the same: it flips the hash of every existing element of that type, site-wide. Fix: resolve a value the way the form does (stored, else default) and leave out params that have no value.internal/server/translation_pages.go:110snapshots onlyres.Page. Undo deletesposts/_index.de.mdbut leavesposts/first/index.de.mdandposts/second/index.de.md.harness checksays nothing.server/translation.go:193-206(openPage) stamps fresh uids on a connected translation's uid-less elements. Repair then deletes them as strays.checkcmd/check.go:471runs Repair without stamping first, so it reports "ok: no problems found".verify-siteandedit-contentlead an agent to read that as safe.langcmd/content.go:166,pagetree.RelativeAddress).content/About Us/_index.mdgets/../About Us/while Hugo published/about-us/, so the old URL 404s after the flip. Fix: apply Hugo's urlize and lowercasing, unlessdisablePathToLoweris set.Aliases:key gets a second, lowercase key (langcmd/content.go:177). Hugo keeps usingAliases, so the flip's aliases are lost.translation_settings.go:110/245checks only the format of a translation slug. Two sibling de translations can both claim/de/leistungen/hosting/.translation.go:120-125: deleting a slugged section translation moves its descendants' addresses and writes no aliases.pageops/translation.go:239/302: Translate never strips an alias that its new address shadows, which contradicts the package doc ("every URL-assigning operation strips…")..mdresource in a leaf bundle cannot be discarded per page.pending.go:293countsposts/first/notes.de.mdas a resource of the en page, butgit.go:190leaves it out of the en file set. Discarding the en page answers 200 and changes nothing.l10n/structure.go:301,targetList).InsertTranslatedfails with "cannot be translated before its parent", although the parent is translated.TranslateAlladds 0, and Repair and Place drop children silently.AGENTS.md:990-992andREADME.md:2557-2560say undo/discard of a source page never touches its translations.README.md:940-956andserver/git.go:650-666,808-822say and do otherwise.README.md:976-977says one-language answers are "exactly what they always were", but every tree node now carriesaddress.Node.Addressignores a section'sslug(pagetree/build.go:437). Hugo 0.167 serves/svc/web/whenservices/_index.mdhasslug: svc, but the tree reports/services/web/. The flip'sRelativeAddressuses the slug, so the two disagree about "the old address".Minor:
linktargets.go:286-305) contradicts PRD §6, which resolves by page.l10nSourcein a free target (element_transfer.go:260).hugo.versionbeforehugo.binary(langcmd.go:380), the reverse ofhugobin.Resolve.For the human, outside this PR:
languageName. This PR writeslabelinstead, so #191 needs a note.backendLayout,weight,hiddenandfooterfrom the default-language page is in PRD §4, but #191's own scope does not list it. Until that lands, a published two-language site renders translations with the wrong layout and menus.[autoland] verdict: fix-done
[autoland] verdict: pass
Lander verdict: CONCERNS (non-blocking), so approved but not merged
What I relied on
ci / build-and-testond1dd324passed in 12m39s (go vet,go test,go build,npm test, with Hugo 0.167.0). I did not re-run it. A localgo vet ./...andgo test ./...also pass.origin/main, and built sites with real Hugo 0.167.0 before and afterharness lang add.Conventions are fine
Closes #190is present.backendlayoutchange ind1dd324only reverts 4e74536's change: its net diff is empty, and every caller passes the source path.The previous rejection is resolved
pagepath.SplitLang(rel, TranslationLangs(...)).SplitLangTagis used only where the language is already known to be configured.languages: [en]site withnode.js.md,faq.en.md,jquery.min.md, a slugged section, paths with spaces andurl:pages, base and head were run through the same scenarios. The tree, page, refs, link resolve, delete-plan, pending, badges, discard, rename and delete all gave the same results, and the resulting file bytes were identical. The only differences are the new treeaddressfield and the{code,label}entries in/api/sitethat the issue specifies.node.js.mdandfaq.en.mdstay pages andx.de.mdis a translation, consistently across the tree, API, pending, badges, discard, rename, delete, check, the flip and purge.Concerns (worth a look; none is a blocker. Most can be follow-ups)
internal/pageops/translation.go:318,translateStripsat:361). The undo snapshot names onlyres.Created(internal/server/translation_pages.go:115), so undo does not restore those files.posts/first/index.de.mdcarries the alias/posts/second/, and translatingposts/secondstrips it. After undo,posts/second/index.de.mdis gone and so is the alias.translationAddressTaken(internal/pageops/translation_settings.go:292-318) compares only against siblings that already have a translation in that language. Grouping folders are skipped (s.Path == "",:314). All of these were reproduced, and Hugo 0.167 silently builds one page over the other:Translate(translation.go:270) has no collision check.PathRules.Segment(internal/pagetree/hugopath.go:30) sanitizes but skips Hugo'sNormalizePathStringBasic, which first lowercases the segment and turns every single space into-.a b.mdandC++ & Go!.mdare published at/a--b/and/c++--go/. The flip aliases/a-b/and/c++-go/, and the old URLs 404 afterlang add. This was confirmed with Hugo 0.167.0.removePathAccents, and a per-language[languages.<code>] disablePathToLower(ReadPathRulesreads only the root key).slugversus stored links under the new Hugo pin.slugto its subtree:/dienste/web/, confirmed against 0.163.3, which publishes/services/web/.pageURL,internal/server/relocate.go:442,505). Link resolve matches only that form (linktargets.go:258). The comment atinternal/pagetree/permalink.go:21-23still says Hugo ignores a section slug.toParentUid(1). Without it (translation_ops.go:386-407), a cut into a container that a connected translation lacks still lists that language asmoved. The current editor (web/src/api/links.ts:136-150) sends notoParentUidand ignorestranslations. #192 has to send it.internal/l10n/hash.go:178)./api/form/inlinewrites defaults back, so pressing OK on a chip dialog without changing anything still marks the owner's translations "changed".scaffold/site/.claude/skills/verify-site/SKILL.md:30-32says to fix a language-config error by redoing it withharness lang add/remove.harness.yaml,lang addreturnsalready-presentand changes nothing.hugo.tomldisagrees,lang addrefuses.AGENTS.md:822itself says to fix it by hand first.Minor doc and comment drift (no action required)
README.md:999andAGENTS.md:912: Hugo buildsfaq.en.mdat/faq/even on a one-language site, as the default-language file. The "exactly as Hugo builds them" claim does not hold for a default-language tag.internal/pagepath/pagepath.go:89says onlyharness checkusesSplitLangTag; there are five other callers.internal/server/publish.go:379describes a branch that can no longer be reached, at:473.AGENTS.md:86-88says the scaffold'sdefaultContentLanguageInSubdirappears only once there is a second language; the scaffold writes= falsefrom the start.README.md:993-995says one-language answers gain onlyaddress./api/sitelanguageschanged shape too.add-page-type/SKILL.md:36-40lists what a translation owns but leaves outdraft,aliasesandl10n.edit-content/SKILL.md:174gives only the_index.<code>.mdform.For the human: each concern is narrow and none breaks the one-language guarantee or the core translation flows, so I did not reject. I approved instead of merging so that someone looks at concerns 1–3 first: fix them before the merge or file them as follow-ups. Concerns 4–5 are for #191 and #192.
Third round: no engine-written aliases, remaining concerns resolved (
e4d8138)Per the product decision recorded in #194, commit
e4d8138removes every alias the engine wrote, then fixes what was left of the second verdict's concerns.Aliases are gone. Rename, move, a translation's slug change, the deletion of a slugged section translation, Translate and the
harness lang addflip no longer record old addresses, and the shadowing guard that stripped entries from other pages is removed with them. A hand-writtenaliaseskey is left byte for byte (tests cover rename, Translate and the flip).harness lang addtouches no content; its result and--jsonlosealiases. This dissolves concern 1 (undo of stripped aliases) and the rename half of concern 4.Concern 3, addresses.
PathRulesnow hasFileandSlug: a file-path segment is lowercased with each single space turned into a hyphen before sanitizing (a b.md→/a--b/,C++ & Go!.md→/c++--go/), unlessdisablePathToLower; a slug is sanitized and lowercased;removePathAccentsis honoured; both keys are read per[languages.<code>]table. Every expectation in the tests was checked against a Hugo 0.167.0 build.Concern 2, collisions. The slug check now treats every sibling as reserving its address in the language: a translated sibling its slug, an untranslated one its source segment, a grouping directory its name. Translate refuses a page whose segment a sibling's translation slug already took (a subpage in that state is skipped).
harness checkreports two pages at one address in a language as an error; verified end to end against a Hugo build that collapsed the two pages.Concern 7. The verify-site skill tells the agent to fix the file the error names, not to rerun
harness lang add.Comments and docs. Permalink's section-slug note now explains it is the stored-link form;
SplitLangTag's caller note, publish's dotted-name note, AGENTS.md'shugo.tomlandfaq.en.mdnotes are corrected. AGENTS.md, the README and the edit-content, migrate-site, setup-site and verify-site skills describe the no-alias behaviour.Still for other issues. Concern 5 (the editor sending
toParentUid) is #192's; concern 6 (inline chips in the body hash) is a follow-up; #191 must resolve link values by content path.go build,go vet, gofmt andgo test ./...are clean, including the AGENTS.md and skills drift tests.web/changed in two comments only.[autoland] verdict: pass
land-pr verdict: PASS with CONCERNS (non-blocking), approved, awaiting confirm
Signal relied on.
ci / build-and-testone4d8138(go vet, go test, go build, npm test with Hugo 0.167.0) passed in 13m13s; not re-run. The first two commits had eleven autoland review passes (see the two lander verdicts above); I read the full diff ofe4d8138, the only commit no reviewer had seen. Base is current with main, no conflict.Conventions. Conventional Commits title;
Closes #190present; head isafk/190.Concerns for the human, before merging
aliases, slug changes and source renames to alias translations.e4d8138removes all of that, citing #194, which records the decision but says it "needs a proper design before anyone implements it"; the AFK run then closed #194 as implemented. It also removes behaviour that existed on main before this PR: rename and move no longer write the page's old URL as an alias, and create no longer strips shadowing aliases, on one-language sites too. So acceptance criterion 1 ("behaves exactly as before") now holds for reads but not for rename/move writes. Confirm this is wanted in this PR rather than as a follow-up.internal/linkrefs/doc.go:53-55andinternal/linkrefs/linkrefs.go:450now say a link that was not rewritten "is reported byharness checkas dangling".internal/checkcmdhas no link check; with aliases gone there is no safety net for a skipped rewrite. Fix the comment or add the check (follow-up).toParentUidso the transfer plan is exact; inline chips in the body hash still mark translations changed on a no-op chip save (follow-up); #191 must read.Language.Labeland resolve link values by content path.Everything else from the second verdict (concerns 1, 2, 3, 7 and the comment drift) is resolved in
e4d8138and covered by tests I read (collision with untranslated siblings and grouping directories,harness checkaddress errors, PathRules File/Slug with per-language keys, hand-written aliases left byte for byte).