Editor polls rebuild the whole site on every tick; on a 2,500-page site the harness pins its CPU and the status poll never answers #358
Labels
No labels
bug
enhancement
needs-triage
ready-for-agent
ready-for-human
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
Cloonar/deckle#358
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
On a site with ~2,500 pages the harness pins its CPU answering the SPA's two-second tree and badge polls, each of which rebuilds the page tree, parses every page body and shells out to git. In hosted mode each editor runs on a single core, so the cheap
GET /api/statuspoll starves behind them and the editor shows "Vorschau startet" / "Preview starting" forever although Hugo has been serving for minutes. The poll endpoints need a cost that does not grow with every request, and the SPA must never pile up requests.Current behavior
Observed on deckle-hosted (deckle-01, site
gbv-aktuell, Hugo 0.167.0, harness buildf09f77f0, 2026-10-31). The site has 2,573 pages, 2,534 of them undernews/. The instance runs under--cpus=1 --memory=1g.site preview ready(Hugo's first build took 5.5 minutes on the throttled CPU).git ls-tree -r -z --name-only refs/remotes/origin/main,git diff --name-status -zandgit ls-files: one badge round trip takes about that long, against a 2 s poll interval.setIntervalwith no in-flight guard, so a slow answer never slows the next request, and the requests pile up.GET /api/statusfails. The status store keeps its last known state on failure, which is the initialstarting; the pill reads "Vorschau startet" and the preview iframe never receivespreviewUrl.curlof the preview answered.Where the cost comes from, per poll cycle (every 2 s):
GET /api/tree→annotatedTree()→pagetree.Build()reads and parses every page's front matter, thenmarkOutdated().GET /api/badges(fetched after every tree poll) →pagetree.Build()again,ChangedPublishFiles()(git diff+git ls-files),MainContentFiles()(git ls-tree), andpageMediaRefs(root, ".", false), which reads every.mdfile and runsmediaops.PageRefsover its body.GET /api/publish/pending(every 3 s while visible) repeats the three git calls and the bundle index.GET /api/status, every 1.5 s, is cheap and takes no lock; the Go server answers it on its own goroutine. It starves only because the single core is saturated and the piled-up connections fill the bridge's backlog. There is no server-side queue to remove: the fix for status is cheap polls plus the SPA in-flight guard.Reproduced in triage (2026-10-31) by driving the handler directly on a generated site of 2,574 pages (2,534 under
news/, 5,660 entries undercontent/), pinned to one core withtaskset -c 0. Every request costs the same whether or not anything changed since the previous one:GET /api/treeGET /api/badgesdiff,ls-files,ls-tree)GET /api/publish/pendingGET /api/statuspagetree.BuildalonepageMediaRefsalonegit diff --name-statusalonegit ls-files --othersalonecontent/(5,660 entries)Reproduction without the hosted platform: scaffold a site, generate ~2,500 pages under one section, run
harness servewithtaskset -c 0(or in a container with--cpus=1), open the editor, and watch the harness's CPU and the time to first/api/treeanswer. On a wide machine the symptom is softer but the request duration still grows linearly with page count.Desired behavior
/api/tree,/api/badges,/api/publish/pending,/api/status) answer in bounded time on a site of several thousand pages, and an idle editor tab leaves the harness close to idle. Work that depends only on the content tree and the git state is done once per change, not once per request: two consecutive polls with no save, no publish, no sync and no change on disk in between must not re-read any page or re-run git.GET /api/statuskeeps answering from the supervisor's snapshot, without any content or git work, and is answered within its normal latency on a one-core run while a badge request is in progress.content/by hand (a site developer or their agent editing files whileharness serveruns locally) is reflected in the next poll's answer, exactly as today.Decisions
/api/statusis the one signal the user has while Hugo starts, and it must stay truthful even when the content endpoints are slow. It takes no lock and does no content or git work; keep it that way.content/, not by the mutation path alone — a cached derivation of the content tree or the git state is valid while a fingerprint ofcontent/(every entry's path, type, size and mtime, files included; no reads, no parses) is unchanged and nothing moved the git state. Why:harness servealso runs locally, where content is edited by hand and must show up at the next poll; the mutating routes alone would miss that. Measured in triage on 2,574 pages on one core: a full walk costs 25 ms, versus ~270 ms of rebuild work per cycle today. A directories-only walk (19 ms) was rejected because an in-place file write changes no directory mtime. A filesystem watcher (inotify) was rejected: it is zero-cost when idle, but Linux-only (the harness also runs on macOS and Windows), subject tofs.inotify.max_user_watches(one watch per directory; 2,577 here, and some containers allow 8,192), races on newly created directories, and would still need the walk as its fallback, i.e. two owners for one rule. The walk may be added as an optimisation later behind the same owner.mutMu, and theX-Draft-Tipheader thedraftTipWriterinjects already names the draft tip; invalidate there, or key on the ref itself (reading a ref is far cheaper than spawning git). How the cache is keyed and stored is the implementer's call.mutMu, seemutatingRoutesininternal/server/mutation.go), the publish job (which takesmutMuitself) or the startup sync. A mutation invalidates the cache directly so the next poll never answers from a stale entry even inside the fingerprint window.Key interfaces
pagetree.Build(contentDir, langs, lang, types)— called twice per cycle (tree and badges); its result for an unchanged content tree should be reusable.Handler.annotatedTree(),Handler.handleTree(),Handler.handleBadges(),Handler.handlePublishPending()— the per-request scans live here. Starting hint:internal/server/modules.go,internal/server/git.go,internal/server/pending.go.pageMediaRefs(root, contentPaths, strict)— reads and parses every page body on each badge request; the non-strict (badge) use is the one that must become cached. Starting hint:internal/server/publishset.go.gitrepo.Repo.ChangedPublishFiles(),gitrepo.Repo.MainContentFiles()— spawngit diff,git ls-filesandgit ls-treeon each call; the results are stable between mutations and content changes. Starting hint:internal/gitrepo/status.go.Handler.ServeHTTP/mutMu— the point every mutation passes; the natural place to invalidate. Starting hint:internal/server/server.go,internal/server/mutation.go.content/withfilepath.WalkDirand hashing each entry's path, mode, size and mtime.createPageTree(TREE_POLL_INTERVAL_MS, which also fetches badges),createPageLocks,createHugoStatus(POLL_INTERVAL_MS), and the two polling loops inweb/src/publish.ts(PUBLISH_POLL_INTERVAL_MS).createModulesalready has one (inFlightinweb/src/stores/modules.ts) and is the pattern to follow. Starting hint:web/src/stores/.Acceptance criteria
GET /api/treeorGET /api/badgesrequests with no mutation and no change on disk between them: the second one performs no per-page file reads and spawns no git process (verifiable withstrace -f -e trace=openat,execveor a counting test double).taskset -c 0),GET /api/statusanswers within its normal latency while aGET /api/badgeson the large site is in progress.POST /api/page/save(and each other mutating route, the publish job and the startup sync) is reflected in the next/api/tree,/api/badgesand/api/publish/pendinganswer, as today; the existing tests for those endpoints pass unchanged.content/(no API call) is reflected in the next/api/treeand/api/badgesanswer; a Go test covers an in-place edit of an existing page file and a new page directory with a file inside it.pagetree.Buildplus the badge computation on a generated large site exists next tohistory_bench_test.go's pattern, so the cost stays visible, and it reports the fingerprint walk's cost separately.docs/architecture.md"Server" describes the cached derivations, the fingerprint and the invalidation points, and a new ADR indocs/adr/records the decision (stat walk over watcher, with the reasons above).docs/api.mdis unchanged because no wire shape changes.How to verify
make test(Go and web suites)./proc/<pid>/statCPU ticks of the harness over 60 s with an editor tab open, before and after; countgitchildren withpgrep -P <harness pid>over the same window.taskset -c 0 curl -w '%{time_total}' http://127.0.0.1:<port>/api/statuswhile a badge request is running on the large site.content/with an editor while the harness runs and the tree is open; the title change shows at the next poll.Assumptions to check
content/; if anything else can move main's tip or the worktree outsidecontent/(the media library underassets/media/feeds the badges too), the key must cover that as well — check whether the library needs to be in the walk or is covered by the mutation path alone.pageMediaRefs's fast path (bytes.Contains(src, LibraryPrefix)) does not save the read: the file is still opened and read for every page.Out of scope
--cpus=1 --memory=1g) and the socat bridge's listen backlog — these are deckle-hosted's and are filed there.hugo serverconfiguration,--renderToMemory, render segments).scaffold/site/.claude/skills/andscaffold/site/AGENTS.mdneed no edit.Blocked by
None - can start immediately