Cloonar/coding-lab

Fork 0

Chat: /clear breaks the session — lab keeps reading the pre-clear transcript file #34

New issue

Closed

opened 2026-07-07 23:45:48 +02:00 by dominik.polakovics · 1 comment

dominik.polakovics commented

2026-07-07 23:45:48 +02:00

Owner

Summary

Using /clear from the embedded chat composer (/runs/:id) breaks the
session: the chat does not clear, and replies sent afterwards produce no
visible response. The only recovery is to stop the instance and start a new
one.

Root cause: lab has no "clear" feature — /clear is forwarded verbatim into
the Claude session as a normal reply. Claude Code's /clear starts a new
session (new sessionId, new transcript file, old file frozen on disk), but
lab pins each run to a write-once runs.transcript_path derived from the
old sessionId and never re-validates it. lab keeps reading the dead
transcript forever.

Reproduction

Open a live instance's chat at /runs/:id.
Type /clear into the composer and send.
Observed: the chat keeps showing the pre-clear messages; subsequent replies
yield no visible response; the composer can get stuck (wrong Send/Interrupt
state). Stopping the instance and starting a new one is the only recovery.

Root cause (code)

The transcript path is resolved once, persisted to runs.transcript_path, and
then returned unconditionally for the life of the run:

internal/chat/chat.go:266-282 — resolvePath returns the persisted path if
set; it only calls LocateTranscript when the path is empty. Never
re-validated after the first hit.
internal/chat/tailer.go:145-147 — the tailer caches path in a goroutine
local and re-resolves only if path == "". It then stats the frozen file
forever → ModTime/Size never change → no run.messages.changed ever
fires again (tailer.go:159-172).
internal/provider/claudecode/chat.go:53-63 (LocateTranscript) /
:69-95 (sessionIDForDir) — the path is
<projects>/<slug(worktree)>/<sessionId>.jsonl, built from the current
registry sessionId in ~/.claude/sessions/<pid>.json. Correct at first
locate, but the function is never called again.
Only writer of the field: internal/store/runs.go:204-212
(UpdateRunTranscriptPath); field at runs.go:51. There is no code anywhere
that invalidates or re-locates it.

Evidence (verified)

Claude Code mechanism: /clear calls setConversationId(randomUUID())
→ a brand-new <newSessionId>.jsonl; the old file is left intact/resumable.
/compact, by contrast, keeps the same sessionId and appends to the same
file — so /compact is not affected by this bug. Refs:
anthropics/claude-code#37451, anthropics/claude-code#3046.
On-host artifact (real claude 2.1.198): a worktree's project dir held two
transcripts for one cwd — an older frozen file and a newer active file with a
different sessionId, whose line 4 was the /clear command marker and which
carried zero back-reference to the old sessionId (a clean rotation, not a
resume).
Registry invariant (verified on live sessions):
~/.claude/sessions/<pid>.json's sessionId field equals the currently
active transcript filename in that worktree's project dir. So re-running
LocateTranscript for a live process resolves to the current session —
which is what makes the fix below work.

Symptom → cause (1:1)

"clear doesn't clear" — the UI keeps reading the old frozen file; the
tailer sees no change, so no refresh event fires.
"sends don't work" — the reply does reach Claude (tmux send-keys is
addressed by session name, stable across /clear —
internal/provider/claudecode/interact.go:44), and Claude answers into the
new file; lab reads the old file, so the message and its reply are
invisible. Composer state is derived from the frozen file, so the Send /
Interrupt / locked control can also be wrong.
"kill and restart" — a new run row has a NULL transcript_path, so
resolvePath re-locates and picks up the current session. The only recovery
today.

Fix (chosen approach: make `/clear` work)

Backend — re-locate on rotation (`internal/chat`)

In resolvePath, for an active run, re-validate the persisted path
against the live session on each read/tick. If LocateTranscript returns a
different, non-empty path, adopt it: UpdateRunTranscriptPath and return
the new path. Rules:
- Ended run: never re-locate (preserve the existing safety that stops it
  grabbing a successor run's transcript — see the comment at
  chat.go:259-266).
- Empty locate: keep the persisted path (do not clobber during a transient
  registry gap / between-states moment).
- Same path: no-op.
In tailer.go, stop caching path write-once — re-resolve each tick (or when
a lightweight sessionId check changes). On a path change, reset
lastMod/lastSz/lastChat and force publishMessagesChanged so the UI
refetches the fresh (cleared) transcript.
Surface a transcript-identity token in the messages response
(messagesResponse, internal/httpapi/chat.go:39-49) — e.g. transcript_id
= the resolved sessionId (or a stable hash of the path). This is what the
frontend watches to reset.

This is general: it also covers rotation triggered by /clear run directly in
claude.ai, or by /rewind.

Frontend — clear the stream on rotation (`web/src/routes/RunChat.tsx`)

Read transcript_id from the messages response into a signal.
Key the reset effect (currently on params.id, RunChat.tsx:188-203) on
(params.id, transcriptId). On change, reset
messages/state/pendingDialogField/hasMore/exhausted/openGroups
(same as the route-change reset), then refetch.
Why required: the new transcript restarts seq at 1, so without a reset
the new messages collide with stale ones in the seq-keyed merge
(web/src/lib/chatStream.ts). It also makes /clear visibly clear the chat
and re-enables the composer.

The reply/answer/interrupt send path needs no change (already addressed by
session name).

Acceptance criteria

After /clear in the composer: the visible chat clears within ~1 poll tick,
the composer returns to a usable Send state, and a subsequent reply appears
with Claude's response.
No regression for: normal replies, pending dialogs, interrupts, ended runs
(read-only), and transcript-gone.
/compact still shows continuous history (same file; unaffected).
Reading an ended run still works from its last-known path (no re-locate).

Tests

internal/chat: with providertest.Fake.SetTranscriptPath
(internal/provider/providertest/fake.go:402), simulate rotation mid-run
(path A → path B with different content). Assert Read follows to B, persists
B via UpdateRunTranscriptPath, and the tailer publishes
run.messages.changed. Assert an ended run does NOT re-locate. Assert an
empty locate does not clobber an already-set path.
internal/httpapi: messages response includes transcript_id, and it changes
across a rotation.
web: RunChat resets the accumulated stream when transcript_id changes
(precedent: web/src/routes/RunChat.test.tsx).

Verify live during implementation

Confirm ~/.claude/sessions/<pid>.json's sessionId updates promptly after an
interactive /clear (the on-host artifact + live invariant both indicate yes;
a ~2-minute manual check with real claude 2.1.198). If it lags, fall back to
"newest actively-growing .jsonl in the cwd-slug project dir" as the locate
strategy.

Docs

This is a newly-found fragile Claude Code coupling: /clear (and /rewind)
rotate the sessionId → transcript file, and lab must follow. Document it in
internal/compat/compat.md §5 (transcript location).
Preserves intervention neutrality (issue #7 / ADR-0016 decision 12): no
budget/claim/three-strikes effects.

## Summary Using **`/clear`** from the embedded chat composer (`/runs/:id`) breaks the session: the chat does not clear, and replies sent afterwards produce no visible response. The only recovery is to stop the instance and start a new one. Root cause: lab has no "clear" feature — `/clear` is forwarded verbatim into the Claude session as a normal reply. Claude Code's `/clear` **starts a new session** (new `sessionId`, new transcript file, old file frozen on disk), but lab pins each run to a **write-once** `runs.transcript_path` derived from the *old* sessionId and never re-validates it. lab keeps reading the dead transcript forever. ## Reproduction 1. Open a live instance's chat at `/runs/:id`. 2. Type `/clear` into the composer and send. 3. Observed: the chat keeps showing the pre-clear messages; subsequent replies yield no visible response; the composer can get stuck (wrong Send/Interrupt state). Stopping the instance and starting a new one is the only recovery. ## Root cause (code) The transcript path is resolved once, persisted to `runs.transcript_path`, and then returned unconditionally for the life of the run: - `internal/chat/chat.go:266-282` — `resolvePath` returns the persisted path if set; it only calls `LocateTranscript` when the path is **empty**. Never re-validated after the first hit. - `internal/chat/tailer.go:145-147` — the tailer caches `path` in a goroutine local and re-resolves only `if path == ""`. It then stats the frozen file forever → `ModTime`/`Size` never change → **no `run.messages.changed` ever fires again** (`tailer.go:159-172`). - `internal/provider/claudecode/chat.go:53-63` (`LocateTranscript`) / `:69-95` (`sessionIDForDir`) — the path is `<projects>/<slug(worktree)>/<sessionId>.jsonl`, built from the **current** registry sessionId in `~/.claude/sessions/<pid>.json`. Correct at first locate, but the function is never called again. - Only writer of the field: `internal/store/runs.go:204-212` (`UpdateRunTranscriptPath`); field at `runs.go:51`. There is no code anywhere that invalidates or re-locates it. ## Evidence (verified) - **Claude Code mechanism:** `/clear` calls `setConversationId(randomUUID())` → a brand-new `<newSessionId>.jsonl`; the old file is left intact/resumable. `/compact`, by contrast, keeps the **same** sessionId and appends to the same file — so `/compact` is **not** affected by this bug. Refs: anthropics/claude-code#37451, anthropics/claude-code#3046. - **On-host artifact (real claude 2.1.198):** a worktree's project dir held two transcripts for one cwd — an older frozen file and a newer active file with a different sessionId, whose line 4 was the `/clear` command marker and which carried zero back-reference to the old sessionId (a clean rotation, not a resume). - **Registry invariant (verified on live sessions):** `~/.claude/sessions/<pid>.json`'s `sessionId` field equals the currently active transcript filename in that worktree's project dir. So re-running `LocateTranscript` for a live process resolves to the *current* session — which is what makes the fix below work. ## Symptom → cause (1:1) 1. **"clear doesn't clear"** — the UI keeps reading the old frozen file; the tailer sees no change, so no refresh event fires. 2. **"sends don't work"** — the reply *does* reach Claude (tmux send-keys is addressed by session **name**, stable across `/clear` — `internal/provider/claudecode/interact.go:44`), and Claude answers into the **new** file; lab reads the **old** file, so the message and its reply are invisible. Composer state is derived from the frozen file, so the Send / Interrupt / locked control can also be wrong. 3. **"kill and restart"** — a new run row has a NULL `transcript_path`, so `resolvePath` re-locates and picks up the current session. The only recovery today. ## Fix (chosen approach: make `/clear` work) ### Backend — re-locate on rotation (`internal/chat`) - In `resolvePath`, for an **active** run, re-validate the persisted path against the live session on each read/tick. If `LocateTranscript` returns a **different, non-empty** path, adopt it: `UpdateRunTranscriptPath` and return the new path. Rules: - **Ended run:** never re-locate (preserve the existing safety that stops it grabbing a *successor* run's transcript — see the comment at `chat.go:259-266`). - **Empty locate:** keep the persisted path (do not clobber during a transient registry gap / between-states moment). - **Same path:** no-op. - In `tailer.go`, stop caching `path` write-once — re-resolve each tick (or when a lightweight sessionId check changes). On a path change, reset `lastMod`/`lastSz`/`lastChat` and force `publishMessagesChanged` so the UI refetches the fresh (cleared) transcript. - Surface a **transcript-identity token** in the messages response (`messagesResponse`, `internal/httpapi/chat.go:39-49`) — e.g. `transcript_id` = the resolved sessionId (or a stable hash of the path). This is what the frontend watches to reset. This is general: it also covers rotation triggered by `/clear` run directly in claude.ai, or by `/rewind`. ### Frontend — clear the stream on rotation (`web/src/routes/RunChat.tsx`) - Read `transcript_id` from the messages response into a signal. - Key the reset effect (currently on `params.id`, `RunChat.tsx:188-203`) on `(params.id, transcriptId)`. On change, reset `messages`/`state`/`pendingDialogField`/`hasMore`/`exhausted`/`openGroups` (same as the route-change reset), then refetch. - **Why required:** the new transcript restarts `seq` at 1, so without a reset the new messages collide with stale ones in the seq-keyed merge (`web/src/lib/chatStream.ts`). It also makes `/clear` visibly clear the chat and re-enables the composer. The reply/answer/interrupt send path needs **no change** (already addressed by session name). ## Acceptance criteria - After `/clear` in the composer: the visible chat clears within ~1 poll tick, the composer returns to a usable Send state, and a subsequent reply appears with Claude's response. - No regression for: normal replies, pending dialogs, interrupts, ended runs (read-only), and transcript-gone. - `/compact` still shows continuous history (same file; unaffected). - Reading an ended run still works from its last-known path (no re-locate). ## Tests - `internal/chat`: with `providertest.Fake.SetTranscriptPath` (`internal/provider/providertest/fake.go:402`), simulate rotation mid-run (path A → path B with different content). Assert `Read` follows to B, persists B via `UpdateRunTranscriptPath`, and the tailer publishes `run.messages.changed`. Assert an **ended** run does NOT re-locate. Assert an **empty** locate does not clobber an already-set path. - `internal/httpapi`: messages response includes `transcript_id`, and it changes across a rotation. - `web`: `RunChat` resets the accumulated stream when `transcript_id` changes (precedent: `web/src/routes/RunChat.test.tsx`). ## Verify live during implementation Confirm `~/.claude/sessions/<pid>.json`'s `sessionId` updates promptly after an interactive `/clear` (the on-host artifact + live invariant both indicate yes; a ~2-minute manual check with real claude 2.1.198). If it lags, fall back to "newest actively-growing `.jsonl` in the cwd-slug project dir" as the locate strategy. ## Docs - This is a newly-found fragile Claude Code coupling: `/clear` (and `/rewind`) rotate the sessionId → transcript file, and lab must follow. Document it in `internal/compat/compat.md` §5 (transcript location). - Preserves intervention neutrality (issue #7 / ADR-0016 decision 12): no budget/claim/three-strikes effects.

dominik.polakovics added the

ready-for-agent

bug

labels

2026-07-07 23:45:48 +02:00

dominik.polakovics commented

2026-07-08 00:17:23 +02:00

Author

Owner

Design decisions (reviewed with maintainer, 2026-07-08) — refines the Fix section above

Where this comment differs from the original Fix section, this comment wins.

Scope — tolerance only; NO Clear affordance

lab does not grow a "Clear" button or a ClearChat provider capability. The
requirement is solely that lab behaves correctly when the operator commands
the agent to clear — e.g. types /clear, which stays an ordinary free-text
passthrough reply. The "interface" here is the provider seam, not UI.

Detect by effect, not by cause

lab must never parse /clear or know any agent's clear command. Each tick it
re-asks the provider for the current transcript of the live session
(LocateTranscript, already on the seam) and follows it when the answer
changes. All agent-specificity stays inside LocateTranscript; lab core
stays ignorant of clear commands. Bonus: keying off the observable effect also
handles a clear run out-of-band (inside claude.ai, or /rewind) for free.

View semantics — fresh-only, re-point & forget

runs.transcript_path stays a single value, re-pointed to the new
transcript. Pre-clear messages vanish from the lab view (the old file remains on
disk, claude --resume-able). No transcript-history chaining (YAGNI): the
transcript is display-only, so re-pointing has no budget / claim / three-strikes
/ CR-done-signal consequence.

NEW — dismiss a stale pending-dialog spool on rotation

(Not in the original body; must be included.) PendingDialog
(internal/provider/claudecode/dialogspool.go:193) suppresses a spooled dialog
only when its tool_use_id appears in the transcript. So after a re-point to
a fresh file, a pre-clear pending-dialog spool (keyed by the stable run ID)
does not self-heal — the composer stays locked and sends stay blocked for
this sub-case. Fix: give PendingDialog the same mtime staleness check
BlockedState already uses (dialogspool.go:208-227), OR'd with the existing
tool-id check (tool-id stays primary): a spool older than the current transcript
is stale. Safe for the normal pending case — compat §5 keeps the transcript
byte-frozen during a genuine pending dialog while the spool is written after
it, so the spool is always newer and the dialog still shows.

Frontend visible clear — explicit `transcript_id`, no inference

The re-point keeps the same run id, so the SPA's route-change reset
(web/src/routes/RunChat.tsx:188) won't fire, and the new transcript restarts
seq at 1 (collides with the stale accumulated messages in the seq-keyed merge,
web/src/lib/chatStream.ts). So:

Backend surfaces transcript_id in the messages response — the located
transcript's identity (for Claude, the sessionId/filename). Provider-neutral
in the envelope, provider-derived in value; the same value the backend
compares to detect the rotation.
The SPA keys its existing reset effect on (runId, transcript_id): on change it
clears the stream/state exactly like a route change, then refetches. That is
what makes the chat visibly empty out and the composer come back to life.

Unchanged from the body

Re-locate only for ACTIVE runs (never ended — preserves successor-safety).
Never clobber a set transcript_path with an empty locate (transient
registry gap / between-states).
Tailer stops caching path write-once; re-resolves each tick, resets its
cached mtime/size on a change, and forces run.messages.changed.
/compact is unaffected (same sessionId, same file).
Reply / answer / interrupt send path unchanged (addressed by tmux session
name, stable across a clear).
Verify live that ~/.claude/sessions/<pid>.json's sessionId updates on
/clear (the linchpin the re-locate relies on).
Document the new coupling in internal/compat/compat.md §5: /clear (and
/rewind) rotate the sessionId → transcript file; lab follows by effect.

Acceptance additions

After a clear with a dialog pending, the composer unlocks and a subsequent
reply is delivered and visible.
After a clear, transcript_id changes and the SPA stream resets to the fresh
conversation within ~1 poll tick.

## Design decisions (reviewed with maintainer, 2026-07-08) — refines the Fix section above Where this comment differs from the original **Fix** section, this comment wins. ### Scope — tolerance only; NO Clear affordance lab does **not** grow a "Clear" button or a `ClearChat` provider capability. The requirement is solely that lab **behaves correctly when the operator commands the agent to clear** — e.g. types `/clear`, which stays an ordinary free-text passthrough reply. The "interface" here is the **provider seam**, not UI. ### Detect by effect, not by cause lab must **never** parse `/clear` or know any agent's clear command. Each tick it re-asks the provider for the current transcript of the live session (`LocateTranscript`, already on the seam) and follows it when the answer **changes**. All agent-specificity stays inside `LocateTranscript`; lab core stays ignorant of clear commands. Bonus: keying off the observable *effect* also handles a clear run out-of-band (inside claude.ai, or `/rewind`) for free. ### View semantics — fresh-only, re-point & forget `runs.transcript_path` stays a **single** value, re-pointed to the new transcript. Pre-clear messages vanish from the lab view (the old file remains on disk, `claude --resume`-able). **No** transcript-history chaining (YAGNI): the transcript is display-only, so re-pointing has no budget / claim / three-strikes / CR-done-signal consequence. ### NEW — dismiss a stale pending-dialog spool on rotation (Not in the original body; **must** be included.) `PendingDialog` (`internal/provider/claudecode/dialogspool.go:193`) suppresses a spooled dialog **only** when its `tool_use_id` appears in the transcript. So after a re-point to a fresh file, a pre-clear pending-dialog spool (keyed by the **stable run ID**) does **not** self-heal — the composer stays locked and sends stay blocked for this sub-case. Fix: give `PendingDialog` the same mtime staleness check `BlockedState` already uses (`dialogspool.go:208-227`), OR'd with the existing tool-id check (tool-id stays primary): a spool older than the current transcript is stale. Safe for the normal pending case — compat §5 keeps the transcript byte-frozen during a genuine pending dialog while the spool is written *after* it, so the spool is always newer and the dialog still shows. ### Frontend visible clear — explicit `transcript_id`, no inference The re-point keeps the **same run id**, so the SPA's route-change reset (`web/src/routes/RunChat.tsx:188`) won't fire, and the new transcript restarts `seq` at 1 (collides with the stale accumulated messages in the seq-keyed merge, `web/src/lib/chatStream.ts`). So: - Backend surfaces **`transcript_id`** in the messages response — the located transcript's identity (for Claude, the `sessionId`/filename). **Provider-neutral in the envelope, provider-derived in value**; the same value the backend compares to detect the rotation. - The SPA keys its existing reset effect on `(runId, transcript_id)`: on change it clears the stream/state exactly like a route change, then refetches. That is what makes the chat visibly empty out and the composer come back to life. ### Unchanged from the body - Re-locate **only for ACTIVE runs** (never ended — preserves successor-safety). - **Never clobber** a set `transcript_path` with an empty locate (transient registry gap / between-states). - Tailer stops caching `path` write-once; re-resolves each tick, resets its cached mtime/size on a change, and forces `run.messages.changed`. - `/compact` is unaffected (same sessionId, same file). - Reply / answer / interrupt send path unchanged (addressed by tmux session name, stable across a clear). - **Verify live** that `~/.claude/sessions/<pid>.json`'s `sessionId` updates on `/clear` (the linchpin the re-locate relies on). - Document the new coupling in `internal/compat/compat.md` §5: `/clear` (and `/rewind`) rotate the sessionId → transcript file; lab follows by effect. ### Acceptance additions - After a clear **with a dialog pending**, the composer unlocks and a subsequent reply is delivered and visible. - After a clear, `transcript_id` changes and the SPA stream resets to the fresh conversation within ~1 poll tick.

dominik.polakovics referenced this issue from a commit

2026-07-08 01:42:04 +02:00

fix(chat): follow transcript rotation on /clear so the chat keeps working (#34)

dominik.polakovics referenced this issue from a pull request that will close it,

2026-07-08 01:42:39 +02:00

fix(chat): follow transcript rotation on /clear so the chat keeps working (#34) #42

dominik.polakovics referenced this issue from a commit

2026-07-08 09:16:49 +02:00

fix(chat): re-guard refetch after transcript_id write so a straddling rotation can't leak a stale tail

dominik.polakovics closed this issue

2026-07-08 09:24:42 +02:00

dominik.polakovics referenced this issue from a commit