feat: stream Reginald's replies token-by-token #49

christian · 2026-07-09T04:22:38Z

christian commented

2026-07-09 04:22:38 +00:00

The 26b is slow (~30s/call); the chat now shows the answer forming instead of freezing until it's done. The final prose streams over SSE; tool-calling turns stay structured (no partial tokens), so streaming kicks in for the narration.

Core (`@commitea/core`)

chat-client.complete(messages, tools, onToken?) — with onToken, requests stream: true and parses the OpenAI SSE stream: emits content deltas and assembles streamed tool-call argument fragments into the final CompletionResult.
GiteaHttpResponse exposes the optional body stream (real fetch has it; stubs don't). agent-loop threads onToken to each completion.

App

model:chat forwards each delta to the renderer (event.sender.send); preload exposes model.onToken(cb) → unsubscribe. useChat accumulates the live stream into a growing bubble (with a cursor), replaced by the authoritative final content when the turn resolves. Unconfigured → scripted reply, unchanged.

Verified

118 core tests green (2 streaming: SSE content deltas + tool-call fragment assembly) · desktop typecheck clean · 14 fixture e2e green. Live: a real turn against gemma-4-26b assembles the correct answer via the streaming path (live-reginald green) — the reply renders token-by-token.

Note: streaming doesn't make the model faster; it replaces the ~60s freeze with visible progress.

🤖 Generated with Claude Code

The 26b is slow (~30s/call); the chat now shows the answer **forming** instead of freezing until it's done. The final prose streams over SSE; tool-calling turns stay structured (no partial tokens), so streaming kicks in for the narration. ## Core (`@commitea/core`) - **`chat-client.complete(messages, tools, onToken?)`** — with `onToken`, requests `stream: true` and parses the OpenAI SSE stream: emits content deltas and assembles streamed tool-call argument fragments into the final `CompletionResult`. - `GiteaHttpResponse` exposes the optional `body` stream (real fetch has it; stubs don't). `agent-loop` threads `onToken` to each completion. ## App - `model:chat` forwards each delta to the renderer (`event.sender.send`); preload exposes `model.onToken(cb)` → unsubscribe. `useChat` accumulates the live stream into a growing bubble (with a cursor), replaced by the authoritative final content when the turn resolves. Unconfigured → scripted reply, unchanged. ## Verified 118 core tests green (2 streaming: SSE content deltas + tool-call fragment assembly) · desktop typecheck clean · 14 fixture e2e green. Live: a real turn against `gemma-4-26b` assembles the correct answer via the streaming path (`live-reginald` green) — the reply renders token-by-token. Note: streaming doesn't make the model faster; it replaces the ~60s freeze with visible progress. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

christian added 1 commit 2026-07-09 04:22:39 +00:00

feat: stream Reginald's replies token-by-token dbcdcda5e7

The 26b is slow (~30s/call); the chat now shows the answer forming instead of
freezing until it's done. The final prose streams over SSE; tool-calling turns
stay structured (no partial tokens), so streaming kicks in for the narration.

core (@commitea/core):
- chat-client.complete gains an optional onToken — when set, it requests
  stream:true and parses the OpenAI SSE stream, emitting content deltas and
  assembling streamed tool-call argument fragments into the final result.
- GiteaHttpResponse exposes the optional `body` stream (real fetch has it; stubs
  don't). agent-loop threads onToken to each completion.

app:
- model:chat forwards each delta to the renderer (event.sender.send); preload
  exposes model.onToken(cb) → unsubscribe. useChat accumulates the live stream
  into a growing bubble (with a cursor), replaced by the authoritative final
  content when the turn resolves. Unconfigured → scripted reply, unchanged.

Verified: 118 core tests green (2 streaming: SSE content deltas + tool-call
fragment assembly), desktop typecheck clean, 14 fixture e2e green. Live: a real
turn against gemma-4-26b assembles the correct answer via the streaming path
(live-reginald green) — the reply now renders token-by-token.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

christian added 1 commit 2026-07-09 04:43:53 +00:00

Merge branch 'main' into feat/streaming-chat cf03827cd5

christian merged commit d80e1266ee into main

2026-07-09 04:43:58 +00:00

christian referenced this issue from a commit

2026-07-09 04:44:00 +00:00

Merge pull request 'feat: stream Reginald's replies token-by-token' (#49) from feat/streaming-chat into main

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: christian/commitea#49

feat: stream Reginald's replies token-by-token #49

Core (@commitea/core)

App

Verified

Core (`@commitea/core`)