* refactor(eval): drop unused agents/graders, collapse registries
Sweep of dead code in the eval app: deleted gemini-computer-use and
yutori-navigator agents, fara/webvoyager/mind2web graders, eight
debug/analyze/test scripts, three stale planning docs, and the orphaned
eval-targets/coordinate-click testbed.
With two agents and three graders left, the Map-backed plugin registries
were over-engineered — collapsed both into plain switches. Removed the
now-dead GraderOptions plumbing (no remaining grader takes API keys),
dropped grader_api_key_env/grader_base_url/grader_model from the schema
and configs, and de-duped PASS_FAIL_GRADER_ORDER (was defined in three
places). Replaced the URL-parsing extractCdpPort hack in single-agent
and orchestrator-executor with workerIndex passed cleanly through
AgentContext.
README and --help text rewritten to match reality. Renamed
configs/test_*.json to test-*.json for kebab-case consistency.
Net: ~10,460 LOC removed across 60 files. Typecheck clean, all tests
pass.
* ci(eval): pull BrowserOS from rolling stable CDN URL
The pinned v0.44.0.1 .deb on GitHub releases regressed on Linux —
servers start but never become healthy. Switch to the canonical rolling
URL at cdn.browseros.com/download/BrowserOS.deb so CI tracks the same
stable channel users get from the marketing site.
When BrowserOS sets up a custom OpenAI-compat provider on the gateway,
the agent UI's "Supports Image" flag (LlmProviderConfig.supportsImages)
was being dropped on the floor. As a result the persisted model entry
had no `input` field, OpenClaw defaulted it to ['text'], and image_url
content parts were silently stripped before the model saw them.
Fix:
- Extend OpenClawSetupInput / OpenClawAgentMutationInput on the agent
side (useOpenClaw.ts) and the route body schema + SetupInput +
createAgent input on the server side with `supportsImages?: boolean`.
- AgentsPage forwards `llmOption?.supportsImages` from the selected
LlmProviderConfig in both handleSetup and handleCreate.
- provider-map.resolveSupportedOpenClawProvider emits
`input: ['text', 'image']` on the model entry when the flag is
truthy; otherwise emits the explicit `['text']` so the value is
always pinned (avoids relying on OpenClaw's implicit default).
- applyBrowserosConfig adds `tools.media.image.enabled = true` to the
bootstrap batch so the gateway's image-understanding pipeline is
always wired up — per-model `input` still gates which models see
images, this just enables the global path.
ACP image content blocks are still dropped by the OpenClaw bridge —
that's a separate bridge bug, not addressed here. This commit
restores image support for the OpenAI-compat /v1/chat/completions
path that the upcoming ACP chat panel will use as a carve-out for
image-bearing prompts.
Existing custom-provider configs are NOT auto-migrated; users will
re-acquire image support either by re-running setup or by editing
their model entries' `input` field manually. A migration pass for
legacy installs is not in scope for this commit because the
"supportsImages" intent isn't recoverable from the persisted config
alone — the source of truth is the LlmProvider record on the agent
side.
* fix(eval): exclude broken tasks + freshen expired card dates
Two AGISDK tasks are unsolvable today for non-model reasons:
- topwork-1: evals-topwork.vercel.app throws Minified React error #185
("Maximum update depth exceeded") on every form submit. The page renders
"Application error: a client-side exception has occurred" instead of saving.
Whole-task failure, every model affected.
- fly-unified-2: hardcodes Exp: 12/25 in both the goal text AND a jmespath
grader criterion. Today is 2026-04, so the eval-site rejects the card.
Freshening the goal alone leaves the grader expecting the original value;
freshening both would require monkey-patching agisdk's TaskConfig at
runtime — too fragile to maintain.
Adds these to a new EXCLUDED_TASKS set alongside the existing
EXCLUDED_WEBSITES (omnizon).
Also adds freshen_goal_dates(): for AGISDK fly-unified tasks whose goal
contains an `Exp: MM/YY` within 6 months of today (or past), rewrites it
to a far-future date (12/30). This rescues fly-unified-5 (had Exp 12/25,
no card-exp grader criterion) and protects fly-unified-4 (had Exp 06/26,
2 months from expiring) from the next eval run hitting the same trap.
Dataset goes from 47 -> 45 tasks; 2 freshened.
* feat(eval): add lenient-strings grader softening
The agisdk grader compares jmespath-extracted values via strict equality.
For tasks where the model adds harmless decoration to a free-text field
(e.g. topwork-3 expects title "Full-Stack Developer" but model produces
"Full-Stack Developer - Enterprise Microservices Platform"), this fails
every other criterion would pass.
Adds a substring fallback in the wrapper: a failed criterion is re-marked
as a softened pass when both actual_value and expected_value are strings
and the (stripped, lower-cased) expected_value is contained in the
actual_value. Numbers/bools/dates/None stay strict.
- Default-on. Set AGISDK_STRICT_STRINGS=1 to recover the strict score.
- Softened criteria are tagged with `softened: true` in per_criterion
output for transparency in run manifests.
- Aggregate `pass`/`reward` are recomputed after softening.
Expected to rescue 4 tasks in our 45-set: topwork-3, topwork-4 (both pure
title-decoration), gomail-8 (grader contradicts goal), and networkin-6
(grader hardcodes profile id).
* fix(eval): exclude 5 more tasks where pipeline (not agent) fails
Extends EXCLUDED_TASKS to 7 entries based on the K2.5 + Opus 4.6
head-to-head deep-dive on the 2026-04-28 runs. The exclusion rule:
remove a task only if it is unsolvable for any agent — either the task
data is invalid, the eval site is broken, or the grader penalizes
correct work. Tasks that fail because of our agent's tool fidelity
(drag, custom-widget fill, click on React submit, etc.) STAY in — those
are real capability gaps the team should see in the score.
New exclusions:
- fly-unified-9: goal references "Dec 18 2024 at 10:00" but the live
eval site has only 2025 inventory and no 10:00 slot. Both models
successfully booked the closest available flight and were penalized
on a grader expectation that can never be met.
- fly-unified-4: eval site stores wall-clock flight times as bare UTC
(T08:00:00.000Z) while the grader expects them shifted by 8h
(T16:00:00.000Z = 8 AM PST). Opus 4.6 completed the entire booking
correctly. Eval-site TZ-storage bug.
- gomail-8: goal says "Clear all emails from GitHub in the inbox", but
criterion 3 expects exactly 1 email updated. Both K2.5 and Opus
correctly cleared all 4 GitHub emails. Grader contradicts goal.
- networkin-6: goal says "Choose a random person you haven't connected
with"; grader hardcodes profilesDiff.updated."4".connectionGrade.
Both models randomized correctly and missed id 4. Grader contradicts
goal.
- networkin-9: eval site's searchHistoryDiff doesn't record queries
submitted via the autocomplete + Enter path. Opus 4.6 completed the
task end-to-end (Stanford alum, connection request, message); only
failed because the search-history criterion was never written
server-side. Eval-site bug.
Dataset goes from 45 -> 40 tasks. Score impact (same K2.5/Opus runs,
recomputed against the cleaned 40-task denominator):
K2.5: 21/45 (46.7%) -> 21/40 (52.5%)
Opus 4.6: 28/45 (62.2%) -> 28/40 (70.0%)
Δ: 15.6 pp -> 17.5 pp (real model gap, less pipeline noise)
The 2026-04-23 weekly run had 42% of AGISDK and 46% of Infinity tasks
fail with `AI_RetryError: ... the service is overloaded` from Fireworks
(20 concurrent kimi-k2p5 streams across both runs at 10 workers each).
Switching to OpenRouter (which fronts the same Moonshot K2.5 weights
and falls back across providers) for the three weekly configs:
- browseros-agent-weekly.json
- agisdk-real-smoke.json
- infinity-hard-50.json
Model accounts/fireworks/models/kimi-k2p5 -> moonshotai/kimi-k2.5
(same weights, same 262K context). API key env var, base URL updated.
OPENROUTER_API_KEY is already wired into .github/workflows/eval-weekly.yml
and present in repo secrets — no GH config changes needed.
Orchestrator-executor configs and test_webvoyager left on Fireworks
intentionally; can switch later if needed.
* feat: deterministic eval graders (AGI SDK + WebArena-Infinity) (#664)
* feat: add deterministic eval graders (AGI SDK + WebArena-Infinity)
Two new benchmark integrations with programmatic grading — no LLM judge.
AGI SDK / REAL Bench (52 tasks):
- 11 React/Next.js clones of consumer apps (DoorDash, Amazon, Gmail, etc.)
- Grader navigates browser to /finish, extracts state diff from <pre> tag
- Python verifier checks exact values via jmespath queries
WebArena-Infinity (50 hard tasks):
- 13 LLM-generated SaaS clones (Gmail, GitLab, Linear, Figma, etc.)
- InfinityAppManager starts fresh app server per task per worker
- Python verifier calls /api/state and asserts on JSON state
Infrastructure:
- GraderInput extended with mcpUrl + infinityAppUrl for parallel workers
- Each worker gets isolated ports (no cross-worker state contamination)
- CI workflow: pip install agisdk, clone webarena-infinity repo
* chore: switch eval configs back to kimi-k2p5
* fix: register deterministic graders in pass rate calculation
Add agisdk_state_diff and infinity_state to PASS_FAIL_GRADER_ORDER
in both runner types and weekly report script, so scores show correctly
in the dashboard.
* chore: temp switch to opus 4.6 for eval run
* chore: restore kimi-k2p5 as default eval config
* ci: add timeout and continue-on-error for trend report step
* fix(eval): drop omnizon from AGISDK dataset (DMCA takedown)
evals-omnizon.vercel.app returns HTTP 451 ("This content has been
blocked for legal reasons / DMCA_TAKEDOWN"). All 5 omnizon-* tasks
fail grading with "Failed to fetch /finish endpoint: JSON Parse error".
Adds an EXCLUDED_WEBSITES set to the dataset builder and regenerates
agisdk-real.jsonl (52 → 47 tasks).
* fix(eval): correct Infinity port-assignment bugs
Two related bugs in the Infinity eval runner that cause silent port
collisions / fallbacks under parallel execution:
1. build-infinity-dataset.py emitted "app_port" but task-executor and
the committed JSONL both read "app_base_port". Re-running the build
script would silently make every task fall back to the 8000 default,
ignoring per-app port assignments. Renamed the key to match.
2. task-executor derived workerIndex as `base_server_port - 9110`, but
parallel-executor doesn't override base_server_port per worker —
only server_url. Every worker computed workerIndex = 0, causing all
parallel workers to spawn Infinity app servers on the same port.
Threading workerIndex explicitly through TaskExecutor instead.
Also drops an unused app_name parameter from load_tasks().
* feat(agent): attach images and text files to chat messages
Adds end-to-end support for image and text file attachments in the chat
composer, with the staged files round-tripping through the OpenClaw
gateway as OpenAI-compatible content blocks and persisting in the JSONL
so they show up in the historical view.
Server
- HTTP client: new OpenClawChatContentPart union and a buildUserContent
helper that emits multimodal content arrays when messageParts is
supplied, falls back to the legacy string content otherwise.
- Service: chatStream takes an optional messageParts array and forwards
it; BrowserOSChatHistoryItem gains an attachments field.
- JSONL reader: PiContentBlock learns the OpenAI image_url and Anthropic
image source/data shapes; user messages now emit user.attachment
events that the history mapper accumulates onto the next user item.
- Route: validates an inbound attachments[] (kind/mime/size/count),
inlines text-shaped files as <attachment> blocks in the message body,
attaches images via image_url parts. Replaces the immediate 409 on
active monitoring session with a 30s waitForSessionFree(agentId) wait
(registry now exposes onSessionEnd) so cron/hook contention does not
reject a user-chat send outright. Returns 503 if the wait times out.
Client
- New lib/attachments.ts: validateAttachment / compressImageIfNeeded
(canvas downscale to 2048px long edge, JPEG 0.85 re-encode for >1.5
MB inputs) / stageAttachment / stageAttachments that produces the
staged-attachment shape the composer renders and the payload the
server accepts.
- ConversationInput: drag-and-drop, paperclip button, clipboard paste,
staged attachment chip strip with thumbnails for images and a
paperclip+name chip for text files. Send button enables on either
text or attachments. Drop-zone overlay during drag.
- chatWithAgent forwards attachments[]; useAgentConversation.send
accepts a SendInput shape and renders user attachments on the
optimistic streaming turn via MessageAttachments / MessageAttachment.
- ClawChatMessage groups historical attachment parts into a single
MessageAttachments strip, ordered before reasoning/tools/text.
- claw-chat-types adds an attachment ClawChatMessagePart variant; the
history mapper emits attachment parts first and skips the text part
when the user only sent media.
- AgentCommandHome forwards the new SendInput shape — home composer
drops attachments at the boundary in v1 (the conversation page is
where staging is most useful; carrying bytes through the URL bar
is not sensible).
Limits: 10 attachments per message, 5 MB per image (post compression),
1 MB per text file, mime types png/jpeg/webp/gif and text/* +
application/json. PDFs and other binaries are deferred to v2.
* feat(agent): outbound message queue for chats while agent is mid-turn
Lets users keep typing and submitting messages while the agent is still
streaming a previous turn. Each press is appended to a single-flight
queue and dispatched as soon as `streaming` flips false; the queued
state renders as a strip above the composer so the user sees what's
pending vs. what's already sending.
- New `useOutboundQueue` hook owns the queue, the worker effect, and
cancel/retry actions. Single-flight by design — a re-entrancy ref
guard prevents two simultaneous dispatches when `streaming` flickers.
- Composer (`ConversationInput`) accepts optional `outboundQueue`,
`onCancelQueued`, `onRetryQueued` props. When the queue is provided
the send-button gate stops blocking on `streaming`; the spinner stays
as the visual cue that the agent is still busy. Legacy direct-send
callers keep the old streaming-blocks-send semantic.
- Renders an OutboundQueueStrip above the staged-attachment strip with
per-item status (queued / sending / failed), a cancel button on
queued items, and retry + discard on failed items.
- AgentCommandConversation wires `onSend` to `queue.enqueue` and routes
the home composer's `?q=` initial-message handoff through the queue
too, so it inherits the same single-flight serialization.
The server-side `waitForSessionFree` (added with attachments) and this
client-side queue together cover both contention sources: cron / hook
turns and back-to-back user sends. Persistence across reloads is
intentionally out of scope for v1 — losing the queue on extension
reload is documented as a known limitation.
* feat(server): server-side outbound message queue
Replaces the client-only React-state queue from 123ef21d with a
proper server-owned queue. Closing the tab is now safe — the server
holds queued messages and dispatches them through the existing
chatStream path the moment the agent's ClawSession status flips to
idle.
Server
- New OutboundQueueService (apps/server/src/api/services/queue) — per
agent FIFO, in-memory. Subscribes to ClawSession.onStateChange
through OpenClawService.onAgentStatusChange, and dispatches via
OpenClawService.chatStream so attachments / history / monitoring
all behave identically to the existing /chat route. The worker
drains the SSE response server-side so the gateway run finalizes
cleanly even with no client connected.
- Four new routes under /claw/agents/:id/queue:
POST /queue enqueue
DELETE /queue/:itemId cancel a queued item
POST /queue/:itemId/retry re-queue a failed item
GET /queue/stream SSE feed of the per-agent queue state.
Validation reuses validateChatAttachments and
buildMessagePartsFromAttachments from the existing chat route.
- Singleton wired in apps/server/src/main.ts; shutdown on SIGTERM.
- New OpenClawService.getAgentState getter for the queue worker's
pre-dispatch sanity check.
Client
- useOutboundQueue rewritten as an SSE-backed projection over server
state. Public API unchanged so the composer still works.
- enqueue POSTs to /queue and shows an optimistic local entry until
the server's SSE snapshot reflects it; local-only entries get a
`local-` id prefix so cancel can short-circuit them without
hitting the server.
- AgentCommandConversation watches the queue for sending items
dropping out and refetches history so the new assistant turn shows
up in the conversation view (the server worker streams the
dispatched turn into OpenClaw without exposing per-turn SSE to
the client).
Out of scope (documented in the plan as v2 follow-ups): disk
persistence (server restart loses queue), per-turn live streaming
of queued sends in the conversation view, and switching the
underlying dispatch from /v1/chat/completions to the chat.send RPC
(which would also fix the multimodal attachment routing problem).
* fix(server): outbound queue must reuse existing session, not spawn UUIDs
The queue worker was generating a fresh randomUUID() as the sessionKey
when the queued item didn't carry one — and the client wasn't sending
one. Result: every queued message kicked off a brand-new OpenClaw
session, orphaning the user's active conversation behind the new
"most recent" entry in sessions.json. The history endpoint then
resolved to the orphan and the chat appeared to disappear.
Fix is layered:
- Client (useOutboundQueue): forward the current resolvedSessionKey
in the POST /queue body so every queued message targets the same
conversation the user is viewing. AgentCommandConversation passes
resolvedSessionKey into the hook.
- Server (OutboundQueueService): the worker now resolves to the
agent's existing user-chat session when no sessionKey is provided
on the queued item, via OpenClawService.resolveAgentSession. UUID
fallback is now reserved for the first-ever message on a brand
new agent — same semantic the existing /chat route has implicitly
through the catalog of historical sessions.
No JSONL data was lost by the original bug (the prior conversations
are intact on disk); the orphan sessions just shadowed the original
in sessions.json.
* fix(agent,server): address PR review feedback for chat queue
- Tighten image data URL cap to base64-aware ~6.7 MB (was ~7.5 MB
through `MAX_IMAGE_BYTES * 2`).
- Forward chat history from useOutboundQueue.enqueue so queued sends
preserve conversation context like direct sends do.
- Match local attachment previews to server snapshots by id (not by
message text), and prune the preview map as items drain.
- Pass an AbortSignal into chatStream so a queue shutdown cancels the
initial OpenClaw handshake, not just the SSE drain loop.
- Track previously gitignored apps/agent/lib/attachments.ts (was caught
by global lib/ ignore) so CI typecheck can resolve @/lib/attachments.
- Update server-api openclaw route tests to the new chatStream signature
and the waitForSessionFree-based busy-agent path.
* fix(agent): dedupe optimistic queue entries for text-only sends
The localId↔serverId map was only populated when the message had
attachments, so plain-text sends left the optimistic local entry in
place after the server snapshot arrived — the user saw the same
message rendered twice in the queue strip.
* fix(agent): prune optimistic queue entry on POST ack, not just SSE
The server broadcasts the new queue snapshot before its POST response
returns, so the SSE handler often runs first — at that point the
localId↔serverId map has no entry for the new server id yet, so the
SSE-based dedupe path can't drop the optimistic local entry. Pruning
on POST success closes the race deterministically.
* fix(agent): hand off optimistic queue entry without a render gap
Pruning the local entry on POST success only worked when the SSE
snapshot had already overwritten it; if the POST response landed
first, the optimistic row disappeared for a frame before the SSE
snapshot brought back the server-keyed row, producing a visible
flicker. Gate the POST-side prune on the SSE snapshot already
carrying the server id, and rely on the SSE-based dedupe (now
guaranteed to find the localId↔serverId link in the map) to clean
up when SSE arrives later.
* fix(agent,server): client-generated queue id eliminates render flicker
The server used to assign its own UUID when an item was enqueued, so
the optimistic client row carried a `local-` id while the SSE snapshot
carried a server UUID — the client had to wait for the POST response
to learn the mapping before it could dedupe, and during that window
both rows rendered.
Now the browser generates the id, sends it in the POST body, and the
server uses it verbatim (falling back to a fresh UUID only if the id
collides with an existing item). The client collapses to a single
id-keyed list, so the optimistic row and the SSE row reconcile on the
same key from the very first render.
* feat: pass per-turn cost and token data through chat history items
- Add costUsd, tokensIn, tokensOut to BrowserOSChatHistoryItem (server)
- Pass through from JSONL agent.message events in jsonlEventsToHistoryItems()
- Add same fields to client-side BrowserOSChatHistoryItem and ClawChatMessage
- Map cost/token data in mapHistoryItemToClawMessage()
Data flows: JSONL message.usage → server history item → API response →
client ClawChatMessage. Available for rendering in ClawChatMessage
component (message toolbar, cost badges).
* feat: add message toolbar with copy button and per-turn cost display
Add MessageToolbar to historical assistant messages in ClawChatMessage:
- Copy button copies message text to clipboard via MessageAction
- Per-turn token count (22.7K → 238) and cost ($0.003) shown as muted
tabular-nums text on the right side of the toolbar
- Toolbar appears on hover (opacity transition via group-hover)
- Only shown when the message has text content
- Cost/token display only shown when data is available from JSONL
* fix: toolbar only on assistant messages, always visible, cost only
- Only render toolbar on assistant messages (not user messages)
- Remove hover-only opacity — toolbar is always visible
- Remove token counts (22.7K → 238 is meaningless to users)
- Show only cost as a budget signal ($0.003)
* feat: group all tool activity into single Task collapsible per turn
Replace flat tool rows with a single ai-elements Task collapsible per
assistant turn that lists every tool/MCP call in sequence.
Live streaming (ConversationMessage):
- Aggregate all tool-batch parts into one Task
- Title: "Working… (N actions)" while running, "Agent activity (N actions)" when done
- Default open while turn is in progress
- Wrench icon in trigger
Historical (ClawChatMessage):
- Group all tool-call parts into one Task
- Title includes failed count if any tools errored
- Default collapsed — expandable on click
- Tool name + status icon + error text per row
Both views show one clean collapsible per turn instead of N individual
tool cards. Collapsed reads "5 actions"; expanded shows the timeline.
* feat: include tool calls in chat history responses
Server: jsonlEventsToHistoryItems() now walks ALL events (not just
messages) and pairs agent.tool_use with agent.tool_result by toolCallId.
The resulting tool call list is attached to the next assistant text
message as toolCalls[]. Each entry includes status, input arguments,
output text, error string, and duration computed from event timestamps.
Client:
- BrowserOSChatHistoryItem gets optional toolCalls field
- Tool-call message part type gets durationMs field
- mapHistoryItemToClawMessage() emits tool-call parts BEFORE the text
part (the order the agent produced them)
- ClawChatMessage Task view now shows tool duration in seconds
Result: historical messages now display the full tool activity
timeline grouped into the single Task collapsible per turn (designed
in step 3), instead of showing only the final text response.
* feat: render activity rows as human verbs sourced from tool registry
Tool calls in the chat activity view now read as sentences:
"Opened tab · news.ycombinator.com" instead of "browseros__new_page".
Server (tool-label-registry.ts):
- Curated verb override map for ~70 BrowserOS first-party tools
- Per-tool subject extractors that pull the meaningful argument from
input (URL → host, query → quoted, element → ID, etc.)
- Generic fallback humanizes snake_case for any unmapped tool
- Strips MCP namespace prefixes (browseros__, mcp_)
Server (openclaw-service.ts):
- jsonlEventsToHistoryItems calls buildToolLabel for each tool_use,
attaches label and subject to the BrowserOSChatHistoryToolCall
Client:
- Mirrored label module at lib/tool-labels.ts
- useAgentConversation tool-start handler computes label/subject
from the SSE tool args
- ClawChatMessage and ConversationMessage render label · subject
with foreground/muted styling, no font-mono
- ToolEntry, BrowserOSChatHistoryToolCall, and tool-call message
part types all carry label and optional subject
* fix: drop meaningless tab N subject from page-read tool rows
Page IDs are internal numbers, not URLs. 'Took screenshot · tab 4'
tells the user nothing. Removed subject extractors for take_snapshot,
take_enhanced_snapshot, get_page_content, get_page_links, get_dom,
and take_screenshot. The verb alone is the right signal.
* fix: gate initial loading on historyQuery.isFetched not isLoading
The session and history queries are sequential: the history query is
disabled until session resolves. After session resolves, there's a render
frame where historyQuery.isLoading is still false (the query hasn't
been kicked off yet). isInitialLoading flipped to false during that
window, exposing an empty chat shell with just Task collapsibles and
copy buttons before the messages filled in.
Switching the guard to isFetched closes that window — the loading state
stays true until the first history fetch actually completes.
* fix: render historical messages immediately instead of through Streamdown's idle-callback debounce
Streamdown defaults to mode="streaming" which uses requestIdleCallback (300ms
debounce, 500ms idle timeout) and lazy/Suspense to optimize for token-by-token
live streams. For finalized historical messages this caused tool collapsibles
and copy buttons to paint while text bodies stayed blank for ~300-500ms after
load. Pass mode="static" + parseIncompleteMarkdown=false on the historical
MessageResponse so completed text paints in the same frame as the surrounding
chrome. Live streaming turns still use the default streaming mode.
Also collapse the redundant /agents/:id/session round-trip into the existing
/history endpoint (server already resolves the most recent user-chat session
when sessionKey is omitted) and tighten the initial-loading gate to stay true
across the render frame where the query is enabled but hasn't started fetching.
* feat: surface thinking duration on historical reasoning collapsibles
Server accumulates agent.thinking events per turn from JSONL and attaches a
single reasoning block (joined text + durationMs from first thinking event
to the closing agent.message) on each assistant history item. Reasoning
buffer resets on user.message alongside the tool-call buffer.
Client mirrors the type, emits the reasoning part before tool calls in
mapHistoryItemToClawMessage (chronological: think → act → answer), and
passes duration in seconds to <Reasoning> so the trigger reads "Thought
for N seconds" instead of just "Thinking" on collapsed historical turns.
* fix: read thinking blocks from the correct JSONL field name
OpenClaw stores reasoning blocks as {type:'thinking', thinking:'...'} but
the JSONL parser was reading block.text, so every thinking event was
silently dropped before it ever reached jsonlEventsToHistoryItems. As a
result the reasoning field on history items was always empty even though
the new accumulator was wired up correctly.
Also guard the client mapping: when durationMs is 0 (think + answer
emitted in the same JSONL line, no real elapsed wall-clock) pass
undefined to <Reasoning> so it renders the static "Thinking" trigger
instead of the streaming shimmer / "Thought for 0 seconds".
* fix: reset reasoning buffer on discarded turns and drop dead session hook
Two cleanups from PR review:
1. jsonlEventsToHistoryItems: when an agent.message is discarded (the
"[Chat messages since your last reply" wrapper without a current-message
marker) the tool buffers were already reset but the reasoning buffer
was not. Accumulated thinking from the discarded turn would bleed onto
the next assistant message. Reset pendingReasoningTexts and
pendingReasoningFirstAt alongside the tool buffers.
2. useClawAgentSession, the AgentSessionResponse type, and the unused
session entry in CLAW_CHAT_QUERY_KEYS became dead code after the
session round-trip was folded into the history endpoint. Removed.
* feat: draft agent chat ui exploration
* feat: refine agent chat ui draft
* feat: remove outer frame from agent chat workspace
* fix: offset agent chat for app sidebar
* fix: simplify agent conversation shell
* fix: remove redundant chat header actions
* fix: unify agent conversation headers
* fix: tighten agent chat spacing
* fix: bound agent chat composer height
* fix: remove agent chat page inset
* fix: align agent header height with sidepanel
* fix: center agent composer resting state
* fix: anchor multiline composer controls
* fix: remove focus grid from agent home
* fix: remove redundant agent home header
* fix: constrain home agent composer
* fix: match home composer default posture
* feat: add openclaw chat history APIs
* feat: add claw chat history hydration
* fix: stabilize claw chat viewport layout
* fix: use conversation scroll base for claw chat
* refactor: split claw chat controller responsibilities
* fix: keep active agent turns in memory
* fix: normalize openclaw chat sessions
* refactor: use HTTP client for agent history instead of CLI client
Replace the CLI-based getChatHistory() call in getAgentHistoryPage()
with the HTTP client's getSessionHistory() from PR #795. This uses
the direct HTTP transport to OpenClaw's /sessions/<key>/history
endpoint instead of shelling out through the CLI.
- Add filterHttpSessionHistoryMessages() for flat-string content format
- Add normalizeHttpHistoryMessages() for OpenClawSessionHistoryMessage shape
- Update getAgentHistoryPage() to call getSessionHistory() via httpClient
- Remove unused getChatHistory(), filterOpenClawSystemMessages(),
normalizeChatHistoryMessages(), and getTextContent()
- Update test mocks from cliClient.getChatHistory to httpClient.getSessionHistory
- Update MutableOpenClawService type: chatClient -> httpClient
* fix: fetch all session messages by iterating OpenClaw pagination
OpenClaw's HTTP history endpoint returns a limited page by default.
When called without a limit, only the first ~27 messages were returned,
causing all newer conversation messages to be silently dropped.
Add fetchAllSessionMessages() that iterates through OpenClaw's cursor-
based pagination (200 messages per page) until hasMore is false, then
feeds the complete message list into the existing BrowserOS normalization
and in-memory pagination layer.
* refactor: migrate chat history from HTTP gateway to direct JSONL file reads
Replace the HTTP-based chat history pipeline (BrowserOS server → OpenClaw
gateway /sessions/:key/history pagination loop) with direct JSONL file reads
from the host filesystem via Lima's virtiofs mount.
- Add OpenClawJsonlReader that reads session JSONL files directly from
~/.browseros/vm/openclaw/.openclaw/agents/<id>/sessions/
- Replace fetchAllSessionMessages() HTTP pagination with single file read
- Replace CLI-based listSessions() with sessions.json file reads
- Make listSessions, resolveAgentSession, getAgentHistoryPage synchronous
- Remove unused toBrowserOSSession, filterHttpSessionHistoryMessages,
normalizeHttpHistoryMessages helpers
- Update route handlers to drop unnecessary async/await
- Update tests to use temp JSONL files instead of mocked HTTP/CLI clients
* fix: restore async route handlers for test compatibility with mocked service
* fix: address review feedback — path traversal guard, lazy reader, exists flag
- Add safePath() to OpenClawJsonlReader that validates resolved paths stay
within stateRoot, preventing path traversal via crafted agentId values
- Use lazy initialization for jsonlReader (nulled on rebuildRuntimeClients)
instead of creating a new instance per property access
- Return exists: false from resolveSpecificAgentSession when no session
matches instead of fabricating a ghost session with sessionId: ''
* feat: add dashboard API and enrich home page agent cards
Server:
- Add summarizeToolActivity() that converts tool events into natural
language descriptions ("Browsed 3 pages, took 2 screenshots")
- Add getDashboard() to OpenClawService that aggregates per-agent stats
from JSONL: latest message, activity summary, cost, session count
- Add GET /claw/dashboard endpoint
Client:
- Add useAgentDashboard() React Query hook (10s refetch, 5s stale)
- Rewrite useAgentCardData from async IndexedDB hook to pure
buildAgentCardData() function merging agent entries with dashboard data
- Add activity summary and cost to AgentCardExpanded footer
- Add activitySummary and costUsd fields to AgentCardData type
- Remove IndexedDB dependency from the home page
* feat: add OpenClawObserver for real-time per-agent status via gateway WS
- Add OpenClawObserver that connects to the OpenClaw gateway WebSocket
control plane and subscribes to chat broadcast events
- Track per-agent status in real time: working (streaming), idle (turn
complete), error (run failed), with current tool name
- Auto-connect when gateway control plane becomes available, auto-
reconnect on disconnect with 5s backoff
- Disconnect observer on stop/shutdown
- Wire live status + currentTool into getDashboard() response
- Update client: AgentOverview includes status + currentTool, card shows
spinning loader + tool name when agent is working
- Status resolution: per-agent WS status takes precedence over gateway-
level status for working/error states
* feat: add SSE dashboard stream for real-time agent status on home page
Server:
- Add GET /claw/dashboard/stream SSE endpoint that sends an initial
snapshot then pushes per-agent status events as they arrive from
the OpenClaw observer
- Add onAgentStatusChange() to OpenClawService exposing the observer's
listener for the route layer
- Heartbeat every 15s to keep connections alive
Client:
- useAgentDashboard() now subscribes to EventSource at /claw/dashboard/stream
- SSE snapshot event hydrates the React Query cache immediately
- SSE status events patch individual agent status + currentTool in the
cache without refetching — agent cards update instantly
- Polling fallback raised to 30s since SSE handles real-time
* fix: observer WS handshake — wait for challenge before sending connect
The OpenClaw gateway sends a connect.challenge event before accepting
the connect request. The observer was sending the connect request on
ws.open which raced with the challenge. Now waits for the challenge
event before sending the handshake.
Also add dangerouslyDisableDeviceAuth to the gateway setup config
batch so the observer can connect without device identity on new
installs.
* fix: JSONL reader falls back to most recent file when sessions.json is stale
OpenClaw's sessions.json can record a Pi session ID that doesn't match
the actual JSONL filename on disk. This happens after context compaction
or session restart — the JSONL file gets a new UUID but sessions.json
keeps the old one.
Previously this caused history to silently disappear (the reader tried
to open a non-existent file and returned empty). Now resolveJsonlPath()
checks if the mapped file exists and, when it doesn't, scans the
sessions directory for the most recently modified .jsonl file as a
fallback.
* feat: add ClawSession state machine for reliable per-agent status
The OpenClawObserver only knows about status changes it witnesses via
WS events. If an agent was already running when the observer connected,
or after a reconnect, statuses were stuck at "unknown".
ClawSession is an in-memory state machine that solves this:
1. Seeds from JSONL on first control plane call — reads the latest
events for each agent and infers working/idle. A session is "working"
if the last event is a user.message with no subsequent agent.message,
or an agent.tool_use with no matching agent.tool_result.
2. Receives live transitions from the WS observer — the observer now
delegates all state management to ClawSession instead of maintaining
its own status map.
3. Applies a 5-minute staleness threshold — if the last JSONL event
is older than 5 minutes, assume idle (handles agent crashes).
Consumers (SSE stream, dashboard endpoint) read from ClawSession and
get correct state from the first call — no "unknown" period.
* fix: remove staleTime so dashboard refetches on every mount
* fix: reset stale working status on WS disconnect, eliminate redundant JSONL reads
- Observer resets all "working" agents to "unknown" when the WS closes,
preventing agents from appearing stuck as Working indefinitely after
a gateway restart. ClawSession re-seeds correct state on reconnect.
- getDashboard() now derives latestAgentMessage and cost from the
already-loaded events array for the latest session instead of calling
latestAgentMessage() and getSessionStats() which each re-read the
same JSONL file. Reduces file reads from 3x to 1x per agent.
* feat: add runtime vm cache sync
* feat: configure runtime vm cache sync
* feat: prefetch vm cache on startup
* feat: await vm cache before vm startup
* fix: recheck vm cache after prefetch wait
* fix: address vm cache review feedback
* build(server): require VM cache manifest env
* feat(openclaw): add Claude CLI as a CLI-backed provider
Extensible registry of "OpenClaw CLI-backed providers" — tools that run
as subprocesses inside the gateway container rather than via an API key.
Claude CLI is the first entry; Gemini CLI / Codex CLI / etc. are
one-line additions in the same shape.
Backend:
- New openclaw-cli-providers/ module: types, registry, claude-cli entry.
- OpenClawService: generic ensureAllCliProvidersInstalled() (runs on
setup/start/restart/auto-start) and getCliProviderAuthStatus(provider).
- Provider dispatch: resolveProviderForAgent() short-circuits CLI
providers (no env var, no custom-provider merge) before falling
through to the API-key resolver. No changes to openclaw-provider-map.
- Container runtime: PATH + NPM_CONFIG_PREFIX env so tools installed
under /home/node/.npm-global/bin (mounted) are discoverable by
OpenClaw's child-process spawns and persist across restarts.
- New route: GET /claw/providers/:providerId/auth-status returns
installed / loggedIn / account / plan / error.
Frontend:
- New openclaw-cli-providers.tsx: mirrors backend registry (id, models,
authLoginCommand), useOpenClawCliProviderAuthStatus hook (2-s poll
while enabled), OpenClawCliProviderStatusPanel component.
- AgentsPage: synthesized CLI-provider options merged into the Create
Agent dropdown, inline status panel, auth modal mounting the existing
AgentTerminal with provider.authLoginCommand, auto-close on loggedIn.
- AgentTerminal: new optional initialCommand + onSessionExit props
(ref-based so parent re-renders don't rebuild the PTY).
No global ProviderType changes. No custom container image — runtime
install into the mounted home dir persists across restarts.
* fix(openclaw): address review comments for claude-cli provider
- Drop redundant providerId field from OpenClawCliProviderOption (type
already carries the same value).
- Reuse SetupInput type in resolveProviderForAgent instead of inlining.
- Split ensureCliProviderInstalled into probe + install so logs
distinguish "already present" from "freshly installed".
- Narrow union in handleCreate via explicit LlmProviderConfig cast; the
'in'-based narrowing stopped working once the two option shapes
overlapped on required fields.
* fix: green up server-api tests after claude-cli additions
- Update container-runtime.test.ts snapshot to include the new
PATH + NPM_CONFIG_PREFIX env args.
- Add a defensive guard in ensureAllCliProvidersInstalled so test
mocks that swap runtime for a partial stub without execInContainer
simply skip the install step; production runtime always provides it.
No production behavior change.
* fix(openclaw): use claude /login for auth flow and render terminal full-page
`claude auth login` in 2.1.x silently discards stdin, so the pasted OAuth
code never reaches claude. Switch to the REPL's `/login` slash command,
which does accept a pasted token. Also render the auth terminal
full-page instead of inside a Radix Dialog — the focus trap was hiding
keyboard events from xterm's helper textarea. Finally, guard the async
WebSocket in AgentTerminal against React 18 StrictMode's double-invoke
so the first mount's orphaned WS doesn't leak a second live session.
- terminal-session: pass PATH on podman exec so user-installed CLIs
resolve in interactive sessions without manual re-exports.
- claude-cli parseAuthStatus: treat exit-code-1 as a valid "not logged
in" JSON payload instead of a hard error.
* fix(openclaw): drop unnecessary PATH override on podman exec
`podman exec` inherits the container's run-time env (PATH includes
/home/node/.npm-global/bin via `podman run -e PATH=…`), so the extra
`-e PATH` on the exec call was redundant. Reverts the export of
GATEWAY_PATH and the exec flag added in the previous commit.
* feat(openclaw): show CLI-backed providers in Set Up dialog
The Set Up OpenClaw dialog previously listed only API-key LLM
providers. Add the CLI-backed ones (currently just Claude CLI) so
users can bootstrap the gateway with a Claude.ai-subscription-backed
agent without round-tripping through the Create Agent flow first.
When the user picks a CLI provider at setup, skip the apiKey/baseUrl
fields and open the auth terminal immediately after the gateway comes
up, so /login runs in one click.
* fix(openclaw): robust claude auth-status parsing and cleaner CLI UX
parseClaudeAuthStatus was doing JSON.parse on the entire stdout, which
fails when Lima/nerdctl appends a stderr line like `level=fatal
msg="exec failed with exit code 1"` whenever the inner command exits
non-zero (claude auth status exits 1 when not logged in). The panel
then surfaced the raw output as an error. Switch to a line-by-line
scan that picks the first parseable JSON object — handles trailing
noise and nested JSON fields cleanly.
UI polish around the Setup dialog:
- Hide the "uses your API key" hint when the selected provider is
CLI-backed — it is inaccurate and confusing.
- When a CLI provider is picked in Setup, show a short helper line
instead of the status panel (the /auth-status poll would be
pre-gateway and would always fail). Set Up & Start boots the
gateway and then auto-opens the auth terminal in one click.
- Track the active CLI provider across both Setup and Create dialogs
so the auth terminal opens for the right provider regardless of
which dialog triggered it.
* feat(terminal): make selection + copy work under TUI mouse tracking
Interactive TUIs like `claude /login` enable xterm mouse-tracking,
which forwards every click to the app and disables click-drag text
selection. Our terminal had no escape hatch, so users couldn't grab
the OAuth URL.
Three general-purpose fixes (none CLI-specific):
- macOptionClickForcesSelection: Opt+drag always selects on Mac,
regardless of what the running program does with mouse events.
- Cmd/Ctrl+A and Cmd/Ctrl+C custom key handler: select-all and copy
to clipboard via navigator.clipboard, even when the TUI would
swallow the keys.
- Copy button in the terminal header: writes the current selection
to the clipboard, or the full visible viewport if nothing is
selected. One-click escape hatch that works in every state.
Applies to any interactive CLI in our terminal (sudo, vim, claude,
gh auth, etc.), not just the claude login flow.
* fix(terminal): make xterm selection actually visible
Selection was registering internally (xterm-selection layer had
correct width/height rects), but the rectangles rendered in
rgb(252,252,251) — practically invisible against the white
background — so users concluded selection was broken.
Root cause: the theme derived selectionBackground from
`withAlpha(resolveCssColor('--accent-orange'), 0.2)`. When the CSS
var failed to resolve it fell back near-white, and the alpha
compositing against the page background made the result
indistinguishable from the background.
Switch to solid terminal-standard selection colors (VSCode-like
light-blue / dark-indigo). Also set selectionInactiveBackground so
the selection persists when focus moves away (useful while copying).
Drop the now-unused withAlpha helper.
* fix(openclaw): handle pretty-printed JSON in claude auth status parser
claude auth status --json emits multi-line pretty-printed JSON. The previous line-by-line parser never matched, so the UI treated every response as an error and surfaced the raw JSON — even when loggedIn was true. Replace with a brace-matching JSON extractor (string- and escape-aware) that tolerates multi-line JSON, leading banners, trailing lima/nerdctl stderr, and nested objects.
* refactor(openclaw): separate exec streams, argv installs, cleaner async cleanup
Audit-driven cleanup. Net -42 lines, four concrete issues fixed:
1. ContainerRuntime.runInContainer() exposes {exitCode, stdout, stderr}
from the nerdctl exec (ContainerCli.runCommand already tracked them
separately; we were just throwing stderr into the same string). The
40-line hand-rolled brace-matching JSON extractor in claude-cli.ts
existed only because the prior merged-stream output had lima/
nerdctl's 'level=fatal' line fused with claude's JSON. parser is
now JSON.parse(stdout.trim()).
2. Replace shell-based 'sh -lc "npm install -g ${pkg}@latest"' with
argv: execInContainer(['npm','install','-g','${pkg}@${version}']).
Registry values no longer flow through a shell (removes injection
surface from future CLI providers). Pinned version instead of
@latest (adds npmPackageVersion to the provider type).
3. AgentTerminal: replace the 'let cancelled' + out-of-effect
disposeSocketBindings pattern with an AbortController scoped to
the effect and a cleanups[] array. Matches the canonical React 18
async-effect pattern — no partial-cleanup race if StrictMode
unmounts between the async await and the resolve.
4. AgentTerminal: drop the full-buffer fallback in the Copy button
(was copying all 8000 scrollback lines when nothing selected —
surprising). Button now only copies the actual xterm selection,
or no-ops silently. Users who want everything can Cmd+A first.
* feat: draft agent chat ui exploration
* feat: refine agent chat ui draft
* feat: remove outer frame from agent chat workspace
* fix: offset agent chat for app sidebar
* fix: simplify agent conversation shell
* fix: remove redundant chat header actions
* fix: unify agent conversation headers
* fix: tighten agent chat spacing
* fix: bound agent chat composer height
* fix: remove agent chat page inset
* fix: align agent header height with sidepanel
* fix: center agent composer resting state
* fix: anchor multiline composer controls
* fix: remove focus grid from agent home
* fix: remove redundant agent home header
* fix: constrain home agent composer
* fix: match home composer default posture
* feat: add openclaw chat history APIs
* feat: add claw chat history hydration
* fix: stabilize claw chat viewport layout
* fix: use conversation scroll base for claw chat
* refactor: split claw chat controller responsibilities
* fix: keep active agent turns in memory
* fix: normalize openclaw chat sessions
* refactor: use HTTP client for agent history instead of CLI client
Replace the CLI-based getChatHistory() call in getAgentHistoryPage()
with the HTTP client's getSessionHistory() from PR #795. This uses
the direct HTTP transport to OpenClaw's /sessions/<key>/history
endpoint instead of shelling out through the CLI.
- Add filterHttpSessionHistoryMessages() for flat-string content format
- Add normalizeHttpHistoryMessages() for OpenClawSessionHistoryMessage shape
- Update getAgentHistoryPage() to call getSessionHistory() via httpClient
- Remove unused getChatHistory(), filterOpenClawSystemMessages(),
normalizeChatHistoryMessages(), and getTextContent()
- Update test mocks from cliClient.getChatHistory to httpClient.getSessionHistory
- Update MutableOpenClawService type: chatClient -> httpClient
* fix: fetch all session messages by iterating OpenClaw pagination
OpenClaw's HTTP history endpoint returns a limited page by default.
When called without a limit, only the first ~27 messages were returned,
causing all newer conversation messages to be silently dropped.
Add fetchAllSessionMessages() that iterates through OpenClaw's cursor-
based pagination (200 messages per page) until hasMore is false, then
feeds the complete message list into the existing BrowserOS normalization
and in-memory pagination layer.
* refactor: migrate chat history from HTTP gateway to direct JSONL file reads
Replace the HTTP-based chat history pipeline (BrowserOS server → OpenClaw
gateway /sessions/:key/history pagination loop) with direct JSONL file reads
from the host filesystem via Lima's virtiofs mount.
- Add OpenClawJsonlReader that reads session JSONL files directly from
~/.browseros/vm/openclaw/.openclaw/agents/<id>/sessions/
- Replace fetchAllSessionMessages() HTTP pagination with single file read
- Replace CLI-based listSessions() with sessions.json file reads
- Make listSessions, resolveAgentSession, getAgentHistoryPage synchronous
- Remove unused toBrowserOSSession, filterHttpSessionHistoryMessages,
normalizeHttpHistoryMessages helpers
- Update route handlers to drop unnecessary async/await
- Update tests to use temp JSONL files instead of mocked HTTP/CLI clients
* fix: restore async route handlers for test compatibility with mocked service
* fix: address review feedback — path traversal guard, lazy reader, exists flag
- Add safePath() to OpenClawJsonlReader that validates resolved paths stay
within stateRoot, preventing path traversal via crafted agentId values
- Use lazy initialization for jsonlReader (nulled on rebuildRuntimeClients)
instead of creating a new instance per property access
- Return exists: false from resolveSpecificAgentSession when no session
matches instead of fabricating a ghost session with sessionId: ''
* feat(build-tools): seed dev agent tarballs
* fix: address review comments for 0423-build_agent_tarball_dev_sync
* chore(build-tools): remove dev cache sync alias
Replace the podman-based runtime with nerdctl running inside the Lima
VM introduced in the previous commit. OpenClaw is cut over to the new
VM-backed container runtime; legacy podman code paths are removed.
- New container CLI (lib/container): nerdctl ContainerCli, ImageLoader
with cache-tarball fallback, shared types
- OpenClaw: container-runtime-factory orchestrates VM lifecycle + gateway
startup; container-runtime.ts rewritten to speak nerdctl; Linux test
startup kept disabled behind the factory
- Terminal: session + routes moved onto Lima shell transport; server
wires the VM-backed runtime via main.ts
- Agent UI: simplify AgentsPage/useOpenClaw after route consolidation
- Remove podman-runtime, podman-overrides, and their tests
- Tests: container-cli, image-loader, container-runtime-factory, and
updated openclaw/terminal/main suites
Introduce a new VM runtime layer using Lima for running containerised
workloads on macOS. Lifecycle covers decompress/create/start/stop with
stubs for upgrade/reset plus version-mismatch warnings.
- Foundation modules: paths, errors, manifest, telemetry
- lima.yaml generator + typed limactl wrapper with structured debug logging
- ssh ControlMaster transport for fast in-VM commands
- Ubuntu 24.04 minimal template, containerd default, 30GiB overlay disk
- browseros-dir helpers (getLimaHomeDir, getVmStateDir, getVmDisksDir);
OpenClaw dir moves into VM state dir
- Test helpers (fake-limactl, fake-ssh, test-env), vm-smoke integration
coverage, NODE_ENV propagation for spawned server test groups
* refactor(openclaw): rename http chat client to http client
Session history is about to land on the same HTTP client. 'Chat client'
will no longer describe it, so rename the class, file, and service field
up front. No behavior change.
* feat(openclaw): add session history fetch + sse stream to http client
Adds getSessionHistory (JSON) and streamSessionHistory (SSE) to the
OpenClaw HTTP client. Both target GET /sessions/<key>/history on the
loopback gateway, reusing the same bearer-token auth as streamChat.
- 404 from the gateway surfaces as OpenClawSessionNotFoundError so
callers can map it to a typed HTTP status.
- The SSE path parses named 'history', 'message', and 'error' events
into a typed OpenClawSessionHistoryEvent union.
- AbortSignal propagates to fetch and cancels the reader mid-stream.
* feat(openclaw): expose session history over GET /claw/session/:key/history
Wire the new getSessionHistory / streamSessionHistory service methods
through a route that defaults to JSON and upgrades to SSE when the
client sends Accept: text/event-stream.
- OpenClawSessionNotFoundError lives in errors.ts alongside the other
OpenClaw errors so routes can import it from one place.
- The route propagates c.req.raw.signal into streamSessionHistory so
client disconnects cancel the upstream fetch.
- Route tests cover the JSON path (with query param forwarding), the
404 path, and the SSE framing.
* chore(openclaw): drop NaN from session history route limit param
Seeds ~/.browseros-dev/cache/vm/ from ./dist/ without touching R2, so
devs can test the server against a freshly-built tarball before anything
is published to cdn.browseros.com. Hardcodes arm64 since all devs are on
Apple Silicon; refuses to run unless NODE_ENV=development; idempotent
(skips copy on sha256 match).
Also fixes the R2_BUCKET default in .env.sample from browseros-artifacts
to browseros to match the actual bucket.
* feat(build-tools): add Lima template for BrowserOS VM
* feat(build-tools): remove build-disk pipeline and recipe directory
Task 2 verification removed the scripts, recipe directory, workflow, and package scripts. Typecheck remains green here because manifest disk fields are removed in the next task, so the plan's expected missing-import failure does not apply yet.
* feat(build-tools): rename VmManifest to AgentManifest, drop disk fields
* feat(build): stage Lima template into server resources
Verified local-resource staging with: bun scripts/build/server.ts --target=darwin-arm64 --ci. The template was copied to dist/prod/server/darwin-arm64/resources/vm/browseros-vm.yaml and included in the zip. bun run build:server:test still fails on the pre-existing R2 limactl resource with: The specified key does not exist.
* docs(build-tools): Lima template dev loop + record D9
Updated the build-tools README in this worktree. Also recorded D9 in the canonical external spec file at /Users/shadowfax/llm/code/browseros-project/grove-ref/browseros-main/specs/decisions.md, which is outside this git checkout.
* chore(build-tools): sweep orphaned references to retired disk pipeline
* chore: self-review fixes
* feat(vm-container): ship the WS1 VM disk image pipeline
New Bun/TS workspace package @browseros/vm-container that produces a
reproducible, versioned Debian 12 + Podman qcow2 disk image for arm64 and
x64, and publishes it to Cloudflare R2 under vm/<version>/ with a per-
version manifest.json and a latest.json pointer.
- virt-customize-driven build with a git-tracked recipe DSL.
- zstd-compressed artifacts; sha256 sidecars for compressed + uncompressed.
- Public surface at @browseros/vm-container/schema exposes zod-validated
VmManifest + R2 key helpers for WS4 to import; /download is a stub
landing pad for WS4 to fill in.
- Rollback on partial upload failure: any exception after the first
successful put deletes all previously uploaded keys for that version.
- GHA workflow build-vm-container.yml runs a matrix build per arch on
native runners, an x64 Lima boot smoke test, and a gated publish job.
- Full unit coverage for arch, r2-keys, manifest, recipe parser, and
publish (rollback + happy path via aws-sdk-client-mock).
* fix(vm-container): address review comments
- Split buildDisk into prepareCustomizedDisk + finalizeArtifacts for
testability.
- Replace resolvePinnedSha's sentinel-prefix check with a positive
sha256-hex regex test, switch base-image.ts placeholder to empty string.
- Drop unused R2_VM_PREFIX from .env.example; document CDN_BASE_URL
override precedence in README.
- Replace SSH host-key explicit list in recipe with `ssh_host_*` glob so
.pub keys and future key types are also removed.
- lima-boot: introduce BunRequestInit type for the unix fetch option and
reject empty limactlPath loudly.
- Extend publish test suite: mid-manifest-upload failure path verifies
both arches' qcow+sha are rolled back and latest.json is never written.
- Add missing tests: parseArch('ARM64') case-sensitivity rejection,
composeVirtCustomizeArgv unresolved-substitution pass-through.
* fix(vm-container): pin a real Debian snapshot, switch verify to SHA-512, streaming download
- Pin Debian base to bookworm/20260413-2447 with real SHA-512 values
from upstream SHA512SUMS (the sentinel placeholder never corresponded
to a real build). Debian cloud images only publish SHA512SUMS today,
so switch base-image verification to SHA-512 throughout: rename
BaseImage.sha256 → sha512, manifest field base_image_sha256 →
base_image_sha512, base_image.sha256_url → sha512_url,
debianSha256SumsUrl → debianSha512SumsUrl. Our own artifact hashes
(compressed_sha256, uncompressed_sha256, recipe_sha256) stay SHA-256.
- Fix downloadTo: previous Bun.write(dest, response) buffered the
entire 300 MB response before writing (100% CPU, empty dir). Replace
with a getReader() loop that streams chunks through Bun.file().writer().
- build CLI now auto-derives --version from today's date when omitted
(defaults to YYYY.MM.DD-dev1); explicit --version still overrides.
Broaden CALVER_REGEX to accept alphanumeric suffixes so -dev1/-rc1
tags are valid. New todayCalver() helper.
- Update GHA workflow fallback to github.run_number (shorter) instead
of run_id.
* fix(vm-container): resolve copy-in paths against recipeDir after substitution
The copy-in path resolver checked op.src.startsWith('/') before running
the {placeholder} substitution, so an absolute-after-substitution path
like {manifest_tmp} → /tmp/vm-dist/manifest-stub-arm64.json was treated
as relative and joined against recipeDir, producing a nonexistent path.
Check the *substituted* value for absoluteness via path.isAbsolute.
* fix: address review comments for 0422-ws1_vm_disk_pipeline
* fix(ci): repair vm-container workflow
* fix(ci): expose vm build logs on failure
* fix(vm-container): expose base_image_sha256 in manifest per PRD
The published manifest contract (consumed by WS4) now uses base_image_sha256
as the PRD specified. Internally the build still verifies the downloaded
Debian base against the pinned sha512 (that's what Debian actually signs in
SHA512SUMS) — then hashes the same bytes as sha256 and records that in the
manifest. One extra digest pass of a ~300 MB file; negligible.
- manifest.json: base_image_sha256 replaces base_image_sha512; sha512_url
removed (not needed — sha256 is the consumer-facing hash).
- CLI: --base-image-sha256 override validates against the locally-computed
sha256 after download.
- BuildResult.baseImage gains sha256 alongside sha512.
- Tests updated to the new field.
The auth.json bug (reviewer #2) is resolved: the source file is
recipe/auth.json and the recipe emits `copy-in auth.json:/etc/containers/`
so libguestfs writes /etc/containers/auth.json.
* ci(vm-container): fix supermin kernel-read + rename sha512 inputs to sha256
- Ubuntu 24.04 GHA runners ship /boot/vmlinuz-* as mode 0600, which blocks
libguestfs's supermin appliance builder when virt-customize runs as a
non-root user. Chmod 0644 before the build — canonical CI workaround.
- Rename workflow_dispatch input base_image_sha512 → base_image_sha256
and CLI flag --base-image-sha512 → --base-image-sha256 to match the
orchestrator's renamed override.
* ci(vm-container): give runner KVM access + install passt for libguestfs
The supermin fix got us past appliance-build, but virt-customize then hit
"passt exited with status 1". The passt networking helper misbehaves when
libguestfs falls back to TCG emulation, which happens because the runner
user isn't in the kvm group even though /dev/kvm exists on the GHA host.
- chmod 0666 /dev/kvm → libguestfs uses hardware acceleration, avoids TCG.
- install passt explicitly so the networking helper is present and current.
* ci(vm-container): disable passt to force libguestfs slirp fallback
libguestfs 1.54+ prefers passt for guest networking, but the passt binary
on GHA ubuntu-24.04 exits with status 1 when invoked from the appliance
— an AppArmor/capability issue that doesn't surface a useful diagnostic.
The reliable workaround is to remove passt so libguestfs picks QEMU's
built-in user-mode SLIRP as the network backend. SLIRP is slower but
functional and doesn't require escalated privileges.
