mirror of
https://github.com/browseros-ai/BrowserOS.git
synced 2026-05-19 11:31:03 +00:00
fix/patch-cli-sync
219 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Nikhil
|
d61d6fc8a9 |
feat: add ACPX agent runtime adapters (#924)
* feat: add acpx claude runtime paths * feat: add acpx adapter preparation * refactor: use acpx adapter preparation * refactor: move openclaw image turns to adapter * fix: keep openclaw independent of host cwd * fix: address acpx review feedback * fix: preserve claude host auth in acpx |
||
|
shivammittal274
|
d383b5e344 |
feat(eval): add claude-generated run report artifact (#892)
* feat(eval): add claude-generated run report artifact * fix(eval): install claude code cli for CI evals * fix(eval): bypass claude code tool permissions * Eval metrics configs (#932) * feat(eval): add agisdk comparison metrics configs * fix(eval): keep cdp crashes from aborting run |
||
|
Dani Akash
|
ce4bb44083 |
feat(agent): /home composer parity with image attachments (#930)
* feat(agent): /home composer parity with image attachments
The /home composer used the same ConversationInput component as the
chat screen but passed attachmentsEnabled={false}, and the home →
chat handoff was a URL search param `?q=<text>` that physically
can't carry binary attachments. Pasting a screenshot at /home did
nothing.
Add a small in-memory registry (pending-initial-message.ts) as the
rich-data side channel for the same navigation: the home composer
writes { agentId, text, attachments } there before navigating; the
chat screen consumes it on mount and replays through the existing
harness send() path that already supports attachments. URL `?q=`
stays for shareable text-only prompts; the registry wins when both
are present. Module-scope, 10s TTL, destructive consume.
Net: home is now flagged attachmentsEnabled={true}; users can paste,
drag, or pick image files at /home and they survive the navigation
into the chat screen with previews intact.
* docs(agent): clarify why initial-message ref reset is safe post-registry-fire
|
||
|
Nikhil
|
0d56815cba |
fix: store server database under BrowserOS dir (#923)
* fix: store server database under browseros dir * fix: address PR review feedback for 923 |
||
|
Nikhil
|
c07d3d95d4 |
feat: add sqlite drizzle persistence (#919)
* feat: add drizzle agent schema * feat: run sqlite drizzle migrations * refactor: remove old sql identity dependency * feat: store harness agents in sqlite * build: package db migrations * refactor: remove sqlite oauth token store * feat: restore oauth token storage * fix: handle empty install id * chore: ignore server runtime state * fix: address review feedback for PR 919 |
||
|
Nikhil
|
1d42a973ea | refactor: extract acpx runtime templates (#918) | ||
|
Nikhil
|
921a797c5b |
feat: add ACPX agent soul and memory support (#917)
* feat: add acpx agent runtime context helpers * feat: add acpx runtime state store * feat: prepare acpx agent runtime context * feat: inject acpx agent command environment * feat: forward acpx agent chat cwd * fix: normalize acpx session record fallback * feat: improve acpx agent soul and memory prompts * fix: address PR review comments for memory-soul-acp * fix: satisfy acpx runtime deepscan checks |
||
|
Nikhil
|
d94597bbf9 |
fix(agent): add CLI model catalog entries (#915)
* fix(agent): add CLI model catalog entries * fix: address PR review comments for acpx-models |
||
|
Dani Akash
|
84e2739663 |
feat(agent): rich rail + header on /agents/:agentId chat (#908)
* feat(agent): rich rail + header on /agents/:agentId chat Replace the chat screen's legacy AgentEntry rail and binary READY header with the same rich data the /agents page already exposes: adapter glyph, liveness dot, pin star, status badge, adapter · model · reasoning chip line, last-used time, lifetime tokens, queue count, and the Adapter Unavailable warning. Source of truth flips from the merged AgentEntry list to useHarnessAgents() directly. Sort order matches /agents (pinned → recency) — not /home (active-first → recency) — because chat is index-shaped and shuffling rows every 5s as turns transition would be jarring while reading. Lift the inline pin-then-recency comparator out of /agents AgentList.tsx into a shared agents-list-order.ts so both surfaces stay on identical sort semantics. * fix(agent): chat header height + composer sticking to bottom Header was clipping descenders because the strip was vertical-content sized at min-h-14 with tight py-2.5; bump padding and lean on natural content height. Drop the AgentTile glyph (the rail row already shows adapter identity) and the cwd path (too long, pushed the meta line off-screen). Header is now name + pin star + status pill, then adapter · model · reasoning, then last-used · tokens · queued. Composer was floating mid-screen on short chats because the chat grid had no grid-template-rows — the implicit auto row collapsed to content height, so the right-column flex wrapper never received the full container height. Add grid-rows-[minmax(0,1fr)] so the single row claims 100% and ClawChat's flex-1 expands to push the composer flush to the bottom. * fix(agent): composer flush to bottom on short chats Match the sidepanel chat's nested-flex pattern. The right-column wrapper got h-full so it expands to the grid row; the conversation controller's root added flex-1 so ClawChat's existing flex-1 has something to actually fill against. Without these, the grid cell stretched but the inner flex columns shrank to content height, leaving the composer floating mid-screen. * fix(agent): align rail header with chat header in shared top band Pull the rail's "Agents" + back-button into the same horizontal strip as the agent identity header. The two halves now sit on a single row that spans both columns, so they can't drift in height as the chat header gains/loses meta lines (last-used, tokens, queued). The rail below the band keeps its scrollable list only; the chat column below holds the conversation + composer. Border-bottom moves from ConversationHeader to the band wrapper so we don't get a double-rule on the boundary. * fix(agent): reserve header height to prevent layout shift on data load The chat header grew from a single line to three lines once the useHarnessAgents() poll resolved (adapter chips + meta line populate asynchronously), shoving the rail and conversation body downward. Lock min-h-[84px] on both the band's left "Agents" cell and the ConversationHeader root, and always render the meta line slot (non-breaking space when empty) so the typographic frame is stable regardless of data state. * refactor(agent): pull status pill + meta to right side of chat header Two-column header layout instead of three stacked rows: name + pin star + adapter chips on the left, status pill stacked on top of the last-used / tokens / queued meta line on the right. Drops min-h from 84px → 60px so the band reclaims ~24px of vertical space and the chat body starts higher on screen. Band's left "Agents" cell matches the new height. |
||
|
Dani Akash
|
974e7e9b86 |
fix(agents): hide BrowserOS ACP envelope from chat history payloads (TKT-774) (#907)
* fix(agents): hide BrowserOS ACP envelope from chat history payloads (TKT-774) The user-message text persisted on the wire carried two nested envelopes — the outer `<role>You are BrowserOS…</role>` + `<user_request>…</user_request>` block from buildBrowserosAcpPrompt and the inner `## Browser Context` + `<selected_text>` + `<USER_QUERY>` block from formatUserMessage. PR #856 had unwrapped only the outer envelope on history reads, so the user bubble in the agent rail still rendered the inner envelope, and the LLM chat-service path leaked the wrapper all the way back to the sidepanel client through AI SDK's stream sync. Two surgical fixes, both server-only: 1) ACP path (acpx-runtime.ts) — replace unwrapBrowserosAcpPrompt with a comprehensive unwrapBrowserosAcpUserMessage that strips both layers and decodes the </>/& escapes the server applied via escapePromptTagText. Each step is independently defensive (anchors that don't match are skipped) so the helper is idempotent and tolerates partial / older / future-shape envelopes. Applied in userContentToText (history mapper) and inherited by extractLastUserMessage (listing's lastUserMessage). 2) LLM chat path (chat-service.ts) — split the persisted user message from the prompt-time copy. session.agent.appendUserMessage now stores the raw user text; a transient promptUiMessages array is built with the wrapped (formatUserMessage + context-change prefix) form and passed to createAgentUIStreamResponse for the model. onFinish restores the raw form before persisting, so the user-visible message and any future history reads see only the user's typed text. Tests: - acpx-runtime.test.ts: new dedicated unwrapBrowserosAcpUserMessage suite covering fully-wrapped messages, only-outer / only-inner inputs, selected_text blocks with attribute strings, idempotency, literal user-typed angle-bracket round-trip, and an integration test that round-trips the real formatUserMessage output through the unwrap to pin the writer/reader contract. - chat-service.test.ts: existing 'rebuilds a managed-app session' test updated for the new behaviour — asserts the persisted user message is the raw text and the prompt copy passed to the agent carries the Klavis context-change notice. * fix(agents): decode entity escapes before stripping inner envelope (TKT-774) The unwrap was running its inner-envelope strips against the literal-tag form (<USER_QUERY>, <selected_text>) but the persisted payload has those tags entity-escaped (<USER_QUERY>, <selected_text>) — buildBrowserosAcpPrompt runs escapePromptTagText over the entire formatUserMessage payload before adding the outer <role>+<user_request> envelope, so the inner anchors never matched against the on-disk text and the user was still seeing <USER_QUERY> in /agents/:id/sessions/main/history responses. Reorder unwrapBrowserosAcpUserMessage to: outer-strip → decode entities → inner-strips. Test fixtures updated to reflect the actual on-wire form (escaped inner tags); the round-trip test duplicates the escape rule inline so the contract between buildBrowserosAcpPrompt and the unwrap is pinned end-to-end. |
||
|
Nikhil
|
dfbce48994 |
feat: remove CLI auto init discovery (#896)
* feat: remove CLI auto init discovery * fix: address review feedback for PR #896 |
||
|
shivammittal274
|
c81906ecbf | feat(eval): add claude code eval agent (#885) | ||
|
Nikhil
|
fd5aba249b |
fix: stabilize OpenClaw gateway startup (#888)
* feat(server): add shared process lock helper * feat(container): add container name reconciliation helpers * feat(openclaw): serialize lifecycle across processes * fix(openclaw): reconcile fixed gateway container startup * test(openclaw): cover lifecycle race recovery * fix(server): satisfy process lock error override * fix(openclaw): address review feedback * test(openclaw): align serialization mock with image check |
||
|
Nikhil
|
492f3fcdf2 |
feat(openclaw): prewarm ghcr image in vm (#887)
* feat(openclaw): add gateway image inspection * feat(openclaw): pull gateway image from registry * refactor(vm): decouple readiness from image cache * refactor(openclaw): remove vm cache from runtime factory * feat(openclaw): detect current gateway image * feat(openclaw): prewarm vm runtime and reuse current gateway * feat(openclaw): prewarm runtime on server startup * refactor(vm): remove browseros image cache runtime * refactor(build-tools): remove openclaw tarball pipeline * chore: self-review fixes * fix(openclaw): suppress prewarm pull progress logs * fix(openclaw): address review feedback * fix(openclaw): resolve review findings * fix(dev): stop stale watch supervisors |
||
|
Nikhil
|
cb0c0dd0c1 |
chore: simplify root test scripts (#886)
* chore: simplify root test scripts * fix: avoid chained root test scripts * fix: update test workflow commands * fix: move app test commands into packages |
||
|
Dani Akash
|
8712f89f18 |
feat(agents): durable per-agent chat message queue + composer Stop (#880)
* feat(agents): durable per-agent chat message queue + composer Stop button
* fix(agents): tighten queue UI — smaller Stop, drop empty indicator, live drain attach
User feedback round 1 on the message-queue UX:
1) The Stop button matched the send/voice mics at h-10 w-10 with a
solid destructive fill, which read as alarming. Shrunk to h-8 w-8,
ghost variant with a soft destructive/10 background, smaller
filled square glyph. Reads as a calm 'stop' affordance instead of
a panic button.
2) The QueueItem's leading <QueueItemIndicator> dot was decorative
only — no state, no interaction. Dropped it from QueuePanel along
with the import; queue items now render as a clean preview line
with the trailing X remove action.
3) When the server drained the queue and started the next turn, the
chat panel didn't pick up the live stream until the user
navigated away and back. The hook's resume effect previously
only fired on agent change, not on listing-observed activeTurnId
change. Surface activeTurnId from useHarnessAgents into
useAgentConversation; effect now re-runs when the id changes,
calls /chat/active, and attaches to the new turn — so a queued
message starts streaming the moment the server drain pops it.
* fix(agents): don't reset streaming state from the resume effect's no-op paths
The Stop button was disappearing while the agent was actively
streaming, even though events were still flowing into the chat. Root
cause: the resume effect's `finally` block reset `streaming`,
`turnIdRef`, and `lastSeqRef` unconditionally — including on the
early-return paths (no active turn, or another mechanism already
owns the stream).
Sequence that triggered it:
1) User sends a message → send() sets streamAbortRef + streaming=true
and starts consuming the SSE.
2) User enqueues another message → enqueue mutation invalidates the
listing query.
3) Listing refetches with the live activeTurnId → the resume
effect re-fires (deps include activeTurnIdDep).
4) attemptResume hits `if (streamAbortRef.current) return` because
send() owns it.
5) The finally clause fires anyway and calls setStreaming(false),
clobbering the live state set by send(). The SSE consumer keeps
running (refs are intact) so text keeps streaming, but the React
flag is wrong, so the Stop button gates off.
Fix: track whether *this* run actually started a stream
(`weStartedStream`). The finally only resets state when it does.
Early-return / no-active-turn paths now leave streaming/turnIdRef/
lastSeqRef alone for whoever does own them.
Also widens the Stop button's visibility (`canStop` prop on
ConversationInput) so it stays steady across the brief gap between
turns when a queue drain is mid-flight; the parent computes
`streaming || activeTurnId !== null || queue.length > 0`. The
visibility widening is independent of the streaming-state fix above
— both are now in place.
* revert: drop canStop widening — Stop only shows while streaming
Reverts the canStop prop on ConversationInput and the OR-with-queue
visibility from AgentCommandConversation. Stop is gated solely on
`streaming` again. Between turns (queue draining) the button stays
hidden — only the actively-streaming turn is interruptible from the
composer, which matches what the user actually expects.
* fix(agents): persist the kicking-off prompt on active turns so the resume placeholder isn't empty
When a queued message drained and started a new turn, the chat
panel's resume effect staged a placeholder turn with userText: ''
because the hook had no way to know what message kicked off the
turn — only the agent-side stream was visible, and the user bubble
above it was blank until the user navigated away and back (at which
point the session record's history loaded normally).
Fix: ActiveTurnRegistry.register now accepts an optional `prompt`
that's stashed on the turn and surfaced via describe() / the
ActiveTurnInfo response. AgentHarnessService.startTurn passes the
incoming message into register. /chat/active returns it. The chat
hook's resume effect uses active.prompt as the placeholder
turn's userText, so the user bubble shows the queued message text
the moment streaming begins. Falls back to '' for older clients
that haven't been refetched yet.
* fix(agents): always release streamAbortRef on resume cleanup, even when cancelled
Greptile P1 follow-up. The previous `weStartedStream` guard correctly
stopped the resume effect's no-op early-returns from clobbering an
in-flight `send()` stream — but it also stopped a *cancelled*
mid-stream resume from clearing its own `streamAbortRef`. When the
cleanup fires (e.g. the 5s listing poll captures a new queue-drain
turn id while the SSE for the prior turn is still finishing), the
next effect run hits the `if (streamAbortRef.current) return` guard
against the now-aborted controller and never reattaches, leaving
`streaming === true` with no live stream until the user navigates
away.
Split the finally block: always release `streamAbortRef` when we
owned the controller (so the next run can take over), but only
reset the streaming flag / turn id / lastSeq on a clean exit (the
new run will set those itself, so resetting on cancel would just
flicker).
|
||
|
Dani Akash
|
ba60bf466f |
feat(agents): rich command-center rows + home grid + dead-code sweep (#879)
* feat(agents): rich-info command center rows + pin/PATCH/adapter-health backbone
Splits AgentRowCard from a 271-line monolith into a shallow tree of
single-responsibility sub-components under `agent-row/`:
AgentTile, AdapterHealthDot, PinToggle, AgentTitleRow,
AgentSparkline, AgentSummaryChips, AgentLastMessage, CwdChip,
AgentTokenSummary, AgentMetaRow, AgentErrorPanel, AgentActions
Adds the data each row consumes:
- pinned: boolean field on AgentDefinition + FileAgentStore.update
+ new PATCH /agents/:id route. useUpdateHarnessAgent mutation
optimistically updates the listing cache so the star flips
instantly; rolls back on error.
- Listing payload extended with lastUserMessage, cwd, tokens
(cumulative + last7d shape — last7d zero-filled until the
activity ledger lands), turnsByDay/failedByDay (zero-filled),
lastError/lastErrorAt, activeTurnId. AcpxRuntime grows a
getRowSnapshot() that reads cwd + cumulative tokens + last user
message from the session record in one pass.
- Adapter health: in-memory AdapterHealthChecker probes
`claude --version` / `codex --version` with a 2s timeout and
caches results for 5 min. /adapters response carries
{ healthy, reason?, checkedAt }. Tile-corner dot exposes the
state via HoverCard; openclaw inherits health from the gateway
snapshot already on the page.
Sub-components are pure: card itself owns no state. Sort order
becomes pinned-first, then recency. HoverCard is the workhorse for
keeping rows compact while exposing depth (full message, token
breakdown, daily turn list, error stack, adapter reason).
* refactor(agents): tighten command-center row design + cut redundant affordances
User feedback round 1:
1) Two green dots on the tile (health + liveness) was confusing. Health
moves out of the tile entirely and surfaces as an inline 'Unavailable'
chip in the model line — silent when the adapter is healthy, with a
warning amber chip + HoverCard reason when not. The tile now shows
one signal: liveness.
2) The last-user-message HoverCard wasn't telegraphing intent. Drop the
HoverCard. The line is informational, italic, with a leading quote
glyph so the row reads like a conversation snippet. To see the full
message the user opens the chat (which is the action they want next
anyway).
3) Resume + Chat were duplicate CTAs. Single primary action per row:
Resume (filled, accent-orange, with a pulsing dot) replaces Chat
when there's an active turn. Both navigate to /agents/:id but the
row tells the user which action they're taking.
4) Tokens weren't visible because the row gated on last7d.requestCount,
which is zero until the activity ledger ships. Switch to lifetime
tokens (which we have today). Drop the '7d stats:' framing — talking
about a window we can't compute would be misleading. The HoverCard
surfaces input/output split + a footnote that per-window stats land
in a follow-up.
5) CWD was rendering the server's own running directory, which is
meaningless to users. Hide it from the row entirely. The cwd field
still rides in the listing payload for future surfaces (chat panel,
debug view) — only the row stops rendering it.
Aesthetic refinements while we're here:
- Whole card carries state, not just the tile: working rows get an
accent-orange tinted border with a soft glow, error rows tint
destructive, idle rows lift on hover.
- Pin star fades in on hover (group-hover) when unpinned and stays
solid amber when pinned — keeps the rail calm by default.
- Tabular-nums on token figures so columns visually align across rows.
- Drop CwdChip and AdapterHealthDot files: no callers left.
* fix(agents): align row title flush-left whether pinned or not
Pin star moved from leading the title to trailing the badges, and
hidden from layout entirely (`hidden group-hover:inline-flex`) when
unpinned. The previous `opacity-0` rule kept the star reserving its
`size-6` slot, which left every unpinned title indented relative to
the model / preview / meta lines underneath it. Title now flushes
left in both states; pinned star stays solid amber so the signal
isn't hidden, and unpinned reveals an outline star on row hover for
the toggle affordance.
* fix(agents): keep pin-toggle slot reserved so row height is constant
Switching the unpinned star from `hidden group-hover:inline-flex`
to `opacity-0 group-hover:opacity-100`. The hidden/show variant was
collapsing the title row's height when the star wasn't rendered,
which made every card below visibly shift on hover. Always rendering
the button (with opacity-only visibility) keeps the row's vertical
metrics constant; the title still flushes left because the slot is
trailing, not leading.
Card hover effect (-translate-y + shadow-md) restored — the layout
shift wasn't coming from the card hover; it was the pin slot
appearing and disappearing.
* fix(agents): quieten row hover — border-tint only, no lift, no shadow
Drop the `-translate-y-px` and `hover:shadow-md` from the row card
plus the working-state inner ring. The translate + shadow grow
combination was visibly noisy as the cursor moved through the rail —
each row 'lifted' as you passed over it. Hover now just tints the
border in accent-orange/30; working and error states keep their
distinct border colours but no inner ring. Card height and shadow
stay constant in every state, so the rail reads as a calm vertical
list of cards.
* feat(home): rich Recent Agents grid + dead-code sweep
The /home Recent Agents grid was a placeholder shell. Every 'rich'
field on the card (lastMessage, lastMessageTimestamp, activitySummary,
currentTool, costUsd) was wired to undefined because AgentCommandHome
called `buildAgentCardData(agents, status?.status, undefined)` — the
dashboard arg has been hard-coded undefined since the harness
migration. Repointing the grid at `useHarnessAgents` + `useAgentAdapters`
gives every card the same enriched data the rail uses.
What the new card shows per agent:
• Adapter glyph tile + liveness dot (working pulses; asleep is
hollow; error is red)
• Name + Working pill (when active)
• Adapter · model · reasoning summary line, with an inline
Unavailable chip + HoverCard reason when the adapter binary
isn't on $PATH
• Italic last-user-message preview (line-clamp-2, leading quote
glyph) — same visual language as the rail
• Footer: 'X ago' + state chip (Asleep / Attention) OR a Resume
button (orange, with pulsing dot) when activeTurnId is non-null
Sort on the home grid is active-turn → recency. Pinning is NOT a
sort key here (and there's no pin indicator on the card) — pinning
belongs to the rail at /agents; the home page is action-oriented
and trusts active-turn + recency to surface the right agent.
Dead code removed:
• useAgentDashboard.ts (96 lines, no callers; subscribed to the
dead /claw/dashboard/stream from the OpenClaw-only era)
• useAgentCardData.ts (the dashboard-merge shim; passed undefined
every call so all enriched fields landed as undefined)
• AgentCard.tsx (AgentCardExpanded replaced by HomeAgentCard;
AgentCardCompact had no callers — the dock's compact mode was
never used)
• AgentCardData interface dropped from lib/agent-conversations/
types.ts; the new card consumes HarnessAgent directly
Visual language stays continuous between rail and grid: same
<AgentTile>, same <LivenessDot>, same italic-quote message
preview, same orange Resume button with a pulsing dot.
|
||
|
Nikhil
|
26afb826c6 |
feat(eval): add viewer manifest contract (#878)
* refactor(eval): canonicalize viewer manifest contract * refactor(eval): publish canonical viewer manifests * feat(eval): make r2 viewer use manifest artifact paths * fix(eval): keep weekly report compatible with viewer manifests * docs(eval): document r2 viewer manifest contract * chore: self-review fixes * fix: address review feedback for PR #878 |
||
|
Nikhil
|
b2340c8afa |
refactor(eval): split orchestrated executor backends (#876)
* refactor(eval): split orchestrated executor backends * fix(eval): address executor backend review comments |
||
|
Nikhil
|
84a79ba0a1 |
feat: refactor eval pipeline workflow (#875)
* feat(eval): add suite variant config bridge * feat(eval): add stable run artifacts * refactor(eval): add shared grader contract * feat(eval): persist grader artifacts * refactor(eval): rename runner layers * refactor(eval): add executor backend boundary * refactor(eval): split clado backend * feat(eval): add workflow compatible cli * feat(eval): add r2 publisher module * ci(eval): migrate weekly workflow to eval cli * docs(eval): document suite pipeline * chore(eval): verify pipeline refactor * fix: address review feedback for PR #875 * docs(eval): add env example * docs(eval): explain suites and variants * chore(eval): organize config layouts * chore(eval): colocate grader python evaluators |
||
|
Nikhil
|
561f2baf97 |
fix(eval): split AGISDK smoke and full configs (#871)
* fix(eval): split agisdk smoke and full configs * fix(eval): default agisdk smoke to openrouter |
||
|
shivammittal274
|
df0f45dd29 |
Feat: eval debug dev ci (#869)
* chore(eval): instrument server startup to root-cause dev CI health-check timeouts Three diagnostics + one config swap to investigate why the eval-weekly workflow has been failing on dev since 2026-04-25 with "Server health check timed out" (every worker, every retry). Background: - Last successful weekly eval on dev: 2026-04-18 (sha |
||
|
Nikhil
|
edfc5c751c |
fix: align OpenClaw gateway image with VM cache (#868)
* fix: load OpenClaw gateway image from VM cache * fix: use container port for OpenClaw ACP bridge * fix: address review feedback for PR #868 |
||
|
Nikhil
|
471256f31c | fix: stop passing native permission flags to ACP adapters (#867) | ||
|
Nikhil
|
4c90ca696b | fix(agents): connect OpenClaw ACP inside gateway container (#866) | ||
|
Nikhil
|
f2ac87d7c3 |
feat: show created agents in sidepanel (#865)
* feat(agent): list created agents in sidepanel target catalog * feat(agent): show created agents in sidepanel selector * feat(server): add sidepanel chat route for created agents * feat(agent): route sidepanel agent sends by agent id * chore(agent): retire virtual sidepanel acp targets * fix: address review feedback for PR #865 |
||
|
shivammittal274
|
231bd6821d |
fix(eval): pin agisdk version + exclude 4 invalid tasks (Phase 2 dataset hygiene) (#844)
* chore(eval): pin agisdk version to prevent silent dataset drift `pip install agisdk` previously fetched whatever version pip resolved at CI time. If agisdk publishes a new version with changed task definitions or grader behavior, the weekly eval silently shifts under our feet — making "did the score move because of code or data?" unanswerable. Pin to agisdk==0.3.5 (the version we currently develop against). Bump intentionally with a documented re-baseline run. * fix(eval): exclude 4 more tasks identified by 8-trial never-passing audit After 8 trials across K2.5 + Opus 4.6 (Phase 1 and Phase 2), 5 tasks never passed. Per-task root-cause investigation via parallel deep-dive subagents flagged 4 of them as fundamentally unfixable in the eval pipeline as it stands; the 5th (`dashdish-5`) is a prompt-rule fix that stays in. - gocalendar-7: goal/grader contradiction. Goal says "move event to July 19, 10 AM"; grader expects `eventsDiff.updated.*.start == "2024-07-18T17:00Z"` (= July 18, 10 AM PDT — same day, 1 hour shift). Even after the Phase 2 HTML5 dnd dispatch fix correctly populates `eventsDiff.updated`, the values are July 19 (matching the goal), which the grader rejects. - staynb-5: grader hardcodes literal `'Oct 13 2025'` and `'Oct 23 2025'` year strings. The staynb date picker interprets bare "Oct 13" as the most-recent-past instance (currently 2024 since today is 2026), not 2025. No agent can produce a persisted date string containing 2025. - staynb-9: under-specified task. Goal says "maximum number of guests supported"; grader requires the very specific string "32 Guests, 16 Infants" — encoding UI knowledge (Adults+Children=Guests display, Infants render separately, per-category cap=16, Pets excluded) that isn't in the prompt. Even Opus 4.6 stopped at 16 across 3 trials. - opendining-3: grader requires `contains(booking.date, '2024-07-20')` but the React-controlled date textbox flakily no-ops on `fill`. 3/8 trial pass rate is essentially coin-flip noise driven by tool-fidelity variance rather than agent capability. Removing to reduce score noise; Phase 2 fill post-validate warning helps when it does work, but the task's signal-to-noise is too low for the eval set. Dataset goes from 40 -> 36 tasks. Total EXCLUDED_TASKS now 11 entries. Validated by 8-trial pass-record audit; deep-dive notes saved to plans/audits/. |
||
|
Dani Akash
|
a228c278c6 |
feat(agents): background-resilient chat — turns survive tab disconnect (#863)
* feat(agents): decouple chat turn lifecycle from SSE response
Introduce a per-process ActiveTurnRegistry that owns each agent turn's
lifecycle and a ring-buffered event stream, so chat tabs that close,
refresh, or navigate away no longer cancel the in-flight turn. New
endpoints:
POST /agents/:id/chat starts a turn (now returns 409 when
one is already running, with the
active turnId for attaching)
GET /agents/:id/chat/active reports the running turn for a UI
that just mounted
GET /agents/:id/chat/stream subscribes to a turn; supports
Last-Event-ID resume via per-event
seq ids
POST /agents/:id/chat/cancel explicit cancel — fetch abort no
longer affects the underlying turn
The chat hook now captures X-Turn-Id, tracks lastSeq from SSE id lines,
re-attaches on mount when the server still has an active turn, and
routes Stop through the cancel endpoint. The runtime call uses the
registry's per-turn AbortController instead of the HTTP request signal,
which is the core decoupling that lets turns outlive their initiator.
* feat(agents): add ActiveTurnRegistry primitive backing the new chat lifecycle
The previous commit referenced these files in tests and the harness
service but global gitignore swallowed them on the first add.
The registry owns the per-turn ring buffer (drop-oldest, terminal frame
preserved), the per-turn AbortController, and subscriber fan-out used
by /chat/stream resume.
|
||
|
Dani Akash
|
e2ec1991cf |
feat(agents): redesign the agent command center for multi-adapter use (#861)
* feat(agents): redesign agent rail to match the rest of the app
Reshape the `/agents` page so it reads as a sibling of `/scheduled`
and `/soul` and adapts to the multi-adapter world (OpenClaw, Claude
Code, Codex). Visual scaffolding only in this commit — per-agent
liveness state ships as `unknown` until the server-side activity
tracker lands.
- New `AgentsHeader` mirrors `SoulHeader`/`ScheduledTasksHeader`:
accent bot tile, title, descriptive subtitle, "+ New Agent"
button. Replaces the loose top toolbar that mixed page-level and
OpenClaw-lifecycle controls.
- New `GatewayStatusBar` collects the OpenClaw lifecycle pills
(running, control plane connected) plus the Terminal/Refresh
affordances into a single labeled bar that only renders when the
gateway is running AND there is at least one OpenClaw agent in
the merged list.
- New `AgentRowCard` per agent: adapter tile with liveness dot,
name + status badge, adapter/model/reasoning chips, last-used
relative time + truncated workspace path, primary "Chat" button,
overflow menu (Copy id / Rename* / Reset history* / Delete).
Rename + Reset are disabled with "coming soon" tooltips until
the corresponding endpoints ship; Delete is hidden for the
protected `main` agent.
- New `AgentsEmptyState` mirrors the scheduled-tasks empty card.
- New `AdapterIcon` + `LivenessDot` + `agent-display.helpers.ts`
keep the row card focused on layout; helpers cover display name
fallbacks for legacy `oc-<uuid>` titles, workspace label rules,
and a tiny relative-time formatter.
- `AgentList` now sorts by `lastUsedAt` desc with `null`s falling
to the bottom; the gateway's `main` agent is pinned to the top
only while it has zero turns so a fresh install has an obvious
starting point. The list also threads a per-agent activity map
so future commits can light up working/idle/asleep without
reshuffling the API.
- `AgentsPage` swaps to the standard `fade-in slide-in-from-bottom-5
animate-in space-y-6 duration-500` shell and threads a
`harnessAgentLookup` Map down to the row card so adapter chips
and reasoning effort render correctly without a re-fetch.
* feat(agents): wire per-agent liveness end-to-end into the rail
Closes the placeholder `unknown` dot from the redesign's first
commit. The rail now shows real working / idle / asleep / error
states per agent, with `lastUsedAt` driving the recency sort.
Server side:
- `AgentHarnessService` keeps an in-memory activity tracker keyed
by agentId. `notifyTurnStarted` flips an entry to `working`,
`notifyTurnEnded({ok})` either drops it (success) or pins it to
`error` (failure / error event).
- `send()` wraps the runtime stream so the lifecycle hook fires
exactly once on natural close, error event, downstream cancel,
or thrown setup. The runtime itself stays unchanged — fork is
contained at the harness layer.
- New `listAgentsWithActivity()` method enriches every agent with
`{ status, lastUsedAt }`. lastUsedAt is read from the acpx
session record's last persisted item via `runtime.getHistory`,
so it survives server restart even though the activity map
doesn't.
- Status derivation: `working`/`error` take precedence; otherwise
timestamp-based — `idle` until 15 min of silence, then `asleep`.
Never-used agents resolve to `idle` (asleep implies "was active,
went quiet").
- `GET /agents` returns the enriched shape.
Client side:
- `HarnessAgent` UI type extended with optional `status` +
`lastUsedAt` so older deployments still typecheck.
- `useHarnessAgents` flips on `refetchInterval: 5_000` (with
`refetchIntervalInBackground: false` so hidden tabs go quiet)
so the per-row dots and last-used copy stay fresh without a
websocket.
- `AgentsPage` builds an activity map from the harness listing
response and threads it into `AgentList` → `AgentRowCard`. The
sort by `lastUsedAt` desc (already in the row card) now has
real data to operate on.
Tests:
- New `marks an agent working while a turn streams and idle once
it ends` exercises the wrap; uses a held upstream stream so
the in-flight `working` state is observable.
- New `flips to error when a turn emits an error event`.
* fix(agents): dedupe agent rail when /claw/agents and /agents share an id
The agents page was rendering every OpenClaw agent twice — once from
the legacy `/claw/agents` listing (`useOpenClawAgents`) and once from
the harness `/agents` listing (`useHarnessAgents`). Post Step 9
backfill the harness store contains every gateway agent, so the
overlap is the rule, not the exception.
Mirror the dedup the chat-panel layout already does: when a gateway
agent's id appears in the harness listing, drop the legacy entry and
keep the harness one (it has adapter/model/reasoning/status/lastUsedAt
the chat path actually consumes).
* feat(agents): swap GatewayStatusBar refresh icon for a Restart Gateway button + tooltips
The manual refresh became redundant once `useHarnessAgents` and
`useOpenClawStatus` started polling on a 5s interval — every visible
field self-refreshes within seconds. The previous AgentsPageHeader
had a real Restart action that the redesign dropped; reinstate it on
the bar so a wedged gateway is one click away again.
- GatewayStatusBar: dropped the `RotateCcw` refresh icon and the
`onRefresh` prop. Added `onRestart` + `actionInProgress` props;
the button shows a spinner while a gateway lifecycle mutation is
in flight.
- Both Terminal and Restart Gateway buttons get tooltips explaining
what they do — Terminal as a power-user shell escape hatch,
Restart for unsticking a wedged gateway or after manual config
edits.
- AgentsPage: drop the now-unused `refreshAll` helper and the
`refetchStatus`/`refetchAdapters`/`refetchOpenClawAgents`
destructures it depended on. Wire `restartOpenClaw` (already
pulled from `useOpenClawMutations`) through
`runWithPageErrorHandling` like the legacy header did.
* feat(agents): consolidate gateway status into the /agents listing
Folds the gateway lifecycle snapshot into the harness listing so the
agents page polls one endpoint instead of two. Drops the dead
`/claw/status` call from the command center while keeping every UI
affordance the page already shipped (Running / Control plane
connected pills, GatewayStateCards setup/start prompts,
ControlPlaneAlert for degraded states).
Server side:
- `OpenClawProvisioner.getStatus()` (optional) — when wired, returns
the same `GatewayStatusSnapshot` shape `/claw/status` does.
- `AgentHarnessService.getGatewayStatus()` — best-effort wrapper
around the provisioner method; logs and swallows errors so a
transient gateway issue doesn't 500 the listing endpoint.
- `GET /agents` now returns `{agents, gateway}` in a single
`Promise.all`. Both fields are independent — agents enrichment
succeeds even if the gateway snapshot is null.
- `server.ts` wires `getOpenClawService().getStatus()` into the
provisioner accessor object alongside `createAgent` /
`removeAgent` / `listAgents`.
Client side:
- `useHarnessAgents` returns `{harnessAgents, gateway}` (plus the
legacy `agents` mapping). Same 5s `refetchInterval` as before —
one round-trip drives the per-row liveness AND the gateway pills.
- `AgentsPage` drops `useOpenClawStatus` entirely; `status` comes
from the harness query. Loader + error/lifecycle plumbing
rewired around the harness query's loading/error.
- `agents-page-utils.getInlineError` and `getAgentsLoading` lose
the now-redundant `statusError` / `statusLoading` /
`openClawAgentsEnabled` params.
The chat-panel layout (`agent-command-layout.tsx`) still consumes
`useOpenClawStatus(5000)` for now — left intact per the user's "only
the command center" scope. Folding that one in is a separate,
smaller pass once we're sure no regression slipped here.
* test(agents): teach the route fake service about the new listing shape
PR #861 CI surfaced two failures in tests/api/routes/agents.test.ts:
both call \`GET /agents\` and the route handler now invokes
\`service.listAgentsWithActivity()\` + \`service.getGatewayStatus()\`
which the fake created here didn't implement. Add both methods to
the fake (returning idle / null) and update the empty-list assertion
to expect the new \`{agents, gateway}\` envelope.
|
||
|
Dani Akash
|
0c84547e8f |
feat(agents): migrate OpenClaw chat onto the unified harness/ACP path (#859)
* chore(acp): smoke-test ACP capabilities against running gateway
Adds apps/server/scripts/acp-smoke.ts which spawns `openclaw acp`
inside the gateway container and exercises every method we plan to
depend on: initialize, newSession, prompt (text + image), cancel,
listSessions, loadSession.
SDK pinned to 0.19.1 (Bun's minimum-release-age policy blocks 0.20+
which were released < 7 days ago).
Findings (full notes in plan outcomes):
- promptCapabilities advertises image:true but the model does NOT see
image bytes — silently dropped at the bridge.
- sessionCapabilities advertises {list:{}} but session/list throws
"Method not found": stale capability advertising.
- loadSession works; replays user/assistant/thought text and
session_info/usage/commands updates. No tool_call replay, as
documented.
- cancel works end-to-end: stopReason=cancelled.
- closeSession/resumeSession are not on ClientSideConnection in
0.19.1; kill child to close, use loadSession for rebind.
Plan revisions triggered by spike are recorded in
plans/browseros-ai/BrowserOS/features/2026-04-28-2310-claude-code-acp-implementation-roadmap.md.
* chore(acp): re-run smoke on SDK 0.21.0 and add mode/config/auth scenarios
After bypassing Bun's minimum-release-age and upgrading the SDK to
0.21.0, restore the previously-skipped resume/close paths and add
three new scenarios: mode (setSessionMode), config (setSessionConfigOption,
correct configId field), and auth (authenticate noop).
Findings, all bridge-side (independent of SDK):
- session/list, session/resume, session/close all throw -32601 on
OpenClaw 2026.4.12 — capability advertising is stale.
- Image content blocks silently dropped; model never sees the bytes.
- setSessionMode and setSessionConfigOption work; latter requires
`configId` (not `optionId`) per the schema.
- loadSession replays user/assistant/thought text + session_info +
usage + available_commands; no tool_call replay (documented).
- authenticate is a noop on OpenClaw (no authMethods advertised).
Plan outcomes updated with full method-support matrix.
* chore(deps): promote @agentclientprotocol/sdk to a runtime dependency
The smoke script in apps/server/scripts/acp-smoke.ts used the SDK as
devDependency. The upcoming ACP bridge (apps/server/src/api/services/acp/)
needs it at runtime, not just for tooling. Move the entry from
devDependencies to dependencies, alphabetically first under @a*.
Pinned to 0.21.0 — same version the smoke script validated against.
README gains a small Dependencies note pointing at the future bridge
location.
No code changes yet. The bridge wiring lands in subsequent commits.
* fix(openclaw): wire LlmProvider.supportsImages through to OpenClaw model config
When BrowserOS sets up a custom OpenAI-compat provider on the gateway,
the agent UI's "Supports Image" flag (LlmProviderConfig.supportsImages)
was being dropped on the floor. As a result the persisted model entry
had no `input` field, OpenClaw defaulted it to ['text'], and image_url
content parts were silently stripped before the model saw them.
Fix:
- Extend OpenClawSetupInput / OpenClawAgentMutationInput on the agent
side (useOpenClaw.ts) and the route body schema + SetupInput +
createAgent input on the server side with `supportsImages?: boolean`.
- AgentsPage forwards `llmOption?.supportsImages` from the selected
LlmProviderConfig in both handleSetup and handleCreate.
- provider-map.resolveSupportedOpenClawProvider emits
`input: ['text', 'image']` on the model entry when the flag is
truthy; otherwise emits the explicit `['text']` so the value is
always pinned (avoids relying on OpenClaw's implicit default).
- applyBrowserosConfig adds `tools.media.image.enabled = true` to the
bootstrap batch so the gateway's image-understanding pipeline is
always wired up — per-model `input` still gates which models see
images, this just enables the global path.
ACP image content blocks are still dropped by the OpenClaw bridge —
that's a separate bridge bug, not addressed here. This commit
restores image support for the OpenAI-compat /v1/chat/completions
path that the upcoming ACP chat panel will use as a carve-out for
image-bearing prompts.
Existing custom-provider configs are NOT auto-migrated; users will
re-acquire image support either by re-running setup or by editing
their model entries' `input` field manually. A migration pass for
legacy installs is not in scope for this commit because the
"supportsImages" intent isn't recoverable from the persisted config
alone — the source of truth is the LlmProvider record on the agent
side.
* feat(agents): add OpenClaw to AgentAdapter union and catalog
Extends AgentAdapter to 'claude' | 'codex' | 'openclaw' and adds the
OpenClaw entry to AGENT_ADAPTER_CATALOG. The new entry has:
- defaultModelId: 'default' — OpenClaw's ACP bridge does not surface
per-session model selection (verified during the ACP spike), so
models live in the OpenClawService config, not in the adapter
catalog. AgentDefinition.modelId carries the gateway-side model
name for display only.
- models: [] — empty list signals "no per-session model picker" in
the UI; isSupportedAgentModel('openclaw', undefined|'default')
returns true via the existing fallback path.
- reasoningEfforts mirror OpenClaw's session-level `thought_level`
config option (off / minimal / low / medium / high / adaptive).
Also extends:
- isAgentAdapter type guard recognizes 'openclaw'
- HarnessAgentAdapter union on the extension side
- agents.test.ts createAgent fake type
- agent-catalog.test.ts asserts on the new entry, empty model list
passthrough behavior, and OpenClaw's reasoning effort set
Lockfile delta is the workspace SDK pin reconciling 0.20.0 (taken
from dev's lock) up to our package.json's 0.21.0 (added in
|
||
|
Nikhil
|
2ff5c12840 |
feat: add sidepanel ACP chat targets (#857)
* feat(agent): add sidepanel chat target catalog * feat(agent): show acp models in sidepanel selector * feat(server): adapt acp events to ui message streams * feat(server): add sidepanel acp chat route * feat(agent): route sidepanel chat through acp targets * chore: self-review fixes * fix: address review feedback for PR #857 |
||
|
Nikhil
|
d87422eea1 | fix: hide BrowserOS ACP wrapper in history (#856) | ||
|
Nikhil
|
1946ca0cf8 | chore: clean up unused agent sdk (#855) | ||
|
Nikhil
|
754f7d0e1d | test: cover terminal limactl resolver errors (#854) | ||
|
Nikhil
|
85bb3f7b42 | fix: avoid eager limactl resolution in server tests (#853) | ||
|
Nikhil
|
cb32b8191d |
fix: show rich ACP harness history from ACPX (#852)
* fix: load ACP harness history from ACPX * fix: address ACP history review comments |
||
|
Nikhil
|
7a92654abc |
feat: add BrowserOS MCP to ACP agents (#851)
* feat: add BrowserOS MCP to ACP agents * fix: bypass ACP agent permissions * fix: address review feedback for PR #851 |
||
|
Nikhil
|
91d3285aa0 |
feat: add ACP agent harness (#849)
* feat: add acp agent runtime spike * feat: add agent harness catalog * feat: persist harness agents in json * feat: persist agent transcripts * feat: route harness service through agent records * feat: expose generic agent harness routes * feat: add harness agent frontend api * feat: create harness agents from agents page * feat: chat with persisted harness agents * chore: remove obsolete agent profile spike * chore: self-review fixes * fix: combine openclaw and harness agents UI * refactor: split agents page components * fix: hide persisted harness turns |
||
|
shivammittal274
|
d9c254053e |
refactor(eval): drop unused agents/graders, collapse registries (#847)
* refactor(eval): drop unused agents/graders, collapse registries Sweep of dead code in the eval app: deleted gemini-computer-use and yutori-navigator agents, fara/webvoyager/mind2web graders, eight debug/analyze/test scripts, three stale planning docs, and the orphaned eval-targets/coordinate-click testbed. With two agents and three graders left, the Map-backed plugin registries were over-engineered — collapsed both into plain switches. Removed the now-dead GraderOptions plumbing (no remaining grader takes API keys), dropped grader_api_key_env/grader_base_url/grader_model from the schema and configs, and de-duped PASS_FAIL_GRADER_ORDER (was defined in three places). Replaced the URL-parsing extractCdpPort hack in single-agent and orchestrator-executor with workerIndex passed cleanly through AgentContext. README and --help text rewritten to match reality. Renamed configs/test_*.json to test-*.json for kebab-case consistency. Net: ~10,460 LOC removed across 60 files. Typecheck clean, all tests pass. * ci(eval): pull BrowserOS from rolling stable CDN URL The pinned v0.44.0.1 .deb on GitHub releases regressed on Linux — servers start but never become healthy. Switch to the canonical rolling URL at cdn.browseros.com/download/BrowserOS.deb so CI tracks the same stable channel users get from the marketing site. |
||
|
Nikhil
|
6b9945f933 | feat(dev): use dev dock icon for browser launches (#848) | ||
|
Dani Akash
|
6a5a7775a9 |
fix(openclaw): wire LlmProvider.supportsImages through to OpenClaw model config (#846)
When BrowserOS sets up a custom OpenAI-compat provider on the gateway, the agent UI's "Supports Image" flag (LlmProviderConfig.supportsImages) was being dropped on the floor. As a result the persisted model entry had no `input` field, OpenClaw defaulted it to ['text'], and image_url content parts were silently stripped before the model saw them. Fix: - Extend OpenClawSetupInput / OpenClawAgentMutationInput on the agent side (useOpenClaw.ts) and the route body schema + SetupInput + createAgent input on the server side with `supportsImages?: boolean`. - AgentsPage forwards `llmOption?.supportsImages` from the selected LlmProviderConfig in both handleSetup and handleCreate. - provider-map.resolveSupportedOpenClawProvider emits `input: ['text', 'image']` on the model entry when the flag is truthy; otherwise emits the explicit `['text']` so the value is always pinned (avoids relying on OpenClaw's implicit default). - applyBrowserosConfig adds `tools.media.image.enabled = true` to the bootstrap batch so the gateway's image-understanding pipeline is always wired up — per-model `input` still gates which models see images, this just enables the global path. ACP image content blocks are still dropped by the OpenClaw bridge — that's a separate bridge bug, not addressed here. This commit restores image support for the OpenAI-compat /v1/chat/completions path that the upcoming ACP chat panel will use as a carve-out for image-bearing prompts. Existing custom-provider configs are NOT auto-migrated; users will re-acquire image support either by re-running setup or by editing their model entries' `input` field manually. A migration pass for legacy installs is not in scope for this commit because the "supportsImages" intent isn't recoverable from the persisted config alone — the source of truth is the LlmProvider record on the agent side. |
||
|
shivammittal274
|
af48a2110c |
feat(eval): Phase 1 — exclude broken tasks, freshen card dates, add grader leniency (#841)
* fix(eval): exclude broken tasks + freshen expired card dates Two AGISDK tasks are unsolvable today for non-model reasons: - topwork-1: evals-topwork.vercel.app throws Minified React error #185 ("Maximum update depth exceeded") on every form submit. The page renders "Application error: a client-side exception has occurred" instead of saving. Whole-task failure, every model affected. - fly-unified-2: hardcodes Exp: 12/25 in both the goal text AND a jmespath grader criterion. Today is 2026-04, so the eval-site rejects the card. Freshening the goal alone leaves the grader expecting the original value; freshening both would require monkey-patching agisdk's TaskConfig at runtime — too fragile to maintain. Adds these to a new EXCLUDED_TASKS set alongside the existing EXCLUDED_WEBSITES (omnizon). Also adds freshen_goal_dates(): for AGISDK fly-unified tasks whose goal contains an `Exp: MM/YY` within 6 months of today (or past), rewrites it to a far-future date (12/30). This rescues fly-unified-5 (had Exp 12/25, no card-exp grader criterion) and protects fly-unified-4 (had Exp 06/26, 2 months from expiring) from the next eval run hitting the same trap. Dataset goes from 47 -> 45 tasks; 2 freshened. * feat(eval): add lenient-strings grader softening The agisdk grader compares jmespath-extracted values via strict equality. For tasks where the model adds harmless decoration to a free-text field (e.g. topwork-3 expects title "Full-Stack Developer" but model produces "Full-Stack Developer - Enterprise Microservices Platform"), this fails every other criterion would pass. Adds a substring fallback in the wrapper: a failed criterion is re-marked as a softened pass when both actual_value and expected_value are strings and the (stripped, lower-cased) expected_value is contained in the actual_value. Numbers/bools/dates/None stay strict. - Default-on. Set AGISDK_STRICT_STRINGS=1 to recover the strict score. - Softened criteria are tagged with `softened: true` in per_criterion output for transparency in run manifests. - Aggregate `pass`/`reward` are recomputed after softening. Expected to rescue 4 tasks in our 45-set: topwork-3, topwork-4 (both pure title-decoration), gomail-8 (grader contradicts goal), and networkin-6 (grader hardcodes profile id). * fix(eval): exclude 5 more tasks where pipeline (not agent) fails Extends EXCLUDED_TASKS to 7 entries based on the K2.5 + Opus 4.6 head-to-head deep-dive on the 2026-04-28 runs. The exclusion rule: remove a task only if it is unsolvable for any agent — either the task data is invalid, the eval site is broken, or the grader penalizes correct work. Tasks that fail because of our agent's tool fidelity (drag, custom-widget fill, click on React submit, etc.) STAY in — those are real capability gaps the team should see in the score. New exclusions: - fly-unified-9: goal references "Dec 18 2024 at 10:00" but the live eval site has only 2025 inventory and no 10:00 slot. Both models successfully booked the closest available flight and were penalized on a grader expectation that can never be met. - fly-unified-4: eval site stores wall-clock flight times as bare UTC (T08:00:00.000Z) while the grader expects them shifted by 8h (T16:00:00.000Z = 8 AM PST). Opus 4.6 completed the entire booking correctly. Eval-site TZ-storage bug. - gomail-8: goal says "Clear all emails from GitHub in the inbox", but criterion 3 expects exactly 1 email updated. Both K2.5 and Opus correctly cleared all 4 GitHub emails. Grader contradicts goal. - networkin-6: goal says "Choose a random person you haven't connected with"; grader hardcodes profilesDiff.updated."4".connectionGrade. Both models randomized correctly and missed id 4. Grader contradicts goal. - networkin-9: eval site's searchHistoryDiff doesn't record queries submitted via the autocomplete + Enter path. Opus 4.6 completed the task end-to-end (Stanford alum, connection request, message); only failed because the search-history criterion was never written server-side. Eval-site bug. Dataset goes from 45 -> 40 tasks. Score impact (same K2.5/Opus runs, recomputed against the cleaned 40-task denominator): K2.5: 21/45 (46.7%) -> 21/40 (52.5%) Opus 4.6: 28/45 (62.2%) -> 28/40 (70.0%) Δ: 15.6 pp -> 17.5 pp (real model gap, less pipeline noise) |
||
|
shivammittal274
|
e045e34b73 |
fix(eval): switch weekly eval configs from Fireworks to OpenRouter (#827)
The 2026-04-23 weekly run had 42% of AGISDK and 46% of Infinity tasks fail with `AI_RetryError: ... the service is overloaded` from Fireworks (20 concurrent kimi-k2p5 streams across both runs at 10 workers each). Switching to OpenRouter (which fronts the same Moonshot K2.5 weights and falls back across providers) for the three weekly configs: - browseros-agent-weekly.json - agisdk-real-smoke.json - infinity-hard-50.json Model accounts/fireworks/models/kimi-k2p5 -> moonshotai/kimi-k2.5 (same weights, same 262K context). API key env var, base URL updated. OPENROUTER_API_KEY is already wired into .github/workflows/eval-weekly.yml and present in repo secrets — no GH config changes needed. Orchestrator-executor configs and test_webvoyager left on Fireworks intentionally; can switch later if needed. |
||
|
shivammittal274
|
01d649da9a |
feat(eval): bring deterministic graders to dev + drop omnizon (#824)
* feat: deterministic eval graders (AGI SDK + WebArena-Infinity) (#664) * feat: add deterministic eval graders (AGI SDK + WebArena-Infinity) Two new benchmark integrations with programmatic grading — no LLM judge. AGI SDK / REAL Bench (52 tasks): - 11 React/Next.js clones of consumer apps (DoorDash, Amazon, Gmail, etc.) - Grader navigates browser to /finish, extracts state diff from <pre> tag - Python verifier checks exact values via jmespath queries WebArena-Infinity (50 hard tasks): - 13 LLM-generated SaaS clones (Gmail, GitLab, Linear, Figma, etc.) - InfinityAppManager starts fresh app server per task per worker - Python verifier calls /api/state and asserts on JSON state Infrastructure: - GraderInput extended with mcpUrl + infinityAppUrl for parallel workers - Each worker gets isolated ports (no cross-worker state contamination) - CI workflow: pip install agisdk, clone webarena-infinity repo * chore: switch eval configs back to kimi-k2p5 * fix: register deterministic graders in pass rate calculation Add agisdk_state_diff and infinity_state to PASS_FAIL_GRADER_ORDER in both runner types and weekly report script, so scores show correctly in the dashboard. * chore: temp switch to opus 4.6 for eval run * chore: restore kimi-k2p5 as default eval config * ci: add timeout and continue-on-error for trend report step * fix(eval): drop omnizon from AGISDK dataset (DMCA takedown) evals-omnizon.vercel.app returns HTTP 451 ("This content has been blocked for legal reasons / DMCA_TAKEDOWN"). All 5 omnizon-* tasks fail grading with "Failed to fetch /finish endpoint: JSON Parse error". Adds an EXCLUDED_WEBSITES set to the dataset builder and regenerates agisdk-real.jsonl (52 → 47 tasks). * fix(eval): correct Infinity port-assignment bugs Two related bugs in the Infinity eval runner that cause silent port collisions / fallbacks under parallel execution: 1. build-infinity-dataset.py emitted "app_port" but task-executor and the committed JSONL both read "app_base_port". Re-running the build script would silently make every task fall back to the 8000 default, ignoring per-app port assignments. Renamed the key to match. 2. task-executor derived workerIndex as `base_server_port - 9110`, but parallel-executor doesn't override base_server_port per worker — only server_url. Every worker computed workerIndex = 0, causing all parallel workers to spawn Infinity app servers on the same port. Threading workerIndex explicitly through TaskExecutor instead. Also drops an unused app_name parameter from load_tasks(). |
||
|
Dani Akash
|
ddbb2cf492 |
feat(agent): composer attachments + server-side outbound message queue (#826)
* feat(agent): attach images and text files to chat messages
Adds end-to-end support for image and text file attachments in the chat
composer, with the staged files round-tripping through the OpenClaw
gateway as OpenAI-compatible content blocks and persisting in the JSONL
so they show up in the historical view.
Server
- HTTP client: new OpenClawChatContentPart union and a buildUserContent
helper that emits multimodal content arrays when messageParts is
supplied, falls back to the legacy string content otherwise.
- Service: chatStream takes an optional messageParts array and forwards
it; BrowserOSChatHistoryItem gains an attachments field.
- JSONL reader: PiContentBlock learns the OpenAI image_url and Anthropic
image source/data shapes; user messages now emit user.attachment
events that the history mapper accumulates onto the next user item.
- Route: validates an inbound attachments[] (kind/mime/size/count),
inlines text-shaped files as <attachment> blocks in the message body,
attaches images via image_url parts. Replaces the immediate 409 on
active monitoring session with a 30s waitForSessionFree(agentId) wait
(registry now exposes onSessionEnd) so cron/hook contention does not
reject a user-chat send outright. Returns 503 if the wait times out.
Client
- New lib/attachments.ts: validateAttachment / compressImageIfNeeded
(canvas downscale to 2048px long edge, JPEG 0.85 re-encode for >1.5
MB inputs) / stageAttachment / stageAttachments that produces the
staged-attachment shape the composer renders and the payload the
server accepts.
- ConversationInput: drag-and-drop, paperclip button, clipboard paste,
staged attachment chip strip with thumbnails for images and a
paperclip+name chip for text files. Send button enables on either
text or attachments. Drop-zone overlay during drag.
- chatWithAgent forwards attachments[]; useAgentConversation.send
accepts a SendInput shape and renders user attachments on the
optimistic streaming turn via MessageAttachments / MessageAttachment.
- ClawChatMessage groups historical attachment parts into a single
MessageAttachments strip, ordered before reasoning/tools/text.
- claw-chat-types adds an attachment ClawChatMessagePart variant; the
history mapper emits attachment parts first and skips the text part
when the user only sent media.
- AgentCommandHome forwards the new SendInput shape — home composer
drops attachments at the boundary in v1 (the conversation page is
where staging is most useful; carrying bytes through the URL bar
is not sensible).
Limits: 10 attachments per message, 5 MB per image (post compression),
1 MB per text file, mime types png/jpeg/webp/gif and text/* +
application/json. PDFs and other binaries are deferred to v2.
* feat(agent): outbound message queue for chats while agent is mid-turn
Lets users keep typing and submitting messages while the agent is still
streaming a previous turn. Each press is appended to a single-flight
queue and dispatched as soon as `streaming` flips false; the queued
state renders as a strip above the composer so the user sees what's
pending vs. what's already sending.
- New `useOutboundQueue` hook owns the queue, the worker effect, and
cancel/retry actions. Single-flight by design — a re-entrancy ref
guard prevents two simultaneous dispatches when `streaming` flickers.
- Composer (`ConversationInput`) accepts optional `outboundQueue`,
`onCancelQueued`, `onRetryQueued` props. When the queue is provided
the send-button gate stops blocking on `streaming`; the spinner stays
as the visual cue that the agent is still busy. Legacy direct-send
callers keep the old streaming-blocks-send semantic.
- Renders an OutboundQueueStrip above the staged-attachment strip with
per-item status (queued / sending / failed), a cancel button on
queued items, and retry + discard on failed items.
- AgentCommandConversation wires `onSend` to `queue.enqueue` and routes
the home composer's `?q=` initial-message handoff through the queue
too, so it inherits the same single-flight serialization.
The server-side `waitForSessionFree` (added with attachments) and this
client-side queue together cover both contention sources: cron / hook
turns and back-to-back user sends. Persistence across reloads is
intentionally out of scope for v1 — losing the queue on extension
reload is documented as a known limitation.
* feat(server): server-side outbound message queue
Replaces the client-only React-state queue from
|
||
|
Dani Akash
|
711934555d |
feat(agent): enrich chat UI with tool activity, reasoning duration, and cost (#825)
* feat: pass per-turn cost and token data through chat history items
- Add costUsd, tokensIn, tokensOut to BrowserOSChatHistoryItem (server)
- Pass through from JSONL agent.message events in jsonlEventsToHistoryItems()
- Add same fields to client-side BrowserOSChatHistoryItem and ClawChatMessage
- Map cost/token data in mapHistoryItemToClawMessage()
Data flows: JSONL message.usage → server history item → API response →
client ClawChatMessage. Available for rendering in ClawChatMessage
component (message toolbar, cost badges).
* feat: add message toolbar with copy button and per-turn cost display
Add MessageToolbar to historical assistant messages in ClawChatMessage:
- Copy button copies message text to clipboard via MessageAction
- Per-turn token count (22.7K → 238) and cost ($0.003) shown as muted
tabular-nums text on the right side of the toolbar
- Toolbar appears on hover (opacity transition via group-hover)
- Only shown when the message has text content
- Cost/token display only shown when data is available from JSONL
* fix: toolbar only on assistant messages, always visible, cost only
- Only render toolbar on assistant messages (not user messages)
- Remove hover-only opacity — toolbar is always visible
- Remove token counts (22.7K → 238 is meaningless to users)
- Show only cost as a budget signal ($0.003)
* feat: group all tool activity into single Task collapsible per turn
Replace flat tool rows with a single ai-elements Task collapsible per
assistant turn that lists every tool/MCP call in sequence.
Live streaming (ConversationMessage):
- Aggregate all tool-batch parts into one Task
- Title: "Working… (N actions)" while running, "Agent activity (N actions)" when done
- Default open while turn is in progress
- Wrench icon in trigger
Historical (ClawChatMessage):
- Group all tool-call parts into one Task
- Title includes failed count if any tools errored
- Default collapsed — expandable on click
- Tool name + status icon + error text per row
Both views show one clean collapsible per turn instead of N individual
tool cards. Collapsed reads "5 actions"; expanded shows the timeline.
* feat: include tool calls in chat history responses
Server: jsonlEventsToHistoryItems() now walks ALL events (not just
messages) and pairs agent.tool_use with agent.tool_result by toolCallId.
The resulting tool call list is attached to the next assistant text
message as toolCalls[]. Each entry includes status, input arguments,
output text, error string, and duration computed from event timestamps.
Client:
- BrowserOSChatHistoryItem gets optional toolCalls field
- Tool-call message part type gets durationMs field
- mapHistoryItemToClawMessage() emits tool-call parts BEFORE the text
part (the order the agent produced them)
- ClawChatMessage Task view now shows tool duration in seconds
Result: historical messages now display the full tool activity
timeline grouped into the single Task collapsible per turn (designed
in step 3), instead of showing only the final text response.
* feat: render activity rows as human verbs sourced from tool registry
Tool calls in the chat activity view now read as sentences:
"Opened tab · news.ycombinator.com" instead of "browseros__new_page".
Server (tool-label-registry.ts):
- Curated verb override map for ~70 BrowserOS first-party tools
- Per-tool subject extractors that pull the meaningful argument from
input (URL → host, query → quoted, element → ID, etc.)
- Generic fallback humanizes snake_case for any unmapped tool
- Strips MCP namespace prefixes (browseros__, mcp_)
Server (openclaw-service.ts):
- jsonlEventsToHistoryItems calls buildToolLabel for each tool_use,
attaches label and subject to the BrowserOSChatHistoryToolCall
Client:
- Mirrored label module at lib/tool-labels.ts
- useAgentConversation tool-start handler computes label/subject
from the SSE tool args
- ClawChatMessage and ConversationMessage render label · subject
with foreground/muted styling, no font-mono
- ToolEntry, BrowserOSChatHistoryToolCall, and tool-call message
part types all carry label and optional subject
* fix: drop meaningless tab N subject from page-read tool rows
Page IDs are internal numbers, not URLs. 'Took screenshot · tab 4'
tells the user nothing. Removed subject extractors for take_snapshot,
take_enhanced_snapshot, get_page_content, get_page_links, get_dom,
and take_screenshot. The verb alone is the right signal.
* fix: gate initial loading on historyQuery.isFetched not isLoading
The session and history queries are sequential: the history query is
disabled until session resolves. After session resolves, there's a render
frame where historyQuery.isLoading is still false (the query hasn't
been kicked off yet). isInitialLoading flipped to false during that
window, exposing an empty chat shell with just Task collapsibles and
copy buttons before the messages filled in.
Switching the guard to isFetched closes that window — the loading state
stays true until the first history fetch actually completes.
* fix: render historical messages immediately instead of through Streamdown's idle-callback debounce
Streamdown defaults to mode="streaming" which uses requestIdleCallback (300ms
debounce, 500ms idle timeout) and lazy/Suspense to optimize for token-by-token
live streams. For finalized historical messages this caused tool collapsibles
and copy buttons to paint while text bodies stayed blank for ~300-500ms after
load. Pass mode="static" + parseIncompleteMarkdown=false on the historical
MessageResponse so completed text paints in the same frame as the surrounding
chrome. Live streaming turns still use the default streaming mode.
Also collapse the redundant /agents/:id/session round-trip into the existing
/history endpoint (server already resolves the most recent user-chat session
when sessionKey is omitted) and tighten the initial-loading gate to stay true
across the render frame where the query is enabled but hasn't started fetching.
* feat: surface thinking duration on historical reasoning collapsibles
Server accumulates agent.thinking events per turn from JSONL and attaches a
single reasoning block (joined text + durationMs from first thinking event
to the closing agent.message) on each assistant history item. Reasoning
buffer resets on user.message alongside the tool-call buffer.
Client mirrors the type, emits the reasoning part before tool calls in
mapHistoryItemToClawMessage (chronological: think → act → answer), and
passes duration in seconds to <Reasoning> so the trigger reads "Thought
for N seconds" instead of just "Thinking" on collapsed historical turns.
* fix: read thinking blocks from the correct JSONL field name
OpenClaw stores reasoning blocks as {type:'thinking', thinking:'...'} but
the JSONL parser was reading block.text, so every thinking event was
silently dropped before it ever reached jsonlEventsToHistoryItems. As a
result the reasoning field on history items was always empty even though
the new accumulator was wired up correctly.
Also guard the client mapping: when durationMs is 0 (think + answer
emitted in the same JSONL line, no real elapsed wall-clock) pass
undefined to <Reasoning> so it renders the static "Thinking" trigger
instead of the streaming shimmer / "Thought for 0 seconds".
* fix: reset reasoning buffer on discarded turns and drop dead session hook
Two cleanups from PR review:
1. jsonlEventsToHistoryItems: when an agent.message is discarded (the
"[Chat messages since your last reply" wrapper without a current-message
marker) the tool buffers were already reset but the reasoning buffer
was not. Accumulated thinking from the discarded turn would bleed onto
the next assistant message. Reset pendingReasoningTexts and
pendingReasoningFirstAt alongside the tool buffers.
2. useClawAgentSession, the AgentSessionResponse type, and the unused
session entry in CLAW_CHAT_QUERY_KEYS became dead code after the
session round-trip was folded into the history endpoint. Removed.
|
||
|
Dani Akash
|
0035893f33 |
feat: dashboard API, JSONL reader, and OpenClaw observer for enriched home page (#810)
* feat: draft agent chat ui exploration * feat: refine agent chat ui draft * feat: remove outer frame from agent chat workspace * fix: offset agent chat for app sidebar * fix: simplify agent conversation shell * fix: remove redundant chat header actions * fix: unify agent conversation headers * fix: tighten agent chat spacing * fix: bound agent chat composer height * fix: remove agent chat page inset * fix: align agent header height with sidepanel * fix: center agent composer resting state * fix: anchor multiline composer controls * fix: remove focus grid from agent home * fix: remove redundant agent home header * fix: constrain home agent composer * fix: match home composer default posture * feat: add openclaw chat history APIs * feat: add claw chat history hydration * fix: stabilize claw chat viewport layout * fix: use conversation scroll base for claw chat * refactor: split claw chat controller responsibilities * fix: keep active agent turns in memory * fix: normalize openclaw chat sessions * refactor: use HTTP client for agent history instead of CLI client Replace the CLI-based getChatHistory() call in getAgentHistoryPage() with the HTTP client's getSessionHistory() from PR #795. This uses the direct HTTP transport to OpenClaw's /sessions/<key>/history endpoint instead of shelling out through the CLI. - Add filterHttpSessionHistoryMessages() for flat-string content format - Add normalizeHttpHistoryMessages() for OpenClawSessionHistoryMessage shape - Update getAgentHistoryPage() to call getSessionHistory() via httpClient - Remove unused getChatHistory(), filterOpenClawSystemMessages(), normalizeChatHistoryMessages(), and getTextContent() - Update test mocks from cliClient.getChatHistory to httpClient.getSessionHistory - Update MutableOpenClawService type: chatClient -> httpClient * fix: fetch all session messages by iterating OpenClaw pagination OpenClaw's HTTP history endpoint returns a limited page by default. When called without a limit, only the first ~27 messages were returned, causing all newer conversation messages to be silently dropped. Add fetchAllSessionMessages() that iterates through OpenClaw's cursor- based pagination (200 messages per page) until hasMore is false, then feeds the complete message list into the existing BrowserOS normalization and in-memory pagination layer. * refactor: migrate chat history from HTTP gateway to direct JSONL file reads Replace the HTTP-based chat history pipeline (BrowserOS server → OpenClaw gateway /sessions/:key/history pagination loop) with direct JSONL file reads from the host filesystem via Lima's virtiofs mount. - Add OpenClawJsonlReader that reads session JSONL files directly from ~/.browseros/vm/openclaw/.openclaw/agents/<id>/sessions/ - Replace fetchAllSessionMessages() HTTP pagination with single file read - Replace CLI-based listSessions() with sessions.json file reads - Make listSessions, resolveAgentSession, getAgentHistoryPage synchronous - Remove unused toBrowserOSSession, filterHttpSessionHistoryMessages, normalizeHttpHistoryMessages helpers - Update route handlers to drop unnecessary async/await - Update tests to use temp JSONL files instead of mocked HTTP/CLI clients * fix: restore async route handlers for test compatibility with mocked service * fix: address review feedback — path traversal guard, lazy reader, exists flag - Add safePath() to OpenClawJsonlReader that validates resolved paths stay within stateRoot, preventing path traversal via crafted agentId values - Use lazy initialization for jsonlReader (nulled on rebuildRuntimeClients) instead of creating a new instance per property access - Return exists: false from resolveSpecificAgentSession when no session matches instead of fabricating a ghost session with sessionId: '' * feat: add dashboard API and enrich home page agent cards Server: - Add summarizeToolActivity() that converts tool events into natural language descriptions ("Browsed 3 pages, took 2 screenshots") - Add getDashboard() to OpenClawService that aggregates per-agent stats from JSONL: latest message, activity summary, cost, session count - Add GET /claw/dashboard endpoint Client: - Add useAgentDashboard() React Query hook (10s refetch, 5s stale) - Rewrite useAgentCardData from async IndexedDB hook to pure buildAgentCardData() function merging agent entries with dashboard data - Add activity summary and cost to AgentCardExpanded footer - Add activitySummary and costUsd fields to AgentCardData type - Remove IndexedDB dependency from the home page * feat: add OpenClawObserver for real-time per-agent status via gateway WS - Add OpenClawObserver that connects to the OpenClaw gateway WebSocket control plane and subscribes to chat broadcast events - Track per-agent status in real time: working (streaming), idle (turn complete), error (run failed), with current tool name - Auto-connect when gateway control plane becomes available, auto- reconnect on disconnect with 5s backoff - Disconnect observer on stop/shutdown - Wire live status + currentTool into getDashboard() response - Update client: AgentOverview includes status + currentTool, card shows spinning loader + tool name when agent is working - Status resolution: per-agent WS status takes precedence over gateway- level status for working/error states * feat: add SSE dashboard stream for real-time agent status on home page Server: - Add GET /claw/dashboard/stream SSE endpoint that sends an initial snapshot then pushes per-agent status events as they arrive from the OpenClaw observer - Add onAgentStatusChange() to OpenClawService exposing the observer's listener for the route layer - Heartbeat every 15s to keep connections alive Client: - useAgentDashboard() now subscribes to EventSource at /claw/dashboard/stream - SSE snapshot event hydrates the React Query cache immediately - SSE status events patch individual agent status + currentTool in the cache without refetching — agent cards update instantly - Polling fallback raised to 30s since SSE handles real-time * fix: observer WS handshake — wait for challenge before sending connect The OpenClaw gateway sends a connect.challenge event before accepting the connect request. The observer was sending the connect request on ws.open which raced with the challenge. Now waits for the challenge event before sending the handshake. Also add dangerouslyDisableDeviceAuth to the gateway setup config batch so the observer can connect without device identity on new installs. * fix: JSONL reader falls back to most recent file when sessions.json is stale OpenClaw's sessions.json can record a Pi session ID that doesn't match the actual JSONL filename on disk. This happens after context compaction or session restart — the JSONL file gets a new UUID but sessions.json keeps the old one. Previously this caused history to silently disappear (the reader tried to open a non-existent file and returned empty). Now resolveJsonlPath() checks if the mapped file exists and, when it doesn't, scans the sessions directory for the most recently modified .jsonl file as a fallback. * feat: add ClawSession state machine for reliable per-agent status The OpenClawObserver only knows about status changes it witnesses via WS events. If an agent was already running when the observer connected, or after a reconnect, statuses were stuck at "unknown". ClawSession is an in-memory state machine that solves this: 1. Seeds from JSONL on first control plane call — reads the latest events for each agent and infers working/idle. A session is "working" if the last event is a user.message with no subsequent agent.message, or an agent.tool_use with no matching agent.tool_result. 2. Receives live transitions from the WS observer — the observer now delegates all state management to ClawSession instead of maintaining its own status map. 3. Applies a 5-minute staleness threshold — if the last JSONL event is older than 5 minutes, assume idle (handles agent crashes). Consumers (SSE stream, dashboard endpoint) read from ClawSession and get correct state from the first call — no "unknown" period. * fix: remove staleTime so dashboard refetches on every mount * fix: reset stale working status on WS disconnect, eliminate redundant JSONL reads - Observer resets all "working" agents to "unknown" when the WS closes, preventing agents from appearing stuck as Working indefinitely after a gateway restart. ClawSession re-seeds correct state on reconnect. - getDashboard() now derives latestAgentMessage and cost from the already-loaded events array for the latest session instead of calling latestAgentMessage() and getSessionStats() which each re-read the same JSONL file. Reduces file reads from 3x to 1x per agent. |
||
|
Neel Gupta
|
4284e88625 |
feat: Implement lazy LLM judge for passive monitoring (#777)
* fix: double close on stream controller * feat: initial lazy llm judge impl * feat: added regex-based matching to insert button context * fix: tests & bugfix fix: redundant truthiness check * fix(tests): stabilize server suites on dev |
||
|
Nikhil
|
0b91c735ab | chore: bump server version, offset and patch for release (#814) | ||
|
Nikhil
|
d189b50b03 |
fix: package bundled Lima guest agent (#813)
* fix(build): upload Lima runtime files * fix(build): stage Lima prefix resources * fix(vm): resolve bundled Lima prefix * docs(build): document Lima runtime packaging * chore: self-review fixes * fix: address review feedback for PR #813 |