Compare commits

...

67 Commits

Author SHA1 Message Date
Nikhil Sonti
8cd7c985e3 fix: address review feedback for PR #891 2026-04-30 12:54:44 -07:00
Nikhil Sonti
548c9dd996 feat(dev): bootstrap setup from dev watch 2026-04-30 12:49:25 -07:00
Nikhil
d38b01a8c7 feat(dev): add guided cleanup and reset commands (#890)
* feat(dev): add guided cleanup and reset commands

* fix: address cleanup reset review feedback
2026-04-30 12:27:15 -07:00
Nikhil
ff36c8412b fix(dev): use run lock for watch cleanup (#889)
* fix(dev): use run lock for watch cleanup

* fix(dev): address watch lock review comments
2026-04-30 11:46:17 -07:00
Nikhil
fd5aba249b fix: stabilize OpenClaw gateway startup (#888)
* feat(server): add shared process lock helper

* feat(container): add container name reconciliation helpers

* feat(openclaw): serialize lifecycle across processes

* fix(openclaw): reconcile fixed gateway container startup

* test(openclaw): cover lifecycle race recovery

* fix(server): satisfy process lock error override

* fix(openclaw): address review feedback

* test(openclaw): align serialization mock with image check
2026-04-30 11:31:40 -07:00
Nikhil
492f3fcdf2 feat(openclaw): prewarm ghcr image in vm (#887)
* feat(openclaw): add gateway image inspection

* feat(openclaw): pull gateway image from registry

* refactor(vm): decouple readiness from image cache

* refactor(openclaw): remove vm cache from runtime factory

* feat(openclaw): detect current gateway image

* feat(openclaw): prewarm vm runtime and reuse current gateway

* feat(openclaw): prewarm runtime on server startup

* refactor(vm): remove browseros image cache runtime

* refactor(build-tools): remove openclaw tarball pipeline

* chore: self-review fixes

* fix(openclaw): suppress prewarm pull progress logs

* fix(openclaw): address review feedback

* fix(openclaw): resolve review findings

* fix(dev): stop stale watch supervisors
2026-04-30 11:18:11 -07:00
Nikhil
cb0c0dd0c1 chore: simplify root test scripts (#886)
* chore: simplify root test scripts

* fix: avoid chained root test scripts

* fix: update test workflow commands

* fix: move app test commands into packages
2026-04-30 10:58:08 -07:00
Dani Akash
8712f89f18 feat(agents): durable per-agent chat message queue + composer Stop (#880)
* feat(agents): durable per-agent chat message queue + composer Stop button

* fix(agents): tighten queue UI — smaller Stop, drop empty indicator, live drain attach

User feedback round 1 on the message-queue UX:

1) The Stop button matched the send/voice mics at h-10 w-10 with a
   solid destructive fill, which read as alarming. Shrunk to h-8 w-8,
   ghost variant with a soft destructive/10 background, smaller
   filled square glyph. Reads as a calm 'stop' affordance instead of
   a panic button.

2) The QueueItem's leading <QueueItemIndicator> dot was decorative
   only — no state, no interaction. Dropped it from QueuePanel along
   with the import; queue items now render as a clean preview line
   with the trailing X remove action.

3) When the server drained the queue and started the next turn, the
   chat panel didn't pick up the live stream until the user
   navigated away and back. The hook's resume effect previously
   only fired on agent change, not on listing-observed activeTurnId
   change. Surface activeTurnId from useHarnessAgents into
   useAgentConversation; effect now re-runs when the id changes,
   calls /chat/active, and attaches to the new turn — so a queued
   message starts streaming the moment the server drain pops it.

* fix(agents): don't reset streaming state from the resume effect's no-op paths

The Stop button was disappearing while the agent was actively
streaming, even though events were still flowing into the chat. Root
cause: the resume effect's `finally` block reset `streaming`,
`turnIdRef`, and `lastSeqRef` unconditionally — including on the
early-return paths (no active turn, or another mechanism already
owns the stream).

Sequence that triggered it:
  1) User sends a message → send() sets streamAbortRef + streaming=true
     and starts consuming the SSE.
  2) User enqueues another message → enqueue mutation invalidates the
     listing query.
  3) Listing refetches with the live activeTurnId → the resume
     effect re-fires (deps include activeTurnIdDep).
  4) attemptResume hits `if (streamAbortRef.current) return` because
     send() owns it.
  5) The finally clause fires anyway and calls setStreaming(false),
     clobbering the live state set by send(). The SSE consumer keeps
     running (refs are intact) so text keeps streaming, but the React
     flag is wrong, so the Stop button gates off.

Fix: track whether *this* run actually started a stream
(`weStartedStream`). The finally only resets state when it does.
Early-return / no-active-turn paths now leave streaming/turnIdRef/
lastSeqRef alone for whoever does own them.

Also widens the Stop button's visibility (`canStop` prop on
ConversationInput) so it stays steady across the brief gap between
turns when a queue drain is mid-flight; the parent computes
`streaming || activeTurnId !== null || queue.length > 0`. The
visibility widening is independent of the streaming-state fix above
— both are now in place.

* revert: drop canStop widening — Stop only shows while streaming

Reverts the canStop prop on ConversationInput and the OR-with-queue
visibility from AgentCommandConversation. Stop is gated solely on
`streaming` again. Between turns (queue draining) the button stays
hidden — only the actively-streaming turn is interruptible from the
composer, which matches what the user actually expects.

* fix(agents): persist the kicking-off prompt on active turns so the resume placeholder isn't empty

When a queued message drained and started a new turn, the chat
panel's resume effect staged a placeholder turn with userText: ''
because the hook had no way to know what message kicked off the
turn — only the agent-side stream was visible, and the user bubble
above it was blank until the user navigated away and back (at which
point the session record's history loaded normally).

Fix: ActiveTurnRegistry.register now accepts an optional `prompt`
that's stashed on the turn and surfaced via describe() / the
ActiveTurnInfo response. AgentHarnessService.startTurn passes the
incoming message into register. /chat/active returns it. The chat
hook's resume effect uses active.prompt as the placeholder
turn's userText, so the user bubble shows the queued message text
the moment streaming begins. Falls back to '' for older clients
that haven't been refetched yet.

* fix(agents): always release streamAbortRef on resume cleanup, even when cancelled

Greptile P1 follow-up. The previous `weStartedStream` guard correctly
stopped the resume effect's no-op early-returns from clobbering an
in-flight `send()` stream — but it also stopped a *cancelled*
mid-stream resume from clearing its own `streamAbortRef`. When the
cleanup fires (e.g. the 5s listing poll captures a new queue-drain
turn id while the SSE for the prior turn is still finishing), the
next effect run hits the `if (streamAbortRef.current) return` guard
against the now-aborted controller and never reattaches, leaving
`streaming === true` with no live stream until the user navigates
away.

Split the finally block: always release `streamAbortRef` when we
owned the controller (so the next run can take over), but only
reset the streaming flag / turn id / lastSeq on a clean exit (the
new run will set those itself, so resetting on cancel would just
flicker).
2026-04-30 18:26:56 +05:30
Dani Akash
ba60bf466f feat(agents): rich command-center rows + home grid + dead-code sweep (#879)
* feat(agents): rich-info command center rows + pin/PATCH/adapter-health backbone

Splits AgentRowCard from a 271-line monolith into a shallow tree of
single-responsibility sub-components under `agent-row/`:

  AgentTile, AdapterHealthDot, PinToggle, AgentTitleRow,
  AgentSparkline, AgentSummaryChips, AgentLastMessage, CwdChip,
  AgentTokenSummary, AgentMetaRow, AgentErrorPanel, AgentActions

Adds the data each row consumes:

- pinned: boolean field on AgentDefinition + FileAgentStore.update
  + new PATCH /agents/:id route. useUpdateHarnessAgent mutation
  optimistically updates the listing cache so the star flips
  instantly; rolls back on error.
- Listing payload extended with lastUserMessage, cwd, tokens
  (cumulative + last7d shape — last7d zero-filled until the
  activity ledger lands), turnsByDay/failedByDay (zero-filled),
  lastError/lastErrorAt, activeTurnId. AcpxRuntime grows a
  getRowSnapshot() that reads cwd + cumulative tokens + last user
  message from the session record in one pass.
- Adapter health: in-memory AdapterHealthChecker probes
  `claude --version` / `codex --version` with a 2s timeout and
  caches results for 5 min. /adapters response carries
  { healthy, reason?, checkedAt }. Tile-corner dot exposes the
  state via HoverCard; openclaw inherits health from the gateway
  snapshot already on the page.

Sub-components are pure: card itself owns no state. Sort order
becomes pinned-first, then recency. HoverCard is the workhorse for
keeping rows compact while exposing depth (full message, token
breakdown, daily turn list, error stack, adapter reason).

* refactor(agents): tighten command-center row design + cut redundant affordances

User feedback round 1:
1) Two green dots on the tile (health + liveness) was confusing. Health
   moves out of the tile entirely and surfaces as an inline 'Unavailable'
   chip in the model line — silent when the adapter is healthy, with a
   warning amber chip + HoverCard reason when not. The tile now shows
   one signal: liveness.
2) The last-user-message HoverCard wasn't telegraphing intent. Drop the
   HoverCard. The line is informational, italic, with a leading quote
   glyph so the row reads like a conversation snippet. To see the full
   message the user opens the chat (which is the action they want next
   anyway).
3) Resume + Chat were duplicate CTAs. Single primary action per row:
   Resume (filled, accent-orange, with a pulsing dot) replaces Chat
   when there's an active turn. Both navigate to /agents/:id but the
   row tells the user which action they're taking.
4) Tokens weren't visible because the row gated on last7d.requestCount,
   which is zero until the activity ledger ships. Switch to lifetime
   tokens (which we have today). Drop the '7d stats:' framing — talking
   about a window we can't compute would be misleading. The HoverCard
   surfaces input/output split + a footnote that per-window stats land
   in a follow-up.
5) CWD was rendering the server's own running directory, which is
   meaningless to users. Hide it from the row entirely. The cwd field
   still rides in the listing payload for future surfaces (chat panel,
   debug view) — only the row stops rendering it.

Aesthetic refinements while we're here:
- Whole card carries state, not just the tile: working rows get an
  accent-orange tinted border with a soft glow, error rows tint
  destructive, idle rows lift on hover.
- Pin star fades in on hover (group-hover) when unpinned and stays
  solid amber when pinned — keeps the rail calm by default.
- Tabular-nums on token figures so columns visually align across rows.
- Drop CwdChip and AdapterHealthDot files: no callers left.

* fix(agents): align row title flush-left whether pinned or not

Pin star moved from leading the title to trailing the badges, and
hidden from layout entirely (`hidden group-hover:inline-flex`) when
unpinned. The previous `opacity-0` rule kept the star reserving its
`size-6` slot, which left every unpinned title indented relative to
the model / preview / meta lines underneath it. Title now flushes
left in both states; pinned star stays solid amber so the signal
isn't hidden, and unpinned reveals an outline star on row hover for
the toggle affordance.

* fix(agents): keep pin-toggle slot reserved so row height is constant

Switching the unpinned star from `hidden group-hover:inline-flex`
to `opacity-0 group-hover:opacity-100`. The hidden/show variant was
collapsing the title row's height when the star wasn't rendered,
which made every card below visibly shift on hover. Always rendering
the button (with opacity-only visibility) keeps the row's vertical
metrics constant; the title still flushes left because the slot is
trailing, not leading.

Card hover effect (-translate-y + shadow-md) restored — the layout
shift wasn't coming from the card hover; it was the pin slot
appearing and disappearing.

* fix(agents): quieten row hover — border-tint only, no lift, no shadow

Drop the `-translate-y-px` and `hover:shadow-md` from the row card
plus the working-state inner ring. The translate + shadow grow
combination was visibly noisy as the cursor moved through the rail —
each row 'lifted' as you passed over it. Hover now just tints the
border in accent-orange/30; working and error states keep their
distinct border colours but no inner ring. Card height and shadow
stay constant in every state, so the rail reads as a calm vertical
list of cards.

* feat(home): rich Recent Agents grid + dead-code sweep

The /home Recent Agents grid was a placeholder shell. Every 'rich'
field on the card (lastMessage, lastMessageTimestamp, activitySummary,
currentTool, costUsd) was wired to undefined because AgentCommandHome
called `buildAgentCardData(agents, status?.status, undefined)` — the
dashboard arg has been hard-coded undefined since the harness
migration. Repointing the grid at `useHarnessAgents` + `useAgentAdapters`
gives every card the same enriched data the rail uses.

What the new card shows per agent:
  • Adapter glyph tile + liveness dot (working pulses; asleep is
    hollow; error is red)
  • Name + Working pill (when active)
  • Adapter · model · reasoning summary line, with an inline
    Unavailable chip + HoverCard reason when the adapter binary
    isn't on $PATH
  • Italic last-user-message preview (line-clamp-2, leading quote
    glyph) — same visual language as the rail
  • Footer: 'X ago' + state chip (Asleep / Attention) OR a Resume
    button (orange, with pulsing dot) when activeTurnId is non-null

Sort on the home grid is active-turn → recency. Pinning is NOT a
sort key here (and there's no pin indicator on the card) — pinning
belongs to the rail at /agents; the home page is action-oriented
and trusts active-turn + recency to surface the right agent.

Dead code removed:
  • useAgentDashboard.ts (96 lines, no callers; subscribed to the
    dead /claw/dashboard/stream from the OpenClaw-only era)
  • useAgentCardData.ts (the dashboard-merge shim; passed undefined
    every call so all enriched fields landed as undefined)
  • AgentCard.tsx (AgentCardExpanded replaced by HomeAgentCard;
    AgentCardCompact had no callers — the dock's compact mode was
    never used)
  • AgentCardData interface dropped from lib/agent-conversations/
    types.ts; the new card consumes HarnessAgent directly

Visual language stays continuous between rail and grid: same
<AgentTile>, same <LivenessDot>, same italic-quote message
preview, same orange Resume button with a pulsing dot.
2026-04-30 16:36:22 +05:30
Nikhil
26afb826c6 feat(eval): add viewer manifest contract (#878)
* refactor(eval): canonicalize viewer manifest contract

* refactor(eval): publish canonical viewer manifests

* feat(eval): make r2 viewer use manifest artifact paths

* fix(eval): keep weekly report compatible with viewer manifests

* docs(eval): document r2 viewer manifest contract

* chore: self-review fixes

* fix: address review feedback for PR #878
2026-04-29 20:50:35 -07:00
Nikhil
b2340c8afa refactor(eval): split orchestrated executor backends (#876)
* refactor(eval): split orchestrated executor backends

* fix(eval): address executor backend review comments
2026-04-29 18:02:32 -07:00
Felarof
790a270f47 Update README.md (#877) 2026-04-29 17:35:15 -07:00
Nikhil
84a79ba0a1 feat: refactor eval pipeline workflow (#875)
* feat(eval): add suite variant config bridge

* feat(eval): add stable run artifacts

* refactor(eval): add shared grader contract

* feat(eval): persist grader artifacts

* refactor(eval): rename runner layers

* refactor(eval): add executor backend boundary

* refactor(eval): split clado backend

* feat(eval): add workflow compatible cli

* feat(eval): add r2 publisher module

* ci(eval): migrate weekly workflow to eval cli

* docs(eval): document suite pipeline

* chore(eval): verify pipeline refactor

* fix: address review feedback for PR #875

* docs(eval): add env example

* docs(eval): explain suites and variants

* chore(eval): organize config layouts

* chore(eval): colocate grader python evaluators
2026-04-29 17:21:02 -07:00
Nikhil
6e3306f5e5 fix: make R2 uploads retryable (#874)
* fix: make R2 uploads retryable

* fix: address review feedback for PR #874
2026-04-29 16:43:33 -07:00
Nikhil
c244462b29 fix: use Node 24 GitHub actions (#872) 2026-04-29 15:31:23 -07:00
Nikhil
ebf97f74f6 fix: bound VM agent cache smoke test (#870)
* fix: bound VM agent cache smoke test

* fix: address review feedback for PR #870
2026-04-29 13:43:37 -07:00
Nikhil
561f2baf97 fix(eval): split AGISDK smoke and full configs (#871)
* fix(eval): split agisdk smoke and full configs

* fix(eval): default agisdk smoke to openrouter
2026-04-29 13:38:55 -07:00
shivammittal274
df0f45dd29 Feat: eval debug dev ci (#869)
* chore(eval): instrument server startup to root-cause dev CI health-check timeouts

Three diagnostics + one config swap to investigate why the eval-weekly
workflow has been failing on dev since 2026-04-25 with "Server health
check timed out" (every worker, every retry).

Background:
- Last successful weekly eval on dev: 2026-04-18 (sha f5a2b73)
- Since then, ~30 server commits landed including Lima/VM runtime,
  OpenClaw service, ACL system, ACP SDK — 108 server files changed,
  ~13K LOC added.
- Server process spawns cleanly in CI (PID logged) but never binds
  /health within the 30s eval-side timeout. Static analysis finds no
  obvious blocker; we need runtime evidence.

Changes:

1. apps/server/package.json — add `start:ci` script (no `--watch`).
   The default `start` uses `bun --watch` which forks a child process
   that watches every file in the import graph. Dev's graph is ~108
   files larger than main's; on a cold CI runner the watcher setup is a
   plausible source of multi-second startup overhead.

2. apps/eval/src/runner/browseros-app-manager.ts:
   - Use `start:ci` when `process.env.CI` is set (true on
     GitHub-hosted runners by default), else `start`.
   - Capture per-worker server stderr to /tmp/browseros-server-logs/
     instead of ignoring it. Without this we have no visibility into
     why the server is hung pre-/health.
   - Bump SERVER_HEALTH_TIMEOUT_MS 30s -> 90s. Dev's larger module
     graph may simply need more cold-start time on CI.

3. .github/workflows/eval-weekly.yml — upload the server logs dir as a
   workflow artifact (always, not just on success) so we can post-mortem
   any startup failure on the next run.

4. configs/agisdk-real-smoke.json — swap K2.5 from OpenRouter ->
   Fireworks (bypasses the OpenRouter per-key spend cap that has been
   eating recent runs) and drop num_workers 10 -> 4 (well below the
   Fireworks per-account TPM threshold that overwhelmed the original
   2026-04-23 run).

Plan: trigger the eval-weekly workflow on this branch with the agisdk
config and observe (a) whether it gets past server startup, and
(b) if it doesn't, what the captured server stderr says.

* fix(eval): capture stdout too — pino logger writes to stdout, not stderr

Previous diagnostic patch only redirected stderr; the captured per-worker
log files came back as 0 bytes because the server uses pino which writes
all log output to stdout (fd 1), not stderr (fd 2). Capture both into
the same file.

* fix(server): catch sync throw from OpenClaw constructor on Linux

The container runtime constructor in OpenClawService throws synchronously
on non-darwin platforms, e.g. GitHub Actions Linux runners. The existing
.catch() on tryAutoStart() only handles async throws inside auto-start —
the sync throw from configureOpenClawService(...) itself propagates up
through Application.start() and crashes the process via index.ts:48
(process.exit(EXIT_CODES.GENERAL_ERROR)).

This is what's been killing dev's eval-weekly CI: the server crashes in
milliseconds, the eval client polls /health, gets nothing, times out.

Fix: wrap the configureOpenClawService call in try/catch matching the
existing .catch() intent (best-effort, don't crash). Server continues
without OpenClaw on platforms where it can't initialize.

Verified by reading captured server stdout from run 25123195126:
  Failed to start server: error: browseros-vm currently supports macOS only
    at buildContainerRuntime (container-runtime-factory.ts:54:11)
    at new OpenClawService (openclaw-service.ts:652:15)
    at configureOpenClawService (openclaw-service.ts:1527:19)
    at start (main.ts:127:5)

* fix(server): defer OpenClaw chat client port lookup to request time

apps/server/src/api/server.ts:149 was calling getOpenClawService().getPort()
synchronously when constructing the OpenClawGatewayChatClient inside the
createHttpServer object literal. On non-darwin platforms this throws via
the OpenClawService constructor → buildContainerRuntime, escaping the
try/catch added in 5cf7b765 (which only protected the configureOpenClawService
call further down in main.ts).

Every other getOpenClawService() reference in server.ts is already wrapped
in an arrow function. This was the lone holdout. Make it lazy too: change
the chat client constructor to take getHostPort: () => number instead of
hostPort: number, evaluate it inside streamTurn at request time. Behavior
on darwin is unchanged.

This unblocks dev's eval-weekly CI on Linux runners where OpenClaw isn't
available — the chat endpoint isn't exercised by the eval, so a deferred
throw is acceptable.

* fix(server): allow Linux to skip OpenClaw via BROWSEROS_SKIP_OPENCLAW=1

Earlier surgical fixes (try/catch in main.ts, lazy chat client port) didn't
unblock dev's Linux CI — same throw kept reproducing. Whether this is bun
caching stale stack frames or a missed eager call site, the safer move is
to fix it at the root: make buildContainerRuntime never throw on Linux
when the runner has explicitly opted out.

Adds BROWSEROS_SKIP_OPENCLAW env check alongside the existing NODE_ENV=test
escape hatch in container-runtime-factory.ts. When set, returns the existing
UnsupportedPlatformTestRuntime stub — server boots normally, /health binds,
any actual OpenClaw API call still fails loudly at request time.

eval-weekly.yml sets the flag for the Linux runner. Darwin behavior and
non-CI Linux behavior unchanged (without the flag they still throw).

* feat(eval): align Clado action executor with new endpoint contract

David Shan shared the updated Clado BrowserOS Action Model spec.
Changes to match it:

- Bump endpoint URL + model id to the 000159-merged checkpoint
  (clado-ai--clado-browseros-action-000159-merged-actionmod-f4a6ef)
  in browseros-oe-clado-weekly.json and the README example.
- CLADO_REQUEST_TIMEOUT_MS 120s → 360s. Cold start can take ~5 min;
  the 2-min ceiling was failing every cold-start request.
- Treat HTTP 200 with action=null / parse_error as an INVALID step
  instead of aborting the executor loop. The model can self-correct
  on the next call. Cap consecutive parse failures at 3 to avoid
  infinite loops.
- Capture final_answer from end actions. Surface it in the observation
  back to the orchestrator so its task answer can use the model's
  declared result.
- Add macOS Cmd-* key mappings (M-a, M-c, M-v, M-x → Meta+A/C/V/X).
- Switch screenshot format from webp → png to match the documented
  "PNG or JPEG" contract.

* chore(eval): refresh test-clado-api script for new Clado contract

Updated the local smoke-test to match the new Clado endpoint and
response contract:

- New action + health URLs (000159-merged checkpoint).
- Drop the grounding-model branch (orchestrator-executor doesn't
  use it; the README David shared only documents the action model).
- Health-check waits up to 6 minutes for cold start with a 30s
  warning so the operator knows it's spinning up.
- Print every documented response field (action, x/y, text, key,
  direction, amount, drag start/end, time, final_answer, thinking,
  parse_error, inference_time_seconds).
- Three-step run that exercises a click, a typing continuation
  with formatted history, and an end+final_answer probe.

* chore(eval): point clado weekly config at agisdk-real

Switches the orchestrator-executor + Clado weekly config to run on the
AGI SDK / REAL Bench task set with the deterministic agisdk_state_diff
grader. Matches the orchestrator-executor smoke target (Fireworks K2.5
orchestrator + Clado action executor) we want to track week-over-week.

* chore(eval): run clado weekly headless

Default to headless so the weekly job (and local repros) don't pop ten
visible Chrome windows. Set headless=false locally if you need to watch
a worker.

* fix(eval): address Greptile P1+P2 on server log fd handling

P1: openSync was outside the mkdirSync try/catch, so a swallowed mkdir
failure (e.g. unwritable custom BROWSEROS_SERVER_LOG_DIR) would leave the
log directory missing and crash the server spawn with ENOENT. Move openSync
into the same try block; fall back to /dev/null so spawn always succeeds.

P2: the log fd was opened on every server start but never closed. Each
restart attempt leaked one fd across all workers — over a long eval run
that could exhaust the process fd limit. Track the fd on the manager and
closeSync it in killApp() right after the server process exits (the child's
dup keeps the file open until it exits, so we don't truncate output).
2026-04-30 01:33:49 +05:30
Nikhil
edfc5c751c fix: align OpenClaw gateway image with VM cache (#868)
* fix: load OpenClaw gateway image from VM cache

* fix: use container port for OpenClaw ACP bridge

* fix: address review feedback for PR #868
2026-04-29 12:11:00 -07:00
Nikhil
471256f31c fix: stop passing native permission flags to ACP adapters (#867) 2026-04-29 11:07:51 -07:00
Nikhil
4c90ca696b fix(agents): connect OpenClaw ACP inside gateway container (#866) 2026-04-29 11:07:29 -07:00
Nikhil
f2ac87d7c3 feat: show created agents in sidepanel (#865)
* feat(agent): list created agents in sidepanel target catalog

* feat(agent): show created agents in sidepanel selector

* feat(server): add sidepanel chat route for created agents

* feat(agent): route sidepanel agent sends by agent id

* chore(agent): retire virtual sidepanel acp targets

* fix: address review feedback for PR #865
2026-04-29 10:15:58 -07:00
shivammittal274
231bd6821d fix(eval): pin agisdk version + exclude 4 invalid tasks (Phase 2 dataset hygiene) (#844)
* chore(eval): pin agisdk version to prevent silent dataset drift

`pip install agisdk` previously fetched whatever version pip resolved at
CI time. If agisdk publishes a new version with changed task definitions
or grader behavior, the weekly eval silently shifts under our feet —
making "did the score move because of code or data?" unanswerable.

Pin to agisdk==0.3.5 (the version we currently develop against). Bump
intentionally with a documented re-baseline run.

* fix(eval): exclude 4 more tasks identified by 8-trial never-passing audit

After 8 trials across K2.5 + Opus 4.6 (Phase 1 and Phase 2), 5 tasks
never passed. Per-task root-cause investigation via parallel deep-dive
subagents flagged 4 of them as fundamentally unfixable in the eval
pipeline as it stands; the 5th (`dashdish-5`) is a prompt-rule fix
that stays in.

- gocalendar-7: goal/grader contradiction. Goal says "move event to
  July 19, 10 AM"; grader expects `eventsDiff.updated.*.start ==
  "2024-07-18T17:00Z"` (= July 18, 10 AM PDT — same day, 1 hour shift).
  Even after the Phase 2 HTML5 dnd dispatch fix correctly populates
  `eventsDiff.updated`, the values are July 19 (matching the goal),
  which the grader rejects.

- staynb-5: grader hardcodes literal `'Oct 13 2025'` and `'Oct 23 2025'`
  year strings. The staynb date picker interprets bare "Oct 13" as the
  most-recent-past instance (currently 2024 since today is 2026), not
  2025. No agent can produce a persisted date string containing 2025.

- staynb-9: under-specified task. Goal says "maximum number of guests
  supported"; grader requires the very specific string "32 Guests, 16
  Infants" — encoding UI knowledge (Adults+Children=Guests display,
  Infants render separately, per-category cap=16, Pets excluded) that
  isn't in the prompt. Even Opus 4.6 stopped at 16 across 3 trials.

- opendining-3: grader requires `contains(booking.date, '2024-07-20')`
  but the React-controlled date textbox flakily no-ops on `fill`. 3/8
  trial pass rate is essentially coin-flip noise driven by tool-fidelity
  variance rather than agent capability. Removing to reduce score noise;
  Phase 2 fill post-validate warning helps when it does work, but the
  task's signal-to-noise is too low for the eval set.

Dataset goes from 40 -> 36 tasks. Total EXCLUDED_TASKS now 11 entries.

Validated by 8-trial pass-record audit; deep-dive notes saved to
plans/audits/.
2026-04-29 22:07:53 +05:30
Dani Akash
a228c278c6 feat(agents): background-resilient chat — turns survive tab disconnect (#863)
* feat(agents): decouple chat turn lifecycle from SSE response

Introduce a per-process ActiveTurnRegistry that owns each agent turn's
lifecycle and a ring-buffered event stream, so chat tabs that close,
refresh, or navigate away no longer cancel the in-flight turn. New
endpoints:

  POST   /agents/:id/chat          starts a turn (now returns 409 when
                                   one is already running, with the
                                   active turnId for attaching)
  GET    /agents/:id/chat/active   reports the running turn for a UI
                                   that just mounted
  GET    /agents/:id/chat/stream   subscribes to a turn; supports
                                   Last-Event-ID resume via per-event
                                   seq ids
  POST   /agents/:id/chat/cancel   explicit cancel — fetch abort no
                                   longer affects the underlying turn

The chat hook now captures X-Turn-Id, tracks lastSeq from SSE id lines,
re-attaches on mount when the server still has an active turn, and
routes Stop through the cancel endpoint. The runtime call uses the
registry's per-turn AbortController instead of the HTTP request signal,
which is the core decoupling that lets turns outlive their initiator.

* feat(agents): add ActiveTurnRegistry primitive backing the new chat lifecycle

The previous commit referenced these files in tests and the harness
service but global gitignore swallowed them on the first add.

The registry owns the per-turn ring buffer (drop-oldest, terminal frame
preserved), the per-turn AbortController, and subscriber fan-out used
by /chat/stream resume.
2026-04-29 21:01:06 +05:30
Dani Akash
e2ec1991cf feat(agents): redesign the agent command center for multi-adapter use (#861)
* feat(agents): redesign agent rail to match the rest of the app

Reshape the `/agents` page so it reads as a sibling of `/scheduled`
and `/soul` and adapts to the multi-adapter world (OpenClaw, Claude
Code, Codex). Visual scaffolding only in this commit — per-agent
liveness state ships as `unknown` until the server-side activity
tracker lands.

  - New `AgentsHeader` mirrors `SoulHeader`/`ScheduledTasksHeader`:
    accent bot tile, title, descriptive subtitle, "+ New Agent"
    button. Replaces the loose top toolbar that mixed page-level and
    OpenClaw-lifecycle controls.
  - New `GatewayStatusBar` collects the OpenClaw lifecycle pills
    (running, control plane connected) plus the Terminal/Refresh
    affordances into a single labeled bar that only renders when the
    gateway is running AND there is at least one OpenClaw agent in
    the merged list.
  - New `AgentRowCard` per agent: adapter tile with liveness dot,
    name + status badge, adapter/model/reasoning chips, last-used
    relative time + truncated workspace path, primary "Chat" button,
    overflow menu (Copy id / Rename* / Reset history* / Delete).
    Rename + Reset are disabled with "coming soon" tooltips until
    the corresponding endpoints ship; Delete is hidden for the
    protected `main` agent.
  - New `AgentsEmptyState` mirrors the scheduled-tasks empty card.
  - New `AdapterIcon` + `LivenessDot` + `agent-display.helpers.ts`
    keep the row card focused on layout; helpers cover display name
    fallbacks for legacy `oc-<uuid>` titles, workspace label rules,
    and a tiny relative-time formatter.
  - `AgentList` now sorts by `lastUsedAt` desc with `null`s falling
    to the bottom; the gateway's `main` agent is pinned to the top
    only while it has zero turns so a fresh install has an obvious
    starting point. The list also threads a per-agent activity map
    so future commits can light up working/idle/asleep without
    reshuffling the API.
  - `AgentsPage` swaps to the standard `fade-in slide-in-from-bottom-5
    animate-in space-y-6 duration-500` shell and threads a
    `harnessAgentLookup` Map down to the row card so adapter chips
    and reasoning effort render correctly without a re-fetch.

* feat(agents): wire per-agent liveness end-to-end into the rail

Closes the placeholder `unknown` dot from the redesign's first
commit. The rail now shows real working / idle / asleep / error
states per agent, with `lastUsedAt` driving the recency sort.

Server side:
  - `AgentHarnessService` keeps an in-memory activity tracker keyed
    by agentId. `notifyTurnStarted` flips an entry to `working`,
    `notifyTurnEnded({ok})` either drops it (success) or pins it to
    `error` (failure / error event).
  - `send()` wraps the runtime stream so the lifecycle hook fires
    exactly once on natural close, error event, downstream cancel,
    or thrown setup. The runtime itself stays unchanged — fork is
    contained at the harness layer.
  - New `listAgentsWithActivity()` method enriches every agent with
    `{ status, lastUsedAt }`. lastUsedAt is read from the acpx
    session record's last persisted item via `runtime.getHistory`,
    so it survives server restart even though the activity map
    doesn't.
  - Status derivation: `working`/`error` take precedence; otherwise
    timestamp-based — `idle` until 15 min of silence, then `asleep`.
    Never-used agents resolve to `idle` (asleep implies "was active,
    went quiet").
  - `GET /agents` returns the enriched shape.

Client side:
  - `HarnessAgent` UI type extended with optional `status` +
    `lastUsedAt` so older deployments still typecheck.
  - `useHarnessAgents` flips on `refetchInterval: 5_000` (with
    `refetchIntervalInBackground: false` so hidden tabs go quiet)
    so the per-row dots and last-used copy stay fresh without a
    websocket.
  - `AgentsPage` builds an activity map from the harness listing
    response and threads it into `AgentList` → `AgentRowCard`. The
    sort by `lastUsedAt` desc (already in the row card) now has
    real data to operate on.

Tests:
  - New `marks an agent working while a turn streams and idle once
    it ends` exercises the wrap; uses a held upstream stream so
    the in-flight `working` state is observable.
  - New `flips to error when a turn emits an error event`.

* fix(agents): dedupe agent rail when /claw/agents and /agents share an id

The agents page was rendering every OpenClaw agent twice — once from
the legacy `/claw/agents` listing (`useOpenClawAgents`) and once from
the harness `/agents` listing (`useHarnessAgents`). Post Step 9
backfill the harness store contains every gateway agent, so the
overlap is the rule, not the exception.

Mirror the dedup the chat-panel layout already does: when a gateway
agent's id appears in the harness listing, drop the legacy entry and
keep the harness one (it has adapter/model/reasoning/status/lastUsedAt
the chat path actually consumes).

* feat(agents): swap GatewayStatusBar refresh icon for a Restart Gateway button + tooltips

The manual refresh became redundant once `useHarnessAgents` and
`useOpenClawStatus` started polling on a 5s interval — every visible
field self-refreshes within seconds. The previous AgentsPageHeader
had a real Restart action that the redesign dropped; reinstate it on
the bar so a wedged gateway is one click away again.

  - GatewayStatusBar: dropped the `RotateCcw` refresh icon and the
    `onRefresh` prop. Added `onRestart` + `actionInProgress` props;
    the button shows a spinner while a gateway lifecycle mutation is
    in flight.
  - Both Terminal and Restart Gateway buttons get tooltips explaining
    what they do — Terminal as a power-user shell escape hatch,
    Restart for unsticking a wedged gateway or after manual config
    edits.
  - AgentsPage: drop the now-unused `refreshAll` helper and the
    `refetchStatus`/`refetchAdapters`/`refetchOpenClawAgents`
    destructures it depended on. Wire `restartOpenClaw` (already
    pulled from `useOpenClawMutations`) through
    `runWithPageErrorHandling` like the legacy header did.

* feat(agents): consolidate gateway status into the /agents listing

Folds the gateway lifecycle snapshot into the harness listing so the
agents page polls one endpoint instead of two. Drops the dead
`/claw/status` call from the command center while keeping every UI
affordance the page already shipped (Running / Control plane
connected pills, GatewayStateCards setup/start prompts,
ControlPlaneAlert for degraded states).

Server side:
  - `OpenClawProvisioner.getStatus()` (optional) — when wired, returns
    the same `GatewayStatusSnapshot` shape `/claw/status` does.
  - `AgentHarnessService.getGatewayStatus()` — best-effort wrapper
    around the provisioner method; logs and swallows errors so a
    transient gateway issue doesn't 500 the listing endpoint.
  - `GET /agents` now returns `{agents, gateway}` in a single
    `Promise.all`. Both fields are independent — agents enrichment
    succeeds even if the gateway snapshot is null.
  - `server.ts` wires `getOpenClawService().getStatus()` into the
    provisioner accessor object alongside `createAgent` /
    `removeAgent` / `listAgents`.

Client side:
  - `useHarnessAgents` returns `{harnessAgents, gateway}` (plus the
    legacy `agents` mapping). Same 5s `refetchInterval` as before —
    one round-trip drives the per-row liveness AND the gateway pills.
  - `AgentsPage` drops `useOpenClawStatus` entirely; `status` comes
    from the harness query. Loader + error/lifecycle plumbing
    rewired around the harness query's loading/error.
  - `agents-page-utils.getInlineError` and `getAgentsLoading` lose
    the now-redundant `statusError` / `statusLoading` /
    `openClawAgentsEnabled` params.

The chat-panel layout (`agent-command-layout.tsx`) still consumes
`useOpenClawStatus(5000)` for now — left intact per the user's "only
the command center" scope. Folding that one in is a separate,
smaller pass once we're sure no regression slipped here.

* test(agents): teach the route fake service about the new listing shape

PR #861 CI surfaced two failures in tests/api/routes/agents.test.ts:
both call \`GET /agents\` and the route handler now invokes
\`service.listAgentsWithActivity()\` + \`service.getGatewayStatus()\`
which the fake created here didn't implement. Add both methods to
the fake (returning idle / null) and update the empty-list assertion
to expect the new \`{agents, gateway}\` envelope.
2026-04-29 19:03:29 +05:30
Dani Akash
0c84547e8f feat(agents): migrate OpenClaw chat onto the unified harness/ACP path (#859)
* chore(acp): smoke-test ACP capabilities against running gateway

Adds apps/server/scripts/acp-smoke.ts which spawns `openclaw acp`
inside the gateway container and exercises every method we plan to
depend on: initialize, newSession, prompt (text + image), cancel,
listSessions, loadSession.

SDK pinned to 0.19.1 (Bun's minimum-release-age policy blocks 0.20+
which were released < 7 days ago).

Findings (full notes in plan outcomes):
- promptCapabilities advertises image:true but the model does NOT see
  image bytes — silently dropped at the bridge.
- sessionCapabilities advertises {list:{}} but session/list throws
  "Method not found": stale capability advertising.
- loadSession works; replays user/assistant/thought text and
  session_info/usage/commands updates. No tool_call replay, as
  documented.
- cancel works end-to-end: stopReason=cancelled.
- closeSession/resumeSession are not on ClientSideConnection in
  0.19.1; kill child to close, use loadSession for rebind.

Plan revisions triggered by spike are recorded in
plans/browseros-ai/BrowserOS/features/2026-04-28-2310-claude-code-acp-implementation-roadmap.md.

* chore(acp): re-run smoke on SDK 0.21.0 and add mode/config/auth scenarios

After bypassing Bun's minimum-release-age and upgrading the SDK to
0.21.0, restore the previously-skipped resume/close paths and add
three new scenarios: mode (setSessionMode), config (setSessionConfigOption,
correct configId field), and auth (authenticate noop).

Findings, all bridge-side (independent of SDK):
- session/list, session/resume, session/close all throw -32601 on
  OpenClaw 2026.4.12 — capability advertising is stale.
- Image content blocks silently dropped; model never sees the bytes.
- setSessionMode and setSessionConfigOption work; latter requires
  `configId` (not `optionId`) per the schema.
- loadSession replays user/assistant/thought text + session_info +
  usage + available_commands; no tool_call replay (documented).
- authenticate is a noop on OpenClaw (no authMethods advertised).

Plan outcomes updated with full method-support matrix.

* chore(deps): promote @agentclientprotocol/sdk to a runtime dependency

The smoke script in apps/server/scripts/acp-smoke.ts used the SDK as
devDependency. The upcoming ACP bridge (apps/server/src/api/services/acp/)
needs it at runtime, not just for tooling. Move the entry from
devDependencies to dependencies, alphabetically first under @a*.

Pinned to 0.21.0 — same version the smoke script validated against.
README gains a small Dependencies note pointing at the future bridge
location.

No code changes yet. The bridge wiring lands in subsequent commits.

* fix(openclaw): wire LlmProvider.supportsImages through to OpenClaw model config

When BrowserOS sets up a custom OpenAI-compat provider on the gateway,
the agent UI's "Supports Image" flag (LlmProviderConfig.supportsImages)
was being dropped on the floor. As a result the persisted model entry
had no `input` field, OpenClaw defaulted it to ['text'], and image_url
content parts were silently stripped before the model saw them.

Fix:
- Extend OpenClawSetupInput / OpenClawAgentMutationInput on the agent
  side (useOpenClaw.ts) and the route body schema + SetupInput +
  createAgent input on the server side with `supportsImages?: boolean`.
- AgentsPage forwards `llmOption?.supportsImages` from the selected
  LlmProviderConfig in both handleSetup and handleCreate.
- provider-map.resolveSupportedOpenClawProvider emits
  `input: ['text', 'image']` on the model entry when the flag is
  truthy; otherwise emits the explicit `['text']` so the value is
  always pinned (avoids relying on OpenClaw's implicit default).
- applyBrowserosConfig adds `tools.media.image.enabled = true` to the
  bootstrap batch so the gateway's image-understanding pipeline is
  always wired up — per-model `input` still gates which models see
  images, this just enables the global path.

ACP image content blocks are still dropped by the OpenClaw bridge —
that's a separate bridge bug, not addressed here. This commit
restores image support for the OpenAI-compat /v1/chat/completions
path that the upcoming ACP chat panel will use as a carve-out for
image-bearing prompts.

Existing custom-provider configs are NOT auto-migrated; users will
re-acquire image support either by re-running setup or by editing
their model entries' `input` field manually. A migration pass for
legacy installs is not in scope for this commit because the
"supportsImages" intent isn't recoverable from the persisted config
alone — the source of truth is the LlmProvider record on the agent
side.

* feat(agents): add OpenClaw to AgentAdapter union and catalog

Extends AgentAdapter to 'claude' | 'codex' | 'openclaw' and adds the
OpenClaw entry to AGENT_ADAPTER_CATALOG. The new entry has:

- defaultModelId: 'default' — OpenClaw's ACP bridge does not surface
  per-session model selection (verified during the ACP spike), so
  models live in the OpenClawService config, not in the adapter
  catalog. AgentDefinition.modelId carries the gateway-side model
  name for display only.
- models: [] — empty list signals "no per-session model picker" in
  the UI; isSupportedAgentModel('openclaw', undefined|'default')
  returns true via the existing fallback path.
- reasoningEfforts mirror OpenClaw's session-level `thought_level`
  config option (off / minimal / low / medium / high / adaptive).

Also extends:
- isAgentAdapter type guard recognizes 'openclaw'
- HarnessAgentAdapter union on the extension side
- agents.test.ts createAgent fake type
- agent-catalog.test.ts asserts on the new entry, empty model list
  passthrough behavior, and OpenClaw's reasoning effort set

Lockfile delta is the workspace SDK pin reconciling 0.20.0 (taken
from dev's lock) up to our package.json's 0.21.0 (added in
c1d987ea). acpx still uses 0.20.0 transitively — both are present.

No runtime wiring yet — the registry override and AcpxRuntime
plumbing land in subsequent commits.

* feat(agents): plumb OpenClaw gateway accessors into AcpxRuntime

Adds an optional `openclawGateway` accessor to AcpxRuntime so the
upcoming registry override (Step 4) can spawn `openclaw acp` inside
the gateway container with the right port, token, and container/VM
identity. All accessors are getter-shaped so values stay live across
gateway restarts (port can change, token can rotate).

The accessor is threaded:
  server.ts → createAgentRoutes → AgentHarnessService → AcpxRuntime
                            ↘ sidepanel lazy AcpxRuntime

Also adds OpenClawService.getGatewayToken() returning the in-memory
token string. We pass it via OPENCLAW_GATEWAY_TOKEN env var on the
spawn (per OpenClaw's documented env-var precedence) instead of via
`--token` flag (which leaks to ps aux) or `--token-file` path (no
discrete token file lives inside the container — the token is nested
inside openclaw.json).

Wiring is dormant — the registry override that consumes these
accessors lands in Step 4. Typecheck + existing acpx/harness/routes
tests pass unchanged.

* refactor(agents): scrub local plan-step references from code comments

Replaces forward-looking comments that referenced internal plan
steps (e.g. "Step 4 wires this into…") with comments that justify
the code on its own merits. Plan files live locally on the
contributor's machine, so cross-references are noise to the rest of
the project.

No behavior change.

* feat(agents): spawn openclaw ACP adapter inside the gateway container

When the harness resolves the `openclaw` adapter, it now returns a
command that runs `openclaw acp` inside the bundled gateway container
via `limactl shell <vm> -- nerdctl exec -i ... openclaw acp --url
ws://127.0.0.1:<port>`. This reuses the openclaw binary already
installed alongside the gateway — no host-side openclaw install is
required.

Auth: the gateway token is injected via OPENCLAW_GATEWAY_TOKEN on
the container exec rather than `--token` on the openclaw CLI, so
the secret never appears in `ps aux`.

Banner output: OPENCLAW_HIDE_BANNER=1 and OPENCLAW_SUPPRESS_NOTES=1
keep stdout JSON-RPC-clean.

LIMA_HOME: prefixed via `env LIMA_HOME=<path>` on the resolved
command so the spawned limactl finds the BrowserOS-owned VM (the
server doesn't set LIMA_HOME on its own process env).

When the gateway accessor is absent, falls through to acpx's
built-in openclaw adapter which assumes a host-side install — that
branch will fail at spawn time with a descriptive error.

Verified end-to-end via the existing acp-smoke script during the
Step 0 spike.

* feat(agents): dual-create OpenClaw harness agents on the gateway

When the harness creates an `openclaw` adapter agent, it now also
provisions a matching agent on the OpenClaw gateway via the existing
CLI path (OpenClawService.createAgent). Symmetric on delete: gateway
removeAgent runs alongside the harness-store delete.

- Adds an OpenClawProvisioner interface (decoupled from OpenClawService
  for testability) and injects it through AgentHarnessService.
- createAgent rolls back the harness record if gateway provisioning
  fails; deleteAgent tolerates gateway-side failures so harness
  identity stays consistent with the user-facing UI.
- New OpenClawProvisionerUnavailableError surfaces as a 503 when an
  openclaw create request lands on a harness with no provisioner
  wired in (instead of a generic 500).
- FileAgentStore mints openclaw agent ids with an 'oc-' prefix so
  the id satisfies the gateway's `^[a-z][a-z0-9-]*$` agent name
  pattern. Other adapters keep raw UUIDs to preserve compatibility.
- POST /agents body schema accepts providerType / providerName /
  baseUrl / apiKey / supportsImages, forwarded to the provisioner
  when adapter='openclaw'.

The agents-page UI still routes openclaw create through the legacy
/claw/agents flow; switching that path to the harness is a separate
UI cutover.

Tests cover: gateway dual-create on success, rollback on gateway
failure, 503 when provisioner is missing, and tolerant delete on
gateway-side failure.

* fix(agents): skip catalog model validation for OpenClaw adapter

OpenClaw agents resolve their model from the gateway-side provider
config (set at agent-create time via OpenClawService) rather than
from the harness catalog, which has an empty `models: []` entry by
design. Without this carve-out, every OpenClaw create body fails
parsing with "Invalid modelId" because no concrete model id can
satisfy isSupportedAgentModel('openclaw', ...).

The reasoning-effort check still runs against the catalog (those
values map directly to OpenClaw's session `thought_level` config
option).

* fix(agents): pass --session to openclaw bridge so newSession routes correctly

acpx's AcpClient.createSession calls connection.newSession with cwd
and mcpServers but never forwards the sessionKey. Without it, the
openclaw bridge falls back to a synthetic acp:<uuid> session that
doesn't resolve to any provisioned gateway agent — every harness
chat returns a generic "Internal error" from -32603.

Fix: bake `--session <key>` into the resolved spawn command. The
bridge then uses that as the default session key for any newSession
the bridge receives, routing the turn to the matching gateway agent.

Per-session keying means each openclaw agent gets its own
AcpxCoreRuntime instance (cached by sessionKey on top of the
existing cwd/permissionMode key). This adds one extra runtime per
active openclaw session — claude/codex are unaffected.

Test asserts the resolved command includes the right --session arg.

* fix(agents): suppress BrowserOS MCP for openclaw bridge

The openclaw ACP bridge rejects newSession when mcpServers is non-empty
because its provider tooling comes from the gateway, not from ACP-side
MCP servers. Forwarding the BrowserOS HTTP MCP made every harness chat
fail with a JSON-RPC -32603 "Internal error" before the session was even
opened. Claude/codex still need the BrowserOS MCP for browser tooling,
so the carve-out is keyed off whether the runtime is for an openclaw
session.

* feat(agents): route OpenClaw chat through the harness behind a flag

Adds the `feature.useAcpxForOpenClaw` extension storage flag. When on,
OpenClaw agents in the agent-command chat panel use the harness
/agents/<id>/chat SSE and harness history hook instead of the legacy
/claw/agents/<id>/chat. When off, behavior is unchanged.

Also dedupes the agent rail when the same id appears in both stores
(dual-created agents from /claw/agents and /agents) by preferring the
harness entry — without this, every dual-created OpenClaw agent shows
up twice after Step 5.

Image attachments are temporarily disabled when the harness path is
active; the carve-out lands in the next commit.

* fix(agents): keep legacy OpenClaw agents on ClawChat

The previous commit's flag-gated branch routed every `source='openclaw'`
agent through `/agents/<id>/chat` when the flag was on, but the layout
dedup means the only agents that ever reach that branch are legacy
gateway-only entries (`main`, orphan agents from rolled-back creates) —
which by definition have no harness record, so the harness path 404s
and chat is unusable. Source is the only routing signal again: harness
agents go through the harness, legacy agents stay on ClawChat. The
storage flag stays for Step 9/10's migration story.

* feat(agents): expose OpenClaw in sidepanel and route through gateway main

`buildSidepanelChatTargets` now emits a single default ACP target for
adapters with no per-session model picker (OpenClaw, whose model is
configured on the gateway-side agent). Without this, OpenClaw never
appeared in the sidepanel target picker because the catalog entry has
`models: []`.

Sidepanel sessions don't have a dedicated provisioned gateway agent.
The openclaw bridge `--session` flag previously got the raw sidepanel
key (`sidepanel:<convId>:openclaw:...`), which doesn't match any
gateway agent — newSession was accepted but every prompt hung
forever. The bridge command now rewrites non-harness session keys
onto the always-present `main` gateway agent, encoding the original
key as a channel suffix to keep state segregated per conversation.
Verified end-to-end via curl: sidepanel openclaw chat streams
`text-delta` + `finish: stop`.

* feat(agents): backfill harness records for legacy gateway agents

Reframes Step 9 of the OpenClaw-on-acpx migration. The plan's literal
Step 9 (route OpenClaw history through the harness when the flag is on)
was already a no-op after the Step 6 walkback — history is routed by
source today. The actual blocker for Steps 10–13 was that legacy
gateway-only agents (e.g. `main`, orphans from rolled-back creates) had
no harness record, so they could never migrate to the harness path
without breaking chat.

`AgentHarnessService.reconcileWithGateway()` now lists every gateway
agent and upserts a matching harness record for any that are missing.
The pass runs lazily on first `listAgents()` call (memoized on success,
retried on failure so a gateway-down boot doesn't permanently disable
backfill). Verified end-to-end: the legacy `agent` agent now streams
`text_delta` + `done(end_turn)` through `/agents/agent/chat`, with the
bridge resolving to the gateway's `agent` record via the existing
`agent:<name>:main` session-key format.

After this, every OpenClaw agent surfaces as `source='agent-harness'`
post-dedup, the legacy `useClawChatHistory` hook becomes unreachable
for OpenClaw, and Steps 11–13 (delete legacy chat/history paths) are
unblocked.

* fix(agents): drop duplicate OpenClaw entry from NewAgentDialog adapter list

The adapter Select hardcoded an `<SelectItem value="openclaw">OpenClaw</SelectItem>`
on top of iterating `adapters`, which now includes OpenClaw post the
catalog change. The dropdown rendered "OpenClaw" twice — once at the
top, once at the bottom of the list. The literal was a pre-catalog
artifact; removing it leaves a single OpenClaw entry sourced from the
catalog. Routing into `handleOpenClawCreate` is unchanged because
the value (`'openclaw'`) is identical either way.

* fix(agents): always reconcile harness with gateway on list, just dedupe concurrent calls

Memoizing the first successful reconcile meant new gateway agents (created
via the legacy /claw/agents path or out-of-band CLI) never appeared in the
harness until server restart. The Promise now serves as a concurrent-call
dedupe only — cleared on settle — so every listAgents call picks up the
current gateway state. Reconcile is one cheap idempotent CLI call.

* chore(agents): remove dormant useAcpxForOpenClaw flag

The flag was scaffolded in Step 6 but its routing effect was walked
back the same day after it broke chat for legacy gateway-only agents.
After the Step 9 backfill, every OpenClaw agent has a harness record
and routes through the harness path purely from `source='agent-harness'`
— no flag is consulted anywhere. Remove the dead storage item, hook,
and stale comment.

* refactor(agents): drop legacy /claw/agents/:id/history endpoint

The harness /agents/:id/sessions/main/history endpoint replaced this
once every OpenClaw agent got a harness record (Step 9 backfill).
Routing is fully source-driven now, so the UI's useClawChatHistory
hook is never enabled today — verified live: legacy URL returns 404,
harness history hydrates correctly for the same agent.

Removes the GET /claw/agents/:id/history route, OpenClawService's
getAgentHistoryPage method plus its cursor/limit helpers and the
history-only types it owned (BrowserOSOpenClawHistoryPageResponse,
HistoryPageInput, normalizeHistoryLimit, encodeHistoryCursor,
decodeHistoryCursor, jsonlEventsToHistoryItems), and the route +
service tests that covered the dropped endpoint.

OpenClawJsonlReader stays alive — still feeds /claw/dashboard,
/claw/agents/:id/sessions, and the boot-time clawSession seed.
Removing those is its own follow-up since the dashboard would need
a harness-side replacement first.

* feat(agents): wire image attachments through the harness ACP path

Composer attachments now flow into the ACP `prompt` request as
spec-compliant `image` content blocks alongside the user's text. End
to end:

  composer → chatWithHarnessAgent({attachments}) →
  POST /agents/:id/chat {message, attachments} →
  parseChatBody decodes data: URLs to {mediaType, base64} →
  AgentHarnessService.send forwards →
  AcpxRuntime.send forwards →
  acpx startTurn({attachments}) → ACP image blocks

UI no longer disables the attach button on harness agents — the
gating was just a placeholder before this commit landed. Verified
end to end with a 1×1 red PNG against a Claude harness agent: model
replies "Red." correctly.

OpenClaw's `acp` bridge still drops image content blocks upstream
(verified by the same probe — Kimi-k2p5 reports "I don't see an
image"). That's an upstream openclaw limitation, not a harness-side
gap; Claude/Codex agents work as advertised today.

* chore(openclaw): delete OpenClawJsonlReader and JSONL-backed routes

* chore(openclaw): remove legacy /claw/agents/:id/chat and /queue routes

* chore(agents): collapse chat panel to harness-only path

* feat(agents): route OpenClaw image turns through the gateway HTTP client

The OpenClaw `acp` bridge silently drops ACP `image` content blocks
(verified during dogfood — model says "I don't see an image"). When
the user attaches images to an OpenClaw agent, the harness now diverts
that turn to the gateway's HTTP `/v1/chat/completions` endpoint, which
accepts OpenAI-style `image_url` parts and forwards them natively to
the provider.

  - New `OpenClawGatewayChatClient` translates an OpenAI streaming
    response into the same `AgentStreamEvent` shape the rest of the
    harness already consumes, so the chat panel renders identically
    whether a turn went through ACP or the gateway carve-out.
  - `AcpxRuntime.send` forks at the top: openclaw + any image
    attachment + a wired gateway client → `sendOpenclawViaGateway`.
    Other turns (text-only openclaw, claude, codex) take the existing
    ACP path unchanged.
  - The diverted path reads the prior turn history from the acpx
    session record so context is preserved, builds the OpenAI
    multimodal user message with text + image_url parts, and pumps
    the gateway SSE back to the caller through a tee that accumulates
    the assistant text. On natural completion, persists a synthetic
    user+assistant message pair to the acpx session record so reload
    shows the image turn in history.
  - Wired `OpenClawGatewayChatClient` into `AgentHarnessService` via
    `server.ts` (gateway port + token accessor, just like the existing
    `openclawGateway`).

Persistence note: the acpx record requires User messages to carry an
`id` and Agent messages to carry `tool_results` — without them the
record fails to round-trip through `parseSessionRecord`. The persist
helper now sets both.

Limitation by design: image recognition only works if the OpenClaw
agent's provider supports vision (e.g. Claude-via-OpenClaw, GPT-4o).
The pipeline routes images correctly to the provider regardless;
text-only providers like Kimi-k2p5 will reply "I don't see an image"
because the model itself has no vision capability — that's a provider
config issue, not a routing bug. The unit test asserts the image_url
part is present in the OpenAI request the gateway client sends.

The wider plan (background-resilient chat, queue, replay) remains in
`plans/.../2026-04-29-1527-...-background-resilient-chat-and-image-uploads.md`
as Phases 3–12; this commit ships only Phases 1–2.

* feat(agents): validate inbound image attachments on /agents/:id/chat

The harness chat body parser was accepting any mediaType and any
dataUrl length. The composer enforces these caps client-side but the
endpoint also serves direct curl/script callers, so the server has to
defend itself.

Restores the same caps the legacy /claw/agents/:id/chat parser had
before it was deleted in the migration:

  - 10 attachments per message
  - 5 MB raw image bytes (≈ 6.7 MB once base64-encoded plus prefix)
  - PNG / JPEG / WebP / GIF only
  - Must start with `data:`

Each violation returns 400 with a specific error message instead of
silently dropping or forwarding the payload.
2026-04-29 16:37:03 +05:30
Nikhil
2ff5c12840 feat: add sidepanel ACP chat targets (#857)
* feat(agent): add sidepanel chat target catalog

* feat(agent): show acp models in sidepanel selector

* feat(server): adapt acp events to ui message streams

* feat(server): add sidepanel acp chat route

* feat(agent): route sidepanel chat through acp targets

* chore: self-review fixes

* fix: address review feedback for PR #857
2026-04-28 18:23:38 -07:00
Nikhil
d87422eea1 fix: hide BrowserOS ACP wrapper in history (#856) 2026-04-28 17:31:11 -07:00
Nikhil
1946ca0cf8 chore: clean up unused agent sdk (#855) 2026-04-28 17:21:46 -07:00
Nikhil
754f7d0e1d test: cover terminal limactl resolver errors (#854) 2026-04-28 17:12:08 -07:00
Nikhil
85bb3f7b42 fix: avoid eager limactl resolution in server tests (#853) 2026-04-28 16:56:41 -07:00
Nikhil
cb32b8191d fix: show rich ACP harness history from ACPX (#852)
* fix: load ACP harness history from ACPX

* fix: address ACP history review comments
2026-04-28 16:40:22 -07:00
Nikhil
7a92654abc feat: add BrowserOS MCP to ACP agents (#851)
* feat: add BrowserOS MCP to ACP agents

* fix: bypass ACP agent permissions

* fix: address review feedback for PR #851
2026-04-28 16:30:20 -07:00
Nikhil
91d3285aa0 feat: add ACP agent harness (#849)
* feat: add acp agent runtime spike

* feat: add agent harness catalog

* feat: persist harness agents in json

* feat: persist agent transcripts

* feat: route harness service through agent records

* feat: expose generic agent harness routes

* feat: add harness agent frontend api

* feat: create harness agents from agents page

* feat: chat with persisted harness agents

* chore: remove obsolete agent profile spike

* chore: self-review fixes

* fix: combine openclaw and harness agents UI

* refactor: split agents page components

* fix: hide persisted harness turns
2026-04-28 15:29:38 -07:00
Nikhil
7bb6dac949 fix(dogfood): copy extension state into dev profile (#850)
* fix(dogfood): copy extension state into dev profile

* fix(dogfood): address profile import review feedback

* fix(dogfood): clarify refresh profile in-use error
2026-04-28 15:25:38 -07:00
shivammittal274
d9c254053e refactor(eval): drop unused agents/graders, collapse registries (#847)
* refactor(eval): drop unused agents/graders, collapse registries

Sweep of dead code in the eval app: deleted gemini-computer-use and
yutori-navigator agents, fara/webvoyager/mind2web graders, eight
debug/analyze/test scripts, three stale planning docs, and the orphaned
eval-targets/coordinate-click testbed.

With two agents and three graders left, the Map-backed plugin registries
were over-engineered — collapsed both into plain switches. Removed the
now-dead GraderOptions plumbing (no remaining grader takes API keys),
dropped grader_api_key_env/grader_base_url/grader_model from the schema
and configs, and de-duped PASS_FAIL_GRADER_ORDER (was defined in three
places). Replaced the URL-parsing extractCdpPort hack in single-agent
and orchestrator-executor with workerIndex passed cleanly through
AgentContext.

README and --help text rewritten to match reality. Renamed
configs/test_*.json to test-*.json for kebab-case consistency.

Net: ~10,460 LOC removed across 60 files. Typecheck clean, all tests
pass.

* ci(eval): pull BrowserOS from rolling stable CDN URL

The pinned v0.44.0.1 .deb on GitHub releases regressed on Linux —
servers start but never become healthy. Switch to the canonical rolling
URL at cdn.browseros.com/download/BrowserOS.deb so CI tracks the same
stable channel users get from the marketing site.
2026-04-29 02:14:47 +05:30
Nikhil
6b9945f933 feat(dev): use dev dock icon for browser launches (#848) 2026-04-28 13:28:19 -07:00
Dani Akash
6a5a7775a9 fix(openclaw): wire LlmProvider.supportsImages through to OpenClaw model config (#846)
When BrowserOS sets up a custom OpenAI-compat provider on the gateway,
the agent UI's "Supports Image" flag (LlmProviderConfig.supportsImages)
was being dropped on the floor. As a result the persisted model entry
had no `input` field, OpenClaw defaulted it to ['text'], and image_url
content parts were silently stripped before the model saw them.

Fix:
- Extend OpenClawSetupInput / OpenClawAgentMutationInput on the agent
  side (useOpenClaw.ts) and the route body schema + SetupInput +
  createAgent input on the server side with `supportsImages?: boolean`.
- AgentsPage forwards `llmOption?.supportsImages` from the selected
  LlmProviderConfig in both handleSetup and handleCreate.
- provider-map.resolveSupportedOpenClawProvider emits
  `input: ['text', 'image']` on the model entry when the flag is
  truthy; otherwise emits the explicit `['text']` so the value is
  always pinned (avoids relying on OpenClaw's implicit default).
- applyBrowserosConfig adds `tools.media.image.enabled = true` to the
  bootstrap batch so the gateway's image-understanding pipeline is
  always wired up — per-model `input` still gates which models see
  images, this just enables the global path.

ACP image content blocks are still dropped by the OpenClaw bridge —
that's a separate bridge bug, not addressed here. This commit
restores image support for the OpenAI-compat /v1/chat/completions
path that the upcoming ACP chat panel will use as a carve-out for
image-bearing prompts.

Existing custom-provider configs are NOT auto-migrated; users will
re-acquire image support either by re-running setup or by editing
their model entries' `input` field manually. A migration pass for
legacy installs is not in scope for this commit because the
"supportsImages" intent isn't recoverable from the persisted config
alone — the source of truth is the LlmProvider record on the agent
side.
2026-04-29 00:23:45 +05:30
shivammittal274
af48a2110c feat(eval): Phase 1 — exclude broken tasks, freshen card dates, add grader leniency (#841)
* fix(eval): exclude broken tasks + freshen expired card dates

Two AGISDK tasks are unsolvable today for non-model reasons:

- topwork-1: evals-topwork.vercel.app throws Minified React error #185
  ("Maximum update depth exceeded") on every form submit. The page renders
  "Application error: a client-side exception has occurred" instead of saving.
  Whole-task failure, every model affected.

- fly-unified-2: hardcodes Exp: 12/25 in both the goal text AND a jmespath
  grader criterion. Today is 2026-04, so the eval-site rejects the card.
  Freshening the goal alone leaves the grader expecting the original value;
  freshening both would require monkey-patching agisdk's TaskConfig at
  runtime — too fragile to maintain.

Adds these to a new EXCLUDED_TASKS set alongside the existing
EXCLUDED_WEBSITES (omnizon).

Also adds freshen_goal_dates(): for AGISDK fly-unified tasks whose goal
contains an `Exp: MM/YY` within 6 months of today (or past), rewrites it
to a far-future date (12/30). This rescues fly-unified-5 (had Exp 12/25,
no card-exp grader criterion) and protects fly-unified-4 (had Exp 06/26,
2 months from expiring) from the next eval run hitting the same trap.

Dataset goes from 47 -> 45 tasks; 2 freshened.

* feat(eval): add lenient-strings grader softening

The agisdk grader compares jmespath-extracted values via strict equality.
For tasks where the model adds harmless decoration to a free-text field
(e.g. topwork-3 expects title "Full-Stack Developer" but model produces
"Full-Stack Developer - Enterprise Microservices Platform"), this fails
every other criterion would pass.

Adds a substring fallback in the wrapper: a failed criterion is re-marked
as a softened pass when both actual_value and expected_value are strings
and the (stripped, lower-cased) expected_value is contained in the
actual_value. Numbers/bools/dates/None stay strict.

- Default-on. Set AGISDK_STRICT_STRINGS=1 to recover the strict score.
- Softened criteria are tagged with `softened: true` in per_criterion
  output for transparency in run manifests.
- Aggregate `pass`/`reward` are recomputed after softening.

Expected to rescue 4 tasks in our 45-set: topwork-3, topwork-4 (both pure
title-decoration), gomail-8 (grader contradicts goal), and networkin-6
(grader hardcodes profile id).

* fix(eval): exclude 5 more tasks where pipeline (not agent) fails

Extends EXCLUDED_TASKS to 7 entries based on the K2.5 + Opus 4.6
head-to-head deep-dive on the 2026-04-28 runs. The exclusion rule:
remove a task only if it is unsolvable for any agent — either the task
data is invalid, the eval site is broken, or the grader penalizes
correct work. Tasks that fail because of our agent's tool fidelity
(drag, custom-widget fill, click on React submit, etc.) STAY in — those
are real capability gaps the team should see in the score.

New exclusions:

- fly-unified-9: goal references "Dec 18 2024 at 10:00" but the live
  eval site has only 2025 inventory and no 10:00 slot. Both models
  successfully booked the closest available flight and were penalized
  on a grader expectation that can never be met.

- fly-unified-4: eval site stores wall-clock flight times as bare UTC
  (T08:00:00.000Z) while the grader expects them shifted by 8h
  (T16:00:00.000Z = 8 AM PST). Opus 4.6 completed the entire booking
  correctly. Eval-site TZ-storage bug.

- gomail-8: goal says "Clear all emails from GitHub in the inbox", but
  criterion 3 expects exactly 1 email updated. Both K2.5 and Opus
  correctly cleared all 4 GitHub emails. Grader contradicts goal.

- networkin-6: goal says "Choose a random person you haven't connected
  with"; grader hardcodes profilesDiff.updated."4".connectionGrade.
  Both models randomized correctly and missed id 4. Grader contradicts
  goal.

- networkin-9: eval site's searchHistoryDiff doesn't record queries
  submitted via the autocomplete + Enter path. Opus 4.6 completed the
  task end-to-end (Stanford alum, connection request, message); only
  failed because the search-history criterion was never written
  server-side. Eval-site bug.

Dataset goes from 45 -> 40 tasks. Score impact (same K2.5/Opus runs,
recomputed against the cleaned 40-task denominator):

  K2.5:     21/45 (46.7%) -> 21/40 (52.5%)
  Opus 4.6: 28/45 (62.2%) -> 28/40 (70.0%)
  Δ:        15.6 pp -> 17.5 pp (real model gap, less pipeline noise)
2026-04-28 23:19:31 +05:30
Nikhil
c5ff8d75bc fix(dogfood): clarify init prompts (#839) 2026-04-28 07:48:42 -07:00
Nikhil
445a6a6c45 fix(dogfood): use alpha dock icon (#837) 2026-04-27 21:47:10 -07:00
Nikhil
72d39b9a0f docs(dogfood): simplify alpha workflow readme (#838) 2026-04-27 21:44:03 -07:00
Nikhil
3b47f330f5 fix(dogfood): separate BrowserOS state root (#836) 2026-04-27 17:38:15 -07:00
Nikhil
15a82ff9cb feat: add dogfood background daemon mode (#833) 2026-04-27 17:15:50 -07:00
Nikhil
427549f081 feat: Add BrowserOS Dock icon variants (#835) 2026-04-27 17:10:36 -07:00
Nikhil
a11f9caa64 fix(dogfood): colorize cli output (#834)
* fix(dogfood): colorize cli output

* fix: address dogfood cli review comments
2026-04-27 16:29:25 -07:00
Nikhil
da1397900b refactor: rename internal BrowserOS CLIs (#832)
* refactor: rename internal BrowserOS CLIs

* fix: update dogfood binary gitignore
2026-04-27 16:18:45 -07:00
Nikhil
368c7dcfe8 fix(alpha): write balpha process logs (#830)
* fix(alpha): write balpha process logs

* fix(alpha): address log review feedback
2026-04-27 15:48:40 -07:00
Nikhil
599f8b6b9c fix: address balpha CLI dogfooding feedback (#831) 2026-04-27 15:43:22 -07:00
Nikhil
27834b1d31 fix: udpate readme (#829) 2026-04-27 15:27:16 -07:00
Nikhil
aa30eb3aaa feat: add balpha dogfooding CLI (#828)
* feat(alpha): scaffold balpha cli

* fix(alpha): address scaffold review

* feat(alpha): add balpha config

* feat(alpha): parse browseros profiles

* feat(alpha): import browseros profile

* feat(alpha): add browser launch helpers

* feat(alpha): add repo build and env pipeline

* feat(alpha): add process supervision

* feat(alpha): add balpha commands

* docs(alpha): document balpha setup

* fix(alpha): reuse dev setup script

* fix(alpha): address review feedback

* fix(alpha): normalize imported browser profile

* fix(alpha): use generic profile fixture names
2026-04-27 15:03:37 -07:00
shivammittal274
e045e34b73 fix(eval): switch weekly eval configs from Fireworks to OpenRouter (#827)
The 2026-04-23 weekly run had 42% of AGISDK and 46% of Infinity tasks
fail with `AI_RetryError: ... the service is overloaded` from Fireworks
(20 concurrent kimi-k2p5 streams across both runs at 10 workers each).

Switching to OpenRouter (which fronts the same Moonshot K2.5 weights
and falls back across providers) for the three weekly configs:
- browseros-agent-weekly.json
- agisdk-real-smoke.json
- infinity-hard-50.json

Model accounts/fireworks/models/kimi-k2p5 -> moonshotai/kimi-k2.5
(same weights, same 262K context). API key env var, base URL updated.

OPENROUTER_API_KEY is already wired into .github/workflows/eval-weekly.yml
and present in repo secrets — no GH config changes needed.

Orchestrator-executor configs and test_webvoyager left on Fireworks
intentionally; can switch later if needed.
2026-04-27 21:52:26 +05:30
shivammittal274
01d649da9a feat(eval): bring deterministic graders to dev + drop omnizon (#824)
* feat: deterministic eval graders (AGI SDK + WebArena-Infinity) (#664)

* feat: add deterministic eval graders (AGI SDK + WebArena-Infinity)

Two new benchmark integrations with programmatic grading — no LLM judge.

AGI SDK / REAL Bench (52 tasks):
- 11 React/Next.js clones of consumer apps (DoorDash, Amazon, Gmail, etc.)
- Grader navigates browser to /finish, extracts state diff from <pre> tag
- Python verifier checks exact values via jmespath queries

WebArena-Infinity (50 hard tasks):
- 13 LLM-generated SaaS clones (Gmail, GitLab, Linear, Figma, etc.)
- InfinityAppManager starts fresh app server per task per worker
- Python verifier calls /api/state and asserts on JSON state

Infrastructure:
- GraderInput extended with mcpUrl + infinityAppUrl for parallel workers
- Each worker gets isolated ports (no cross-worker state contamination)
- CI workflow: pip install agisdk, clone webarena-infinity repo

* chore: switch eval configs back to kimi-k2p5

* fix: register deterministic graders in pass rate calculation

Add agisdk_state_diff and infinity_state to PASS_FAIL_GRADER_ORDER
in both runner types and weekly report script, so scores show correctly
in the dashboard.

* chore: temp switch to opus 4.6 for eval run

* chore: restore kimi-k2p5 as default eval config

* ci: add timeout and continue-on-error for trend report step

* fix(eval): drop omnizon from AGISDK dataset (DMCA takedown)

evals-omnizon.vercel.app returns HTTP 451 ("This content has been
blocked for legal reasons / DMCA_TAKEDOWN"). All 5 omnizon-* tasks
fail grading with "Failed to fetch /finish endpoint: JSON Parse error".

Adds an EXCLUDED_WEBSITES set to the dataset builder and regenerates
agisdk-real.jsonl (52 → 47 tasks).

* fix(eval): correct Infinity port-assignment bugs

Two related bugs in the Infinity eval runner that cause silent port
collisions / fallbacks under parallel execution:

1. build-infinity-dataset.py emitted "app_port" but task-executor and
   the committed JSONL both read "app_base_port". Re-running the build
   script would silently make every task fall back to the 8000 default,
   ignoring per-app port assignments. Renamed the key to match.

2. task-executor derived workerIndex as `base_server_port - 9110`, but
   parallel-executor doesn't override base_server_port per worker —
   only server_url. Every worker computed workerIndex = 0, causing all
   parallel workers to spawn Infinity app servers on the same port.
   Threading workerIndex explicitly through TaskExecutor instead.

Also drops an unused app_name parameter from load_tasks().
2026-04-27 21:35:43 +05:30
Dani Akash
ddbb2cf492 feat(agent): composer attachments + server-side outbound message queue (#826)
* feat(agent): attach images and text files to chat messages

Adds end-to-end support for image and text file attachments in the chat
composer, with the staged files round-tripping through the OpenClaw
gateway as OpenAI-compatible content blocks and persisting in the JSONL
so they show up in the historical view.

Server
- HTTP client: new OpenClawChatContentPart union and a buildUserContent
  helper that emits multimodal content arrays when messageParts is
  supplied, falls back to the legacy string content otherwise.
- Service: chatStream takes an optional messageParts array and forwards
  it; BrowserOSChatHistoryItem gains an attachments field.
- JSONL reader: PiContentBlock learns the OpenAI image_url and Anthropic
  image source/data shapes; user messages now emit user.attachment
  events that the history mapper accumulates onto the next user item.
- Route: validates an inbound attachments[] (kind/mime/size/count),
  inlines text-shaped files as <attachment> blocks in the message body,
  attaches images via image_url parts. Replaces the immediate 409 on
  active monitoring session with a 30s waitForSessionFree(agentId) wait
  (registry now exposes onSessionEnd) so cron/hook contention does not
  reject a user-chat send outright. Returns 503 if the wait times out.

Client
- New lib/attachments.ts: validateAttachment / compressImageIfNeeded
  (canvas downscale to 2048px long edge, JPEG 0.85 re-encode for >1.5
  MB inputs) / stageAttachment / stageAttachments that produces the
  staged-attachment shape the composer renders and the payload the
  server accepts.
- ConversationInput: drag-and-drop, paperclip button, clipboard paste,
  staged attachment chip strip with thumbnails for images and a
  paperclip+name chip for text files. Send button enables on either
  text or attachments. Drop-zone overlay during drag.
- chatWithAgent forwards attachments[]; useAgentConversation.send
  accepts a SendInput shape and renders user attachments on the
  optimistic streaming turn via MessageAttachments / MessageAttachment.
- ClawChatMessage groups historical attachment parts into a single
  MessageAttachments strip, ordered before reasoning/tools/text.
- claw-chat-types adds an attachment ClawChatMessagePart variant; the
  history mapper emits attachment parts first and skips the text part
  when the user only sent media.
- AgentCommandHome forwards the new SendInput shape — home composer
  drops attachments at the boundary in v1 (the conversation page is
  where staging is most useful; carrying bytes through the URL bar
  is not sensible).

Limits: 10 attachments per message, 5 MB per image (post compression),
1 MB per text file, mime types png/jpeg/webp/gif and text/* +
application/json. PDFs and other binaries are deferred to v2.

* feat(agent): outbound message queue for chats while agent is mid-turn

Lets users keep typing and submitting messages while the agent is still
streaming a previous turn. Each press is appended to a single-flight
queue and dispatched as soon as `streaming` flips false; the queued
state renders as a strip above the composer so the user sees what's
pending vs. what's already sending.

- New `useOutboundQueue` hook owns the queue, the worker effect, and
  cancel/retry actions. Single-flight by design — a re-entrancy ref
  guard prevents two simultaneous dispatches when `streaming` flickers.
- Composer (`ConversationInput`) accepts optional `outboundQueue`,
  `onCancelQueued`, `onRetryQueued` props. When the queue is provided
  the send-button gate stops blocking on `streaming`; the spinner stays
  as the visual cue that the agent is still busy. Legacy direct-send
  callers keep the old streaming-blocks-send semantic.
- Renders an OutboundQueueStrip above the staged-attachment strip with
  per-item status (queued / sending / failed), a cancel button on
  queued items, and retry + discard on failed items.
- AgentCommandConversation wires `onSend` to `queue.enqueue` and routes
  the home composer's `?q=` initial-message handoff through the queue
  too, so it inherits the same single-flight serialization.

The server-side `waitForSessionFree` (added with attachments) and this
client-side queue together cover both contention sources: cron / hook
turns and back-to-back user sends. Persistence across reloads is
intentionally out of scope for v1 — losing the queue on extension
reload is documented as a known limitation.

* feat(server): server-side outbound message queue

Replaces the client-only React-state queue from 123ef21d with a
proper server-owned queue. Closing the tab is now safe — the server
holds queued messages and dispatches them through the existing
chatStream path the moment the agent's ClawSession status flips to
idle.

Server
- New OutboundQueueService (apps/server/src/api/services/queue) — per
  agent FIFO, in-memory. Subscribes to ClawSession.onStateChange
  through OpenClawService.onAgentStatusChange, and dispatches via
  OpenClawService.chatStream so attachments / history / monitoring
  all behave identically to the existing /chat route. The worker
  drains the SSE response server-side so the gateway run finalizes
  cleanly even with no client connected.
- Four new routes under /claw/agents/:id/queue:
  POST   /queue            enqueue
  DELETE /queue/:itemId    cancel a queued item
  POST   /queue/:itemId/retry  re-queue a failed item
  GET    /queue/stream     SSE feed of the per-agent queue state.
  Validation reuses validateChatAttachments and
  buildMessagePartsFromAttachments from the existing chat route.
- Singleton wired in apps/server/src/main.ts; shutdown on SIGTERM.
- New OpenClawService.getAgentState getter for the queue worker's
  pre-dispatch sanity check.

Client
- useOutboundQueue rewritten as an SSE-backed projection over server
  state. Public API unchanged so the composer still works.
- enqueue POSTs to /queue and shows an optimistic local entry until
  the server's SSE snapshot reflects it; local-only entries get a
  `local-` id prefix so cancel can short-circuit them without
  hitting the server.
- AgentCommandConversation watches the queue for sending items
  dropping out and refetches history so the new assistant turn shows
  up in the conversation view (the server worker streams the
  dispatched turn into OpenClaw without exposing per-turn SSE to
  the client).

Out of scope (documented in the plan as v2 follow-ups): disk
persistence (server restart loses queue), per-turn live streaming
of queued sends in the conversation view, and switching the
underlying dispatch from /v1/chat/completions to the chat.send RPC
(which would also fix the multimodal attachment routing problem).

* fix(server): outbound queue must reuse existing session, not spawn UUIDs

The queue worker was generating a fresh randomUUID() as the sessionKey
when the queued item didn't carry one — and the client wasn't sending
one. Result: every queued message kicked off a brand-new OpenClaw
session, orphaning the user's active conversation behind the new
"most recent" entry in sessions.json. The history endpoint then
resolved to the orphan and the chat appeared to disappear.

Fix is layered:
- Client (useOutboundQueue): forward the current resolvedSessionKey
  in the POST /queue body so every queued message targets the same
  conversation the user is viewing. AgentCommandConversation passes
  resolvedSessionKey into the hook.
- Server (OutboundQueueService): the worker now resolves to the
  agent's existing user-chat session when no sessionKey is provided
  on the queued item, via OpenClawService.resolveAgentSession. UUID
  fallback is now reserved for the first-ever message on a brand
  new agent — same semantic the existing /chat route has implicitly
  through the catalog of historical sessions.

No JSONL data was lost by the original bug (the prior conversations
are intact on disk); the orphan sessions just shadowed the original
in sessions.json.

* fix(agent,server): address PR review feedback for chat queue

- Tighten image data URL cap to base64-aware ~6.7 MB (was ~7.5 MB
  through `MAX_IMAGE_BYTES * 2`).
- Forward chat history from useOutboundQueue.enqueue so queued sends
  preserve conversation context like direct sends do.
- Match local attachment previews to server snapshots by id (not by
  message text), and prune the preview map as items drain.
- Pass an AbortSignal into chatStream so a queue shutdown cancels the
  initial OpenClaw handshake, not just the SSE drain loop.
- Track previously gitignored apps/agent/lib/attachments.ts (was caught
  by global lib/ ignore) so CI typecheck can resolve @/lib/attachments.
- Update server-api openclaw route tests to the new chatStream signature
  and the waitForSessionFree-based busy-agent path.

* fix(agent): dedupe optimistic queue entries for text-only sends

The localId↔serverId map was only populated when the message had
attachments, so plain-text sends left the optimistic local entry in
place after the server snapshot arrived — the user saw the same
message rendered twice in the queue strip.

* fix(agent): prune optimistic queue entry on POST ack, not just SSE

The server broadcasts the new queue snapshot before its POST response
returns, so the SSE handler often runs first — at that point the
localId↔serverId map has no entry for the new server id yet, so the
SSE-based dedupe path can't drop the optimistic local entry. Pruning
on POST success closes the race deterministically.

* fix(agent): hand off optimistic queue entry without a render gap

Pruning the local entry on POST success only worked when the SSE
snapshot had already overwritten it; if the POST response landed
first, the optimistic row disappeared for a frame before the SSE
snapshot brought back the server-keyed row, producing a visible
flicker. Gate the POST-side prune on the SSE snapshot already
carrying the server id, and rely on the SSE-based dedupe (now
guaranteed to find the localId↔serverId link in the map) to clean
up when SSE arrives later.

* fix(agent,server): client-generated queue id eliminates render flicker

The server used to assign its own UUID when an item was enqueued, so
the optimistic client row carried a `local-` id while the SSE snapshot
carried a server UUID — the client had to wait for the POST response
to learn the mapping before it could dedupe, and during that window
both rows rendered.

Now the browser generates the id, sends it in the POST body, and the
server uses it verbatim (falling back to a fresh UUID only if the id
collides with an existing item). The client collapses to a single
id-keyed list, so the optimistic row and the SSE row reconcile on the
same key from the very first render.
2026-04-27 21:31:03 +05:30
Dani Akash
711934555d feat(agent): enrich chat UI with tool activity, reasoning duration, and cost (#825)
* feat: pass per-turn cost and token data through chat history items

- Add costUsd, tokensIn, tokensOut to BrowserOSChatHistoryItem (server)
- Pass through from JSONL agent.message events in jsonlEventsToHistoryItems()
- Add same fields to client-side BrowserOSChatHistoryItem and ClawChatMessage
- Map cost/token data in mapHistoryItemToClawMessage()

Data flows: JSONL message.usage → server history item → API response →
client ClawChatMessage. Available for rendering in ClawChatMessage
component (message toolbar, cost badges).

* feat: add message toolbar with copy button and per-turn cost display

Add MessageToolbar to historical assistant messages in ClawChatMessage:
- Copy button copies message text to clipboard via MessageAction
- Per-turn token count (22.7K → 238) and cost ($0.003) shown as muted
  tabular-nums text on the right side of the toolbar
- Toolbar appears on hover (opacity transition via group-hover)
- Only shown when the message has text content
- Cost/token display only shown when data is available from JSONL

* fix: toolbar only on assistant messages, always visible, cost only

- Only render toolbar on assistant messages (not user messages)
- Remove hover-only opacity — toolbar is always visible
- Remove token counts (22.7K → 238 is meaningless to users)
- Show only cost as a budget signal ($0.003)

* feat: group all tool activity into single Task collapsible per turn

Replace flat tool rows with a single ai-elements Task collapsible per
assistant turn that lists every tool/MCP call in sequence.

Live streaming (ConversationMessage):
- Aggregate all tool-batch parts into one Task
- Title: "Working… (N actions)" while running, "Agent activity (N actions)" when done
- Default open while turn is in progress
- Wrench icon in trigger

Historical (ClawChatMessage):
- Group all tool-call parts into one Task
- Title includes failed count if any tools errored
- Default collapsed — expandable on click
- Tool name + status icon + error text per row

Both views show one clean collapsible per turn instead of N individual
tool cards. Collapsed reads "5 actions"; expanded shows the timeline.

* feat: include tool calls in chat history responses

Server: jsonlEventsToHistoryItems() now walks ALL events (not just
messages) and pairs agent.tool_use with agent.tool_result by toolCallId.
The resulting tool call list is attached to the next assistant text
message as toolCalls[]. Each entry includes status, input arguments,
output text, error string, and duration computed from event timestamps.

Client:
- BrowserOSChatHistoryItem gets optional toolCalls field
- Tool-call message part type gets durationMs field
- mapHistoryItemToClawMessage() emits tool-call parts BEFORE the text
  part (the order the agent produced them)
- ClawChatMessage Task view now shows tool duration in seconds

Result: historical messages now display the full tool activity
timeline grouped into the single Task collapsible per turn (designed
in step 3), instead of showing only the final text response.

* feat: render activity rows as human verbs sourced from tool registry

Tool calls in the chat activity view now read as sentences:
"Opened tab · news.ycombinator.com" instead of "browseros__new_page".

Server (tool-label-registry.ts):
- Curated verb override map for ~70 BrowserOS first-party tools
- Per-tool subject extractors that pull the meaningful argument from
  input (URL → host, query → quoted, element → ID, etc.)
- Generic fallback humanizes snake_case for any unmapped tool
- Strips MCP namespace prefixes (browseros__, mcp_)

Server (openclaw-service.ts):
- jsonlEventsToHistoryItems calls buildToolLabel for each tool_use,
  attaches label and subject to the BrowserOSChatHistoryToolCall

Client:
- Mirrored label module at lib/tool-labels.ts
- useAgentConversation tool-start handler computes label/subject
  from the SSE tool args
- ClawChatMessage and ConversationMessage render label · subject
  with foreground/muted styling, no font-mono
- ToolEntry, BrowserOSChatHistoryToolCall, and tool-call message
  part types all carry label and optional subject

* fix: drop meaningless tab N subject from page-read tool rows

Page IDs are internal numbers, not URLs. 'Took screenshot · tab 4'
tells the user nothing. Removed subject extractors for take_snapshot,
take_enhanced_snapshot, get_page_content, get_page_links, get_dom,
and take_screenshot. The verb alone is the right signal.

* fix: gate initial loading on historyQuery.isFetched not isLoading

The session and history queries are sequential: the history query is
disabled until session resolves. After session resolves, there's a render
frame where historyQuery.isLoading is still false (the query hasn't
been kicked off yet). isInitialLoading flipped to false during that
window, exposing an empty chat shell with just Task collapsibles and
copy buttons before the messages filled in.

Switching the guard to isFetched closes that window — the loading state
stays true until the first history fetch actually completes.

* fix: render historical messages immediately instead of through Streamdown's idle-callback debounce

Streamdown defaults to mode="streaming" which uses requestIdleCallback (300ms
debounce, 500ms idle timeout) and lazy/Suspense to optimize for token-by-token
live streams. For finalized historical messages this caused tool collapsibles
and copy buttons to paint while text bodies stayed blank for ~300-500ms after
load. Pass mode="static" + parseIncompleteMarkdown=false on the historical
MessageResponse so completed text paints in the same frame as the surrounding
chrome. Live streaming turns still use the default streaming mode.

Also collapse the redundant /agents/:id/session round-trip into the existing
/history endpoint (server already resolves the most recent user-chat session
when sessionKey is omitted) and tighten the initial-loading gate to stay true
across the render frame where the query is enabled but hasn't started fetching.

* feat: surface thinking duration on historical reasoning collapsibles

Server accumulates agent.thinking events per turn from JSONL and attaches a
single reasoning block (joined text + durationMs from first thinking event
to the closing agent.message) on each assistant history item. Reasoning
buffer resets on user.message alongside the tool-call buffer.

Client mirrors the type, emits the reasoning part before tool calls in
mapHistoryItemToClawMessage (chronological: think → act → answer), and
passes duration in seconds to <Reasoning> so the trigger reads "Thought
for N seconds" instead of just "Thinking" on collapsed historical turns.

* fix: read thinking blocks from the correct JSONL field name

OpenClaw stores reasoning blocks as {type:'thinking', thinking:'...'} but
the JSONL parser was reading block.text, so every thinking event was
silently dropped before it ever reached jsonlEventsToHistoryItems. As a
result the reasoning field on history items was always empty even though
the new accumulator was wired up correctly.

Also guard the client mapping: when durationMs is 0 (think + answer
emitted in the same JSONL line, no real elapsed wall-clock) pass
undefined to <Reasoning> so it renders the static "Thinking" trigger
instead of the streaming shimmer / "Thought for 0 seconds".

* fix: reset reasoning buffer on discarded turns and drop dead session hook

Two cleanups from PR review:

1. jsonlEventsToHistoryItems: when an agent.message is discarded (the
   "[Chat messages since your last reply" wrapper without a current-message
   marker) the tool buffers were already reset but the reasoning buffer
   was not. Accumulated thinking from the discarded turn would bleed onto
   the next assistant message. Reset pendingReasoningTexts and
   pendingReasoningFirstAt alongside the tool buffers.

2. useClawAgentSession, the AgentSessionResponse type, and the unused
   session entry in CLAW_CHAT_QUERY_KEYS became dead code after the
   session round-trip was folded into the history endpoint. Removed.
2026-04-27 18:29:15 +05:30
Nikhil
5125dffbf3 fix: sign limactl with VZ entitlement (#822) 2026-04-26 13:30:09 -07:00
Dani Akash
0035893f33 feat: dashboard API, JSONL reader, and OpenClaw observer for enriched home page (#810)
* feat: draft agent chat ui exploration

* feat: refine agent chat ui draft

* feat: remove outer frame from agent chat workspace

* fix: offset agent chat for app sidebar

* fix: simplify agent conversation shell

* fix: remove redundant chat header actions

* fix: unify agent conversation headers

* fix: tighten agent chat spacing

* fix: bound agent chat composer height

* fix: remove agent chat page inset

* fix: align agent header height with sidepanel

* fix: center agent composer resting state

* fix: anchor multiline composer controls

* fix: remove focus grid from agent home

* fix: remove redundant agent home header

* fix: constrain home agent composer

* fix: match home composer default posture

* feat: add openclaw chat history APIs

* feat: add claw chat history hydration

* fix: stabilize claw chat viewport layout

* fix: use conversation scroll base for claw chat

* refactor: split claw chat controller responsibilities

* fix: keep active agent turns in memory

* fix: normalize openclaw chat sessions

* refactor: use HTTP client for agent history instead of CLI client

Replace the CLI-based getChatHistory() call in getAgentHistoryPage()
with the HTTP client's getSessionHistory() from PR #795. This uses
the direct HTTP transport to OpenClaw's /sessions/<key>/history
endpoint instead of shelling out through the CLI.

- Add filterHttpSessionHistoryMessages() for flat-string content format
- Add normalizeHttpHistoryMessages() for OpenClawSessionHistoryMessage shape
- Update getAgentHistoryPage() to call getSessionHistory() via httpClient
- Remove unused getChatHistory(), filterOpenClawSystemMessages(),
  normalizeChatHistoryMessages(), and getTextContent()
- Update test mocks from cliClient.getChatHistory to httpClient.getSessionHistory
- Update MutableOpenClawService type: chatClient -> httpClient

* fix: fetch all session messages by iterating OpenClaw pagination

OpenClaw's HTTP history endpoint returns a limited page by default.
When called without a limit, only the first ~27 messages were returned,
causing all newer conversation messages to be silently dropped.

Add fetchAllSessionMessages() that iterates through OpenClaw's cursor-
based pagination (200 messages per page) until hasMore is false, then
feeds the complete message list into the existing BrowserOS normalization
and in-memory pagination layer.

* refactor: migrate chat history from HTTP gateway to direct JSONL file reads

Replace the HTTP-based chat history pipeline (BrowserOS server → OpenClaw
gateway /sessions/:key/history pagination loop) with direct JSONL file reads
from the host filesystem via Lima's virtiofs mount.

- Add OpenClawJsonlReader that reads session JSONL files directly from
  ~/.browseros/vm/openclaw/.openclaw/agents/<id>/sessions/
- Replace fetchAllSessionMessages() HTTP pagination with single file read
- Replace CLI-based listSessions() with sessions.json file reads
- Make listSessions, resolveAgentSession, getAgentHistoryPage synchronous
- Remove unused toBrowserOSSession, filterHttpSessionHistoryMessages,
  normalizeHttpHistoryMessages helpers
- Update route handlers to drop unnecessary async/await
- Update tests to use temp JSONL files instead of mocked HTTP/CLI clients

* fix: restore async route handlers for test compatibility with mocked service

* fix: address review feedback — path traversal guard, lazy reader, exists flag

- Add safePath() to OpenClawJsonlReader that validates resolved paths stay
  within stateRoot, preventing path traversal via crafted agentId values
- Use lazy initialization for jsonlReader (nulled on rebuildRuntimeClients)
  instead of creating a new instance per property access
- Return exists: false from resolveSpecificAgentSession when no session
  matches instead of fabricating a ghost session with sessionId: ''

* feat: add dashboard API and enrich home page agent cards

Server:
- Add summarizeToolActivity() that converts tool events into natural
  language descriptions ("Browsed 3 pages, took 2 screenshots")
- Add getDashboard() to OpenClawService that aggregates per-agent stats
  from JSONL: latest message, activity summary, cost, session count
- Add GET /claw/dashboard endpoint

Client:
- Add useAgentDashboard() React Query hook (10s refetch, 5s stale)
- Rewrite useAgentCardData from async IndexedDB hook to pure
  buildAgentCardData() function merging agent entries with dashboard data
- Add activity summary and cost to AgentCardExpanded footer
- Add activitySummary and costUsd fields to AgentCardData type
- Remove IndexedDB dependency from the home page

* feat: add OpenClawObserver for real-time per-agent status via gateway WS

- Add OpenClawObserver that connects to the OpenClaw gateway WebSocket
  control plane and subscribes to chat broadcast events
- Track per-agent status in real time: working (streaming), idle (turn
  complete), error (run failed), with current tool name
- Auto-connect when gateway control plane becomes available, auto-
  reconnect on disconnect with 5s backoff
- Disconnect observer on stop/shutdown
- Wire live status + currentTool into getDashboard() response
- Update client: AgentOverview includes status + currentTool, card shows
  spinning loader + tool name when agent is working
- Status resolution: per-agent WS status takes precedence over gateway-
  level status for working/error states

* feat: add SSE dashboard stream for real-time agent status on home page

Server:
- Add GET /claw/dashboard/stream SSE endpoint that sends an initial
  snapshot then pushes per-agent status events as they arrive from
  the OpenClaw observer
- Add onAgentStatusChange() to OpenClawService exposing the observer's
  listener for the route layer
- Heartbeat every 15s to keep connections alive

Client:
- useAgentDashboard() now subscribes to EventSource at /claw/dashboard/stream
- SSE snapshot event hydrates the React Query cache immediately
- SSE status events patch individual agent status + currentTool in the
  cache without refetching — agent cards update instantly
- Polling fallback raised to 30s since SSE handles real-time

* fix: observer WS handshake — wait for challenge before sending connect

The OpenClaw gateway sends a connect.challenge event before accepting
the connect request. The observer was sending the connect request on
ws.open which raced with the challenge. Now waits for the challenge
event before sending the handshake.

Also add dangerouslyDisableDeviceAuth to the gateway setup config
batch so the observer can connect without device identity on new
installs.

* fix: JSONL reader falls back to most recent file when sessions.json is stale

OpenClaw's sessions.json can record a Pi session ID that doesn't match
the actual JSONL filename on disk. This happens after context compaction
or session restart — the JSONL file gets a new UUID but sessions.json
keeps the old one.

Previously this caused history to silently disappear (the reader tried
to open a non-existent file and returned empty). Now resolveJsonlPath()
checks if the mapped file exists and, when it doesn't, scans the
sessions directory for the most recently modified .jsonl file as a
fallback.

* feat: add ClawSession state machine for reliable per-agent status

The OpenClawObserver only knows about status changes it witnesses via
WS events. If an agent was already running when the observer connected,
or after a reconnect, statuses were stuck at "unknown".

ClawSession is an in-memory state machine that solves this:

1. Seeds from JSONL on first control plane call — reads the latest
   events for each agent and infers working/idle. A session is "working"
   if the last event is a user.message with no subsequent agent.message,
   or an agent.tool_use with no matching agent.tool_result.

2. Receives live transitions from the WS observer — the observer now
   delegates all state management to ClawSession instead of maintaining
   its own status map.

3. Applies a 5-minute staleness threshold — if the last JSONL event
   is older than 5 minutes, assume idle (handles agent crashes).

Consumers (SSE stream, dashboard endpoint) read from ClawSession and
get correct state from the first call — no "unknown" period.

* fix: remove staleTime so dashboard refetches on every mount

* fix: reset stale working status on WS disconnect, eliminate redundant JSONL reads

- Observer resets all "working" agents to "unknown" when the WS closes,
  preventing agents from appearing stuck as Working indefinitely after
  a gateway restart. ClawSession re-seeds correct state on reconnect.

- getDashboard() now derives latestAgentMessage and cost from the
  already-loaded events array for the latest session instead of calling
  latestAgentMessage() and getSessionStats() which each re-read the
  same JSONL file. Reduces file reads from 3x to 1x per agent.
2026-04-25 19:03:03 +05:30
Neel Gupta
4284e88625 feat: Implement lazy LLM judge for passive monitoring (#777)
* fix: double close on stream controller

* feat: initial lazy llm judge impl

* feat: added regex-based matching to insert button context

* fix: tests & bugfix

fix: redundant truthiness check

* fix(tests): stabilize server suites on dev
2026-04-25 12:52:41 +01:00
Nikhil
0b91c735ab chore: bump server version, offset and patch for release (#814) 2026-04-24 12:05:47 -07:00
Nikhil
d189b50b03 fix: package bundled Lima guest agent (#813)
* fix(build): upload Lima runtime files

* fix(build): stage Lima prefix resources

* fix(vm): resolve bundled Lima prefix

* docs(build): document Lima runtime packaging

* chore: self-review fixes

* fix: address review feedback for PR #813
2026-04-24 12:03:26 -07:00
Nikhil
a407e48209 Prefetch runtime VM cache (#811)
* feat: add runtime vm cache sync

* feat: configure runtime vm cache sync

* feat: prefetch vm cache on startup

* feat: await vm cache before vm startup

* fix: recheck vm cache after prefetch wait

* fix: address vm cache review feedback

* build(server): require VM cache manifest env
2026-04-24 10:41:20 -07:00
shivammittal274
1f75b91fba feat(openclaw): add Claude CLI as a CLI-backed provider (#791)
* feat(openclaw): add Claude CLI as a CLI-backed provider

Extensible registry of "OpenClaw CLI-backed providers" — tools that run
as subprocesses inside the gateway container rather than via an API key.
Claude CLI is the first entry; Gemini CLI / Codex CLI / etc. are
one-line additions in the same shape.

Backend:
- New openclaw-cli-providers/ module: types, registry, claude-cli entry.
- OpenClawService: generic ensureAllCliProvidersInstalled() (runs on
  setup/start/restart/auto-start) and getCliProviderAuthStatus(provider).
- Provider dispatch: resolveProviderForAgent() short-circuits CLI
  providers (no env var, no custom-provider merge) before falling
  through to the API-key resolver. No changes to openclaw-provider-map.
- Container runtime: PATH + NPM_CONFIG_PREFIX env so tools installed
  under /home/node/.npm-global/bin (mounted) are discoverable by
  OpenClaw's child-process spawns and persist across restarts.
- New route: GET /claw/providers/:providerId/auth-status returns
  installed / loggedIn / account / plan / error.

Frontend:
- New openclaw-cli-providers.tsx: mirrors backend registry (id, models,
  authLoginCommand), useOpenClawCliProviderAuthStatus hook (2-s poll
  while enabled), OpenClawCliProviderStatusPanel component.
- AgentsPage: synthesized CLI-provider options merged into the Create
  Agent dropdown, inline status panel, auth modal mounting the existing
  AgentTerminal with provider.authLoginCommand, auto-close on loggedIn.
- AgentTerminal: new optional initialCommand + onSessionExit props
  (ref-based so parent re-renders don't rebuild the PTY).

No global ProviderType changes. No custom container image — runtime
install into the mounted home dir persists across restarts.

* fix(openclaw): address review comments for claude-cli provider

- Drop redundant providerId field from OpenClawCliProviderOption (type
  already carries the same value).
- Reuse SetupInput type in resolveProviderForAgent instead of inlining.
- Split ensureCliProviderInstalled into probe + install so logs
  distinguish "already present" from "freshly installed".
- Narrow union in handleCreate via explicit LlmProviderConfig cast; the
  'in'-based narrowing stopped working once the two option shapes
  overlapped on required fields.

* fix: green up server-api tests after claude-cli additions

- Update container-runtime.test.ts snapshot to include the new
  PATH + NPM_CONFIG_PREFIX env args.
- Add a defensive guard in ensureAllCliProvidersInstalled so test
  mocks that swap runtime for a partial stub without execInContainer
  simply skip the install step; production runtime always provides it.

No production behavior change.

* fix(openclaw): use claude /login for auth flow and render terminal full-page

`claude auth login` in 2.1.x silently discards stdin, so the pasted OAuth
code never reaches claude. Switch to the REPL's `/login` slash command,
which does accept a pasted token. Also render the auth terminal
full-page instead of inside a Radix Dialog — the focus trap was hiding
keyboard events from xterm's helper textarea. Finally, guard the async
WebSocket in AgentTerminal against React 18 StrictMode's double-invoke
so the first mount's orphaned WS doesn't leak a second live session.

- terminal-session: pass PATH on podman exec so user-installed CLIs
  resolve in interactive sessions without manual re-exports.
- claude-cli parseAuthStatus: treat exit-code-1 as a valid "not logged
  in" JSON payload instead of a hard error.

* fix(openclaw): drop unnecessary PATH override on podman exec

`podman exec` inherits the container's run-time env (PATH includes
/home/node/.npm-global/bin via `podman run -e PATH=…`), so the extra
`-e PATH` on the exec call was redundant. Reverts the export of
GATEWAY_PATH and the exec flag added in the previous commit.

* feat(openclaw): show CLI-backed providers in Set Up dialog

The Set Up OpenClaw dialog previously listed only API-key LLM
providers. Add the CLI-backed ones (currently just Claude CLI) so
users can bootstrap the gateway with a Claude.ai-subscription-backed
agent without round-tripping through the Create Agent flow first.

When the user picks a CLI provider at setup, skip the apiKey/baseUrl
fields and open the auth terminal immediately after the gateway comes
up, so /login runs in one click.

* fix(openclaw): robust claude auth-status parsing and cleaner CLI UX

parseClaudeAuthStatus was doing JSON.parse on the entire stdout, which
fails when Lima/nerdctl appends a stderr line like `level=fatal
msg="exec failed with exit code 1"` whenever the inner command exits
non-zero (claude auth status exits 1 when not logged in). The panel
then surfaced the raw output as an error. Switch to a line-by-line
scan that picks the first parseable JSON object — handles trailing
noise and nested JSON fields cleanly.

UI polish around the Setup dialog:
- Hide the "uses your API key" hint when the selected provider is
  CLI-backed — it is inaccurate and confusing.
- When a CLI provider is picked in Setup, show a short helper line
  instead of the status panel (the /auth-status poll would be
  pre-gateway and would always fail). Set Up & Start boots the
  gateway and then auto-opens the auth terminal in one click.
- Track the active CLI provider across both Setup and Create dialogs
  so the auth terminal opens for the right provider regardless of
  which dialog triggered it.

* feat(terminal): make selection + copy work under TUI mouse tracking

Interactive TUIs like `claude /login` enable xterm mouse-tracking,
which forwards every click to the app and disables click-drag text
selection. Our terminal had no escape hatch, so users couldn't grab
the OAuth URL.

Three general-purpose fixes (none CLI-specific):
- macOptionClickForcesSelection: Opt+drag always selects on Mac,
  regardless of what the running program does with mouse events.
- Cmd/Ctrl+A and Cmd/Ctrl+C custom key handler: select-all and copy
  to clipboard via navigator.clipboard, even when the TUI would
  swallow the keys.
- Copy button in the terminal header: writes the current selection
  to the clipboard, or the full visible viewport if nothing is
  selected. One-click escape hatch that works in every state.

Applies to any interactive CLI in our terminal (sudo, vim, claude,
gh auth, etc.), not just the claude login flow.

* fix(terminal): make xterm selection actually visible

Selection was registering internally (xterm-selection layer had
correct width/height rects), but the rectangles rendered in
rgb(252,252,251) — practically invisible against the white
background — so users concluded selection was broken.

Root cause: the theme derived selectionBackground from
`withAlpha(resolveCssColor('--accent-orange'), 0.2)`. When the CSS
var failed to resolve it fell back near-white, and the alpha
compositing against the page background made the result
indistinguishable from the background.

Switch to solid terminal-standard selection colors (VSCode-like
light-blue / dark-indigo). Also set selectionInactiveBackground so
the selection persists when focus moves away (useful while copying).
Drop the now-unused withAlpha helper.

* fix(openclaw): handle pretty-printed JSON in claude auth status parser

claude auth status --json emits multi-line pretty-printed JSON. The previous line-by-line parser never matched, so the UI treated every response as an error and surfaced the raw JSON — even when loggedIn was true. Replace with a brace-matching JSON extractor (string- and escape-aware) that tolerates multi-line JSON, leading banners, trailing lima/nerdctl stderr, and nested objects.

* refactor(openclaw): separate exec streams, argv installs, cleaner async cleanup

Audit-driven cleanup. Net -42 lines, four concrete issues fixed:

1. ContainerRuntime.runInContainer() exposes {exitCode, stdout, stderr}
   from the nerdctl exec (ContainerCli.runCommand already tracked them
   separately; we were just throwing stderr into the same string). The
   40-line hand-rolled brace-matching JSON extractor in claude-cli.ts
   existed only because the prior merged-stream output had lima/
   nerdctl's 'level=fatal' line fused with claude's JSON. parser is
   now JSON.parse(stdout.trim()).

2. Replace shell-based 'sh -lc "npm install -g ${pkg}@latest"' with
   argv: execInContainer(['npm','install','-g','${pkg}@${version}']).
   Registry values no longer flow through a shell (removes injection
   surface from future CLI providers). Pinned version instead of
   @latest (adds npmPackageVersion to the provider type).

3. AgentTerminal: replace the 'let cancelled' + out-of-effect
   disposeSocketBindings pattern with an AbortController scoped to
   the effect and a cleanups[] array. Matches the canonical React 18
   async-effect pattern — no partial-cleanup race if StrictMode
   unmounts between the async await and the resolve.

4. AgentTerminal: drop the full-buffer fallback in the Copy button
   (was copying all 8000 scrollback lines when nothing selected —
   surprising). Button now only copies the actual xterm selection,
   or no-ops silently. Users who want everything can Cmd+A first.
2026-04-24 20:13:18 +05:30
Dani Akash
752f42d1fe refactor: migrate chat history to direct JSONL file reads via Lima filesystem (#808)
* feat: draft agent chat ui exploration

* feat: refine agent chat ui draft

* feat: remove outer frame from agent chat workspace

* fix: offset agent chat for app sidebar

* fix: simplify agent conversation shell

* fix: remove redundant chat header actions

* fix: unify agent conversation headers

* fix: tighten agent chat spacing

* fix: bound agent chat composer height

* fix: remove agent chat page inset

* fix: align agent header height with sidepanel

* fix: center agent composer resting state

* fix: anchor multiline composer controls

* fix: remove focus grid from agent home

* fix: remove redundant agent home header

* fix: constrain home agent composer

* fix: match home composer default posture

* feat: add openclaw chat history APIs

* feat: add claw chat history hydration

* fix: stabilize claw chat viewport layout

* fix: use conversation scroll base for claw chat

* refactor: split claw chat controller responsibilities

* fix: keep active agent turns in memory

* fix: normalize openclaw chat sessions

* refactor: use HTTP client for agent history instead of CLI client

Replace the CLI-based getChatHistory() call in getAgentHistoryPage()
with the HTTP client's getSessionHistory() from PR #795. This uses
the direct HTTP transport to OpenClaw's /sessions/<key>/history
endpoint instead of shelling out through the CLI.

- Add filterHttpSessionHistoryMessages() for flat-string content format
- Add normalizeHttpHistoryMessages() for OpenClawSessionHistoryMessage shape
- Update getAgentHistoryPage() to call getSessionHistory() via httpClient
- Remove unused getChatHistory(), filterOpenClawSystemMessages(),
  normalizeChatHistoryMessages(), and getTextContent()
- Update test mocks from cliClient.getChatHistory to httpClient.getSessionHistory
- Update MutableOpenClawService type: chatClient -> httpClient

* fix: fetch all session messages by iterating OpenClaw pagination

OpenClaw's HTTP history endpoint returns a limited page by default.
When called without a limit, only the first ~27 messages were returned,
causing all newer conversation messages to be silently dropped.

Add fetchAllSessionMessages() that iterates through OpenClaw's cursor-
based pagination (200 messages per page) until hasMore is false, then
feeds the complete message list into the existing BrowserOS normalization
and in-memory pagination layer.

* refactor: migrate chat history from HTTP gateway to direct JSONL file reads

Replace the HTTP-based chat history pipeline (BrowserOS server → OpenClaw
gateway /sessions/:key/history pagination loop) with direct JSONL file reads
from the host filesystem via Lima's virtiofs mount.

- Add OpenClawJsonlReader that reads session JSONL files directly from
  ~/.browseros/vm/openclaw/.openclaw/agents/<id>/sessions/
- Replace fetchAllSessionMessages() HTTP pagination with single file read
- Replace CLI-based listSessions() with sessions.json file reads
- Make listSessions, resolveAgentSession, getAgentHistoryPage synchronous
- Remove unused toBrowserOSSession, filterHttpSessionHistoryMessages,
  normalizeHttpHistoryMessages helpers
- Update route handlers to drop unnecessary async/await
- Update tests to use temp JSONL files instead of mocked HTTP/CLI clients

* fix: restore async route handlers for test compatibility with mocked service

* fix: address review feedback — path traversal guard, lazy reader, exists flag

- Add safePath() to OpenClawJsonlReader that validates resolved paths stay
  within stateRoot, preventing path traversal via crafted agentId values
- Use lazy initialization for jsonlReader (nulled on rebuildRuntimeClients)
  instead of creating a new instance per property access
- Return exists: false from resolveSpecificAgentSession when no session
  matches instead of fabricating a ghost session with sessionId: ''
2026-04-24 13:19:46 +05:30
Nikhil
2f8e36546f fix: resize BrowserOS VM resources (#807) 2026-04-23 18:24:49 -07:00
Nikhil
461dcd29e8 fix: upload Lima resources under vendor prefix (#805) 2026-04-23 17:19:45 -07:00
Nikhil
c6c902a4ab feat: improve dev watch Lima preflights (#802)
* feat: improve dev watch lima preflights

* fix: note vm cache sync duration

* fix: address review feedback for PR #802
2026-04-23 17:16:50 -07:00
Nikhil
6e37742a5a feat: reuse agent command chat for agents page (#803) 2026-04-23 17:09:49 -07:00
496 changed files with 38583 additions and 21922 deletions

View File

@@ -1,157 +0,0 @@
name: build-agent
on:
workflow_dispatch:
inputs:
agent:
description: "Agent name from bundle.json"
required: true
type: string
default: openclaw
publish:
description: "Upload to R2 and merge manifest slice"
required: false
default: false
type: boolean
pull_request:
paths:
- "packages/browseros-agent/packages/build-tools/**"
- ".github/workflows/build-agent.yml"
env:
BUN_VERSION: "1.3.6"
PKG_DIR: packages/browseros-agent/packages/build-tools
permissions:
contents: read
jobs:
check:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- uses: oven-sh/setup-bun@v2
with:
bun-version: ${{ env.BUN_VERSION }}
- working-directory: packages/browseros-agent
run: bun install --frozen-lockfile
- working-directory: packages/browseros-agent
run: bun run --filter @browseros/build-tools typecheck
- working-directory: packages/browseros-agent
run: bun run --filter @browseros/build-tools test
build:
needs: check
strategy:
fail-fast: false
matrix:
include:
- arch: arm64
runner: ubuntu-24.04-arm
runs-on: ${{ matrix.runner }}
steps:
- uses: actions/checkout@v4
- uses: oven-sh/setup-bun@v2
with:
bun-version: ${{ env.BUN_VERSION }}
- name: Install podman
run: |
sudo apt-get update
sudo apt-get install -y podman
- working-directory: packages/browseros-agent
run: bun install --frozen-lockfile
- name: Build tarball
working-directory: ${{ env.PKG_DIR }}
env:
AGENT: ${{ inputs.agent || 'openclaw' }}
OUT: ${{ github.workspace }}/dist/images
run: bun run build:tarball -- --agent "$AGENT" --arch "${{ matrix.arch }}" --output-dir "$OUT"
- uses: actions/upload-artifact@v4
with:
name: tarball-${{ inputs.agent || 'openclaw' }}-${{ matrix.arch }}
path: dist/images/
retention-days: 7
smoke:
needs: build
runs-on: ubuntu-24.04-arm
steps:
- uses: actions/checkout@v4
- uses: oven-sh/setup-bun@v2
with:
bun-version: ${{ env.BUN_VERSION }}
- uses: actions/download-artifact@v4
with:
name: tarball-${{ inputs.agent || 'openclaw' }}-arm64
path: dist/images
- name: Install podman
run: |
sudo apt-get update
sudo apt-get install -y podman
- working-directory: packages/browseros-agent
run: bun install --frozen-lockfile
- name: Smoke test tarball
working-directory: ${{ env.PKG_DIR }}
env:
AGENT: ${{ inputs.agent || 'openclaw' }}
run: |
set -euo pipefail
tarball="$(find "$GITHUB_WORKSPACE/dist/images" -name "${AGENT}-*-arm64.tar.gz" -print -quit)"
if [ -z "$tarball" ]; then
echo "missing arm64 tarball artifact for ${AGENT}" >&2
exit 1
fi
bun run smoke:tarball -- --agent "$AGENT" --arch arm64 --tarball "$tarball"
publish:
needs: [build, smoke]
if: ${{ github.event_name == 'workflow_dispatch' && inputs.publish == true }}
runs-on: ubuntu-24.04
environment: release
concurrency:
group: r2-manifest-publish
cancel-in-progress: false
steps:
- uses: actions/checkout@v4
- uses: oven-sh/setup-bun@v2
with:
bun-version: ${{ env.BUN_VERSION }}
- uses: actions/download-artifact@v4
with:
pattern: tarball-*
path: dist/images
merge-multiple: true
- working-directory: packages/browseros-agent
run: bun install --frozen-lockfile
- name: Upload tarballs to R2
working-directory: ${{ env.PKG_DIR }}
env:
R2_ACCOUNT_ID: ${{ secrets.R2_ACCOUNT_ID }}
R2_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
R2_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
R2_BUCKET: ${{ secrets.R2_BUCKET }}
run: |
set -euo pipefail
for file in "$GITHUB_WORKSPACE"/dist/images/*.tar.gz; do
base="$(basename "$file")"
bun run upload -- --file "$file" --key "vm/images/$base" --content-type "application/gzip" --sidecar-sha
done
- name: Merge agent slice into manifest
working-directory: ${{ env.PKG_DIR }}
env:
AGENT: ${{ inputs.agent || 'openclaw' }}
R2_ACCOUNT_ID: ${{ secrets.R2_ACCOUNT_ID }}
R2_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
R2_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
R2_BUCKET: ${{ secrets.R2_BUCKET }}
run: |
set -euo pipefail
mkdir -p dist/images
cp -R "$GITHUB_WORKSPACE"/dist/images/* dist/images/
bun run download -- --key vm/manifest.json --out dist/baseline-manifest.json
bun run emit-manifest -- \
--slice "agents:${AGENT}" \
--dist-dir dist \
--merge-from dist/baseline-manifest.json \
--out dist/manifest.json
bun run upload -- --file dist/manifest.json --key vm/manifest.json --content-type "application/json"

View File

@@ -14,7 +14,7 @@ on:
config:
description: 'Eval config file (relative to apps/eval/)'
required: false
default: 'configs/browseros-agent-weekly.json'
default: 'configs/legacy/browseros-agent-weekly.json'
permissions:
contents: read
@@ -30,8 +30,9 @@ jobs:
- name: Install BrowserOS
run: |
wget -q https://github.com/browseros-ai/BrowserOS/releases/download/v0.44.0.1/BrowserOS_v0.44.0.1_amd64.deb
sudo dpkg -i BrowserOS_v0.44.0.1_amd64.deb
# Rolling stable channel — see https://cdn.browseros.com/download/BrowserOS.deb
wget -q -O BrowserOS.deb https://cdn.browseros.com/download/BrowserOS.deb
sudo dpkg -i BrowserOS.deb
browseros --version || echo "BrowserOS installed at $(which browseros)"
- name: Install Bun
@@ -41,7 +42,15 @@ jobs:
- name: Install dependencies
working-directory: packages/browseros-agent
run: bun install --ignore-scripts && bun run build:agent-sdk
run: bun install --ignore-scripts
- name: Install Python eval dependencies
# agisdk pinned so silent upstream releases can't shift task definitions
# or grader behavior. Bump intentionally with a documented re-baseline.
run: pip install agisdk==0.3.5 requests
- name: Clone WebArena-Infinity
run: git clone --depth 1 https://github.com/web-arena-x/webarena-infinity.git /tmp/webarena-infinity
- name: Install xvfb
run: sudo apt-get update && sudo apt-get install -y xvfb
@@ -53,34 +62,32 @@ jobs:
curl -sL -o /tmp/nopecha.zip https://github.com/NopeCHALLC/nopecha-extension/releases/latest/download/chromium_automation.zip
unzip -qo /tmp/nopecha.zip -d extensions/nopecha
- name: Run eval
- name: Run eval and publish to R2
working-directory: packages/browseros-agent/apps/eval
env:
FIREWORKS_API_KEY: ${{ secrets.FIREWORKS_API_KEY }}
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
CLAUDE_CODE_OAUTH_TOKEN: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
NOPECHA_API_KEY: ${{ secrets.NOPECHA_API_KEY }}
BROWSEROS_BINARY: /usr/bin/browseros
EVAL_CONFIG: ${{ github.event.inputs.config || 'configs/browseros-agent-weekly.json' }}
run: |
echo "Running eval with config: $EVAL_CONFIG"
xvfb-run --auto-servernum --server-args="-screen 0 1440x900x24" bun run src/index.ts -c "$EVAL_CONFIG"
- name: Upload runs to R2
if: success()
working-directory: packages/browseros-agent/apps/eval
env:
EVAL_R2_ACCOUNT_ID: ${{ secrets.EVAL_R2_ACCOUNT_ID }}
EVAL_R2_ACCESS_KEY_ID: ${{ secrets.EVAL_R2_ACCESS_KEY_ID }}
EVAL_R2_SECRET_ACCESS_KEY: ${{ secrets.EVAL_R2_SECRET_ACCESS_KEY }}
EVAL_R2_BUCKET: ${{ secrets.EVAL_R2_BUCKET }}
EVAL_R2_CDN_BASE_URL: ${{ secrets.EVAL_R2_CDN_BASE_URL }}
EVAL_CONFIG: ${{ github.event.inputs.config || 'configs/browseros-agent-weekly.json' }}
BROWSEROS_BINARY: /usr/bin/browseros
WEBARENA_INFINITY_DIR: /tmp/webarena-infinity
# OpenClaw container runtime is macOS-only; opt the Linux runner
# into the no-op stub so the server can boot and the eval can run.
BROWSEROS_SKIP_OPENCLAW: '1'
EVAL_CONFIG: ${{ github.event.inputs.config || 'configs/legacy/browseros-agent-weekly.json' }}
run: |
CONFIG_NAME=$(basename "$EVAL_CONFIG" .json)
bun scripts/upload-run.ts "results/$CONFIG_NAME"
echo "Running eval with config: $EVAL_CONFIG"
xvfb-run --auto-servernum --server-args="-screen 0 1440x900x24" bun run src/index.ts suite --config "$EVAL_CONFIG" --publish r2
- name: Generate trend report
if: success()
timeout-minutes: 5
continue-on-error: true
working-directory: packages/browseros-agent
env:
EVAL_R2_ACCOUNT_ID: ${{ secrets.EVAL_R2_ACCOUNT_ID }}
@@ -96,3 +103,11 @@ jobs:
with:
name: eval-report-${{ github.run_id }}
path: /tmp/eval-report.html
- name: Upload server stderr logs (for post-mortem on startup failures)
if: always()
uses: actions/upload-artifact@v4
with:
name: browseros-server-logs-${{ github.run_id }}
path: /tmp/browseros-server-logs/
if-no-files-found: ignore

View File

@@ -1,168 +1,11 @@
name: Release BrowserOS Agent SDK
name: Release BrowserOS Agent SDK (disabled)
on:
workflow_dispatch:
concurrency:
group: release-agent-sdk
cancel-in-progress: false
jobs:
publish:
if: github.ref == 'refs/heads/main'
disabled:
if: ${{ false }}
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
defaults:
run:
working-directory: packages/browseros-agent/packages/agent-sdk
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0
- uses: oven-sh/setup-bun@v2
- uses: actions/setup-node@v6
with:
node-version: "20"
registry-url: "https://registry.npmjs.org"
- name: Install dependencies
run: bun ci
working-directory: packages/browseros-agent
- name: Build
run: bun run build
- name: Test
run: bun test
- name: Get version
id: version
run: |
echo "version=$(node -p "require('./package.json').version")" >> "$GITHUB_OUTPUT"
echo "release_sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
- name: Generate release notes
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
SDK_PATH="packages/browseros-agent/packages/agent-sdk"
CURRENT_TAG="agent-sdk-v${{ steps.version.outputs.version }}"
# Find the previous tag, excluding the current version's tag
# (which may already exist from a prior failed run)
PREV_TAG=$(git tag -l "agent-sdk-v*" --sort=-v:refname | grep -v "^${CURRENT_TAG}$" | head -n 1)
if [ -z "$PREV_TAG" ]; then
echo "Initial release" > /tmp/release-notes.md
else
# Get commits scoped to the SDK directory
COMMITS=$(git log "$PREV_TAG"..HEAD --pretty=format:"%H" -- "$SDK_PATH")
if [ -z "$COMMITS" ]; then
echo "No notable changes." > /tmp/release-notes.md
else
echo "## What's Changed" > /tmp/release-notes.md
echo "" >> /tmp/release-notes.md
# For each commit, find the associated PR and format with author
CONTRIBUTORS=""
while IFS= read -r SHA; do
# Get commit subject and author
SUBJECT=$(git log -1 --pretty=format:"%s" "$SHA")
AUTHOR=$(git log -1 --pretty=format:"%an" "$SHA")
GITHUB_USER=$(gh api "/repos/${{ github.repository }}/commits/${SHA}" --jq '.author.login // empty' 2>/dev/null)
# Find associated PR number
PR_NUM=$(gh api "/repos/${{ github.repository }}/commits/${SHA}/pulls" --jq '.[0].number // empty' 2>/dev/null)
# Format line: skip PR number if already in the commit subject
# (squash merges include "(#123)" in the subject automatically)
if [ -n "$PR_NUM" ] && ! echo "$SUBJECT" | grep -qF "(#${PR_NUM})"; then
echo "- ${SUBJECT} (#${PR_NUM})" >> /tmp/release-notes.md
else
echo "- ${SUBJECT}" >> /tmp/release-notes.md
fi
done <<< "$COMMITS"
fi
fi
working-directory: ${{ github.workspace }}
- name: Publish
run: npm publish --access public
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
- name: Create GitHub release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
TAG="agent-sdk-v${{ steps.version.outputs.version }}"
RELEASE_SHA="${{ steps.version.outputs.release_sha }}"
TITLE="BrowserOS Agent SDK - v${{ steps.version.outputs.version }}"
# Create or reuse tag (idempotent for re-runs)
if git rev-parse "$TAG" >/dev/null 2>&1; then
echo "Tag $TAG already exists, skipping tag creation"
else
git tag "$TAG" "$RELEASE_SHA"
fi
# Push tag (skip if already on remote)
if git ls-remote --tags origin "$TAG" | grep -q "$TAG"; then
echo "Tag $TAG already on remote, skipping push"
else
git push origin "$TAG"
fi
# Create or update release
if gh release view "$TAG" >/dev/null 2>&1; then
echo "Release $TAG already exists, updating"
gh release edit "$TAG" --title "$TITLE" --notes-file /tmp/release-notes.md
else
gh release create "$TAG" --title "$TITLE" --notes-file /tmp/release-notes.md
fi
working-directory: ${{ github.workspace }}
- name: Update CHANGELOG.md via PR
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
VERSION="${{ steps.version.outputs.version }}"
DATE=$(date -u +"%Y-%m-%d")
BRANCH="docs/agent-sdk-changelog-v${VERSION}"
CHANGELOG="packages/browseros-agent/packages/agent-sdk/CHANGELOG.md"
# Return to main before branching
git checkout main
# Use head/tail to safely insert without sed quoting issues
{
head -n 1 "$CHANGELOG"
echo ""
echo "## v${VERSION} (${DATE})"
echo ""
cat /tmp/release-notes.md
echo ""
tail -n +2 "$CHANGELOG"
} > /tmp/new-changelog.md
mv /tmp/new-changelog.md "$CHANGELOG"
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git checkout -b "$BRANCH"
git add "$CHANGELOG"
git commit -m "docs: update agent-sdk changelog for v${VERSION}"
git push origin "$BRANCH"
gh pr create \
--title "docs: update agent-sdk changelog for v${VERSION}" \
--body "Auto-generated changelog update for BrowserOS Agent SDK v${VERSION}." \
--base main \
--head "$BRANCH"
gh pr merge "$BRANCH" --squash --auto || true
working-directory: ${{ github.workspace }}
- run: echo "Agent SDK publishing is disabled."

View File

@@ -54,28 +54,24 @@ jobs:
command: (cd apps/server && bun run test:integration)
junit_path: test-results/server-integration.xml
needs_browser: true
- suite: server-sdk
command: (cd apps/server && bun run test:sdk)
junit_path: test-results/server-sdk.xml
needs_browser: true
- suite: server-lib
command: (cd apps/server && bun run test:lib)
junit_path: test-results/server-lib.xml
needs_browser: false
- suite: server-root
command: (cd apps/server && bun run test:root)
junit_path: test-results/server-root.xml
needs_browser: false
- suite: agent
command: bun run test:agent
command: (cd apps/agent && bun run test)
junit_path: test-results/agent.xml
needs_browser: false
- suite: eval
command: bun run test:eval
command: (cd apps/eval && bun run test)
junit_path: test-results/eval.xml
needs_browser: false
- suite: agent-sdk
command: bun run test:agent-sdk
junit_path: test-results/agent-sdk.xml
needs_browser: false
- suite: build
command: bun run test:build
command: bun run ./scripts/run-bun-test.ts ./scripts/build
junit_path: test-results/build.xml
needs_browser: false

View File

@@ -188,6 +188,21 @@ We'd love your help making BrowserOS better! See our [Contributing Guide](CONTRI
- [ungoogled-chromium](https://github.com/ungoogled-software/ungoogled-chromium) — BrowserOS uses some patches for enhanced privacy. Thanks to everyone behind this project!
- [The Chromium Project](https://www.chromium.org/) — at the core of BrowserOS, making it possible to exist in the first place.
## Citation
If you use BrowserOS in your research or project, please cite:
```bibtex
@software{browseros2025,
author = {Nithin Sonti and Nikhil Sonti and {BrowserOS-team}},
title = {BrowserOS: The open-source Agentic browser},
url = {https://github.com/browseros-ai/BrowserOS},
year = {2025},
publisher = {GitHub},
license = {AGPL-3.0},
}
```
## License
BrowserOS is open source under the [AGPL-3.0 license](LICENSE).

View File

@@ -180,6 +180,7 @@ packages/*/dist
browseros-server
browseros-server.exe
browseros-server-*
tools/dogfood/browseros-dogfood
tools/dev/browseros-dev
log.txt

View File

@@ -1,6 +1,6 @@
# BrowserOS Agent
The agent platform powering [BrowserOS](https://github.com/browseros-ai/BrowserOS) — contains the MCP server, agent UI, CLI, evaluation framework, and SDK.
The agent platform powering [BrowserOS](https://github.com/browseros-ai/BrowserOS) — contains the MCP server, agent UI, CLI, and evaluation framework.
## Monorepo Structure
@@ -12,7 +12,6 @@ apps/
eval/ # Evaluation framework for benchmarking agents
packages/
agent-sdk/ # Node.js SDK (@browseros-ai/agent-sdk)
cdp-protocol/ # Type-safe Chrome DevTools Protocol bindings
shared/ # Shared constants (ports, timeouts, limits)
```
@@ -23,7 +22,6 @@ packages/
| `apps/agent` | Agent UI — Chrome extension for the chat interface |
| `apps/cli` | Go CLI — control BrowserOS from the terminal or AI coding agents |
| `apps/eval` | Benchmark framework — WebVoyager, Mind2Web evaluation |
| `packages/agent-sdk` | Node.js SDK for browser automation with natural language |
| `packages/cdp-protocol` | Auto-generated CDP type bindings used by the server |
| `packages/shared` | Shared constants used across packages |
@@ -81,14 +79,15 @@ cp apps/server/.env.example apps/server/.env.development
cp apps/agent/.env.example apps/agent/.env.development
cp apps/server/.env.production.example apps/server/.env.production
# Install deps, generate agent code, and sync the VM cache
# Install deps and generate agent code
bun run dev:setup
# Start the full dev environment
bun run dev:watch
```
`dev:watch` exits when the VM cache manifest is missing, but setup stays in `dev:setup`.
`dev:watch` starts the server immediately. OpenClaw VM/image prewarm runs from
the server startup path and pulls the configured GHCR image on demand.
### Environment Variables
@@ -158,9 +157,14 @@ bun run build:server # Build production server resource artifacts and u
bun run build:agent # Build agent extension
# Test
bun run test # Run standard tests
bun run test:cdp # Run CDP-based tests
bun run test:integration # Run integration tests
bun run test # Run all tests
bun run test:all # Run all tests
bun run test:main # Run key server tools and integration tests
# App-specific test groups (from packages/browseros-agent)
cd apps/server && bun run test:tools
cd apps/server && bun run test:cdp
cd apps/server && bun run test:integration
# Quality
bun run lint # Check with Biome

View File

@@ -0,0 +1,50 @@
import type { Provider } from './chatComponentTypes'
export interface ProviderOptionGroup {
key: 'llm' | 'acp'
label: string
options: Provider[]
}
export function groupProviderOptions(
providers: Provider[],
): ProviderOptionGroup[] {
const llm = providers.filter((provider) => provider.kind !== 'acp')
const acp = providers.filter((provider) => provider.kind === 'acp')
return [
...(llm.length
? [{ key: 'llm' as const, label: 'AI Providers', options: llm }]
: []),
...(acp.length
? [{ key: 'acp' as const, label: 'Agents', options: acp }]
: []),
]
}
export function getProviderSearchValue(
provider: Provider,
groupLabel: string,
): string {
return [
provider.id,
provider.name,
provider.type,
groupLabel,
provider.adapterName,
provider.modelLabel,
]
.filter(Boolean)
.join(' ')
}
export function getProviderSubtitle(provider: Provider): string | undefined {
if (provider.kind !== 'acp') return undefined
return [
provider.adapterName,
provider.modelLabel,
provider.modelControl === 'best-effort' ? 'best effort' : undefined,
]
.filter(Boolean)
.join(' · ')
}

View File

@@ -0,0 +1,72 @@
import { describe, expect, it } from 'bun:test'
import {
getProviderSearchValue,
getProviderSubtitle,
groupProviderOptions,
} from './ChatProviderSelector.helpers'
import type { Provider } from './chatComponentTypes'
const options: Provider[] = [
{ kind: 'llm', id: 'browseros', name: 'BrowserOS', type: 'browseros' },
{
kind: 'llm',
id: 'anthropic-sonnet',
name: 'Anthropic Sonnet',
type: 'anthropic',
},
{
kind: 'acp',
id: 'agent-claude-review',
name: 'Review Bot',
type: 'acp',
adapterName: 'Claude Code',
modelLabel: 'Haiku',
modelControl: 'best-effort',
},
{
kind: 'acp',
id: 'agent-codex-browser',
name: 'Browser Driver',
type: 'acp',
adapterName: 'Codex',
modelLabel: 'GPT-5.5',
modelControl: 'runtime-supported',
},
]
describe('groupProviderOptions', () => {
it('groups normal providers separately from created agents', () => {
expect(groupProviderOptions(options)).toEqual([
{
key: 'llm',
label: 'AI Providers',
options: [options[0], options[1]],
},
{
key: 'acp',
label: 'Agents',
options: [options[2], options[3]],
},
])
})
})
describe('getProviderSearchValue', () => {
it('matches created-agent group labels and item labels', () => {
expect(getProviderSearchValue(options[2], 'Agents')).toContain('Agents')
expect(getProviderSearchValue(options[2], 'Agents')).toContain('Review Bot')
expect(getProviderSearchValue(options[2], 'Agents')).toContain(
'Claude Code',
)
})
})
describe('getProviderSubtitle', () => {
it('describes created-agent runtime context without model-target copy', () => {
expect(getProviderSubtitle(options[2])).toBe(
'Claude Code · Haiku · best effort',
)
expect(getProviderSubtitle(options[3])).toBe('Codex · GPT-5.5')
expect(getProviderSubtitle(options[0])).toBeUndefined()
})
})

View File

@@ -1,4 +1,4 @@
import { Check, Plus } from 'lucide-react'
import { Bot, Check, Plus } from 'lucide-react'
import type { FC, PropsWithChildren } from 'react'
import { useState } from 'react'
import {
@@ -17,6 +17,11 @@ import {
import { BrowserOSIcon, ProviderIcon } from '@/lib/llm-providers/providerIcons'
import type { ProviderType } from '@/lib/llm-providers/types'
import { cn } from '@/lib/utils'
import {
getProviderSearchValue,
getProviderSubtitle,
groupProviderOptions,
} from './ChatProviderSelector.helpers'
import type { Provider } from './chatComponentTypes'
interface ChatProviderSelectorProps {
@@ -29,54 +34,58 @@ export const ChatProviderSelector: FC<
PropsWithChildren<ChatProviderSelectorProps>
> = ({ children, providers, selectedProvider, onSelectProvider }) => {
const [open, setOpen] = useState(false)
const groups = groupProviderOptions(providers)
return (
<Popover open={open} onOpenChange={setOpen}>
<PopoverTrigger asChild>{children}</PopoverTrigger>
<PopoverContent side="bottom" align="start" className="w-48 p-0">
<PopoverContent side="bottom" align="start" className="w-64 p-0">
<Command>
<CommandInput placeholder="Search providers..." className="h-9" />
<CommandInput
placeholder="Search providers or agents..."
className="h-9"
/>
<CommandList>
<div className="my-2 px-2 font-semibold text-muted-foreground text-xs uppercase tracking-wide">
AI Provider
</div>
<CommandEmpty>No provider found</CommandEmpty>
<CommandGroup>
{providers.map((provider) => {
const isSelected = selectedProvider.id === provider.id
return (
<CommandItem
key={provider.id}
value={`${provider.id} ${provider.name}`}
onSelect={() => {
onSelectProvider(provider)
setOpen(false)
}}
className={cn(
'flex w-full items-center gap-3 rounded-md p-2 transition-colors',
isSelected && 'bg-[var(--accent-orange)]/10',
)}
>
<span className="text-muted-foreground">
{provider.type === 'browseros' ? (
<BrowserOSIcon size={18} />
) : (
<ProviderIcon
type={provider.type as ProviderType}
size={18}
/>
{groups.map((group) => (
<CommandGroup key={group.key} heading={group.label}>
{group.options.map((provider) => {
const isSelected = selectedProvider.id === provider.id
const subtitle = getProviderSubtitle(provider)
return (
<CommandItem
key={provider.id}
value={getProviderSearchValue(provider, group.label)}
onSelect={() => {
onSelectProvider(provider)
setOpen(false)
}}
className={cn(
'flex w-full items-center gap-3 rounded-md p-2 transition-colors',
isSelected && 'bg-[var(--accent-orange)]/10',
)}
</span>
<span className="flex-1 text-left text-sm">
{provider.name}
</span>
{isSelected && (
<Check className="h-3.5 w-3.5 text-[var(--accent-orange)]" />
)}
</CommandItem>
)
})}
</CommandGroup>
>
<span className="text-muted-foreground">
<ProviderOptionIcon provider={provider} />
</span>
<span className="min-w-0 flex-1 text-left">
<span className="block truncate text-sm">
{provider.name}
</span>
{subtitle && (
<span className="block truncate text-muted-foreground text-xs">
{subtitle}
</span>
)}
</span>
{isSelected && (
<Check className="h-3.5 w-3.5 text-[var(--accent-orange)]" />
)}
</CommandItem>
)
})}
</CommandGroup>
))}
<div className="border-border border-t p-1">
<button
type="button"
@@ -96,3 +105,9 @@ export const ChatProviderSelector: FC<
</Popover>
)
}
function ProviderOptionIcon({ provider }: { provider: Provider }) {
if (provider.kind === 'acp') return <Bot size={18} />
if (provider.type === 'browseros') return <BrowserOSIcon size={18} />
return <ProviderIcon type={provider.type as ProviderType} size={18} />
}

View File

@@ -1,7 +1,14 @@
import type { ProviderType } from '@/lib/llm-providers/types'
export type ChatProviderType = ProviderType | 'acp'
export interface Provider {
id: string
name: string
type: ProviderType
type: ChatProviderType
kind: 'llm' | 'acp'
agentId?: string
adapterName?: string
modelLabel?: string
modelControl?: 'runtime-supported' | 'best-effort'
}

View File

@@ -74,6 +74,18 @@ const primaryNavItems: NavItem[] = [
{ name: 'Settings', to: '/settings/ai', icon: Settings },
]
function isNavItemActive(item: NavItem, pathname: string): boolean {
if (item.to === '/settings/ai') {
return pathname.startsWith('/settings')
}
if (item.to === '/agents') {
return pathname === '/agents' || pathname.startsWith('/agents/')
}
return pathname === item.to
}
export const SidebarNavigation: FC<SidebarNavigationProps> = ({
expanded = true,
}) => {
@@ -90,10 +102,7 @@ export const SidebarNavigation: FC<SidebarNavigationProps> = ({
<nav className="space-y-1">
{filteredItems.map((item) => {
const Icon = item.icon
const isActive =
item.to === '/settings/ai'
? location.pathname.startsWith('/settings')
: location.pathname === item.to
const isActive = isNavItemActive(item, location.pathname)
const navItem = (
<NavLink

View File

@@ -113,7 +113,22 @@ export const App: FC = () => {
<Route path="connect-apps" element={<ConnectMCP />} />
<Route path="scheduled" element={<ScheduledTasksPage />} />
{alphaEnabled ? (
<Route path="agents" element={<AgentsPage />} />
<>
<Route path="agents" element={<AgentsPage />} />
<Route element={<AgentCommandLayout />}>
<Route
path="agents/:agentId"
element={
<AgentCommandConversation
variant="page"
backPath="/agents"
agentPathPrefix="/agents"
createAgentPath="/agents"
/>
}
/>
</Route>
</>
) : null}
{alphaEnabled ? (
<Route path="admin" element={<AdminDashboardPage />} />

View File

@@ -1,114 +0,0 @@
import { Bot } from 'lucide-react'
import type { FC } from 'react'
import type { AgentCardData } from '@/lib/agent-conversations/types'
import { cn } from '@/lib/utils'
interface AgentCardProps {
agent: AgentCardData
onClick: () => void
active?: boolean
}
function formatTimestamp(timestamp?: number): string {
if (!timestamp) return 'No activity yet'
const diff = Date.now() - timestamp
const minutes = Math.floor(diff / 60000)
if (minutes < 1) return 'just now'
if (minutes < 60) return `${minutes}m ago`
const hours = Math.floor(minutes / 60)
if (hours < 24) return `${hours}h ago`
return `${Math.floor(hours / 24)}d ago`
}
function getStatusLabel(status: AgentCardData['status']): string {
if (status === 'working') return 'Working'
if (status === 'error') return 'Error'
return 'Ready'
}
function getStatusTone(status: AgentCardData['status']): string {
if (status === 'working') return 'bg-amber-500'
if (status === 'error') return 'bg-destructive'
return 'bg-emerald-500'
}
export const AgentCardExpanded: FC<AgentCardProps> = ({
agent,
onClick,
active,
}) => (
<button
type="button"
onClick={onClick}
className={cn(
'group flex min-h-32 w-full min-w-0 flex-col rounded-2xl border p-4 text-left shadow-sm transition-all duration-200',
active
? 'border-border/80 bg-card shadow-md ring-1 ring-[var(--accent-orange)]/20'
: 'border-border/60 bg-card/85 hover:border-border hover:bg-card hover:shadow-md',
)}
>
<div className="flex items-start justify-between gap-3">
<div className="flex min-w-0 items-center gap-3">
<div
className={cn(
'flex size-10 shrink-0 items-center justify-center rounded-xl',
active
? 'bg-[var(--accent-orange)]/10 text-[var(--accent-orange)]'
: 'bg-muted text-muted-foreground',
)}
>
<Bot className="size-5" />
</div>
<div className="min-w-0">
<div className="truncate font-semibold text-sm">{agent.name}</div>
<div className="truncate text-muted-foreground text-xs">
{agent.model ?? 'OpenClaw agent'}
</div>
</div>
</div>
<div className="flex items-center gap-2 rounded-full border border-border/60 bg-background/70 px-2.5 py-1 text-[11px] text-muted-foreground">
<span
className={cn('size-2 rounded-full', getStatusTone(agent.status))}
/>
<span>{getStatusLabel(agent.status)}</span>
</div>
</div>
<div className="mt-4 flex-1">
<p className="line-clamp-2 text-foreground/90 text-sm">
{agent.lastMessage ??
'Start a conversation to see recent work and summaries.'}
</p>
</div>
<div className="mt-4 flex items-center justify-between gap-3 text-muted-foreground text-xs">
<span>{formatTimestamp(agent.lastMessageTimestamp)}</span>
<span>Open conversation</span>
</div>
</button>
)
export const AgentCardCompact: FC<AgentCardProps> = ({
agent,
onClick,
active,
}) => (
<button
type="button"
onClick={onClick}
className={cn(
'inline-flex items-center gap-2 rounded-full border px-3 py-2 text-sm transition-colors',
active
? 'border-border bg-card shadow-sm ring-1 ring-[var(--accent-orange)]/20'
: 'border-border/60 bg-card/85 text-foreground hover:border-border hover:bg-card',
)}
>
<span
className={cn(
'size-2 rounded-full',
active ? 'bg-[var(--accent-orange)]' : getStatusTone(agent.status),
)}
/>
<span className="truncate">{agent.name}</span>
</button>
)

View File

@@ -1,70 +1,71 @@
import { Plus } from 'lucide-react'
import type { FC } from 'react'
import type { AgentCardData } from '@/lib/agent-conversations/types'
import type {
HarnessAdapterDescriptor,
HarnessAdapterHealth,
HarnessAgent,
HarnessAgentAdapter,
} from '@/entrypoints/app/agents/agent-harness-types'
import { cn } from '@/lib/utils'
import { AgentCardCompact, AgentCardExpanded } from './AgentCard'
import { HomeAgentCard } from './HomeAgentCard'
interface AgentCardDockProps {
agents: AgentCardData[]
agents: HarnessAgent[]
adapters: HarnessAdapterDescriptor[]
activeAgentId?: string
onSelectAgent: (agentId: string) => void
onCreateAgent?: () => void
compact?: boolean
}
function CreateAgentButton({
compact,
onCreateAgent,
}: {
compact?: boolean
onCreateAgent: () => void
}) {
function CreateAgentButton({ onCreateAgent }: { onCreateAgent: () => void }) {
return (
<button
type="button"
onClick={onCreateAgent}
className={cn(
'flex shrink-0 items-center justify-center gap-2 border border-dashed text-muted-foreground transition-colors hover:border-[var(--accent-orange)] hover:text-[var(--accent-orange)]',
compact
? 'rounded-full px-3 py-2 text-sm'
: 'min-h-32 rounded-2xl px-5 py-4',
'flex min-h-32 shrink-0 items-center justify-center gap-2 rounded-2xl border border-dashed px-5 py-4 text-muted-foreground transition-colors',
'hover:border-[var(--accent-orange)] hover:text-[var(--accent-orange)]',
)}
>
<Plus className={compact ? 'size-3.5' : 'size-5'} />
<span>{compact ? 'New' : 'Create agent'}</span>
<Plus className="size-5" />
<span>Create agent</span>
</button>
)
}
/**
* 3-column grid of HomeAgentCards plus a trailing "Create agent"
* tile. The previous `compact` mode (rendered a horizontal pill rail)
* had no callers and was dropped along with the legacy AgentCard.
*/
export const AgentCardDock: FC<AgentCardDockProps> = ({
agents,
adapters,
activeAgentId,
onSelectAgent,
onCreateAgent,
compact,
}) => {
if (agents.length === 0 && !onCreateAgent) return null
const Card = compact ? AgentCardCompact : AgentCardExpanded
const adapterHealth = new Map<HarnessAgentAdapter, HarnessAdapterHealth>()
for (const descriptor of adapters) {
if (descriptor.health) adapterHealth.set(descriptor.id, descriptor.health)
}
return (
<div
className={cn(
compact
? 'flex items-center gap-2 overflow-x-auto pb-1'
: 'grid gap-4 md:grid-cols-3',
)}
>
<div className="grid gap-4 md:grid-cols-3">
{agents.map((agent) => (
<Card
key={agent.agentId}
<HomeAgentCard
key={agent.id}
agent={agent}
active={agent.agentId === activeAgentId}
onClick={() => onSelectAgent(agent.agentId)}
adapter={agent.adapter}
adapterHealth={adapterHealth.get(agent.adapter) ?? null}
active={agent.id === activeAgentId}
onClick={() => onSelectAgent(agent.id)}
/>
))}
{onCreateAgent ? (
<CreateAgentButton compact={compact} onCreateAgent={onCreateAgent} />
<CreateAgentButton onCreateAgent={onCreateAgent} />
) : null}
</div>
)

View File

@@ -1,194 +1,438 @@
import { Bot, Home, RotateCcw } from 'lucide-react'
import { type FC, useEffect, useRef } from 'react'
import { ArrowLeft, Bot, Home } from 'lucide-react'
import { type FC, useEffect, useMemo, useRef } from 'react'
import { Navigate, useNavigate, useParams, useSearchParams } from 'react-router'
import { Button } from '@/components/ui/button'
import type { AgentEntry } from '@/entrypoints/app/agents/useOpenClaw'
import {
cancelHarnessTurn,
useEnqueueHarnessMessage,
useHarnessAgents,
useRemoveHarnessQueuedMessage,
} from '@/entrypoints/app/agents/useAgents'
import {
type AgentEntry,
getModelDisplayName,
} from '@/entrypoints/app/agents/useOpenClaw'
import { cn } from '@/lib/utils'
import { useAgentCommandData } from './agent-command-layout'
import { ClawChat } from './ClawChat'
import { ConversationInput } from './ConversationInput'
import { ConversationMessage } from './ConversationMessage'
import {
buildChatHistoryFromClawMessages,
filterTurnsPersistedInHistory,
flattenHistoryPages,
} from './claw-chat-types'
import { QueuePanel } from './QueuePanel'
import { useAgentConversation } from './useAgentConversation'
import { useHarnessChatHistory } from './useHarnessChatHistory'
function StatusBadge({ status }: { status: string }) {
return (
<div className="inline-flex items-center gap-2 rounded-full border border-border/60 bg-card px-3 py-1 text-[11px] text-muted-foreground uppercase tracking-[0.18em]">
<span
className={cn(
'size-1.5 rounded-full',
status === 'Working on your request'
? 'bg-amber-500'
: status === 'Ready'
? 'bg-emerald-500'
: status === 'Offline'
? 'bg-muted-foreground/50'
: 'bg-[var(--accent-orange)]',
)}
/>
<span>{status}</span>
</div>
)
}
function AgentIdentity({
name,
meta,
className,
}: {
name: string
meta: string
className?: string
}) {
return (
<div className={cn('min-w-0', className)}>
<div className="truncate font-semibold text-[15px] leading-5">{name}</div>
<div className="truncate text-muted-foreground text-xs leading-5">
{meta}
</div>
</div>
)
}
function ConversationHeader({
agentName,
agentMeta,
status,
backLabel,
backTarget,
onGoHome,
onReset,
}: {
agentName: string
agentMeta: string
status: string
backLabel: string
backTarget: 'home' | 'page'
onGoHome: () => void
onReset: () => void
}) {
const BackIcon = backTarget === 'home' ? Home : ArrowLeft
return (
<div className="overflow-hidden rounded-[1.5rem] border border-border/60 bg-card/95 shadow-sm backdrop-blur">
<div className="flex items-center justify-between gap-3 px-5 py-4">
<div className="flex min-w-0 items-center gap-3">
<Button
variant="ghost"
size="icon"
onClick={onGoHome}
className="rounded-xl"
title="Back to home"
>
<Home className="size-4" />
</Button>
<div className="flex size-11 shrink-0 items-center justify-center rounded-2xl bg-muted text-muted-foreground">
<Bot className="size-5" />
</div>
<div className="min-w-0">
<div className="truncate font-semibold text-sm">{agentName}</div>
<div className="truncate text-muted-foreground text-sm">
{status}
</div>
</div>
</div>
<div className="flex h-14 items-center justify-between gap-4 border-border/50 border-b px-5">
<div className="flex min-w-0 items-center gap-3">
<Button
variant="ghost"
size="sm"
onClick={onReset}
className="rounded-xl text-muted-foreground"
size="icon"
onClick={onGoHome}
className="size-8 rounded-xl lg:hidden"
title={backLabel}
>
<RotateCcw className="mr-2 size-4" />
New conversation
<BackIcon className="size-4" />
</Button>
</div>
</div>
)
}
function EmptyConversationState({ agentName }: { agentName: string }) {
return (
<div className="flex min-h-full items-center justify-center py-10">
<div className="max-w-md rounded-[1.5rem] border border-border/60 bg-card/90 px-8 py-10 text-center shadow-sm backdrop-blur">
<div className="mx-auto flex size-14 items-center justify-center rounded-2xl bg-muted text-muted-foreground">
<Bot className="size-6" />
<div className="flex size-8 shrink-0 items-center justify-center rounded-xl bg-muted text-muted-foreground">
<Bot className="size-4" />
</div>
<AgentIdentity name={agentName} meta={agentMeta} />
</div>
<StatusBadge status={status} />
</div>
)
}
function AgentRailHeader({ onGoHome }: { onGoHome: () => void }) {
return (
<div className="hidden h-14 items-center border-border/50 border-r border-b bg-background/70 px-4 lg:flex">
<div className="flex min-w-0 items-center gap-3">
<Button
variant="ghost"
size="icon"
onClick={onGoHome}
className="size-8 rounded-xl"
title="Back to home"
>
<ArrowLeft className="size-4" />
</Button>
<div className="truncate font-semibold text-[15px] leading-5">
Agents
</div>
<h2 className="mt-4 font-semibold text-lg">{agentName}</h2>
<p className="mt-2 text-muted-foreground text-sm">
Send a message to start a focused conversation with this agent.
</p>
</div>
</div>
)
}
function getConversationStatusCopy(
status: string | undefined,
streaming: boolean,
): string {
if (streaming) return 'Working on your request'
if (status === 'running') return 'Ready for the next task'
if (status === 'starting') return 'Connecting to OpenClaw'
if (status === 'error') return 'OpenClaw needs attention'
if (status === 'stopped') return 'OpenClaw is offline'
return 'Open agent setup to continue'
function AgentRailList({
activeAgentId,
agents,
onSelectAgent,
}: {
activeAgentId: string
agents: AgentEntry[]
onSelectAgent: (entry: AgentEntry) => void
}) {
return (
<aside className="hidden min-h-0 flex-col border-border/50 border-r bg-background/70 lg:flex">
<div className="styled-scrollbar min-h-0 flex-1 space-y-2 overflow-y-auto px-3 py-3">
{agents.map((entry) => {
const active = entry.agentId === activeAgentId
const modelName = getAgentEntryMeta(entry)
return (
<button
key={entry.agentId}
type="button"
onClick={() => onSelectAgent(entry)}
className={cn(
'w-full rounded-2xl border px-3 py-3 text-left transition-all',
active
? 'border-[var(--accent-orange)]/30 bg-[var(--accent-orange)]/8 shadow-sm'
: 'border-transparent bg-transparent hover:border-border/60 hover:bg-card',
)}
>
<div className="flex items-center gap-3">
<div
className={cn(
'flex size-9 items-center justify-center rounded-xl',
active
? 'bg-[var(--accent-orange)]/12 text-[var(--accent-orange)]'
: 'bg-muted text-muted-foreground',
)}
>
<Bot className="size-4" />
</div>
<AgentIdentity name={entry.name} meta={modelName} />
</div>
</button>
)
})}
</div>
</aside>
)
}
export const AgentCommandConversation: FC = () => {
const { agentId } = useParams<{ agentId: string }>()
const [searchParams, setSearchParams] = useSearchParams()
function getAgentEntryMeta(agent: AgentEntry | undefined): string {
if (agent?.source === 'agent-harness') {
return getModelDisplayName(agent.model) ?? 'ACP agent'
}
return getModelDisplayName(agent?.model) ?? 'OpenClaw agent'
}
function AgentConversationController({
agentId,
initialMessage,
onInitialMessageConsumed,
agents,
agentPathPrefix,
createAgentPath,
}: {
agentId: string
initialMessage: string | null
onInitialMessageConsumed: () => void
agents: AgentEntry[]
agentPathPrefix: string
createAgentPath: string
}) {
const navigate = useNavigate()
const scrollRef = useRef<HTMLDivElement>(null)
const initialQuerySent = useRef(false)
const { status, agents } = useAgentCommandData()
const shouldRedirectHome = !agentId
const resolvedAgentId = agentId ?? ''
const agent = agents.find((entry) => entry.agentId === resolvedAgentId)
const agentName = agent?.name || resolvedAgentId || 'Agent'
const { turns, streaming, loading, send, resetConversation } =
useAgentConversation(resolvedAgentId, agentName)
const lastTurn = turns[turns.length - 1]
const lastTurnPartCount = lastTurn?.parts.length ?? 0
const initialMessageSentRef = useRef<string | null>(null)
const onInitialMessageConsumedRef = useRef(onInitialMessageConsumed)
const agent = agents.find((entry) => entry.agentId === agentId)
const agentName = agent?.name || agentId || 'Agent'
// Routing is now harness-only. Every OpenClaw agent has a harness
// record post the gateway → harness backfill, so the chat panel
// always talks to /agents/<id>/chat. The legacy ClawChat surface
// was deleted with the /claw/agents/:id/chat server route.
const harnessHistoryQuery = useHarnessChatHistory(agentId, Boolean(agent))
const historyMessages = useMemo(
() =>
flattenHistoryPages(
harnessHistoryQuery.data ? [harnessHistoryQuery.data] : [],
),
[harnessHistoryQuery.data],
)
const chatHistory = useMemo(
() => buildChatHistoryFromClawMessages(historyMessages),
[historyMessages],
)
// Listing query feeds queue + active-turn state for this agent. We
// already poll it every 5s for the rail; reusing the same cache
// keeps cross-tab queue state in sync without a second poll.
const { harnessAgents } = useHarnessAgents()
const harnessAgent = harnessAgents.find((entry) => entry.id === agentId)
const queue = harnessAgent?.queue ?? []
const activeTurnId = harnessAgent?.activeTurnId ?? null
const { turns, streaming, send } = useAgentConversation(agentId, {
runtime: 'agent-harness',
sessionKey: null,
history: chatHistory,
activeTurnId,
onComplete: () => {
void harnessHistoryQuery.refetch()
},
onSessionKeyChange: () => {},
})
const enqueueMessage = useEnqueueHarnessMessage()
const removeQueuedMessage = useRemoveHarnessQueuedMessage()
const handleStop = () => {
void cancelHarnessTurn(agentId, {
turnId: activeTurnId ?? undefined,
reason: 'user pressed stop',
})
}
const visibleTurns = useMemo(
() => filterTurnsPersistedInHistory(turns, historyMessages),
[historyMessages, turns],
)
onInitialMessageConsumedRef.current = onInitialMessageConsumed
const disabled = !agent
const historyReady =
harnessHistoryQuery.isFetched || harnessHistoryQuery.isError
const initialMessageKey = initialMessage
? `${agentId}:${initialMessage}`
: null
const error = harnessHistoryQuery.error ?? null
const sendRef = useRef(send)
sendRef.current = send
useEffect(() => {
if (shouldRedirectHome) return
const query = searchParams.get('q')
if (query && !initialQuerySent.current && !loading) {
initialQuerySent.current = true
setSearchParams({}, { replace: true })
void send(query)
const query = initialMessage?.trim()
if (!initialMessageKey) {
initialMessageSentRef.current = null
return
}
}, [loading, searchParams, send, setSearchParams, shouldRedirectHome])
useEffect(() => {
if (
shouldRedirectHome ||
(turns.length === 0 && lastTurnPartCount === 0 && !streaming)
!query ||
initialMessageSentRef.current === initialMessageKey ||
disabled ||
!historyReady
) {
return
}
scrollRef.current?.scrollTo({
top: scrollRef.current.scrollHeight,
behavior: 'smooth',
})
}, [lastTurnPartCount, shouldRedirectHome, streaming, turns.length])
if (shouldRedirectHome) {
return <Navigate to="/home" replace />
}
initialMessageSentRef.current = initialMessageKey
onInitialMessageConsumedRef.current()
void sendRef.current({ text: query })
}, [disabled, historyReady, initialMessage, initialMessageKey])
const handleSelectAgent = (entry: AgentEntry) => {
navigate(`/home/agents/${entry.agentId}`)
navigate(`${agentPathPrefix}/${entry.agentId}`)
}
const statusCopy = getConversationStatusCopy(status?.status, streaming)
return (
<div className="absolute inset-0 overflow-hidden">
<div className="fade-in slide-in-from-bottom-5 mx-auto flex h-full w-full max-w-3xl animate-in flex-col gap-3 px-4 pt-4 pb-2 duration-300">
<ConversationHeader
agentName={agentName}
status={statusCopy}
onGoHome={() => navigate('/home')}
onReset={resetConversation}
/>
<div className="flex min-h-0 flex-col overflow-hidden">
<ClawChat
agentName={agentName}
historyMessages={historyMessages}
turns={visibleTurns}
streaming={streaming}
isInitialLoading={harnessHistoryQuery.isLoading}
error={error}
hasNextPage={false}
isFetchingNextPage={false}
onFetchNextPage={() => {}}
onRetry={() => {
void harnessHistoryQuery.refetch()
}}
/>
<main
ref={scrollRef}
className={cn(
'styled-scrollbar min-h-0 flex-1 overflow-y-auto overflow-x-hidden rounded-[1.5rem] border border-border/50 bg-card/85 px-5 py-5 shadow-sm',
'[&_[data-streamdown="code-block"]]:!max-w-full [&_[data-streamdown="table-wrapper"]]:!max-w-full [&_[data-streamdown="code-block"]]:overflow-x-auto [&_[data-streamdown="table-wrapper"]]:overflow-x-auto',
)}
>
{loading ? (
<div className="flex h-full items-center justify-center text-muted-foreground text-sm">
Loading conversation...
</div>
) : turns.length === 0 ? (
<EmptyConversationState agentName={agentName} />
) : (
<div className="w-full space-y-4">
{turns.map((turn, index) => (
<ConversationMessage
key={turn.id}
turn={turn}
streaming={streaming && index === turns.length - 1}
/>
))}
</div>
)}
</main>
<div className="w-full flex-shrink-0">
<div className="border-border/50 border-t bg-background/88 px-4 py-3 backdrop-blur-md">
<div className="mx-auto max-w-3xl space-y-3">
{queue.length > 0 ? (
<QueuePanel
queue={queue}
onRemove={(messageId) =>
removeQueuedMessage.mutate({ agentId, messageId })
}
/>
) : null}
<ConversationInput
variant="conversation"
agents={agents}
selectedAgentId={resolvedAgentId}
selectedAgentId={agentId}
onSelectAgent={handleSelectAgent}
onSend={(text) => {
void send(text)
onSend={(input) => {
const attachments = input.attachments.map((a) => a.payload)
const attachmentPreviews = input.attachments.map((a) => ({
id: a.id,
kind: a.kind,
mediaType: a.mediaType,
name: a.name,
dataUrl: a.dataUrl,
}))
// When the agent already has an in-flight turn, route
// the new message into the durable queue instead of
// starting a parallel turn. Drains automatically as
// soon as the active turn ends.
if (streaming || activeTurnId) {
enqueueMessage.mutate({
agentId,
message: input.text,
attachments,
})
return
}
void send({ text: input.text, attachments, attachmentPreviews })
}}
onCreateAgent={() => navigate('/agents')}
onCreateAgent={() => navigate(createAgentPath)}
onStop={handleStop}
streaming={streaming}
disabled={status?.status !== 'running'}
status={status?.status}
placeholder={`Message ${agentName}...`}
disabled={disabled}
status="running"
attachmentsEnabled={true}
placeholder={
streaming
? `Type to queue another message for ${agentName}...`
: `Message ${agentName}...`
}
/>
</div>
</div>
</div>
)
}
interface AgentCommandConversationProps {
variant?: 'command' | 'page'
backPath?: string
agentPathPrefix?: string
createAgentPath?: string
}
export const AgentCommandConversation: FC<AgentCommandConversationProps> = ({
variant = 'command',
backPath = '/home',
agentPathPrefix = '/home/agents',
createAgentPath = '/agents',
}) => {
const { agentId } = useParams<{ agentId: string }>()
const [searchParams, setSearchParams] = useSearchParams()
const navigate = useNavigate()
const { agents } = useAgentCommandData()
const shouldRedirectHome = !agentId
const resolvedAgentId = agentId ?? ''
const agent = agents.find((entry) => entry.agentId === resolvedAgentId)
const agentName = agent?.name || resolvedAgentId || 'Agent'
const agentMeta = getAgentEntryMeta(agent)
const initialMessage = searchParams.get('q')
const isPageVariant = variant === 'page'
const backLabel = isPageVariant ? 'Back to agents' : 'Back to home'
if (shouldRedirectHome) {
return <Navigate to="/home" replace />
}
const handleSelectAgent = (entry: AgentEntry) => {
navigate(`${agentPathPrefix}/${entry.agentId}`)
}
// Every visible agent runs through the harness now, so per-agent
// runtime status doesn't gate chat the way OpenClaw's legacy
// gateway lifecycle did. Show "Ready" once the agent record is
// resolved from the rail, "Setup" otherwise.
const statusCopy = agent ? 'Ready' : 'Setup'
return (
<div className="absolute inset-0 overflow-hidden bg-background md:pl-[theme(spacing.14)]">
<div className="mx-auto grid h-full w-full max-w-[1480px] lg:grid-cols-[288px_minmax(0,1fr)] lg:grid-rows-[3.5rem_minmax(0,1fr)]">
<AgentRailHeader onGoHome={() => navigate(backPath)} />
<ConversationHeader
agentName={agentName}
agentMeta={agentMeta}
status={statusCopy}
backLabel={backLabel}
backTarget={isPageVariant ? 'page' : 'home'}
onGoHome={() => navigate(backPath)}
/>
<AgentRailList
activeAgentId={resolvedAgentId}
agents={agents}
onSelectAgent={handleSelectAgent}
/>
<AgentConversationController
key={resolvedAgentId}
agentId={resolvedAgentId}
agents={agents}
initialMessage={initialMessage}
onInitialMessageConsumed={() =>
setSearchParams({}, { replace: true })
}
agentPathPrefix={agentPathPrefix}
createAgentPath={createAgentPath}
/>
</div>
</div>
)
}

View File

@@ -1,178 +1,195 @@
import { ArrowRight } from 'lucide-react'
import { type FC, useEffect, useState } from 'react'
import { Plus } from 'lucide-react'
import { type FC, useEffect, useMemo, useState } from 'react'
import { useNavigate } from 'react-router'
import { Button } from '@/components/ui/button'
import { Card, CardContent } from '@/components/ui/card'
import { Separator } from '@/components/ui/separator'
import type {
HarnessAdapterDescriptor,
HarnessAgent,
} from '@/entrypoints/app/agents/agent-harness-types'
import {
useAgentAdapters,
useHarnessAgents,
} from '@/entrypoints/app/agents/useAgents'
import type { AgentEntry } from '@/entrypoints/app/agents/useOpenClaw'
import { ImportDataHint } from '@/entrypoints/newtab/index/ImportDataHint'
import { NewTabBranding } from '@/entrypoints/newtab/index/NewTabBranding'
import { NewTabTip } from '@/entrypoints/newtab/index/NewTabTip'
import { ScheduleResults } from '@/entrypoints/newtab/index/ScheduleResults'
import { SignInHint } from '@/entrypoints/newtab/index/SignInHint'
import { TopSites } from '@/entrypoints/newtab/index/TopSites'
import { useActiveHint } from '@/entrypoints/newtab/index/useActiveHint'
import { AgentCardDock } from './AgentCardDock'
import { useAgentCommandData } from './agent-command-layout'
import { ConversationInput } from './ConversationInput'
import { useAgentCardData } from './useAgentCardData'
function AgentCommandSetupState({
onOpenAgents,
}: {
onOpenAgents: () => void
}) {
return (
<Card className="border-border/60 bg-card/85 shadow-sm">
<CardContent className="flex flex-col items-center gap-4 p-6 text-center">
<p className="max-w-xl text-muted-foreground text-sm">
Set up OpenClaw agents to turn your new tab into an agent command
center.
</p>
<Button onClick={onOpenAgents} className="gap-2">
Open Agent Setup
<ArrowRight className="size-4" />
</Button>
</CardContent>
</Card>
)
}
import { orderHomeAgents } from './home-agent-card.helpers'
function EmptyAgentsState({ onOpenAgents }: { onOpenAgents: () => void }) {
return (
<Card className="border-border/60 bg-card/85 shadow-sm">
<CardContent className="flex flex-col items-center gap-4 p-6 text-center">
<p className="max-w-xl text-muted-foreground text-sm">
OpenClaw is running, but you do not have any agents yet.
</p>
<Button variant="outline" onClick={onOpenAgents}>
Create your first agent
<Card className="border-border/60 bg-card/90 shadow-sm">
<CardContent className="flex flex-col items-center gap-4 p-8 text-center">
<div className="flex size-12 items-center justify-center rounded-2xl bg-muted text-muted-foreground">
<Plus className="size-5" />
</div>
<div className="space-y-2">
<h2 className="font-semibold text-lg">No agents yet</h2>
<p className="max-w-md text-muted-foreground text-sm leading-6">
Create an agent to start using BrowserOS as an agent-first new tab.
</p>
</div>
<Button variant="outline" onClick={onOpenAgents} className="rounded-xl">
Create agent
</Button>
</CardContent>
</Card>
)
}
function OpenClawUnavailableState({
function RecentThreads({
activeAgentId,
agents,
adapters,
onOpenAgents,
onSelectAgent,
}: {
activeAgentId?: string | null
agents: HarnessAgent[]
adapters: HarnessAdapterDescriptor[]
onOpenAgents: () => void
onSelectAgent: (agentId: string) => void
}) {
if (agents.length === 0) return null
return (
<Card className="border-border/60 bg-card/85 shadow-sm">
<CardContent className="flex flex-col items-center gap-4 p-6 text-center">
<p className="max-w-xl text-muted-foreground text-sm">
OpenClaw is unavailable right now. Open the Agents page to restart the
gateway or review setup.
</p>
<Button onClick={onOpenAgents} className="gap-2">
Open Agent Setup
<ArrowRight className="size-4" />
<section className="space-y-4">
<div className="flex items-center justify-between gap-4">
<div>
<h2 className="font-semibold text-base">Recent agents</h2>
<p className="text-muted-foreground text-sm">
Continue from where you left off.
</p>
</div>
<Button
variant="outline"
onClick={onOpenAgents}
className="rounded-xl"
size="sm"
>
Manage agents
</Button>
</CardContent>
</Card>
</div>
<AgentCardDock
agents={agents}
adapters={adapters}
activeAgentId={activeAgentId ?? undefined}
onSelectAgent={onSelectAgent}
onCreateAgent={onOpenAgents}
/>
</section>
)
}
export const AgentCommandHome: FC = () => {
const navigate = useNavigate()
const activeHint = useActiveHint()
const { status, agents } = useAgentCommandData()
const [mounted, setMounted] = useState(false)
// The conversation input still consumes the merged AgentEntry list
// from the layout context (handles legacy /claw/agents entries that
// haven't yet been backfilled into the harness store). The Recent
// Agents grid below reads the richer harness payload directly.
const { agents: legacyAgents, status } = useAgentCommandData()
const { harnessAgents } = useHarnessAgents()
const { adapters } = useAgentAdapters()
const [selectedAgentId, setSelectedAgentId] = useState<string | null>(null)
const cardData = useAgentCardData(agents, status?.status)
const orderedAgents = useMemo(
() => orderHomeAgents(harnessAgents),
[harnessAgents],
)
useEffect(() => {
setMounted(true)
}, [])
useEffect(() => {
if (agents.length === 0) {
if (selectedAgentId) {
setSelectedAgentId(null)
}
if (legacyAgents.length === 0) {
if (selectedAgentId) setSelectedAgentId(null)
return
}
if (
!selectedAgentId ||
!agents.some((agent) => agent.agentId === selectedAgentId)
!legacyAgents.some((agent) => agent.agentId === selectedAgentId)
) {
setSelectedAgentId(agents[0].agentId)
setSelectedAgentId(legacyAgents[0].agentId)
}
}, [agents, selectedAgentId])
}, [legacyAgents, selectedAgentId])
const handleSend = (text: string) => {
const handleSend = (input: { text: string }) => {
if (!selectedAgentId) return
navigate(`/home/agents/${selectedAgentId}?q=${encodeURIComponent(text)}`)
navigate(
`/home/agents/${selectedAgentId}?q=${encodeURIComponent(input.text)}`,
)
}
const handleSelectAgent = (agent: AgentEntry) => {
setSelectedAgentId(agent.agentId)
}
const openClawStatus = status?.status
const isSetup = openClawStatus != null && openClawStatus !== 'uninitialized'
const shouldShowUnavailableState =
openClawStatus != null &&
openClawStatus !== 'running' &&
openClawStatus !== 'uninitialized' &&
cardData.length === 0
const selectedAgent = legacyAgents.find(
(agent) => agent.agentId === selectedAgentId,
)
const selectedAgentReady = selectedAgent
? selectedAgent.source === 'agent-harness' || status?.status === 'running'
: false
const selectedAgentStatus =
selectedAgent?.source === 'agent-harness' ? 'running' : status?.status
const selectedAgentName =
selectedAgent?.name ?? orderedAgents[0]?.name ?? 'your agent'
const hasAgents = legacyAgents.length > 0
return (
<div className="pt-[max(25vh,16px)]">
<div className="relative w-full space-y-8 md:w-3xl">
<NewTabBranding />
<ConversationInput
variant="home"
agents={agents}
selectedAgentId={selectedAgentId}
onSelectAgent={handleSelectAgent}
onSend={handleSend}
onCreateAgent={() => navigate('/agents')}
streaming={false}
disabled={status?.status !== 'running'}
status={status?.status}
placeholder={
status?.status === 'running'
? undefined
: 'OpenClaw is not running...'
}
/>
{mounted ? <NewTabTip /> : null}
{isSetup ? (
shouldShowUnavailableState ? (
<OpenClawUnavailableState
onOpenAgents={() => navigate('/agents')}
/>
) : cardData.length > 0 ? (
<section className="space-y-3">
<div className="flex items-center justify-between">
<div>
<h2 className="font-semibold text-base">Agents</h2>
<p className="text-muted-foreground text-sm">
Pick up where your agents left off.
</p>
</div>
<div className="min-h-full px-4 py-6">
<div className="mx-auto flex w-full max-w-5xl flex-col gap-8">
{hasAgents ? (
<>
<div className="flex flex-col items-center gap-5 pt-[max(10vh,24px)] text-center">
<div className="space-y-3">
<h1 className="font-semibold text-[clamp(2rem,4vw,3.25rem)] leading-tight tracking-tight">
What should your agent work on next?
</h1>
<p className="mx-auto max-w-2xl text-muted-foreground text-sm leading-6">
Start with a task, continue a thread, or switch to another
agent without leaving the new tab.
</p>
</div>
<AgentCardDock
agents={cardData}
activeAgentId={selectedAgentId ?? undefined}
onSelectAgent={(agentId) => navigate(`/home/agents/${agentId}`)}
onCreateAgent={() => navigate('/agents')}
/>
</section>
) : (
<EmptyAgentsState onOpenAgents={() => navigate('/agents')} />
)
) : (
<AgentCommandSetupState onOpenAgents={() => navigate('/agents')} />
)}
{mounted ? <TopSites /> : null}
{mounted ? <ScheduleResults /> : null}
<div className="w-full max-w-3xl">
<ConversationInput
variant="home"
agents={legacyAgents}
selectedAgentId={selectedAgentId}
onSelectAgent={handleSelectAgent}
onSend={handleSend}
onCreateAgent={() => navigate('/agents')}
streaming={false}
disabled={!selectedAgentReady}
status={selectedAgentStatus}
attachmentsEnabled={false}
placeholder={
selectedAgentReady
? `Ask ${selectedAgentName} to handle a task...`
: 'Agent runtime is not running...'
}
/>
</div>
</div>
<Separator />
<RecentThreads
activeAgentId={selectedAgentId}
agents={orderedAgents}
adapters={adapters}
onOpenAgents={() => navigate('/agents')}
onSelectAgent={(agentId) => navigate(`/home/agents/${agentId}`)}
/>
</>
) : (
<EmptyAgentsState onOpenAgents={() => navigate('/agents')} />
)}
</div>
{activeHint === 'signin' ? <SignInHint /> : null}

View File

@@ -0,0 +1,172 @@
import { Bot, Loader2, RefreshCw } from 'lucide-react'
import { type FC, useEffect, useRef } from 'react'
import {
Conversation,
ConversationContent,
ConversationScrollButton,
} from '@/components/ai-elements/conversation'
import type { AgentConversationTurn } from '@/lib/agent-conversations/types'
import { cn } from '@/lib/utils'
import { ClawChatMessage } from './ClawChatMessage'
import { ConversationMessage } from './ConversationMessage'
import type { ClawChatMessage as ClawChatMessageModel } from './claw-chat-types'
interface ClawChatProps {
agentName: string
historyMessages: ClawChatMessageModel[]
turns: AgentConversationTurn[]
streaming: boolean
isInitialLoading: boolean
error: Error | null
hasNextPage: boolean
isFetchingNextPage: boolean
onFetchNextPage: () => void
onRetry: () => void
className?: string
}
function EmptyConversationState({ agentName }: { agentName: string }) {
return (
<div className="flex h-full items-center justify-center px-6 py-12">
<div className="max-w-md text-center">
<div className="mx-auto flex size-14 items-center justify-center rounded-3xl bg-muted text-muted-foreground">
<Bot className="size-6" />
</div>
<h2 className="mt-5 font-semibold text-xl">{agentName}</h2>
<p className="mt-2 text-muted-foreground text-sm leading-6">
Ask {agentName} to start a task.
</p>
</div>
</div>
)
}
function LoadingConversationState() {
return (
<div className="flex h-full items-center justify-center gap-2 text-muted-foreground text-sm">
<Loader2 className="size-4 animate-spin" />
Loading conversation...
</div>
)
}
function ConversationErrorState({
message,
onRetry,
}: {
message: string
onRetry: () => void
}) {
return (
<div className="flex h-full items-center justify-center px-6 py-12">
<div className="max-w-md rounded-2xl border border-border/60 bg-card px-5 py-4 text-center shadow-sm">
<p className="text-sm">{message}</p>
<button
type="button"
onClick={onRetry}
className="mt-3 inline-flex items-center gap-2 rounded-lg border border-border/60 px-3 py-1.5 font-medium text-muted-foreground text-xs transition-colors hover:bg-accent hover:text-foreground"
>
<RefreshCw className="size-3.5" />
Retry
</button>
</div>
</div>
)
}
export const ClawChat: FC<ClawChatProps> = ({
agentName,
historyMessages,
turns,
streaming,
isInitialLoading,
error,
hasNextPage,
isFetchingNextPage,
onFetchNextPage,
onRetry,
className,
}) => {
const topSentinelRef = useRef<HTMLDivElement>(null)
const onFetchNextPageRef = useRef(onFetchNextPage)
onFetchNextPageRef.current = onFetchNextPage
const hasMessages = historyMessages.length > 0 || turns.length > 0
useEffect(() => {
const sentinel = topSentinelRef.current
if (!sentinel) return
const observer = new IntersectionObserver(
(entries) => {
const [entry] = entries
if (!entry?.isIntersecting || !hasNextPage || isFetchingNextPage) {
return
}
onFetchNextPageRef.current()
},
{
root: null,
rootMargin: '160px 0px 0px 0px',
threshold: 0,
},
)
observer.observe(sentinel)
return () => observer.disconnect()
}, [hasNextPage, isFetchingNextPage])
return (
<div
className={cn('flex min-h-0 flex-1 flex-col overflow-hidden', className)}
>
<Conversation
className={cn(
'bg-background',
'[&_[data-streamdown="code-block"]]:!w-full [&_[data-streamdown="code-block"]]:!max-w-full [&_[data-streamdown="table-wrapper"]]:!w-full [&_[data-streamdown="table-wrapper"]]:!max-w-full [&_[data-streamdown="code-block"]]:overflow-x-auto [&_[data-streamdown="table-wrapper"]]:overflow-x-auto',
)}
>
<ConversationContent className="min-h-full px-5 py-5">
{isInitialLoading ? (
<LoadingConversationState />
) : error && !hasMessages ? (
<ConversationErrorState message={error.message} onRetry={onRetry} />
) : !hasMessages ? (
<EmptyConversationState agentName={agentName} />
) : (
<div className="mx-auto flex w-full max-w-3xl flex-col gap-3">
<div ref={topSentinelRef} aria-hidden="true" className="h-px" />
{isFetchingNextPage ? (
<div className="flex justify-center py-2 text-muted-foreground text-xs">
<Loader2 className="mr-2 size-3.5 animate-spin" />
Loading older messages...
</div>
) : null}
{!hasNextPage && historyMessages.length > 0 ? (
<div className="py-1 text-center text-muted-foreground text-xs">
Start of conversation
</div>
) : null}
{historyMessages.map((message) => (
<ClawChatMessage key={message.id} message={message} />
))}
{turns.map((turn, index) => (
<ConversationMessage
key={turn.id}
turn={turn}
streaming={streaming && index === turns.length - 1}
/>
))}
{error ? (
<div className="rounded-xl border border-border/60 bg-card px-4 py-3 text-muted-foreground text-sm">
{error.message}
</div>
) : null}
</div>
)}
</ConversationContent>
<ConversationScrollButton />
</Conversation>
</div>
)
}

View File

@@ -0,0 +1,248 @@
import { CheckCircle2, Copy, Loader2, Wrench, XCircle } from 'lucide-react'
import { type FC, useCallback, useMemo } from 'react'
import {
Message,
MessageAction,
MessageActions,
MessageAttachment,
MessageAttachments,
MessageContent,
MessageResponse,
MessageToolbar,
} from '@/components/ai-elements/message'
import {
Reasoning,
ReasoningContent,
ReasoningTrigger,
} from '@/components/ai-elements/reasoning'
import {
Task,
TaskContent,
TaskItem,
TaskTrigger,
} from '@/components/ai-elements/task'
import { cn } from '@/lib/utils'
import type {
ClawChatMessagePart,
ClawChatMessage as ClawChatMessageType,
} from './claw-chat-types'
function formatCost(usd: number): string {
if (usd < 0.005) return `$${usd.toFixed(4)}`
return `$${usd.toFixed(2)}`
}
type ToolCallPart = Extract<ClawChatMessagePart, { type: 'tool-call' }>
type AttachmentPart = Extract<ClawChatMessagePart, { type: 'attachment' }>
interface RenderEntry {
kind: 'text' | 'reasoning' | 'meta' | 'task' | 'attachments'
partIndex: number
part?: ClawChatMessagePart
tools?: ToolCallPart[]
attachments?: AttachmentPart[]
}
/**
* Build a render plan that groups all tool-call parts into a single Task
* collapsible and all attachment parts into a single attachment strip at
* their respective first-appearance positions. Other parts render in place.
*/
function buildRenderEntries(parts: ClawChatMessagePart[]): RenderEntry[] {
const entries: RenderEntry[] = []
const tools: ToolCallPart[] = []
const attachments: AttachmentPart[] = []
let taskInserted = false
let attachmentsInserted = false
parts.forEach((part, partIndex) => {
if (part.type === 'tool-call') {
tools.push(part)
if (!taskInserted) {
entries.push({ kind: 'task', partIndex, tools })
taskInserted = true
}
} else if (part.type === 'attachment') {
attachments.push(part)
if (!attachmentsInserted) {
entries.push({ kind: 'attachments', partIndex, attachments })
attachmentsInserted = true
}
} else if (part.type === 'text') {
entries.push({ kind: 'text', partIndex, part })
} else if (part.type === 'reasoning') {
entries.push({ kind: 'reasoning', partIndex, part })
} else if (part.type === 'meta') {
entries.push({ kind: 'meta', partIndex, part })
}
})
return entries
}
function ToolStatusIcon({ status }: { status: ToolCallPart['status'] }) {
if (status === 'running' || status === 'pending') {
return (
<Loader2 className="size-3.5 shrink-0 animate-spin text-muted-foreground" />
)
}
if (status === 'completed') {
return <CheckCircle2 className="size-3.5 shrink-0 text-green-500" />
}
return <XCircle className="size-3.5 shrink-0 text-destructive" />
}
interface ClawChatMessageProps {
message: ClawChatMessageType
}
export const ClawChatMessage: FC<ClawChatMessageProps> = ({ message }) => {
const messageText = message.parts
.filter((p) => p.type === 'text')
.map((p) => p.text)
.join('\n')
const handleCopy = useCallback(() => {
if (messageText) navigator.clipboard.writeText(messageText)
}, [messageText])
const entries = useMemo(
() => buildRenderEntries(message.parts),
[message.parts],
)
return (
<Message
from={message.role}
className="max-w-full group-[.is-user]:max-w-[80%]"
>
<MessageContent className="max-w-full overflow-hidden group-[.is-assistant]:w-full group-[.is-user]:max-w-full">
{entries.map((entry) => {
const key = `${message.id}-entry-${entry.partIndex}`
if (entry.kind === 'attachments' && entry.attachments) {
return (
<MessageAttachments key={key}>
{entry.attachments.map((attachment, idx) => (
<MessageAttachment
// biome-ignore lint/suspicious/noArrayIndexKey: attachment order is stable within a finalized message
key={`${attachment.kind}-${idx}`}
data={{
type: 'file',
url: attachment.dataUrl ?? '',
mediaType: attachment.mediaType,
filename: attachment.name,
}}
/>
))}
</MessageAttachments>
)
}
if (entry.kind === 'text' && entry.part?.type === 'text') {
return (
<MessageResponse
key={key}
// Historical messages are finalized — render immediately.
// Streamdown's default "streaming" mode uses an idle-callback
// debounce (300ms / 500ms idle) that paints empty content
// first, which made history flash blank tool collapsibles
// before text on every load.
mode="static"
parseIncompleteMarkdown={false}
className={cn(
'max-w-full overflow-hidden break-words',
'[&_[data-streamdown="code-block"]]:!w-full [&_[data-streamdown="code-block"]]:!max-w-full [&_[data-streamdown="code-block"]]:overflow-x-auto',
'[&_[data-streamdown="table-wrapper"]]:!w-full [&_[data-streamdown="table-wrapper"]]:!max-w-full [&_[data-streamdown="table-wrapper"]]:overflow-x-auto',
'[&_table]:w-max [&_table]:min-w-full',
)}
>
{entry.part.text}
</MessageResponse>
)
}
if (entry.kind === 'reasoning' && entry.part?.type === 'reasoning') {
return (
<Reasoning
key={key}
className="w-full"
defaultOpen={false}
duration={entry.part.duration}
>
<ReasoningTrigger />
<ReasoningContent>{entry.part.text}</ReasoningContent>
</Reasoning>
)
}
if (entry.kind === 'meta' && entry.part?.type === 'meta') {
return (
<div key={key} className="text-muted-foreground text-xs">
{entry.part.label}: {entry.part.value}
</div>
)
}
if (entry.kind === 'task' && entry.tools) {
const tools = entry.tools
const errorCount = tools.filter((t) => t.status === 'failed').length
const taskTitle = `Agent activity (${tools.length} ${tools.length === 1 ? 'action' : 'actions'}${errorCount > 0 ? `, ${errorCount} failed` : ''})`
return (
<Task key={key} defaultOpen={false}>
<TaskTrigger title={taskTitle} TriggerIcon={Wrench} />
<TaskContent>
{tools.map((tool, idx) => (
<TaskItem
// biome-ignore lint/suspicious/noArrayIndexKey: tool order is stable within a finalized historical message
key={`${tool.name}-${tool.status}-${idx}`}
className="flex items-center gap-2"
>
<ToolStatusIcon status={tool.status} />
<span className="text-foreground text-xs">
{tool.label}
</span>
{tool.subject ? (
<span className="ml-1.5 truncate text-muted-foreground/70 text-xs">
· {tool.subject}
</span>
) : null}
{tool.error ? (
<span className="ml-2 truncate text-destructive text-xs">
{tool.error}
</span>
) : null}
{tool.durationMs != null ? (
<span className="ml-auto text-muted-foreground/60 text-xs tabular-nums">
{(tool.durationMs / 1000).toFixed(1)}s
</span>
) : null}
</TaskItem>
))}
</TaskContent>
</Task>
)
}
return null
})}
{message.role === 'assistant' && messageText ? (
<MessageToolbar>
<MessageActions>
<MessageAction tooltip="Copy" onClick={handleCopy}>
<Copy className="size-3.5" />
</MessageAction>
</MessageActions>
{message.costUsd ? (
<span className="text-[11px] text-muted-foreground/50 tabular-nums">
{formatCost(message.costUsd)}
</span>
) : null}
</MessageToolbar>
) : null}
</MessageContent>
</Message>
)
}

View File

@@ -2,20 +2,33 @@ import {
ArrowRight,
Bot,
ChevronDown,
FileText,
Folder,
Layers,
Loader2,
Mic,
Paperclip,
Square,
X,
} from 'lucide-react'
import { type FC, type ReactNode, useEffect, useState } from 'react'
import {
type DragEvent,
type FC,
type ReactNode,
useEffect,
useLayoutEffect,
useRef,
useState,
} from 'react'
import { AppSelector } from '@/components/elements/AppSelector'
import { TabPickerPopover } from '@/components/elements/tab-picker-popover'
import { WorkspaceSelector } from '@/components/elements/workspace-selector'
import { Button } from '@/components/ui/button'
import { Textarea } from '@/components/ui/textarea'
import type { AgentEntry } from '@/entrypoints/app/agents/useOpenClaw'
import { McpServerIcon } from '@/entrypoints/app/connect-mcp/McpServerIcon'
import { useGetUserMCPIntegrations } from '@/entrypoints/app/connect-mcp/useGetUserMCPIntegrations'
import { type StagedAttachment, stageAttachments } from '@/lib/attachments'
import { Feature } from '@/lib/browseros/capabilities'
import { useCapabilities } from '@/lib/browseros/useCapabilities'
import { useMcpServers } from '@/lib/mcp/mcpServerStorage'
@@ -24,36 +37,57 @@ import { useVoiceInput } from '@/lib/voice/useVoiceInput'
import { useWorkspace } from '@/lib/workspace/use-workspace'
import { AgentSelector } from './AgentSelector'
export interface ConversationInputSendInput {
text: string
attachments: StagedAttachment[]
}
interface ConversationInputProps {
agents: AgentEntry[]
selectedAgentId: string | null
onSelectAgent: (agent: AgentEntry) => void
onSend: (text: string) => void
onSend: (input: ConversationInputSendInput) => void
onCreateAgent?: () => void
streaming: boolean
disabled?: boolean
status?: string
placeholder?: string
attachmentsEnabled?: boolean
variant?: 'home' | 'conversation'
/**
* When set, a Stop button surfaces to the left of the voice mic
* while `streaming === true`. Click cancels the active turn
* server-side via the chat-cancel endpoint. Absent → no Stop
* button (legacy behaviour for the home composer).
*/
onStop?: () => void
}
function InputActionButton({
disabled,
onClick,
streaming,
hasContent,
}: {
disabled: boolean
onClick: () => void
streaming: boolean
hasContent: boolean
}) {
// Show the spinner while streaming only when there's nothing to
// send — once the user types something, the icon flips back to the
// paper-plane so it reads as "queue this message" instead of
// "still working".
const showSpinner = streaming && !hasContent
return (
<Button
onClick={onClick}
size="icon"
disabled={disabled}
title={streaming && hasContent ? 'Queue message' : undefined}
className="h-10 w-10 flex-shrink-0 rounded-xl bg-primary text-primary-foreground hover:bg-primary/90"
>
{streaming ? (
{showSpinner ? (
<Loader2 className="h-5 w-5 animate-spin" />
) : (
<ArrowRight className="h-5 w-5" />
@@ -62,6 +96,22 @@ function InputActionButton({
)
}
function StopButton({ onStop }: { onStop: () => void }) {
return (
<Button
type="button"
size="icon"
variant="ghost"
onClick={onStop}
title="Stop current turn — queued messages will start next."
aria-label="Stop current turn"
className="h-8 w-8 flex-shrink-0 rounded-lg bg-destructive/10 text-destructive transition-colors hover:bg-destructive/15 hover:text-destructive"
>
<Square className="h-3.5 w-3.5 fill-current" />
</Button>
)
}
function VoiceButton({
isRecording,
isTranscribing,
@@ -123,6 +173,9 @@ function ContextControls({
onToggleTab,
showAgentSelector,
status,
onAttachClick,
attachDisabled,
attachmentsEnabled,
}: {
agents: AgentEntry[]
onCreateAgent?: () => void
@@ -132,6 +185,9 @@ function ContextControls({
onToggleTab: (tab: chrome.tabs.Tab) => void
showAgentSelector: boolean
status?: string
onAttachClick: () => void
attachDisabled: boolean
attachmentsEnabled: boolean
}) {
const { supports } = useCapabilities()
const { selectedFolder } = useWorkspace()
@@ -146,7 +202,7 @@ function ContextControls({
})
return (
<div className="flex items-center justify-between border-border/50 border-t px-5 py-3">
<div className="flex items-center justify-between border-border/40 border-t px-4 py-2.5">
<div className="flex items-center gap-1">
{showAgentSelector ? (
<AgentSelector
@@ -191,6 +247,20 @@ function ContextControls({
<span>Tabs</span>
</Button>
</TabPickerPopover>
<Button
type="button"
variant="ghost"
onClick={onAttachClick}
disabled={attachDisabled || !attachmentsEnabled}
title="Attach files"
className={cn(
'flex items-center gap-2 rounded-lg px-3 py-1.5 font-medium text-sm transition-all',
'bg-transparent text-muted-foreground hover:bg-accent hover:text-accent-foreground',
)}
>
<Paperclip className="h-4 w-4" />
<span>Attach</span>
</Button>
</div>
{supports(Feature.MANAGED_MCP_SUPPORT) ? (
@@ -234,7 +304,7 @@ function ContextControls({
function HomeShell({ children }: { children: ReactNode }) {
return (
<div className="overflow-hidden rounded-[1.5rem] border border-border/60 bg-card/95 shadow-sm backdrop-blur">
<div className="overflow-hidden rounded-[1.55rem] border border-border/60 bg-card/95 shadow-sm">
{children}
</div>
)
@@ -242,7 +312,7 @@ function HomeShell({ children }: { children: ReactNode }) {
function ConversationShell({ children }: { children: ReactNode }) {
return (
<div className="overflow-hidden rounded-[1.5rem] border border-border/60 bg-card/95 shadow-sm backdrop-blur">
<div className="overflow-hidden rounded-[1.35rem] border border-border/50 bg-background/95 shadow-[0_10px_30px_rgba(15,23,42,0.06)] backdrop-blur-md">
{children}
</div>
)
@@ -258,14 +328,64 @@ export const ConversationInput: FC<ConversationInputProps> = ({
disabled,
status,
placeholder,
attachmentsEnabled = true,
variant = 'conversation',
onStop,
}) => {
const [input, setInput] = useState('')
const [selectedTabs, setSelectedTabs] = useState<chrome.tabs.Tab[]>([])
const [isExpandedDraft, setIsExpandedDraft] = useState(false)
const [attachments, setAttachments] = useState<StagedAttachment[]>([])
const [attachmentError, setAttachmentError] = useState<string | null>(null)
const [isStaging, setIsStaging] = useState(false)
const [isDragOver, setIsDragOver] = useState(false)
const fileInputRef = useRef<HTMLInputElement>(null)
const voice = useVoiceInput()
const textareaRef = useRef<HTMLTextAreaElement>(null)
const selectedAgent = agents.find(
(agent) => agent.agentId === selectedAgentId,
)
const isConversation = variant === 'conversation'
const stageFiles = async (files: File[]) => {
if (files.length === 0) return
if (!attachmentsEnabled) {
setAttachmentError('Attachments are not supported for this agent yet.')
return
}
setIsStaging(true)
setAttachmentError(null)
try {
const result = await stageAttachments(files, attachments.length)
if (result.staged.length > 0) {
setAttachments((prev) => [...prev, ...result.staged])
}
if (result.errors.length > 0) {
setAttachmentError(result.errors.map((e) => e.message).join(' \u2022 '))
}
} finally {
setIsStaging(false)
}
}
const removeAttachment = (id: string) => {
setAttachments((prev) => prev.filter((a) => a.id !== id))
setAttachmentError(null)
}
useLayoutEffect(() => {
const element = textareaRef.current
if (!element) return
const maxHeight = isConversation ? 176 : 100
const collapsedHeight = isConversation ? 56 : 72
element.style.height = '0px'
const nextHeight = Math.min(element.scrollHeight, maxHeight)
element.style.height = `${nextHeight}px`
element.style.overflowY =
element.scrollHeight > maxHeight ? 'auto' : 'hidden'
setIsExpandedDraft(nextHeight > collapsedHeight)
})
useEffect(() => {
if (voice.transcript && !voice.isTranscribing) {
@@ -274,6 +394,12 @@ export const ConversationInput: FC<ConversationInputProps> = ({
}
}, [voice.transcript, voice.isTranscribing, voice])
useEffect(() => {
if (attachmentsEnabled) return
setAttachments([])
setAttachmentError(null)
}, [attachmentsEnabled])
const toggleTab = (tab: chrome.tabs.Tab) => {
setSelectedTabs((prev) => {
const isSelected = prev.some((selected) => selected.id === tab.id)
@@ -284,11 +410,77 @@ export const ConversationInput: FC<ConversationInputProps> = ({
})
}
const hasContent = input.trim().length > 0 || attachments.length > 0
// Queue-aware composers (the conversation panel passes `onStop`)
// accept input while streaming — the parent decides whether the
// submission opens a new turn or enqueues onto the active one.
// Surfaces without a Stop hook (home) keep the legacy behaviour
// and block input until the current turn finishes.
const queueAware = Boolean(onStop)
const handleSend = () => {
const text = input.trim()
if (!text || streaming || disabled) return
onSend(text)
if (disabled || isStaging) return
if (streaming && !queueAware) return
if (!text && attachments.length === 0) return
onSend({ text, attachments })
setInput('')
setAttachments([])
setAttachmentError(null)
}
const handlePaste = (event: React.ClipboardEvent<HTMLTextAreaElement>) => {
const items = event.clipboardData?.items
if (!items) return
const files: File[] = []
for (const item of items) {
if (item.kind === 'file') {
const file = item.getAsFile()
if (file) files.push(file)
}
}
if (files.length > 0) {
event.preventDefault()
void stageFiles(files)
}
}
const handleDrop = (event: DragEvent<HTMLDivElement>) => {
event.preventDefault()
setIsDragOver(false)
const files = Array.from(event.dataTransfer?.files ?? [])
if (files.length > 0) {
void stageFiles(files)
}
}
const handleDragOver = (event: DragEvent<HTMLDivElement>) => {
if (!event.dataTransfer?.types.includes('Files')) return
event.preventDefault()
setIsDragOver(true)
}
const handleDragLeave = (event: DragEvent<HTMLDivElement>) => {
if (event.currentTarget.contains(event.relatedTarget as Node | null)) {
return
}
setIsDragOver(false)
}
const openFilePicker = () => {
if (!attachmentsEnabled) {
setAttachmentError('Attachments are not supported for this agent yet.')
return
}
fileInputRef.current?.click()
}
const handleFileInputChange = (
event: React.ChangeEvent<HTMLInputElement>,
) => {
const files = Array.from(event.target.files ?? [])
event.target.value = ''
if (files.length > 0) void stageFiles(files)
}
const shell = variant === 'home' ? HomeShell : ConversationShell
@@ -296,73 +488,203 @@ export const ConversationInput: FC<ConversationInputProps> = ({
return (
<Shell>
<div className="flex items-center gap-3 px-5 py-4">
<BotInputIcon variant={variant} />
<section
// Drag/drop on a region isn't a click affordance — wrap the
// composer in a labeled <section> so the a11y rule is satisfied
// without misrepresenting the surface as interactive.
aria-label="Message composer"
className={cn('relative', isDragOver && 'ring-2 ring-primary/60')}
onDragOver={handleDragOver}
onDragLeave={handleDragLeave}
onDrop={handleDrop}
>
<input
type="text"
value={input}
onChange={(event) => setInput(event.currentTarget.value)}
onKeyDown={(event) => {
if (event.key === 'Enter') {
event.preventDefault()
handleSend()
ref={fileInputRef}
type="file"
multiple
accept="image/png,image/jpeg,image/webp,image/gif,text/*,application/json"
className="hidden"
onChange={handleFileInputChange}
/>
{attachments.length > 0 || attachmentError ? (
<AttachmentStrip
attachments={attachments}
onRemove={removeAttachment}
error={attachmentError}
/>
) : null}
<div
className={cn(
'flex gap-3',
variant === 'home' ? 'px-4 py-3' : 'px-4 py-3',
isExpandedDraft ? 'items-end' : 'items-center',
)}
>
<BotInputIcon variant={variant} />
<div className="flex-1">
<Textarea
ref={textareaRef}
value={input}
onChange={(event) => setInput(event.currentTarget.value)}
onKeyDown={(event) => {
if (event.key === 'Enter' && !event.shiftKey) {
event.preventDefault()
handleSend()
}
}}
onPaste={handlePaste}
rows={1}
placeholder={
voice.isTranscribing
? 'Transcribing...'
: (placeholder ??
`Message ${selectedAgent?.name ?? 'agent'}...`)
}
disabled={disabled || voice.isTranscribing}
className={cn(
'resize-none border-none bg-transparent px-0 text-[15px] shadow-none focus-visible:ring-0',
'[field-sizing:fixed]',
variant === 'home'
? 'min-h-[40px] py-2 leading-6'
: 'min-h-[40px] py-2 leading-6',
'placeholder:text-muted-foreground/80',
)}
/>
</div>
{streaming && onStop ? <StopButton onStop={onStop} /> : null}
<VoiceButton
isRecording={voice.isRecording}
isTranscribing={voice.isTranscribing}
onStart={() => {
void voice.startRecording()
}}
onStop={() => {
void voice.stopRecording()
}}
/>
<InputActionButton
disabled={
!hasContent ||
isStaging ||
!!disabled ||
voice.isRecording ||
voice.isTranscribing ||
(streaming && !queueAware)
}
}}
placeholder={
voice.isTranscribing
? 'Transcribing...'
: (placeholder ?? `Message ${selectedAgent?.name ?? 'agent'}...`)
}
disabled={disabled || voice.isTranscribing}
className="flex-1 border-none bg-transparent text-base text-foreground outline-none placeholder:text-muted-foreground disabled:opacity-60"
onClick={handleSend}
// Spinner stays the user-facing "agent is busy" hint; with the
// queue active we still spin while a turn is in flight.
streaming={streaming}
hasContent={hasContent}
/>
</div>
{voice.error ? (
<div className="px-5 pb-2 text-destructive text-xs">
{voice.error}
</div>
) : null}
<ContextControls
agents={agents}
onCreateAgent={onCreateAgent}
onSelectAgent={onSelectAgent}
selectedAgentId={selectedAgentId}
selectedTabs={selectedTabs}
onToggleTab={toggleTab}
showAgentSelector={variant === 'home'}
status={status}
onAttachClick={openFilePicker}
attachDisabled={attachments.length >= 10 || isStaging || !!disabled}
attachmentsEnabled={attachmentsEnabled}
/>
<VoiceButton
isRecording={voice.isRecording}
isTranscribing={voice.isTranscribing}
onStart={() => {
void voice.startRecording()
}}
onStop={() => {
void voice.stopRecording()
}}
/>
<InputActionButton
disabled={
!input.trim() ||
streaming ||
!!disabled ||
voice.isRecording ||
voice.isTranscribing
}
onClick={handleSend}
streaming={streaming}
/>
</div>
{voice.error ? (
<div className="px-5 pb-2 text-destructive text-xs">{voice.error}</div>
) : null}
<ContextControls
agents={agents}
onCreateAgent={onCreateAgent}
onSelectAgent={onSelectAgent}
selectedAgentId={selectedAgentId}
selectedTabs={selectedTabs}
onToggleTab={toggleTab}
showAgentSelector={variant === 'home'}
status={status}
/>
{isDragOver ? (
<div className="pointer-events-none absolute inset-0 flex items-center justify-center rounded-[inherit] bg-background/80 font-medium text-foreground text-sm backdrop-blur-sm">
Drop files to attach
</div>
) : null}
</section>
</Shell>
)
}
function AttachmentStrip({
attachments,
onRemove,
error,
}: {
attachments: StagedAttachment[]
onRemove: (id: string) => void
error: string | null
}) {
return (
<div className="border-border/40 border-b px-4 pt-3 pb-2">
{attachments.length > 0 ? (
<div className="flex flex-wrap gap-2">
{attachments.map((attachment) => (
<AttachmentChip
key={attachment.id}
attachment={attachment}
onRemove={() => onRemove(attachment.id)}
/>
))}
</div>
) : null}
{error ? (
<div className="mt-2 text-destructive text-xs">{error}</div>
) : null}
</div>
)
}
function AttachmentChip({
attachment,
onRemove,
}: {
attachment: StagedAttachment
onRemove: () => void
}) {
if (attachment.kind === 'image' && attachment.dataUrl) {
return (
<div className="group relative size-16 overflow-hidden rounded-md border border-border/60">
<img
src={attachment.dataUrl}
alt={attachment.name}
className="size-full object-cover"
/>
<button
type="button"
onClick={onRemove}
className="absolute top-1 right-1 inline-flex size-5 items-center justify-center rounded-full bg-background/80 text-muted-foreground opacity-0 transition-opacity hover:text-foreground group-hover:opacity-100"
aria-label={`Remove ${attachment.name}`}
>
<X className="size-3" />
</button>
</div>
)
}
return (
<div className="group flex max-w-[220px] items-center gap-2 rounded-md border border-border/60 bg-background/60 px-2 py-1.5">
<FileText className="size-4 shrink-0 text-muted-foreground" />
<span className="truncate text-xs">{attachment.name}</span>
<button
type="button"
onClick={onRemove}
className="ml-1 inline-flex size-4 items-center justify-center text-muted-foreground hover:text-foreground"
aria-label={`Remove ${attachment.name}`}
>
<X className="size-3" />
</button>
</div>
)
}
function BotInputIcon({ variant }: { variant: 'home' | 'conversation' }) {
return (
<div
className={cn(
'flex items-center justify-center text-[var(--accent-orange)]',
variant === 'home'
? 'h-10 w-10 rounded-xl bg-[var(--accent-orange)]/10'
: 'h-9 w-9 rounded-xl bg-[var(--accent-orange)]/12',
? 'h-8 w-8 rounded-lg bg-[var(--accent-orange)]/10'
: 'h-8 w-8 rounded-lg bg-[var(--accent-orange)]/10',
)}
>
<Bot className="h-4 w-4" />

View File

@@ -1,7 +1,9 @@
import { Bot, CheckCircle2, Loader2, XCircle } from 'lucide-react'
import type { FC } from 'react'
import { Bot, CheckCircle2, Loader2, Wrench, XCircle } from 'lucide-react'
import { type FC, useMemo } from 'react'
import {
Message,
MessageAttachment,
MessageAttachments,
MessageContent,
MessageResponse,
} from '@/components/ai-elements/message'
@@ -10,96 +12,191 @@ import {
ReasoningContent,
ReasoningTrigger,
} from '@/components/ai-elements/reasoning'
import type { AgentConversationTurn } from '@/lib/agent-conversations/types'
import {
Task,
TaskContent,
TaskItem,
TaskTrigger,
} from '@/components/ai-elements/task'
import type {
AgentConversationTurn,
ToolEntry,
} from '@/lib/agent-conversations/types'
interface ConversationMessageProps {
turn: AgentConversationTurn
streaming: boolean
}
interface RenderEntry {
kind: 'thinking' | 'text' | 'task'
partIndex: number
text?: string
done?: boolean
tools?: ToolEntry[]
}
/**
* Build the render plan for an assistant turn:
* - thinking and text parts render in place
* - all tool-batch parts collapse into a single Task entry at their first
* appearance position, with tools listed in arrival order
*/
function buildRenderEntries(turn: AgentConversationTurn): RenderEntry[] {
const entries: RenderEntry[] = []
const aggregatedTools: ToolEntry[] = []
let taskInserted = false
turn.parts.forEach((part, partIndex) => {
if (part.kind === 'thinking') {
entries.push({
kind: 'thinking',
partIndex,
text: part.text,
done: part.done,
})
} else if (part.kind === 'text') {
entries.push({ kind: 'text', partIndex, text: part.text })
} else if (part.kind === 'tool-batch') {
aggregatedTools.push(...part.tools)
if (!taskInserted) {
entries.push({
kind: 'task',
partIndex,
tools: aggregatedTools,
})
taskInserted = true
}
}
})
return entries
}
function ToolStatusIcon({ status }: { status: ToolEntry['status'] }) {
if (status === 'running') {
return (
<Loader2 className="size-3.5 shrink-0 animate-spin text-muted-foreground" />
)
}
if (status === 'completed') {
return <CheckCircle2 className="size-3.5 shrink-0 text-green-500" />
}
return <XCircle className="size-3.5 shrink-0 text-destructive" />
}
export const ConversationMessage: FC<ConversationMessageProps> = ({
turn,
streaming,
}) => (
<div className="space-y-3">
<Message from="user">
<MessageContent>
<pre className="whitespace-pre-wrap font-sans text-sm">
{turn.userText}
</pre>
</MessageContent>
</Message>
}) => {
const entries = useMemo(() => buildRenderEntries(turn), [turn])
{turn.parts.length > 0 && (
<Message from="assistant">
return (
<div className="space-y-3">
<Message from="user">
<MessageContent>
{turn.parts.map((part, i) => {
const key = `${turn.id}-part-${i}`
{turn.userAttachments && turn.userAttachments.length > 0 && (
<MessageAttachments>
{turn.userAttachments.map((attachment) => (
<MessageAttachment
key={attachment.id}
data={{
type: 'file',
url: attachment.dataUrl ?? '',
mediaType: attachment.mediaType,
filename: attachment.name,
}}
/>
))}
</MessageAttachments>
)}
{turn.userText && (
<pre className="whitespace-pre-wrap font-sans text-sm">
{turn.userText}
</pre>
)}
</MessageContent>
</Message>
switch (part.kind) {
case 'thinking':
{entries.length > 0 && (
<Message from="assistant">
<MessageContent>
{entries.map((entry) => {
const key = `${turn.id}-entry-${entry.partIndex}`
if (entry.kind === 'thinking') {
return (
<Reasoning
key={key}
className="w-full"
isStreaming={!part.done}
defaultOpen={!part.done}
isStreaming={!entry.done}
defaultOpen={!entry.done}
>
<ReasoningTrigger />
<ReasoningContent>{part.text}</ReasoningContent>
<ReasoningContent>{entry.text ?? ''}</ReasoningContent>
</Reasoning>
)
}
case 'tool-batch':
if (entry.kind === 'text') {
return (
<div key={key} className="w-full space-y-1">
{part.tools.map((tool) => (
<div
<MessageResponse key={key}>
{entry.text ?? ''}
</MessageResponse>
)
}
const tools = entry.tools ?? []
const allDone = tools.every((t) => t.status !== 'running')
const taskTitle = allDone
? `Agent activity (${tools.length} ${tools.length === 1 ? 'action' : 'actions'})`
: `Working… (${tools.length} ${tools.length === 1 ? 'action' : 'actions'})`
return (
<Task key={key} defaultOpen={!turn.done}>
<TaskTrigger title={taskTitle} TriggerIcon={Wrench} />
<TaskContent>
{tools.map((tool) => (
<TaskItem
key={tool.id}
className="flex items-center gap-2 rounded-md border px-3 py-2 text-sm"
className="flex items-center gap-2"
>
{tool.status === 'running' && (
<Loader2 className="size-3.5 animate-spin text-muted-foreground" />
)}
{tool.status === 'completed' && (
<CheckCircle2 className="size-3.5 text-green-500" />
)}
{tool.status === 'error' && (
<XCircle className="size-3.5 text-destructive" />
)}
<span className="font-mono text-xs">{tool.name}</span>
<ToolStatusIcon status={tool.status} />
<span className="text-foreground text-xs">
{tool.label}
</span>
{tool.subject ? (
<span className="ml-1.5 truncate text-muted-foreground/70 text-xs">
· {tool.subject}
</span>
) : null}
{tool.durationMs != null && (
<span className="ml-auto text-muted-foreground text-xs">
<span className="ml-auto text-muted-foreground/60 text-xs tabular-nums">
{(tool.durationMs / 1000).toFixed(1)}s
</span>
)}
</div>
</TaskItem>
))}
</div>
)
</TaskContent>
</Task>
)
})}
</MessageContent>
</Message>
)}
case 'text':
return <MessageResponse key={key}>{part.text}</MessageResponse>
default:
return null
}
})}
</MessageContent>
</Message>
)}
{!turn.done && turn.parts.length === 0 && streaming && (
<div className="flex gap-2">
<div className="flex size-7 shrink-0 items-center justify-center rounded-full bg-[var(--accent-orange)] text-white">
<Bot className="size-3.5" />
{!turn.done && turn.parts.length === 0 && streaming && (
<div className="flex gap-2">
<div className="flex size-7 shrink-0 items-center justify-center rounded-full bg-[var(--accent-orange)] text-white">
<Bot className="size-3.5" />
</div>
<div className="flex items-center gap-1 rounded-xl rounded-tl-none border border-border/50 bg-card px-3 py-2.5 shadow-sm">
<span className="size-1.5 animate-bounce rounded-full bg-[var(--accent-orange)] [animation-delay:-0.3s]" />
<span className="size-1.5 animate-bounce rounded-full bg-[var(--accent-orange)] [animation-delay:-0.15s]" />
<span className="size-1.5 animate-bounce rounded-full bg-[var(--accent-orange)]" />
</div>
</div>
<div className="flex items-center gap-1 rounded-xl rounded-tl-none border border-border/50 bg-card px-3 py-2.5 shadow-sm">
<span className="size-1.5 animate-bounce rounded-full bg-[var(--accent-orange)] [animation-delay:-0.3s]" />
<span className="size-1.5 animate-bounce rounded-full bg-[var(--accent-orange)] [animation-delay:-0.15s]" />
<span className="size-1.5 animate-bounce rounded-full bg-[var(--accent-orange)]" />
</div>
</div>
)}
</div>
)
)}
</div>
)
}

View File

@@ -0,0 +1,243 @@
import { Quote, TriangleAlert } from 'lucide-react'
import type { FC } from 'react'
import { Badge } from '@/components/ui/badge'
import {
HoverCard,
HoverCardContent,
HoverCardTrigger,
} from '@/components/ui/hover-card'
import { adapterLabel } from '@/entrypoints/app/agents/AdapterIcon'
import { formatRelativeTime } from '@/entrypoints/app/agents/agent-display.helpers'
import type {
HarnessAdapterHealth,
HarnessAgent,
HarnessAgentAdapter,
} from '@/entrypoints/app/agents/agent-harness-types'
import { AgentTile } from '@/entrypoints/app/agents/agent-row/AgentTile'
import {
firstNonBlankLine,
truncate,
} from '@/entrypoints/app/agents/agent-row/agent-row.helpers'
import type { AgentLiveness } from '@/entrypoints/app/agents/LivenessDot'
import { cn } from '@/lib/utils'
interface HomeAgentCardProps {
agent: HarnessAgent
adapter: HarnessAgentAdapter | 'unknown'
/** Per-adapter health snapshot, shared across cards rendering the
* same adapter. `null` when the /adapters response hasn't surfaced
* health yet (we treat that as healthy until proven otherwise). */
adapterHealth: HarnessAdapterHealth | null
/** Highlights the card with an accent ring; tells the user which
* agent the conversation input is bound to. */
active?: boolean
onClick: () => void
}
const PREVIEW_CHARS = 100
/**
* Grid-shaped card for the /home Recent agents section. Composition
* mirrors the rail's `AgentRowCard` but the layout is a vertical
* column sized for a 1/3-width tile rather than a full-width row.
*
* Reuses `<AgentTile>`, `<LivenessDot>`, `livenessDetail`,
* `formatRelativeTime`, `firstNonBlankLine`, `truncate`, and the
* inline `Unavailable` chip pattern so the visual language is
* continuous between rail and grid.
*/
export const HomeAgentCard: FC<HomeAgentCardProps> = ({
agent,
adapter,
adapterHealth,
active,
onClick,
}) => {
const status = agent.status ?? 'unknown'
const lastUsedAt = agent.lastUsedAt ?? null
const isWorking = status === 'working'
const isAsleep = status === 'asleep'
const isError = status === 'error'
const hasActiveTurn = Boolean(agent.activeTurnId)
return (
<button
type="button"
onClick={onClick}
className={cn(
'group flex min-h-32 w-full min-w-0 flex-col rounded-2xl border bg-card p-4 text-left shadow-sm transition-colors',
active && 'ring-1 ring-[var(--accent-orange)]/30',
isWorking
? 'border-[var(--accent-orange)]/40'
: isError
? 'border-destructive/30'
: 'border-border/60 hover:border-[var(--accent-orange)]/30',
)}
>
<div className="flex items-start gap-3">
<AgentTile adapter={adapter} status={status} lastUsedAt={lastUsedAt} />
<div className="min-w-0 flex-1">
<div className="flex items-center gap-1.5">
<span className="truncate font-semibold text-sm">
{displayName(agent)}
</span>
{isWorking && (
<Badge
variant="secondary"
className="ml-auto bg-amber-50 text-amber-900 hover:bg-amber-50"
>
Working
</Badge>
)}
</div>
<SummaryLine
adapter={adapter}
modelId={agent.modelId ?? null}
reasoningEffort={agent.reasoningEffort ?? null}
adapterHealth={adapterHealth}
/>
</div>
</div>
<LastMessage message={agent.lastUserMessage ?? null} />
<div className="mt-3 flex items-center justify-between gap-2 text-muted-foreground text-xs">
<span>{statusFootnote(status, lastUsedAt)}</span>
{hasActiveTurn ? (
<ResumeChip />
) : isAsleep ? (
<Badge variant="outline" className="text-muted-foreground">
Asleep
</Badge>
) : isError ? (
<ErrorChip lastError={agent.lastError ?? null} />
) : null}
</div>
</button>
)
}
const SummaryLine: FC<{
adapter: HarnessAgentAdapter | 'unknown'
modelId: string | null
reasoningEffort: string | null
adapterHealth: HarnessAdapterHealth | null
}> = ({ adapter, modelId, reasoningEffort, adapterHealth }) => {
const parts = [adapterLabel(adapter)]
if (modelId) parts.push(modelId)
if (reasoningEffort) parts.push(reasoningEffort)
const unhealthy = adapterHealth?.healthy === false
return (
<div
className={cn(
'mt-0.5 flex items-center gap-1.5 text-muted-foreground text-xs',
unhealthy && 'text-muted-foreground/70',
)}
>
<span className="truncate">{parts.join(' · ')}</span>
{unhealthy && (
<HoverCard openDelay={200}>
<HoverCardTrigger asChild>
<Badge
variant="outline"
className="h-5 cursor-default gap-1 border-amber-500/40 bg-amber-50 px-1.5 text-amber-900 hover:bg-amber-50"
>
<TriangleAlert className="size-2.5" />
<span className="font-normal">Unavailable</span>
</Badge>
</HoverCardTrigger>
<HoverCardContent side="right" className="w-72 text-sm">
<div className="font-medium">
{adapterLabel(adapter)} CLI not available
</div>
<div className="mt-1 text-muted-foreground text-xs">
{adapterHealth?.reason ??
'Adapter binary missing on $PATH. Install it from the adapter docs to use this agent.'}
</div>
</HoverCardContent>
</HoverCard>
)}
</div>
)
}
const LastMessage: FC<{ message: string | null }> = ({ message }) => {
if (!message) {
return (
<p className="mt-3 flex-1 text-muted-foreground/70 text-xs italic">
No messages yet start a chat
</p>
)
}
return (
<p className="mt-3 line-clamp-2 flex flex-1 items-start gap-1.5 text-foreground/85 text-sm italic leading-snug">
<Quote
className="mt-1 size-3 shrink-0 text-muted-foreground/60"
aria-hidden
/>
<span className="line-clamp-2">
{truncate(firstNonBlankLine(message), PREVIEW_CHARS)}
</span>
</p>
)
}
const ResumeChip: FC = () => (
<span className="inline-flex items-center gap-1.5 rounded-full bg-[var(--accent-orange)] px-2.5 py-0.5 font-medium text-[11px] text-white shadow-sm">
<span className="relative flex size-1.5">
<span className="absolute inline-flex h-full w-full animate-ping rounded-full bg-white/70 opacity-75" />
<span className="relative inline-flex size-1.5 rounded-full bg-white" />
</span>
Resume
</span>
)
const ErrorChip: FC<{ lastError: string | null }> = ({ lastError }) => {
if (!lastError) {
return <Badge variant="destructive">Attention</Badge>
}
return (
<HoverCard openDelay={200}>
<HoverCardTrigger asChild>
<Badge variant="destructive" className="cursor-default">
Attention
</Badge>
</HoverCardTrigger>
<HoverCardContent
side="left"
className="max-w-xs whitespace-pre-wrap font-mono text-xs"
>
{lastError}
</HoverCardContent>
</HoverCard>
)
}
/**
* Footer left side: relative time on every state EXCEPT working,
* which shows `now` (the dot is already pulsing — restating it as
* "Working" would duplicate the pill in the title row).
*/
function statusFootnote(
status: AgentLiveness,
lastUsedAt: number | null,
): string {
if (status === 'working') return 'now'
return formatRelativeTime(lastUsedAt)
}
const UUID_PATTERN =
/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i
const OC_UUID_PATTERN =
/^oc-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i
function displayName(agent: HarnessAgent): string {
const name = agent.name?.trim()
const id = agent.id
if (!name || name === id) {
if (OC_UUID_PATTERN.test(id)) return id.slice(0, 11)
if (UUID_PATTERN.test(id)) return id.slice(0, 8)
return id
}
return name
}

View File

@@ -0,0 +1,94 @@
import { ListPlus, X } from 'lucide-react'
import type { FC } from 'react'
import {
Queue,
QueueItem,
QueueItemAction,
QueueItemActions,
QueueItemAttachment,
QueueItemContent,
QueueItemFile,
QueueItemImage,
QueueList,
QueueSection,
QueueSectionContent,
QueueSectionLabel,
QueueSectionTrigger,
} from '@/components/ai-elements/queue'
import type {
HarnessQueuedMessage,
HarnessQueuedMessageAttachment,
} from '@/entrypoints/app/agents/agent-harness-types'
import { firstNonBlankLine } from '@/entrypoints/app/agents/agent-row/agent-row.helpers'
interface QueuePanelProps {
queue: HarnessQueuedMessage[]
onRemove: (messageId: string) => void
}
/**
* Renders the agent's pending message queue using the shared AI
* Elements `Queue` primitives. Caller is expected to gate render on
* `queue.length > 0` — when empty, this returns null so the panel
* disappears cleanly between turns.
*/
export const QueuePanel: FC<QueuePanelProps> = ({ queue, onRemove }) => {
if (queue.length === 0) return null
return (
<Queue>
<QueueSection>
<QueueSectionTrigger>
<QueueSectionLabel
count={queue.length}
label={queue.length === 1 ? 'queued message' : 'queued messages'}
icon={<ListPlus className="size-3.5" />}
/>
</QueueSectionTrigger>
<QueueSectionContent>
<QueueList>
{queue.map((entry) => (
<QueueItem key={entry.id}>
<div className="flex items-center gap-2">
<QueueItemContent>
{firstNonBlankLine(entry.message)}
</QueueItemContent>
<QueueItemActions>
<QueueItemAction
aria-label="Remove from queue"
onClick={() => onRemove(entry.id)}
>
<X className="size-3" />
</QueueItemAction>
</QueueItemActions>
</div>
{entry.attachments && entry.attachments.length > 0 ? (
<QueueItemAttachment>
{entry.attachments.map((attachment, idx) =>
renderAttachment(entry.id, attachment, idx),
)}
</QueueItemAttachment>
) : null}
</QueueItem>
))}
</QueueList>
</QueueSectionContent>
</QueueSection>
</Queue>
)
}
function renderAttachment(
messageId: string,
attachment: HarnessQueuedMessageAttachment,
idx: number,
) {
if (attachment.mediaType.startsWith('image/')) {
const src = `data:${attachment.mediaType};base64,${attachment.data}`
return <QueueItemImage key={`${messageId}-${idx}`} src={src} />
}
return (
<QueueItemFile key={`${messageId}-${idx}`}>
{attachment.mediaType}
</QueueItemFile>
)
}

View File

@@ -1,8 +1,11 @@
import type { FC } from 'react'
import { Outlet, useOutletContext } from 'react-router'
import { useHarnessAgents } from '@/entrypoints/app/agents/useAgents'
import type {
AgentEntry,
OpenClawStatus,
} from '@/entrypoints/app/agents/useOpenClaw'
import {
type AgentEntry,
type OpenClawStatus,
useOpenClawAgents,
useOpenClawStatus,
} from '@/entrypoints/app/agents/useOpenClaw'
@@ -16,16 +19,32 @@ interface AgentCommandContextValue {
export const AgentCommandLayout: FC = () => {
const { status, loading: statusLoading } = useOpenClawStatus(5000)
const { agents, loading: agentsLoading } = useOpenClawAgents(
status?.status === 'running' && status.controlPlaneStatus === 'connected',
const openClawEnabled =
status?.status === 'running' && status.controlPlaneStatus === 'connected'
const { agents: openClawAgents, loading: openClawAgentsLoading } =
useOpenClawAgents(openClawEnabled)
const { agents: harnessAgents, loading: harnessAgentsLoading } =
useHarnessAgents()
const visibleOpenClawAgents = openClawEnabled ? openClawAgents : []
// Dual-created OpenClaw agents appear in both `/claw/agents` (gateway
// record) and `/agents` (harness record) under the same id. Prefer the
// harness entry so the chat panel can route through the harness path
// and the rail doesn't show duplicates.
const harnessAgentIds = new Set(harnessAgents.map((entry) => entry.agentId))
const dedupedOpenClawAgents = visibleOpenClawAgents.filter(
(entry) => !harnessAgentIds.has(entry.agentId),
)
const agents = [...dedupedOpenClawAgents, ...harnessAgents]
return (
<Outlet
context={
{
agents,
agentsLoading,
agentsLoading:
harnessAgentsLoading ||
statusLoading ||
(openClawEnabled && openClawAgentsLoading),
status,
statusLoading,
} satisfies AgentCommandContextValue

View File

@@ -0,0 +1,12 @@
import { describe, expect, it } from 'bun:test'
import { mapAgentHarnessToolStatus } from './agent-stream-events'
describe('mapAgentHarnessToolStatus', () => {
it('normalizes ACP tool statuses for the chat renderer', () => {
expect(mapAgentHarnessToolStatus('running')).toBe('running')
expect(mapAgentHarnessToolStatus('completed')).toBe('completed')
expect(mapAgentHarnessToolStatus('failed')).toBe('error')
expect(mapAgentHarnessToolStatus('incomplete')).toBe('running')
expect(mapAgentHarnessToolStatus(undefined)).toBe('running')
})
})

View File

@@ -0,0 +1,19 @@
import type { ToolEntry } from '@/lib/agent-conversations/types'
export function mapAgentHarnessToolStatus(
status: string | undefined,
): ToolEntry['status'] {
if (!status) return 'running'
const normalized = status.toLowerCase()
if (['error', 'failed', 'failure', 'denied'].includes(normalized)) {
return 'error'
}
if (
['complete', 'completed', 'done', 'success', 'succeeded'].includes(
normalized,
)
) {
return 'completed'
}
return 'running'
}

View File

@@ -0,0 +1,183 @@
import { describe, expect, it } from 'bun:test'
import type { AgentConversationTurn } from '@/lib/agent-conversations/types'
import {
type AgentHistoryPageResponse,
type BrowserOSChatHistoryItem,
buildChatHistoryFromClawMessages,
filterTurnsPersistedInHistory,
flattenHistoryPages,
mapHistoryItemToClawMessage,
} from './claw-chat-types'
function historyItem(
overrides: Partial<BrowserOSChatHistoryItem>,
): BrowserOSChatHistoryItem {
return {
id: 'session-1:0',
role: 'user',
text: 'Hello',
timestamp: 1000,
messageSeq: 0,
sessionKey: 'session-1',
source: 'user-chat',
...overrides,
}
}
function page(items: BrowserOSChatHistoryItem[]): AgentHistoryPageResponse {
return {
agentId: 'main',
sessionKey: 'session-1',
session: null,
items,
page: {
hasMore: false,
limit: 50,
},
}
}
describe('claw-chat-types', () => {
it('maps backend history items into text-first ClawChat messages', () => {
const message = mapHistoryItemToClawMessage(
historyItem({
id: 'session-1:1',
role: 'assistant',
text: 'Hi there',
messageSeq: 1,
}),
)
expect(message).toEqual({
id: 'session-1:1',
role: 'assistant',
sessionKey: 'session-1',
timestamp: 1000,
source: 'user-chat',
messageSeq: 1,
status: 'historical',
parts: [{ type: 'text', text: 'Hi there' }],
})
})
it('flattens paginated history into oldest-to-newest render order', () => {
const messages = flattenHistoryPages([
page([
historyItem({
id: 'session-1:2',
role: 'user',
text: 'newer',
timestamp: 3000,
messageSeq: 2,
}),
]),
page([
historyItem({
id: 'session-1:0',
role: 'user',
text: 'older',
timestamp: 1000,
messageSeq: 0,
}),
historyItem({
id: 'session-1:1',
role: 'assistant',
text: 'middle',
timestamp: 2000,
messageSeq: 1,
}),
]),
])
expect(messages.map((message) => message.id)).toEqual([
'session-1:0',
'session-1:1',
'session-1:2',
])
})
it('builds OpenClaw chat history from text message parts only', () => {
const history = buildChatHistoryFromClawMessages([
{
id: 'user-1',
role: 'user',
sessionKey: 'session-1',
parts: [{ type: 'text', text: ' User request ' }],
},
{
id: 'assistant-1',
role: 'assistant',
sessionKey: 'session-1',
parts: [
{ type: 'reasoning', text: 'private reasoning' },
{ type: 'text', text: 'Assistant answer' },
],
},
])
expect(history).toEqual([
{ role: 'user', content: 'User request' },
{ role: 'assistant', content: 'Assistant answer' },
])
})
it('hides completed live turns once harness history contains the same turn', () => {
const turn: AgentConversationTurn = {
id: 'live-turn',
userText: 'hello',
parts: [{ kind: 'text', text: 'hi there' }],
done: true,
timestamp: 1_000,
}
const visible = filterTurnsPersistedInHistory(
[turn],
[
{
id: 'history-user',
role: 'user',
sessionKey: 'main',
timestamp: 1_050,
status: 'historical',
parts: [{ type: 'text', text: 'hello' }],
},
{
id: 'history-assistant',
role: 'assistant',
sessionKey: 'main',
timestamp: 1_100,
status: 'historical',
parts: [{ type: 'text', text: 'hi there' }],
},
],
)
expect(visible).toEqual([])
})
it('keeps completed live turns until matching assistant history arrives', () => {
const turn: AgentConversationTurn = {
id: 'live-turn',
userText: 'hello',
parts: [{ kind: 'text', text: 'hi there' }],
done: true,
timestamp: 1_000,
}
const visible = filterTurnsPersistedInHistory(
[turn],
[
{
id: 'history-user',
role: 'user',
sessionKey: 'main',
timestamp: 1_050,
status: 'historical',
parts: [{ type: 'text', text: 'hello' }],
},
],
)
expect(visible).toEqual([turn])
})
})

View File

@@ -0,0 +1,287 @@
import type { OpenClawChatHistoryMessage } from '@/entrypoints/app/agents/useOpenClaw'
import type { AgentConversationTurn } from '@/lib/agent-conversations/types'
export type ClawChatRole = 'user' | 'assistant'
export type ClawChatSource = 'user-chat' | 'cron' | 'hook' | 'channel' | 'other'
export interface BrowserOSOpenClawSession {
key: string
updatedAt: number
sessionId: string
agentId: string
kind: string
source: ClawChatSource
status?: string
totalTokens?: number
model?: string
modelProvider?: string
}
export interface BrowserOSChatHistoryToolCall {
toolCallId?: string
toolName: string
label: string
subject?: string
status: 'pending' | 'running' | 'completed' | 'failed'
input?: unknown
output?: unknown
error?: string
durationMs?: number
}
export interface BrowserOSChatHistoryReasoning {
text: string
durationMs?: number
}
export interface BrowserOSChatHistoryAttachment {
kind: 'image' | 'file'
mediaType: string
// Images carry a `data:` URL so we can render directly without any
// additional fetch; files (text/PDF) currently round-trip via inline
// text in the message body and do not populate this field in v1.
dataUrl?: string
name?: string
}
export interface BrowserOSChatHistoryItem {
id: string
role: ClawChatRole
text: string
timestamp?: number
messageSeq: number
sessionKey: string
source: ClawChatSource
costUsd?: number
tokensIn?: number
tokensOut?: number
toolCalls?: BrowserOSChatHistoryToolCall[]
reasoning?: BrowserOSChatHistoryReasoning
attachments?: BrowserOSChatHistoryAttachment[]
}
export interface AgentHistoryPageResponse {
agentId: string
sessionKey: string | null
session: BrowserOSOpenClawSession | null
items: BrowserOSChatHistoryItem[]
page: {
cursor?: string
hasMore: boolean
limit: number
}
}
export type ClawChatMessageStatus =
| 'historical'
| 'sending'
| 'streaming'
| 'error'
export type ClawChatMessagePart =
| { type: 'text'; text: string }
| { type: 'reasoning'; text: string; duration?: number }
| {
type: 'tool-call'
name: string
label: string
subject?: string
status: 'pending' | 'running' | 'completed' | 'failed'
input?: unknown
output?: unknown
error?: string
durationMs?: number
}
| {
type: 'attachment'
kind: 'image' | 'file'
mediaType: string
dataUrl?: string
name?: string
}
| { type: 'meta'; label: string; value: string }
export interface ClawChatMessage {
id: string
role: ClawChatRole
sessionKey: string
timestamp?: number
source?: ClawChatSource
messageSeq?: number
status?: ClawChatMessageStatus
parts: ClawChatMessagePart[]
costUsd?: number
tokensIn?: number
tokensOut?: number
}
export function mapHistoryItemToClawMessage(
item: BrowserOSChatHistoryItem,
): ClawChatMessage {
const parts: ClawChatMessagePart[] = []
// Attachments first — they belong above the text in user messages and
// never appear on assistant messages today (assistant images come back
// through tool results, which render via the Task collapsible).
if (item.attachments && item.attachments.length > 0) {
for (const attachment of item.attachments) {
parts.push({
type: 'attachment',
kind: attachment.kind,
mediaType: attachment.mediaType,
dataUrl: attachment.dataUrl,
name: attachment.name,
})
}
}
// Reasoning, then tool calls, then text — the chronological order the
// agent produced them (think → act → answer).
if (item.reasoning && item.reasoning.text.trim().length > 0) {
// 0ms means thinking and the final answer were emitted in the same JSONL
// line (no tool calls between them) — there's no real elapsed wall-clock,
// so fall through to the "Thinking" trigger instead of "Thought for 0
// seconds" / streaming shimmer. Real multi-line turns floor at 1s.
const durationMs = item.reasoning.durationMs ?? 0
const duration =
durationMs > 0 ? Math.max(1, Math.round(durationMs / 1000)) : undefined
parts.push({
type: 'reasoning',
text: item.reasoning.text,
duration,
})
}
if (item.toolCalls && item.toolCalls.length > 0) {
for (const tc of item.toolCalls) {
parts.push({
type: 'tool-call',
name: tc.toolName,
label: tc.label,
subject: tc.subject,
status: tc.status,
input: tc.input,
output: tc.output,
error: tc.error,
durationMs: tc.durationMs,
})
}
}
// Only emit a text part when there's actual content. User messages with
// only attachments and no caption shouldn't render an empty bubble.
if (item.text.trim().length > 0) {
parts.push({ type: 'text', text: item.text })
}
return {
id: item.id,
role: item.role,
sessionKey: item.sessionKey,
timestamp: item.timestamp,
source: item.source,
messageSeq: item.messageSeq,
status: 'historical',
parts,
costUsd: item.costUsd,
tokensIn: item.tokensIn,
tokensOut: item.tokensOut,
}
}
export function flattenHistoryPages(
pages: AgentHistoryPageResponse[],
): ClawChatMessage[] {
return pages
.flatMap((page) => page.items)
.sort((a, b) => {
if (a.timestamp != null && b.timestamp != null) {
return a.timestamp - b.timestamp
}
return a.messageSeq - b.messageSeq
})
.map(mapHistoryItemToClawMessage)
}
export function buildChatHistoryFromClawMessages(
messages: ClawChatMessage[],
): OpenClawChatHistoryMessage[] {
return messages
.map((message) => {
const content = message.parts
.filter((part): part is { type: 'text'; text: string } => {
return part.type === 'text' && part.text.trim().length > 0
})
.map((part) => part.text.trim())
.join('\n\n')
return content ? { role: message.role, content } : null
})
.filter((message): message is OpenClawChatHistoryMessage =>
Boolean(message),
)
}
const TURN_HISTORY_MATCH_WINDOW_MS = 5_000
export function filterTurnsPersistedInHistory(
turns: AgentConversationTurn[],
historyMessages: ClawChatMessage[],
): AgentConversationTurn[] {
return turns.filter(
(turn) => !isTurnPersistedInHistory(turn, historyMessages),
)
}
function isTurnPersistedInHistory(
turn: AgentConversationTurn,
historyMessages: ClawChatMessage[],
): boolean {
if (!turn.done) return false
const assistantText = getTurnAssistantText(turn)
if (!assistantText) return false
const minTimestamp = turn.timestamp - TURN_HISTORY_MATCH_WINDOW_MS
const userText = turn.userText.trim()
const userPersisted =
!userText ||
historyMessages.some(
(message) =>
message.role === 'user' &&
isHistoryMessageAfter(message, minTimestamp) &&
getClawMessageText(message) === userText,
)
const assistantPersisted = historyMessages.some(
(message) =>
message.role === 'assistant' &&
isHistoryMessageAfter(message, minTimestamp) &&
getClawMessageText(message) === assistantText,
)
return userPersisted && assistantPersisted
}
function isHistoryMessageAfter(
message: ClawChatMessage,
minTimestamp: number,
): boolean {
return message.timestamp == null || message.timestamp >= minTimestamp
}
function getTurnAssistantText(turn: AgentConversationTurn): string {
return turn.parts
.filter((part) => part.kind === 'text')
.map((part) => part.text)
.join('')
.trim()
}
function getClawMessageText(message: ClawChatMessage): string {
return message.parts
.filter((part) => part.type === 'text')
.map((part) => part.text)
.join('')
.trim()
}

View File

@@ -0,0 +1,71 @@
import { buildToolLabel } from '../../../lib/tool-labels'
import type { HarnessAgentHistoryPage } from '../agents/agent-harness-types'
import type {
AgentHistoryPageResponse,
BrowserOSChatHistoryItem,
BrowserOSChatHistoryToolCall,
} from './claw-chat-types'
export function mapHarnessHistoryPage(
page: HarnessAgentHistoryPage,
): AgentHistoryPageResponse {
const items: BrowserOSChatHistoryItem[] = page.items.map((item, index) => {
const toolCalls = item.toolCalls?.map(
(tool): BrowserOSChatHistoryToolCall => {
const input = asRecord(tool.input)
const { label, subject } = buildToolLabel(tool.toolName, input)
return {
toolName: tool.toolName,
label,
status: tool.status,
...(tool.toolCallId ? { toolCallId: tool.toolCallId } : {}),
...(subject ? { subject } : {}),
...(tool.input !== undefined ? { input: tool.input } : {}),
...(tool.output !== undefined ? { output: tool.output } : {}),
...(tool.error ? { error: tool.error } : {}),
...(tool.durationMs != null ? { durationMs: tool.durationMs } : {}),
}
},
)
return {
id: item.id,
role: item.role,
text: item.text,
timestamp: item.createdAt,
messageSeq: index + 1,
sessionKey: 'main',
source: 'user-chat',
...(item.reasoning ? { reasoning: item.reasoning } : {}),
...(toolCalls && toolCalls.length > 0 ? { toolCalls } : {}),
}
})
const updatedAt =
page.items.length > 0
? Math.max(...page.items.map((item) => item.createdAt))
: Date.now()
return {
agentId: page.agentId,
sessionKey: 'main',
session: {
key: 'main',
updatedAt,
sessionId: 'main',
agentId: page.agentId,
kind: 'agent-harness',
source: 'user-chat',
},
items,
page: {
hasMore: false,
limit: items.length,
},
}
}
function asRecord(value: unknown): Record<string, unknown> | undefined {
return value && typeof value === 'object' && !Array.isArray(value)
? (value as Record<string, unknown>)
: undefined
}

View File

@@ -0,0 +1,69 @@
import { describe, expect, it } from 'bun:test'
import type { HarnessAgent } from '@/entrypoints/app/agents/agent-harness-types'
import { orderHomeAgents } from './home-agent-card.helpers'
function agent(overrides: Partial<HarnessAgent>): HarnessAgent {
return {
id: overrides.id ?? 'agent-x',
name: overrides.name ?? overrides.id ?? 'agent-x',
adapter: overrides.adapter ?? 'codex',
permissionMode: 'approve-all',
sessionKey: `agent:${overrides.id ?? 'agent-x'}:main`,
createdAt: 1000,
updatedAt: 1000,
...overrides,
}
}
describe('orderHomeAgents', () => {
it('places active-turn agents before everyone else', () => {
const sorted = orderHomeAgents([
agent({ id: 'a', lastUsedAt: 5000 }),
agent({ id: 'b', lastUsedAt: 9000, activeTurnId: 'turn-1' }),
agent({ id: 'c', lastUsedAt: 7000 }),
])
expect(sorted.map((a) => a.id)).toEqual(['b', 'c', 'a'])
})
it('orders non-active agents by lastUsedAt desc', () => {
const sorted = orderHomeAgents([
agent({ id: 'old', lastUsedAt: 1000 }),
agent({ id: 'new', lastUsedAt: 9000 }),
agent({ id: 'mid', lastUsedAt: 5000 }),
])
expect(sorted.map((a) => a.id)).toEqual(['new', 'mid', 'old'])
})
it('puts the gateway `main` seed agent above other never-used agents', () => {
const sorted = orderHomeAgents([
agent({ id: 'oc-aaaaaa', lastUsedAt: null }),
agent({ id: 'main', lastUsedAt: null }),
agent({ id: 'oc-bbbbbb', lastUsedAt: null }),
])
expect(sorted.map((a) => a.id)).toEqual(['main', 'oc-aaaaaa', 'oc-bbbbbb'])
})
it('sends never-used agents to the bottom even when `main` is among them', () => {
const sorted = orderHomeAgents([
agent({ id: 'main', lastUsedAt: null }),
agent({ id: 'used', lastUsedAt: 5000 }),
])
expect(sorted.map((a) => a.id)).toEqual(['used', 'main'])
})
it('does NOT sort by pinned — pinned agents are treated like any other', () => {
const sorted = orderHomeAgents([
agent({ id: 'unpinned-recent', lastUsedAt: 9000, pinned: false }),
agent({ id: 'pinned-old', lastUsedAt: 1000, pinned: true }),
])
expect(sorted.map((a) => a.id)).toEqual(['unpinned-recent', 'pinned-old'])
})
it('falls back to id-stable ordering when lastUsedAt ties', () => {
const sorted = orderHomeAgents([
agent({ id: 'b', lastUsedAt: 5000 }),
agent({ id: 'a', lastUsedAt: 5000 }),
])
expect(sorted.map((a) => a.id)).toEqual(['a', 'b'])
})
})

View File

@@ -0,0 +1,42 @@
import type { HarnessAgent } from '@/entrypoints/app/agents/agent-harness-types'
/**
* Order for the /home Recent agents grid.
*
* 1. Active turn first — agents mid-turn float to the top so the
* Resume affordance is the first thing the user sees on /home.
* 2. The protected gateway-side `main` agent stays pinned-to-top in
* the never-used group on a fresh install (mirrors the rail).
* 3. Recency (`lastUsedAt` desc).
* 4. `id` tiebreaker for stability so the grid doesn't reshuffle on
* every 5-second poll.
*
* Pin is NOT a sort key. The home grid is action-oriented and trusts
* recency + active-turn to surface the right agent; pinning is an
* organisation tool that lives on the rail at /agents.
*/
export function orderHomeAgents(agents: HarnessAgent[]): HarnessAgent[] {
return [...agents].sort((a, b) => {
const aActive = a.activeTurnId != null
const bActive = b.activeTurnId != null
if (aActive !== bActive) return aActive ? -1 : 1
// Recency wins outright. Never-used agents (`lastUsedAt == null`)
// both fall to the same `-Infinity` bucket and the seed/id rules
// below decide their order — but a used agent always beats any
// never-used agent regardless of id.
const aValue = a.lastUsedAt ?? Number.NEGATIVE_INFINITY
const bValue = b.lastUsedAt ?? Number.NEGATIVE_INFINITY
if (aValue !== bValue) return bValue - aValue
// Inside the never-used (or exact-tie) group: pin the gateway
// `main` seed to the top of the group on a fresh install, then
// fall back to id-stable order so the grid doesn't reshuffle on
// every poll.
const aSeed = a.id === 'main' && a.lastUsedAt == null
const bSeed = b.id === 'main' && b.lastUsedAt == null
if (aSeed !== bSeed) return aSeed ? -1 : 1
return a.id.localeCompare(b.id)
})
}

View File

@@ -1,69 +0,0 @@
import { useEffect, useState } from 'react'
import {
type AgentEntry,
getModelDisplayName,
type OpenClawStatus,
} from '@/entrypoints/app/agents/useOpenClaw'
import { getLatestConversation } from '@/lib/agent-conversations/storage'
import type { AgentCardData } from '@/lib/agent-conversations/types'
function getAgentStatusTone(
status: OpenClawStatus['status'] | undefined,
): AgentCardData['status'] {
if (status === 'error') return 'error'
if (status === 'starting') return 'working'
return 'idle'
}
async function getAgentCardData(
agent: AgentEntry,
status: OpenClawStatus['status'] | undefined,
): Promise<AgentCardData> {
const conversation = await getLatestConversation(agent.agentId)
const lastTurn = conversation?.turns[conversation.turns.length - 1]
const lastTextPart = lastTurn?.parts.findLast((part) => part.kind === 'text')
return {
agentId: agent.agentId,
name: agent.name,
model: getModelDisplayName(agent.model),
status: getAgentStatusTone(status),
lastMessage:
lastTextPart?.kind === 'text'
? lastTextPart.text.slice(0, 120)
: undefined,
lastMessageTimestamp: lastTurn?.timestamp,
}
}
export function useAgentCardData(
agents: AgentEntry[],
status: OpenClawStatus['status'] | undefined,
) {
const [cardData, setCardData] = useState<AgentCardData[]>([])
useEffect(() => {
let active = true
const loadCardData = async () => {
const nextCardData = await Promise.all(
agents.map((agent) => getAgentCardData(agent, status)),
)
if (active) {
setCardData(nextCardData)
}
}
if (agents.length > 0) {
void loadCardData()
} else {
setCardData([])
}
return () => {
active = false
}
}, [agents, status])
return cardData
}

View File

@@ -1,52 +1,86 @@
import { useEffect, useRef, useState } from 'react'
import {
buildChatHistoryFromTurns,
chatWithAgent,
type OpenClawStreamEvent,
} from '@/entrypoints/app/agents/useOpenClaw'
import {
getLatestConversation,
saveConversation,
} from '@/lib/agent-conversations/storage'
type AgentHarnessStreamEvent,
attachToHarnessTurn,
cancelHarnessTurn,
chatWithHarnessAgent,
fetchActiveHarnessTurn,
} from '@/entrypoints/app/agents/useAgents'
import type { OpenClawChatHistoryMessage } from '@/entrypoints/app/agents/useOpenClaw'
import type {
AgentConversation,
AgentConversationTurn,
AssistantPart,
ToolEntry,
UserAttachmentPreview,
} from '@/lib/agent-conversations/types'
import type { ServerAttachmentPayload } from '@/lib/attachments'
import { consumeSSEStream } from '@/lib/sse'
import { buildToolLabel } from '@/lib/tool-labels'
import { mapAgentHarnessToolStatus } from './agent-stream-events'
export function useAgentConversation(agentId: string, agentName: string) {
export interface SendInput {
text: string
attachments?: ServerAttachmentPayload[]
// Optional preview metadata used to render the optimistic user turn.
// Built by the composer at staging time; the server only sees the
// payload array.
attachmentPreviews?: UserAttachmentPreview[]
}
interface UseAgentConversationOptions {
// The hook always speaks to the harness chat path now; the OpenClaw
// legacy /claw/agents/:id/chat surface was removed in Step 12. The
// option remains for forward-compatibility.
runtime?: 'agent-harness'
sessionKey?: string | null
history?: OpenClawChatHistoryMessage[]
onComplete?: () => void
onSessionKeyChange?: (sessionKey: string) => void
/**
* Server-side active turn id, surfaced via the listing query. When
* this changes from null/<id> to a different non-null id while we
* aren't already streaming (e.g. the server just popped a queued
* message and started a new turn), the hook reattaches via
* /chat/active so the chat panel picks up the live stream without
* waiting for a remount.
*/
activeTurnId?: string | null
}
export function useAgentConversation(
agentId: string,
options: UseAgentConversationOptions = {},
) {
const [turns, setTurns] = useState<AgentConversationTurn[]>([])
const [streaming, setStreaming] = useState(false)
const [loading, setLoading] = useState(true)
const sessionKeyRef = useRef('')
const sessionKeyRef = useRef(options.sessionKey ?? '')
const historyRef = useRef<OpenClawChatHistoryMessage[]>(options.history ?? [])
const textAccRef = useRef('')
const thinkAccRef = useRef('')
const streamAbortRef = useRef<AbortController | null>(null)
const onCompleteRef = useRef(options.onComplete)
const onSessionKeyChangeRef = useRef(options.onSessionKeyChange)
// Per-turn resume bookkeeping. `turnId` is captured from the response
// header; `lastSeq` advances with every SSE event so a reconnect can
// resume via Last-Event-ID.
const turnIdRef = useRef<string | null>(null)
const lastSeqRef = useRef<number | null>(null)
useEffect(() => {
let active = true
getLatestConversation(agentId)
.then((conv) => {
if (!active) return
if (conv) {
setTurns(conv.turns)
sessionKeyRef.current = conv.sessionKey
} else {
sessionKeyRef.current = crypto.randomUUID()
}
setLoading(false)
})
.catch(() => {
if (active) {
sessionKeyRef.current = crypto.randomUUID()
setLoading(false)
}
})
return () => {
active = false
}
}, [agentId])
sessionKeyRef.current = options.sessionKey ?? ''
}, [options.sessionKey])
useEffect(() => {
historyRef.current = options.history ?? []
}, [options.history])
useEffect(() => {
onCompleteRef.current = options.onComplete
}, [options.onComplete])
useEffect(() => {
onSessionKeyChangeRef.current = options.onSessionKeyChange
}, [options.onSessionKeyChange])
useEffect(() => {
return () => {
@@ -54,17 +88,11 @@ export function useAgentConversation(agentId: string, agentName: string) {
}
}, [])
const persistTurns = (updatedTurns: AgentConversationTurn[]) => {
const conv: AgentConversation = {
agentId,
agentName,
sessionKey: sessionKeyRef.current,
turns: updatedTurns,
createdAt: updatedTurns[0]?.timestamp ?? Date.now(),
updatedAt: Date.now(),
}
saveConversation(conv).catch(() => {})
}
// Indirection for the resume effect below: lets it call the latest
// event handler without re-subscribing on every render.
const processEventRef = useRef<(event: AgentHarnessStreamEvent) => void>(
() => {},
)
const updateCurrentTurnParts = (
updater: (parts: AssistantPart[]) => AssistantPart[],
@@ -76,123 +104,236 @@ export function useAgentConversation(agentId: string, agentName: string) {
})
}
const processStreamEvent = (event: OpenClawStreamEvent) => {
switch (event.type) {
case 'text-delta': {
const delta = (event.data.text as string) ?? ''
textAccRef.current += delta
const text = textAccRef.current
updateCurrentTurnParts((parts) => {
const last = parts[parts.length - 1]
if (last?.kind === 'text') {
return [...parts.slice(0, -1), { ...last, text }]
}
return [...parts, { kind: 'text', text }]
})
break
const appendTextDelta = (delta: string) => {
textAccRef.current += delta
const text = textAccRef.current
updateCurrentTurnParts((parts) => {
const last = parts[parts.length - 1]
if (last?.kind === 'text') {
return [...parts.slice(0, -1), { ...last, text }]
}
case 'thinking': {
const delta = (event.data.text as string) ?? ''
thinkAccRef.current += delta
const text = thinkAccRef.current
updateCurrentTurnParts((parts) => {
const idx = parts.findIndex((p) => p.kind === 'thinking' && !p.done)
if (idx >= 0) {
return [
...parts.slice(0, idx),
{ ...parts[idx], text, done: false },
...parts.slice(idx + 1),
]
}
return [...parts, { kind: 'thinking', text, done: false }]
})
break
}
case 'tool-start': {
const tool = {
id: (event.data.toolCallId as string) ?? crypto.randomUUID(),
name: (event.data.toolName as string) ?? 'unknown',
status: 'running' as const,
}
updateCurrentTurnParts((parts) => {
const last = parts[parts.length - 1]
if (last?.kind === 'tool-batch') {
return [
...parts.slice(0, -1),
{ ...last, tools: [...last.tools, tool] },
]
}
return [...parts, { kind: 'tool-batch', tools: [tool] }]
})
break
}
case 'tool-end': {
const toolId = event.data.toolCallId as string
const toolStatus: 'completed' | 'error' =
(event.data.status as string) === 'error' ? 'error' : 'completed'
const durationMs = event.data.durationMs as number | undefined
updateCurrentTurnParts((parts) => {
for (let i = parts.length - 1; i >= 0; i--) {
const part = parts[i]
if (
part.kind === 'tool-batch' &&
part.tools.some((t) => t.id === toolId)
) {
const updatedTools = part.tools.map((t) =>
t.id === toolId ? { ...t, status: toolStatus, durationMs } : t,
)
return [
...parts.slice(0, i),
{ ...part, tools: updatedTools },
...parts.slice(i + 1),
]
}
}
return parts
})
break
}
case 'done': {
updateCurrentTurnParts((parts) =>
parts.map((part) =>
part.kind === 'thinking' ? { ...part, done: true } : part,
),
)
setTurns((prev) => {
const last = prev[prev.length - 1]
if (!last) return prev
const updated = [...prev.slice(0, -1), { ...last, done: true }]
persistTurns(updated)
return updated
})
break
}
case 'error': {
const msg =
(event.data.message as string) ??
(event.data.error as string) ??
'Unknown error'
updateCurrentTurnParts((parts) => [
...parts,
{ kind: 'text', text: `Error: ${msg}` },
])
break
}
}
return [...parts, { kind: 'text', text }]
})
}
const send = async (text: string) => {
if (!text.trim() || streaming) return
const history = buildChatHistoryFromTurns(turns)
const appendThinkingDelta = (delta: string) => {
thinkAccRef.current += delta
const text = thinkAccRef.current
updateCurrentTurnParts((parts) => {
const idx = parts.findIndex((p) => p.kind === 'thinking' && !p.done)
if (idx >= 0) {
return [
...parts.slice(0, idx),
{ ...parts[idx], text, done: false },
...parts.slice(idx + 1),
]
}
return [...parts, { kind: 'thinking', text, done: false }]
})
}
const appendErrorText = (message: string) => {
updateCurrentTurnParts((parts) => [
...parts,
{ kind: 'text', text: `Error: ${message}` },
])
}
const markCurrentTurnDone = () => {
updateCurrentTurnParts((parts) =>
parts.map((part) =>
part.kind === 'thinking' ? { ...part, done: true } : part,
),
)
setTurns((prev) => {
const last = prev[prev.length - 1]
if (!last) return prev
return [...prev.slice(0, -1), { ...last, done: true }]
})
}
const upsertAgentHarnessTool = (event: AgentHarnessStreamEvent) => {
if (event.type !== 'tool_call') return
const rawName = event.title || event.rawType || 'tool call'
const { label, subject } = buildToolLabel(
rawName,
event.text ? { description: event.text } : undefined,
)
const tool: ToolEntry = {
id: event.id ?? crypto.randomUUID(),
name: rawName,
label,
subject,
status: mapAgentHarnessToolStatus(event.status),
}
updateCurrentTurnParts((parts) => {
for (let i = parts.length - 1; i >= 0; i--) {
const part = parts[i]
if (
part.kind === 'tool-batch' &&
part.tools.some((existing) => existing.id === tool.id)
) {
const tools = part.tools.map((existing) =>
existing.id === tool.id ? { ...existing, ...tool } : existing,
)
return [
...parts.slice(0, i),
{ ...part, tools },
...parts.slice(i + 1),
]
}
}
const last = parts[parts.length - 1]
if (last?.kind === 'tool-batch') {
return [
...parts.slice(0, -1),
{ ...last, tools: [...last.tools, tool] },
]
}
return [...parts, { kind: 'tool-batch', tools: [tool] }]
})
}
const processAgentHarnessStreamEvent = (event: AgentHarnessStreamEvent) => {
switch (event.type) {
case 'text_delta':
if (event.stream === 'thought') {
appendThinkingDelta(event.text)
} else {
appendTextDelta(event.text)
}
break
case 'tool_call':
upsertAgentHarnessTool(event)
break
case 'done':
markCurrentTurnDone()
break
case 'error':
appendErrorText(event.message)
break
case 'status':
break
}
}
processEventRef.current = processAgentHarnessStreamEvent
const activeTurnIdDep = options.activeTurnId ?? null
// On mount, on agent change, and whenever the listing reports a
// *new* active turn id, check whether the server has an in-flight
// turn for this agent and reattach to it. This catches three
// cases at once: the chat resilience flow (tab close/reopen),
// navigation between agents, AND queue drain (the server starts a
// new turn from a queued message → activeTurnId flips → attach).
useEffect(() => {
let cancelled = false
const abortController = new AbortController()
// Reference the dep inside the body so biome's exhaustive-deps
// rule sees it consumed; the value is just an "any non-null
// active turn id" trigger — the actual id we attach to comes
// from the fresh fetchActiveHarnessTurn call below.
void activeTurnIdDep
const attemptResume = async () => {
// Track whether *we* started a stream in this run. When the
// early-return paths fire (no active turn, or a `send()` /
// earlier resume already owns `streamAbortRef`), the finally
// block must NOT touch streaming/turnIdRef/lastSeqRef —
// otherwise we clobber the in-flight stream's state and the
// Stop button drops out mid-turn while events keep arriving.
let weStartedStream = false
try {
const active = await fetchActiveHarnessTurn(agentId)
if (cancelled || !active || active.status !== 'running') return
if (streamAbortRef.current) return // someone else already owns the stream
// Stage a placeholder turn so the streamed events have a row
// to render into. The server now persists the kicking-off
// prompt on the active turn, so we render it as the user
// bubble immediately — no empty-bubble flicker when a queued
// message starts running.
setTurns((prev) => [
...prev,
{
id: crypto.randomUUID(),
userText: active.prompt ?? '',
parts: [],
done: false,
timestamp: active.startedAt,
},
])
textAccRef.current = ''
thinkAccRef.current = ''
turnIdRef.current = active.turnId
lastSeqRef.current = null
streamAbortRef.current = abortController
setStreaming(true)
weStartedStream = true
const response = await attachToHarnessTurn(agentId, {
turnId: active.turnId,
signal: abortController.signal,
})
if (!response.ok) return
await consumeSSEStream<AgentHarnessStreamEvent>(
response,
(event, meta) => {
if (typeof meta.seq === 'number') lastSeqRef.current = meta.seq
processEventRef.current(event)
},
abortController.signal,
)
} catch {
// Resume is best-effort; transient errors fall back to the
// user starting a new turn manually.
} finally {
// Always release `streamAbortRef` if we owned it — even when
// the effect was cancelled mid-stream (a listing poll
// captured the next queue-drain turn id, for example). If we
// don't, the next effect run hits `if (streamAbortRef.current)
// return` against our now-aborted controller and never
// reattaches, leaving `streaming === true` with no live stream.
if (weStartedStream && streamAbortRef.current === abortController) {
streamAbortRef.current = null
}
// The other state (streaming flag, turn id, lastSeq) is the
// *current run's* lifecycle: only reset it on a clean exit.
// When `cancelled` is true the next run will set these
// itself, so resetting here would only cause a brief flicker.
if (!cancelled && weStartedStream) {
turnIdRef.current = null
lastSeqRef.current = null
setStreaming(false)
}
}
}
void attemptResume()
return () => {
cancelled = true
abortController.abort()
}
}, [agentId, activeTurnIdDep])
const send = async (input: string | SendInput) => {
const normalized: SendInput =
typeof input === 'string' ? { text: input } : input
const trimmed = normalized.text.trim()
const attachments = normalized.attachments ?? []
if (streaming) return
if (!trimmed && attachments.length === 0) return
const turn: AgentConversationTurn = {
id: crypto.randomUUID(),
userText: text.trim(),
userText: trimmed,
userAttachments:
normalized.attachmentPreviews &&
normalized.attachmentPreviews.length > 0
? normalized.attachmentPreviews
: undefined,
parts: [],
done: false,
timestamp: Date.now(),
@@ -205,13 +346,37 @@ export function useAgentConversation(agentId: string, agentName: string) {
streamAbortRef.current = abortController
try {
const response = await chatWithAgent(
let response = await chatWithHarnessAgent(
agentId,
text.trim(),
sessionKeyRef.current,
history,
trimmed,
abortController.signal,
attachments,
)
// 409 means the server already has an active turn for this
// agent (e.g. a previous tab kicked one off and we're a fresh
// mount that missed the resume window). Attach to it instead of
// double-sending.
if (response.status === 409) {
const body = (await response.json()) as { turnId?: string }
if (body.turnId) {
response = await attachToHarnessTurn(agentId, {
turnId: body.turnId,
signal: abortController.signal,
})
}
}
const responseSessionKey =
response.headers.get('X-Session-Key') ??
response.headers.get('X-Session-Id')
if (responseSessionKey) {
sessionKeyRef.current = responseSessionKey
onSessionKeyChangeRef.current?.(responseSessionKey)
}
const responseTurnId = response.headers.get('X-Turn-Id')
if (responseTurnId) {
turnIdRef.current = responseTurnId
lastSeqRef.current = null
}
if (!response.ok) {
const err = await response.text()
updateCurrentTurnParts((parts) => [
@@ -220,9 +385,12 @@ export function useAgentConversation(agentId: string, agentName: string) {
])
return
}
await consumeSSEStream(
await consumeSSEStream<AgentHarnessStreamEvent>(
response,
processStreamEvent,
(event, meta) => {
if (typeof meta.seq === 'number') lastSeqRef.current = meta.seq
processAgentHarnessStreamEvent(event)
},
abortController.signal,
)
} catch (err) {
@@ -236,24 +404,45 @@ export function useAgentConversation(agentId: string, agentName: string) {
if (streamAbortRef.current === abortController) {
streamAbortRef.current = null
}
turnIdRef.current = null
lastSeqRef.current = null
onCompleteRef.current?.()
setStreaming(false)
}
}
const resetConversation = () => {
/**
* Stop button. The fetch abort only detaches *this* SSE subscriber
* now — the underlying turn would otherwise keep running on the
* server. So we explicitly cancel via the new endpoint, then unwind
* the local stream.
*/
const stop = async () => {
const turnId = turnIdRef.current ?? undefined
streamAbortRef.current?.abort()
streamAbortRef.current = null
try {
await cancelHarnessTurn(agentId, {
turnId,
reason: 'user pressed stop',
})
} catch {
// Best-effort — UI already aborted.
}
}
const resetConversation = () => {
void stop()
setTurns([])
setStreaming(false)
sessionKeyRef.current = crypto.randomUUID()
}
return {
turns,
streaming,
loading,
sessionKey: sessionKeyRef.current,
send,
stop,
resetConversation,
}
}

View File

@@ -0,0 +1,55 @@
import { describe, expect, it } from 'bun:test'
import { mapHarnessHistoryPage } from './harness-history-mapper'
describe('mapHarnessHistoryPage', () => {
it('maps rich harness history into chat history items', () => {
const page = mapHarnessHistoryPage({
agentId: 'agent-1',
sessionId: 'main',
items: [
{
id: 'agent:agent-1:main:1',
agentId: 'agent-1',
sessionId: 'main',
role: 'assistant',
text: 'Done.',
createdAt: 1000,
reasoning: { text: 'checking state' },
toolCalls: [
{
toolCallId: 'tool-1',
toolName: 'read_file',
status: 'completed',
input: { path: 'src/index.ts' },
output: 'file contents',
},
],
},
],
})
expect(page.items).toEqual([
{
id: 'agent:agent-1:main:1',
role: 'assistant',
text: 'Done.',
timestamp: 1000,
messageSeq: 1,
sessionKey: 'main',
source: 'user-chat',
reasoning: { text: 'checking state' },
toolCalls: [
{
toolCallId: 'tool-1',
toolName: 'read_file',
label: 'Read file',
subject: 'index.ts',
status: 'completed',
input: { path: 'src/index.ts' },
output: 'file contents',
},
],
},
])
})
})

View File

@@ -0,0 +1,29 @@
import { useQuery } from '@tanstack/react-query'
import { fetchHarnessAgentHistory } from '@/entrypoints/app/agents/useAgents'
import { useAgentServerUrl } from '@/lib/browseros/useBrowserOSProviders'
import type { AgentHistoryPageResponse } from './claw-chat-types'
import { mapHarnessHistoryPage } from './harness-history-mapper'
const HISTORY_QUERY_KEY = 'harness-agent-history'
export function useHarnessChatHistory(agentId: string, enabled = true) {
const {
baseUrl,
isLoading: urlLoading,
error: urlError,
} = useAgentServerUrl()
const query = useQuery<AgentHistoryPageResponse, Error>({
queryKey: [HISTORY_QUERY_KEY, baseUrl, agentId, 'main'],
queryFn: async () => {
return mapHarnessHistoryPage(await fetchHarnessAgentHistory(agentId))
},
enabled: Boolean(baseUrl) && !urlLoading && enabled && Boolean(agentId),
})
return {
...query,
error: query.error ?? urlError,
isLoading: query.isLoading || urlLoading,
}
}

View File

@@ -0,0 +1,42 @@
import { Bot, Cpu, Sparkles } from 'lucide-react'
import type { FC } from 'react'
import type { HarnessAgentAdapter } from './agent-harness-types'
/**
* Single icon component for any adapter the agent rail can render.
* Falls back to a generic bot when the adapter is unknown so future
* adapters land without a code change at the call site.
*/
interface AdapterIconProps {
adapter: HarnessAgentAdapter | 'unknown'
className?: string
}
export const AdapterIcon: FC<AdapterIconProps> = ({ adapter, className }) => {
switch (adapter) {
case 'claude':
// Claude Code — text-based agent, sparkles to evoke the "AI assistant" feel.
return <Sparkles className={className} aria-label="Claude Code" />
case 'codex':
// Codex — code-leaning, CPU mark.
return <Cpu className={className} aria-label="Codex" />
case 'openclaw':
// OpenClaw — bot/automation framing.
return <Bot className={className} aria-label="OpenClaw" />
default:
return <Bot className={className} aria-label="Agent" />
}
}
export function adapterLabel(adapter: HarnessAgentAdapter | 'unknown'): string {
switch (adapter) {
case 'claude':
return 'Claude Code'
case 'codex':
return 'Codex'
case 'openclaw':
return 'OpenClaw'
default:
return 'Agent'
}
}

View File

@@ -1,399 +0,0 @@
import {
ArrowLeft,
Bot,
CheckCircle2,
Loader2,
Send,
XCircle,
} from 'lucide-react'
import { type FC, useEffect, useRef, useState } from 'react'
import {
Message,
MessageContent,
MessageResponse,
} from '@/components/ai-elements/message'
import {
Reasoning,
ReasoningContent,
ReasoningTrigger,
} from '@/components/ai-elements/reasoning'
import { Button } from '@/components/ui/button'
import { Textarea } from '@/components/ui/textarea'
import { consumeSSEStream } from '@/lib/sse'
import {
buildChatHistoryFromTurns,
chatWithAgent,
type OpenClawStreamEvent,
} from './useOpenClaw'
interface ToolEntry {
id: string
name: string
status: 'running' | 'completed' | 'error'
durationMs?: number
}
type AssistantPart =
| { kind: 'thinking'; text: string; done: boolean }
| { kind: 'tool-batch'; tools: ToolEntry[] }
| { kind: 'text'; text: string }
interface ChatTurn {
id: string
userText: string
parts: AssistantPart[]
done: boolean
}
interface AgentChatProps {
agentId: string
agentName: string
onBack: () => void
}
export const AgentChat: FC<AgentChatProps> = ({
agentId,
agentName,
onBack,
}) => {
const [turns, setTurns] = useState<ChatTurn[]>([])
const [input, setInput] = useState('')
const [streaming, setStreaming] = useState(false)
const scrollRef = useRef<HTMLDivElement>(null)
const sessionKeyRef = useRef(crypto.randomUUID())
const streamAbortRef = useRef<AbortController | null>(null)
const textAccRef = useRef('')
const thinkAccRef = useRef('')
const scrollToBottom = () => {
scrollRef.current?.scrollTo(0, scrollRef.current.scrollHeight)
}
// biome-ignore lint/correctness/useExhaustiveDependencies: scroll on every turns change
useEffect(() => {
scrollToBottom()
}, [turns])
useEffect(() => {
return () => {
streamAbortRef.current?.abort()
}
}, [])
const updateCurrentTurnParts = (
updater: (parts: AssistantPart[]) => AssistantPart[],
) => {
setTurns((prev) => {
const last = prev[prev.length - 1]
if (!last) return prev
return [...prev.slice(0, -1), { ...last, parts: updater(last.parts) }]
})
}
const processStreamEvent = (event: OpenClawStreamEvent) => {
switch (event.type) {
case 'text-delta': {
const delta = (event.data.text as string) ?? ''
textAccRef.current += delta
const text = textAccRef.current
updateCurrentTurnParts((parts) => {
const last = parts[parts.length - 1]
if (last?.kind === 'text') {
return [...parts.slice(0, -1), { ...last, text }]
}
return [...parts, { kind: 'text', text }]
})
break
}
case 'thinking': {
const delta = (event.data.text as string) ?? ''
thinkAccRef.current += delta
const text = thinkAccRef.current
updateCurrentTurnParts((parts) => {
const idx = parts.findIndex((p) => p.kind === 'thinking' && !p.done)
if (idx >= 0) {
return [
...parts.slice(0, idx),
{ ...parts[idx], text, done: false },
...parts.slice(idx + 1),
]
}
return [...parts, { kind: 'thinking', text, done: false }]
})
break
}
case 'tool-start': {
const tool: ToolEntry = {
id: (event.data.toolCallId as string) ?? crypto.randomUUID(),
name: (event.data.toolName as string) ?? 'unknown',
status: 'running',
}
updateCurrentTurnParts((parts) => {
const last = parts[parts.length - 1]
if (last?.kind === 'tool-batch') {
return [
...parts.slice(0, -1),
{ ...last, tools: [...last.tools, tool] },
]
}
return [...parts, { kind: 'tool-batch', tools: [tool] }]
})
break
}
case 'tool-end': {
const toolId = event.data.toolCallId as string
const status =
(event.data.status as string) === 'error' ? 'error' : 'completed'
const durationMs = event.data.durationMs as number | undefined
updateCurrentTurnParts((parts) => {
for (let i = parts.length - 1; i >= 0; i--) {
const part = parts[i]
if (
part.kind === 'tool-batch' &&
part.tools.some((t) => t.id === toolId)
) {
const updatedTools = part.tools.map((t) =>
t.id === toolId
? {
...t,
status: status as ToolEntry['status'],
durationMs,
}
: t,
)
return [
...parts.slice(0, i),
{ ...part, tools: updatedTools },
...parts.slice(i + 1),
]
}
}
return parts
})
break
}
case 'done': {
updateCurrentTurnParts((parts) =>
parts.map((part) =>
part.kind === 'thinking' ? { ...part, done: true } : part,
),
)
setTurns((prev) => {
const last = prev[prev.length - 1]
if (!last) return prev
return [...prev.slice(0, -1), { ...last, done: true }]
})
break
}
case 'error': {
const msg =
(event.data.message as string) ??
(event.data.error as string) ??
'Unknown error'
updateCurrentTurnParts((parts) => [
...parts,
{ kind: 'text', text: `Error: ${msg}` },
])
break
}
}
}
const handleSend = async () => {
const text = input.trim()
if (!text || streaming) return
const history = buildChatHistoryFromTurns(turns)
const turn: ChatTurn = {
id: crypto.randomUUID(),
userText: text,
parts: [],
done: false,
}
setTurns((prev) => [...prev, turn])
setInput('')
setStreaming(true)
textAccRef.current = ''
thinkAccRef.current = ''
const abortController = new AbortController()
streamAbortRef.current = abortController
try {
const response = await chatWithAgent(
agentId,
text,
sessionKeyRef.current,
history,
abortController.signal,
)
if (!response.ok) {
const err = await response.text()
updateCurrentTurnParts((parts) => [
...parts,
{ kind: 'text', text: `Error: ${err}` },
])
return
}
await consumeSSEStream(
response,
processStreamEvent,
abortController.signal,
)
} catch (err) {
if (abortController.signal.aborted) return
const msg = err instanceof Error ? err.message : String(err)
updateCurrentTurnParts((parts) => [
...parts,
{ kind: 'text', text: `Error: ${msg}` },
])
} finally {
if (streamAbortRef.current === abortController) {
streamAbortRef.current = null
}
setStreaming(false)
}
}
return (
<div className="flex h-[calc(100vh-4rem)] flex-col">
<div className="flex items-center gap-2 border-b px-4 py-3">
<Button variant="ghost" size="icon" onClick={onBack}>
<ArrowLeft className="size-4" />
</Button>
<h2 className="font-semibold text-lg">{agentName}</h2>
</div>
<div ref={scrollRef} className="flex-1 space-y-4 overflow-y-auto p-4">
{turns.map((turn) => (
<div key={turn.id} className="space-y-3">
{/* User message */}
<Message from="user">
<MessageContent>
<pre className="whitespace-pre-wrap font-sans text-sm">
{turn.userText}
</pre>
</MessageContent>
</Message>
{/* Assistant response — all parts grouped */}
{turn.parts.length > 0 && (
<Message from="assistant">
<MessageContent>
{turn.parts.map((part, i) => {
const key = `${turn.id}-part-${i}`
switch (part.kind) {
case 'thinking':
return (
<Reasoning
key={key}
className="w-full"
isStreaming={!part.done}
defaultOpen={!part.done}
>
<ReasoningTrigger />
<ReasoningContent>{part.text}</ReasoningContent>
</Reasoning>
)
case 'tool-batch':
return (
<div key={key} className="w-full space-y-1">
{part.tools.map((tool) => (
<div
key={tool.id}
className="flex items-center gap-2 rounded-md border px-3 py-2 text-sm"
>
{tool.status === 'running' && (
<Loader2 className="size-3.5 animate-spin text-muted-foreground" />
)}
{tool.status === 'completed' && (
<CheckCircle2 className="size-3.5 text-green-500" />
)}
{tool.status === 'error' && (
<XCircle className="size-3.5 text-destructive" />
)}
<span className="font-mono text-xs">
{tool.name}
</span>
{tool.durationMs != null && (
<span className="ml-auto text-muted-foreground text-xs">
{(tool.durationMs / 1000).toFixed(1)}s
</span>
)}
</div>
))}
</div>
)
case 'text':
return (
<MessageResponse key={key}>
{part.text}
</MessageResponse>
)
default:
return null
}
})}
</MessageContent>
</Message>
)}
{/* Streaming indicator when waiting for first part */}
{!turn.done && turn.parts.length === 0 && streaming && (
<div className="flex gap-2">
<div className="flex h-7 w-7 shrink-0 items-center justify-center rounded-full bg-[var(--accent-orange)] text-white">
<Bot className="h-3.5 w-3.5" />
</div>
<div className="flex items-center gap-1 rounded-xl rounded-tl-none border border-border/50 bg-card px-3 py-2.5 shadow-sm">
<span className="h-1.5 w-1.5 animate-bounce rounded-full bg-[var(--accent-orange)] [animation-delay:-0.3s]" />
<span className="h-1.5 w-1.5 animate-bounce rounded-full bg-[var(--accent-orange)] [animation-delay:-0.15s]" />
<span className="h-1.5 w-1.5 animate-bounce rounded-full bg-[var(--accent-orange)]" />
</div>
</div>
)}
</div>
))}
</div>
<div className="border-t p-4">
<div className="flex gap-2">
<Textarea
value={input}
onChange={(e) => setInput(e.target.value)}
onKeyDown={(e) => {
if (e.key === 'Enter' && !e.shiftKey) {
e.preventDefault()
handleSend()
}
}}
placeholder="Send a message..."
className="min-h-[44px] resize-none"
rows={1}
/>
<Button
onClick={handleSend}
disabled={!input.trim() || streaming}
size="icon"
>
{streaming ? (
<Loader2 className="size-4 animate-spin" />
) : (
<Send className="size-4" />
)}
</Button>
</div>
</div>
</div>
)
}

View File

@@ -0,0 +1,176 @@
import { Loader2 } from 'lucide-react'
import { type FC, useMemo } from 'react'
import { AgentRowCard } from './AgentRowCard'
import { AgentsEmptyState } from './AgentsEmptyState'
import type {
HarnessAdapterDescriptor,
HarnessAgent,
HarnessAgentAdapter,
} from './agent-harness-types'
import type {
AgentAdapterHealth,
AgentRowData,
} from './agent-row/agent-row.types'
import type { AgentListItem } from './agents-page-types'
import type { AgentLiveness } from './LivenessDot'
interface AgentListProps {
agents: AgentListItem[]
/** Optional per-agent activity metadata, keyed by `agentId`. */
activity?: Record<
string,
{ status: AgentLiveness; lastUsedAt: number | null }
>
/** Lookup table from harness id → enriched agent record. */
harnessAgentLookup?: Map<string, HarnessAgent>
/** Adapter catalog (carries per-adapter health). */
adapters: HarnessAdapterDescriptor[]
loading: boolean
deletingAgentKey: string | null
onCreateAgent: () => void
onDeleteAgent: (agent: AgentListItem) => void
onPinToggle: (agent: AgentListItem, next: boolean) => void
}
export const AgentList: FC<AgentListProps> = ({
agents,
activity,
harnessAgentLookup,
adapters,
loading,
deletingAgentKey,
onCreateAgent,
onDeleteAgent,
onPinToggle,
}) => {
const adapterHealth = useMemo(() => {
const map = new Map<HarnessAgentAdapter, AgentAdapterHealth>()
for (const adapter of adapters) {
if (adapter.health) {
map.set(adapter.id, {
healthy: adapter.health.healthy,
reason: adapter.health.reason,
})
}
}
return map
}, [adapters])
// Sort: pinned rows first, then most recently used, then never-used
// agents in id-stable order. The gateway's `main` agent stays
// pinned-to-top when never touched so a fresh install has an
// obvious starting point.
const ordered = useMemo(() => {
const withMeta = agents.map((agent) => {
const harness = harnessAgentLookup?.get(agent.agentId)
return {
agent,
pinned: harness?.pinned ?? false,
lastUsedAt: activity?.[agent.agentId]?.lastUsedAt ?? null,
}
})
return withMeta
.sort((a, b) => {
if (a.pinned !== b.pinned) return a.pinned ? -1 : 1
const aSeed = a.agent.agentId === 'main' && a.lastUsedAt === null
const bSeed = b.agent.agentId === 'main' && b.lastUsedAt === null
if (aSeed && !bSeed) return -1
if (!aSeed && bSeed) return 1
const aValue = a.lastUsedAt ?? -Infinity
const bValue = b.lastUsedAt ?? -Infinity
if (aValue !== bValue) return bValue - aValue
return a.agent.agentId.localeCompare(b.agent.agentId)
})
.map((entry) => entry.agent)
}, [activity, agents, harnessAgentLookup])
if (loading && agents.length === 0) {
return (
<div className="flex h-36 items-center justify-center rounded-xl border border-border border-dashed bg-card/50">
<Loader2 className="size-5 animate-spin text-muted-foreground" />
</div>
)
}
if (agents.length === 0) {
return <AgentsEmptyState onCreateAgent={onCreateAgent} />
}
return (
<div className="grid gap-3">
{ordered.map((agent) => {
const harness = harnessAgentLookup?.get(agent.agentId)
const adapter: HarnessAgentAdapter | 'unknown' =
harness?.adapter ?? inferAdapterFromLabel(agent.runtimeLabel)
const data = buildRowData({
agent,
adapter,
harness,
activity: activity?.[agent.agentId],
adapterHealth:
adapterHealth.get(adapter as HarnessAgentAdapter) ?? null,
})
return (
<AgentRowCard
key={agent.key}
data={data}
deleting={deletingAgentKey === agent.key}
onDelete={onDeleteAgent}
onPinToggle={onPinToggle}
/>
)
})}
</div>
)
}
function inferAdapterFromLabel(label: string): HarnessAgentAdapter | 'unknown' {
const lower = label?.toLowerCase()
if (lower === 'claude code') return 'claude'
if (lower === 'codex') return 'codex'
if (lower === 'openclaw') return 'openclaw'
return 'unknown'
}
const ZERO_BUCKETS = (): number[] => Array.from({ length: 14 }, () => 0)
function buildRowData(input: {
agent: AgentListItem
adapter: HarnessAgentAdapter | 'unknown'
harness: HarnessAgent | undefined
activity: { status: AgentLiveness; lastUsedAt: number | null } | undefined
adapterHealth: AgentAdapterHealth | null
}): AgentRowData {
const { agent, adapter, harness, activity, adapterHealth } = input
return {
agent,
adapter,
modelLabel: deriveModelLabel(agent, harness),
reasoningEffort: harness?.reasoningEffort ?? null,
status: activity?.status ?? 'unknown',
lastUsedAt: activity?.lastUsedAt ?? harness?.lastUsedAt ?? null,
pinned: harness?.pinned ?? false,
cwd: harness?.cwd ?? null,
lastUserMessage: harness?.lastUserMessage ?? null,
tokens: harness?.tokens ?? null,
turnsByDay: harness?.turnsByDay ?? ZERO_BUCKETS(),
failedByDay: harness?.failedByDay ?? ZERO_BUCKETS(),
lastError: harness?.lastError ?? null,
lastErrorAt: harness?.lastErrorAt ?? null,
activeTurnId: harness?.activeTurnId ?? null,
adapterHealth,
}
}
function deriveModelLabel(
agent: AgentListItem,
harness: HarnessAgent | undefined,
): string | null {
// Prefer the agent rail's modelLabel when meaningful; harness's
// modelId is a stable identifier but the rail's `modelLabel`
// already maps to a friendly display string.
if (agent.modelLabel && agent.modelLabel !== 'default') {
return agent.modelLabel
}
return harness?.modelId ?? null
}

View File

@@ -0,0 +1,99 @@
import type { FC } from 'react'
import { cn } from '@/lib/utils'
import { AgentActions } from './agent-row/AgentActions'
import { AgentErrorPanel } from './agent-row/AgentErrorPanel'
import { AgentLastMessage } from './agent-row/AgentLastMessage'
import { AgentMetaRow } from './agent-row/AgentMetaRow'
import { AgentSummaryChips } from './agent-row/AgentSummaryChips'
import { AgentTile } from './agent-row/AgentTile'
import { AgentTitleRow } from './agent-row/AgentTitleRow'
import type {
AgentRowCallbacks,
AgentRowData,
} from './agent-row/agent-row.types'
interface AgentRowCardProps extends AgentRowCallbacks {
data: AgentRowData
/** Whether THIS agent is mid-delete; renders a spinner in the menu. */
deleting?: boolean
}
/**
* Composition shell for the agent rail. Owns no state; sub-components
* each handle their own micro-state (error-panel collapse, etc.) and
* emit callbacks (delete, pin/unpin) for the page to act on.
*
* The whole card carries state — not just the tile — so the row's
* border subtly tells the user what's going on at a glance:
* working → accent-orange border with a soft glow
* error → destructive border
* idle → muted border, lifts on hover
*/
export const AgentRowCard: FC<AgentRowCardProps> = ({
data,
deleting,
onDelete,
onPinToggle,
}) => {
return (
<div
className={cn(
// Layout-stable hover. No translate, no shadow change — both
// visibly perturb neighbouring rows. Only the border tint
// shifts on hover, and the rail's vertical rhythm stays
// exactly the same in every state.
'group rounded-xl border bg-card p-4 shadow-sm transition-colors',
data.status === 'working'
? 'border-[var(--accent-orange)]/40'
: data.status === 'error'
? 'border-destructive/40'
: 'border-border hover:border-[var(--accent-orange)]/30',
)}
>
<div className="flex items-start gap-4">
<AgentTile
adapter={data.adapter}
status={data.status}
lastUsedAt={data.lastUsedAt}
/>
<div className="min-w-0 flex-1">
<AgentTitleRow
agent={data.agent}
status={data.status}
pinned={data.pinned}
turnsByDay={data.turnsByDay}
failedByDay={data.failedByDay}
onPinToggle={(next) => onPinToggle(data.agent, next)}
/>
<AgentSummaryChips
adapter={data.adapter}
modelLabel={data.modelLabel}
reasoningEffort={data.reasoningEffort}
adapterHealth={data.adapterHealth}
/>
<AgentLastMessage message={data.lastUserMessage} />
<AgentMetaRow lastUsedAt={data.lastUsedAt} tokens={data.tokens} />
{data.status === 'error' && data.lastError && (
<AgentErrorPanel
agentId={data.agent.agentId}
message={data.lastError}
errorAt={data.lastErrorAt}
/>
)}
</div>
<AgentActions
agent={data.agent}
activeTurnId={data.activeTurnId}
deleting={deleting}
onDelete={onDelete}
/>
</div>
</div>
)
}

View File

@@ -5,14 +5,16 @@ import {
import { FitAddon } from '@xterm/addon-fit'
import { WebLinksAddon } from '@xterm/addon-web-links'
import { Terminal } from '@xterm/xterm'
import { ArrowLeft } from 'lucide-react'
import { type FC, useEffect, useRef } from 'react'
import { ArrowLeft, Check, Copy } from 'lucide-react'
import { type FC, useEffect, useRef, useState } from 'react'
import '@xterm/xterm/css/xterm.css'
import { Button } from '@/components/ui/button'
import { getAgentServerUrl } from '@/lib/browseros/helpers'
interface AgentTerminalProps {
onBack: () => void
initialCommand?: string
onSessionExit?: () => void
}
type TerminalServerMessage =
@@ -36,26 +38,22 @@ function resolveCssColor(variableName: string): string {
return color
}
function withAlpha(color: string, alpha: number): string {
const channels = color.match(/[\d.]+/g)
if (!channels || channels.length < 3) return color
const [red, green, blue] = channels
return `rgb(${red} ${green} ${blue} / ${alpha})`
}
function createTerminalTheme() {
const isDark = document.documentElement.classList.contains('dark')
const background = resolveCssColor('--background')
const foreground = resolveCssColor('--foreground')
const muted = resolveCssColor('--muted-foreground')
const accent = resolveCssColor('--accent-orange')
return {
background,
foreground,
cursor: foreground,
cursorAccent: background,
selectionBackground: withAlpha(accent, isDark ? 0.3 : 0.2),
// Solid terminal-standard selection colors. Deriving from a CSS var
// with alpha composed against the background produced near-white
// rectangles on light mode, making selection invisible.
selectionBackground: isDark ? '#3a4463' : '#b4d4f4',
selectionInactiveBackground: isDark ? '#2b3348' : '#d9e5f3',
selectionForeground: foreground,
black: isDark ? '#16131a' : '#1f1b22',
red: isDark ? '#ef8c7c' : '#c25544',
@@ -118,8 +116,38 @@ function parseTerminalMessage(data: unknown): TerminalServerMessage | null {
return null
}
export const AgentTerminal: FC<AgentTerminalProps> = ({ onBack }) => {
export const AgentTerminal: FC<AgentTerminalProps> = ({
onBack,
initialCommand,
onSessionExit,
}) => {
const containerRef = useRef<HTMLDivElement>(null)
const terminalRef = useRef<Terminal | null>(null)
// Refs keep the mount-once effect from tearing down the PTY when the
// parent re-renders with new inline callbacks.
const initialCommandRef = useRef(initialCommand)
const onSessionExitRef = useRef(onSessionExit)
initialCommandRef.current = initialCommand
onSessionExitRef.current = onSessionExit
const [copied, setCopied] = useState(false)
// Copy the current xterm selection to the browser clipboard. No-op
// if nothing is selected — users who want the whole buffer can
// Cmd+A first. Uses the browser clipboard, not the container's, so
// it works even when the running TUI has mouse tracking enabled
// (Opt+drag forces a selection regardless, see terminal config).
const handleCopy = async (): Promise<void> => {
const text = terminalRef.current?.getSelection()
if (!text) return
try {
await navigator.clipboard.writeText(text)
setCopied(true)
window.setTimeout(() => setCopied(false), 1500)
} catch {
// clipboard permission denied or unavailable — swallow, user will retry
}
}
useEffect(() => {
if (!containerRef.current) return
@@ -132,6 +160,34 @@ export const AgentTerminal: FC<AgentTerminalProps> = ({ onBack }) => {
lineHeight: 1.25,
scrollback: 8000,
theme: createTerminalTheme(),
// Opt+click+drag forces a native text selection even when the
// running TUI has mouse-tracking enabled (xterm would otherwise
// forward every click to the app and selection wouldn't work).
macOptionClickForcesSelection: true,
})
terminalRef.current = terminal
// Cmd+A → select all, Cmd+C → copy selection via the browser
// clipboard. Return false so xterm doesn't also forward the keys
// to the running program.
terminal.attachCustomKeyEventHandler((event) => {
if (event.type !== 'keydown') return true
const isMac = navigator.platform.toUpperCase().includes('MAC')
const mod = isMac ? event.metaKey : event.ctrlKey
if (!mod) return true
const key = event.key.toLowerCase()
if (key === 'a') {
terminal.selectAll()
return false
}
if (key === 'c') {
const sel = terminal.getSelection()
if (sel) {
void navigator.clipboard.writeText(sel)
return false
}
}
return true
})
const fitAddon = new FitAddon()
@@ -139,6 +195,12 @@ export const AgentTerminal: FC<AgentTerminalProps> = ({ onBack }) => {
terminal.loadAddon(new WebLinksAddon())
terminal.open(containerRef.current)
// React 18 StrictMode double-invokes effects in dev. Everything
// async inside this effect is scoped to an AbortController; the
// cleanup aborts it and any pending awaits bail out, so we never
// leak a second live WebSocket or duplicate xterm listeners.
const ac = new AbortController()
const cleanups: Array<() => void> = []
let ws: WebSocket | null = null
let sawExit = false
@@ -159,17 +221,28 @@ export const AgentTerminal: FC<AgentTerminalProps> = ({ onBack }) => {
sendMessage({ type: 'resize', cols, rows })
}
const connect = async () => {
const connect = async (): Promise<void> => {
const baseUrl = await getAgentServerUrl()
if (ac.signal.aborted) return
const wsUrl = new URL('/terminal/ws', baseUrl)
wsUrl.protocol = wsUrl.protocol === 'https:' ? 'wss:' : 'ws:'
ws = new WebSocket(wsUrl)
// If the effect was cleaned up between the await above and now,
// close the socket we just opened and bail.
if (ac.signal.aborted) {
ws.close()
ws = null
return
}
cleanups.push(() => ws?.close())
ws.onopen = () => {
fitAddon.fit()
terminal.focus()
sendResize()
const cmd = initialCommandRef.current
if (cmd) sendMessage({ type: 'input', data: `${cmd}\n` })
}
ws.onmessage = (event) => {
@@ -185,6 +258,7 @@ export const AgentTerminal: FC<AgentTerminalProps> = ({ onBack }) => {
terminal.write(
`\r\n\x1b[90m[session ended with exit ${message.exitCode}]\x1b[0m\r\n`,
)
onSessionExitRef.current?.()
}
}
@@ -200,49 +274,41 @@ export const AgentTerminal: FC<AgentTerminalProps> = ({ onBack }) => {
const inputDisposable = terminal.onData((data) => {
sendMessage({ type: 'input', data })
})
const resizeDisposable = terminal.onResize(({ cols, rows }) => {
sendResize(cols, rows)
})
return () => {
inputDisposable.dispose()
resizeDisposable.dispose()
}
cleanups.push(() => inputDisposable.dispose())
cleanups.push(() => resizeDisposable.dispose())
}
let disposeSocketBindings: (() => void) | undefined
void connect().then((disposeBindings) => {
disposeSocketBindings = disposeBindings
})
void connect()
const resizeObserver = new ResizeObserver(() => {
fitAddon.fit()
sendResize()
})
resizeObserver.observe(containerRef.current)
cleanups.push(() => resizeObserver.disconnect())
const themeObserver = new MutationObserver(() => {
applyTheme()
})
const themeObserver = new MutationObserver(() => applyTheme())
themeObserver.observe(document.documentElement, {
attributes: true,
attributeFilter: ['class'],
})
cleanups.push(() => themeObserver.disconnect())
return () => {
resizeObserver.disconnect()
themeObserver.disconnect()
disposeSocketBindings?.()
ws?.close()
ac.abort()
for (const dispose of cleanups) dispose()
terminal.dispose()
terminalRef.current = null
}
}, [])
return (
<div className="flex h-[calc(100dvh-10rem)] min-h-[32rem] w-full flex-col py-2 sm:min-h-[42rem] sm:py-4">
<div className="flex min-h-0 flex-1 flex-col overflow-hidden rounded-xl border border-border bg-card shadow-sm">
<div className="flex items-center gap-3 border-border border-b px-4 py-3 sm:px-6">
<div className="flex items-center justify-between gap-3 border-border border-b px-4 py-3 sm:px-6">
<div className="flex min-w-0 items-center gap-3">
<Button variant="ghost" size="icon" onClick={onBack}>
<ArrowLeft className="size-4" />
@@ -256,6 +322,14 @@ export const AgentTerminal: FC<AgentTerminalProps> = ({ onBack }) => {
</div>
</div>
</div>
<Button variant="outline" size="sm" onClick={handleCopy}>
{copied ? (
<Check className="mr-1 size-3.5" />
) : (
<Copy className="mr-1 size-3.5" />
)}
{copied ? 'Copied' : 'Copy'}
</Button>
</div>
<div className="min-h-0 flex-1 p-4 sm:p-6">
@@ -269,7 +343,7 @@ export const AgentTerminal: FC<AgentTerminalProps> = ({ onBack }) => {
</div>
</div>
<div className="min-h-0 flex-1 px-4 py-4 sm:px-5 sm:py-5">
<div className="min-h-0 flex-1 cursor-text px-4 py-4 sm:px-5 sm:py-5">
<div ref={containerRef} className="h-full w-full" />
</div>
</div>

View File

@@ -0,0 +1,32 @@
import { Bot, Plus } from 'lucide-react'
import type { FC } from 'react'
import { Button } from '@/components/ui/button'
interface AgentsEmptyStateProps {
onCreateAgent: () => void
}
export const AgentsEmptyState: FC<AgentsEmptyStateProps> = ({
onCreateAgent,
}) => {
return (
<div className="rounded-xl border border-border border-dashed bg-card/50 p-12 text-center">
<div className="mx-auto mb-4 flex h-12 w-12 items-center justify-center rounded-xl bg-[var(--accent-orange)]/10">
<Bot className="h-6 w-6 text-[var(--accent-orange)]" />
</div>
<h3 className="mb-1 font-semibold">No agents yet</h3>
<p className="mx-auto mb-4 max-w-sm text-muted-foreground text-sm">
Spin up an OpenClaw, Claude Code, or Codex agent to chat with, schedule,
or run in the background.
</p>
<Button
onClick={onCreateAgent}
variant="outline"
className="border-[var(--accent-orange)] bg-[var(--accent-orange)]/10 text-[var(--accent-orange)] hover:bg-[var(--accent-orange)]/20 hover:text-[var(--accent-orange)]"
>
<Plus className="mr-1.5 h-4 w-4" />
Create your first agent
</Button>
</div>
)
}

View File

@@ -0,0 +1,41 @@
import { Bot, Plus } from 'lucide-react'
import type { FC } from 'react'
import { Button } from '@/components/ui/button'
interface AgentsHeaderProps {
onCreateAgent: () => void
}
/**
* Mirrors the visual shape of `SoulHeader` and `ScheduledTasksHeader`
* so the page reads as part of the same family. Loose lifecycle
* controls that used to sit next to the title moved into
* `GatewayStatusBar` — they're OpenClaw-specific and don't apply to
* Claude/Codex agents.
*/
export const AgentsHeader: FC<AgentsHeaderProps> = ({ onCreateAgent }) => {
return (
<div className="rounded-xl border border-border bg-card p-6 shadow-sm transition-all hover:shadow-md">
<div className="flex items-start gap-4">
<div className="flex h-12 w-12 shrink-0 items-center justify-center rounded-xl bg-[var(--accent-orange)]/10">
<Bot className="h-6 w-6 text-[var(--accent-orange)]" />
</div>
<div className="flex-1">
<h2 className="mb-1 font-semibold text-xl">Agents</h2>
<p className="text-muted-foreground text-sm">
OpenClaw, Claude Code, and Codex agents chat, schedule, and run
them in the background.
</p>
</div>
<Button
onClick={onCreateAgent}
className="border-[var(--accent-orange)] bg-[var(--accent-orange)]/10 text-[var(--accent-orange)] hover:bg-[var(--accent-orange)]/20 hover:text-[var(--accent-orange)]"
variant="outline"
>
<Plus className="mr-1.5 h-4 w-4" />
New Agent
</Button>
</div>
</div>
)
}

View File

@@ -0,0 +1,206 @@
import { Loader2, RotateCcw, Terminal } from 'lucide-react'
import type { FC, ReactNode } from 'react'
import { Badge } from '@/components/ui/badge'
import { Button } from '@/components/ui/button'
import { Separator } from '@/components/ui/separator'
import {
Tooltip,
TooltipContent,
TooltipProvider,
TooltipTrigger,
} from '@/components/ui/tooltip'
import { cn } from '@/lib/utils'
import type { OpenClawStatus } from './useOpenClaw'
interface GatewayStatusBarProps {
status: OpenClawStatus | null
/** Disabled while a gateway lifecycle mutation is mid-flight. */
actionInProgress: boolean
onOpenTerminal: () => void
onRestart: () => void
}
/**
* Compact one-line status bar for the OpenClaw gateway. Renders the
* lifecycle pills (Running / Control plane connected) plus a Terminal
* escape hatch and a Restart Gateway action. Lives between the page
* header and the agent list when at least one OpenClaw agent is in
* the merged list; collapses to nothing for Claude/Codex-only setups.
*
* Status is sourced from `GET /agents`'s `gateway` field — the agents
* page no longer polls `/claw/status` directly. One endpoint, one
* 5s interval, no duplicate state.
*/
export const GatewayStatusBar: FC<GatewayStatusBarProps> = ({
status,
actionInProgress,
onOpenTerminal,
onRestart,
}) => {
if (!status) return null
const runningPill = pillForRuntimeStatus(status.status)
const controlPlanePill = pillForControlPlane(status.controlPlaneStatus)
return (
<div className="rounded-xl border border-border bg-card px-4 py-3 shadow-sm">
<div className="flex items-center gap-3 text-sm">
<span className="font-medium text-muted-foreground">
OpenClaw gateway
</span>
<Badge
variant={runningPill.variant}
className={cn('gap-1.5', runningPill.className)}
>
<span
className={cn(
'inline-block h-1.5 w-1.5 rounded-full',
runningPill.dot,
)}
/>
{runningPill.label}
</Badge>
<Badge
variant={controlPlanePill.variant}
className={cn('gap-1.5', controlPlanePill.className)}
>
<span
className={cn(
'inline-block h-1.5 w-1.5 rounded-full',
controlPlanePill.dot,
)}
/>
{controlPlanePill.label}
</Badge>
<Separator orientation="vertical" className="h-4" />
<WithTooltip label="Open a shell into the OpenClaw gateway container for raw CLI access (config edits, session inspection).">
<Button variant="ghost" size="sm" onClick={onOpenTerminal}>
<Terminal className="mr-1.5 h-3.5 w-3.5" />
Terminal
</Button>
</WithTooltip>
<WithTooltip label="Restart the OpenClaw gateway. Useful when the gateway is stuck or after editing provider config.">
<Button
variant="ghost"
size="sm"
onClick={onRestart}
disabled={actionInProgress}
className="ml-auto"
>
{actionInProgress ? (
<Loader2 className="mr-1.5 h-3.5 w-3.5 animate-spin" />
) : (
<RotateCcw className="mr-1.5 h-3.5 w-3.5" />
)}
Restart Gateway
</Button>
</WithTooltip>
</div>
</div>
)
}
const WithTooltip: FC<{ label: string; children: ReactNode }> = ({
label,
children,
}) => (
<TooltipProvider delayDuration={250}>
<Tooltip>
<TooltipTrigger asChild>{children}</TooltipTrigger>
<TooltipContent side="bottom" className="max-w-xs text-xs">
{label}
</TooltipContent>
</Tooltip>
</TooltipProvider>
)
type PillKind = {
variant: 'default' | 'secondary' | 'outline' | 'destructive'
label: string
dot: string
className?: string
}
function pillForRuntimeStatus(status: OpenClawStatus['status']): PillKind {
switch (status) {
case 'running':
return {
variant: 'secondary',
label: 'Running',
dot: 'bg-emerald-500',
className: 'bg-emerald-50 text-emerald-900 hover:bg-emerald-50',
}
case 'starting':
return {
variant: 'secondary',
label: 'Starting',
dot: 'bg-amber-500 animate-pulse',
className: 'bg-amber-50 text-amber-900 hover:bg-amber-50',
}
case 'stopped':
return {
variant: 'outline',
label: 'Stopped',
dot: 'bg-muted-foreground/40',
}
case 'error':
return {
variant: 'destructive',
label: 'Error',
dot: 'bg-destructive-foreground',
}
default:
return {
variant: 'outline',
label: 'Unknown',
dot: 'bg-muted-foreground/40',
}
}
}
function pillForControlPlane(
status: OpenClawStatus['controlPlaneStatus'],
): PillKind {
switch (status) {
case 'connected':
return {
variant: 'secondary',
label: 'Control plane connected',
dot: 'bg-emerald-500',
className: 'bg-emerald-50 text-emerald-900 hover:bg-emerald-50',
}
case 'connecting':
return {
variant: 'secondary',
label: 'Connecting',
dot: 'bg-amber-500 animate-pulse',
className: 'bg-amber-50 text-amber-900 hover:bg-amber-50',
}
case 'reconnecting':
return {
variant: 'secondary',
label: 'Reconnecting',
dot: 'bg-amber-500 animate-pulse',
className: 'bg-amber-50 text-amber-900 hover:bg-amber-50',
}
case 'recovering':
return {
variant: 'secondary',
label: 'Recovering',
dot: 'bg-amber-500 animate-pulse',
className: 'bg-amber-50 text-amber-900 hover:bg-amber-50',
}
case 'failed':
return {
variant: 'destructive',
label: 'Needs attention',
dot: 'bg-destructive-foreground',
}
default:
return {
variant: 'outline',
label: 'Disconnected',
dot: 'bg-muted-foreground/40',
}
}
}

View File

@@ -0,0 +1,83 @@
import type { FC } from 'react'
import {
Tooltip,
TooltipContent,
TooltipProvider,
TooltipTrigger,
} from '@/components/ui/tooltip'
import { cn } from '@/lib/utils'
export type AgentLiveness = 'working' | 'idle' | 'asleep' | 'error' | 'unknown'
interface LivenessDotProps {
status: AgentLiveness
/**
* Optional human-friendly secondary line, e.g. "Idle for 4 min" or
* "Asleep — no activity for 22 min". When absent the tooltip just
* reads the status label.
*/
detail?: string
className?: string
}
const VARIANT: Record<
AgentLiveness,
{ dot: string; ring: string; label: string }
> = {
working: {
// Animated amber pulse + soft halo so the eye catches an active
// agent in a long list without the dot screaming for attention.
dot: 'bg-amber-500 animate-pulse',
ring: 'ring-2 ring-amber-200',
label: 'Working on a turn',
},
idle: {
dot: 'bg-emerald-500',
ring: 'ring-2 ring-emerald-100',
label: 'Idle',
},
asleep: {
dot: 'bg-muted-foreground/40',
ring: 'ring-2 ring-muted',
label: 'Asleep',
},
error: {
dot: 'bg-destructive',
ring: 'ring-2 ring-destructive/30',
label: 'Attention',
},
unknown: {
dot: 'bg-muted-foreground/30',
ring: 'ring-2 ring-muted',
label: 'Status unknown',
},
}
export const LivenessDot: FC<LivenessDotProps> = ({
status,
detail,
className,
}) => {
const variant = VARIANT[status]
return (
<TooltipProvider delayDuration={150}>
<Tooltip>
<TooltipTrigger asChild>
<span
role="img"
aria-label={detail ?? variant.label}
className={cn(
'inline-block h-3 w-3 rounded-full',
variant.dot,
variant.ring,
className,
)}
/>
</TooltipTrigger>
<TooltipContent side="right" className="text-xs">
{detail ?? variant.label}
</TooltipContent>
</Tooltip>
</TooltipProvider>
)
}

View File

@@ -0,0 +1,260 @@
import { AlertCircle, Loader2 } from 'lucide-react'
import type { FC } from 'react'
import { Alert, AlertDescription, AlertTitle } from '@/components/ui/alert'
import { Button } from '@/components/ui/button'
import {
Dialog,
DialogContent,
DialogFooter,
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog'
import { Input } from '@/components/ui/input'
import { Label } from '@/components/ui/label'
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select'
import type {
HarnessAdapterDescriptor,
HarnessAgentAdapter,
} from './agent-harness-types'
import type { CreateAgentRuntime, ProviderOption } from './agents-page-types'
import { ProviderSelector } from './OpenClawControls'
import {
type OpenClawCliProvider,
type OpenClawCliProviderAuthStatus,
OpenClawCliProviderStatusPanel,
} from './openclaw-cli-providers'
interface NewAgentDialogProps {
adapters: HarnessAdapterDescriptor[]
canManageOpenClaw: boolean
createError: string | null
createRuntime: CreateAgentRuntime
creating: boolean
defaultProviderId: string
harnessAdapterId: HarnessAgentAdapter
harnessModelId: string
harnessReasoningEffort: string
name: string
open: boolean
providers: ProviderOption[]
selectedCliProvider: OpenClawCliProvider | undefined
selectedProviderId: string
cliAuthError: Error | null
cliAuthLoading: boolean
cliAuthStatus: OpenClawCliProviderAuthStatus | undefined
onConnectCliProvider: () => void
onCreate: () => void
onOpenChange: (open: boolean) => void
onRuntimeChange: (runtime: CreateAgentRuntime) => void
onHarnessAdapterChange: (adapter: HarnessAgentAdapter) => void
onHarnessModelChange: (modelId: string) => void
onHarnessReasoningChange: (reasoningEffort: string) => void
onNameChange: (name: string) => void
onProviderChange: (providerId: string) => void
}
export const NewAgentDialog: FC<NewAgentDialogProps> = ({
adapters,
canManageOpenClaw,
createError,
createRuntime,
creating,
defaultProviderId,
harnessAdapterId,
harnessModelId,
harnessReasoningEffort,
name,
open,
providers,
selectedCliProvider,
selectedProviderId,
cliAuthError,
cliAuthLoading,
cliAuthStatus,
onConnectCliProvider,
onCreate,
onOpenChange,
onRuntimeChange,
onHarnessAdapterChange,
onHarnessModelChange,
onHarnessReasoningChange,
onNameChange,
onProviderChange,
}) => {
const selectedHarnessAdapter =
adapters.find((adapter) => adapter.id === harnessAdapterId) ?? adapters[0]
const isHarnessRuntime = createRuntime !== 'openclaw'
const openClawBlocked = createRuntime === 'openclaw' && !canManageOpenClaw
const cliBlocked =
createRuntime === 'openclaw' &&
!!selectedCliProvider &&
!cliAuthStatus?.loggedIn
const canCreate =
Boolean(name.trim()) &&
!creating &&
!openClawBlocked &&
!cliBlocked &&
(createRuntime === 'openclaw'
? providers.length > 0
: Boolean(selectedHarnessAdapter))
return (
<Dialog open={open} onOpenChange={onOpenChange}>
<DialogContent>
<DialogHeader>
<DialogTitle>New Agent</DialogTitle>
</DialogHeader>
<div className="grid gap-4 py-2">
{createError ? (
<Alert variant="destructive">
<AlertCircle className="size-4" />
<AlertTitle>Create failed</AlertTitle>
<AlertDescription>{createError}</AlertDescription>
</Alert>
) : null}
<div className="grid gap-2">
<Label htmlFor="agent-name">Name</Label>
<Input
id="agent-name"
value={name}
onChange={(event) => onNameChange(event.target.value)}
placeholder={
createRuntime === 'openclaw' ? 'research-agent' : 'Review bot'
}
onKeyDown={(event) => {
if (event.key === 'Enter' && canCreate) onCreate()
}}
/>
</div>
<div className="grid gap-2">
<Label htmlFor="agent-runtime">Adapter</Label>
<Select
value={createRuntime}
onValueChange={(value) => {
if (
value === 'openclaw' ||
value === 'claude' ||
value === 'codex'
) {
onRuntimeChange(value)
if (value !== 'openclaw') onHarnessAdapterChange(value)
}
}}
>
<SelectTrigger id="agent-runtime">
<SelectValue />
</SelectTrigger>
<SelectContent>
{adapters.map((adapter) => (
<SelectItem key={adapter.id} value={adapter.id}>
{adapter.name}
</SelectItem>
))}
</SelectContent>
</Select>
</div>
{createRuntime === 'openclaw' ? (
<>
{openClawBlocked ? (
<Alert>
<AlertCircle className="size-4" />
<AlertTitle>OpenClaw is not ready</AlertTitle>
<AlertDescription>
Start or set up the OpenClaw gateway before creating an
OpenClaw agent.
</AlertDescription>
</Alert>
) : null}
<ProviderSelector
providers={providers}
defaultProviderId={defaultProviderId}
selectedId={selectedProviderId}
onSelect={onProviderChange}
hideApiKeyHint={!!selectedCliProvider}
/>
{selectedCliProvider ? (
<OpenClawCliProviderStatusPanel
provider={selectedCliProvider}
status={cliAuthStatus}
loading={cliAuthLoading}
fetchError={cliAuthError}
onConnect={onConnectCliProvider}
/>
) : null}
</>
) : null}
{isHarnessRuntime ? (
<>
<div className="grid gap-2">
<Label htmlFor="harness-model">Model</Label>
<Select
value={harnessModelId}
onValueChange={onHarnessModelChange}
>
<SelectTrigger id="harness-model">
<SelectValue />
</SelectTrigger>
<SelectContent>
{(selectedHarnessAdapter?.models ?? []).map((model) => (
<SelectItem key={model.id} value={model.id}>
{model.label}
</SelectItem>
))}
</SelectContent>
</Select>
</div>
<div className="grid gap-2">
<Label htmlFor="harness-effort">Reasoning</Label>
<Select
value={harnessReasoningEffort}
onValueChange={onHarnessReasoningChange}
>
<SelectTrigger id="harness-effort">
<SelectValue />
</SelectTrigger>
<SelectContent>
{(selectedHarnessAdapter?.reasoningEfforts ?? []).map(
(effort) => (
<SelectItem key={effort.id} value={effort.id}>
{effort.label}
</SelectItem>
),
)}
</SelectContent>
</Select>
</div>
</>
) : null}
</div>
<DialogFooter>
<Button
variant="outline"
onClick={() => onOpenChange(false)}
disabled={creating}
>
Cancel
</Button>
<Button disabled={!canCreate} onClick={onCreate}>
{creating ? <Loader2 className="mr-2 size-4 animate-spin" /> : null}
Create
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
)
}

View File

@@ -0,0 +1,387 @@
import {
AlertCircle,
Cpu,
Loader2,
Plus,
RefreshCw,
ShieldAlert,
Square,
TerminalSquare,
WifiOff,
Wrench,
} from 'lucide-react'
import type { FC } from 'react'
import { Alert, AlertDescription, AlertTitle } from '@/components/ui/alert'
import { Badge } from '@/components/ui/badge'
import { Button } from '@/components/ui/button'
import { Card, CardContent } from '@/components/ui/card'
import { Label } from '@/components/ui/label'
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select'
import type { ProviderOption } from './agents-page-types'
import {
CONTROL_PLANE_COPY,
FALLBACK_CONTROL_PLANE_COPY,
} from './agents-page-types'
import type { getControlPlaneCopy } from './agents-page-utils'
import type { OpenClawStatus } from './useOpenClaw'
const StatusBadge: FC<{ status: OpenClawStatus['status'] }> = ({ status }) => {
const variants: Record<
OpenClawStatus['status'],
{
variant: 'default' | 'secondary' | 'outline' | 'destructive'
label: string
}
> = {
running: { variant: 'default', label: 'Running' },
starting: { variant: 'secondary', label: 'Starting...' },
stopped: { variant: 'outline', label: 'Stopped' },
error: { variant: 'destructive', label: 'Error' },
uninitialized: { variant: 'outline', label: 'Not Set Up' },
}
const current = variants[status] ?? {
variant: 'outline' as const,
label: 'Unknown',
}
return <Badge variant={current.variant}>{current.label}</Badge>
}
const ControlPlaneBadge: FC<{
status: OpenClawStatus['controlPlaneStatus']
}> = ({ status }) => {
const current = CONTROL_PLANE_COPY[status] ?? FALLBACK_CONTROL_PLANE_COPY
return <Badge variant={current.badgeVariant}>{current.badgeLabel}</Badge>
}
interface ProviderSelectorProps {
providers: ProviderOption[]
defaultProviderId: string
selectedId: string
onSelect: (id: string) => void
hideApiKeyHint?: boolean
}
export const ProviderSelector: FC<ProviderSelectorProps> = ({
providers,
defaultProviderId,
selectedId,
onSelect,
hideApiKeyHint,
}) => {
if (providers.length === 0) {
return (
<div className="space-y-2">
<p className="font-medium text-sm">LLM Provider</p>
<p className="text-muted-foreground text-sm">
No compatible LLM providers configured.{' '}
<a href="#/settings/ai" className="underline">
Add one in AI settings
</a>{' '}
first.
</p>
</div>
)
}
return (
<div className="space-y-2">
<Label htmlFor="provider-select">LLM Provider</Label>
<Select value={selectedId} onValueChange={onSelect}>
<SelectTrigger id="provider-select">
<SelectValue placeholder="Select a provider" />
</SelectTrigger>
<SelectContent>
{providers.map((provider) => (
<SelectItem key={provider.id} value={provider.id}>
{provider.name} - {provider.modelId}
{provider.id === defaultProviderId ? ' (default)' : ''}
</SelectItem>
))}
</SelectContent>
</Select>
{!hideApiKeyHint && (
<p className="text-muted-foreground text-xs">
Uses your existing API key from BrowserOS settings. The key is passed
to the container and never leaves your machine.
</p>
)}
</div>
)
}
interface AgentsPageHeaderProps {
actionInProgress: boolean
controlPlaneBusy: boolean
reconnecting: boolean
status: OpenClawStatus | null
onCreateAgent: () => void
onOpenTerminal: () => void
onReconnect: () => void
onRefresh: () => void
onRestart: () => void
onStop: () => void
}
export const AgentsPageHeader: FC<AgentsPageHeaderProps> = ({
actionInProgress,
controlPlaneBusy,
reconnecting,
status,
onCreateAgent,
onOpenTerminal,
onReconnect,
onRefresh,
onRestart,
onStop,
}) => (
<div className="flex flex-wrap items-center justify-between gap-3">
<div>
<h1 className="font-semibold text-2xl tracking-normal">Agents</h1>
<p className="text-muted-foreground text-sm">
OpenClaw, Claude Code, and Codex agents
</p>
</div>
<div className="flex flex-wrap items-center gap-2">
{status ? (
<>
<StatusBadge status={status.status} />
{status.status !== 'uninitialized' && (
<ControlPlaneBadge status={status.controlPlaneStatus} />
)}
</>
) : null}
{status?.status === 'running' &&
status.controlPlaneStatus !== 'connected' ? (
<Button
variant="outline"
onClick={onReconnect}
disabled={actionInProgress || controlPlaneBusy}
>
{reconnecting ? (
<Loader2 className="mr-2 size-4 animate-spin" />
) : (
<RefreshCw className="mr-2 size-4" />
)}
Retry Connection
</Button>
) : null}
{status?.status === 'running' ? (
<>
<Button
variant="ghost"
size="icon"
onClick={onRestart}
disabled={actionInProgress}
title="Restart gateway"
>
<RefreshCw className="size-4" />
</Button>
<Button
variant="ghost"
size="icon"
onClick={onStop}
disabled={actionInProgress}
title="Stop gateway"
>
<Square className="size-4" />
</Button>
<Button variant="outline" onClick={onOpenTerminal}>
<TerminalSquare className="mr-2 size-4" />
Terminal
</Button>
</>
) : null}
<Button variant="ghost" size="icon" onClick={onRefresh} title="Refresh">
<RefreshCw className="size-4" />
</Button>
<Button onClick={onCreateAgent}>
<Plus className="mr-2 size-4" />
New Agent
</Button>
</div>
</div>
)
export function LifecycleAlert({ message }: { message: string }) {
return (
<Alert>
<Loader2 className="size-4 animate-spin" />
<AlertTitle>{message}</AlertTitle>
</Alert>
)
}
export function InlineErrorAlert({
message,
onDismiss,
}: {
message: string
onDismiss: () => void
}) {
return (
<Alert variant="destructive">
<AlertCircle className="size-4" />
<AlertTitle>Agent action failed</AlertTitle>
<AlertDescription>
<p>{message}</p>
<div className="mt-2">
<Button variant="outline" size="sm" onClick={onDismiss}>
Dismiss
</Button>
</div>
</AlertDescription>
</Alert>
)
}
interface ControlPlaneAlertProps {
actionInProgress: boolean
controlPlaneBusy: boolean
controlPlaneCopy: ReturnType<typeof getControlPlaneCopy>
reconnecting: boolean
recoveryDetail: string | null
status: OpenClawStatus
onReconnect: () => void
onRestart: () => void
}
export const ControlPlaneAlert: FC<ControlPlaneAlertProps> = ({
actionInProgress,
controlPlaneBusy,
controlPlaneCopy,
reconnecting,
recoveryDetail,
status,
onReconnect,
onRestart,
}) => (
<Alert
variant={status.controlPlaneStatus === 'failed' ? 'destructive' : 'default'}
>
{status.controlPlaneStatus === 'failed' ? (
<ShieldAlert className="size-4" />
) : status.controlPlaneStatus === 'recovering' ? (
<Wrench className="size-4" />
) : (
<WifiOff className="size-4" />
)}
<AlertTitle>{controlPlaneCopy.title}</AlertTitle>
<AlertDescription>
<p>{controlPlaneCopy.description}</p>
{recoveryDetail ? <p>{recoveryDetail}</p> : null}
<div className="mt-2 flex flex-wrap gap-2">
<Button
variant="outline"
size="sm"
onClick={onReconnect}
disabled={actionInProgress || controlPlaneBusy}
>
{reconnecting ? (
<Loader2 className="mr-2 size-4 animate-spin" />
) : (
<RefreshCw className="mr-2 size-4" />
)}
Retry Connection
</Button>
<Button
variant="outline"
size="sm"
onClick={onRestart}
disabled={actionInProgress}
>
Restart Gateway
</Button>
</div>
</AlertDescription>
</Alert>
)
interface GatewayStateCardsProps {
actionInProgress: boolean
status: OpenClawStatus | null
onOpenSetup: () => void
onRestart: () => void
onStart: () => void
}
export const GatewayStateCards: FC<GatewayStateCardsProps> = ({
actionInProgress,
status,
onOpenSetup,
onRestart,
onStart,
}) => (
<>
{status?.status === 'uninitialized' ? (
<Card>
<CardContent className="flex flex-col items-center gap-4 py-12">
<Cpu className="size-12 text-muted-foreground" />
<div className="text-center">
<h3 className="font-semibold text-lg">Set Up OpenClaw</h3>
<p className="text-muted-foreground text-sm">
{status.podmanAvailable
? 'Create a local BrowserOS VM to run autonomous agents with full tool access.'
: 'BrowserOS VM runtime is unavailable on this system.'}
</p>
</div>
{status.podmanAvailable ? (
<Button onClick={onOpenSetup}>Set Up Now</Button>
) : null}
</CardContent>
</Card>
) : null}
{status?.status === 'stopped' ? (
<Card>
<CardContent className="flex flex-col items-center gap-4 py-12">
<Cpu className="size-12 text-muted-foreground" />
<div className="text-center">
<h3 className="font-semibold text-lg">Gateway Stopped</h3>
<p className="text-muted-foreground text-sm">
The OpenClaw gateway is not running.
</p>
</div>
<Button onClick={onStart} disabled={actionInProgress}>
Start Gateway
</Button>
</CardContent>
</Card>
) : null}
{status?.status === 'error' ? (
<Card className="border-destructive">
<CardContent className="flex flex-col items-center gap-4 py-12">
<AlertCircle className="size-12 text-destructive" />
<div className="text-center">
<h3 className="font-semibold text-lg">Gateway Error</h3>
<p className="text-muted-foreground text-sm">
{status.error ?? status.lastGatewayError}
</p>
</div>
<div className="flex gap-2">
<Button onClick={onStart} disabled={actionInProgress}>
Start Gateway
</Button>
<Button
variant="outline"
onClick={onRestart}
disabled={actionInProgress}
>
Restart Gateway
</Button>
</div>
</CardContent>
</Card>
) : null}
</>
)

View File

@@ -0,0 +1,76 @@
import { Loader2 } from 'lucide-react'
import type { FC } from 'react'
import { Button } from '@/components/ui/button'
import {
Dialog,
DialogContent,
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog'
import type { ProviderOption } from './agents-page-types'
import { ProviderSelector } from './OpenClawControls'
import type { OpenClawCliProvider } from './openclaw-cli-providers'
interface SetupOpenClawDialogProps {
defaultProviderId: string
open: boolean
providers: ProviderOption[]
selectedProviderId: string
selectedCliProvider: OpenClawCliProvider | undefined
settingUp: boolean
onOpenChange: (open: boolean) => void
onProviderChange: (providerId: string) => void
onSetup: () => void
}
export const SetupOpenClawDialog: FC<SetupOpenClawDialogProps> = ({
defaultProviderId,
open,
providers,
selectedProviderId,
selectedCliProvider,
settingUp,
onOpenChange,
onProviderChange,
onSetup,
}) => (
<Dialog open={open} onOpenChange={onOpenChange}>
<DialogContent>
<DialogHeader>
<DialogTitle>Set Up OpenClaw</DialogTitle>
</DialogHeader>
<div className="space-y-4 py-2">
<ProviderSelector
providers={providers}
defaultProviderId={defaultProviderId}
selectedId={selectedProviderId}
onSelect={onProviderChange}
hideApiKeyHint={!!selectedCliProvider}
/>
{selectedCliProvider ? (
<p className="rounded-md border border-border bg-muted/30 px-3 py-2 text-muted-foreground text-xs">
{selectedCliProvider.description}. Clicking{' '}
<span className="font-medium">Set Up &amp; Start</span> starts the
gateway and opens a terminal to sign in.
</p>
) : null}
<Button
onClick={onSetup}
disabled={settingUp || providers.length === 0}
className="w-full"
>
{settingUp ? (
<>
<Loader2 className="mr-2 size-4 animate-spin" />
Setting up...
</>
) : (
'Set Up & Start'
)}
</Button>
</div>
</DialogContent>
</Dialog>
)

View File

@@ -0,0 +1,4 @@
export function buildAgentApiUrl(baseUrl: string, path: string): string {
const normalizedPath = path === '/' ? '' : path
return `${baseUrl}/agents${normalizedPath}`
}

View File

@@ -0,0 +1,107 @@
import type { AgentListItem } from './agents-page-types'
import type { AgentLiveness } from './LivenessDot'
/**
* Display rules for the redesigned agent rows. Pure helpers — no React,
* no API calls — so they're trivial to unit-test and the row card stays
* focused on layout.
*/
const UUID_PATTERN =
/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i
const OC_UUID_PATTERN =
/^oc-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i
/**
* The agent rail used to render whatever the gateway returned for `name`.
* Post-migration that's frequently the agent's UUID — readable to nobody.
* Prefer the explicit `name` when it differs meaningfully from the id;
* otherwise fall back to a short prefix users can recognize on second
* glance.
*/
export function displayName(agent: AgentListItem): string {
const name = agent.name?.trim()
const id = agent.agentId
if (!name || name === id) {
if (OC_UUID_PATTERN.test(id)) return id.slice(0, 11) // "oc-XXXXXXXX"
if (UUID_PATTERN.test(id)) return id.slice(0, 8)
return id
}
return name
}
export function canDelete(agent: AgentListItem): boolean {
// The gateway's protected `main` agent must not be deletable. The
// server enforces this too, but disabling the menu item avoids users
// hitting an opaque 400.
if (agent.agentId === 'main') return false
return agent.canDelete
}
/**
* Rename will be wired to a future `PATCH /agents/:id` endpoint. The
* legacy `/claw/agents` create flow named the agent on the gateway via
* the `name` field but the field isn't editable post-create today.
*/
export function canRename(_agent: AgentListItem): boolean {
return false
}
/**
* The detail line carries the agent's workspace path. The `detail`
* field on AgentListItem already holds it for OpenClaw entries
* (`/home/node/.openclaw/workspace-...`); for harness agents it's the
* synthetic `<adapter>:main` marker that's not informative — hide it.
*/
export function workspaceLabel(agent: AgentListItem): string | null {
if (!agent.detail) return null
if (/^(claude|codex|openclaw):main$/.test(agent.detail)) return null
return agent.detail
}
const ONE_MINUTE = 60_000
const ONE_HOUR = 60 * ONE_MINUTE
const ONE_DAY = 24 * ONE_HOUR
/**
* Lightweight relative-time formatter. We don't want to drag in
* `dayjs/relativeTime` just for a few labels.
*/
export function formatRelativeTime(epochMs: number | null): string {
if (epochMs === null || !Number.isFinite(epochMs)) return 'never'
const diff = Math.max(0, Date.now() - epochMs)
if (diff < ONE_MINUTE) return 'just now'
if (diff < ONE_HOUR) {
const m = Math.floor(diff / ONE_MINUTE)
return `${m} min ago`
}
if (diff < ONE_DAY) {
const h = Math.floor(diff / ONE_HOUR)
return h === 1 ? '1 hr ago' : `${h} hr ago`
}
const d = Math.floor(diff / ONE_DAY)
return d === 1 ? '1 day ago' : `${d} days ago`
}
/**
* Tooltip-friendly description of a row's current liveness state.
* Returns `undefined` when the state has nothing extra to add (e.g.
* `unknown` with no timestamp).
*/
export function livenessDetail(
status: AgentLiveness,
lastUsedAt: number | null | undefined,
): string | undefined {
if (lastUsedAt == null) return undefined
const diffMin = Math.floor((Date.now() - lastUsedAt) / 60_000)
if (status === 'idle') return `Idle for ${Math.max(0, diffMin)} min`
if (status === 'asleep') {
if (diffMin < 60) return `Asleep — quiet for ${diffMin} min`
const hr = Math.floor(diffMin / 60)
return `Asleep — quiet for ${hr} hr`
}
if (status === 'working') return 'Working on a turn'
if (status === 'error') return 'Attention — last turn failed'
return undefined
}

View File

@@ -0,0 +1,156 @@
import type { AgentEntry } from './useOpenClaw'
export type HarnessAgentAdapter = 'claude' | 'codex' | 'openclaw'
export type AgentHarnessStreamEvent =
| {
type: 'text_delta'
text: string
stream: 'output' | 'thought'
rawType?: string
}
| {
type: 'tool_call'
text: string
title: string
id?: string
status?: string
rawType?: string
}
| {
type: 'status'
text: string
rawType?: string
}
| {
type: 'done'
text?: string
stopReason?: string
}
| {
type: 'error'
message: string
code?: string
}
export type HarnessAgentLiveness = 'working' | 'idle' | 'asleep' | 'error'
export interface HarnessAgent {
id: string
name: string
adapter: HarnessAgentAdapter
modelId?: string
reasoningEffort?: string
permissionMode: 'approve-all'
sessionKey: string
createdAt: number
updatedAt: number
/**
* Server-derived liveness state. When the listing endpoint hasn't
* been enriched yet (older deployments) this is undefined and the UI
* falls back to `unknown`.
*/
status?: HarnessAgentLiveness
/**
* Wall-clock ms of the last persisted turn. `null` for never-used
* agents. Drives the recency sort and the "Last used X min ago" copy.
*/
lastUsedAt?: number | null
/** Pinned agents float to the top of the list. Defaults to `false`. */
pinned?: boolean
/** First non-blank line of the most recent user message; null if none. */
lastUserMessage?: string | null
/** Working directory the agent runs in; null when no session record yet. */
cwd?: string | null
/** Cumulative + 7-day rolling token usage; null when no record. */
tokens?: {
last7d: { input: number; output: number; requestCount: number }
cumulative: { input: number; output: number }
} | null
turnsByDay?: number[]
failedByDay?: number[]
lastError?: string | null
lastErrorAt?: number | null
/** When non-null, an in-flight turn this row can be resumed from. */
activeTurnId?: string | null
/** Persistent FIFO queue of messages waiting for this agent. */
queue?: HarnessQueuedMessage[]
}
export interface HarnessQueuedMessageAttachment {
mediaType: string
data: string
}
export interface HarnessQueuedMessage {
id: string
createdAt: number
message: string
attachments?: ReadonlyArray<HarnessQueuedMessageAttachment>
}
export interface HarnessAdapterHealth {
healthy: boolean
reason?: string
checkedAt: number
}
export interface HarnessAdapterDescriptor {
id: HarnessAgentAdapter
name: string
defaultModelId: string
defaultReasoningEffort: string
modelControl: 'runtime-supported' | 'best-effort'
models: Array<{ id: string; label: string; recommended?: boolean }>
reasoningEfforts: Array<{ id: string; label: string; recommended?: boolean }>
health?: HarnessAdapterHealth
}
export interface CreateHarnessAgentInput {
name: string
adapter: HarnessAgentAdapter
modelId?: string
reasoningEffort?: string
}
export interface HarnessHistoryReasoning {
text: string
durationMs?: number
}
export interface HarnessHistoryToolCall {
toolCallId?: string
toolName: string
status: 'pending' | 'running' | 'completed' | 'failed'
input?: unknown
output?: unknown
error?: string
durationMs?: number
}
export interface HarnessHistoryEntry {
id: string
agentId: string
sessionId: 'main'
role: 'user' | 'assistant'
text: string
createdAt: number
reasoning?: HarnessHistoryReasoning
toolCalls?: HarnessHistoryToolCall[]
}
export interface HarnessAgentHistoryPage {
agentId: string
sessionId: 'main'
items: HarnessHistoryEntry[]
}
export function mapHarnessAgentToEntry(agent: HarnessAgent): AgentEntry {
return {
agentId: agent.id,
name: agent.name,
workspace: `${agent.adapter}:main`,
model: agent.modelId,
source: 'agent-harness',
}
}

View File

@@ -0,0 +1,160 @@
import {
Copy,
Loader2,
MessageSquare,
MoreHorizontal,
Pencil,
RotateCcw,
Trash2,
} from 'lucide-react'
import type { FC } from 'react'
import { useNavigate } from 'react-router'
import { toast } from 'sonner'
import { Button } from '@/components/ui/button'
import {
DropdownMenu,
DropdownMenuContent,
DropdownMenuItem,
DropdownMenuSeparator,
DropdownMenuTrigger,
} from '@/components/ui/dropdown-menu'
import {
Tooltip,
TooltipContent,
TooltipProvider,
TooltipTrigger,
} from '@/components/ui/tooltip'
import {
canDelete as canDeleteAgent,
canRename as canRenameAgent,
displayName,
} from '../agent-display.helpers'
import type { AgentListItem } from '../agents-page-types'
interface AgentActionsProps {
agent: AgentListItem
activeTurnId: string | null
deleting?: boolean
onDelete: (agent: AgentListItem) => void
}
/**
* Single primary CTA per row: `Resume` (filled, accent-orange, with a
* pulsing dot) when an active turn exists; otherwise `Chat` (outline).
* Both navigate to the same place — the chat hook auto-attaches via
* `/chat/active` when there's a live turn — but the row signals which
* action the user is actually taking.
*/
export const AgentActions: FC<AgentActionsProps> = ({
agent,
activeTurnId,
deleting,
onDelete,
}) => {
const navigate = useNavigate()
const allowDelete = canDeleteAgent(agent)
const allowRename = canRenameAgent(agent)
const handleChat = () => navigate(`/agents/${agent.agentId}`)
const handleCopyId = async () => {
try {
await navigator.clipboard.writeText(agent.agentId)
toast.success('Agent id copied')
} catch {
toast.error('Could not copy agent id')
}
}
return (
<div className="flex shrink-0 items-center gap-1.5">
{activeTurnId ? (
<Button
variant="default"
size="sm"
onClick={handleChat}
className="gap-2 bg-[var(--accent-orange)] text-white shadow-sm hover:bg-[var(--accent-orange)]/90"
>
<span className="relative flex size-2">
<span className="absolute inline-flex h-full w-full animate-ping rounded-full bg-white/70 opacity-75" />
<span className="relative inline-flex size-2 rounded-full bg-white" />
</span>
Resume
</Button>
) : (
<Button variant="outline" size="sm" onClick={handleChat}>
<MessageSquare className="mr-1.5 size-3" />
Chat
</Button>
)}
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button
variant="ghost"
size="icon"
aria-label={`More actions for ${displayName(agent)}`}
className="size-8 text-muted-foreground hover:text-foreground"
>
<MoreHorizontal className="size-4" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end" className="w-44">
<DropdownMenuItem onSelect={() => void handleCopyId()}>
<Copy className="mr-2 size-3.5" />
Copy id
</DropdownMenuItem>
<ComingSoonItem
icon={Pencil}
label="Rename"
disabled={!allowRename}
/>
<ComingSoonItem icon={RotateCcw} label="Reset history" disabled />
<DropdownMenuSeparator />
<DropdownMenuItem
onSelect={() => onDelete(agent)}
disabled={!allowDelete || deleting}
className="text-destructive focus:text-destructive"
>
{deleting ? (
<Loader2 className="mr-2 size-3.5 animate-spin" />
) : (
<Trash2 className="mr-2 size-3.5" />
)}
Delete
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</div>
)
}
interface ComingSoonItemProps {
icon: typeof Pencil
label: string
disabled: boolean
}
const ComingSoonItem: FC<ComingSoonItemProps> = ({
icon: Icon,
label,
disabled,
}) => {
const item = (
<DropdownMenuItem disabled className="text-muted-foreground">
<Icon className="mr-2 size-3.5" />
{label}
</DropdownMenuItem>
)
if (!disabled) return item
return (
<TooltipProvider delayDuration={300}>
<Tooltip>
<TooltipTrigger asChild>
<span className="block w-full">{item}</span>
</TooltipTrigger>
<TooltipContent side="left" className="text-xs">
{label} coming soon
</TooltipContent>
</Tooltip>
</TooltipProvider>
)
}

View File

@@ -0,0 +1,96 @@
import { AlertTriangle, ChevronDown } from 'lucide-react'
import { type FC, useEffect, useState } from 'react'
import { Button } from '@/components/ui/button'
import {
Collapsible,
CollapsibleContent,
CollapsibleTrigger,
} from '@/components/ui/collapsible'
import {
HoverCard,
HoverCardContent,
HoverCardTrigger,
} from '@/components/ui/hover-card'
import { cn } from '@/lib/utils'
import { truncate } from './agent-row.helpers'
interface AgentErrorPanelProps {
agentId: string
message: string
errorAt: number | null
}
const STORAGE_PREFIX = 'agent-row:lastErrorSeenAt:'
const PREVIEW_CHARS = 200
export const AgentErrorPanel: FC<AgentErrorPanelProps> = ({
agentId,
message,
errorAt,
}) => {
const storageKey = `${STORAGE_PREFIX}${agentId}`
// Open if we've never seen this `errorAt` for this agent. Once the
// user collapses the panel (or refreshes after seeing it), we mark
// it seen so it doesn't re-pop on every poll.
const [open, setOpen] = useState<boolean>(() => {
if (typeof window === 'undefined' || !errorAt) return true
const seen = Number(window.localStorage.getItem(storageKey) ?? 0)
return !Number.isFinite(seen) || errorAt > seen
})
useEffect(() => {
if (!open && errorAt && typeof window !== 'undefined') {
window.localStorage.setItem(storageKey, String(errorAt))
}
}, [open, errorAt, storageKey])
const preview = truncate(message, PREVIEW_CHARS)
const truncated = preview.length < message.length
return (
<Collapsible open={open} onOpenChange={setOpen} className="mt-3">
<div className="flex items-center justify-between rounded-md border border-destructive/30 bg-destructive/5 px-3 py-2">
<div className="flex items-center gap-2 font-medium text-destructive text-xs">
<AlertTriangle className="size-3.5" />
Last error
</div>
<CollapsibleTrigger asChild>
<Button
variant="ghost"
size="sm"
className="h-6 px-2 text-muted-foreground"
>
<span className="text-xs">{open ? 'hide' : 'show'}</span>
<ChevronDown
className={cn(
'ml-1 size-3 transition-transform',
open && 'rotate-180',
)}
/>
</Button>
</CollapsibleTrigger>
</div>
<CollapsibleContent>
<div className="mt-1 rounded-md border-destructive/30 border-x border-b bg-destructive/5 px-3 pb-2 text-xs">
{truncated ? (
<HoverCard openDelay={300}>
<HoverCardTrigger asChild>
<span className="cursor-default font-mono text-foreground/80">
{preview}
</span>
</HoverCardTrigger>
<HoverCardContent
side="bottom"
className="max-w-md whitespace-pre-wrap font-mono text-xs"
>
{message}
</HoverCardContent>
</HoverCard>
) : (
<span className="font-mono text-foreground/80">{message}</span>
)}
</div>
</CollapsibleContent>
</Collapsible>
)
}

View File

@@ -0,0 +1,35 @@
import { Quote } from 'lucide-react'
import type { FC } from 'react'
import { firstNonBlankLine, truncate } from './agent-row.helpers'
interface AgentLastMessageProps {
message: string | null
}
const PREVIEW_CHARS = 110
/**
* Inline preview of the most recent user message. Renders as a quoted,
* italic line so the row reads like a conversation snippet rather than
* a label-and-value pair. No hover-card — opening the agent's chat is
* the canonical way to read the full message.
*/
export const AgentLastMessage: FC<AgentLastMessageProps> = ({ message }) => {
if (!message) {
return (
<p className="mt-1 text-muted-foreground/70 text-xs italic">
No messages yet start a chat
</p>
)
}
const preview = truncate(firstNonBlankLine(message), PREVIEW_CHARS)
return (
<p className="mt-1.5 flex items-start gap-1.5 text-foreground/85 text-sm italic leading-snug">
<Quote
className="mt-1 size-3 shrink-0 text-muted-foreground/60"
aria-hidden
/>
<span className="truncate">{preview}</span>
</p>
)
}

View File

@@ -0,0 +1,37 @@
import type { FC } from 'react'
import { formatRelativeTime } from '../agent-display.helpers'
import { AgentTokenSummary } from './AgentTokenSummary'
import type { AgentTokenUsage } from './agent-row.types'
interface AgentMetaRowProps {
lastUsedAt: number | null
tokens: AgentTokenUsage | null
}
/**
* Bottom-of-row meta line. Intentionally sparse — last activity time
* and lifetime tokens. CWD is no longer surfaced here because the path
* the server happens to be running from isn't actionable; if a future
* surface needs the cwd (chat panel, debug view) it reads from the
* listing payload directly.
*/
export const AgentMetaRow: FC<AgentMetaRowProps> = ({ lastUsedAt, tokens }) => {
const lastUsedLabel = formatRelativeTime(lastUsedAt)
const tokensTotal =
(tokens?.cumulative.input ?? 0) + (tokens?.cumulative.output ?? 0)
const showTokens = tokensTotal > 0
return (
<div className="mt-2 flex flex-wrap items-center gap-x-2 text-muted-foreground text-xs">
<span>{lastUsedLabel}</span>
{showTokens && (
<>
<span aria-hidden className="text-muted-foreground/50">
·
</span>
<AgentTokenSummary tokens={tokens} />
</>
)}
</div>
)
}

View File

@@ -0,0 +1,92 @@
import type { FC } from 'react'
import {
HoverCard,
HoverCardContent,
HoverCardTrigger,
} from '@/components/ui/hover-card'
import { cn } from '@/lib/utils'
import { formatLocalDate, ROW_BAR_COUNT } from './agent-row.helpers'
interface AgentSparklineProps {
/** 14 entries, oldest → newest. Today's bucket is the last index. */
turnsByDay: number[]
/** Same length, same order. Failed turns counted separately. */
failedByDay: number[]
className?: string
}
const MIN_BAR_HEIGHT_PX = 2
const MAX_BAR_HEIGHT_PX = 18
export const AgentSparkline: FC<AgentSparklineProps> = ({
turnsByDay,
failedByDay,
className,
}) => {
if (turnsByDay.length === 0 || turnsByDay.every((n) => n === 0)) return null
const max = Math.max(1, ...turnsByDay)
return (
<HoverCard openDelay={250}>
<HoverCardTrigger asChild>
<div
role="img"
aria-label={`Last ${ROW_BAR_COUNT} days of activity`}
className={cn('flex h-5 items-end gap-px', className)}
>
{turnsByDay.map((count, idx) => {
const ratio = count / max
const height = Math.max(
MIN_BAR_HEIGHT_PX,
Math.round(ratio * MAX_BAR_HEIGHT_PX),
)
const isToday = idx === ROW_BAR_COUNT - 1
const failed = failedByDay[idx] ?? 0
return (
<div
// biome-ignore lint/suspicious/noArrayIndexKey: fixed-length sparkline buckets keyed by day position
key={`bar-${idx}`}
className={cn(
'w-1.5 rounded-sm',
count === 0
? 'bg-muted-foreground/15'
: failed > 0
? 'bg-destructive/50'
: 'bg-[var(--accent-orange)]/50',
isToday && 'ring-1 ring-foreground/30',
)}
style={{ height }}
/>
)
})}
</div>
</HoverCardTrigger>
<HoverCardContent side="left" className="w-56 text-xs">
<div className="mb-2 font-medium text-sm">Last 14 days</div>
<ul className="space-y-0.5">
{turnsByDay.map((count, idx) => {
const failed = failedByDay[idx] ?? 0
const dayLabel = formatLocalDate(idx)
return (
<li
// biome-ignore lint/suspicious/noArrayIndexKey: fixed-length list keyed by day position
key={`day-${idx}`}
className="flex items-center justify-between text-muted-foreground"
>
<span>{dayLabel}</span>
<span>
{count}
{failed > 0 && (
<span className="ml-1 text-destructive">
({failed} failed)
</span>
)}
</span>
</li>
)
})}
</ul>
</HoverCardContent>
</HoverCard>
)
}

View File

@@ -0,0 +1,71 @@
import { TriangleAlert } from 'lucide-react'
import type { FC } from 'react'
import { Badge } from '@/components/ui/badge'
import {
HoverCard,
HoverCardContent,
HoverCardTrigger,
} from '@/components/ui/hover-card'
import { cn } from '@/lib/utils'
import { adapterLabel } from '../AdapterIcon'
import type { HarnessAgentAdapter } from '../agent-harness-types'
import type { AgentAdapterHealth } from './agent-row.types'
interface AgentSummaryChipsProps {
adapter: HarnessAgentAdapter | 'unknown'
modelLabel: string | null
reasoningEffort: string | null
/** When unhealthy, the adapter label dims and a warning chip appears. */
adapterHealth: AgentAdapterHealth | null
}
/**
* Adapter / model / reasoning summary line. Always rendered (so OpenClaw
* rows that fall back to defaults still expose what they're set up to do)
* and surfaces adapter-health *only when unhealthy* — keeping the calm
* default state silent and reserving visual noise for things the user
* needs to act on.
*/
export const AgentSummaryChips: FC<AgentSummaryChipsProps> = ({
adapter,
modelLabel,
reasoningEffort,
adapterHealth,
}) => {
const parts = [adapterLabel(adapter)]
if (modelLabel) parts.push(modelLabel)
if (reasoningEffort) parts.push(reasoningEffort)
const unhealthy = adapterHealth?.healthy === false
return (
<div
className={cn(
'flex items-center gap-1.5 text-muted-foreground text-xs',
unhealthy && 'text-muted-foreground/70',
)}
>
<span className="truncate">{parts.join(' · ')}</span>
{unhealthy && adapterHealth && (
<HoverCard openDelay={200}>
<HoverCardTrigger asChild>
<Badge
variant="outline"
className="h-5 cursor-default gap-1 border-amber-500/40 bg-amber-50 px-1.5 text-amber-900 hover:bg-amber-50"
>
<TriangleAlert className="size-2.5" />
<span className="font-normal">Unavailable</span>
</Badge>
</HoverCardTrigger>
<HoverCardContent side="right" className="w-72 text-sm">
<div className="font-medium">
{adapterLabel(adapter)} CLI not available
</div>
<div className="mt-1 text-muted-foreground text-xs">
{adapterHealth.reason ??
'Adapter binary missing on $PATH. Install it from the adapter docs to use this agent.'}
</div>
</HoverCardContent>
</HoverCard>
)}
</div>
)
}

View File

@@ -0,0 +1,37 @@
import type { FC } from 'react'
import { cn } from '@/lib/utils'
import { AdapterIcon } from '../AdapterIcon'
import { livenessDetail } from '../agent-display.helpers'
import type { HarnessAgentAdapter } from '../agent-harness-types'
import { type AgentLiveness, LivenessDot } from '../LivenessDot'
export interface AgentTileProps {
adapter: HarnessAgentAdapter | 'unknown'
status: AgentLiveness
lastUsedAt: number | null
}
/**
* Adapter glyph + a single liveness dot. Adapter health is no longer
* surfaced here — it lives as an inline pill inside `AgentSummaryChips`
* so the user isn't asked to disambiguate two dots on the same tile.
*/
export const AgentTile: FC<AgentTileProps> = ({
adapter,
status,
lastUsedAt,
}) => (
<div className="relative shrink-0">
<div className="flex h-12 w-12 items-center justify-center rounded-xl bg-muted text-muted-foreground">
<AdapterIcon adapter={adapter} className="h-6 w-6" />
</div>
<LivenessDot
status={status}
detail={livenessDetail(status, lastUsedAt)}
className={cn(
'absolute -right-0.5 -bottom-0.5',
status === 'working' && 'animate-pulse',
)}
/>
</div>
)

View File

@@ -0,0 +1,55 @@
import type { FC } from 'react'
import { Badge } from '@/components/ui/badge'
import { displayName } from '../agent-display.helpers'
import type { AgentListItem } from '../agents-page-types'
import type { AgentLiveness } from '../LivenessDot'
import { AgentSparkline } from './AgentSparkline'
import { PinToggle } from './PinToggle'
interface AgentTitleRowProps {
agent: AgentListItem
status: AgentLiveness
pinned: boolean
turnsByDay: number[]
failedByDay: number[]
onPinToggle: (next: boolean) => void
}
/**
* Title strip: name + status badge + (right-aligned) sparkline. The
* pin toggle sits trailing the title so the title always flushes left
* regardless of pin state — moving the star left of the title indents
* the row's first line off-axis from the model/preview/meta lines
* below it. When unpinned and not hovered, the toggle is removed from
* layout entirely so it reserves no space at all.
*/
export const AgentTitleRow: FC<AgentTitleRowProps> = ({
agent,
status,
pinned,
turnsByDay,
failedByDay,
onPinToggle,
}) => (
<div className="mb-1 flex items-center gap-2">
<span className="truncate font-semibold">{displayName(agent)}</span>
{status === 'working' && (
<Badge
variant="secondary"
className="bg-amber-50 text-amber-900 hover:bg-amber-50"
>
Working
</Badge>
)}
{status === 'asleep' && (
<Badge variant="outline" className="text-muted-foreground">
Asleep
</Badge>
)}
{status === 'error' && <Badge variant="destructive">Attention</Badge>}
<PinToggle pinned={pinned} onToggle={onPinToggle} />
<div className="ml-auto">
<AgentSparkline turnsByDay={turnsByDay} failedByDay={failedByDay} />
</div>
</div>
)

View File

@@ -0,0 +1,63 @@
import type { FC } from 'react'
import {
HoverCard,
HoverCardContent,
HoverCardTrigger,
} from '@/components/ui/hover-card'
import { Progress } from '@/components/ui/progress'
import { formatTokens } from './agent-row.helpers'
import type { AgentTokenUsage } from './agent-row.types'
interface AgentTokenSummaryProps {
tokens: AgentTokenUsage | null
}
/**
* Inline token total + a HoverCard breakdown. Surfaces lifetime tokens
* (the only window we can compute reliably from the session record).
* Per-window stats land in a follow-up once the activity ledger ships.
*/
export const AgentTokenSummary: FC<AgentTokenSummaryProps> = ({ tokens }) => {
if (!tokens) return null
const { input, output } = tokens.cumulative
const total = input + output
if (total === 0) return null
const inputPct = (input / total) * 100
return (
<HoverCard openDelay={200}>
<HoverCardTrigger asChild>
<span className="cursor-default text-muted-foreground tabular-nums transition-colors hover:text-foreground">
{formatTokens(total)} tokens
</span>
</HoverCardTrigger>
<HoverCardContent side="top" align="end" className="w-72 text-sm">
<div className="mb-3 flex items-center justify-between">
<span className="font-medium">Lifetime tokens</span>
<span className="text-muted-foreground text-xs tabular-nums">
{formatTokens(total)} total
</span>
</div>
<div className="space-y-2">
<div className="flex items-center justify-between text-xs">
<span className="text-muted-foreground">Input</span>
<span className="tabular-nums">{formatTokens(input)}</span>
</div>
<Progress value={inputPct} className="h-1.5" />
<div className="mt-2 flex items-center justify-between text-xs">
<span className="text-muted-foreground">Output</span>
<span className="tabular-nums">{formatTokens(output)}</span>
</div>
<Progress value={100 - inputPct} className="h-1.5" />
</div>
<p className="mt-3 border-t pt-2 text-muted-foreground text-xs leading-snug">
Cumulative across every turn this agent has run. Per-window stats
arrive in a future release.
</p>
</HoverCardContent>
</HoverCard>
)
}

View File

@@ -0,0 +1,60 @@
import { Star } from 'lucide-react'
import type { FC } from 'react'
import { Button } from '@/components/ui/button'
import {
Tooltip,
TooltipContent,
TooltipProvider,
TooltipTrigger,
} from '@/components/ui/tooltip'
import { cn } from '@/lib/utils'
interface PinToggleProps {
pinned: boolean
onToggle: (next: boolean) => void
}
/**
* Trailing star toggle. The button is *always rendered* — only its
* opacity changes between pinned/unpinned/hover states — so the title
* row's height is constant. Hiding the slot via `display: none` would
* collapse the row's vertical metrics on hover and shift every card
* below in the rail.
*
* Placement is trailing the title (after the status badge) so the
* title itself flushes left regardless of pin state — leading the
* row with the star would indent the title relative to the model /
* preview / meta lines beneath it.
*/
export const PinToggle: FC<PinToggleProps> = ({ pinned, onToggle }) => (
<TooltipProvider delayDuration={300}>
<Tooltip>
<TooltipTrigger asChild>
<Button
variant="ghost"
size="icon"
className={cn(
'size-6 text-muted-foreground transition-opacity hover:text-foreground',
pinned ? 'opacity-100' : 'opacity-0 group-hover:opacity-100',
)}
aria-pressed={pinned}
aria-label={pinned ? 'Unpin agent' : 'Pin agent'}
onClick={(event) => {
event.stopPropagation()
onToggle(!pinned)
}}
>
<Star
className={cn(
'size-3.5',
pinned && 'fill-amber-400 text-amber-500',
)}
/>
</Button>
</TooltipTrigger>
<TooltipContent side="top" className="text-xs">
{pinned ? 'Unpin' : 'Pin to top'}
</TooltipContent>
</Tooltip>
</TooltipProvider>
)

View File

@@ -0,0 +1,73 @@
import { describe, expect, it } from 'bun:test'
import {
firstNonBlankLine,
formatLocalDate,
formatTokens,
ROW_BAR_COUNT,
truncate,
} from './agent-row.helpers'
describe('formatTokens', () => {
it('renders zero / NaN as "0"', () => {
expect(formatTokens(0)).toBe('0')
expect(formatTokens(Number.NaN)).toBe('0')
})
it('renders sub-1K as integer', () => {
expect(formatTokens(142)).toBe('142')
})
it('renders K with one decimal under 10', () => {
expect(formatTokens(8_400)).toBe('8.4K')
})
it('drops the decimal at >=10K', () => {
expect(formatTokens(120_000)).toBe('120K')
})
it('renders M with one decimal under 10', () => {
expect(formatTokens(1_200_000)).toBe('1.2M')
})
})
describe('firstNonBlankLine', () => {
it('returns the first non-blank line', () => {
expect(firstNonBlankLine('\n\nhello\nworld')).toBe('hello')
})
it('skips USER_QUERY envelope tags', () => {
expect(firstNonBlankLine('<USER_QUERY>\nfix tests\n</USER_QUERY>')).toBe(
'fix tests',
)
})
it('falls back to the trimmed input when nothing matches', () => {
expect(firstNonBlankLine(' single ')).toBe('single')
})
})
describe('truncate', () => {
it('returns input unchanged when within limit', () => {
expect(truncate('hello', 10)).toBe('hello')
})
it('appends an ellipsis when over limit', () => {
expect(truncate('hello world', 6)).toBe('hello…')
})
})
describe('formatLocalDate', () => {
const today = new Date('2026-04-30T12:00:00Z')
it('labels today and yesterday explicitly', () => {
expect(formatLocalDate(ROW_BAR_COUNT - 1, today)).toBe('today')
expect(formatLocalDate(ROW_BAR_COUNT - 2, today)).toBe('yesterday')
})
it('returns a "Mon D" format for older days', () => {
const label = formatLocalDate(0, today)
// "Apr 17" or "Apr 17," depending on locale; just assert it
// contains a month abbreviation and a day number.
expect(label).toMatch(/[A-Za-z]+ \d+/)
})
})

View File

@@ -0,0 +1,64 @@
/**
* Pure formatters consumed by row sub-components. Kept distinct from
* `agent-display.helpers.ts` (page-level helpers) so the row internals
* have an obvious single home.
*/
const TOKEN_THRESHOLDS: Array<[number, string]> = [
[1_000_000, 'M'],
[1_000, 'K'],
]
/** `1.2M`, `820K`, `8.4K`, `142`, `0`. */
export function formatTokens(n: number): string {
if (!Number.isFinite(n) || n <= 0) return '0'
for (const [threshold, suffix] of TOKEN_THRESHOLDS) {
if (n >= threshold) {
const value = n / threshold
const decimal = value < 10 ? value.toFixed(1) : value.toFixed(0)
return `${decimal}${suffix}`
}
}
return String(Math.round(n))
}
const USER_QUERY_OPEN = /^<USER_QUERY>$/i
const USER_QUERY_CLOSE = /^<\/USER_QUERY>$/i
/**
* First non-blank line, with the BrowserOS user-system-prompt
* `<USER_QUERY>` envelope tags stripped so previews don't show
* structural noise.
*/
export function firstNonBlankLine(text: string): string {
const lines = text.split('\n').map((line) => line.trim())
for (const line of lines) {
if (!line) continue
if (USER_QUERY_OPEN.test(line) || USER_QUERY_CLOSE.test(line)) continue
return line
}
return text.trim()
}
export function truncate(text: string, max: number): string {
if (text.length <= max) return text
return `${text.slice(0, max - 1).trimEnd()}`
}
const SPARKLINE_DAYS = 14
/**
* "today" / "yesterday" / "Apr 17" — given an index 0..13 from
* oldest → newest. `today` defaults to `new Date()` so callers don't
* have to thread a clock through.
*/
export function formatLocalDate(idx: number, today: Date = new Date()): string {
if (idx === SPARKLINE_DAYS - 1) return 'today'
if (idx === SPARKLINE_DAYS - 2) return 'yesterday'
const offset = SPARKLINE_DAYS - 1 - idx
const date = new Date(today)
date.setDate(date.getDate() - offset)
return date.toLocaleDateString(undefined, { month: 'short', day: 'numeric' })
}
export const ROW_BAR_COUNT = SPARKLINE_DAYS

View File

@@ -0,0 +1,51 @@
import type { HarnessAgentAdapter } from '../agent-harness-types'
import type { AgentListItem } from '../agents-page-types'
import type { AgentLiveness } from '../LivenessDot'
/**
* Window-bounded token usage. Server returns `null` when no session
* record exists yet for the agent.
*/
export interface AgentTokenUsage {
last7d: { input: number; output: number; requestCount: number }
cumulative: { input: number; output: number }
}
export interface AgentAdapterHealth {
healthy: boolean
reason?: string
}
/**
* Everything an `AgentRowCard` needs to render. Mirrors the shape
* `useHarnessAgents` exposes; the page assembles one entry per row in
* `AgentList` and passes it down. Sub-components only see slices of
* this object — no prop drilling beyond two levels.
*/
export interface AgentRowData {
agent: AgentListItem
adapter: HarnessAgentAdapter | 'unknown'
modelLabel: string | null
reasoningEffort: string | null
status: AgentLiveness
lastUsedAt: number | null
pinned: boolean
cwd: string | null
lastUserMessage: string | null
tokens: AgentTokenUsage | null
/** 14 entries, oldest → newest. Today is the last index. */
turnsByDay: number[]
/** Same length and ordering as `turnsByDay`. */
failedByDay: number[]
lastError: string | null
lastErrorAt: number | null
/** When non-null, an in-flight turn this row can be resumed from. */
activeTurnId: string | null
/** Adapter-level health, shared across rows for the same adapter. */
adapterHealth: AgentAdapterHealth | null
}
export interface AgentRowCallbacks {
onDelete: (agent: AgentListItem) => void
onPinToggle: (agent: AgentListItem, next: boolean) => void
}

View File

@@ -0,0 +1,172 @@
import type { NavigateFunction } from 'react-router'
import {
AGENT_CREATED_EVENT,
AGENT_DELETED_EVENT,
} from '@/lib/constants/analyticsEvents'
import { track } from '@/lib/metrics/track'
import type { HarnessAgent, HarnessAgentAdapter } from './agent-harness-types'
import type {
AgentListItem,
CreateAgentRuntime,
ProviderOption,
} from './agents-page-types'
import { findOpenClawCliProviderById } from './openclaw-cli-providers'
import type {
AgentEntry,
OpenClawAgentMutationInput,
OpenClawSetupInput,
} from './useOpenClaw'
export interface AgentPageActionInput {
createProviderId: string
createRuntime: CreateAgentRuntime
harnessModelId: string
harnessReasoningEffort: string
navigate: NavigateFunction
newName: string
selectableOpenClawProviders: ProviderOption[]
setupProviderId: string
createHarnessAgent: (input: {
name: string
adapter: HarnessAgentAdapter
modelId?: string
reasoningEffort?: string
}) => Promise<HarnessAgent>
createOpenClawAgent: (
input: OpenClawAgentMutationInput,
) => Promise<{ agent: AgentEntry }>
deleteHarnessAgent: (agentId: string) => Promise<unknown>
deleteOpenClawAgent: (agentId: string) => Promise<unknown>
setCliAuthModalOpen: (open: boolean) => void
setCreateError: (error: string | null) => void
setCreateOpen: (open: boolean) => void
setDeletingAgentKey: (key: string | null) => void
setNewName: (name: string) => void
setPageError: (error: string | null) => void
setSetupOpen: (open: boolean) => void
setupOpenClaw: (input: OpenClawSetupInput) => Promise<unknown>
}
export function createAgentPageActions(input: AgentPageActionInput) {
const runWithPageErrorHandling = async (fn: () => Promise<unknown>) => {
input.setPageError(null)
try {
await fn()
} catch (err) {
input.setPageError(err instanceof Error ? err.message : String(err))
}
}
const handleSetup = async () => {
const option = input.selectableOpenClawProviders.find(
(item) => item.id === input.setupProviderId,
)
const isCli = !!option && !!findOpenClawCliProviderById(option.type)
const llmOption = !isCli && option ? option : undefined
await runWithPageErrorHandling(async () => {
await input.setupOpenClaw({
providerType: option?.type,
providerName: isCli ? undefined : option?.name,
baseUrl: llmOption?.baseUrl,
apiKey: llmOption?.apiKey,
modelId: option?.modelId,
})
input.setSetupOpen(false)
if (isCli) input.setCliAuthModalOpen(true)
})
}
const handleOpenClawCreate = async () => {
if (!input.newName.trim()) return
const option = input.selectableOpenClawProviders.find(
(item) => item.id === input.createProviderId,
)
const normalizedName = input.newName
.trim()
.toLowerCase()
.replace(/\s+/g, '-')
const isCli = !!option && !!findOpenClawCliProviderById(option.type)
const llmOption = !isCli && option ? option : undefined
input.setCreateError(null)
try {
const result = await input.createOpenClawAgent({
name: normalizedName,
providerType: option?.type,
providerName: isCli ? undefined : option?.name,
baseUrl: llmOption?.baseUrl,
apiKey: llmOption?.apiKey,
modelId: option?.modelId,
})
input.setCreateOpen(false)
input.setNewName('')
track(AGENT_CREATED_EVENT, {
runtime: 'openclaw',
provider_type: option?.type,
})
input.navigate(`/agents/${result.agent.agentId}`)
} catch (err) {
input.setCreateError(err instanceof Error ? err.message : String(err))
}
}
const handleHarnessCreate = async () => {
if (!input.newName.trim()) return
input.setCreateError(null)
try {
const agent = await input.createHarnessAgent({
name: input.newName.trim(),
adapter: input.createRuntime as HarnessAgentAdapter,
modelId: input.harnessModelId || undefined,
reasoningEffort: input.harnessReasoningEffort || undefined,
})
input.setCreateOpen(false)
input.setNewName('')
track(AGENT_CREATED_EVENT, {
runtime: input.createRuntime,
model_id: input.harnessModelId || undefined,
reasoning_effort: input.harnessReasoningEffort || undefined,
})
input.navigate(`/agents/${agent.id}`)
} catch (err) {
input.setCreateError(err instanceof Error ? err.message : String(err))
}
}
const handleCreate = () => {
const createByRuntime: Record<CreateAgentRuntime, () => Promise<void>> = {
openclaw: handleOpenClawCreate,
claude: handleHarnessCreate,
codex: handleHarnessCreate,
}
void createByRuntime[input.createRuntime]()
}
const handleDelete = async (agent: AgentListItem) => {
input.setDeletingAgentKey(agent.key)
await runWithPageErrorHandling(async () => {
const deleteBySource: Record<
AgentListItem['source'],
(agentId: string) => Promise<unknown>
> = {
openclaw: (agentId) => input.deleteOpenClawAgent(agentId),
'agent-harness': (agentId) => input.deleteHarnessAgent(agentId),
}
await deleteBySource[agent.source](agent.agentId)
track(AGENT_DELETED_EVENT, {
runtime: agent.source,
agent_id: agent.agentId,
})
})
input.setDeletingAgentKey(null)
}
return {
handleCreate,
handleDelete,
handleSetup,
runWithPageErrorHandling,
}
}

View File

@@ -0,0 +1,173 @@
import { type Dispatch, type SetStateAction, useEffect, useMemo } from 'react'
import type { LlmProviderConfig } from '@/lib/llm-providers/types'
import type {
HarnessAdapterDescriptor,
HarnessAgentAdapter,
} from './agent-harness-types'
import type { CreateAgentRuntime } from './agents-page-types'
import { toProviderOptions } from './agents-page-utils'
import {
buildOpenClawCliProviderOptions,
findOpenClawCliProviderById,
useOpenClawCliProviderAuthStatus,
} from './openclaw-cli-providers'
export function useDefaultAgentName(
createOpen: boolean,
setNewName: Dispatch<SetStateAction<string>>,
): void {
useEffect(() => {
if (!createOpen) return
setNewName((current) => current || 'agent')
}, [createOpen, setNewName])
}
export function useHarnessAgentDefaults(input: {
adapters: HarnessAdapterDescriptor[]
createOpen: boolean
harnessAdapterId: HarnessAgentAdapter
setHarnessAdapterId: Dispatch<SetStateAction<HarnessAgentAdapter>>
setHarnessModelId: Dispatch<SetStateAction<string>>
setHarnessReasoningEffort: Dispatch<SetStateAction<string>>
}): void {
const {
adapters,
createOpen,
harnessAdapterId,
setHarnessAdapterId,
setHarnessModelId,
setHarnessReasoningEffort,
} = input
useEffect(() => {
if (!createOpen) return
const adapter =
adapters.find((entry) => entry.id === harnessAdapterId) ?? adapters[0]
if (!adapter) return
setHarnessAdapterId(adapter.id)
setHarnessModelId((current) => current || adapter.defaultModelId)
setHarnessReasoningEffort(
(current) => current || adapter.defaultReasoningEffort,
)
}, [
adapters,
createOpen,
harnessAdapterId,
setHarnessAdapterId,
setHarnessModelId,
setHarnessReasoningEffort,
])
}
export function useOpenClawProviderSelection(input: {
providers: LlmProviderConfig[]
defaultProviderId: string
createOpen: boolean
createRuntime: CreateAgentRuntime
createProviderId: string
setCreateProviderId: Dispatch<SetStateAction<string>>
setupOpen: boolean
setupProviderId: string
setSetupProviderId: Dispatch<SetStateAction<string>>
cliAuthModalOpen: boolean
setCliAuthModalOpen: Dispatch<SetStateAction<boolean>>
}) {
const {
providers,
defaultProviderId,
createOpen,
createRuntime,
createProviderId,
setCreateProviderId,
setupOpen,
setupProviderId,
setSetupProviderId,
cliAuthModalOpen,
setCliAuthModalOpen,
} = input
const cliProviderOptions = useMemo(
() => buildOpenClawCliProviderOptions(),
[],
)
const selectableOpenClawProviders = useMemo(
() => toProviderOptions(providers, cliProviderOptions),
[providers, cliProviderOptions],
)
useEffect(() => {
if (selectableOpenClawProviders.length === 0) return
const fallbackId =
selectableOpenClawProviders.find(
(provider) => provider.id === defaultProviderId,
)?.id ?? selectableOpenClawProviders[0].id
if (createOpen && !createProviderId) {
setCreateProviderId(fallbackId)
}
}, [
createOpen,
createProviderId,
defaultProviderId,
selectableOpenClawProviders,
setCreateProviderId,
])
useEffect(() => {
if (selectableOpenClawProviders.length === 0) return
const fallbackId =
selectableOpenClawProviders.find(
(provider) => provider.id === defaultProviderId,
)?.id ?? selectableOpenClawProviders[0].id
if (setupOpen && !setupProviderId) {
setSetupProviderId(fallbackId)
}
}, [
defaultProviderId,
selectableOpenClawProviders,
setSetupProviderId,
setupOpen,
setupProviderId,
])
const selectedCreateOption = selectableOpenClawProviders.find(
(provider) => provider.id === createProviderId,
)
const selectedCliProvider = selectedCreateOption
? findOpenClawCliProviderById(selectedCreateOption.type)
: undefined
const selectedSetupOption = selectableOpenClawProviders.find(
(provider) => provider.id === setupProviderId,
)
const selectedSetupCliProvider = selectedSetupOption
? findOpenClawCliProviderById(selectedSetupOption.type)
: undefined
const activeCliProvider =
(setupOpen && selectedSetupCliProvider) ||
(createOpen && createRuntime === 'openclaw' && selectedCliProvider) ||
undefined
const {
data: cliAuthStatus,
isLoading: cliAuthLoading,
error: cliAuthError,
} = useOpenClawCliProviderAuthStatus(
activeCliProvider?.id ?? '',
!!activeCliProvider,
)
useEffect(() => {
if (cliAuthModalOpen && cliAuthStatus?.loggedIn) {
setCliAuthModalOpen(false)
}
}, [cliAuthModalOpen, cliAuthStatus?.loggedIn, setCliAuthModalOpen])
return {
selectableOpenClawProviders,
selectedCliProvider,
selectedSetupCliProvider,
authTerminalProvider: selectedSetupCliProvider ?? selectedCliProvider,
cliAuthStatus,
cliAuthLoading,
cliAuthError,
}
}

View File

@@ -0,0 +1,119 @@
import type { HarnessAgentAdapter } from './agent-harness-types'
import type { GatewayLifecycleAction, OpenClawStatus } from './useOpenClaw'
export type CreateAgentRuntime = 'openclaw' | HarnessAgentAdapter
export interface ProviderOption {
id: string
type: string
name: string
modelId: string
baseUrl?: string
apiKey?: string
}
export interface AgentListItem {
key: string
agentId: string
name: string
source: 'openclaw' | 'agent-harness'
runtimeLabel: string
modelLabel: string
detail: string
canChat: boolean
canDelete: boolean
}
export interface GatewayUiState {
canManageAgents: boolean
controlPlaneDegraded: boolean
controlPlaneBusy: boolean
}
export const DEFAULT_HARNESS_ADAPTER: HarnessAgentAdapter = 'claude'
export const DEFAULT_CREATE_RUNTIME: CreateAgentRuntime = 'openclaw'
export const LIFECYCLE_BANNER_COPY: Record<GatewayLifecycleAction, string> = {
setup: 'Setting up OpenClaw...',
start: 'Starting gateway...',
stop: 'Stopping gateway...',
restart: 'Restarting gateway...',
reconnect: 'Restoring gateway connection...',
}
export const CONTROL_PLANE_COPY: Record<
OpenClawStatus['controlPlaneStatus'],
{
badgeVariant: 'default' | 'secondary' | 'outline' | 'destructive'
badgeLabel: string
title: string
description: string
}
> = {
connected: {
badgeVariant: 'default',
badgeLabel: 'Control Plane Ready',
title: 'Gateway Connected',
description: 'OpenClaw can create, manage, and chat with agents normally.',
},
connecting: {
badgeVariant: 'secondary',
badgeLabel: 'Connecting',
title: 'Connecting to Gateway',
description:
'BrowserOS is establishing the OpenClaw control channel for agent operations.',
},
reconnecting: {
badgeVariant: 'secondary',
badgeLabel: 'Reconnecting',
title: 'Reconnecting Control Plane',
description:
'The gateway process is up, but BrowserOS is restoring the control channel.',
},
recovering: {
badgeVariant: 'secondary',
badgeLabel: 'Recovering',
title: 'Recovering Gateway Connection',
description:
'BrowserOS detected a control-plane fault and is trying a safe recovery path.',
},
disconnected: {
badgeVariant: 'outline',
badgeLabel: 'Disconnected',
title: 'Gateway Disconnected',
description: 'The gateway process is not available to BrowserOS right now.',
},
failed: {
badgeVariant: 'destructive',
badgeLabel: 'Needs Attention',
title: 'Gateway Recovery Failed',
description:
'BrowserOS could not restore the OpenClaw control channel automatically.',
},
}
export const FALLBACK_CONTROL_PLANE_COPY = {
badgeVariant: 'outline' as const,
badgeLabel: 'Unknown',
title: 'Gateway State Unknown',
description:
'BrowserOS received a gateway status it does not recognize yet. Refreshing or reconnecting should restore a known state.',
}
export const RECOVERY_REASON_COPY: Record<
NonNullable<OpenClawStatus['lastRecoveryReason']>,
string
> = {
transient_disconnect:
'The control channel dropped briefly and BrowserOS is retrying it.',
signature_expired:
'The gateway rejected the signed device handshake because its clock drifted.',
pairing_required:
'The gateway asked BrowserOS to approve its local device identity again.',
token_mismatch:
'BrowserOS had to reload the gateway token before reconnecting.',
container_not_ready:
'The OpenClaw gateway process is not ready yet, so control-plane recovery cannot start.',
unknown:
'BrowserOS hit an unexpected gateway error and could not classify it cleanly.',
}

View File

@@ -0,0 +1,167 @@
import type { LlmProviderConfig } from '@/lib/llm-providers/types'
import type { HarnessAgent, HarnessAgentAdapter } from './agent-harness-types'
import {
type AgentListItem,
CONTROL_PLANE_COPY,
FALLBACK_CONTROL_PLANE_COPY,
type GatewayUiState,
LIFECYCLE_BANNER_COPY,
type ProviderOption,
RECOVERY_REASON_COPY,
} from './agents-page-types'
import { getOpenClawSupportedProviders } from './openclaw-supported-providers'
import {
type AgentEntry,
type GatewayLifecycleAction,
getModelDisplayName,
type OpenClawStatus,
} from './useOpenClaw'
export function getControlPlaneCopy(
status: OpenClawStatus['controlPlaneStatus'],
) {
return CONTROL_PLANE_COPY[status] ?? FALLBACK_CONTROL_PLANE_COPY
}
export function getRecoveryDetail(status: OpenClawStatus): string | null {
if (!status.lastRecoveryReason && !status.lastGatewayError) return null
const detail = status.lastRecoveryReason
? RECOVERY_REASON_COPY[status.lastRecoveryReason]
: null
if (status.lastGatewayError && detail) {
return `${detail} Latest gateway error: ${status.lastGatewayError}`
}
return status.lastGatewayError ?? detail
}
export function formatHarnessAdapter(adapter: HarnessAgentAdapter): string {
return adapter === 'claude' ? 'Claude Code' : 'Codex'
}
export function toProviderOptions(
providers: LlmProviderConfig[],
cliProviders: ProviderOption[],
): ProviderOption[] {
return [...getOpenClawSupportedProviders(providers), ...cliProviders]
}
export function toOpenClawListItem(
agent: AgentEntry,
canManageAgents: boolean,
): AgentListItem {
return {
key: `openclaw:${agent.agentId}`,
agentId: agent.agentId,
name: agent.name,
source: 'openclaw',
runtimeLabel: 'OpenClaw',
modelLabel: getModelDisplayName(agent.model) ?? 'default',
detail: agent.workspace,
canChat: canManageAgents,
canDelete: canManageAgents && agent.agentId !== 'main',
}
}
export function toHarnessListItem(agent: HarnessAgent): AgentListItem {
return {
key: `agent-harness:${agent.id}`,
agentId: agent.id,
name: agent.name,
source: 'agent-harness',
runtimeLabel: formatHarnessAdapter(agent.adapter),
modelLabel: agent.modelId ?? 'default',
detail: `${agent.adapter}:main`,
canChat: true,
canDelete: true,
}
}
export function getGatewayUiState(
status: OpenClawStatus | null,
): GatewayUiState {
if (!status) {
return {
canManageAgents: false,
controlPlaneDegraded: false,
controlPlaneBusy: false,
}
}
const controlPlaneBusy =
status.controlPlaneStatus === 'connecting' ||
status.controlPlaneStatus === 'reconnecting' ||
status.controlPlaneStatus === 'recovering'
return {
canManageAgents:
status.status === 'running' && status.controlPlaneStatus === 'connected',
controlPlaneBusy,
controlPlaneDegraded:
status.status === 'running' && status.controlPlaneStatus !== 'connected',
}
}
export function getLifecycleBanner(
action: GatewayLifecycleAction | null,
): string | null {
return action ? LIFECYCLE_BANNER_COPY[action] : null
}
export function canManageOpenClawAgents(
state: GatewayUiState,
lifecyclePending: boolean,
): boolean {
return state.canManageAgents && !lifecyclePending
}
export function shouldShowControlPlaneDegraded(
state: GatewayUiState,
lifecyclePending: boolean,
): boolean {
return state.controlPlaneDegraded && !lifecyclePending
}
export function getControlPlaneCopyForStatus(status: OpenClawStatus | null) {
return status
? getControlPlaneCopy(status.controlPlaneStatus)
: FALLBACK_CONTROL_PLANE_COPY
}
export function getVisibleOpenClawAgents(
enabled: boolean,
agents: AgentEntry[],
): AgentEntry[] {
return enabled ? agents : []
}
export function getAgentsLoading(input: {
adaptersLoading: boolean
harnessAgentsLoading: boolean
openClawAgentsLoading: boolean
}): boolean {
return (
input.adaptersLoading ||
input.harnessAgentsLoading ||
input.openClawAgentsLoading
)
}
export function getInlineError(input: {
lifecyclePending: boolean
pageError: string | null
openClawAgentsError: Error | null
adaptersError: Error | null
harnessAgentsError: Error | null
}): string | null {
if (input.lifecyclePending) return null
return (
input.pageError ??
input.openClawAgentsError?.message ??
input.adaptersError?.message ??
input.harnessAgentsError?.message ??
null
)
}

View File

@@ -0,0 +1,185 @@
import { useQuery } from '@tanstack/react-query'
import { CheckCircle2, Loader2, Terminal, TriangleAlert } from 'lucide-react'
import type { FC } from 'react'
import { Button } from '@/components/ui/button'
import { useAgentServerUrl } from '@/lib/browseros/useBrowserOSProviders'
export interface OpenClawCliProvider {
id: string
displayName: string
description: string
models: readonly string[]
authLoginCommand: string
}
export interface OpenClawCliProviderAuthStatus {
installed: boolean
loggedIn: boolean
accountLabel?: string
subscriptionLabel?: string
error?: string
}
export interface OpenClawCliProviderOption {
id: string
type: string
name: string
modelId: string
}
const CLAUDE_CLI_PROVIDER: OpenClawCliProvider = {
id: 'claude-cli',
displayName: 'Anthropic Claude CLI',
description: 'Uses your Claude.ai subscription via the Claude Code CLI',
models: ['claude-sonnet-4-6', 'claude-opus-4-6', 'claude-haiku-4-5'],
authLoginCommand: 'claude /login',
}
export const OPENCLAW_CLI_PROVIDERS: readonly OpenClawCliProvider[] = [
CLAUDE_CLI_PROVIDER,
]
export function findOpenClawCliProviderById(
id: string,
): OpenClawCliProvider | undefined {
return OPENCLAW_CLI_PROVIDERS.find((provider) => provider.id === id)
}
export function buildOpenClawCliProviderOptions(): OpenClawCliProviderOption[] {
return OPENCLAW_CLI_PROVIDERS.flatMap((provider) =>
provider.models.map((modelId) => ({
id: `${provider.id}/${modelId}`,
type: provider.id,
name: provider.displayName,
modelId,
})),
)
}
async function fetchCliProviderAuthStatus(
baseUrl: string,
providerId: string,
): Promise<OpenClawCliProviderAuthStatus> {
const res = await fetch(`${baseUrl}/claw/providers/${providerId}/auth-status`)
if (!res.ok) {
let message = `Auth status request failed (${res.status})`
try {
const body = (await res.json()) as { error?: string }
if (body.error) message = body.error
} catch {}
throw new Error(message)
}
return res.json() as Promise<OpenClawCliProviderAuthStatus>
}
export function useOpenClawCliProviderAuthStatus(
providerId: string,
enabled: boolean,
) {
const { baseUrl, isLoading: urlLoading } = useAgentServerUrl()
return useQuery<OpenClawCliProviderAuthStatus, Error>({
queryKey: ['openclaw-cli-auth', baseUrl, providerId],
queryFn: () => fetchCliProviderAuthStatus(baseUrl as string, providerId),
enabled: !!baseUrl && !urlLoading && enabled,
refetchInterval: enabled ? 2000 : false,
})
}
interface OpenClawCliProviderStatusPanelProps {
provider: OpenClawCliProvider
status: OpenClawCliProviderAuthStatus | undefined
loading: boolean
fetchError: Error | null
onConnect: () => void
}
export const OpenClawCliProviderStatusPanel: FC<
OpenClawCliProviderStatusPanelProps
> = ({ provider, status, loading, fetchError, onConnect }) => {
// Initial fetch (no data yet).
if (loading && !status) {
return (
<div className="flex items-center gap-2 rounded-md border border-border bg-muted/30 px-3 py-2 text-sm">
<Loader2 className="size-4 animate-spin text-muted-foreground" />
<span className="text-muted-foreground">
Checking {provider.displayName} status
</span>
</div>
)
}
if (fetchError) {
return (
<div className="flex items-start gap-2 rounded-md border border-destructive/30 bg-destructive/5 px-3 py-2 text-sm">
<TriangleAlert className="mt-0.5 size-4 text-destructive" />
<div>
<div className="font-medium text-destructive">
Could not read {provider.displayName} status
</div>
<div className="text-muted-foreground text-xs">
{fetchError.message}
</div>
</div>
</div>
)
}
if (!status) return null
// Install failed or binary missing.
if (!status.installed) {
return (
<div className="flex items-start gap-2 rounded-md border border-amber-500/40 bg-amber-500/5 px-3 py-2 text-sm">
<TriangleAlert className="mt-0.5 size-4 text-amber-600" />
<div>
<div className="font-medium">
{provider.displayName} not installed
</div>
<div className="text-muted-foreground text-xs">
The gateway will try to install it on the next restart. If this
persists, check your network and the gateway logs.
</div>
</div>
</div>
)
}
// Happy path.
if (status.loggedIn) {
const identityBits = [
status.accountLabel,
status.subscriptionLabel ? `(${status.subscriptionLabel})` : null,
].filter(Boolean)
const identity = identityBits.length > 0 ? identityBits.join(' ') : 'Ready'
return (
<div className="flex items-center gap-2 rounded-md border border-emerald-500/40 bg-emerald-500/5 px-3 py-2 text-sm">
<CheckCircle2 className="size-4 text-emerald-600" />
<div className="min-w-0 flex-1">
<div className="font-medium">Connected to {provider.displayName}</div>
<div className="truncate text-muted-foreground text-xs">
{identity}
</div>
</div>
</div>
)
}
// Installed but not logged in.
return (
<div className="flex flex-col gap-2 rounded-md border border-border bg-muted/30 px-3 py-3 text-sm">
<div>
<div className="font-medium">{provider.displayName} not set up</div>
<div className="text-muted-foreground text-xs">
{provider.description}
</div>
{status.error && (
<div className="mt-1 text-destructive text-xs">{status.error}</div>
)}
</div>
<Button size="sm" variant="outline" onClick={onConnect} className="w-fit">
<Terminal className="mr-1 size-4" />
Connect {provider.displayName}
</Button>
</div>
)
}

View File

@@ -0,0 +1,38 @@
import { describe, expect, it } from 'bun:test'
import { buildAgentApiUrl } from './agent-api-url'
import { mapHarnessAgentToEntry } from './agent-harness-types'
describe('mapHarnessAgentToEntry', () => {
it('maps created harness agents into chat-compatible entries', () => {
expect(
mapHarnessAgentToEntry({
id: 'agent-1',
name: 'Review bot',
adapter: 'codex',
modelId: 'gpt-5.5',
reasoningEffort: 'medium',
permissionMode: 'approve-all',
sessionKey: 'agent:agent-1:main',
createdAt: 1000,
updatedAt: 1000,
}),
).toEqual({
agentId: 'agent-1',
name: 'Review bot',
workspace: 'codex:main',
model: 'gpt-5.5',
source: 'agent-harness',
})
})
})
describe('buildAgentApiUrl', () => {
it('does not add a trailing slash for the harness root route', () => {
expect(buildAgentApiUrl('http://127.0.0.1:9105', '/')).toBe(
'http://127.0.0.1:9105/agents',
)
expect(buildAgentApiUrl('http://127.0.0.1:9105', '/adapters')).toBe(
'http://127.0.0.1:9105/agents/adapters',
)
})
})

View File

@@ -0,0 +1,464 @@
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query'
import { getAgentServerUrl } from '@/lib/browseros/helpers'
import { useAgentServerUrl } from '@/lib/browseros/useBrowserOSProviders'
import { buildAgentApiUrl } from './agent-api-url'
import {
type AgentHarnessStreamEvent,
type CreateHarnessAgentInput,
type HarnessAdapterDescriptor,
type HarnessAgent,
type HarnessAgentHistoryPage,
type HarnessQueuedMessage,
mapHarnessAgentToEntry,
} from './agent-harness-types'
import type { OpenClawStatus } from './useOpenClaw'
/**
* Combined response shape of `GET /agents`. The page polls this once
* and consumes both fields, replacing the dedicated `/claw/status`
* poll the previous design carried.
*/
interface HarnessAgentsResponse {
agents: HarnessAgent[]
gateway: OpenClawStatus | null
}
export type { AgentHarnessStreamEvent }
const AGENT_QUERY_KEYS = {
adapters: 'agent-harness-adapters',
agents: 'agent-harness-agents',
} as const
async function agentsFetch<T>(
baseUrl: string,
path: string,
init?: RequestInit,
): Promise<T> {
const res = await fetch(buildAgentApiUrl(baseUrl, path), init)
if (!res.ok) {
let message = `Request failed with status ${res.status}`
try {
const body = (await res.json()) as { error?: string }
if (body.error) message = body.error
} catch {}
throw new Error(message)
}
return res.json() as Promise<T>
}
export function useAgentAdapters(enabled = true) {
const {
baseUrl,
isLoading: urlLoading,
error: urlError,
} = useAgentServerUrl()
const query = useQuery<HarnessAdapterDescriptor[], Error>({
queryKey: [AGENT_QUERY_KEYS.adapters, baseUrl],
queryFn: async () => {
const data = await agentsFetch<{ adapters: HarnessAdapterDescriptor[] }>(
baseUrl as string,
'/adapters',
)
return data.adapters ?? []
},
enabled: Boolean(baseUrl) && !urlLoading && enabled,
})
return {
adapters: query.data ?? [],
loading: query.isLoading || urlLoading,
error: query.error ?? urlError,
refetch: query.refetch,
}
}
export function useHarnessAgents(enabled = true) {
const {
baseUrl,
isLoading: urlLoading,
error: urlError,
} = useAgentServerUrl()
const query = useQuery<HarnessAgentsResponse, Error>({
queryKey: [AGENT_QUERY_KEYS.agents, baseUrl],
queryFn: async () => {
const data = await agentsFetch<HarnessAgentsResponse>(
baseUrl as string,
'/',
)
return {
agents: data.agents ?? [],
gateway: data.gateway ?? null,
}
},
enabled: Boolean(baseUrl) && !urlLoading && enabled,
// Poll every 5s so the per-agent liveness state (working / idle /
// asleep / error) and last-used timestamps stay fresh without a
// websocket. `refetchIntervalInBackground: false` lets a hidden
// tab go quiet — react-query's default, made explicit.
refetchInterval: 5_000,
refetchIntervalInBackground: false,
})
return {
agents: (query.data?.agents ?? []).map(mapHarnessAgentToEntry),
harnessAgents: query.data?.agents ?? [],
gateway: query.data?.gateway ?? null,
loading: query.isLoading || urlLoading,
error: query.error ?? urlError,
refetch: query.refetch,
}
}
export function useCreateHarnessAgent() {
const { baseUrl, isLoading: urlLoading } = useAgentServerUrl()
const queryClient = useQueryClient()
return useMutation({
mutationFn: async (input: CreateHarnessAgentInput) => {
if (!baseUrl || urlLoading) {
throw new Error('BrowserOS agent server URL is not ready')
}
const data = await agentsFetch<{ agent: HarnessAgent }>(baseUrl, '/', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(input),
})
return data.agent
},
onSuccess: async () => {
await queryClient.invalidateQueries({
queryKey: [AGENT_QUERY_KEYS.agents],
})
},
})
}
/**
* Apply a partial update to a harness agent. Used by the pin-toggle
* star and (eventually) the inline rename UI. Optimistically writes
* the patch into the listing query cache so the row updates instantly,
* then rolls back if the server rejects the change.
*/
export function useUpdateHarnessAgent() {
const { baseUrl, isLoading: urlLoading } = useAgentServerUrl()
const queryClient = useQueryClient()
return useMutation({
mutationFn: async (input: {
agentId: string
patch: { name?: string; pinned?: boolean }
}) => {
if (!baseUrl || urlLoading) {
throw new Error('BrowserOS agent server URL is not ready')
}
const data = await agentsFetch<{ agent: HarnessAgent }>(
baseUrl,
`/${encodeURIComponent(input.agentId)}`,
{
method: 'PATCH',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(input.patch),
},
)
return data.agent
},
onMutate: async ({ agentId, patch }) => {
const queryKey = [AGENT_QUERY_KEYS.agents, baseUrl]
await queryClient.cancelQueries({ queryKey })
const previous = queryClient.getQueryData<HarnessAgentsResponse>(queryKey)
if (!previous) return { previous: undefined }
queryClient.setQueryData<HarnessAgentsResponse>(queryKey, {
...previous,
agents: previous.agents.map((agent) =>
agent.id === agentId ? { ...agent, ...patch } : agent,
),
})
return { previous }
},
onError: (_err, _vars, context) => {
if (!context?.previous) return
queryClient.setQueryData(
[AGENT_QUERY_KEYS.agents, baseUrl],
context.previous,
)
},
onSettled: async () => {
await queryClient.invalidateQueries({
queryKey: [AGENT_QUERY_KEYS.agents],
})
},
})
}
export function useDeleteHarnessAgent() {
const { baseUrl, isLoading: urlLoading } = useAgentServerUrl()
const queryClient = useQueryClient()
return useMutation({
mutationFn: async (agentId: string) => {
if (!baseUrl || urlLoading) {
throw new Error('BrowserOS agent server URL is not ready')
}
return agentsFetch<{ success: boolean }>(
baseUrl,
`/${encodeURIComponent(agentId)}`,
{ method: 'DELETE' },
)
},
onSuccess: async () => {
await queryClient.invalidateQueries({
queryKey: [AGENT_QUERY_KEYS.agents],
})
},
})
}
export async function chatWithHarnessAgent(
agentId: string,
message: string,
signal?: AbortSignal,
attachments?: ReadonlyArray<unknown>,
): Promise<Response> {
const baseUrl = await getAgentServerUrl()
return fetch(`${baseUrl}/agents/${encodeURIComponent(agentId)}/chat`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
message,
...(attachments && attachments.length > 0 ? { attachments } : {}),
}),
signal,
})
}
/**
* Subscribe to an existing turn (the server's `ActiveTurnRegistry`
* decoupled the turn lifecycle from POST /chat). `lastSeq` lets the
* client resume after a disconnect — the server replays buffered
* frames with seq > lastSeq, then tails new ones.
*/
export async function attachToHarnessTurn(
agentId: string,
options: { turnId?: string; lastSeq?: number; signal?: AbortSignal } = {},
): Promise<Response> {
const baseUrl = await getAgentServerUrl()
const url = new URL(
`${baseUrl}/agents/${encodeURIComponent(agentId)}/chat/stream`,
)
if (options.turnId) url.searchParams.set('turnId', options.turnId)
const headers: Record<string, string> = {}
if (typeof options.lastSeq === 'number') {
headers['Last-Event-ID'] = String(options.lastSeq)
}
return fetch(url.toString(), { signal: options.signal, headers })
}
export interface HarnessActiveTurnInfo {
turnId: string
agentId: string
sessionId: 'main'
status: 'running' | 'done' | 'error' | 'cancelled'
lastSeq: number
startedAt: number
endedAt?: number
/** User message that kicked off the turn; null when not captured. */
prompt: string | null
}
/**
* Discover an in-flight turn for an agent. Used on chat mount so the
* UI reattaches instead of starting a new turn after a tab/refresh.
*/
export async function fetchActiveHarnessTurn(
agentId: string,
): Promise<HarnessActiveTurnInfo | null> {
const baseUrl = await getAgentServerUrl()
const response = await fetch(
`${baseUrl}/agents/${encodeURIComponent(agentId)}/chat/active`,
)
if (!response.ok) return null
const body = (await response.json()) as {
active: HarnessActiveTurnInfo | null
}
return body.active
}
/**
* Stop button. Hits the explicit cancel endpoint instead of just
* aborting the fetch (which now only detaches *this* subscriber from
* the buffer; the underlying turn would otherwise keep running).
*/
export async function cancelHarnessTurn(
agentId: string,
options: { turnId?: string; reason?: string } = {},
): Promise<{ cancelled: boolean }> {
const baseUrl = await getAgentServerUrl()
const response = await fetch(
`${baseUrl}/agents/${encodeURIComponent(agentId)}/chat/cancel`,
{
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
...(options.turnId ? { turnId: options.turnId } : {}),
...(options.reason ? { reason: options.reason } : {}),
}),
},
)
if (!response.ok) return { cancelled: false }
return (await response.json()) as { cancelled: boolean }
}
export async function fetchHarnessAgentHistory(
agentId: string,
): Promise<HarnessAgentHistoryPage> {
const baseUrl = await getAgentServerUrl()
return agentsFetch<HarnessAgentHistoryPage>(
baseUrl,
`/${encodeURIComponent(agentId)}/sessions/main/history`,
)
}
export interface EnqueueMessageInput {
message: string
attachments?: ReadonlyArray<unknown>
}
export async function enqueueHarnessMessage(
agentId: string,
input: EnqueueMessageInput,
): Promise<HarnessQueuedMessage> {
const baseUrl = await getAgentServerUrl()
const response = await fetch(
`${baseUrl}/agents/${encodeURIComponent(agentId)}/queue`,
{
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
message: input.message,
...(input.attachments && input.attachments.length > 0
? { attachments: input.attachments }
: {}),
}),
},
)
if (!response.ok) {
let message = `Request failed with status ${response.status}`
try {
const body = (await response.json()) as { error?: string }
if (body.error) message = body.error
} catch {}
throw new Error(message)
}
const body = (await response.json()) as { queued: HarnessQueuedMessage }
return body.queued
}
export async function removeHarnessQueuedMessage(
agentId: string,
messageId: string,
): Promise<{ removed: boolean }> {
const baseUrl = await getAgentServerUrl()
const response = await fetch(
`${baseUrl}/agents/${encodeURIComponent(agentId)}/queue/${encodeURIComponent(
messageId,
)}`,
{ method: 'DELETE' },
)
if (!response.ok) return { removed: false }
return (await response.json()) as { removed: boolean }
}
/**
* Optimistic enqueue: writes the new queued message into the listing
* cache immediately so the queue panel reflects the change without
* waiting for the next poll. Rolls back if the server rejects.
*/
export function useEnqueueHarnessMessage() {
const { baseUrl } = useAgentServerUrl()
const queryClient = useQueryClient()
return useMutation({
mutationFn: async (input: { agentId: string } & EnqueueMessageInput) =>
enqueueHarnessMessage(input.agentId, input),
onMutate: async (input) => {
const queryKey = [AGENT_QUERY_KEYS.agents, baseUrl]
await queryClient.cancelQueries({ queryKey })
const previous = queryClient.getQueryData<HarnessAgentsResponse>(queryKey)
if (!previous) return { previous: undefined }
const optimistic: HarnessQueuedMessage = {
id: `optimistic-${Math.random().toString(36).slice(2, 10)}`,
createdAt: Date.now(),
message: input.message,
}
queryClient.setQueryData<HarnessAgentsResponse>(queryKey, {
...previous,
agents: previous.agents.map((agent) =>
agent.id === input.agentId
? { ...agent, queue: [...(agent.queue ?? []), optimistic] }
: agent,
),
})
return { previous }
},
onError: (_err, _vars, context) => {
if (!context?.previous) return
queryClient.setQueryData(
[AGENT_QUERY_KEYS.agents, baseUrl],
context.previous,
)
},
onSettled: async () => {
await queryClient.invalidateQueries({
queryKey: [AGENT_QUERY_KEYS.agents],
})
},
})
}
/**
* Optimistic queue removal mirror of `useEnqueueHarnessMessage`.
*/
export function useRemoveHarnessQueuedMessage() {
const { baseUrl } = useAgentServerUrl()
const queryClient = useQueryClient()
return useMutation({
mutationFn: async (input: { agentId: string; messageId: string }) =>
removeHarnessQueuedMessage(input.agentId, input.messageId),
onMutate: async (input) => {
const queryKey = [AGENT_QUERY_KEYS.agents, baseUrl]
await queryClient.cancelQueries({ queryKey })
const previous = queryClient.getQueryData<HarnessAgentsResponse>(queryKey)
if (!previous) return { previous: undefined }
queryClient.setQueryData<HarnessAgentsResponse>(queryKey, {
...previous,
agents: previous.agents.map((agent) =>
agent.id === input.agentId
? {
...agent,
queue: (agent.queue ?? []).filter(
(entry) => entry.id !== input.messageId,
),
}
: agent,
),
})
return { previous }
},
onError: (_err, _vars, context) => {
if (!context?.previous) return
queryClient.setQueryData(
[AGENT_QUERY_KEYS.agents, baseUrl],
context.previous,
)
},
onSettled: async () => {
await queryClient.invalidateQueries({
queryKey: [AGENT_QUERY_KEYS.agents],
})
},
})
}

View File

@@ -1,5 +1,4 @@
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query'
import { getAgentServerUrl } from '@/lib/browseros/helpers'
import { useAgentServerUrl } from '@/lib/browseros/useBrowserOSProviders'
export interface AgentEntry {
@@ -7,6 +6,7 @@ export interface AgentEntry {
name: string
workspace: string
model?: unknown
source?: 'openclaw' | 'agent-harness'
}
export interface OpenClawStatus {
@@ -41,6 +41,7 @@ export interface OpenClawAgentMutationInput {
baseUrl?: string
apiKey?: string
modelId?: string
supportsImages?: boolean
}
export interface OpenClawSetupInput {
@@ -49,6 +50,10 @@ export interface OpenClawSetupInput {
baseUrl?: string
apiKey?: string
modelId?: string
// Mirrors LlmProviderConfig.supportsImages — pass-through so the gateway
// can declare the model's input modalities correctly when persisting the
// custom-provider config.
supportsImages?: boolean
}
export function getModelDisplayName(model: unknown): string | undefined {
@@ -93,7 +98,10 @@ async function fetchOpenClawStatus(baseUrl: string): Promise<OpenClawStatus> {
async function fetchOpenClawAgents(baseUrl: string): Promise<AgentEntry[]> {
const data = await clawFetch<{ agents: AgentEntry[] }>(baseUrl, '/agents')
return data.agents ?? []
return (data.agents ?? []).map((agent) => ({
...agent,
source: 'openclaw',
}))
}
async function invalidateOpenClawQueries(
@@ -310,19 +318,3 @@ export function buildChatHistoryFromTurns(
return messages
}
export async function chatWithAgent(
agentId: string,
message: string,
sessionKey?: string,
history: OpenClawChatHistoryMessage[] = [],
signal?: AbortSignal,
): Promise<Response> {
const baseUrl = await getAgentServerUrl()
return fetch(`${baseUrl}/claw/agents/${agentId}/chat`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ message, sessionKey, history }),
signal,
})
}

View File

@@ -164,9 +164,17 @@ export const NewScheduledTaskDialog: FC<NewScheduledTaskDialogProps> = ({
const resolvedProvider: Provider | null = (() => {
const id = selectedProviderId ?? defaultProviderId
const found = providers.find((p) => p.id === id)
if (found) return { id: found.id, name: found.name, type: found.type }
if (found) {
return {
kind: 'llm' as const,
id: found.id,
name: found.name,
type: found.type,
}
}
if (providers[0])
return {
kind: 'llm' as const,
id: providers[0].id,
name: providers[0].name,
type: providers[0].type,
@@ -175,6 +183,7 @@ export const NewScheduledTaskDialog: FC<NewScheduledTaskDialogProps> = ({
})()
const providerOptions: Provider[] = providers.map((p) => ({
kind: 'llm',
id: p.id,
name: p.name,
type: p.type,

View File

@@ -18,8 +18,8 @@ describe('route-utils', () => {
expect(shouldUseChatSession('/home/chat')).toBe(true)
})
it('keeps the focus grid on home while hiding it on dedicated full-screen routes', () => {
expect(shouldHideFocusGrid('/home')).toBe(false)
it('hides the focus grid on full-screen routes', () => {
expect(shouldHideFocusGrid('/home')).toBe(true)
expect(shouldHideFocusGrid('/home/agents/main')).toBe(true)
expect(shouldHideFocusGrid('/home/chat')).toBe(true)
expect(shouldHideFocusGrid('/home/skills')).toBe(true)

View File

@@ -1,4 +1,5 @@
const HIDE_FOCUS_GRID_PATHS = new Set([
'/home',
'/home/soul',
'/home/memory',
'/home/skills',

View File

@@ -1,4 +1,4 @@
import { Github, History, Plus, SettingsIcon } from 'lucide-react'
import { Bot, Github, History, Plus, SettingsIcon } from 'lucide-react'
import type { FC } from 'react'
import { Link, useLocation, useNavigate } from 'react-router'
import { ChatProviderSelector } from '@/components/chat/ChatProviderSelector'
@@ -64,7 +64,9 @@ export const ChatHeader: FC<ChatHeaderProps> = ({
className="group relative inline-flex cursor-pointer items-center gap-2 rounded-lg p-2 text-muted-foreground transition-colors hover:bg-muted/50 hover:text-foreground data-[state=open]:bg-accent"
title="Change AI Provider"
>
{selectedProvider.type === 'browseros' ? (
{selectedProvider.kind === 'acp' ? (
<Bot className="h-[18px] w-[18px]" />
) : selectedProvider.type === 'browseros' ? (
<BrowserOSIcon size={18} />
) : (
<ProviderIcon

View File

@@ -0,0 +1,258 @@
import { describe, expect, it } from 'bun:test'
import type {
HarnessAdapterDescriptor,
HarnessAgent,
} from '@/entrypoints/app/agents/agent-harness-types'
import type { LlmProviderConfig } from '@/lib/llm-providers/types'
import {
buildSidepanelChatTargets,
persistSidepanelChatTargetSelection,
resolveSidepanelChatTarget,
type SidepanelChatTargetSelection,
toLlmProviderConfig,
} from './sidepanel-chat-targets'
const timestamp = 1000
const providers: LlmProviderConfig[] = [
{
id: 'browseros',
type: 'browseros',
name: 'BrowserOS',
baseUrl: 'https://api.browseros.com/v1',
modelId: 'browseros-auto',
supportsImages: true,
contextWindow: 200000,
temperature: 0.2,
createdAt: timestamp,
updatedAt: timestamp,
},
{
id: 'anthropic-sonnet',
type: 'anthropic',
name: 'Anthropic Sonnet',
modelId: 'claude-sonnet-4-6',
apiKey: 'sk-ant',
supportsImages: true,
contextWindow: 200000,
temperature: 0.2,
createdAt: timestamp,
updatedAt: timestamp,
},
]
const adapters: HarnessAdapterDescriptor[] = [
{
id: 'claude',
name: 'Claude Code',
defaultModelId: 'haiku',
defaultReasoningEffort: 'medium',
modelControl: 'best-effort',
models: [
{ id: 'sonnet', label: 'Sonnet' },
{ id: 'haiku', label: 'Haiku', recommended: true },
],
reasoningEfforts: [
{ id: 'medium', label: 'Medium', recommended: true },
{ id: 'high', label: 'High' },
],
},
{
id: 'codex',
name: 'Codex',
defaultModelId: 'gpt-5.5',
defaultReasoningEffort: 'medium',
modelControl: 'runtime-supported',
models: [{ id: 'gpt-5.5', label: 'GPT-5.5', recommended: true }],
reasoningEfforts: [{ id: 'medium', label: 'Medium', recommended: true }],
},
{
id: 'openclaw',
name: 'OpenClaw',
defaultModelId: 'default',
defaultReasoningEffort: 'medium',
modelControl: 'best-effort',
models: [],
reasoningEfforts: [
{ id: 'medium', label: 'Medium', recommended: true },
{ id: 'high', label: 'High' },
],
},
]
const agents: HarnessAgent[] = [
{
id: 'agent-codex',
name: 'Review Bot',
adapter: 'codex',
modelId: 'gpt-5.5',
reasoningEffort: 'medium',
permissionMode: 'approve-all',
sessionKey: 'agent:agent-codex:main',
createdAt: timestamp,
updatedAt: timestamp,
},
{
id: 'agent-openclaw',
name: 'Research Claw',
adapter: 'openclaw',
modelId: 'default',
reasoningEffort: 'high',
permissionMode: 'approve-all',
sessionKey: 'agent:agent-openclaw:main',
createdAt: timestamp,
updatedAt: timestamp,
},
]
describe('buildSidepanelChatTargets', () => {
it('returns LLM targets plus one ACP target per persisted harness agent', () => {
const targets = buildSidepanelChatTargets({ providers, adapters, agents })
expect(targets.map((target) => target.id)).toEqual([
'browseros',
'anthropic-sonnet',
'agent-codex',
'agent-openclaw',
])
})
it('does not emit catalog-only ACP targets without persisted agents', () => {
const targets = buildSidepanelChatTargets({
providers,
adapters,
agents: [],
})
expect(targets.map((target) => target.id)).toEqual([
'browseros',
'anthropic-sonnet',
])
})
it('uses the created OpenClaw agent name instead of a generic adapter target', () => {
const targets = buildSidepanelChatTargets({ providers, adapters, agents })
const openclaw = targets.find((target) => target.id === 'agent-openclaw')
expect(openclaw).toMatchObject({
kind: 'acp',
id: 'agent-openclaw',
agentId: 'agent-openclaw',
adapter: 'openclaw',
adapterName: 'OpenClaw',
modelId: 'default',
modelLabel: 'default',
name: 'Research Claw',
modelControl: 'best-effort',
reasoningEffort: 'high',
})
})
it('preserves adapter metadata for created agent targets', () => {
const targets = buildSidepanelChatTargets({ providers, adapters, agents })
const codex = targets.find((target) => target.id === 'agent-codex')
expect(codex).toMatchObject({
kind: 'acp',
agentId: 'agent-codex',
adapter: 'codex',
adapterName: 'Codex',
modelId: 'gpt-5.5',
modelLabel: 'GPT-5.5',
modelControl: 'runtime-supported',
recommended: true,
reasoningEffort: 'medium',
reasoningEffortLabel: 'Medium',
})
})
it('still returns LLM targets when agents and adapters are unavailable', () => {
expect(
buildSidepanelChatTargets({ providers, adapters: [], agents: [] }),
).toEqual([
{
kind: 'llm',
id: 'browseros',
name: 'BrowserOS',
type: 'browseros',
provider: providers[0],
},
{
kind: 'llm',
id: 'anthropic-sonnet',
name: 'Anthropic Sonnet',
type: 'anthropic',
provider: providers[1],
},
])
})
})
describe('resolveSidepanelChatTarget', () => {
it('resolves selected LLM targets back to their provider config', () => {
const targets = buildSidepanelChatTargets({ providers, adapters, agents })
const resolved = resolveSidepanelChatTarget({
targets,
defaultProviderId: 'browseros',
selection: { kind: 'llm', id: 'anthropic-sonnet' },
})
expect(resolved?.kind).toBe('llm')
expect(toLlmProviderConfig(resolved)?.modelId).toBe('claude-sonnet-4-6')
})
it('falls back to the current default LLM provider when a persisted ACP target is stale', () => {
const targets = buildSidepanelChatTargets({
providers,
adapters,
agents: [],
})
expect(
resolveSidepanelChatTarget({
targets,
defaultProviderId: 'anthropic-sonnet',
selection: { kind: 'acp', id: 'agent-codex' },
}),
).toMatchObject({
kind: 'llm',
id: 'anthropic-sonnet',
})
})
it('falls back when an old catalog-style ACP target id is persisted', () => {
const targets = buildSidepanelChatTargets({ providers, adapters, agents })
expect(
resolveSidepanelChatTarget({
targets,
defaultProviderId: 'anthropic-sonnet',
selection: { kind: 'acp', id: 'acp:codex:gpt-5.5:medium' },
}),
).toMatchObject({
kind: 'llm',
id: 'anthropic-sonnet',
})
})
})
describe('persistSidepanelChatTargetSelection', () => {
it('stores only target identity and does not mutate LLM provider arrays', async () => {
let savedSelection: SidepanelChatTargetSelection | null = null
const originalProviders = providers.map((provider) => ({ ...provider }))
const targets = buildSidepanelChatTargets({ providers, adapters, agents })
const target = targets.find((candidate) => candidate.id === 'agent-codex')
await persistSidepanelChatTargetSelection(target, {
setValue: async (value) => {
savedSelection = value
},
})
expect(savedSelection as SidepanelChatTargetSelection | null).toEqual({
kind: 'acp',
id: 'agent-codex',
})
expect(providers).toEqual(originalProviders)
})
})

View File

@@ -0,0 +1,178 @@
import type {
HarnessAdapterDescriptor,
HarnessAgent,
HarnessAgentAdapter,
} from '@/entrypoints/app/agents/agent-harness-types'
import type { LlmProviderConfig, ProviderType } from '@/lib/llm-providers/types'
export type SidepanelTargetKind = 'llm' | 'acp'
export type SidepanelChatTarget =
| {
kind: 'llm'
id: string
name: string
type: ProviderType
provider: LlmProviderConfig
}
| {
kind: 'acp'
id: string
name: string
type: 'acp'
agentId: string
adapter: HarnessAgentAdapter
adapterName: string
modelId: string
modelLabel: string
modelControl: HarnessAdapterDescriptor['modelControl']
recommended?: boolean
reasoningEffort: string
reasoningEffortLabel?: string
}
export type SidepanelChatTargetSelection = Pick<
SidepanelChatTarget,
'kind' | 'id'
>
interface BuildSidepanelChatTargetsInput {
providers: LlmProviderConfig[]
adapters: HarnessAdapterDescriptor[]
agents?: HarnessAgent[]
}
interface ResolveSidepanelChatTargetInput {
targets: SidepanelChatTarget[]
defaultProviderId: string
selection?: SidepanelChatTargetSelection | null
}
interface SidepanelChatTargetSelectionWriter {
setValue(value: SidepanelChatTargetSelection | null): Promise<void>
}
interface SidepanelChatTargetSelectionReader {
getValue(): Promise<SidepanelChatTargetSelection | null>
}
type SidepanelChatTargetSelectionStore = SidepanelChatTargetSelectionReader &
SidepanelChatTargetSelectionWriter
let sidepanelChatTargetSelectionStorage:
| SidepanelChatTargetSelectionStore
| undefined
export function buildSidepanelChatTargets({
providers,
adapters,
agents = [],
}: BuildSidepanelChatTargetsInput): SidepanelChatTarget[] {
return [
...providers.map(toLlmTarget),
...agents.map((agent) => toAcpTargetForAgent(agent, adapters)),
]
}
function toAcpTargetForAgent(
agent: HarnessAgent,
adapters: HarnessAdapterDescriptor[],
): SidepanelChatTarget {
const adapter = adapters.find((entry) => entry.id === agent.adapter)
const modelId = agent.modelId ?? adapter?.defaultModelId ?? 'default'
const reasoningEffort =
agent.reasoningEffort ?? adapter?.defaultReasoningEffort ?? 'medium'
const model = adapter?.models.find((entry) => entry.id === modelId)
const reasoning = adapter?.reasoningEfforts.find(
(effort) => effort.id === reasoningEffort,
)
return {
kind: 'acp',
id: agent.id,
name: agent.name,
type: 'acp',
agentId: agent.id,
adapter: agent.adapter,
adapterName: adapter?.name ?? formatAdapterName(agent.adapter),
modelId,
modelLabel: model?.label ?? modelId,
modelControl: adapter?.modelControl ?? 'best-effort',
recommended: model?.recommended,
reasoningEffort,
reasoningEffortLabel: reasoning?.label,
}
}
function formatAdapterName(adapter: HarnessAgentAdapter): string {
if (adapter === 'claude') return 'Claude Code'
if (adapter === 'codex') return 'Codex'
if (adapter === 'openclaw') return 'OpenClaw'
return adapter
}
export function resolveSidepanelChatTarget({
targets,
defaultProviderId,
selection,
}: ResolveSidepanelChatTargetInput): SidepanelChatTarget | undefined {
if (selection) {
const selected = targets.find(
(target) => target.kind === selection.kind && target.id === selection.id,
)
if (selected) return selected
}
return (
targets.find(
(target) => target.kind === 'llm' && target.id === defaultProviderId,
) ?? targets.find((target) => target.kind === 'llm')
)
}
export function toLlmProviderConfig(
target: SidepanelChatTarget | undefined,
): LlmProviderConfig | undefined {
return target?.kind === 'llm' ? target.provider : undefined
}
export async function persistSidepanelChatTargetSelection(
target: SidepanelChatTarget | undefined,
store?: SidepanelChatTargetSelectionWriter,
): Promise<void> {
const targetStore = store ?? (await getSidepanelChatTargetSelectionStorage())
await targetStore.setValue(
target ? { kind: target.kind, id: target.id } : null,
)
}
export async function loadSidepanelChatTargetSelection(
store?: SidepanelChatTargetSelectionReader,
): Promise<SidepanelChatTargetSelection | null> {
const targetStore = store ?? (await getSidepanelChatTargetSelectionStorage())
return targetStore.getValue()
}
function toLlmTarget(provider: LlmProviderConfig): SidepanelChatTarget {
return {
kind: 'llm',
id: provider.id,
name: provider.name,
type: provider.type,
provider,
}
}
async function getSidepanelChatTargetSelectionStorage(): Promise<SidepanelChatTargetSelectionStore> {
if (sidepanelChatTargetSelectionStorage) {
return sidepanelChatTargetSelectionStorage
}
const { storage } = await import('@wxt-dev/storage')
sidepanelChatTargetSelectionStorage =
storage.defineItem<SidepanelChatTargetSelection | null>(
'local:sidepanel-chat-target-selection',
{ fallback: null },
)
return sidepanelChatTargetSelectionStorage
}

View File

@@ -1,9 +1,21 @@
import { useEffect, useRef } from 'react'
import { useCallback, useEffect, useMemo, useRef, useState } from 'react'
import useDeepCompareEffect from 'use-deep-compare-effect'
import {
useAgentAdapters,
useHarnessAgents,
} from '@/entrypoints/app/agents/useAgents'
import type { LlmProviderConfig } from '@/lib/llm-providers/types'
import { useLlmProviders } from '@/lib/llm-providers/useLlmProviders'
import { type McpServer, useMcpServers } from '@/lib/mcp/mcpServerStorage'
import { usePersonalization } from '@/lib/personalization/personalizationStorage'
import {
buildSidepanelChatTargets,
loadSidepanelChatTargetSelection,
persistSidepanelChatTargetSelection,
resolveSidepanelChatTarget,
type SidepanelChatTarget,
type SidepanelChatTargetSelection,
} from './sidepanel-chat-targets'
const constructMcpServers = (servers: McpServer[]) => {
return servers
@@ -23,14 +35,53 @@ const constructCustomServers = (servers: McpServer[]) => {
export const useChatRefs = () => {
const { servers: mcpServers } = useMcpServers()
const {
providers: llmProviders,
selectedProvider: selectedLlmProvider,
setDefaultProvider,
isLoading: isLoadingProviders,
} = useLlmProviders()
const { adapters, loading: isLoadingAdapters } = useAgentAdapters()
const { harnessAgents, loading: isLoadingAgents } = useHarnessAgents()
const { personalization } = usePersonalization()
const [targetSelection, setTargetSelection] =
useState<SidepanelChatTargetSelection | null>(null)
useEffect(() => {
let cancelled = false
loadSidepanelChatTargetSelection().then((selection) => {
if (!cancelled) setTargetSelection(selection)
})
return () => {
cancelled = true
}
}, [])
const chatTargets = useMemo(
() =>
buildSidepanelChatTargets({
providers: llmProviders,
adapters,
agents: harnessAgents,
}),
[llmProviders, adapters, harnessAgents],
)
const selectedChatTarget = useMemo(
() =>
resolveSidepanelChatTarget({
targets: chatTargets,
defaultProviderId: selectedLlmProvider?.id ?? llmProviders[0]?.id ?? '',
selection: targetSelection,
}),
[chatTargets, llmProviders, selectedLlmProvider, targetSelection],
)
const selectedLlmProviderRef = useRef<LlmProviderConfig | null>(
selectedLlmProvider,
)
const selectedChatTargetRef = useRef<SidepanelChatTarget | undefined>(
selectedChatTarget,
)
const enabledMcpServersRef = useRef(constructMcpServers(mcpServers))
const enabledCustomServersRef = useRef(constructCustomServers(mcpServers))
const personalizationRef = useRef(personalization)
@@ -41,16 +92,36 @@ export const useChatRefs = () => {
enabledCustomServersRef.current = constructCustomServers(mcpServers)
}, [selectedLlmProvider, mcpServers])
useEffect(() => {
selectedChatTargetRef.current = selectedChatTarget
}, [selectedChatTarget])
useEffect(() => {
personalizationRef.current = personalization
}, [personalization])
const selectChatTarget = useCallback(
async (target: SidepanelChatTarget | undefined) => {
selectedChatTargetRef.current = target
setTargetSelection(target ? { kind: target.kind, id: target.id } : null)
await persistSidepanelChatTargetSelection(target)
},
[],
)
return {
selectedLlmProviderRef,
selectedChatTargetRef,
enabledMcpServersRef,
enabledCustomServersRef,
personalizationRef,
llmProviders,
setDefaultProvider,
chatTargets,
selectedChatTarget,
selectChatTarget,
selectedLlmProvider,
isLoadingProviders,
isLoadingProviders:
isLoadingProviders || isLoadingAdapters || isLoadingAgents,
}
}

View File

@@ -0,0 +1,153 @@
import { describe, expect, it } from 'bun:test'
import type { LlmProviderConfig } from '@/lib/llm-providers/types'
import type { ChatMode } from './chatTypes'
import type { SidepanelChatTarget } from './sidepanel-chat-targets'
import { buildSidepanelPreparedSendMessagesRequest } from './useChatSessionRequest'
const conversationId = '00000000-0000-4000-8000-000000000001'
describe('buildSidepanelPreparedSendMessagesRequest', () => {
it('keeps LLM targets on the existing /chat request body', () => {
const request = buildSidepanelPreparedSendMessagesRequest({
agentServerUrl: 'http://127.0.0.1:5151',
target: llmTarget,
fallbackProvider,
message: 'Summarize this page',
...commonRequestInput(),
})
expect(request.api).toBe('http://127.0.0.1:5151/chat')
expect(request.body).toMatchObject({
message: 'Summarize this page',
conversationId,
provider: 'browseros',
providerType: 'browseros',
providerName: 'BrowserOS',
model: 'gpt-5',
mode: 'agent',
browserContext: {
activeTab: { id: 10, url: 'https://example.com', title: 'Example' },
enabledMcpServers: ['slack'],
},
userSystemPrompt: 'Be concise',
userWorkingDir: '/tmp/work',
previousConversation: [{ role: 'assistant', content: 'Prior answer' }],
selectedText: 'selected text',
selectedTextSource: {
url: 'https://example.com',
title: 'Example',
},
})
})
it('sends created-agent targets to the agent-id sidepanel route', () => {
const request = buildSidepanelPreparedSendMessagesRequest({
agentServerUrl: 'http://127.0.0.1:5151',
target: acpTarget,
fallbackProvider,
message: 'Inspect the current tab',
approvalResponses: [
{ approvalId: 'approval-1', approved: true, reason: 'ok' },
],
...commonRequestInput(),
})
expect(request.api).toBe(
'http://127.0.0.1:5151/agents/agent-codex/sidepanel/chat',
)
expect(request.body).toEqual({
conversationId,
message: 'Inspect the current tab',
browserContext: {
activeTab: { id: 10, url: 'https://example.com', title: 'Example' },
enabledMcpServers: ['slack'],
},
userSystemPrompt: 'Be concise',
userWorkingDir: '/tmp/work',
selectedText: 'selected text',
selectedTextSource: {
url: 'https://example.com',
title: 'Example',
},
})
})
it('keeps tool approval retry payloads scoped to LLM chat', () => {
const request = buildSidepanelPreparedSendMessagesRequest({
agentServerUrl: 'http://127.0.0.1:5151',
target: llmTarget,
fallbackProvider,
approvalResponses: [
{ approvalId: 'approval-1', approved: false, reason: 'no' },
],
...commonRequestInput(),
})
expect(request.api).toBe('http://127.0.0.1:5151/chat')
expect(request.body).toMatchObject({
message: '',
toolApprovalResponses: [
{ approvalId: 'approval-1', approved: false, reason: 'no' },
],
})
})
})
function commonRequestInput() {
return {
conversationId,
mode: 'agent' as ChatMode,
browserContext: {
activeTab: { id: 10, url: 'https://example.com', title: 'Example' },
enabledMcpServers: ['slack'],
},
userSystemPrompt: 'Be concise',
userWorkingDir: '/tmp/work',
previousConversation: [
{ role: 'assistant' as const, content: 'Prior answer' },
],
declinedApps: ['gmail'],
aclRules: [{ id: 'rule-1', sitePattern: '*://*/*', enabled: true }],
selectedText: 'selected text',
selectedTextSource: {
url: 'https://example.com',
title: 'Example',
},
toolApprovalConfig: { categories: { navigation: true } },
}
}
const fallbackProvider: LlmProviderConfig = {
id: 'browseros',
type: 'browseros',
name: 'BrowserOS',
modelId: 'gpt-5',
supportsImages: true,
contextWindow: 128000,
temperature: 0.7,
createdAt: 1000,
updatedAt: 1000,
}
const llmTarget: SidepanelChatTarget = {
kind: 'llm',
id: fallbackProvider.id,
name: fallbackProvider.name,
type: fallbackProvider.type,
provider: fallbackProvider,
}
const acpTarget: SidepanelChatTarget = {
kind: 'acp',
id: 'agent-codex',
name: 'Review bot',
type: 'acp',
agentId: 'agent-codex',
adapter: 'codex',
adapterName: 'Codex',
modelId: 'gpt-5.5',
modelLabel: 'GPT-5.5',
modelControl: 'best-effort',
reasoningEffort: 'medium',
reasoningEffortLabel: 'Medium',
}

View File

@@ -26,15 +26,14 @@ import { useInvalidateCredits } from '@/lib/credits/useCredits'
import { declinedAppsStorage } from '@/lib/declined-apps/storage'
import { useGraphqlQuery } from '@/lib/graphql/useGraphqlQuery'
import { createDefaultBrowserOSProvider } from '@/lib/llm-providers/storage'
import { useLlmProviders } from '@/lib/llm-providers/useLlmProviders'
import {
type ApprovalResponseData,
buildChatRequestBody,
type ChatRequestBrowserContext,
import type {
ApprovalResponseData,
ChatRequestBrowserContext,
} from '@/lib/messaging/server/buildChatRequestBody'
import { track } from '@/lib/metrics/track'
import { searchActionsStorage } from '@/lib/search-actions/searchActionsStorage'
import { selectedTextStorage } from '@/lib/selected-text/selectedTextStorage'
import { sentry } from '@/lib/sentry/sentry'
import { stopAgentStorage } from '@/lib/stop-agent/stop-agent-storage'
import {
type ApprovalResponse,
@@ -52,7 +51,12 @@ import {
import { selectedWorkspaceStorage } from '@/lib/workspace/workspace-storage'
import type { ChatMode } from './chatTypes'
import { GetConversationWithMessagesDocument } from './graphql/chatSessionDocument'
import { toLlmProviderConfig } from './sidepanel-chat-targets'
import { useChatRefs } from './useChatRefs'
import {
buildSidepanelPreparedSendMessagesRequest,
toProviderOption,
} from './useChatSessionRequest'
import { useExecutionHistoryTracker } from './useExecutionHistoryTracker'
import { useNotifyActiveTab } from './useNotifyActiveTab'
import { useRemoteConversationSave } from './useRemoteConversationSave'
@@ -186,16 +190,19 @@ const buildRequestBrowserContext = ({
export const useChatSession = (options?: ChatSessionOptions) => {
const {
selectedLlmProviderRef,
selectedChatTargetRef,
enabledMcpServersRef,
enabledCustomServersRef,
personalizationRef,
setDefaultProvider,
chatTargets,
selectedChatTarget,
selectChatTarget,
selectedLlmProvider,
isLoadingProviders,
} = useChatRefs()
const invalidateCredits = useInvalidateCredits()
const { providers: llmProviders, setDefaultProvider } = useLlmProviders()
const {
baseUrl: agentServerUrl,
isLoading: isLoadingAgentUrl,
@@ -218,11 +225,7 @@ export const useChatSession = (options?: ChatSessionOptions) => {
agentUrlRef.current = agentServerUrl
}, [agentServerUrl])
const providers: Provider[] = llmProviders.map((p) => ({
id: p.id,
name: p.name,
type: p.type,
}))
const providers: Provider[] = chatTargets.map(toProviderOption)
const [mode, setMode] = useState<ChatMode>('agent')
const [textToAction, setTextToAction] = useState<Map<string, ChatAction>>(
@@ -324,15 +327,8 @@ export const useChatSession = (options?: ChatSessionOptions) => {
textToActionRef.current = textToAction
}, [mode, textToAction])
const selectedProvider = selectedLlmProvider
? {
id: selectedLlmProvider.id,
name: selectedLlmProvider.name,
type:
selectedLlmProvider.id === 'browseros'
? ('browseros' as const)
: selectedLlmProvider.type,
}
const selectedProvider = selectedChatTarget
? toProviderOption(selectedChatTarget)
: providers[0]
const {
@@ -346,7 +342,8 @@ export const useChatSession = (options?: ChatSessionOptions) => {
} = useChat({
transport: new DefaultChatTransport({
prepareSendMessagesRequest: async ({ messages }) => {
const provider =
const target = selectedChatTargetRef.current
const fallbackProvider =
selectedLlmProviderRef.current ?? createDefaultBrowserOSProvider()
const activeTabsList = await chrome.tabs.query({
active: true,
@@ -395,51 +392,46 @@ export const useChatSession = (options?: ChatSessionOptions) => {
personalizationRef.current,
)
const approvalResponses = extractApprovalResponses(messages)
const commonRequest = {
conversationId: conversationIdRef.current,
mode: currentMode,
browserContext: requestBrowserContext,
userSystemPrompt,
userWorkingDir: workingDirRef.current,
previousConversation,
declinedApps,
aclRules: enabledAclRules,
toolApprovalConfig: approvalConfig,
}
const approvalResponses =
target?.kind === 'acp' ? null : extractApprovalResponses(messages)
if (approvalResponses) {
return {
api: `${agentUrlRef.current}/chat`,
body: buildChatRequestBody({
conversationId: conversationIdRef.current,
provider,
mode: currentMode,
browserContext: requestBrowserContext,
userSystemPrompt,
userWorkingDir: workingDirRef.current,
previousConversation,
declinedApps,
aclRules: enabledAclRules,
toolApprovalConfig: approvalConfig,
toolApprovalResponses: approvalResponses,
}),
}
return buildSidepanelPreparedSendMessagesRequest({
agentServerUrl: agentUrlRef.current ?? undefined,
target,
fallbackProvider,
...commonRequest,
approvalResponses,
})
}
const message = getLastMessageText(messages)
const result = {
api: `${agentUrlRef.current}/chat`,
body: buildChatRequestBody({
message,
conversationId: conversationIdRef.current,
provider,
mode: currentMode,
browserContext: requestBrowserContext,
userSystemPrompt,
userWorkingDir: workingDirRef.current,
previousConversation,
declinedApps,
aclRules: enabledAclRules,
selectedText: activeTabSelection?.text,
selectedTextSource: activeTabSelection
? {
url: activeTabSelection.url,
title: activeTabSelection.title,
}
: undefined,
toolApprovalConfig: approvalConfig,
}),
}
const result = buildSidepanelPreparedSendMessagesRequest({
agentServerUrl: agentUrlRef.current ?? undefined,
target,
fallbackProvider,
message,
...commonRequest,
selectedText: activeTabSelection?.text,
selectedTextSource: activeTabSelection
? {
url: activeTabSelection.url,
title: activeTabSelection.title,
}
: undefined,
})
// Track which tab's selection was sent so we can clear it on success
pendingSelectionTabKeyRef.current =
@@ -451,7 +443,7 @@ export const useChatSession = (options?: ChatSessionOptions) => {
sendAutomaticallyWhen: () => {
if (approvalJustRespondedRef.current) {
approvalJustRespondedRef.current = false
return true
return selectedChatTargetRef.current?.kind !== 'acp'
}
return false
},
@@ -686,10 +678,22 @@ export const useChatSession = (options?: ChatSessionOptions) => {
}, [dispatchMessage, isIntegrationsSynced])
const sendMessage = (params: { text: string; action?: ChatAction }) => {
const target = selectedChatTargetRef.current
const llmTargetProvider = toLlmProviderConfig(target)
const agentTarget = target?.kind === 'acp' ? target : undefined
track(MESSAGE_SENT_EVENT, {
mode,
provider_type: selectedLlmProvider?.type,
model: selectedLlmProvider?.modelId,
provider_id:
agentTarget?.agentId ??
llmTargetProvider?.id ??
selectedLlmProvider?.id,
provider_type: agentTarget ? 'acp' : llmTargetProvider?.type,
agent_id: agentTarget?.agentId,
adapter: agentTarget?.adapter,
model:
agentTarget?.modelId ??
llmTargetProvider?.modelId ??
selectedLlmProvider?.modelId,
})
if (!isIntegrationsSyncedRef.current) {
@@ -741,14 +745,54 @@ export const useChatSession = (options?: ChatSessionOptions) => {
addToolApprovalResponse(params)
}
const resetConversationState = () => {
stop()
void finishExecutionTask({ isAbort: true })
setConversationId(crypto.randomUUID())
setMessages([])
setTextToAction(new Map())
setLiked({})
setDisliked({})
setRestoredConversationId(null)
resetRemoteConversation()
}
const handleSelectProvider = (provider: Provider) => {
const fullProvider = llmProviders.find((p) => p.id === provider.id)
const target = chatTargets.find(
(candidate) =>
candidate.id === provider.id && candidate.kind === provider.kind,
)
if (!target) return
const previousTarget = selectedChatTargetRef.current
track(PROVIDER_SELECTED_EVENT, {
provider_id: provider.id,
provider_type: provider.type,
model_id: fullProvider?.modelId,
provider_id: target.id,
provider_type: target.kind === 'acp' ? 'acp' : target.type,
model_id:
target.kind === 'acp' ? target.modelId : target.provider.modelId,
agent_id: target.kind === 'acp' ? target.agentId : undefined,
adapter: target.kind === 'acp' ? target.adapter : undefined,
})
setDefaultProvider(provider.id)
void selectChatTarget(target).catch((error) => {
sentry.captureException(error, {
extra: {
message: 'Failed to persist sidepanel chat target selection',
targetId: target.id,
targetKind: target.kind,
},
})
})
if (target.kind === 'llm') setDefaultProvider(target.provider.id)
if (
previousTarget &&
(previousTarget.kind !== target.kind ||
previousTarget.id !== target.id) &&
messagesRef.current.length > 0
) {
resetConversationState()
}
}
const getActionForMessage = (message: UIMessage) => {
@@ -762,15 +806,7 @@ export const useChatSession = (options?: ChatSessionOptions) => {
const resetConversation = () => {
track(CONVERSATION_RESET_EVENT, { message_count: messages.length })
stop()
void finishExecutionTask({ isAbort: true })
setConversationId(crypto.randomUUID())
setMessages([])
setTextToAction(new Map())
setLiked({})
setDisliked({})
setRestoredConversationId(null)
resetRemoteConversation()
resetConversationState()
}
const isRestoringConversation =

View File

@@ -0,0 +1,74 @@
import type { Provider } from '../../../components/chat/chatComponentTypes'
import type { LlmProviderConfig } from '../../../lib/llm-providers/types'
import {
type ApprovalResponseData,
buildChatRequestBody,
} from '../../../lib/messaging/server/buildChatRequestBody'
import {
type SidepanelChatTarget,
toLlmProviderConfig,
} from './sidepanel-chat-targets'
type LlmChatRequestBodyInput = Parameters<typeof buildChatRequestBody>[0]
type CommonSidepanelRequestInput = Omit<
LlmChatRequestBodyInput,
'provider' | 'message' | 'toolApprovalResponses' | 'isScheduledTask'
>
interface BuildSidepanelPreparedSendMessagesRequestInput
extends CommonSidepanelRequestInput {
agentServerUrl: string | undefined
target: SidepanelChatTarget | undefined
fallbackProvider: LlmProviderConfig
message?: string
approvalResponses?: ApprovalResponseData[] | null
}
export function buildSidepanelPreparedSendMessagesRequest({
agentServerUrl,
target,
fallbackProvider,
message,
approvalResponses,
...common
}: BuildSidepanelPreparedSendMessagesRequestInput) {
if (target?.kind === 'acp') {
return {
api: `${agentServerUrl}/agents/${encodeURIComponent(target.agentId)}/sidepanel/chat`,
body: {
conversationId: common.conversationId,
message: message ?? '',
browserContext: common.browserContext,
userSystemPrompt: common.userSystemPrompt,
userWorkingDir: common.userWorkingDir,
selectedText: common.selectedText,
selectedTextSource: common.selectedTextSource,
},
}
}
const provider = toLlmProviderConfig(target) ?? fallbackProvider
return {
api: `${agentServerUrl}/chat`,
body: buildChatRequestBody({
...common,
provider,
message,
toolApprovalResponses: approvalResponses ?? undefined,
}),
}
}
export function toProviderOption(target: SidepanelChatTarget): Provider {
return {
id: target.id,
name: target.name,
type: target.type,
kind: target.kind,
agentId: target.kind === 'acp' ? target.agentId : undefined,
adapterName: target.kind === 'acp' ? target.adapterName : undefined,
modelLabel: target.kind === 'acp' ? target.modelLabel : undefined,
modelControl: target.kind === 'acp' ? target.modelControl : undefined,
}
}

View File

@@ -12,6 +12,8 @@ export interface AssistantThinkingPart {
export interface ToolEntry {
id: string
name: string
label: string
subject?: string
status: 'running' | 'completed' | 'error'
durationMs?: number
}
@@ -26,9 +28,24 @@ export type AssistantPart =
| AssistantThinkingPart
| AssistantToolBatchPart
/**
* Attachments rendered alongside the user's text on the optimistic turn
* — populated when the composer staged any images/files. The dataUrl is
* the same one the server received; we keep it in memory only for the
* lifetime of the live turn (history reload re-fetches via the JSONL).
*/
export interface UserAttachmentPreview {
id: string
kind: 'image' | 'file'
mediaType: string
name: string
dataUrl?: string
}
export interface AgentConversationTurn {
id: string
userText: string
userAttachments?: UserAttachmentPreview[]
parts: AssistantPart[]
done: boolean
timestamp: number
@@ -42,12 +59,3 @@ export interface AgentConversation {
createdAt: number
updatedAt: number
}
export interface AgentCardData {
agentId: string
name: string
model?: string
status: 'idle' | 'working' | 'error'
lastMessage?: string
lastMessageTimestamp?: number
}

View File

@@ -0,0 +1,369 @@
/**
* Composer attachment helpers — validation, image compression, and the
* client-side payload shape sent to /agents/:id/chat.
*
* Image attachments travel as `data:` URLs (base64) so the gateway, which
* runs on 127.0.0.1 over Lima virtiofs, can ingest them as standard
* OpenAI-style content blocks. Non-image text-shaped files are read into
* memory and travel as their extracted text body — the server inlines
* them as a fenced `<attachment>` block on the user message.
*/
export const MAX_ATTACHMENTS_PER_MESSAGE = 10
export const MAX_IMAGE_BYTES = 5 * 1024 * 1024 // 5 MB after compression
export const MAX_FILE_TEXT_BYTES = 1 * 1024 * 1024 // 1 MB extracted text
export const IMAGE_LONG_EDGE_CAP = 2048
export const ALLOWED_IMAGE_MEDIA_TYPES = [
'image/png',
'image/jpeg',
'image/jpg',
'image/webp',
'image/gif',
] as const
export const ALLOWED_FILE_MEDIA_TYPE_PREFIXES = [
'text/',
'application/json',
] as const
export type ServerImageAttachment = {
kind: 'image'
mediaType: string
dataUrl: string
name?: string
}
export type ServerFileAttachment = {
kind: 'file'
mediaType: string
name: string
text: string
}
export type ServerAttachmentPayload =
| ServerImageAttachment
| ServerFileAttachment
/** UI-side representation: what the composer needs to render a chip. */
export interface StagedAttachment {
id: string
kind: 'image' | 'file'
mediaType: string
name: string
// Set for images so the chip thumbnail can render directly. For files
// we don't need a preview yet, but the field exists for v2 PDF previews.
dataUrl?: string
// Pre-computed payload for the server. Built once at staging time so
// re-renders don't re-encode large blobs.
payload: ServerAttachmentPayload
}
export type AttachmentValidationError =
| { code: 'too_many'; message: string }
| { code: 'unsupported_type'; message: string; mediaType: string }
| { code: 'too_large'; message: string }
| { code: 'read_failed'; message: string }
export type StageAttachmentResult =
| { ok: true; attachment: StagedAttachment }
| { ok: false; error: AttachmentValidationError }
function isImageMediaType(mediaType: string): boolean {
return (ALLOWED_IMAGE_MEDIA_TYPES as readonly string[]).includes(mediaType)
}
function isAllowedFileMediaType(mediaType: string): boolean {
return ALLOWED_FILE_MEDIA_TYPE_PREFIXES.some((prefix) =>
mediaType.startsWith(prefix),
)
}
/** Build a unique id without depending on `crypto.randomUUID` outside DOM. */
function makeId(): string {
if (typeof crypto !== 'undefined' && crypto.randomUUID) {
return crypto.randomUUID()
}
return `att-${Date.now().toString(36)}-${Math.random().toString(36).slice(2, 10)}`
}
/**
* Read a `File` and produce the staged-attachment shape — validate type,
* compress if it's a large image, and pre-build the server payload.
*/
export async function stageAttachment(
file: File,
): Promise<StageAttachmentResult> {
const mediaType = file.type || 'application/octet-stream'
if (isImageMediaType(mediaType)) {
try {
const compressed = await compressImageIfNeeded(file)
const dataUrl = await readAsDataUrl(compressed)
// Rough byte ceiling — `data:image/png;base64,...` doubles size with
// base64. Reject early so we never POST something the route will 400.
if (dataUrl.length > MAX_IMAGE_BYTES * 2) {
return {
ok: false,
error: {
code: 'too_large',
message: `Image "${file.name}" is too large (max ${humanBytes(
MAX_IMAGE_BYTES,
)}).`,
},
}
}
return {
ok: true,
attachment: {
id: makeId(),
kind: 'image',
mediaType,
name: file.name || 'image',
dataUrl,
payload: {
kind: 'image',
mediaType,
dataUrl,
name: file.name || undefined,
},
},
}
} catch (err) {
return {
ok: false,
error: {
code: 'read_failed',
message:
err instanceof Error
? err.message
: `Failed to read image "${file.name}".`,
},
}
}
}
if (isAllowedFileMediaType(mediaType)) {
let text: string
try {
text = await file.text()
} catch (err) {
return {
ok: false,
error: {
code: 'read_failed',
message:
err instanceof Error
? err.message
: `Failed to read file "${file.name}".`,
},
}
}
if (text.length > MAX_FILE_TEXT_BYTES) {
return {
ok: false,
error: {
code: 'too_large',
message: `File "${file.name}" is too large (max ${humanBytes(
MAX_FILE_TEXT_BYTES,
)}).`,
},
}
}
return {
ok: true,
attachment: {
id: makeId(),
kind: 'file',
mediaType,
name: file.name || 'attachment',
payload: {
kind: 'file',
mediaType,
name: file.name || 'attachment',
text,
},
},
}
}
return {
ok: false,
error: {
code: 'unsupported_type',
message: `Unsupported attachment type: ${mediaType || 'unknown'}`,
mediaType,
},
}
}
/**
* Stage multiple files at once, enforcing the per-message cap. The result
* partitions successful stages and any errors so the caller can show
* granular toasts.
*/
export async function stageAttachments(
files: File[],
alreadyStaged: number,
): Promise<{
staged: StagedAttachment[]
errors: AttachmentValidationError[]
}> {
const remainingSlots = Math.max(
0,
MAX_ATTACHMENTS_PER_MESSAGE - alreadyStaged,
)
const staged: StagedAttachment[] = []
const errors: AttachmentValidationError[] = []
if (remainingSlots === 0 && files.length > 0) {
errors.push({
code: 'too_many',
message: `At most ${MAX_ATTACHMENTS_PER_MESSAGE} attachments per message.`,
})
return { staged, errors }
}
const overflow = files.length - remainingSlots
if (overflow > 0) {
errors.push({
code: 'too_many',
message: `Only the first ${remainingSlots} of ${files.length} files were attached (max ${MAX_ATTACHMENTS_PER_MESSAGE}).`,
})
}
for (const file of files.slice(0, remainingSlots)) {
const result = await stageAttachment(file)
if (result.ok) {
staged.push(result.attachment)
} else {
errors.push(result.error)
}
}
return { staged, errors }
}
/**
* Resize images that are oversized to a sane long-edge cap. JPEG/WebP
* source files are re-encoded to JPEG; PNGs/GIFs that are already small
* are passed through untouched.
*/
export async function compressImageIfNeeded(file: File): Promise<Blob> {
// Cheap path: small files don't need any transform.
if (file.size <= 1.5 * 1024 * 1024) return file
const bitmap = await blobToImageBitmap(file)
const { width, height } = bitmap
const longEdge = Math.max(width, height)
if (longEdge <= IMAGE_LONG_EDGE_CAP && file.size <= MAX_IMAGE_BYTES) {
bitmap.close?.()
return file
}
const scale = Math.min(1, IMAGE_LONG_EDGE_CAP / longEdge)
const targetWidth = Math.max(1, Math.round(width * scale))
const targetHeight = Math.max(1, Math.round(height * scale))
const canvas =
typeof OffscreenCanvas !== 'undefined'
? new OffscreenCanvas(targetWidth, targetHeight)
: Object.assign(document.createElement('canvas'), {
width: targetWidth,
height: targetHeight,
})
const ctx = canvas.getContext('2d') as
| CanvasRenderingContext2D
| OffscreenCanvasRenderingContext2D
| null
if (!ctx) {
bitmap.close?.()
return file
}
ctx.drawImage(bitmap, 0, 0, targetWidth, targetHeight)
bitmap.close?.()
const outputType = 'image/jpeg'
if (canvas instanceof HTMLCanvasElement) {
return await new Promise<Blob>((resolve, reject) => {
canvas.toBlob(
(blob) => {
if (blob) resolve(blob)
else reject(new Error('Image compression failed.'))
},
outputType,
0.85,
)
})
}
return await (canvas as OffscreenCanvas).convertToBlob({
type: outputType,
quality: 0.85,
})
}
async function blobToImageBitmap(blob: Blob): Promise<ImageBitmap> {
if (typeof createImageBitmap === 'function') {
return createImageBitmap(blob)
}
// Fallback: load via an Image element and use the canvas decode path.
const url = URL.createObjectURL(blob)
try {
const img = await new Promise<HTMLImageElement>((resolve, reject) => {
const el = new Image()
el.onload = () => resolve(el)
el.onerror = () =>
reject(new Error('Failed to decode image for compression.'))
el.src = url
})
const canvas = document.createElement('canvas')
canvas.width = img.naturalWidth
canvas.height = img.naturalHeight
const ctx = canvas.getContext('2d')
if (!ctx) throw new Error('Canvas 2D context unavailable.')
ctx.drawImage(img, 0, 0)
const blobOut = await new Promise<Blob | null>((resolve) =>
canvas.toBlob(resolve, 'image/png'),
)
if (!blobOut) throw new Error('Canvas toBlob returned null.')
return await createImageBitmap(blobOut)
} finally {
URL.revokeObjectURL(url)
}
}
async function readAsDataUrl(blob: Blob): Promise<string> {
if ('arrayBuffer' in blob && typeof FileReader === 'undefined') {
const buffer = await blob.arrayBuffer()
const base64 = arrayBufferToBase64(buffer)
const type = blob.type || 'application/octet-stream'
return `data:${type};base64,${base64}`
}
return await new Promise<string>((resolve, reject) => {
const reader = new FileReader()
reader.onload = () => resolve(reader.result as string)
reader.onerror = () =>
reject(reader.error ?? new Error('FileReader failed to read blob.'))
reader.readAsDataURL(blob)
})
}
function arrayBufferToBase64(buffer: ArrayBuffer): string {
const bytes = new Uint8Array(buffer)
let binary = ''
const chunkSize = 0x8000
for (let i = 0; i < bytes.byteLength; i += chunkSize) {
binary += String.fromCharCode.apply(
null,
Array.from(bytes.subarray(i, Math.min(i + chunkSize, bytes.byteLength))),
)
}
return btoa(binary)
}
function humanBytes(bytes: number): string {
if (bytes >= 1024 * 1024) return `${(bytes / 1024 / 1024).toFixed(0)} MB`
if (bytes >= 1024) return `${(bytes / 1024).toFixed(0)} KB`
return `${bytes} B`
}

View File

@@ -75,6 +75,12 @@ export const MCP_EXTERNAL_ACCESS_DISABLED_EVENT =
/** @public */
export const MCP_SERVER_RESTARTED_EVENT = 'settings.mcp_server.restarted'
/** @public */
export const AGENT_CREATED_EVENT = 'agents.agent.created'
/** @public */
export const AGENT_DELETED_EVENT = 'agents.agent.deleted'
/** @public */
export const NEW_SCHEDULED_TASK_CREATED_EVENT =
'settings.scheduled_task.created'

View File

@@ -2,29 +2,75 @@ function isAbortError(error: unknown): boolean {
return error instanceof DOMException && error.name === 'AbortError'
}
export interface ParsedSSEEvent<T> {
data: T
/** Numeric `id:` line on the same SSE event, if any. */
seq?: number
}
export function parseSSELines<T>(buffer: string): {
events: T[]
events: ParsedSSEEvent<T>[]
remainder: string
} {
// SSE events are separated by blank lines. Buffer lines until we hit
// a blank, then assemble each event. Lines we recognise: `id: <n>`
// and `data: <payload>`. Everything else is ignored.
const events: ParsedSSEEvent<T>[] = []
const lines = buffer.split('\n')
const remainder = lines.pop() ?? ''
const events: T[] = []
for (const line of lines) {
if (!line.startsWith('data: ')) continue
const payload = line.slice(6)
if (payload === '[DONE]') continue
try {
events.push(JSON.parse(payload) as T)
} catch {}
// Find the last blank-line boundary; everything after it is the
// remainder (next event partially received).
let lastBoundary = -1
for (let i = lines.length - 1; i >= 0; i--) {
if (lines[i] === '') {
lastBoundary = i
break
}
}
const completeLines = lastBoundary >= 0 ? lines.slice(0, lastBoundary) : []
const remainder =
lastBoundary >= 0 ? lines.slice(lastBoundary + 1).join('\n') : buffer
let currentSeq: number | undefined
let currentData: string | null = null
const flush = () => {
if (currentData != null && currentData !== '[DONE]') {
try {
events.push({
data: JSON.parse(currentData) as T,
seq: currentSeq,
})
} catch {
// ignore
}
}
currentSeq = undefined
currentData = null
}
for (const line of completeLines) {
if (line === '') {
flush()
continue
}
if (line.startsWith('id: ')) {
const n = Number.parseInt(line.slice(4).trim(), 10)
if (Number.isFinite(n)) currentSeq = n
continue
}
if (line.startsWith('data: ')) {
currentData = line.slice(6)
}
}
// Catch a complete trailing event with no terminating blank line —
// shouldn't happen in well-formed SSE, but be tolerant.
flush()
return { events, remainder }
}
export async function consumeSSEStream<T>(
response: Response,
onEvent: (event: T) => void,
onEvent: (event: T, meta: { seq?: number }) => void,
signal?: AbortSignal,
): Promise<void> {
const reader = response.body?.getReader()
@@ -49,7 +95,7 @@ export async function consumeSSEStream<T>(
buffer = remainder
for (const event of events) {
onEvent(event)
onEvent(event.data, { seq: event.seq })
}
}
} catch (error) {
@@ -64,7 +110,7 @@ export async function consumeSSEStream<T>(
if (buffer) {
const { events } = parseSSELines<T>(buffer)
for (const event of events) {
onEvent(event)
onEvent(event.data, { seq: event.seq })
}
}
}

View File

@@ -0,0 +1,325 @@
/**
* @license
* Copyright 2025 BrowserOS
* SPDX-License-Identifier: AGPL-3.0-or-later
*
* Maps raw tool names + arguments to human-readable activity labels for
* the chat UI activity view. The MCP ToolRegistry is the source of truth
* for tool *existence*; this file is the editorial layer that turns
* snake_case identifiers into agent-speak verbs.
*/
const VERB_OVERRIDES: Record<string, string> = {
// Navigation
navigate_page: 'Navigated to',
new_page: 'Opened tab',
new_hidden_page: 'Opened tab',
show_page: 'Showed tab',
close_page: 'Closed tab',
list_pages: 'Listed open tabs',
get_active_page: 'Got active tab',
move_page: 'Moved tab',
group_tabs: 'Grouped tabs',
// Page reading
take_snapshot: 'Captured page snapshot',
take_enhanced_snapshot: 'Captured detailed snapshot',
get_page_content: 'Read page content',
get_page_links: 'Extracted page links',
get_dom: 'Read page DOM',
search_dom: 'Searched page DOM',
take_screenshot: 'Took screenshot',
// Input
click: 'Clicked',
click_at: 'Clicked at coordinates',
hover: 'Hovered',
hover_at: 'Hovered at coordinates',
type_at: 'Typed at coordinates',
drag_at: 'Dragged',
focus: 'Focused element',
fill: 'Filled field',
clear: 'Cleared field',
check: 'Checked box',
uncheck: 'Unchecked box',
press_key: 'Pressed key',
upload_file: 'Uploaded file',
// Console / scripts
evaluate_script: 'Ran script',
get_console_logs: 'Read console logs',
// History / bookmarks
search_history: 'Searched history',
get_recent_history: 'Read recent history',
delete_history_url: 'Deleted history entry',
delete_history_range: 'Deleted history range',
get_bookmarks: 'Listed bookmarks',
create_bookmark: 'Created bookmark',
remove_bookmark: 'Removed bookmark',
update_bookmark: 'Updated bookmark',
move_bookmark: 'Moved bookmark',
search_bookmarks: 'Searched bookmarks',
// Filesystem (sandboxed)
read_file: 'Read file',
write_file: 'Wrote file',
find_files: 'Searched files',
// Memory
read_soul: 'Read soul memory',
read_core: 'Read core memory',
write_memory: 'Wrote memory',
search_memory: 'Searched memory',
update_soul: 'Updated soul memory',
update_core: 'Updated core memory',
// Web
web_search: 'Searched the web',
web_fetch: 'Fetched URL',
// Klavis / external apps (Strata)
connector_mcp_servers: 'Listed connected apps',
discover_server_categories_or_actions: 'Browsed available actions',
get_category_actions: 'Listed actions',
get_action_details: 'Looked up action',
execute_action: 'Ran external action',
search_documentation: 'Searched docs',
handle_auth_failure: 'Handled auth issue',
// Suggestions
suggest_schedule: 'Suggested schedule',
suggest_app_connection: 'Suggested app connect',
// BrowserOS info
browseros_info: 'Read BrowserOS info',
// Windows
list_windows: 'Listed windows',
focus_window: 'Focused window',
close_window: 'Closed window',
create_window: 'Created window',
}
// ──────────────────────────────────────────────────────────────────────
// Helpers
// ──────────────────────────────────────────────────────────────────────
function asString(value: unknown): string | undefined {
return typeof value === 'string' && value.length > 0 ? value : undefined
}
function stringField(
input: Record<string, unknown>,
...keys: string[]
): string | undefined {
for (const k of keys) {
const v = asString(input[k])
if (v) return v
}
return undefined
}
function truncate(text: string | undefined, max: number): string | undefined {
if (!text) return undefined
return text.length > max ? `${text.slice(0, max - 1)}` : text
}
function quote(value: string | undefined): string | undefined {
if (!value) return undefined
return `"${truncate(value, 60)}"`
}
function basename(path: string | undefined): string | undefined {
if (!path) return undefined
const parts = path.split(/[/\\]/).filter(Boolean)
return parts[parts.length - 1] ?? path
}
function formatUrl(value: unknown): string | undefined {
const url = asString(value)
if (!url) return undefined
try {
const parsed = new URL(url)
const host = parsed.host
const path = parsed.pathname === '/' ? '' : parsed.pathname
const display = path && path.length > 0 ? `${host}${path}` : host
return truncate(display, 60)
} catch {
return truncate(url, 60)
}
}
function coords(x: unknown, y: unknown): string | undefined {
if (typeof x === 'number' && typeof y === 'number') {
return `${Math.round(x)}, ${Math.round(y)}`
}
return undefined
}
// ──────────────────────────────────────────────────────────────────────
// Subject extractors
// ──────────────────────────────────────────────────────────────────────
type SubjectExtractor = (input: Record<string, unknown>) => string | undefined
const SUBJECT_EXTRACTORS: Record<string, SubjectExtractor> = {
// URL-bearing tools
new_page: (i) => formatUrl(i.url),
new_hidden_page: (i) => formatUrl(i.url),
navigate_page: (i) => {
const action = asString(i.action)
if (action === 'back') return 'back'
if (action === 'forward') return 'forward'
if (action === 'reload') return 'reload'
return formatUrl(i.url)
},
web_fetch: (i) => formatUrl(i.url),
// Search queries
web_search: (i) => quote(stringField(i, 'query', 'q')),
search_history: (i) => quote(stringField(i, 'query', 'text')),
search_bookmarks: (i) => quote(stringField(i, 'query', 'text')),
search_memory: (i) => quote(stringField(i, 'query', 'q')),
search_dom: (i) => quote(stringField(i, 'query', 'selector')),
search_documentation: (i) => quote(stringField(i, 'query', 'q')),
find_files: (i) => quote(stringField(i, 'pattern', 'query')),
// Element interactions
click: (i) => stringField(i, 'element'),
hover: (i) => stringField(i, 'element'),
focus: (i) => stringField(i, 'element'),
clear: (i) => stringField(i, 'element'),
check: (i) => stringField(i, 'element'),
uncheck: (i) => stringField(i, 'element'),
fill: (i) => {
const target = stringField(i, 'element')
const text = stringField(i, 'text')
if (target && text) return `${target}: ${truncate(text, 40)}`
return target ?? truncate(text, 40)
},
press_key: (i) => stringField(i, 'key'),
// Coordinate-based input
click_at: (i) => coords(i.x, i.y),
hover_at: (i) => coords(i.x, i.y),
type_at: (i) => {
const at = coords(i.x, i.y)
const text = stringField(i, 'text')
if (at && text) return `${at}: ${truncate(text, 40)}`
return at ?? truncate(text, 40)
},
drag_at: (i) => {
const from = coords(i.fromX, i.fromY)
const to = coords(i.toX, i.toY)
if (from && to) return `${from}${to}`
return from ?? to
},
// Tab management
show_page: (i) => {
const page = i.page
return typeof page === 'number' ? `tab ${page}` : asString(page)
},
close_page: (i) => {
const page = i.page
return typeof page === 'number' ? `tab ${page}` : asString(page)
},
move_page: (i) => {
const page = i.page
return typeof page === 'number' ? `tab ${page}` : asString(page)
},
// Page reads (take_snapshot, take_enhanced_snapshot, get_page_content,
// get_page_links, get_dom, take_screenshot) intentionally omit a
// subject — the only argument is a numeric page ID that's internal
// to the agent and meaningless to the user ("tab 4" tells them nothing).
// The verb alone communicates what happened.
// External actions via Strata
execute_action: (i) => {
const server = stringField(i, 'server_name')
const action = stringField(i, 'action_name')
if (server && action) return `${server} · ${action}`
return action ?? server
},
get_category_actions: (i) => stringField(i, 'category_name', 'server_name'),
get_action_details: (i) => stringField(i, 'action_name'),
discover_server_categories_or_actions: (i) =>
stringField(i, 'server_name', 'category_name'),
connector_mcp_servers: (i) => stringField(i, 'server_name'),
// Filesystem
read_file: (i) => basename(stringField(i, 'path')),
write_file: (i) => basename(stringField(i, 'path')),
// Memory writes — show first chars of content
write_memory: (i) => truncate(stringField(i, 'content', 'text'), 40),
update_soul: (i) => truncate(stringField(i, 'content'), 40),
update_core: (i) => truncate(stringField(i, 'content'), 40),
// Bookmarks
create_bookmark: (i) => stringField(i, 'title') ?? formatUrl(i.url),
remove_bookmark: (i) => stringField(i, 'id', 'title'),
update_bookmark: (i) => stringField(i, 'id', 'title'),
move_bookmark: (i) => stringField(i, 'id', 'title'),
// History
delete_history_url: (i) => formatUrl(i.url),
}
// ──────────────────────────────────────────────────────────────────────
// Public API
// ──────────────────────────────────────────────────────────────────────
export interface ToolLabelResult {
label: string
subject?: string
}
/**
* Strip MCP namespace prefixes (e.g. "browseros__", "mcp_") to find the
* canonical tool name used in the override maps.
*/
function canonicalName(rawName: string): string {
return rawName.replace(/^browseros__/, '').replace(/^mcp_/, '')
}
/**
* Convert a snake_case tool name into Sentence-case English as a fallback
* when no curated override exists.
*/
function humanizeToolName(rawName: string): string {
const stripped = canonicalName(rawName)
const words = stripped.split(/[_-]/).filter((w) => w.length > 0)
if (words.length === 0) return rawName
const first = words[0]!
return [
first.charAt(0).toUpperCase() + first.slice(1),
...words.slice(1),
].join(' ')
}
/**
* Build a human-readable label and subject string for a tool call,
* suitable for rendering in the chat activity view.
*/
export function buildToolLabel(
rawName: string,
input?: Record<string, unknown>,
): ToolLabelResult {
const canonical = canonicalName(rawName)
const label =
VERB_OVERRIDES[canonical] ??
VERB_OVERRIDES[rawName] ??
humanizeToolName(rawName)
const extractor = Object.hasOwn(SUBJECT_EXTRACTORS, canonical)
? SUBJECT_EXTRACTORS[canonical]
: Object.hasOwn(SUBJECT_EXTRACTORS, rawName)
? SUBJECT_EXTRACTORS[rawName]
: undefined
const subject = extractor && input ? extractor(input) : undefined
return { label, subject }
}

View File

@@ -9,6 +9,7 @@
"build": "bun run codegen && wxt build",
"build:dev": "bun --env-file=.env.development wxt build --mode development",
"zip": "wxt zip",
"test": "bun run ../../scripts/run-bun-test.ts ./apps/agent",
"compile": "bun --env-file=.env.development wxt prepare && tsgo --noEmit",
"lint": "bunx biome check",
"typecheck": "bun --env-file=.env.development wxt prepare && tsgo --noEmit",

View File

@@ -8,6 +8,7 @@ const chromiumArgs = [
'--show-component-extension-options',
'--disable-browseros-server',
'--disable-browseros-extensions',
'--browseros-dock-icon=dev',
]
if (env.BROWSEROS_CDP_PORT) {

View File

@@ -0,0 +1,51 @@
# Copy to .env.development for local eval runs.
# Provider keys used by existing config files.
OPENROUTER_API_KEY=
FIREWORKS_API_KEY=
ANTHROPIC_API_KEY=
OPENAI_API_KEY=
GOOGLE_GENERATIVE_AI_API_KEY=
# Claude Agent SDK token used by performance_grader.
CLAUDE_CODE_OAUTH_TOKEN=
# Suite-mode model selection.
EVAL_VARIANT=local
EVAL_AGENT_PROVIDER=openai-compatible
EVAL_AGENT_MODEL=
EVAL_AGENT_API_KEY=
EVAL_AGENT_BASE_URL=
EVAL_AGENT_SUPPORTS_IMAGES=true
# Optional suite-mode executor override for orchestrator suites.
EVAL_EXECUTOR_MODEL=
EVAL_EXECUTOR_API_KEY=
EVAL_EXECUTOR_BASE_URL=
# Clado visual action executor.
CLADO_ACTION_MODEL=
CLADO_ACTION_API_KEY=
CLADO_ACTION_BASE_URL=
# Backward-compatible alias used by older local scripts.
CLADO_ACTION_URL=
# BrowserOS runner.
BROWSEROS_BINARY=/Applications/BrowserOS.app/Contents/MacOS/BrowserOS
BROWSEROS_SERVER_URL=http://127.0.0.1:9110
BROWSEROS_SERVER_LOG_DIR=/tmp/browseros-server-logs
BROWSEROS_CONFIG_URL=
# Captcha solver extension.
NOPECHA_API_KEY=
# WebArena-Infinity.
WEBARENA_INFINITY_DIR=
INFINITY_APP_URL=
# R2 publishing and weekly report.
EVAL_R2_ACCOUNT_ID=
EVAL_R2_ACCESS_KEY_ID=
EVAL_R2_SECRET_ACCESS_KEY=
EVAL_R2_BUCKET=browseros-eval
EVAL_R2_CDN_BASE_URL=https://eval.browseros.com

View File

@@ -1,875 +0,0 @@
# Eval System - Production Grade Design Doc
## Current State Analysis
### What's Working Well
1. **Zod validation** - Already exists in `config-validator.ts`, reuses `LLMConfigSchema` from `@browseros/shared`
2. **Grader registry pattern** - `createGrader()` factory works well, easy to add new graders
3. **AgentEvaluator interface** - Clean interface: `execute() → AgentResult`
4. **Discriminated unions** - Messages, agent types use proper TypeScript patterns
5. **Capture utilities** - `ScreenshotCapture`, `MessageLogger`, `TrajectorySaver` are modular
### Key Problems
**1. No Agent Registry/Factory**
Agent creation is hardcoded if-else in `task-executor.ts`:
```typescript
// Current approach - not scalable
if (this.config.agent.type === 'single') {
const evaluator = new SingleAgentEvaluator(...)
} else if (this.config.agent.type === 'orchestrator-executor') {
const evaluator = new OrchestratorExecutorEvaluator(...)
}
// Adding new agent = modify this file
```
**2. Heavy Server Dependency**
Imports from `@browseros/server`:
- `GeminiAgent` - Core agent (necessary)
- `ToolExecutionHooks` - Hook interface
- `ResolvedAgentConfig` - Agent config type
- `AgentExecutionError` - Error type
- `VercelAIContentGenerator` - Provider adapter
- Gateway client functions
**3. Scattered Types**
- `src/types.ts` - Main types
- `agents/types.ts` - Agent interface
- `agents/orchestrator-executor/types.ts` - Orchestrator types
- `runner/types.ts` - Runner types
- `graders/types.ts` - Grader types
**4. Duplicated Capture Logic**
Both agent evaluators duplicate:
- Initialize ScreenshotCapture
- Initialize MessageLogger
- Set up tool hooks
- Handle timeouts
- Collect errors/warnings
**5. No Unified Utils**
Hooks, screenshot capture, message logging code is copy-pasted per agent type.
---
## Design Goals
1. **Easy to add new agents** - Register new agent type, implement interface, done
2. **Shared capture infrastructure** - All agents use same screenshot/logging utils
3. **Type-safe with Zod** - Config validation at entry point
4. **Minimal server coupling** - Only import what's necessary
5. **Clear folder structure** - Types where they belong
6. **Production patterns** - Factory, registry, composition
---
## Proposed Architecture
### Folder Structure
```
eval/src/
├── index.ts # Entry point, CLI
├── types/
│ ├── index.ts # Re-exports all types
│ ├── config.ts # EvalConfig, AgentConfig (Zod schemas + types)
│ ├── task.ts # Task, TaskMetadata
│ ├── message.ts # Message discriminated union
│ ├── result.ts # AgentResult, GraderResult
│ └── errors.ts # ErrorSource, TaskError, EvalWarning
├── agents/
│ ├── index.ts # Re-exports + auto-registration
│ ├── registry.ts # Agent registry + factory
│ ├── types.ts # AgentEvaluator interface, AgentContext
│ ├── single/
│ │ └── index.ts # SingleAgentEvaluator
│ └── orchestrator-executor/
│ ├── index.ts # OrchestratorExecutorEvaluator
│ ├── types.ts # Orchestrator-specific types only
│ ├── orchestrator.ts
│ ├── orchestrator-agent.ts
│ ├── orchestrator-tools.ts
│ ├── executor.ts
│ └── executor-store.ts
├── capture/
│ ├── index.ts # Re-exports
│ ├── types.ts # CaptureContext interface
│ ├── context.ts # CaptureContext class (bundles all capture)
│ ├── hooks.ts # createCaptureHooks() utility
│ ├── screenshot.ts # ScreenshotCapture
│ ├── message-logger.ts # MessageLogger
│ ├── trajectory-saver.ts # TrajectorySaver
│ └── window-manager.ts # WindowManager
├── graders/
│ ├── index.ts # Re-exports
│ ├── registry.ts # Grader registry (existing pattern)
│ ├── types.ts # Grader interface
│ ├── benchmark/
│ │ ├── webvoyager.ts
│ │ └── mind2web.ts
│ └── fara/
│ ├── alignment.ts
│ ├── rubric.ts
│ ├── multimodal.ts
│ └── combined.ts
├── runner/
│ ├── index.ts # runEval() main entry
│ ├── types.ts # RunEvalOptions, TaskResult, BatchSummary
│ ├── task-loader.ts
│ ├── task-executor.ts
│ └── parallel-executor.ts
└── utils/
├── env.ts # resolveEnvValue() helper
└── validation.ts # Config validation logic
```
---
## Key Components
### 1. Type System (`types/`)
**`types/config.ts`** - Zod schemas + inferred types:
```typescript
import { LLMConfigSchema, LLMProviderSchema } from '@browseros/shared/schemas/llm'
import { z } from 'zod'
// Single agent config
export const SingleAgentConfigSchema = LLMConfigSchema.extend({
type: z.literal('single'),
})
export type SingleAgentConfig = z.infer<typeof SingleAgentConfigSchema>
// Orchestrator-executor config
export const OrchestratorExecutorConfigSchema = z.object({
type: z.literal('orchestrator-executor'),
orchestrator: LLMConfigSchema.extend({
maxTurns: z.number().int().min(1).optional(),
}),
executor: LLMConfigSchema.extend({
maxStepsPerDelegation: z.number().int().min(1).optional(),
}),
})
export type OrchestratorExecutorConfig = z.infer<typeof OrchestratorExecutorConfigSchema>
// Discriminated union
export const AgentConfigSchema = z.discriminatedUnion('type', [
SingleAgentConfigSchema,
OrchestratorExecutorConfigSchema,
])
export type AgentConfig = z.infer<typeof AgentConfigSchema>
// Full eval config
export const EvalConfigSchema = z.object({
agent: AgentConfigSchema,
dataset: z.string().min(1),
output_dir: z.string().optional(),
num_workers: z.number().int().min(1).max(20).default(1),
browseros: z.object({
server_url: z.string().url(),
}),
grader_model: z.string().optional(),
grader_api_key_env: z.string().optional(),
grader_base_url: z.string().url().optional(),
timeout_ms: z.number().int().min(30000).max(3600000).optional(),
})
export type EvalConfig = z.infer<typeof EvalConfigSchema>
```
**`types/message.ts`** - Message types:
```typescript
import { z } from 'zod'
const BaseMessageSchema = z.object({
timestamp: z.string().datetime(),
})
export const UserMessageSchema = BaseMessageSchema.extend({
type: z.literal('user'),
content: z.string(),
})
export const AssistantMessageSchema = BaseMessageSchema.extend({
type: z.literal('assistant'),
content: z.string(),
})
export const ToolCallMessageSchema = BaseMessageSchema.extend({
type: z.literal('tool_call'),
tool: z.string(),
toolCallId: z.string(),
params: z.record(z.unknown()),
})
export const ToolResultMessageSchema = BaseMessageSchema.extend({
type: z.literal('tool_result'),
toolCallId: z.string(),
result: z.unknown(),
isError: z.boolean(),
screenshot: z.number().optional(),
})
export const ErrorMessageSchema = BaseMessageSchema.extend({
type: z.literal('error'),
content: z.string(),
errorCode: z.string().optional(),
})
// Orchestrator-specific messages
export const DelegationMessageSchema = BaseMessageSchema.extend({
type: z.literal('delegation'),
instruction: z.string(),
executorId: z.string(),
maxSteps: z.number().optional(),
})
export const DelegationResultMessageSchema = BaseMessageSchema.extend({
type: z.literal('delegation_result'),
executorId: z.string(),
summary: z.string(),
status: z.enum(['done', 'blocked', 'max_steps']),
stepsUsed: z.number(),
currentUrl: z.string().optional(),
})
export const MessageSchema = z.discriminatedUnion('type', [
UserMessageSchema,
AssistantMessageSchema,
ToolCallMessageSchema,
ToolResultMessageSchema,
ErrorMessageSchema,
DelegationMessageSchema,
DelegationResultMessageSchema,
])
export type Message = z.infer<typeof MessageSchema>
export type UserMessage = z.infer<typeof UserMessageSchema>
export type AssistantMessage = z.infer<typeof AssistantMessageSchema>
export type ToolCallMessage = z.infer<typeof ToolCallMessageSchema>
export type ToolResultMessage = z.infer<typeof ToolResultMessageSchema>
export type ErrorMessage = z.infer<typeof ErrorMessageSchema>
export type DelegationMessage = z.infer<typeof DelegationMessageSchema>
export type DelegationResultMessage = z.infer<typeof DelegationResultMessageSchema>
// Type guards
export const isToolCallMessage = (m: Message): m is ToolCallMessage => m.type === 'tool_call'
export const isDelegationMessage = (m: Message): m is DelegationMessage => m.type === 'delegation'
// ... etc
```
---
### 2. Agent Registry (`agents/registry.ts`)
```typescript
import type { AgentContext, AgentEvaluator } from './types'
type AgentFactory = (context: AgentContext) => AgentEvaluator
const registry = new Map<string, AgentFactory>()
/**
* Register an agent type
*/
export function registerAgent(type: string, factory: AgentFactory): void {
if (registry.has(type)) {
throw new Error(`Agent type "${type}" already registered`)
}
registry.set(type, factory)
}
/**
* Create agent evaluator from context
*/
export function createAgent(context: AgentContext): AgentEvaluator {
const factory = registry.get(context.config.agent.type)
if (!factory) {
const available = Array.from(registry.keys()).join(', ')
throw new Error(
`Unknown agent type: "${context.config.agent.type}". Available: ${available}`
)
}
return factory(context)
}
/**
* Get all registered agent types
*/
export function getRegisteredAgentTypes(): string[] {
return Array.from(registry.keys())
}
```
**`agents/index.ts`** - Auto-registration:
```typescript
import { registerAgent } from './registry'
import { SingleAgentEvaluator } from './single'
import { OrchestratorExecutorEvaluator } from './orchestrator-executor'
// Auto-register built-in agents
registerAgent('single', (ctx) => new SingleAgentEvaluator(ctx))
registerAgent('orchestrator-executor', (ctx) => new OrchestratorExecutorEvaluator(ctx))
// Re-exports
export { createAgent, registerAgent, getRegisteredAgentTypes } from './registry'
export type { AgentContext, AgentEvaluator, AgentResult } from './types'
```
---
### 3. Agent Context (`agents/types.ts`)
```typescript
import type { CaptureContext } from '../capture/types'
import type { EvalConfig, Task, TaskMetadata, Message } from '../types'
/**
* All dependencies an agent needs - passed to factory
*/
export interface AgentContext {
// Config
config: EvalConfig
task: Task
// Browser window
windowId: number
tabId: number
// Output
outputDir: string // Root output dir
taskOutputDir: string // Task-specific: outputDir/query_id/
// Capture infrastructure (pre-initialized)
capture: CaptureContext
}
/**
* Result returned by agent execution
*/
export interface AgentResult {
metadata: TaskMetadata
messages: Message[]
finalAnswer: string | null
}
/**
* Interface all agent evaluators must implement
*/
export interface AgentEvaluator {
/**
* Execute the agent on the task
*/
execute(): Promise<AgentResult>
}
```
---
### 4. Capture Context (`capture/context.ts`)
Bundle all capture utilities:
```typescript
import { randomUUID } from 'node:crypto'
import type { ToolExecutionHooks, ToolExecutionResult } from '@browseros/server/agent'
import type { Message, TaskError, EvalWarning, ErrorSource } from '../types'
import { MessageLogger } from './message-logger'
import { ScreenshotCapture } from './screenshot'
import { TrajectorySaver } from './trajectory-saver'
export interface CaptureContextConfig {
serverUrl: string
outputDir: string
taskId: string
tabId: number
windowId: number
}
/**
* Unified capture context - bundles screenshot, message logging, errors/warnings
*/
export class CaptureContext {
readonly screenshot: ScreenshotCapture
readonly messageLogger: MessageLogger
readonly trajectorySaver: TrajectorySaver
private errors: TaskError[] = []
private warnings: EvalWarning[] = []
private currentToolCallId: string | null = null
private readonly tabId: number
private readonly windowId: number
constructor(private config: CaptureContextConfig) {
this.tabId = config.tabId
this.windowId = config.windowId
this.trajectorySaver = new TrajectorySaver(config.outputDir, config.taskId)
}
/**
* Initialize - must be called before use
*/
async init(): Promise<string> {
const taskOutputDir = await this.trajectorySaver.init()
this.screenshot = new ScreenshotCapture(this.config.serverUrl, taskOutputDir)
await this.screenshot.init()
this.messageLogger = new MessageLogger(taskOutputDir)
return taskOutputDir
}
/**
* Create tool execution hooks for GeminiAgent
*/
createToolHooks(): ToolExecutionHooks {
return {
onBeforeToolCall: async (toolName: string, args: unknown) => {
try {
this.currentToolCallId = randomUUID()
await this.messageLogger.logToolCall(
toolName,
this.currentToolCallId,
args as Record<string, unknown>
)
} catch (err) {
this.addWarning('message_logging', `Failed to log tool call ${toolName}: ${err}`)
}
},
onAfterToolCall: async (toolName: string, result: ToolExecutionResult) => {
let screenshotNum = 0
// Capture screenshot
try {
screenshotNum = await this.screenshot.capture(this.tabId, this.windowId)
} catch (err) {
this.addWarning('screenshot', `Screenshot after ${toolName} failed: ${err}`)
screenshotNum = this.screenshot.getCount()
}
// Log tool errors
if (result.isError) {
this.addWarning('mcp_tool', `Tool ${toolName} error: ${result.errorMessage}`)
}
// Log result
if (this.currentToolCallId) {
try {
await this.messageLogger.logToolResult(
this.currentToolCallId,
result.isError ? { error: result.errorMessage } : result.parts,
result.isError,
screenshotNum
)
} catch (err) {
this.addWarning('message_logging', `Failed to log tool result: ${err}`)
}
}
this.currentToolCallId = null
},
}
}
// Error/warning collection
addError(source: ErrorSource, message: string, details?: Record<string, unknown>): void {
this.errors.push({ source, message, timestamp: new Date().toISOString(), details })
}
addWarning(source: ErrorSource, message: string): void {
this.warnings.push({ source, message, timestamp: new Date().toISOString() })
console.warn(`[${source}] ${message}`)
}
getErrors(): TaskError[] { return [...this.errors] }
getWarnings(): EvalWarning[] { return [...this.warnings] }
getMessages(): Message[] { return this.messageLogger.getMessages() }
getScreenshotCount(): number { return this.screenshot.getCount() }
getLastAssistantMessage(): string | null { return this.messageLogger.getLastAssistantMessage() }
// Delegation logging (for orchestrator-executor)
async logDelegation(instruction: string, executorId: string, maxSteps?: number): Promise<void> {
await this.messageLogger.logDelegation(instruction, executorId, maxSteps)
}
async logDelegationResult(
executorId: string,
summary: string,
status: 'done' | 'blocked' | 'max_steps',
stepsUsed: number,
currentUrl?: string
): Promise<void> {
await this.messageLogger.logDelegationResult(executorId, summary, status, stepsUsed, currentUrl)
}
}
```
---
### 5. Single Agent Evaluator (`agents/single/index.ts`)
Clean implementation using context:
```typescript
import { randomUUID } from 'node:crypto'
import { GeminiAgent } from '@browseros/server/agent'
import { AgentExecutionError } from '@browseros/server/agent/errors'
import type { ResolvedAgentConfig } from '@browseros/server/agent/types'
import { MCPServerConfig } from '@google/gemini-cli-core'
import type { AgentContext, AgentEvaluator, AgentResult } from '../types'
import type { SingleAgentConfig, TaskMetadata } from '../../types'
import { resolveEnvValue } from '../../utils/env'
const DEFAULT_TIMEOUT_MS = 15 * 60 * 1000
export class SingleAgentEvaluator implements AgentEvaluator {
constructor(private ctx: AgentContext) {}
async execute(): Promise<AgentResult> {
const startTime = Date.now()
const { config, task, capture } = this.ctx
const agentConfig = config.agent as SingleAgentConfig
const timeoutMs = config.timeout_ms ?? DEFAULT_TIMEOUT_MS
// Log initial user message
await capture.messageLogger.logUser(task.query)
// Set up timeout
const abortController = new AbortController()
const timeoutHandle = setTimeout(() => abortController.abort(), timeoutMs)
// Create agent
const resolvedConfig: ResolvedAgentConfig = {
conversationId: randomUUID(),
provider: agentConfig.provider,
model: agentConfig.model ?? 'gemini-2.0-flash',
apiKey: resolveEnvValue(agentConfig.apiKey),
baseUrl: agentConfig.baseUrl,
sessionExecutionDir: '/tmp/browseros-eval',
evalMode: true,
}
const mcpServers = {
'browseros-mcp': new MCPServerConfig(
undefined, undefined, undefined, undefined, undefined,
`${config.browseros.server_url}/mcp`,
{ Accept: 'application/json, text/event-stream', 'X-BrowserOS-Source': 'eval' },
undefined, undefined, true
),
}
const agent = await GeminiAgent.create(resolvedConfig, mcpServers)
// Set capture hooks
agent.setToolHooks(capture.createToolHooks())
// Create mock stream to capture assistant messages
let lastAssistantMessage = ''
const mockStream = {
write: async (data: string) => {
if (data.includes('"type":"text-delta"')) {
const match = data.match(/"delta":"((?:[^"\\]|\\.)*)"/)
if (match) lastAssistantMessage += JSON.parse(`"${match[1]}"`)
} else if (data.includes('"type":"finish"')) {
if (lastAssistantMessage) {
await capture.messageLogger.logAssistant(lastAssistantMessage)
lastAssistantMessage = ''
}
}
},
}
// Execute
let terminationReason: TaskMetadata['termination_reason'] = 'completed'
try {
await agent.execute(
task.query,
mockStream as Parameters<typeof agent.execute>[1],
abortController.signal,
{ windowId: this.ctx.windowId, activeTab: { id: this.ctx.tabId, url: task.start_url } }
)
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err))
if (abortController.signal.aborted) {
terminationReason = 'timeout'
capture.addError('agent_execution', `Task timed out after ${timeoutMs / 1000}s`)
} else {
terminationReason = 'error'
const msg = err instanceof AgentExecutionError && err.originalError
? `${error.message}: ${err.originalError.message}`
: error.message
capture.addError('agent_execution', msg, { stack: error.stack })
}
await capture.messageLogger.logError(error.message)
} finally {
clearTimeout(timeoutHandle)
}
// Build metadata
const metadata: TaskMetadata = {
query_id: task.query_id,
dataset: task.dataset,
query: task.query,
started_at: new Date(startTime).toISOString(),
completed_at: new Date().toISOString(),
total_duration_ms: Date.now() - startTime,
total_steps: capture.getScreenshotCount(),
termination_reason: terminationReason,
final_answer: capture.getLastAssistantMessage(),
errors: capture.getErrors(),
warnings: capture.getWarnings(),
agent_config: { type: 'single', model: resolvedConfig.model },
grader_results: {},
}
await capture.trajectorySaver.saveMetadata(metadata)
return {
metadata,
messages: capture.getMessages(),
finalAnswer: metadata.final_answer,
}
}
}
```
---
### 6. Task Executor (`runner/task-executor.ts`)
Uses agent registry:
```typescript
import { createAgent } from '../agents'
import type { AgentContext } from '../agents/types'
import { CaptureContext } from '../capture/context'
import type { EvalConfig, Task } from '../types'
import type { WindowManager } from '../capture/window-manager'
export class TaskExecutor {
constructor(
private config: EvalConfig,
private outputDir: string,
private windowManager: WindowManager,
private graderOptions: GraderOptions | null,
) {}
async execute(task: Task): Promise<TaskResult> {
const startTime = Date.now()
let window: { windowId: number; tabId: number } | null = null
try {
// Create window
window = await this.windowManager.createWindow(task.query_id, task.start_url)
// Initialize capture context
const capture = new CaptureContext({
serverUrl: this.config.browseros.server_url,
outputDir: this.outputDir,
taskId: task.query_id,
tabId: window.tabId,
windowId: window.windowId,
})
const taskOutputDir = await capture.init()
// Build agent context
const context: AgentContext = {
config: this.config,
task,
windowId: window.windowId,
tabId: window.tabId,
outputDir: this.outputDir,
taskOutputDir,
capture,
}
// Create and execute agent (via registry)
const agent = createAgent(context)
const agentResult = await agent.execute()
// Run graders
const graderResults = await this.runGraders(task, agentResult)
return {
status: agentResult.metadata.termination_reason === 'timeout' ? 'timeout' : 'completed',
task,
agentResult,
graderResults,
durationMs: Date.now() - startTime,
}
} catch (error) {
return {
status: 'failed',
task,
error: error instanceof Error ? error : new Error(String(error)),
errorSource: 'unknown',
durationMs: Date.now() - startTime,
}
} finally {
if (window) {
await this.windowManager.closeWindow(task.query_id)
}
}
}
}
```
---
## Server Dependencies
### What We MUST Import from Server
These are necessary - `GeminiAgent` IS the agent:
```typescript
// Core agent
import { GeminiAgent, type ToolExecutionHooks, type ToolExecutionResult } from '@browseros/server/agent'
import { AgentExecutionError } from '@browseros/server/agent/errors'
import type { ResolvedAgentConfig } from '@browseros/server/agent/types'
// Provider adapter (for orchestrator-agent)
import { VercelAIContentGenerator } from '@browseros/server/agent/provider-adapter'
// Gateway client (for browseros provider only)
import { fetchBrowserOSConfig, getLLMConfigFromProvider } from '@browseros/server/lib/clients/gateway'
```
### What Could Move to Shared (Future)
If we want to decouple more:
```typescript
// These types could be in @browseros/shared
export interface ToolExecutionHooks { ... }
export interface ToolExecutionResult { ... }
export interface ResolvedAgentConfig { ... }
```
But for now, importing from server is fine - eval is tightly coupled to server anyway.
---
## Import Guidelines
```typescript
// Shared package - schemas, constants
import { LLMConfigSchema, LLMProviderSchema, LLM_PROVIDERS } from '@browseros/shared/schemas/llm'
import { TIMEOUTS } from '@browseros/shared/constants/timeouts'
import { AGENT_LIMITS } from '@browseros/shared/constants/limits'
import type { BrowserContext } from '@browseros/shared/schemas/browser-context'
// Server - only agent-related imports
import { GeminiAgent, type ToolExecutionHooks } from '@browseros/server/agent'
import type { ResolvedAgentConfig } from '@browseros/server/agent/types'
// Internal eval types - from types/ folder
import type { EvalConfig, Task, Message, AgentResult } from '../types'
import type { AgentContext, AgentEvaluator } from '../agents/types'
```
---
## Adding a New Agent Type
1. Create folder: `agents/my-new-agent/`
2. Implement `AgentEvaluator` interface:
```typescript
// agents/my-new-agent/index.ts
import type { AgentContext, AgentEvaluator, AgentResult } from '../types'
export class MyNewAgentEvaluator implements AgentEvaluator {
constructor(private ctx: AgentContext) {}
async execute(): Promise<AgentResult> {
const { config, task, capture } = this.ctx
// Use capture.createToolHooks() for screenshot/logging
// Use capture.messageLogger for messages
// Use capture.addError/addWarning for errors
// Return AgentResult
}
}
```
3. Register in `agents/index.ts`:
```typescript
import { MyNewAgentEvaluator } from './my-new-agent'
registerAgent('my-new-agent', (ctx) => new MyNewAgentEvaluator(ctx))
```
4. Add config schema in `types/config.ts`:
```typescript
export const MyNewAgentConfigSchema = z.object({
type: z.literal('my-new-agent'),
// ... specific fields
})
export const AgentConfigSchema = z.discriminatedUnion('type', [
SingleAgentConfigSchema,
OrchestratorExecutorConfigSchema,
MyNewAgentConfigSchema, // Add here
])
```
Done - no changes to runner code needed.
---
## Implementation Order
1. **Phase 1: Types** (~1 hour)
- Create `types/` folder with proper structure
- Move/consolidate all types
- Add Zod schemas for messages
2. **Phase 2: Capture Context** (~1 hour)
- Create `CaptureContext` class
- Add delegation message methods
- Create `createToolHooks()` utility
3. **Phase 3: Agent Registry** (~30 min)
- Create `registry.ts`
- Create `AgentContext` interface
- Update exports
4. **Phase 4: Refactor Single Agent** (~1 hour)
- Use `AgentContext`
- Use `CaptureContext`
- Clean up code
5. **Phase 5: Refactor Orchestrator-Executor** (~2 hours)
- Use `AgentContext`
- Integrate `CaptureContext`
- Wire up hooks properly
6. **Phase 6: Update Runner** (~30 min)
- Use `createAgent()` instead of if-else
- Initialize `CaptureContext` in executor
7. **Phase 7: Testing** (~1 hour)
- Run single-agent eval
- Run orchestrator-executor eval
- Verify screenshots/messages captured
---
## Summary
| Before | After |
|--------|-------|
| If-else agent creation | Registry + factory pattern |
| Duplicated capture code | Shared `CaptureContext` |
| Scattered types | Organized `types/` folder |
| Copy-paste hooks | `createToolHooks()` utility |
| Tight coupling | Clear interfaces |
| Hard to add agents | Register + implement |

View File

@@ -1,431 +0,0 @@
# Implementation Phases - Parallel Execution Plan
## Dependency Graph
```
Phase 1: Types (4 parallel subagents)
├──────────────────┬──────────────────┐
▼ ▼ │
Phase 2: Capture Phase 3: Agent │
(2 parallel) Registry │
│ (1 subagent) │
│ │ │
└────────┬─────────┘ │
▼ │
Phase 4: Agent Refactors │
(2 parallel - after 2+3) │
│ │
▼ │
Phase 5: Runner Update │
(1 subagent - after 4) │
│ │
▼ │
Phase 6: Cleanup & Test ◄─────────────────┘
(1 subagent)
```
---
## Phase 1: Types (4 Parallel Subagents)
No dependencies - can all run simultaneously.
### Subagent 1A: Config Types
```
Create /apps/eval/src/types/config.ts
Requirements:
1. Import LLMConfigSchema, LLMProviderSchema from @browseros/shared/schemas/llm
2. Import z from zod
Create Zod schemas:
- SingleAgentConfigSchema = LLMConfigSchema.extend({ type: z.literal('single') })
- OrchestratorExecutorConfigSchema with orchestrator + executor nested configs
- AgentConfigSchema = z.discriminatedUnion('type', [...])
- EvalConfigSchema with all fields (agent, dataset, output_dir, num_workers, browseros, grader_*, timeout_ms)
Export both schemas and inferred types (z.infer<>)
Reference: Current implementation in /apps/eval/src/utils/config-validator.ts (lines 1-42)
```
### Subagent 1B: Message Types
```
Create /apps/eval/src/types/message.ts
Requirements:
1. Use Zod for all schemas
2. Create BaseMessageSchema with timestamp field
Create schemas for:
- UserMessageSchema (type: 'user', content)
- AssistantMessageSchema (type: 'assistant', content)
- ToolCallMessageSchema (type: 'tool_call', tool, toolCallId, params)
- ToolResultMessageSchema (type: 'tool_result', toolCallId, result, isError, screenshot?)
- ErrorMessageSchema (type: 'error', content, errorCode?)
- DelegationMessageSchema (type: 'delegation', instruction, executorId, maxSteps?)
- DelegationResultMessageSchema (type: 'delegation_result', executorId, summary, status, stepsUsed, currentUrl?)
Create MessageSchema = z.discriminatedUnion('type', [...all schemas])
Export schemas, types, and type guards (isToolCallMessage, isDelegationMessage, etc.)
Reference: Current types in /apps/eval/src/types.ts (lines 62-127)
```
### Subagent 1C: Task & Result Types
```
Create /apps/eval/src/types/task.ts
Requirements:
1. Use Zod schemas with inferred types
Create:
- TaskMetadataSchema (original_task_id, website?, category?, additional?)
- TaskSchema (query_id, dataset, query, graders[], start_url?, setup_script?, metadata)
Export schemas and types.
---
Create /apps/eval/src/types/result.ts
Create:
- GraderResultSchema (score, pass, reasoning, details?)
- TaskMetadataSchema (query_id, dataset, query, started_at, completed_at, total_duration_ms, total_steps, termination_reason, final_answer, errors, warnings, agent_config, grader_results)
- AgentResultSchema (metadata, messages, finalAnswer)
Export schemas and types.
Reference: Current types in /apps/eval/src/types.ts (lines 6-20, 156-182)
```
### Subagent 1D: Error Types + Index
```
Create /apps/eval/src/types/errors.ts
Create:
- ErrorSourceSchema = z.enum(['window_creation', 'agent_execution', 'mcp_tool', 'screenshot', 'grader', 'message_logging', 'cleanup', 'unknown'])
- TaskErrorSchema (source, message, timestamp, details?)
- EvalWarningSchema (source, message, timestamp)
Export schemas and types.
---
Create /apps/eval/src/types/index.ts
Re-export everything from:
- ./config
- ./message
- ./task
- ./result
- ./errors
This becomes the single import point: import { EvalConfig, Message, Task } from '../types'
Reference: Current types in /apps/eval/src/types.ts (lines 129-154)
```
---
## Phase 2: Capture Infrastructure (2 Parallel Subagents)
**Depends on:** Phase 1 (types)
### Subagent 2A: CaptureContext Class
```
Create /apps/eval/src/capture/types.ts
Define interface:
- CaptureContextConfig { serverUrl, outputDir, taskId, tabId, windowId }
---
Create /apps/eval/src/capture/context.ts
Requirements:
1. Import ToolExecutionHooks, ToolExecutionResult from @browseros/server/agent
2. Import types from ../types
3. Import existing ScreenshotCapture, MessageLogger, TrajectorySaver
Implement CaptureContext class:
- Constructor takes CaptureContextConfig
- async init() - initializes screenshot, messageLogger, trajectorySaver, returns taskOutputDir
- createToolHooks(): ToolExecutionHooks - returns hooks for GeminiAgent
- addError(source, message, details?)
- addWarning(source, message)
- getErrors(), getWarnings(), getMessages(), getScreenshotCount(), getLastAssistantMessage()
- logDelegation(instruction, executorId, maxSteps?)
- logDelegationResult(executorId, summary, status, stepsUsed, currentUrl?)
Reference implementation details in DESIGN_DOC.md section "4. Capture Context"
Update /apps/eval/src/capture/index.ts to export CaptureContext
```
### Subagent 2B: MessageLogger Extensions
```
Update /apps/eval/src/capture/message-logger.ts
Add two new methods:
1. logDelegation(instruction: string, executorId: string, maxSteps?: number): Promise<void>
- Creates DelegationMessage with type: 'delegation'
- Appends to messages
2. logDelegationResult(executorId: string, summary: string, status: 'done' | 'blocked' | 'max_steps', stepsUsed: number, currentUrl?: string): Promise<void>
- Creates DelegationResultMessage with type: 'delegation_result'
- Appends to messages
Import DelegationMessage, DelegationResultMessage from ../types
Reference: Current MessageLogger in /apps/eval/src/capture/message-logger.ts
```
---
## Phase 3: Agent Registry (1 Subagent)
**Depends on:** Phase 1 (types)
**Can run parallel with:** Phase 2
### Subagent 3A: Agent Registry + Types
```
Create /apps/eval/src/agents/types.ts
Define:
- AgentContext interface:
{
config: EvalConfig
task: Task
windowId: number
tabId: number
outputDir: string
taskOutputDir: string
capture: CaptureContext
}
- AgentResult interface (re-export from ../types or define here)
- AgentEvaluator interface { execute(): Promise<AgentResult> }
---
Create /apps/eval/src/agents/registry.ts
Implement:
- type AgentFactory = (context: AgentContext) => AgentEvaluator
- const registry = new Map<string, AgentFactory>()
- registerAgent(type: string, factory: AgentFactory): void
- createAgent(context: AgentContext): AgentEvaluator
- getRegisteredAgentTypes(): string[]
Reference: DESIGN_DOC.md section "2. Agent Registry"
---
Update /apps/eval/src/agents/index.ts
- Import registerAgent from ./registry
- Import SingleAgentEvaluator (will be updated later)
- Import OrchestratorExecutorEvaluator (will be updated later)
- Call registerAgent for both
- Re-export createAgent, registerAgent, getRegisteredAgentTypes
- Re-export types
Note: Registration calls will fail initially until agents are refactored.
That's OK - add TODO comments for now.
```
---
## Phase 4: Agent Refactors (2 Parallel Subagents)
**Depends on:** Phase 2 + Phase 3
### Subagent 4A: Single Agent Refactor
```
Refactor /apps/eval/src/agents/single-agent.ts
Changes:
1. Change constructor to accept AgentContext instead of individual params:
constructor(private ctx: AgentContext) {}
2. Use ctx.capture instead of creating ScreenshotCapture/MessageLogger:
- Remove local ScreenshotCapture initialization
- Remove local MessageLogger initialization
- Remove local hooks setup
- Use ctx.capture.createToolHooks() for GeminiAgent hooks
- Use ctx.capture.messageLogger.logUser/logAssistant
- Use ctx.capture.addError/addWarning
- Use ctx.capture.getMessages(), getScreenshotCount(), etc.
3. Build metadata using capture methods
4. Remove TrajectorySaver init (done in CaptureContext)
5. Keep the core agent execution logic (GeminiAgent.create, agent.execute)
Reference:
- Current implementation: /apps/eval/src/agents/single-agent.ts
- Target implementation: DESIGN_DOC.md section "5. Single Agent Evaluator"
```
### Subagent 4B: Orchestrator-Executor Refactor
```
Refactor /apps/eval/src/agents/orchestrator-executor/index.ts
Changes:
1. Change OrchestratorExecutorEvaluator constructor to accept AgentContext:
constructor(private ctx: AgentContext) {}
2. Initialize capture from context (already done in runner)
3. Add hook integration:
- Create executor hooks that use ctx.capture.createToolHooks()
- Wire hooks through Orchestrator → ExecutorStore → Executor
- Call ctx.capture.logDelegation() when orchestrator delegates
- Call ctx.capture.logDelegationResult() when executor returns
4. Update return to include messages:
return {
metadata,
messages: ctx.capture.getMessages(), // Now populated!
finalAnswer,
}
Also update supporting files if needed:
- orchestrator.ts - add setExecutorHooks() method
- executor.ts - accept external hooks via setObservationHooks()
- executor-store.ts - pass hooks to new executors
Reference:
- Current: /apps/eval/src/agents/orchestrator-executor/index.ts
- Target: DESIGN_DOC.md and previous IMPLEMENTATION_PLAN.md
```
---
## Phase 5: Runner Update (1 Subagent)
**Depends on:** Phase 4
### Subagent 5A: Task Executor Update
```
Update /apps/eval/src/runner/task-executor.ts
Changes:
1. Import createAgent from ../agents instead of individual evaluators
2. Import CaptureContext from ../capture
3. In execute() method:
- Create CaptureContext and call init()
- Build AgentContext with all required fields
- Use createAgent(context) instead of if-else switch
- Remove the if (config.agent.type === 'single') / else if blocks
4. Remove direct imports of SingleAgentEvaluator, OrchestratorExecutorEvaluator
Before:
```typescript
if (this.config.agent.type === 'single') {
const evaluator = new SingleAgentEvaluator(this.config, task, window.windowId, ...)
} else if (this.config.agent.type === 'orchestrator-executor') {
const evaluator = new OrchestratorExecutorEvaluator(this.config, task, ...)
}
```
After:
```typescript
const capture = new CaptureContext({ serverUrl, outputDir, taskId, tabId, windowId })
const taskOutputDir = await capture.init()
const context: AgentContext = {
config: this.config,
task,
windowId: window.windowId,
tabId: window.tabId,
outputDir: this.outputDir,
taskOutputDir,
capture,
}
const agent = createAgent(context)
const agentResult = await agent.execute()
```
Reference:
- Current: /apps/eval/src/runner/task-executor.ts (lines 143-186)
- Target: DESIGN_DOC.md section "6. Task Executor"
```
---
## Phase 6: Cleanup & Test (1 Subagent)
**Depends on:** Phase 5
### Subagent 6A: Cleanup Old Files + Verify
```
Tasks:
1. Delete old /apps/eval/src/types.ts (replaced by types/ folder)
2. Update all imports across the codebase:
- Change: import { EvalConfig, Task, Message } from '../types'
- Keep same (types/index.ts re-exports everything)
3. Update /apps/eval/src/utils/config-validator.ts:
- Import schemas from ../types/config instead of defining locally
- Remove duplicate schema definitions
4. Verify no TypeScript errors:
- Run: cd apps/eval && bun run typecheck
5. Test single-agent eval:
- Run: cd apps/eval && bun run eval -c configs/webvoyager-test.json
- Verify screenshots captured
- Verify messages.jsonl populated
6. Test orchestrator-executor eval:
- Run: cd apps/eval && bun run eval -c configs/orchestrator-executor-test.json
- Verify screenshots captured
- Verify messages.jsonl has delegation messages
- Verify graders pass (no "no_screenshots" error)
Report any issues found.
```
---
## Execution Summary
| Phase | Subagents | Can Parallelize? | Dependencies |
|-------|-----------|------------------|--------------|
| 1 | 4 (1A, 1B, 1C, 1D) | Yes - all parallel | None |
| 2 | 2 (2A, 2B) | Yes - both parallel | Phase 1 |
| 3 | 1 (3A) | Yes - parallel with Phase 2 | Phase 1 |
| 4 | 2 (4A, 4B) | Yes - both parallel | Phase 2 + 3 |
| 5 | 1 (5A) | No | Phase 4 |
| 6 | 1 (6A) | No | Phase 5 |
**Total: 11 subagent tasks**
**Parallel execution timeline:**
```
Time →
─────────────────────────────────────────────────────────────────
Phase 1: [1A] [1B] [1C] [1D] (4 parallel)
─────────────────
Phase 2: [2A] [2B] (2 parallel)
Phase 3: [3A] (parallel with Phase 2)
───────────
Phase 4: [4A] [4B] (2 parallel)
──────────
Phase 5: [5A]
────
Phase 6: [6A]
────
```
**Maximum parallelism: 4 subagents** (Phase 1)

View File

@@ -1,888 +0,0 @@
# Eval System - Production Grade Implementation Plan
## Overview
This plan outlines the changes needed to make the eval system production-grade with uniform agent observation across all agent patterns (single-agent, orchestrator-executor, future patterns).
**Goal:** All agent evaluators produce consistent `AgentResult` with screenshots, message traces, and verifiable action sequences.
---
## Phase 1: Type System Extensions
### 1.1 Add New Message Types
**File:** `src/types.ts`
Add delegation-specific message types for orchestrator pattern:
```typescript
// After ErrorMessage definition (~line 99)
export interface DelegationMessage extends BaseMessage {
type: 'delegation'
instruction: string
executorId: string
maxSteps?: number
}
export interface DelegationResultMessage extends BaseMessage {
type: 'delegation_result'
executorId: string
summary: string
status: 'done' | 'blocked' | 'max_steps'
stepsUsed: number
currentUrl?: string
}
// Update Message union (~line 101)
export type Message =
| UserMessage
| AssistantMessage
| ToolCallMessage
| ToolResultMessage
| ErrorMessage
| DelegationMessage // NEW
| DelegationResultMessage // NEW
// Add type guards
export function isDelegationMessage(msg: Message): msg is DelegationMessage {
return msg.type === 'delegation'
}
export function isDelegationResultMessage(msg: Message): msg is DelegationResultMessage {
return msg.type === 'delegation_result'
}
```
### 1.2 Add Orchestrator Hook Types
**File:** `src/agents/orchestrator-executor/types.ts`
```typescript
// Add after existing types
export interface OrchestratorHooks {
onDelegation?: (instruction: string, executorId: string, maxSteps?: number) => Promise<void>
onDelegationResult?: (result: ExecutorResult) => Promise<void>
onTurnStart?: (turn: number) => Promise<void>
onTurnComplete?: (turn: number) => Promise<void>
onComplete?: (answer: string) => Promise<void>
onFailed?: (reason: string) => Promise<void>
}
export interface ExecutorObservationHooks {
onBeforeToolCall?: (toolName: string, args: unknown) => Promise<string> // returns toolCallId
onAfterToolCall?: (toolName: string, toolCallId: string, result: unknown, isError: boolean) => Promise<void>
}
```
---
## Phase 2: Unified Capture Infrastructure
### 2.1 Create EvalCapture Class
**File:** `src/capture/eval-capture.ts` (NEW)
```typescript
/**
* EvalCapture - Unified capture infrastructure for all agent evaluators
*
* Combines screenshot capture, message logging, and provides hooks for
* both single-agent and orchestrator-executor patterns.
*/
import { randomUUID } from 'node:crypto'
import type {
AssistantMessage,
DelegationMessage,
DelegationResultMessage,
ErrorMessage,
Message,
ToolCallMessage,
ToolResultMessage,
UserMessage,
} from '../types'
import { MessageLogger } from './message-logger'
import { ScreenshotCapture } from './screenshot'
export interface EvalCaptureConfig {
serverUrl: string
outputDir: string
tabId: number
windowId: number
}
export class EvalCapture {
private screenshotCapture: ScreenshotCapture
private messageLogger: MessageLogger
private tabId: number
private windowId: number
private currentToolCallId: string | null = null
constructor(config: EvalCaptureConfig) {
this.screenshotCapture = new ScreenshotCapture(config.serverUrl, config.outputDir)
this.messageLogger = new MessageLogger(config.outputDir)
this.tabId = config.tabId
this.windowId = config.windowId
}
async init(): Promise<void> {
await this.screenshotCapture.init()
}
// ============================================================================
// Screenshot Capture
// ============================================================================
async captureScreenshot(): Promise<number> {
return this.screenshotCapture.capture(this.tabId, this.windowId)
}
getScreenshotCount(): number {
return this.screenshotCapture.getCount()
}
// ============================================================================
// Message Logging - Basic Types
// ============================================================================
async logUser(content: string): Promise<void> {
await this.messageLogger.logUser(content)
}
async logAssistant(content: string): Promise<void> {
await this.messageLogger.logAssistant(content)
}
async logError(content: string, errorCode?: string): Promise<void> {
await this.messageLogger.logError(content, errorCode)
}
// ============================================================================
// Tool Call Logging (for single-agent and executor)
// ============================================================================
async logToolCall(tool: string, params: Record<string, unknown>): Promise<string> {
const toolCallId = randomUUID()
this.currentToolCallId = toolCallId
await this.messageLogger.logToolCall(tool, toolCallId, params)
return toolCallId
}
async logToolResult(
toolCallId: string,
result: unknown,
isError: boolean,
screenshot?: number,
): Promise<void> {
await this.messageLogger.logToolResult(toolCallId, result, isError, screenshot)
this.currentToolCallId = null
}
getCurrentToolCallId(): string | null {
return this.currentToolCallId
}
// ============================================================================
// Delegation Logging (for orchestrator-executor)
// ============================================================================
async logDelegation(
instruction: string,
executorId: string,
maxSteps?: number,
): Promise<void> {
const message: DelegationMessage = {
type: 'delegation',
timestamp: new Date().toISOString(),
instruction,
executorId,
...(maxSteps !== undefined && { maxSteps }),
}
// Extend MessageLogger to handle this, or append directly
await this.appendMessage(message)
}
async logDelegationResult(
executorId: string,
summary: string,
status: 'done' | 'blocked' | 'max_steps',
stepsUsed: number,
currentUrl?: string,
): Promise<void> {
const message: DelegationResultMessage = {
type: 'delegation_result',
timestamp: new Date().toISOString(),
executorId,
summary,
status,
stepsUsed,
...(currentUrl && { currentUrl }),
}
await this.appendMessage(message)
}
// ============================================================================
// Helpers
// ============================================================================
private async appendMessage(message: Message): Promise<void> {
// Access internal messages array and file
// This requires either extending MessageLogger or using a shared approach
const messages = this.messageLogger.getMessages()
messages.push(message)
// Write to file - MessageLogger needs extension for this
}
getMessages(): Message[] {
return this.messageLogger.getMessages()
}
getLastAssistantMessage(): string | null {
return this.messageLogger.getLastAssistantMessage()
}
}
```
### 2.2 Extend MessageLogger for New Types
**File:** `src/capture/message-logger.ts`
Add methods for delegation messages:
```typescript
// Add after logError method
async logDelegation(
instruction: string,
executorId: string,
maxSteps?: number,
): Promise<void> {
const message: DelegationMessage = {
type: 'delegation',
timestamp: new Date().toISOString(),
instruction,
executorId,
...(maxSteps !== undefined && { maxSteps }),
}
await this.append(message)
}
async logDelegationResult(
executorId: string,
summary: string,
status: 'done' | 'blocked' | 'max_steps',
stepsUsed: number,
currentUrl?: string,
): Promise<void> {
const message: DelegationResultMessage = {
type: 'delegation_result',
timestamp: new Date().toISOString(),
executorId,
summary,
status,
stepsUsed,
...(currentUrl && { currentUrl }),
}
await this.append(message)
}
```
---
## Phase 3: Executor Hook Integration
### 3.1 Modify Executor to Accept External Hooks
**File:** `src/agents/orchestrator-executor/executor.ts`
```typescript
// Add import
import type { ExecutorObservationHooks } from './types'
export class Executor {
private agent: GeminiAgent | null = null
private stepsUsed = 0
private currentUrl = ''
private config: ExecutorConfig
private serverUrl: string
private windowId: number
private tabId: number
private observationHooks?: ExecutorObservationHooks // NEW
// ... existing constructor ...
/**
* Set external observation hooks for capture integration
*/
setObservationHooks(hooks: ExecutorObservationHooks): void {
this.observationHooks = hooks
}
async execute(
instruction: string,
maxSteps?: number,
signal?: AbortSignal,
): Promise<Omit<ExecutorResult, 'executorId'>> {
// ... existing setup ...
// Track steps via hooks - MODIFIED to include external observation
let stepsThisRun = 0
const hooks: ToolExecutionHooks = {
onBeforeToolCall: async (toolName: string, args: unknown) => {
// Call external hook if set (for logging)
if (this.observationHooks?.onBeforeToolCall) {
await this.observationHooks.onBeforeToolCall(toolName, args)
}
},
onAfterToolCall: async (toolName: string, result: ToolExecutionResult) => {
stepsThisRun++
this.stepsUsed++
// Call external hook if set (for screenshot capture and logging)
if (this.observationHooks?.onAfterToolCall) {
const toolCallId = 'current' // Will be tracked by EvalCapture
await this.observationHooks.onAfterToolCall(
toolName,
toolCallId,
result.parts,
result.isError,
)
}
},
}
this.agent.setToolHooks(hooks)
// ... rest of execute method ...
}
}
```
### 3.2 Pass Hooks Through ExecutorStore
**File:** `src/agents/orchestrator-executor/executor-store.ts`
```typescript
import type { ExecutorObservationHooks } from './types'
export class ExecutorStore {
private executors = new Map<string, Executor>()
private observationHooks?: ExecutorObservationHooks // NEW
/**
* Set observation hooks that will be applied to all executors
*/
setObservationHooks(hooks: ExecutorObservationHooks): void {
this.observationHooks = hooks
// Apply to existing executors
for (const executor of this.executors.values()) {
executor.setObservationHooks(hooks)
}
}
getOrCreate(
id: string,
config: ExecutorConfig,
serverUrl: string,
windowId: number,
tabId: number,
): Executor {
if (!this.executors.has(id)) {
const executor = new Executor(config, serverUrl, windowId, tabId)
// Apply observation hooks to new executor
if (this.observationHooks) {
executor.setObservationHooks(this.observationHooks)
}
this.executors.set(id, executor)
}
return this.executors.get(id)!
}
// ... rest unchanged ...
}
```
---
## Phase 4: Orchestrator Hook Integration
### 4.1 Add Hooks to OrchestratorAgent
**File:** `src/agents/orchestrator-executor/orchestrator-agent.ts`
```typescript
import type { ExecutorObservationHooks, OrchestratorHooks } from './types'
export class OrchestratorAgent {
private orchestratorHooks?: OrchestratorHooks // NEW
private constructor(
private client: GeminiClient,
private geminiConfig: GeminiConfig,
private state: OrchestratorState,
private executorStore: ExecutorStore,
private maxTurns: number,
) {}
/**
* Set orchestrator-level hooks for delegation tracking
*/
setHooks(hooks: OrchestratorHooks): void {
this.orchestratorHooks = hooks
}
/**
* Set executor observation hooks (passed through to ExecutorStore)
*/
setExecutorObservationHooks(hooks: ExecutorObservationHooks): void {
this.executorStore.setObservationHooks(hooks)
}
/**
* Get hooks for tool context (used by orchestrator-tools.ts)
*/
getOrchestratorHooks(): OrchestratorHooks | undefined {
return this.orchestratorHooks
}
async run(taskQuery: string): Promise<OrchestratorAgentResult> {
let currentParts: Part[] = [{ text: taskQuery }]
let turns = 0
while (
!this.state.isComplete &&
!this.state.isFailed &&
turns < this.maxTurns
) {
turns++
// Fire turn start hook
await this.orchestratorHooks?.onTurnStart?.(turns)
// ... existing turn logic ...
// Fire turn complete hook
await this.orchestratorHooks?.onTurnComplete?.(turns)
}
// Fire completion hooks
if (this.state.isComplete && this.state.finalAnswer) {
await this.orchestratorHooks?.onComplete?.(this.state.finalAnswer)
} else if (this.state.isFailed && this.state.failureReason) {
await this.orchestratorHooks?.onFailed?.(this.state.failureReason)
}
return {
success: this.state.isComplete,
answer: this.state.finalAnswer,
reason: this.state.failureReason,
delegationCount: this.state.delegationCount,
totalExecutorSteps: this.state.totalExecutorSteps,
turns,
}
}
// ... rest unchanged ...
}
```
### 4.2 Fire Hooks in Orchestrator Tools
**File:** `src/agents/orchestrator-executor/orchestrator-tools.ts`
Modify the delegate tool handler to fire hooks:
```typescript
// In createOrchestratorTools function, modify the delegate tool handler
// Inside the delegate tool's handler:
handler: async (args) => {
const { instruction, executorId, maxSteps } = args as DelegateParams
// Fire delegation hook BEFORE execution
const hooks = context.getOrchestratorHooks?.()
const actualExecutorId = executorId ?? randomUUID()
await hooks?.onDelegation?.(instruction, actualExecutorId, maxSteps)
// Get or create executor
const executor = context.executorStore.getOrCreate(
actualExecutorId,
context.executorConfig,
context.serverUrl,
context.windowId,
context.tabId,
)
// Execute
const result = await executor.execute(instruction, maxSteps)
// Update state
context.state.delegationCount++
context.state.totalExecutorSteps += result.stepsUsed
// Fire delegation result hook AFTER execution
await hooks?.onDelegationResult?.({
...result,
executorId: actualExecutorId,
})
// Return result to orchestrator
return {
executorId: actualExecutorId,
...result,
}
}
```
---
## Phase 5: Update OrchestratorExecutorEvaluator
### 5.1 Full Integration
**File:** `src/agents/orchestrator-executor/index.ts`
```typescript
import { ScreenshotCapture } from '../../capture/screenshot'
import { MessageLogger } from '../../capture/message-logger'
import { TrajectorySaver } from '../../capture/trajectory-saver'
import type { ExecutorObservationHooks, OrchestratorHooks } from './types'
export class OrchestratorExecutorEvaluator implements AgentEvaluator {
constructor(
private config: EvalConfig,
private task: Task,
private windowId: number,
private tabId: number,
private outputDir: string,
) {}
async execute(): Promise<AgentResult> {
const startTime = Date.now()
const timeoutMs = this.config.timeout_ms ?? DEFAULT_TIMEOUT_MS
const errors: TaskError[] = []
const warnings: EvalWarning[] = []
const addError = (source: TaskError['source'], message: string, details?: Record<string, unknown>) => {
errors.push({ source, message, timestamp: new Date().toISOString(), details })
}
const addWarning = (source: EvalWarning['source'], message: string) => {
warnings.push({ source, message, timestamp: new Date().toISOString() })
console.warn(`[${source}] ${message}`)
}
// Initialize trajectory saver
const saver = new TrajectorySaver(this.outputDir, this.task.query_id)
const taskOutputDir = await saver.init()
// NEW: Initialize capture infrastructure (same as single-agent)
const screenshotCapture = new ScreenshotCapture(
this.config.browseros.server_url,
taskOutputDir,
)
await screenshotCapture.init()
const messageLogger = new MessageLogger(taskOutputDir)
// Log initial user message
await messageLogger.logUser(this.task.query)
// Validate config type
if (this.config.agent.type !== 'orchestrator-executor') {
throw new Error('OrchestratorExecutorEvaluator requires orchestrator-executor config')
}
const agentConfig = this.config.agent as OrchestratorExecutorConfig
const { orchestrator: orchestratorConfig, executor: executorConfig } =
resolveAgentConfig(agentConfig)
// Create orchestrator
const orchestrator = new Orchestrator(
orchestratorConfig,
executorConfig,
this.config.browseros.server_url,
this.windowId,
this.tabId,
)
// NEW: Set up executor observation hooks (for tool call/result capture)
let currentToolCallId: string | null = null
const executorHooks: ExecutorObservationHooks = {
onBeforeToolCall: async (toolName: string, args: unknown) => {
try {
currentToolCallId = randomUUID()
await messageLogger.logToolCall(toolName, currentToolCallId, args as Record<string, unknown>)
} catch (err) {
addWarning('message_logging', `Failed to log tool call ${toolName}: ${err instanceof Error ? err.message : String(err)}`)
}
return currentToolCallId
},
onAfterToolCall: async (toolName: string, _toolCallId: string, result: unknown, isError: boolean) => {
let screenshotNum = 0
// Capture screenshot after tool execution
try {
screenshotNum = await screenshotCapture.capture(this.tabId, this.windowId)
} catch (err) {
addWarning('screenshot', `Screenshot after ${toolName} failed: ${err instanceof Error ? err.message : String(err)}`)
screenshotNum = screenshotCapture.getCount()
}
// Log tool errors
if (isError) {
addWarning('mcp_tool', `Tool ${toolName} returned error`)
}
if (!currentToolCallId) {
addWarning('message_logging', 'Tool result without matching tool call')
return
}
try {
await messageLogger.logToolResult(currentToolCallId, result, isError, screenshotNum)
} catch (err) {
addWarning('message_logging', `Failed to log tool result: ${err instanceof Error ? err.message : String(err)}`)
}
currentToolCallId = null
},
}
// NEW: Set up orchestrator hooks (for delegation tracking)
const orchestratorHooks: OrchestratorHooks = {
onDelegation: async (instruction: string, executorId: string, maxSteps?: number) => {
try {
await messageLogger.logDelegation(instruction, executorId, maxSteps)
} catch (err) {
addWarning('message_logging', `Failed to log delegation: ${err instanceof Error ? err.message : String(err)}`)
}
},
onDelegationResult: async (result) => {
try {
await messageLogger.logDelegationResult(
result.executorId,
result.summary,
result.status,
result.stepsUsed,
result.currentUrl,
)
} catch (err) {
addWarning('message_logging', `Failed to log delegation result: ${err instanceof Error ? err.message : String(err)}`)
}
},
}
// Apply hooks to orchestrator
orchestrator.setHooks(orchestratorHooks)
orchestrator.setExecutorObservationHooks(executorHooks)
// Set up timeout
const abortController = new AbortController()
const timeoutHandle = setTimeout(() => {
abortController.abort()
}, timeoutMs)
let terminationReason: 'completed' | 'max_steps' | 'error' | 'timeout' = 'completed'
let finalAnswer: string | null = null
let orchestratorResult: Awaited<ReturnType<typeof orchestrator.run>> | null = null
try {
const runPromise = orchestrator.run(this.task.query)
orchestratorResult = await Promise.race([
runPromise,
new Promise<never>((_, reject) => {
abortController.signal.addEventListener('abort', () => {
reject(new Error('Timeout'))
})
}),
])
if (orchestratorResult.success) {
finalAnswer = orchestratorResult.answer
terminationReason = 'completed'
// Log final assistant message
if (finalAnswer) {
await messageLogger.logAssistant(finalAnswer)
}
} else {
terminationReason = 'error'
addError('agent_execution', orchestratorResult.reason ?? 'Unknown failure')
await messageLogger.logError(orchestratorResult.reason ?? 'Unknown failure')
}
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err))
if (error.message === 'Timeout' || abortController.signal.aborted) {
terminationReason = 'timeout'
addError('agent_execution', `Task timed out after ${timeoutMs / 1000}s`)
} else {
terminationReason = 'error'
addError('agent_execution', error.message, { stack: error.stack })
}
await messageLogger.logError(error.message)
} finally {
clearTimeout(timeoutHandle)
orchestrator.getExecutorStore().clear()
}
const endTime = Date.now()
// Create metadata
const metadata: TaskMetadata = {
query_id: this.task.query_id,
dataset: this.task.dataset,
query: this.task.query,
started_at: new Date(startTime).toISOString(),
completed_at: new Date(endTime).toISOString(),
total_duration_ms: endTime - startTime,
total_steps: screenshotCapture.getCount(), // Now accurate
termination_reason: terminationReason,
final_answer: finalAnswer,
errors,
warnings,
agent_config: {
type: 'orchestrator-executor',
model: `${orchestratorConfig.model} / ${executorConfig.model}`,
},
grader_results: {},
}
await saver.saveMetadata(metadata)
return {
metadata,
messages: messageLogger.getMessages(), // NOW POPULATED
finalAnswer,
}
}
}
```
---
## Phase 6: Orchestrator Class Updates
### 6.1 Add Hook Passthrough Methods
**File:** `src/agents/orchestrator-executor/orchestrator.ts`
```typescript
import type { ExecutorObservationHooks, OrchestratorHooks } from './types'
export class Orchestrator {
private agent: OrchestratorAgent | null = null
private executorStore: ExecutorStore
private pendingOrchestratorHooks?: OrchestratorHooks
private pendingExecutorHooks?: ExecutorObservationHooks
constructor(
private orchestratorConfig: OrchestratorConfig,
private executorConfig: ExecutorConfig,
private serverUrl: string,
private windowId: number,
private tabId: number,
) {
this.executorStore = new ExecutorStore()
}
/**
* Set orchestrator-level hooks (must be called before run())
*/
setHooks(hooks: OrchestratorHooks): void {
this.pendingOrchestratorHooks = hooks
if (this.agent) {
this.agent.setHooks(hooks)
}
}
/**
* Set executor observation hooks (must be called before run())
*/
setExecutorObservationHooks(hooks: ExecutorObservationHooks): void {
this.pendingExecutorHooks = hooks
this.executorStore.setObservationHooks(hooks)
if (this.agent) {
this.agent.setExecutorObservationHooks(hooks)
}
}
async run(taskQuery: string): Promise<OrchestratorAgentResult> {
this.agent = await OrchestratorAgent.create(
this.orchestratorConfig,
this.executorConfig,
this.serverUrl,
this.windowId,
this.tabId,
)
// Apply pending hooks
if (this.pendingOrchestratorHooks) {
this.agent.setHooks(this.pendingOrchestratorHooks)
}
if (this.pendingExecutorHooks) {
this.agent.setExecutorObservationHooks(this.pendingExecutorHooks)
}
const result = await this.agent.run(taskQuery)
this.executorStore = this.agent.getExecutorStore()
return result
}
getExecutorStore(): ExecutorStore {
return this.agent?.getExecutorStore() ?? this.executorStore
}
}
```
---
## Implementation Order
1. **Phase 1** - Type extensions (types.ts) - 30 min
2. **Phase 2** - MessageLogger extensions - 30 min
3. **Phase 3** - Executor hook integration - 1 hour
4. **Phase 4** - OrchestratorAgent hooks - 1 hour
5. **Phase 5** - OrchestratorExecutorEvaluator update - 1.5 hours
6. **Phase 6** - Orchestrator passthrough - 30 min
7. **Testing** - End-to-end verification - 1 hour
**Total estimated time:** ~6 hours
---
## Testing Checklist
- [ ] Single-agent eval still works (regression test)
- [ ] Orchestrator-executor produces screenshots in output folder
- [ ] Orchestrator-executor produces messages.jsonl with:
- [ ] user message
- [ ] delegation messages
- [ ] tool_call messages (from executor)
- [ ] tool_result messages with screenshot numbers
- [ ] delegation_result messages
- [ ] assistant message (final answer)
- [ ] Graders pass with orchestrator-executor (no "no_screenshots" error)
- [ ] metadata.json has accurate `total_steps` count
- [ ] Error/warning capture works for both patterns
---
## Future Considerations
1. **New Agent Patterns:** Any new agent type just needs to:
- Accept hooks in constructor or via setter
- Fire hooks at appropriate points
- Use shared capture infrastructure
2. **Grader Updates:** May need to update graders to understand delegation messages
3. **Parallel Executors:** If orchestrator delegates to multiple executors in parallel, need to handle concurrent screenshot capture
4. **Memory/Performance:** Screenshot capture creates MCP connection per capture - consider connection pooling for high-volume evals

View File

@@ -2,155 +2,158 @@
[![License: AGPL v3](https://img.shields.io/badge/License-AGPL%20v3-blue.svg)](../../../../LICENSE)
Evaluation framework for benchmarking BrowserOS browser automation agents. Runs tasks from standard datasets ([WebVoyager](https://arxiv.org/abs/2401.13919), [Mind2Web](https://arxiv.org/abs/2306.06070)), captures trajectories with screenshots, and grades results automatically.
Evaluation framework for BrowserOS browser automation agents. Runs tasks from standard datasets ([WebVoyager](https://arxiv.org/abs/2401.13919), [Mind2Web](https://arxiv.org/abs/2306.06070), AGI SDK / REAL Bench, WebArena-Infinity, WebBench), captures trajectories with screenshots, and grades results automatically.
## Prerequisites
- **BrowserOS binary** installed at `/Applications/BrowserOS.app` (macOS)
- **BrowserOS binary** at `/Applications/BrowserOS.app` (macOS) or `BROWSEROS_BINARY` pointing at it
- **Bun** runtime
- **API keys** for your chosen LLM provider and grader model
- **API keys** for your LLM provider (and `CLAUDE_CODE_OAUTH_TOKEN` if you use `performance_grader`)
## Quick Start
### 1. Set up environment
```bash
cd apps/eval
```
Edit `.env.development` and add your API keys:
```bash
# Pick ONE provider for the orchestrator (whichever you have access to)
OPENAI_API_KEY=sk-xxxxx
ANTHROPIC_API_KEY=sk-ant-xxxxx
FIREWORKS_API_KEY=fw_xxxxx
GOOGLE_API_KEY=AIza-xxxxx
# For grading results (OpenRouter recommended — gives access to many models)
OPENROUTER_API_KEY=sk-or-v1-xxxxx
```
### 2. Launch the dashboard
```bash
cp .env.example .env.development
# Edit .env.development with your keys, then:
bun run eval
```
Opens the **Eval Dashboard** at `http://localhost:9900` in config mode.
Opens the eval dashboard at `http://localhost:9900` in config mode. From there: load a preset, edit settings, click **Run**.
### 3. Configure and run
From the dashboard:
1. **Load a preset** — select from the dropdown or click **Load File** to import a config JSON
2. **Edit settings** — change agent type, provider, model, API keys, dataset, workers, timeouts
3. **Save Config** — export your configuration for reuse
4. **Click Run** — starts the evaluation with live progress
### Alternative: Run from CLI
### CLI mode
```bash
bun run eval -c configs/orchestrator-executor-clado-test.json
bun run eval -c configs/legacy/browseros-agent-weekly.json
bun run eval suite --config configs/legacy/browseros-agent-weekly.json --publish r2
```
Runs immediately. Dashboard still available at `http://localhost:9900` for live progress.
## Agent Types
The `suite` command is the workflow-compatible full loop: execute tasks, run graders, write artifacts, and optionally publish to R2. The old `-c` form remains supported during migration.
### Orchestrator-Executor with Clado
```bash
bun run eval run --config configs/legacy/browseros-agent-weekly.json
bun run eval suite --suite configs/suites/agisdk-daily-10.json --variant kimi-fireworks --publish r2
bun run eval grade --run results/browseros-agent-weekly/2026-04-29-1430
bun run eval publish --run results/browseros-agent-weekly/2026-04-29-1430 --target r2
```
The recommended architecture for visual model evals. Two tiers:
Config files live in two groups:
- **Orchestrator** — An LLM that plans and issues high-level instructions
- **Executor** — The **Clado Action** visual model that takes screenshots and predicts click/type/scroll coordinates
```txt
configs/legacy/ # Complete EvalConfig files used by older workflows and the dashboard
configs/suites/ # Suite definitions; model/provider comes from CLI flags or env
```
The orchestrator works with **any LLM provider**. Pick whichever you have access to:
Suite mode takes model settings from CLI flags first, then env:
#### OpenAI orchestrator
```bash
EVAL_VARIANT=kimi-fireworks \
EVAL_AGENT_PROVIDER=openai-compatible \
EVAL_AGENT_MODEL=accounts/fireworks/models/kimi-k2p5 \
EVAL_AGENT_API_KEY=$FIREWORKS_API_KEY \
EVAL_AGENT_BASE_URL=https://api.fireworks.ai/inference/v1 \
bun run eval suite --suite configs/suites/agisdk-daily-10.json --publish r2
```
### Suites and variants
A **suite** is what we run: the task dataset, graders, worker count, timeout, and browser settings. For example, `agisdk-daily-10` means "run these 10 AGI SDK tasks and grade them with `agisdk_state_diff`."
A **variant** is the model setup we are testing on that suite. `EVAL_VARIANT` is just the human-readable name for that setup. The actual model connection still comes from `EVAL_AGENT_PROVIDER`, `EVAL_AGENT_MODEL`, `EVAL_AGENT_API_KEY`, and `EVAL_AGENT_BASE_URL`.
This lets us run the same suite against multiple model setups without copying the benchmark config:
```txt
agisdk-daily-10 + kimi-fireworks
agisdk-daily-10 + claude-sonnet
agisdk-daily-10 + clado-action-000159
```
For `orchestrator-executor` suites, there can also be an executor model/backend. The `EVAL_AGENT_*` vars describe the main agent or orchestrator. The optional `EVAL_EXECUTOR_*` or `CLADO_ACTION_*` vars describe the delegated executor.
## Agent types
| Type | Description |
|------|-------------|
| `single` | Single LLM agent driven by the BrowserOS tool loop (CDP) |
| `orchestrator-executor` | High-level orchestrator + per-step executor (LLM or Clado visual model) |
### Single agent
```json
{
"agent": {
"type": "single",
"provider": "openai-compatible",
"model": "moonshotai/kimi-k2.5",
"apiKey": "OPENROUTER_API_KEY",
"baseUrl": "https://openrouter.ai/api/v1",
"supportsImages": true
}
}
```
### Orchestrator-Executor
The orchestrator works with any LLM provider. The executor can be another LLM, or the **Clado action** visual model that takes screenshots and predicts click/type/scroll coordinates.
```json
{
"agent": {
"type": "orchestrator-executor",
"orchestrator": {
"provider": "openai",
"model": "gpt-4o",
"apiKey": "OPENAI_API_KEY"
"provider": "openai-compatible",
"model": "accounts/fireworks/models/kimi-k2p5",
"apiKey": "FIREWORKS_API_KEY",
"baseUrl": "https://api.fireworks.ai/inference/v1"
},
"executor": {
"provider": "clado-action",
"model": "qwen3-vl-30b-a3b-instruct",
"model": "Qwen3.5-35B-A3B-action-000159-merged",
"apiKey": "",
"baseUrl": "https://clado-ai--clado-browseros-action-actionmodel-generate.modal.run"
"baseUrl": "https://clado-ai--clado-browseros-action-000159-merged-actionmod-f4a6ef.modal.run"
}
},
"dataset": "../data/webvoyager_e2e_test.jsonl",
"output_dir": "../results/oe-clado-openai",
"num_workers": 3,
"browseros": {
"server_url": "http://127.0.0.1:9110",
"base_cdp_port": 9010,
"base_server_port": 9110,
"base_extension_port": 9310,
"headless": true
},
"grader_api_key_env": "OPENROUTER_API_KEY",
"grader_base_url": "https://openrouter.ai/api/v1",
"grader_model": "openai/gpt-4.1",
"timeout_ms": 1200000
}
}
```
#### Anthropic orchestrator
## Graders
| Name | Description |
|------|-------------|
| `performance_grader` | Multi-axis grader running on Claude Agent SDK (uses its own credentials via `CLAUDE_CODE_OAUTH_TOKEN`) |
| `agisdk_state_diff` | AGI SDK / REAL Bench environment state-diff grader (deterministic) |
| `infinity_state` | WebArena-Infinity verifier-script grader (deterministic) |
Set `graders` in your config to override the per-task `graders` field from the dataset:
```json
"orchestrator": {
"provider": "anthropic",
"model": "claude-sonnet-4-20250514",
"apiKey": "ANTHROPIC_API_KEY"
}
"graders": ["performance_grader"]
```
#### Google orchestrator
```json
"orchestrator": {
"provider": "google",
"model": "gemini-2.0-flash",
"apiKey": "GOOGLE_API_KEY"
}
```
#### Fireworks orchestrator (OpenAI-compatible)
```json
"orchestrator": {
"provider": "openai-compatible",
"model": "accounts/fireworks/models/kimi-k2p5",
"apiKey": "FIREWORKS_API_KEY",
"baseUrl": "https://api.fireworks.ai/inference/v1"
}
```
The executor config stays the same across all orchestrator providers — it always uses the Clado action model.
### Other Agent Types
| Type | Description | Example config |
|------|-------------|----------------|
| `single` | Single LLM agent via Gemini CLI + MCP | `webvoyager-test.json` |
| `tool-loop` | AI SDK tool loop, connects via CDP | `tool-loop-test.json` |
| `gemini-computer-use` | Google native computer use API | `gemini-computer-use.json` |
| `yutori-navigator` | Yutori N1 visual model | `yutori-navigator.json` |
## Configuration Reference
## Configuration reference
### API keys
The `apiKey` field supports two formats:
- **Env var name**: `"OPENAI_API_KEY"` — resolved from `.env.development` at runtime
- **Direct value**: `"sk-xxxxx"` — used as-is (not recommended, prefer env vars)
- **Direct value**: `"sk-xxxxx"` — used as-is (not recommended)
### Environment variables
| Variable | Used for |
|----------|----------|
| `EVAL_AGENT_PROVIDER`, `EVAL_AGENT_MODEL`, `EVAL_AGENT_API_KEY`, `EVAL_AGENT_BASE_URL`, `EVAL_AGENT_SUPPORTS_IMAGES` | Suite variant model selection |
| `FIREWORKS_API_KEY`, `OPENROUTER_API_KEY`, `ANTHROPIC_API_KEY`, provider-specific keys | Config-file or provider-backed model calls |
| `EVAL_EXECUTOR_MODEL`, `EVAL_EXECUTOR_API_KEY`, `EVAL_EXECUTOR_BASE_URL` | Suite-mode orchestrator executor override |
| `CLADO_ACTION_MODEL`, `CLADO_ACTION_API_KEY`, `CLADO_ACTION_BASE_URL` | Clado executor defaults |
| `BROWSEROS_BINARY` | BrowserOS binary path in CI/local smoke runs |
| `BROWSEROS_SERVER_URL` | Optional grader MCP URL override |
| `WEBARENA_INFINITY_DIR` | Local WebArena-Infinity checkout for Infinity tasks |
| `NOPECHA_API_KEY` | CAPTCHA solver extension |
| `EVAL_R2_ACCOUNT_ID`, `EVAL_R2_ACCESS_KEY_ID`, `EVAL_R2_SECRET_ACCESS_KEY`, `EVAL_R2_BUCKET`, `EVAL_R2_CDN_BASE_URL` | R2 upload and viewer URL |
### Supported providers
@@ -160,12 +163,28 @@ The `apiKey` field supports two formats:
| Anthropic | `anthropic` | No |
| Google | `google` | No |
| Azure OpenAI | `azure` | Yes |
| AWS Bedrock | `bedrock` | No (uses `region`, `accessKeyId`, `secretAccessKey`) |
| AWS Bedrock | `bedrock` | No |
| OpenRouter | `openrouter` | No |
| Fireworks, Together, etc. | `openai-compatible` | Yes |
| Ollama | `ollama` | No |
| Clado Action (executor only) | `clado-action` | Yes |
### R2 publishing
`suite --config ... --publish r2` and `publish --target r2` upload the run artifacts plus `viewer.html` to the viewer-compatible R2 layout:
```bash
export EVAL_R2_ACCOUNT_ID=...
export EVAL_R2_ACCESS_KEY_ID=...
export EVAL_R2_SECRET_ACCESS_KEY=...
export EVAL_R2_BUCKET=browseros-eval
export EVAL_R2_CDN_BASE_URL=https://eval.browseros.com
```
`EVAL_R2_CDN_BASE_URL` must be a public R2 custom domain, `r2.dev` URL, or Worker URL. Do not set it to the private `*.r2.cloudflarestorage.com` S3 API endpoint.
Published runs are available at `EVAL_R2_CDN_BASE_URL/viewer.html?run=<run-id>`.
### BrowserOS infrastructure
```json
@@ -179,34 +198,29 @@ The `apiKey` field supports two formats:
}
```
Each worker gets its own Chrome instance. Worker N uses `base_port + N` for CDP and server ports. `base_extension_port` is still reserved as a legacy BrowserOS launch argument for compatibility with Chromium builds that still pass it.
Each worker gets its own Chrome instance. Worker N uses `base_port + N` for CDP and server ports.
### Execution settings
| Field | Description | Default |
|-------|-------------|---------|
| `num_workers` | Parallel workers (each gets its own Chrome) | `1` |
| `timeout_ms` | Per-task timeout in ms | `900000` (15 min) |
| `timeout_ms` | Per-task timeout in ms | `1800000` (30 min) |
| `restart_server_per_task` | Restart Chrome between tasks (cleaner state, slower) | `false` |
### Grading
Results are auto-graded after each task. The grader uses an LLM judge.
| Field | Description |
|-------|-------------|
| `grader_model` | Model for grading (e.g., `openai/gpt-4.1`) |
| `grader_api_key_env` | Env var name for grader API key |
| `grader_base_url` | API endpoint (e.g., `https://openrouter.ai/api/v1`) |
## Datasets
| File | Tasks | Description |
|------|-------|-------------|
| `webvoyager_e2e_test.jsonl` | 10 | WebVoyager test subset (quick smoke test) |
| `agisdk-daily-10.jsonl` | 10 | Daily AGI SDK / REAL Bench subset |
| `webvoyager.jsonl` | 643 | Full WebVoyager benchmark |
| `mind2web_e2e_test.jsonl` | 10 | Mind2Web test subset |
| `mind2web.jsonl` | 300 | Full Mind2Web benchmark |
| `mind2web.jsonl` | 300 | Online-Mind2Web |
| `webbench-{0,1,2}of4-50.jsonl` | 50 each | WebBench shards (50-task subsets) |
| `agisdk-real-smoke.jsonl` | 1 | AGI SDK / REAL Bench smoke task |
| `agisdk-real.jsonl` | 36 | AGI SDK / REAL Bench (action-only tasks) |
| `webarena-infinity-hard-50.jsonl` | 50 | WebArena-Infinity hard set |
| `browsecomp-medium-hard-50.jsonl` | 50 | BrowseComp medium-hard |
| `browsecomp-very-hard-50.jsonl` | 50 | BrowseComp very-hard |
Task format (JSONL, one per line):
@@ -215,7 +229,7 @@ Task format (JSONL, one per line):
"query_id": "Amazon--0",
"dataset": "webvoyager",
"query": "Search an Xbox Wireless controller with green color and rated above 4 stars.",
"graders": ["webvoyager_grader", "fara_combined"],
"graders": ["performance_grader"],
"start_url": "https://www.amazon.com/",
"metadata": { "original_task_id": "Amazon--0", "website": "Amazon" }
}
@@ -227,24 +241,58 @@ Results are saved to `output_dir`:
```
results/
oe-clado-openai/
Amazon--0/
metadata.json # Task result, timing, grader scores
messages.jsonl # Full message log
screenshots/
001.png # Step-by-step screenshots
002.png
summary.json # Aggregate pass rates
browseros-agent-weekly/
2026-04-29-1430/
Amazon--0/
attempt.json # Stable attempt summary for viewer/reporting
metadata.json # Task result, timing, grader scores
grades.json # Compact grader results
messages.jsonl # Full message log
grader-artifacts/ # Grader-specific inputs/outputs/stderr
screenshots/
001.png # Step-by-step screenshots
002.png
summary.json # Aggregate pass rates
```
R2 publishing preserves the task files under `runs/<run-id>/...`, writes `runs/<run-id>/manifest.json`, and uploads `viewer.html` at the bucket root. The viewer URL is `EVAL_R2_CDN_BASE_URL/viewer.html?run=<run-id>`.
### R2 viewer manifest
`runs/<run-id>/manifest.json` is the source of truth for the public viewer. New manifests include `schemaVersion: 2` and each task includes explicit artifact paths:
```json
{
"schemaVersion": 2,
"runId": "agisdk-real-smoke-2026-04-30-0000",
"tasks": [
{
"queryId": "agisdk-dashdish-10",
"paths": {
"metadata": "tasks/agisdk-dashdish-10/metadata.json",
"messages": "tasks/agisdk-dashdish-10/messages.jsonl",
"grades": "tasks/agisdk-dashdish-10/grades.json",
"trace": "tasks/agisdk-dashdish-10/trace.jsonl",
"screenshots": "tasks/agisdk-dashdish-10/screenshots",
"graderArtifacts": "tasks/agisdk-dashdish-10/grader-artifacts"
}
}
]
}
```
The static viewer uses `task.paths` when present. Older uploaded runs without `schemaVersion` or `task.paths` still work through the legacy inferred layout: `runs/<run-id>/<task-id>/metadata.json`, `messages.jsonl`, and `screenshots/<n>.png`.
Manifest paths are stable artifact locations, not a guarantee that every optional artifact exists for every task. For example, `attempt.json`, `trace.jsonl`, or grader artifact directories may be absent when that artifact was not produced by the run.
## Troubleshooting
**BrowserOS not found**: Expects `/Applications/BrowserOS.app/Contents/MacOS/BrowserOS`. Make sure it's installed.
**BrowserOS not found**: Expects `/Applications/BrowserOS.app/Contents/MacOS/BrowserOS`. Set `BROWSEROS_BINARY` to override.
**Port conflicts**: Each worker uses `base_port + workerIndex`. 3 workers on base 9110 → ports 9110, 9111, 9112. Stop other BrowserOS instances first.
**API key not resolving**: If your config has `"apiKey": "OPENAI_API_KEY"`, ensure the env var is set in `.env.development`.
**Tasks timing out**: Increase `timeout_ms`. Default is 15 minutes; complex tasks may need 20+ minutes.
**Tasks timing out**: Increase `timeout_ms`. Default is 30 minutes.
**Headless vs headed**: Set `"headless": false` to watch Chrome in real-time. Useful for debugging.
**Headless vs headed**: Set `"headless": false` to watch Chrome in real time.

View File

@@ -1,18 +0,0 @@
{
"agent": {
"type": "single",
"provider": "openrouter",
"model": "openai/gpt-4o",
"apiKey": "OPENROUTER_API_KEY"
},
"dataset": "data/webvoyager_e2e_test.jsonl",
"output_dir": "results",
"num_workers": 5,
"browseros": {
"server_url": "http://127.0.0.1:9110"
},
"grader_api_key_env": "OPENROUTER_API_KEY",
"grader_base_url": "https://openrouter.ai/api/v1",
"grader_model": "openai/gpt-4.1",
"timeout_ms": 300000
}

View File

@@ -0,0 +1,26 @@
{
"agent": {
"type": "single",
"provider": "openai-compatible",
"model": "moonshotai/kimi-k2.5",
"apiKey": "OPENROUTER_API_KEY",
"baseUrl": "https://openrouter.ai/api/v1",
"supportsImages": true
},
"dataset": "../../data/agisdk-real-smoke.jsonl",
"num_workers": 1,
"restart_server_per_task": true,
"browseros": {
"server_url": "http://127.0.0.1:9110",
"base_cdp_port": 9010,
"base_server_port": 9110,
"base_extension_port": 9310,
"load_extensions": false,
"headless": false
},
"captcha": {
"api_key_env": "NOPECHA_API_KEY"
},
"graders": ["agisdk_state_diff"],
"timeout_ms": 1800000
}

View File

@@ -7,8 +7,8 @@
"baseUrl": "https://api.fireworks.ai/inference/v1",
"supportsImages": true
},
"dataset": "../data/webbench-2of4-50.jsonl",
"num_workers": 10,
"dataset": "../../data/agisdk-real.jsonl",
"num_workers": 4,
"restart_server_per_task": true,
"browseros": {
"server_url": "http://127.0.0.1:9110",
@@ -21,9 +21,6 @@
"captcha": {
"api_key_env": "NOPECHA_API_KEY"
},
"graders": ["performance_grader"],
"grader_api_key_env": "OPENROUTER_API_KEY",
"grader_base_url": "https://openrouter.ai/api/v1",
"grader_model": "openai/gpt-4.1",
"graders": ["agisdk_state_diff"],
"timeout_ms": 1800000
}

View File

@@ -0,0 +1,26 @@
{
"agent": {
"type": "single",
"provider": "openai-compatible",
"model": "moonshotai/kimi-k2.5",
"apiKey": "OPENROUTER_API_KEY",
"baseUrl": "https://openrouter.ai/api/v1",
"supportsImages": true
},
"dataset": "../../data/webbench-2of4-50.jsonl",
"num_workers": 10,
"restart_server_per_task": true,
"browseros": {
"server_url": "http://127.0.0.1:9110",
"base_cdp_port": 9010,
"base_server_port": 9110,
"base_extension_port": 9310,
"load_extensions": false,
"headless": false
},
"captcha": {
"api_key_env": "NOPECHA_API_KEY"
},
"graders": ["performance_grader"],
"timeout_ms": 1800000
}

View File

@@ -14,7 +14,7 @@
"baseUrl": "https://api.fireworks.ai/inference/v1"
}
},
"dataset": "../data/webbench-2of4-50.jsonl",
"dataset": "../../data/webbench-2of4-50.jsonl",
"num_workers": 10,
"restart_server_per_task": true,
"browseros": {
@@ -29,8 +29,5 @@
"api_key_env": "NOPECHA_API_KEY"
},
"graders": ["performance_grader"],
"grader_api_key_env": "OPENROUTER_API_KEY",
"grader_base_url": "https://openrouter.ai/api/v1",
"grader_model": "openai/gpt-4.1",
"timeout_ms": 1800000
}

View File

@@ -9,12 +9,12 @@
},
"executor": {
"provider": "clado-action",
"model": "qwen3-vl-30b-a3b-instruct",
"model": "Qwen3.5-35B-A3B-action-000159-merged",
"apiKey": "",
"baseUrl": "https://clado-ai--clado-browseros-action-actionmodel-generate.modal.run"
"baseUrl": "https://clado-ai--clado-browseros-action-000159-merged-actionmod-f4a6ef.modal.run"
}
},
"dataset": "../data/webbench-2of4-50.jsonl",
"dataset": "../../data/agisdk-real.jsonl",
"num_workers": 10,
"restart_server_per_task": true,
"browseros": {
@@ -23,14 +23,11 @@
"base_server_port": 9110,
"base_extension_port": 9310,
"load_extensions": false,
"headless": false
"headless": true
},
"captcha": {
"api_key_env": "NOPECHA_API_KEY"
},
"graders": ["performance_grader"],
"grader_api_key_env": "OPENROUTER_API_KEY",
"grader_base_url": "https://openrouter.ai/api/v1",
"grader_model": "openai/gpt-4.1",
"graders": ["agisdk_state_diff"],
"timeout_ms": 1800000
}

Some files were not shown because too many files have changed in this diff Show More