19 Commits

Author	SHA1	Message	Date
shivammittal274	7a2a8e09bc	feat(agent): add Hermes as 4th ACPX adapter (in-VM container, BrowserOS-managed providers) (#956) ... * feat(agent): add Hermes as a 4th ACPX adapter (Phase A) Adds Hermes Agent (NousResearch/hermes-agent) as a host-process ACPX adapter, mirroring the Claude Code pattern. - agent-types.ts: extend AgentAdapter union with 'hermes' - agent-catalog.ts: add Hermes catalog entry - lib/agents/hermes/prepare.ts (new): minimal prepare using prepareBrowserosManagedContext - acpx-agent-adapter.ts: register the adapter - acpx-runtime.ts: add 'hermes' branch returning 'hermes acp' (host) - AdapterIcon.tsx: add Hermes icon - db schema + supporting frontend types/literals updated for the new adapter Phase A scope: host-process only. Phase A.5 swaps to nerdctl exec into a Hermes container. OpenClaw is untouched. Verified by all 6 POC spikes (plans/features/claude-browseros-hermes-poc/findings.md). * fix(agent): address Hermes adapter review issues - NewAgentDialog: add 'hermes' to onValueChange guard so the dropdown option actually wires through onRuntimeChange/onHarnessAdapterChange (was a no-op before — selecting Hermes silently kept previous value) - tests/acpx-runtime: add coverage for the new 'hermes' registry branch - tests/acpx-agent-adapter: fold hermes prepare test into existing file, matching the pattern used for claude/codex/openclaw - Delete tests/lib/agents/hermes-prepare.test.ts (now redundant) - Reconcile install-mechanism comment between acpx-runtime.ts and agent-catalog.ts * fix(agent): make Hermes adapter actually work end-to-end Two surgical fixes uncovered while running the Phase A smoke test through the BrowserOS chat HTTP API: 1. lib/agents/hermes/prepare.ts — seed per-agent HERMES_HOME from the user's global ~/.hermes/ on first use. ensureAgentHome only writes SOUL.md and MEMORY.md; without seeding config.yaml, .env, and auth.json, hermes acp comes up unconfigured and either hangs or errors with "No LLM provider configured." Copy is idempotent (skip if dest exists) so subsequent prepare calls don't clobber per-agent edits. 2. lib/agents/acpx-runtime.ts — wrap the hermes spawn in `bash -c "exec hermes acp | tee /dev/null"` to bridge Bun's socketpair-based child stdio with Python's asyncio.connect_write_pipe (which only drains correctly to a real pipe(2)). Without it, hermes' stdout never reaches the harness — verified by inspecting hermes process FDs: Bun gives the child unix sockets, asyncio queues writes that never become readable on Bun's end. With tee in the middle, hermes writes to a real pipe and tee bridges the bytes through the socket. Verified 2026-05-06 against hermes-agent 0.12.0 on macOS arm64 + Bun 1.3.6. Smoke-test result with both fixes: - ACP session created end-to-end - BrowserOS MCP wired (96 browser tools registered with hermes) - Reasoning + text streamed back through /agents/:id/sidepanel/chat - Final stream: text-delta "PONG", finishReason "stop" Updates the existing acpx-runtime test to assert the new spawn shape (bash -c, tee /dev/null bridge) so the workaround can't silently regress. * feat(agent): run Hermes adapter in Lima container (Phase A.5) Move Hermes ACPX adapter from host-process spawn to running inside docker.io/nousresearch/hermes-agent:v2026.4.30 in the existing BrowserOS Lima VM, mirroring the OpenClaw container pattern. Container lifecycle (api/services/hermes/hermes-container.ts): - prewarm: ensure VM ready, pull image (or skip if already in containerd), start an idle container with /bin/sh -c "exec sleep infinity" so the harness can nerdctl exec into it per turn - Tini bypassed — tini 0.19.0 in upstream image getopt-parses any -x token even after PROGRAM, breaking /bin/sh -c - --add-host host.containers.internal:<vm-gateway> so hermes inside the container can reach the BrowserOS HTTP MCP endpoint - Bind-mount <browserosDir>/vm/hermes/harness onto /data/agents/harness so per-agent HERMES_HOME dirs are visible to the container Spawn (acpx-runtime.ts): - HermesGatewayAccessor interface (mirrors OpenclawGatewayAccessor) - resolveHermesAcpCommand builds: env LIMA_HOME=... limactl shell --workdir / browseros-vm -- nerdctl exec -i -e PYTHONUNBUFFERED=1 -e HERMES_HOME=... <container> /opt/hermes/.venv/bin/hermes acp - Absolute path /opt/hermes/.venv/bin/hermes (not bare "hermes") since upstream image's PATH is set by its entrypoint script which we override to keep the container idle - Falls back to host-process spawn when no HermesGatewayAccessor wired (test path / dev fallback) - Drops the host-mode bash+tee workaround — limactl/SSH/nerdctl pipe chain is sufficient for asyncio's pipe writer MCP wiring: - New PreparedAcpxAgentContext.browserosMcpHost field threads through prepare → getRuntime → createBrowserosMcpServers - Hermes prepare sets browserosMcpHost='host.containers.internal' so the URL injected into newSession.mcpServers resolves from inside the container; other adapters keep '127.0.0.1' default Per-agent home (lib/agents/hermes/prepare.ts): - HERMES_HOME points at /data/agents/harness/<agentId>/home (in-container) - Host-side seedHermesHomeFromGlobal still copies ~/.hermes/{config.yaml, .env, auth.json} into the per-agent home; the volume mount makes them visible inside the container - New api/services/hermes/hermes-paths.ts holds host/container path helpers End-to-end smoke tests against the dev server (clean Lima state): - Plain text: PONG round-trip via /sidepanel/chat ✓ - Multi-turn context: RUBY-7421 stored + recalled ✓ - Multi-agent isolation: agent 2 doesn't see agent 1's secret ✓ - MCP tool execution: mcp_browseros_browseros_info fires ✓ - Image attachment via /chat: model identifies "Red" from a 128x128 PNG ✓ - Concurrent turns + 409 attachUrl: full attach streams the in-flight Pacific Ocean essay turn cleanly ✓ - Cancel midstream + recovery turn: ALIVE response ✓ - Persistence across server restart: agents survive ✓ Companion knowledge doc: plans/features/claude-browseros-hermes-acp-knowledge.md * feat(agent): per-agent provider/key for Hermes adapter Lets users create multiple Hermes agents each with its own provider, model, and API key. NewAgentDialog now shows provider/model/key fields inline when 'Hermes' is selected. On submit, the harness writes the per-agent <browserosDir>/vm/hermes/harness/<agentId>/home/{config.yaml, .env} directly so the agent has the right config from turn 1 — no dependency on the user having run `hermes setup` outside BrowserOS. The existing seedHermesHomeFromGlobal flow remains as a fallback for agents created without provider fields (e.g. via direct API or with an existing ~/.hermes/ install). Backend: - shared/constants/hermes.ts: HERMES_SUPPORTED_PROVIDERS registry (openrouter, anthropic, openai, custom — bedrock follow-up) - api/services/hermes/hermes-paths.ts: writeHermesPerAgentProvider - agent-harness-service: writes per-agent config.yaml + .env in createAgent when adapter=hermes and apiKey present - routes/agents.ts: relax modelId catalog validation for adapter=hermes (catalog has empty models[] by design; per-agent modelId is free-form) - tests/agent-harness-service: cover write + skip paths Frontend: - HermesProviderFields.tsx (new): provider dropdown, model field, API key + optional baseUrl when provider=custom - NewAgentDialog: render the new fields when adapter=hermes - agents-page-actions: thread fields through createHarnessAgent - AgentsPage / agent-harness-types: minor pass-through edits Smoke-tested end-to-end against the dev server (clean Hermes per-agent home, no ~/.hermes/ seed): create agent with apiKey + modelId, files written at the per-agent path with mode 0600, first chat returns the expected response, all without touching ~/.hermes/. * feat(agent): source Hermes provider config from BrowserOS LLM providers Replace the Hermes-specific provider/model/API-key form in New Agent with a chooser that pulls from the same global LLM providers OpenClaw uses (Settings → BrowserOS AI). Backend rejects creation with a 400 when the selected provider is missing required fields (apiKey, modelId, plus baseUrl for openai-compatible) or is not in the Hermes-supported set; the ~/.hermes/ fallback is removed so Hermes agents always carry their own per-agent config.	2026-05-07 21:54:36 +05:30
Dani Akash	db5e55a174	feat(agent-files): expose openclaw produced files inline + outputs rail (#946) ... * feat(server): foundation for OpenClaw agent file-output attribution Phase 1 of TKT-762 — surface files OpenClaw agents produce as artifacts inline in chat + a per-agent Outputs rail. This commit lays the storage + I/O foundation only; turn-lifecycle wiring, HTTP routes, and UI follow in subsequent phases. - New `produced_files` Drizzle table (FK→agent_definitions with cascade, unique on (agent, path) so re-modifications upsert). Migration 0002_chemical_whirlwind.sql. Adapter-agnostic schema — V1 only enables the watcher for openclaw, V2 can plug Claude / Codex into the same table without migrating. - `ProducedFilesStore` — snapshot/finalize-turn diff API plus by-turn / by-agent queries and a path-resolver that enforces workspace-root containment for the download / preview routes. - `walkWorkspace` — bounded recursive workspace walker; skips symlinks (no host-fs smuggling), excludes node_modules / .git / .cache, hard-capped at 50k entries / depth 16. - `file-preview` helper — extension + magic-byte MIME detection, bounded text-snippet reader (1 MB cap), inline image base64 reader (4 MB cap). Streaming download path lives in the route layer (next phase) — this module only handles the small in-memory reads the preview UX needs. * feat(server): attribute openclaw turn outputs to the harness layer Phase 2 of TKT-762 — wire the per-turn workspace diff into the single dispatch path that owns every turn's lifecycle. Two prior wiring points the original plan named (the OpenClaw HTTP chat route + OutboundQueueService.tryDispatch) were collapsed in dev into agent-harness-service.runDetachedTurn — both direct sends and queued sends route through it now, so a single hook covers both. The old `OutboundQueueService` is gone; its successor `message-queue.ts` re-enters runDetachedTurn for the queued case, so we still only need to bracket once. Changes: - New `produced_files` variant on `AgentStreamEvent` so the inline artifact card has a wire-format hook independent of the REST API. - `ProducedFilesStore` gains `resolveAgentDefinitionId` to bridge gateway-side openclaw agent names to the harness's `agent_definitions.id`, handling both the reconciled-row shape (id == openclaw name) and the BrowserOS-created shape (id = oc-<uuid>, name = openclaw display name). - `AgentHarnessService.runDetachedTurn`: snapshot the openclaw workspace before `runtime.send(...)`, finalize the diff in the outer finally, push the resulting rows as a `produced_files` event. Adapter-gated to openclaw only — Claude / Codex agents write to the user's own filesystem and don't need attribution. - Skip attribution on user-cancel (`abort.signal.aborted`) so the side effects of an aborted turn don't get surfaced as "outputs you asked for." On runtime errors we still attribute, because partial outputs are what the user is most likely to want to recover. - Lazy-init the store via `tryGetProducedFilesStore()` so tests that swap in a fake `agentStore` don't trip the process-wide `getDb()` initialisation guard. - File attribution extracted into `attributeTurnFiles` helper to keep `runDetachedTurn`'s cognitive complexity under the lint ceiling. Verifications: - Server tsgo --noEmit clean for changed files. - 162/162 server-api tests pass. - Biome lint clean on all three changed files. * feat(server): expose produced-files HTTP API for /agents Phase 3 of TKT-762 — surface the rows Phase 2 attributes via four read-only endpoints under the existing `/agents` router. Mounted where the agents page already polls so the rail UI doesn't add a second router/origin to its trust boundary. Routes: - GET /agents/:agentId/files Outputs-rail data, grouped by the assistant turn that produced each batch, newest first. `?limit=` clamps to N rows server-side (default 200). - GET /agents/:agentId/files/turn/:turnId Per-turn refresh — used by the inline-card consumer to rebuild metadata after the SSE `produced_files` event lands, and by direct fetches that missed the live event. - GET /agents/files/:fileId/preview Discriminated `FilePreview` JSON: text snippet (≤1MB), base64 image (≤4MB), pdf metadata, or `binary` placeholder when neither preview path applies. 404 when the file id is unknown OR the on-disk file disappeared after attribution. - GET /agents/files/:fileId/download Streams raw bytes via `Bun.file().stream()` with `Content-Disposition: attachment` and the detected MIME type. The fileId is opaque — the server resolves the agent and on-disk path; the client never sees a path, so traversal is impossible by construction. Service layer: - `AgentHarnessService` gains `listAgentFiles`, `listAgentFilesForTurn`, `previewProducedFile`, and `resolveProducedFileForDownload`. All four are no-ops for claude / codex adapters (they return null/[]) so the route contract stays uniform across adapters even though only openclaw produces rows in v1. - New `ProducedFileEntry` and `ProducedFilesRailGroup` DTOs — trimmed wire shapes that strip `agentDefinitionId` and `sessionKey` from the on-disk row. Verifications: - Server tsgo --noEmit clean for changed files (only pre- existing `Bun` global warning). - 162/162 server-api tests pass. - Biome clean on both changed files. Smoke-test instructions for the route shape live in the plan under §6 and §8; full end-to-end smoke happens in Phase 6. * feat(agent): client-side hooks + types for agent file outputs Phase 4 of TKT-762 — frontend foundation for the inline artifact card and the per-agent Outputs rail. UI components themselves land in Phase 5; this commit only adds types, hooks, and shared helpers so the wiring is in place when the components arrive. New module: `apps/agent/lib/agent-files/` - `types.ts` — `ProducedFile`, `ProducedFilesRailGroup`, and the discriminated `FilePreview` union, mirrored from the server-side DTOs in `apps/server/src/api/services/agents/agent-harness-service.ts`. The `agentDefinitionId` / `sessionKey` columns on the on-disk rows deliberately do NOT exist at the type boundary — clients refer to files by opaque `id`. - `file-helpers.ts` — pure helpers: `inferFileKind` (icon routing), `formatFileSize`, `extensionOf`, `basenameOf`, `buildFileDownloadUrl`. No React, no fetch, no DOM — anything stateful belongs in the hooks. - `useAgentOutputs.ts` — `useAgentOutputs(agentId)` for the rail, `useAgentTurnFiles(agentId, turnId)` for the inline card, `useInvalidateAgentOutputs()` for the chat-stream-completion hook (Phase 5 will plumb this), and `useRefreshAgentOutputs()` for the rail's manual refresh button. - `useFilePreview.ts` — `useFilePreview(fileId)` with `staleTime: Infinity` (previews are immutable for a given id; no point refetching on focus). Always opt-in (`enabled`) — the preview only loads when the user clicks a row. - `index.ts` — barrel re-export so consumers import from one path. Touched in `apps/agent/entrypoints/app/agents/`: - `agent-harness-types.ts` — added `produced_files` variant + the `HarnessProducedFile` type to `AgentHarnessStreamEvent`. Mirrors the server-side change from Phase 2 so the client SSE consumer type-narrows correctly. - `useAgents.ts` — exported the previously-private `agentsFetch` helper and the `AGENT_QUERY_KEYS` registry so the agent-files hooks reuse them without duplicating fetch / key conventions. Three new keys added: `agentOutputs`, `agentTurnFiles`, `filePreview`. Verifications: - Agent tsgo --noEmit clean. - Biome clean on all touched files. * feat(agent): inline artifact card + per-agent outputs rail Wires the chat surface to the produced-files API shipped earlier: - Inline artifact card under each assistant turn that produced files, populated by the live `produced_files` SSE event (resumes also stamp `turnId` so a missed live event can fall back to the per-turn fetch). - Collapsible right-side Outputs rail on the agent conversation page, grouped by turn, with Refresh + per-agent open/close persistence in localStorage. Gated to openclaw adapters in v1. - Shared file preview Sheet branches on the FilePreview union: text snippet (markdown for `.md`/`.mdx`, otherwise pre+code), image data URL, and download-only fallback for pdf/binary/missing. - Conversation hook invalidates the rail's React Query cache from its finally block so newly attributed files appear without a manual refresh. * feat(agent-files): polish — symlink-safe paths + toast on failures - `resolveFilePath` now rejects symlink-escapes from the workspace by realpath-resolving both endpoints and re-checking containment. Lexical traversal (`..` segments) still fails fast without touching the filesystem. - Added `produced-files-store.test.ts` with 6 path-resolution cases including a symlink whose target lives outside the workspace root — the prior string-only check would have allowed this. - File preview Sheet: surfaces preview-load failures in a toast (in addition to the inline error block, which is easy to miss when the body has scrolled). Download button now intercepts the click so a missing baseUrl shows a toast instead of silently hiding the button. - Outputs rail: refresh failures fire `toast.error` with the underlying message. * fix(agent-files): drop duplicate `/agents` prefix from client paths `agentsFetch` / `buildAgentApiUrl` already prepend `/agents`, but the file-output hooks were passing fully-qualified paths (`/agents/<id>/files`, `/agents/files/<id>/preview`, etc.) which resolved to `/agents/agents/...` and 404'd. Fixed the four call sites to pass paths relative to the `/agents` root. * fix(agents): strip openclaw role envelope from chat history PR #924 introduced a second `<role>…</role>` prefix for openclaw turns — a single-line block distinct from the multi-line BrowserOS role TKT-774 wired the unwrap against. Because TKT-774's `stripOuterRoleEnvelope` matched the BrowserOS prefix exactly, the openclaw envelope sailed through unstripped and user messages on openclaw agents rendered the full preamble in /sessions/main/history responses. Make the strip adapter-agnostic: any `<role …>…</role>\n\n<user_request>\n…\n</user_request>` shape gets unwrapped. Drops the now-unused BROWSEROS_ACP_AGENT_INSTRUCTIONS constant and adds a regression test that uses the openclaw form verbatim. * feat(agent-files): inline file-card strip with rail deep-link Replaces Phase 5's row-list ArtifactCard with a horizontal strip of small file cards under any assistant turn that produced files. Click a card → opens the FilePreviewSheet directly (preview + download). Click View / +N → opens the per-agent Outputs rail and scrolls / expands the matching turn group. The card strip: - Caps at 4 visible cards; remainder collapses into a +N pill that shares the View handler. - Owns its own FilePreviewSheet instance (parallel to the deprecated ArtifactCard) so the per-card preview path doesn't fight with the rail's Sheet. - Hidden during streaming and absent when producedFiles is empty. - Adapter-gated upstream: AgentCommandConversation only passes the open-rail callback when adapter==='openclaw', so claude / codex agents render no rail-opening affordance. Rail changes: - Accepts focusTurnId + onFocusTurnConsumed; the matching RailTurnGroup expands and scrollIntoView's on focus, then fires the consumed callback so the parent can drop the URL state. - ?outputsTurn=<turnId> deep-links work: external nav opens the rail, sets focusTurnId, and clears the param after consumption. ArtifactCard is marked @deprecated; remove in a follow-up once nothing imports it. * fix(agent-files): keep file-card strip visible after history reload After Phase 7 the inline FileCardStrip vanished as soon as a turn finished: `filterTurnsPersistedInHistory` dropped the optimistic turn once history reloaded, and history items don't carry `producedFiles`. So the user could see a file produced inside an assistant message but no card to open it. Two fixes in tandem so the strip survives both the just-finished case AND a fresh page load: - New `selectStripOnlyTurns` keeps persisted turns that still carry `producedFiles`. `ConversationMessage` learns a `stripOnly` mode that renders only the trailing strip (no duplicate user/assistant bubbles, since those are rendered by `ClawChatMessage`). - `AgentCommandConversation` now also calls `useAgentOutputs` and passes `tailStripGroups` to `ClawChat`. Each rail group not already covered by a live or strip-only turn renders as its own tail `FileCardStrip` after history. Dedup keys on `turnId` so the same turn never doubles up. Adapter-gated upstream — claude / codex agents skip the useAgentOutputs fetch entirely. The card click still opens the preview Sheet directly; View / +N still deep-link to the rail at the matching turn group. * fix(agent-files): per-turn association + cache invalidation Two fixes for the inline file-card strip: 1. Strips were stacking at the conversation tail because every produced-files group rendered as a tail strip after history. New `mapHistoryToProducedFilesGroups` matches each group to the assistant history message that came from its turn — by `group.turnPrompt` vs the first non-blank line of the preceding user message — and ClawChat renders the strip directly under that bubble. Groups that don't match any history pair (orphans) still fall through to the tail. 2. `useInvalidateAgentOutputs` was passing `undefined` as the baseUrl placeholder to `invalidateQueries({ queryKey })` — react-query's positional partial-match doesn't treat undefined as a wildcard, so the cache stayed stale until the query refetched on its own (e.g. window focus). Switched to predicate-based invalidation that matches by [agentOutputs marker, agentId] regardless of baseUrl. Same for the per-turn files key. Net effect: send a turn that produces files → strip appears under the just-finished assistant message; reload the page → strips still appear under the right bubbles, not bunched at the bottom. * fix(agent-files): review feedback — name guard, RFC 5987, limit cap Three review-flagged issues: 1. Path traversal via agent display name — `getHostWorkspaceDir` accepted any string and `path.join`'d it, so a name like `../../tmp` escaped `.openclaw`. The pre-turn snapshot would then walk that escaped directory and attribute every file to the new turn; resolveSafeWorkspacePath's containment check is relative to the same escaped root so it would later serve arbitrary host paths. Added `isAgentWorkspaceNameSafe` (rejects `..`, separators, control chars, leading dots, empty); the builder now throws on unsafe names plus a defensive realpath-style containment check after the join. Harness wraps the call so the path-traversal trip just disables file attribution for the turn instead of failing the whole send. Six-case regression test pinned. 2. `encodeRfc6266Filename` JSDoc claimed an RFC 5987 `filename*=UTF-8''<percent-encoded>` fallback but the impl only stripped CRLFs/quotes. Now actually emits the fallback when non-ASCII is present; helper returns the full `filename="…"; filename*=UTF-8''…` attribute pair so the call site doesn't have to wrap in quotes. 3. `/agents/:agentId/files` `?limit=` was forwarded to the DB uncapped — extracted `parseAgentFilesLimit` that clamps to [1, 500] before forwarding. Also extracted `resolveSafeWorkspaceDir` + `snapshotWorkspaceForTurn` helpers off `runDetachedTurn` so the new safety branch doesn't push it past biome's cognitive-complexity cap.	2026-05-05 19:48:28 +05:30
Dani Akash	fbae45eb97	feat(agent): calm composer + redesigned hero (#931) ... * feat(agent): calm composer + redesigned hero on /home Adopt the Variant A redesign aesthetic on /home — hero text and composer styling only. shadcn primitives and CSS variables unchanged; conversation-screen composer untouched. Hero: - Larger display title (clamp 36→56px, weight 600, tighter letter-spacing, balanced wrap). - Italic muted span around "work on" — small typographic accent that makes the hero read as designed rather than default. Composer (variant="home" only): - Internal dashed divider between the typing area and the footer chip row. The visual cornerstone of the calm aesthetic. - Footer chips become 24px pill-shaped (rounded-full), ghost-on- idle / muted-bg-on-hover. Workspace and Tabs show muted trailing values inline (none / 0). - Agent selector on the far left of the footer gets a filled-pill trigger variant (bordered, accent/40 background, mono name) to visually anchor the row. AgentSelector exposes a triggerVariant prop (ghost | pill); chat surface keeps the existing ghost. - Subtle 1px vertical divider between the agent pill and the rest. - Right-aligned keyboard hint (↵ to run · ⇧↵ new line) using kbd elements with the existing accent/border tokens. - Outer shell gains a soft accent-orange focus-within ring. Out of scope (future PRs): TRY suggestion chips, eyebrow strip, recent-agents redesign, activity log. * fix(agent): textarea bg leaks in dark mode * style(agent): paint hero italic span in accent orange * feat(agent): adopt calm composer aesthetic on chat-screen too Bring the calm-composer footer (dashed divider, pill chips, keyboard hint) over from /home to /agents/:agentId so both surfaces share one design language. - Rename HomeContextControls → CalmContextControls; the agent selector is conditional via showAgentSelector, so chat hides it while home keeps the filled agent pill on the left. - Drop the legacy ContextControls function entirely (~140 LOC) and collapse the variant branching at the call site to a single CalmContextControls render. - Add the same focus-within accent ring to ConversationShell that HomeShell already has, so the focus signal is consistent. The chat composer's Stop button (between textarea and voice mic) is unchanged — it lives outside the footer chip row and only surfaces while streaming. --------- Co-authored-by: DaniAkash <DaniAkash@users.noreply.function>	2026-05-05 14:18:29 +05:30
Dani Akash	ce4bb44083	feat(agent): /home composer parity with image attachments (#930) ... * feat(agent): /home composer parity with image attachments The /home composer used the same ConversationInput component as the chat screen but passed attachmentsEnabled={false}, and the home → chat handoff was a URL search param `?q=<text>` that physically can't carry binary attachments. Pasting a screenshot at /home did nothing. Add a small in-memory registry (pending-initial-message.ts) as the rich-data side channel for the same navigation: the home composer writes { agentId, text, attachments } there before navigating; the chat screen consumes it on mount and replays through the existing harness send() path that already supports attachments. URL `?q=` stays for shareable text-only prompts; the registry wins when both are present. Module-scope, 10s TTL, destructive consume. Net: home is now flagged attachmentsEnabled={true}; users can paste, drag, or pick image files at /home and they survive the navigation into the chat screen with previews intact. * docs(agent): clarify why initial-message ref reset is safe post-registry-fire	2026-05-04 18:02:31 +05:30
Dani Akash	84e2739663	feat(agent): rich rail + header on /agents/:agentId chat (#908) ... * feat(agent): rich rail + header on /agents/:agentId chat Replace the chat screen's legacy AgentEntry rail and binary READY header with the same rich data the /agents page already exposes: adapter glyph, liveness dot, pin star, status badge, adapter · model · reasoning chip line, last-used time, lifetime tokens, queue count, and the Adapter Unavailable warning. Source of truth flips from the merged AgentEntry list to useHarnessAgents() directly. Sort order matches /agents (pinned → recency) — not /home (active-first → recency) — because chat is index-shaped and shuffling rows every 5s as turns transition would be jarring while reading. Lift the inline pin-then-recency comparator out of /agents AgentList.tsx into a shared agents-list-order.ts so both surfaces stay on identical sort semantics. * fix(agent): chat header height + composer sticking to bottom Header was clipping descenders because the strip was vertical-content sized at min-h-14 with tight py-2.5; bump padding and lean on natural content height. Drop the AgentTile glyph (the rail row already shows adapter identity) and the cwd path (too long, pushed the meta line off-screen). Header is now name + pin star + status pill, then adapter · model · reasoning, then last-used · tokens · queued. Composer was floating mid-screen on short chats because the chat grid had no grid-template-rows — the implicit auto row collapsed to content height, so the right-column flex wrapper never received the full container height. Add grid-rows-[minmax(0,1fr)] so the single row claims 100% and ClawChat's flex-1 expands to push the composer flush to the bottom. * fix(agent): composer flush to bottom on short chats Match the sidepanel chat's nested-flex pattern. The right-column wrapper got h-full so it expands to the grid row; the conversation controller's root added flex-1 so ClawChat's existing flex-1 has something to actually fill against. Without these, the grid cell stretched but the inner flex columns shrank to content height, leaving the composer floating mid-screen. * fix(agent): align rail header with chat header in shared top band Pull the rail's "Agents" + back-button into the same horizontal strip as the agent identity header. The two halves now sit on a single row that spans both columns, so they can't drift in height as the chat header gains/loses meta lines (last-used, tokens, queued). The rail below the band keeps its scrollable list only; the chat column below holds the conversation + composer. Border-bottom moves from ConversationHeader to the band wrapper so we don't get a double-rule on the boundary. * fix(agent): reserve header height to prevent layout shift on data load The chat header grew from a single line to three lines once the useHarnessAgents() poll resolved (adapter chips + meta line populate asynchronously), shoving the rail and conversation body downward. Lock min-h-[84px] on both the band's left "Agents" cell and the ConversationHeader root, and always render the meta line slot (non-breaking space when empty) so the typographic frame is stable regardless of data state. * refactor(agent): pull status pill + meta to right side of chat header Two-column header layout instead of three stacked rows: name + pin star + adapter chips on the left, status pill stacked on top of the last-used / tokens / queued meta line on the right. Drops min-h from 84px → 60px so the band reclaims ~24px of vertical space and the chat body starts higher on screen. Band's left "Agents" cell matches the new height.	2026-05-01 20:19:16 +05:30
Dani Akash	8712f89f18	feat(agents): durable per-agent chat message queue + composer Stop (#880) ... * feat(agents): durable per-agent chat message queue + composer Stop button * fix(agents): tighten queue UI — smaller Stop, drop empty indicator, live drain attach User feedback round 1 on the message-queue UX: 1) The Stop button matched the send/voice mics at h-10 w-10 with a solid destructive fill, which read as alarming. Shrunk to h-8 w-8, ghost variant with a soft destructive/10 background, smaller filled square glyph. Reads as a calm 'stop' affordance instead of a panic button. 2) The QueueItem's leading <QueueItemIndicator> dot was decorative only — no state, no interaction. Dropped it from QueuePanel along with the import; queue items now render as a clean preview line with the trailing X remove action. 3) When the server drained the queue and started the next turn, the chat panel didn't pick up the live stream until the user navigated away and back. The hook's resume effect previously only fired on agent change, not on listing-observed activeTurnId change. Surface activeTurnId from useHarnessAgents into useAgentConversation; effect now re-runs when the id changes, calls /chat/active, and attaches to the new turn — so a queued message starts streaming the moment the server drain pops it. * fix(agents): don't reset streaming state from the resume effect's no-op paths The Stop button was disappearing while the agent was actively streaming, even though events were still flowing into the chat. Root cause: the resume effect's `finally` block reset `streaming`, `turnIdRef`, and `lastSeqRef` unconditionally — including on the early-return paths (no active turn, or another mechanism already owns the stream). Sequence that triggered it: 1) User sends a message → send() sets streamAbortRef + streaming=true and starts consuming the SSE. 2) User enqueues another message → enqueue mutation invalidates the listing query. 3) Listing refetches with the live activeTurnId → the resume effect re-fires (deps include activeTurnIdDep). 4) attemptResume hits `if (streamAbortRef.current) return` because send() owns it. 5) The finally clause fires anyway and calls setStreaming(false), clobbering the live state set by send(). The SSE consumer keeps running (refs are intact) so text keeps streaming, but the React flag is wrong, so the Stop button gates off. Fix: track whether *this* run actually started a stream (`weStartedStream`). The finally only resets state when it does. Early-return / no-active-turn paths now leave streaming/turnIdRef/ lastSeqRef alone for whoever does own them. Also widens the Stop button's visibility (`canStop` prop on ConversationInput) so it stays steady across the brief gap between turns when a queue drain is mid-flight; the parent computes `streaming || activeTurnId !== null || queue.length > 0`. The visibility widening is independent of the streaming-state fix above — both are now in place. * revert: drop canStop widening — Stop only shows while streaming Reverts the canStop prop on ConversationInput and the OR-with-queue visibility from AgentCommandConversation. Stop is gated solely on `streaming` again. Between turns (queue draining) the button stays hidden — only the actively-streaming turn is interruptible from the composer, which matches what the user actually expects. * fix(agents): persist the kicking-off prompt on active turns so the resume placeholder isn't empty When a queued message drained and started a new turn, the chat panel's resume effect staged a placeholder turn with userText: '' because the hook had no way to know what message kicked off the turn — only the agent-side stream was visible, and the user bubble above it was blank until the user navigated away and back (at which point the session record's history loaded normally). Fix: ActiveTurnRegistry.register now accepts an optional `prompt` that's stashed on the turn and surfaced via describe() / the ActiveTurnInfo response. AgentHarnessService.startTurn passes the incoming message into register. /chat/active returns it. The chat hook's resume effect uses active.prompt as the placeholder turn's userText, so the user bubble shows the queued message text the moment streaming begins. Falls back to '' for older clients that haven't been refetched yet. * fix(agents): always release streamAbortRef on resume cleanup, even when cancelled Greptile P1 follow-up. The previous `weStartedStream` guard correctly stopped the resume effect's no-op early-returns from clobbering an in-flight `send()` stream — but it also stopped a *cancelled* mid-stream resume from clearing its own `streamAbortRef`. When the cleanup fires (e.g. the 5s listing poll captures a new queue-drain turn id while the SSE for the prior turn is still finishing), the next effect run hits the `if (streamAbortRef.current) return` guard against the now-aborted controller and never reattaches, leaving `streaming === true` with no live stream until the user navigates away. Split the finally block: always release `streamAbortRef` when we owned the controller (so the next run can take over), but only reset the streaming flag / turn id / lastSeq on a clean exit (the new run will set those itself, so resetting on cancel would just flicker).	2026-04-30 18:26:56 +05:30
Dani Akash	ba60bf466f	feat(agents): rich command-center rows + home grid + dead-code sweep (#879) ... * feat(agents): rich-info command center rows + pin/PATCH/adapter-health backbone Splits AgentRowCard from a 271-line monolith into a shallow tree of single-responsibility sub-components under `agent-row/`: AgentTile, AdapterHealthDot, PinToggle, AgentTitleRow, AgentSparkline, AgentSummaryChips, AgentLastMessage, CwdChip, AgentTokenSummary, AgentMetaRow, AgentErrorPanel, AgentActions Adds the data each row consumes: - pinned: boolean field on AgentDefinition + FileAgentStore.update + new PATCH /agents/:id route. useUpdateHarnessAgent mutation optimistically updates the listing cache so the star flips instantly; rolls back on error. - Listing payload extended with lastUserMessage, cwd, tokens (cumulative + last7d shape — last7d zero-filled until the activity ledger lands), turnsByDay/failedByDay (zero-filled), lastError/lastErrorAt, activeTurnId. AcpxRuntime grows a getRowSnapshot() that reads cwd + cumulative tokens + last user message from the session record in one pass. - Adapter health: in-memory AdapterHealthChecker probes `claude --version` / `codex --version` with a 2s timeout and caches results for 5 min. /adapters response carries { healthy, reason?, checkedAt }. Tile-corner dot exposes the state via HoverCard; openclaw inherits health from the gateway snapshot already on the page. Sub-components are pure: card itself owns no state. Sort order becomes pinned-first, then recency. HoverCard is the workhorse for keeping rows compact while exposing depth (full message, token breakdown, daily turn list, error stack, adapter reason). * refactor(agents): tighten command-center row design + cut redundant affordances User feedback round 1: 1) Two green dots on the tile (health + liveness) was confusing. Health moves out of the tile entirely and surfaces as an inline 'Unavailable' chip in the model line — silent when the adapter is healthy, with a warning amber chip + HoverCard reason when not. The tile now shows one signal: liveness. 2) The last-user-message HoverCard wasn't telegraphing intent. Drop the HoverCard. The line is informational, italic, with a leading quote glyph so the row reads like a conversation snippet. To see the full message the user opens the chat (which is the action they want next anyway). 3) Resume + Chat were duplicate CTAs. Single primary action per row: Resume (filled, accent-orange, with a pulsing dot) replaces Chat when there's an active turn. Both navigate to /agents/:id but the row tells the user which action they're taking. 4) Tokens weren't visible because the row gated on last7d.requestCount, which is zero until the activity ledger ships. Switch to lifetime tokens (which we have today). Drop the '7d stats:' framing — talking about a window we can't compute would be misleading. The HoverCard surfaces input/output split + a footnote that per-window stats land in a follow-up. 5) CWD was rendering the server's own running directory, which is meaningless to users. Hide it from the row entirely. The cwd field still rides in the listing payload for future surfaces (chat panel, debug view) — only the row stops rendering it. Aesthetic refinements while we're here: - Whole card carries state, not just the tile: working rows get an accent-orange tinted border with a soft glow, error rows tint destructive, idle rows lift on hover. - Pin star fades in on hover (group-hover) when unpinned and stays solid amber when pinned — keeps the rail calm by default. - Tabular-nums on token figures so columns visually align across rows. - Drop CwdChip and AdapterHealthDot files: no callers left. * fix(agents): align row title flush-left whether pinned or not Pin star moved from leading the title to trailing the badges, and hidden from layout entirely (`hidden group-hover:inline-flex`) when unpinned. The previous `opacity-0` rule kept the star reserving its `size-6` slot, which left every unpinned title indented relative to the model / preview / meta lines underneath it. Title now flushes left in both states; pinned star stays solid amber so the signal isn't hidden, and unpinned reveals an outline star on row hover for the toggle affordance. * fix(agents): keep pin-toggle slot reserved so row height is constant Switching the unpinned star from `hidden group-hover:inline-flex` to `opacity-0 group-hover:opacity-100`. The hidden/show variant was collapsing the title row's height when the star wasn't rendered, which made every card below visibly shift on hover. Always rendering the button (with opacity-only visibility) keeps the row's vertical metrics constant; the title still flushes left because the slot is trailing, not leading. Card hover effect (-translate-y + shadow-md) restored — the layout shift wasn't coming from the card hover; it was the pin slot appearing and disappearing. * fix(agents): quieten row hover — border-tint only, no lift, no shadow Drop the `-translate-y-px` and `hover:shadow-md` from the row card plus the working-state inner ring. The translate + shadow grow combination was visibly noisy as the cursor moved through the rail — each row 'lifted' as you passed over it. Hover now just tints the border in accent-orange/30; working and error states keep their distinct border colours but no inner ring. Card height and shadow stay constant in every state, so the rail reads as a calm vertical list of cards. * feat(home): rich Recent Agents grid + dead-code sweep The /home Recent Agents grid was a placeholder shell. Every 'rich' field on the card (lastMessage, lastMessageTimestamp, activitySummary, currentTool, costUsd) was wired to undefined because AgentCommandHome called `buildAgentCardData(agents, status?.status, undefined)` — the dashboard arg has been hard-coded undefined since the harness migration. Repointing the grid at `useHarnessAgents` + `useAgentAdapters` gives every card the same enriched data the rail uses. What the new card shows per agent: • Adapter glyph tile + liveness dot (working pulses; asleep is hollow; error is red) • Name + Working pill (when active) • Adapter · model · reasoning summary line, with an inline Unavailable chip + HoverCard reason when the adapter binary isn't on $PATH • Italic last-user-message preview (line-clamp-2, leading quote glyph) — same visual language as the rail • Footer: 'X ago' + state chip (Asleep / Attention) OR a Resume button (orange, with pulsing dot) when activeTurnId is non-null Sort on the home grid is active-turn → recency. Pinning is NOT a sort key here (and there's no pin indicator on the card) — pinning belongs to the rail at /agents; the home page is action-oriented and trusts active-turn + recency to surface the right agent. Dead code removed: • useAgentDashboard.ts (96 lines, no callers; subscribed to the dead /claw/dashboard/stream from the OpenClaw-only era) • useAgentCardData.ts (the dashboard-merge shim; passed undefined every call so all enriched fields landed as undefined) • AgentCard.tsx (AgentCardExpanded replaced by HomeAgentCard; AgentCardCompact had no callers — the dock's compact mode was never used) • AgentCardData interface dropped from lib/agent-conversations/ types.ts; the new card consumes HarnessAgent directly Visual language stays continuous between rail and grid: same <AgentTile>, same <LivenessDot>, same italic-quote message preview, same orange Resume button with a pulsing dot.	2026-04-30 16:36:22 +05:30
Dani Akash	a228c278c6	feat(agents): background-resilient chat — turns survive tab disconnect (#863) ... * feat(agents): decouple chat turn lifecycle from SSE response Introduce a per-process ActiveTurnRegistry that owns each agent turn's lifecycle and a ring-buffered event stream, so chat tabs that close, refresh, or navigate away no longer cancel the in-flight turn. New endpoints: POST /agents/:id/chat starts a turn (now returns 409 when one is already running, with the active turnId for attaching) GET /agents/:id/chat/active reports the running turn for a UI that just mounted GET /agents/:id/chat/stream subscribes to a turn; supports Last-Event-ID resume via per-event seq ids POST /agents/:id/chat/cancel explicit cancel — fetch abort no longer affects the underlying turn The chat hook now captures X-Turn-Id, tracks lastSeq from SSE id lines, re-attaches on mount when the server still has an active turn, and routes Stop through the cancel endpoint. The runtime call uses the registry's per-turn AbortController instead of the HTTP request signal, which is the core decoupling that lets turns outlive their initiator. * feat(agents): add ActiveTurnRegistry primitive backing the new chat lifecycle The previous commit referenced these files in tests and the harness service but global gitignore swallowed them on the first add. The registry owns the per-turn ring buffer (drop-oldest, terminal frame preserved), the per-turn AbortController, and subscriber fan-out used by /chat/stream resume.	2026-04-29 21:01:06 +05:30
Dani Akash	0c84547e8f	feat(agents): migrate OpenClaw chat onto the unified harness/ACP path (#859) ... * chore(acp): smoke-test ACP capabilities against running gateway Adds apps/server/scripts/acp-smoke.ts which spawns `openclaw acp` inside the gateway container and exercises every method we plan to depend on: initialize, newSession, prompt (text + image), cancel, listSessions, loadSession. SDK pinned to 0.19.1 (Bun's minimum-release-age policy blocks 0.20+ which were released < 7 days ago). Findings (full notes in plan outcomes): - promptCapabilities advertises image:true but the model does NOT see image bytes — silently dropped at the bridge. - sessionCapabilities advertises {list:{}} but session/list throws "Method not found": stale capability advertising. - loadSession works; replays user/assistant/thought text and session_info/usage/commands updates. No tool_call replay, as documented. - cancel works end-to-end: stopReason=cancelled. - closeSession/resumeSession are not on ClientSideConnection in 0.19.1; kill child to close, use loadSession for rebind. Plan revisions triggered by spike are recorded in plans/browseros-ai/BrowserOS/features/2026-04-28-2310-claude-code-acp-implementation-roadmap.md. * chore(acp): re-run smoke on SDK 0.21.0 and add mode/config/auth scenarios After bypassing Bun's minimum-release-age and upgrading the SDK to 0.21.0, restore the previously-skipped resume/close paths and add three new scenarios: mode (setSessionMode), config (setSessionConfigOption, correct configId field), and auth (authenticate noop). Findings, all bridge-side (independent of SDK): - session/list, session/resume, session/close all throw -32601 on OpenClaw 2026.4.12 — capability advertising is stale. - Image content blocks silently dropped; model never sees the bytes. - setSessionMode and setSessionConfigOption work; latter requires `configId` (not `optionId`) per the schema. - loadSession replays user/assistant/thought text + session_info + usage + available_commands; no tool_call replay (documented). - authenticate is a noop on OpenClaw (no authMethods advertised). Plan outcomes updated with full method-support matrix. * chore(deps): promote @agentclientprotocol/sdk to a runtime dependency The smoke script in apps/server/scripts/acp-smoke.ts used the SDK as devDependency. The upcoming ACP bridge (apps/server/src/api/services/acp/) needs it at runtime, not just for tooling. Move the entry from devDependencies to dependencies, alphabetically first under @a*. Pinned to 0.21.0 — same version the smoke script validated against. README gains a small Dependencies note pointing at the future bridge location. No code changes yet. The bridge wiring lands in subsequent commits. * fix(openclaw): wire LlmProvider.supportsImages through to OpenClaw model config When BrowserOS sets up a custom OpenAI-compat provider on the gateway, the agent UI's "Supports Image" flag (LlmProviderConfig.supportsImages) was being dropped on the floor. As a result the persisted model entry had no `input` field, OpenClaw defaulted it to ['text'], and image_url content parts were silently stripped before the model saw them. Fix: - Extend OpenClawSetupInput / OpenClawAgentMutationInput on the agent side (useOpenClaw.ts) and the route body schema + SetupInput + createAgent input on the server side with `supportsImages?: boolean`. - AgentsPage forwards `llmOption?.supportsImages` from the selected LlmProviderConfig in both handleSetup and handleCreate. - provider-map.resolveSupportedOpenClawProvider emits `input: ['text', 'image']` on the model entry when the flag is truthy; otherwise emits the explicit `['text']` so the value is always pinned (avoids relying on OpenClaw's implicit default). - applyBrowserosConfig adds `tools.media.image.enabled = true` to the bootstrap batch so the gateway's image-understanding pipeline is always wired up — per-model `input` still gates which models see images, this just enables the global path. ACP image content blocks are still dropped by the OpenClaw bridge — that's a separate bridge bug, not addressed here. This commit restores image support for the OpenAI-compat /v1/chat/completions path that the upcoming ACP chat panel will use as a carve-out for image-bearing prompts. Existing custom-provider configs are NOT auto-migrated; users will re-acquire image support either by re-running setup or by editing their model entries' `input` field manually. A migration pass for legacy installs is not in scope for this commit because the "supportsImages" intent isn't recoverable from the persisted config alone — the source of truth is the LlmProvider record on the agent side. * feat(agents): add OpenClaw to AgentAdapter union and catalog Extends AgentAdapter to 'claude' | 'codex' | 'openclaw' and adds the OpenClaw entry to AGENT_ADAPTER_CATALOG. The new entry has: - defaultModelId: 'default' — OpenClaw's ACP bridge does not surface per-session model selection (verified during the ACP spike), so models live in the OpenClawService config, not in the adapter catalog. AgentDefinition.modelId carries the gateway-side model name for display only. - models: [] — empty list signals "no per-session model picker" in the UI; isSupportedAgentModel('openclaw', undefined|'default') returns true via the existing fallback path. - reasoningEfforts mirror OpenClaw's session-level `thought_level` config option (off / minimal / low / medium / high / adaptive). Also extends: - isAgentAdapter type guard recognizes 'openclaw' - HarnessAgentAdapter union on the extension side - agents.test.ts createAgent fake type - agent-catalog.test.ts asserts on the new entry, empty model list passthrough behavior, and OpenClaw's reasoning effort set Lockfile delta is the workspace SDK pin reconciling 0.20.0 (taken from dev's lock) up to our package.json's 0.21.0 (added in c1d987ea). acpx still uses 0.20.0 transitively — both are present. No runtime wiring yet — the registry override and AcpxRuntime plumbing land in subsequent commits. * feat(agents): plumb OpenClaw gateway accessors into AcpxRuntime Adds an optional `openclawGateway` accessor to AcpxRuntime so the upcoming registry override (Step 4) can spawn `openclaw acp` inside the gateway container with the right port, token, and container/VM identity. All accessors are getter-shaped so values stay live across gateway restarts (port can change, token can rotate). The accessor is threaded: server.ts → createAgentRoutes → AgentHarnessService → AcpxRuntime ↘ sidepanel lazy AcpxRuntime Also adds OpenClawService.getGatewayToken() returning the in-memory token string. We pass it via OPENCLAW_GATEWAY_TOKEN env var on the spawn (per OpenClaw's documented env-var precedence) instead of via `--token` flag (which leaks to ps aux) or `--token-file` path (no discrete token file lives inside the container — the token is nested inside openclaw.json). Wiring is dormant — the registry override that consumes these accessors lands in Step 4. Typecheck + existing acpx/harness/routes tests pass unchanged. * refactor(agents): scrub local plan-step references from code comments Replaces forward-looking comments that referenced internal plan steps (e.g. "Step 4 wires this into…") with comments that justify the code on its own merits. Plan files live locally on the contributor's machine, so cross-references are noise to the rest of the project. No behavior change. * feat(agents): spawn openclaw ACP adapter inside the gateway container When the harness resolves the `openclaw` adapter, it now returns a command that runs `openclaw acp` inside the bundled gateway container via `limactl shell <vm> -- nerdctl exec -i ... openclaw acp --url ws://127.0.0.1:<port>`. This reuses the openclaw binary already installed alongside the gateway — no host-side openclaw install is required. Auth: the gateway token is injected via OPENCLAW_GATEWAY_TOKEN on the container exec rather than `--token` on the openclaw CLI, so the secret never appears in `ps aux`. Banner output: OPENCLAW_HIDE_BANNER=1 and OPENCLAW_SUPPRESS_NOTES=1 keep stdout JSON-RPC-clean. LIMA_HOME: prefixed via `env LIMA_HOME=<path>` on the resolved command so the spawned limactl finds the BrowserOS-owned VM (the server doesn't set LIMA_HOME on its own process env). When the gateway accessor is absent, falls through to acpx's built-in openclaw adapter which assumes a host-side install — that branch will fail at spawn time with a descriptive error. Verified end-to-end via the existing acp-smoke script during the Step 0 spike. * feat(agents): dual-create OpenClaw harness agents on the gateway When the harness creates an `openclaw` adapter agent, it now also provisions a matching agent on the OpenClaw gateway via the existing CLI path (OpenClawService.createAgent). Symmetric on delete: gateway removeAgent runs alongside the harness-store delete. - Adds an OpenClawProvisioner interface (decoupled from OpenClawService for testability) and injects it through AgentHarnessService. - createAgent rolls back the harness record if gateway provisioning fails; deleteAgent tolerates gateway-side failures so harness identity stays consistent with the user-facing UI. - New OpenClawProvisionerUnavailableError surfaces as a 503 when an openclaw create request lands on a harness with no provisioner wired in (instead of a generic 500). - FileAgentStore mints openclaw agent ids with an 'oc-' prefix so the id satisfies the gateway's `^[a-z][a-z0-9-]*$` agent name pattern. Other adapters keep raw UUIDs to preserve compatibility. - POST /agents body schema accepts providerType / providerName / baseUrl / apiKey / supportsImages, forwarded to the provisioner when adapter='openclaw'. The agents-page UI still routes openclaw create through the legacy /claw/agents flow; switching that path to the harness is a separate UI cutover. Tests cover: gateway dual-create on success, rollback on gateway failure, 503 when provisioner is missing, and tolerant delete on gateway-side failure. * fix(agents): skip catalog model validation for OpenClaw adapter OpenClaw agents resolve their model from the gateway-side provider config (set at agent-create time via OpenClawService) rather than from the harness catalog, which has an empty `models: []` entry by design. Without this carve-out, every OpenClaw create body fails parsing with "Invalid modelId" because no concrete model id can satisfy isSupportedAgentModel('openclaw', ...). The reasoning-effort check still runs against the catalog (those values map directly to OpenClaw's session `thought_level` config option). * fix(agents): pass --session to openclaw bridge so newSession routes correctly acpx's AcpClient.createSession calls connection.newSession with cwd and mcpServers but never forwards the sessionKey. Without it, the openclaw bridge falls back to a synthetic acp:<uuid> session that doesn't resolve to any provisioned gateway agent — every harness chat returns a generic "Internal error" from -32603. Fix: bake `--session <key>` into the resolved spawn command. The bridge then uses that as the default session key for any newSession the bridge receives, routing the turn to the matching gateway agent. Per-session keying means each openclaw agent gets its own AcpxCoreRuntime instance (cached by sessionKey on top of the existing cwd/permissionMode key). This adds one extra runtime per active openclaw session — claude/codex are unaffected. Test asserts the resolved command includes the right --session arg. * fix(agents): suppress BrowserOS MCP for openclaw bridge The openclaw ACP bridge rejects newSession when mcpServers is non-empty because its provider tooling comes from the gateway, not from ACP-side MCP servers. Forwarding the BrowserOS HTTP MCP made every harness chat fail with a JSON-RPC -32603 "Internal error" before the session was even opened. Claude/codex still need the BrowserOS MCP for browser tooling, so the carve-out is keyed off whether the runtime is for an openclaw session. * feat(agents): route OpenClaw chat through the harness behind a flag Adds the `feature.useAcpxForOpenClaw` extension storage flag. When on, OpenClaw agents in the agent-command chat panel use the harness /agents/<id>/chat SSE and harness history hook instead of the legacy /claw/agents/<id>/chat. When off, behavior is unchanged. Also dedupes the agent rail when the same id appears in both stores (dual-created agents from /claw/agents and /agents) by preferring the harness entry — without this, every dual-created OpenClaw agent shows up twice after Step 5. Image attachments are temporarily disabled when the harness path is active; the carve-out lands in the next commit. * fix(agents): keep legacy OpenClaw agents on ClawChat The previous commit's flag-gated branch routed every `source='openclaw'` agent through `/agents/<id>/chat` when the flag was on, but the layout dedup means the only agents that ever reach that branch are legacy gateway-only entries (`main`, orphan agents from rolled-back creates) — which by definition have no harness record, so the harness path 404s and chat is unusable. Source is the only routing signal again: harness agents go through the harness, legacy agents stay on ClawChat. The storage flag stays for Step 9/10's migration story. * feat(agents): expose OpenClaw in sidepanel and route through gateway main `buildSidepanelChatTargets` now emits a single default ACP target for adapters with no per-session model picker (OpenClaw, whose model is configured on the gateway-side agent). Without this, OpenClaw never appeared in the sidepanel target picker because the catalog entry has `models: []`. Sidepanel sessions don't have a dedicated provisioned gateway agent. The openclaw bridge `--session` flag previously got the raw sidepanel key (`sidepanel:<convId>:openclaw:...`), which doesn't match any gateway agent — newSession was accepted but every prompt hung forever. The bridge command now rewrites non-harness session keys onto the always-present `main` gateway agent, encoding the original key as a channel suffix to keep state segregated per conversation. Verified end-to-end via curl: sidepanel openclaw chat streams `text-delta` + `finish: stop`. * feat(agents): backfill harness records for legacy gateway agents Reframes Step 9 of the OpenClaw-on-acpx migration. The plan's literal Step 9 (route OpenClaw history through the harness when the flag is on) was already a no-op after the Step 6 walkback — history is routed by source today. The actual blocker for Steps 10–13 was that legacy gateway-only agents (e.g. `main`, orphans from rolled-back creates) had no harness record, so they could never migrate to the harness path without breaking chat. `AgentHarnessService.reconcileWithGateway()` now lists every gateway agent and upserts a matching harness record for any that are missing. The pass runs lazily on first `listAgents()` call (memoized on success, retried on failure so a gateway-down boot doesn't permanently disable backfill). Verified end-to-end: the legacy `agent` agent now streams `text_delta` + `done(end_turn)` through `/agents/agent/chat`, with the bridge resolving to the gateway's `agent` record via the existing `agent:<name>:main` session-key format. After this, every OpenClaw agent surfaces as `source='agent-harness'` post-dedup, the legacy `useClawChatHistory` hook becomes unreachable for OpenClaw, and Steps 11–13 (delete legacy chat/history paths) are unblocked. * fix(agents): drop duplicate OpenClaw entry from NewAgentDialog adapter list The adapter Select hardcoded an `<SelectItem value="openclaw">OpenClaw</SelectItem>` on top of iterating `adapters`, which now includes OpenClaw post the catalog change. The dropdown rendered "OpenClaw" twice — once at the top, once at the bottom of the list. The literal was a pre-catalog artifact; removing it leaves a single OpenClaw entry sourced from the catalog. Routing into `handleOpenClawCreate` is unchanged because the value (`'openclaw'`) is identical either way. * fix(agents): always reconcile harness with gateway on list, just dedupe concurrent calls Memoizing the first successful reconcile meant new gateway agents (created via the legacy /claw/agents path or out-of-band CLI) never appeared in the harness until server restart. The Promise now serves as a concurrent-call dedupe only — cleared on settle — so every listAgents call picks up the current gateway state. Reconcile is one cheap idempotent CLI call. * chore(agents): remove dormant useAcpxForOpenClaw flag The flag was scaffolded in Step 6 but its routing effect was walked back the same day after it broke chat for legacy gateway-only agents. After the Step 9 backfill, every OpenClaw agent has a harness record and routes through the harness path purely from `source='agent-harness'` — no flag is consulted anywhere. Remove the dead storage item, hook, and stale comment. * refactor(agents): drop legacy /claw/agents/:id/history endpoint The harness /agents/:id/sessions/main/history endpoint replaced this once every OpenClaw agent got a harness record (Step 9 backfill). Routing is fully source-driven now, so the UI's useClawChatHistory hook is never enabled today — verified live: legacy URL returns 404, harness history hydrates correctly for the same agent. Removes the GET /claw/agents/:id/history route, OpenClawService's getAgentHistoryPage method plus its cursor/limit helpers and the history-only types it owned (BrowserOSOpenClawHistoryPageResponse, HistoryPageInput, normalizeHistoryLimit, encodeHistoryCursor, decodeHistoryCursor, jsonlEventsToHistoryItems), and the route + service tests that covered the dropped endpoint. OpenClawJsonlReader stays alive — still feeds /claw/dashboard, /claw/agents/:id/sessions, and the boot-time clawSession seed. Removing those is its own follow-up since the dashboard would need a harness-side replacement first. * feat(agents): wire image attachments through the harness ACP path Composer attachments now flow into the ACP `prompt` request as spec-compliant `image` content blocks alongside the user's text. End to end: composer → chatWithHarnessAgent({attachments}) → POST /agents/:id/chat {message, attachments} → parseChatBody decodes data: URLs to {mediaType, base64} → AgentHarnessService.send forwards → AcpxRuntime.send forwards → acpx startTurn({attachments}) → ACP image blocks UI no longer disables the attach button on harness agents — the gating was just a placeholder before this commit landed. Verified end to end with a 1×1 red PNG against a Claude harness agent: model replies "Red." correctly. OpenClaw's `acp` bridge still drops image content blocks upstream (verified by the same probe — Kimi-k2p5 reports "I don't see an image"). That's an upstream openclaw limitation, not a harness-side gap; Claude/Codex agents work as advertised today. * chore(openclaw): delete OpenClawJsonlReader and JSONL-backed routes * chore(openclaw): remove legacy /claw/agents/:id/chat and /queue routes * chore(agents): collapse chat panel to harness-only path * feat(agents): route OpenClaw image turns through the gateway HTTP client The OpenClaw `acp` bridge silently drops ACP `image` content blocks (verified during dogfood — model says "I don't see an image"). When the user attaches images to an OpenClaw agent, the harness now diverts that turn to the gateway's HTTP `/v1/chat/completions` endpoint, which accepts OpenAI-style `image_url` parts and forwards them natively to the provider. - New `OpenClawGatewayChatClient` translates an OpenAI streaming response into the same `AgentStreamEvent` shape the rest of the harness already consumes, so the chat panel renders identically whether a turn went through ACP or the gateway carve-out. - `AcpxRuntime.send` forks at the top: openclaw + any image attachment + a wired gateway client → `sendOpenclawViaGateway`. Other turns (text-only openclaw, claude, codex) take the existing ACP path unchanged. - The diverted path reads the prior turn history from the acpx session record so context is preserved, builds the OpenAI multimodal user message with text + image_url parts, and pumps the gateway SSE back to the caller through a tee that accumulates the assistant text. On natural completion, persists a synthetic user+assistant message pair to the acpx session record so reload shows the image turn in history. - Wired `OpenClawGatewayChatClient` into `AgentHarnessService` via `server.ts` (gateway port + token accessor, just like the existing `openclawGateway`). Persistence note: the acpx record requires User messages to carry an `id` and Agent messages to carry `tool_results` — without them the record fails to round-trip through `parseSessionRecord`. The persist helper now sets both. Limitation by design: image recognition only works if the OpenClaw agent's provider supports vision (e.g. Claude-via-OpenClaw, GPT-4o). The pipeline routes images correctly to the provider regardless; text-only providers like Kimi-k2p5 will reply "I don't see an image" because the model itself has no vision capability — that's a provider config issue, not a routing bug. The unit test asserts the image_url part is present in the OpenAI request the gateway client sends. The wider plan (background-resilient chat, queue, replay) remains in `plans/.../2026-04-29-1527-...-background-resilient-chat-and-image-uploads.md` as Phases 3–12; this commit ships only Phases 1–2. * feat(agents): validate inbound image attachments on /agents/:id/chat The harness chat body parser was accepting any mediaType and any dataUrl length. The composer enforces these caps client-side but the endpoint also serves direct curl/script callers, so the server has to defend itself. Restores the same caps the legacy /claw/agents/:id/chat parser had before it was deleted in the migration: - 10 attachments per message - 5 MB raw image bytes (≈ 6.7 MB once base64-encoded plus prefix) - PNG / JPEG / WebP / GIF only - Must start with `data:` Each violation returns 400 with a specific error message instead of silently dropping or forwarding the payload.	2026-04-29 16:37:03 +05:30
Nikhil	cb32b8191d	fix: show rich ACP harness history from ACPX (#852) ... * fix: load ACP harness history from ACPX * fix: address ACP history review comments	2026-04-28 16:40:22 -07:00
Nikhil	91d3285aa0	feat: add ACP agent harness (#849) ... * feat: add acp agent runtime spike * feat: add agent harness catalog * feat: persist harness agents in json * feat: persist agent transcripts * feat: route harness service through agent records * feat: expose generic agent harness routes * feat: add harness agent frontend api * feat: create harness agents from agents page * feat: chat with persisted harness agents * chore: remove obsolete agent profile spike * chore: self-review fixes * fix: combine openclaw and harness agents UI * refactor: split agents page components * fix: hide persisted harness turns	2026-04-28 15:29:38 -07:00
Dani Akash	ddbb2cf492	feat(agent): composer attachments + server-side outbound message queue (#826) ... * feat(agent): attach images and text files to chat messages Adds end-to-end support for image and text file attachments in the chat composer, with the staged files round-tripping through the OpenClaw gateway as OpenAI-compatible content blocks and persisting in the JSONL so they show up in the historical view. Server - HTTP client: new OpenClawChatContentPart union and a buildUserContent helper that emits multimodal content arrays when messageParts is supplied, falls back to the legacy string content otherwise. - Service: chatStream takes an optional messageParts array and forwards it; BrowserOSChatHistoryItem gains an attachments field. - JSONL reader: PiContentBlock learns the OpenAI image_url and Anthropic image source/data shapes; user messages now emit user.attachment events that the history mapper accumulates onto the next user item. - Route: validates an inbound attachments[] (kind/mime/size/count), inlines text-shaped files as <attachment> blocks in the message body, attaches images via image_url parts. Replaces the immediate 409 on active monitoring session with a 30s waitForSessionFree(agentId) wait (registry now exposes onSessionEnd) so cron/hook contention does not reject a user-chat send outright. Returns 503 if the wait times out. Client - New lib/attachments.ts: validateAttachment / compressImageIfNeeded (canvas downscale to 2048px long edge, JPEG 0.85 re-encode for >1.5 MB inputs) / stageAttachment / stageAttachments that produces the staged-attachment shape the composer renders and the payload the server accepts. - ConversationInput: drag-and-drop, paperclip button, clipboard paste, staged attachment chip strip with thumbnails for images and a paperclip+name chip for text files. Send button enables on either text or attachments. Drop-zone overlay during drag. - chatWithAgent forwards attachments[]; useAgentConversation.send accepts a SendInput shape and renders user attachments on the optimistic streaming turn via MessageAttachments / MessageAttachment. - ClawChatMessage groups historical attachment parts into a single MessageAttachments strip, ordered before reasoning/tools/text. - claw-chat-types adds an attachment ClawChatMessagePart variant; the history mapper emits attachment parts first and skips the text part when the user only sent media. - AgentCommandHome forwards the new SendInput shape — home composer drops attachments at the boundary in v1 (the conversation page is where staging is most useful; carrying bytes through the URL bar is not sensible). Limits: 10 attachments per message, 5 MB per image (post compression), 1 MB per text file, mime types png/jpeg/webp/gif and text/* + application/json. PDFs and other binaries are deferred to v2. * feat(agent): outbound message queue for chats while agent is mid-turn Lets users keep typing and submitting messages while the agent is still streaming a previous turn. Each press is appended to a single-flight queue and dispatched as soon as `streaming` flips false; the queued state renders as a strip above the composer so the user sees what's pending vs. what's already sending. - New `useOutboundQueue` hook owns the queue, the worker effect, and cancel/retry actions. Single-flight by design — a re-entrancy ref guard prevents two simultaneous dispatches when `streaming` flickers. - Composer (`ConversationInput`) accepts optional `outboundQueue`, `onCancelQueued`, `onRetryQueued` props. When the queue is provided the send-button gate stops blocking on `streaming`; the spinner stays as the visual cue that the agent is still busy. Legacy direct-send callers keep the old streaming-blocks-send semantic. - Renders an OutboundQueueStrip above the staged-attachment strip with per-item status (queued / sending / failed), a cancel button on queued items, and retry + discard on failed items. - AgentCommandConversation wires `onSend` to `queue.enqueue` and routes the home composer's `?q=` initial-message handoff through the queue too, so it inherits the same single-flight serialization. The server-side `waitForSessionFree` (added with attachments) and this client-side queue together cover both contention sources: cron / hook turns and back-to-back user sends. Persistence across reloads is intentionally out of scope for v1 — losing the queue on extension reload is documented as a known limitation. * feat(server): server-side outbound message queue Replaces the client-only React-state queue from 123ef21d with a proper server-owned queue. Closing the tab is now safe — the server holds queued messages and dispatches them through the existing chatStream path the moment the agent's ClawSession status flips to idle. Server - New OutboundQueueService (apps/server/src/api/services/queue) — per agent FIFO, in-memory. Subscribes to ClawSession.onStateChange through OpenClawService.onAgentStatusChange, and dispatches via OpenClawService.chatStream so attachments / history / monitoring all behave identically to the existing /chat route. The worker drains the SSE response server-side so the gateway run finalizes cleanly even with no client connected. - Four new routes under /claw/agents/:id/queue: POST /queue enqueue DELETE /queue/:itemId cancel a queued item POST /queue/:itemId/retry re-queue a failed item GET /queue/stream SSE feed of the per-agent queue state. Validation reuses validateChatAttachments and buildMessagePartsFromAttachments from the existing chat route. - Singleton wired in apps/server/src/main.ts; shutdown on SIGTERM. - New OpenClawService.getAgentState getter for the queue worker's pre-dispatch sanity check. Client - useOutboundQueue rewritten as an SSE-backed projection over server state. Public API unchanged so the composer still works. - enqueue POSTs to /queue and shows an optimistic local entry until the server's SSE snapshot reflects it; local-only entries get a `local-` id prefix so cancel can short-circuit them without hitting the server. - AgentCommandConversation watches the queue for sending items dropping out and refetches history so the new assistant turn shows up in the conversation view (the server worker streams the dispatched turn into OpenClaw without exposing per-turn SSE to the client). Out of scope (documented in the plan as v2 follow-ups): disk persistence (server restart loses queue), per-turn live streaming of queued sends in the conversation view, and switching the underlying dispatch from /v1/chat/completions to the chat.send RPC (which would also fix the multimodal attachment routing problem). * fix(server): outbound queue must reuse existing session, not spawn UUIDs The queue worker was generating a fresh randomUUID() as the sessionKey when the queued item didn't carry one — and the client wasn't sending one. Result: every queued message kicked off a brand-new OpenClaw session, orphaning the user's active conversation behind the new "most recent" entry in sessions.json. The history endpoint then resolved to the orphan and the chat appeared to disappear. Fix is layered: - Client (useOutboundQueue): forward the current resolvedSessionKey in the POST /queue body so every queued message targets the same conversation the user is viewing. AgentCommandConversation passes resolvedSessionKey into the hook. - Server (OutboundQueueService): the worker now resolves to the agent's existing user-chat session when no sessionKey is provided on the queued item, via OpenClawService.resolveAgentSession. UUID fallback is now reserved for the first-ever message on a brand new agent — same semantic the existing /chat route has implicitly through the catalog of historical sessions. No JSONL data was lost by the original bug (the prior conversations are intact on disk); the orphan sessions just shadowed the original in sessions.json. * fix(agent,server): address PR review feedback for chat queue - Tighten image data URL cap to base64-aware ~6.7 MB (was ~7.5 MB through `MAX_IMAGE_BYTES * 2`). - Forward chat history from useOutboundQueue.enqueue so queued sends preserve conversation context like direct sends do. - Match local attachment previews to server snapshots by id (not by message text), and prune the preview map as items drain. - Pass an AbortSignal into chatStream so a queue shutdown cancels the initial OpenClaw handshake, not just the SSE drain loop. - Track previously gitignored apps/agent/lib/attachments.ts (was caught by global lib/ ignore) so CI typecheck can resolve @/lib/attachments. - Update server-api openclaw route tests to the new chatStream signature and the waitForSessionFree-based busy-agent path. * fix(agent): dedupe optimistic queue entries for text-only sends The localId↔serverId map was only populated when the message had attachments, so plain-text sends left the optimistic local entry in place after the server snapshot arrived — the user saw the same message rendered twice in the queue strip. * fix(agent): prune optimistic queue entry on POST ack, not just SSE The server broadcasts the new queue snapshot before its POST response returns, so the SSE handler often runs first — at that point the localId↔serverId map has no entry for the new server id yet, so the SSE-based dedupe path can't drop the optimistic local entry. Pruning on POST success closes the race deterministically. * fix(agent): hand off optimistic queue entry without a render gap Pruning the local entry on POST success only worked when the SSE snapshot had already overwritten it; if the POST response landed first, the optimistic row disappeared for a frame before the SSE snapshot brought back the server-keyed row, producing a visible flicker. Gate the POST-side prune on the SSE snapshot already carrying the server id, and rely on the SSE-based dedupe (now guaranteed to find the localId↔serverId link in the map) to clean up when SSE arrives later. * fix(agent,server): client-generated queue id eliminates render flicker The server used to assign its own UUID when an item was enqueued, so the optimistic client row carried a `local-` id while the SSE snapshot carried a server UUID — the client had to wait for the POST response to learn the mapping before it could dedupe, and during that window both rows rendered. Now the browser generates the id, sends it in the POST body, and the server uses it verbatim (falling back to a fresh UUID only if the id collides with an existing item). The client collapses to a single id-keyed list, so the optimistic row and the SSE row reconcile on the same key from the very first render.	2026-04-27 21:31:03 +05:30
Dani Akash	711934555d	feat(agent): enrich chat UI with tool activity, reasoning duration, and cost (#825) ... * feat: pass per-turn cost and token data through chat history items - Add costUsd, tokensIn, tokensOut to BrowserOSChatHistoryItem (server) - Pass through from JSONL agent.message events in jsonlEventsToHistoryItems() - Add same fields to client-side BrowserOSChatHistoryItem and ClawChatMessage - Map cost/token data in mapHistoryItemToClawMessage() Data flows: JSONL message.usage → server history item → API response → client ClawChatMessage. Available for rendering in ClawChatMessage component (message toolbar, cost badges). * feat: add message toolbar with copy button and per-turn cost display Add MessageToolbar to historical assistant messages in ClawChatMessage: - Copy button copies message text to clipboard via MessageAction - Per-turn token count (22.7K → 238) and cost ($0.003) shown as muted tabular-nums text on the right side of the toolbar - Toolbar appears on hover (opacity transition via group-hover) - Only shown when the message has text content - Cost/token display only shown when data is available from JSONL * fix: toolbar only on assistant messages, always visible, cost only - Only render toolbar on assistant messages (not user messages) - Remove hover-only opacity — toolbar is always visible - Remove token counts (22.7K → 238 is meaningless to users) - Show only cost as a budget signal ($0.003) * feat: group all tool activity into single Task collapsible per turn Replace flat tool rows with a single ai-elements Task collapsible per assistant turn that lists every tool/MCP call in sequence. Live streaming (ConversationMessage): - Aggregate all tool-batch parts into one Task - Title: "Working… (N actions)" while running, "Agent activity (N actions)" when done - Default open while turn is in progress - Wrench icon in trigger Historical (ClawChatMessage): - Group all tool-call parts into one Task - Title includes failed count if any tools errored - Default collapsed — expandable on click - Tool name + status icon + error text per row Both views show one clean collapsible per turn instead of N individual tool cards. Collapsed reads "5 actions"; expanded shows the timeline. * feat: include tool calls in chat history responses Server: jsonlEventsToHistoryItems() now walks ALL events (not just messages) and pairs agent.tool_use with agent.tool_result by toolCallId. The resulting tool call list is attached to the next assistant text message as toolCalls[]. Each entry includes status, input arguments, output text, error string, and duration computed from event timestamps. Client: - BrowserOSChatHistoryItem gets optional toolCalls field - Tool-call message part type gets durationMs field - mapHistoryItemToClawMessage() emits tool-call parts BEFORE the text part (the order the agent produced them) - ClawChatMessage Task view now shows tool duration in seconds Result: historical messages now display the full tool activity timeline grouped into the single Task collapsible per turn (designed in step 3), instead of showing only the final text response. * feat: render activity rows as human verbs sourced from tool registry Tool calls in the chat activity view now read as sentences: "Opened tab · news.ycombinator.com" instead of "browseros__new_page". Server (tool-label-registry.ts): - Curated verb override map for ~70 BrowserOS first-party tools - Per-tool subject extractors that pull the meaningful argument from input (URL → host, query → quoted, element → ID, etc.) - Generic fallback humanizes snake_case for any unmapped tool - Strips MCP namespace prefixes (browseros__, mcp_) Server (openclaw-service.ts): - jsonlEventsToHistoryItems calls buildToolLabel for each tool_use, attaches label and subject to the BrowserOSChatHistoryToolCall Client: - Mirrored label module at lib/tool-labels.ts - useAgentConversation tool-start handler computes label/subject from the SSE tool args - ClawChatMessage and ConversationMessage render label · subject with foreground/muted styling, no font-mono - ToolEntry, BrowserOSChatHistoryToolCall, and tool-call message part types all carry label and optional subject * fix: drop meaningless tab N subject from page-read tool rows Page IDs are internal numbers, not URLs. 'Took screenshot · tab 4' tells the user nothing. Removed subject extractors for take_snapshot, take_enhanced_snapshot, get_page_content, get_page_links, get_dom, and take_screenshot. The verb alone is the right signal. * fix: gate initial loading on historyQuery.isFetched not isLoading The session and history queries are sequential: the history query is disabled until session resolves. After session resolves, there's a render frame where historyQuery.isLoading is still false (the query hasn't been kicked off yet). isInitialLoading flipped to false during that window, exposing an empty chat shell with just Task collapsibles and copy buttons before the messages filled in. Switching the guard to isFetched closes that window — the loading state stays true until the first history fetch actually completes. * fix: render historical messages immediately instead of through Streamdown's idle-callback debounce Streamdown defaults to mode="streaming" which uses requestIdleCallback (300ms debounce, 500ms idle timeout) and lazy/Suspense to optimize for token-by-token live streams. For finalized historical messages this caused tool collapsibles and copy buttons to paint while text bodies stayed blank for ~300-500ms after load. Pass mode="static" + parseIncompleteMarkdown=false on the historical MessageResponse so completed text paints in the same frame as the surrounding chrome. Live streaming turns still use the default streaming mode. Also collapse the redundant /agents/:id/session round-trip into the existing /history endpoint (server already resolves the most recent user-chat session when sessionKey is omitted) and tighten the initial-loading gate to stay true across the render frame where the query is enabled but hasn't started fetching. * feat: surface thinking duration on historical reasoning collapsibles Server accumulates agent.thinking events per turn from JSONL and attaches a single reasoning block (joined text + durationMs from first thinking event to the closing agent.message) on each assistant history item. Reasoning buffer resets on user.message alongside the tool-call buffer. Client mirrors the type, emits the reasoning part before tool calls in mapHistoryItemToClawMessage (chronological: think → act → answer), and passes duration in seconds to <Reasoning> so the trigger reads "Thought for N seconds" instead of just "Thinking" on collapsed historical turns. * fix: read thinking blocks from the correct JSONL field name OpenClaw stores reasoning blocks as {type:'thinking', thinking:'...'} but the JSONL parser was reading block.text, so every thinking event was silently dropped before it ever reached jsonlEventsToHistoryItems. As a result the reasoning field on history items was always empty even though the new accumulator was wired up correctly. Also guard the client mapping: when durationMs is 0 (think + answer emitted in the same JSONL line, no real elapsed wall-clock) pass undefined to <Reasoning> so it renders the static "Thinking" trigger instead of the streaming shimmer / "Thought for 0 seconds". * fix: reset reasoning buffer on discarded turns and drop dead session hook Two cleanups from PR review: 1. jsonlEventsToHistoryItems: when an agent.message is discarded (the "[Chat messages since your last reply" wrapper without a current-message marker) the tool buffers were already reset but the reasoning buffer was not. Accumulated thinking from the discarded turn would bleed onto the next assistant message. Reset pendingReasoningTexts and pendingReasoningFirstAt alongside the tool buffers. 2. useClawAgentSession, the AgentSessionResponse type, and the unused session entry in CLAW_CHAT_QUERY_KEYS became dead code after the session round-trip was folded into the history endpoint. Removed.	2026-04-27 18:29:15 +05:30
Dani Akash	0035893f33	feat: dashboard API, JSONL reader, and OpenClaw observer for enriched home page (#810) ... * feat: draft agent chat ui exploration * feat: refine agent chat ui draft * feat: remove outer frame from agent chat workspace * fix: offset agent chat for app sidebar * fix: simplify agent conversation shell * fix: remove redundant chat header actions * fix: unify agent conversation headers * fix: tighten agent chat spacing * fix: bound agent chat composer height * fix: remove agent chat page inset * fix: align agent header height with sidepanel * fix: center agent composer resting state * fix: anchor multiline composer controls * fix: remove focus grid from agent home * fix: remove redundant agent home header * fix: constrain home agent composer * fix: match home composer default posture * feat: add openclaw chat history APIs * feat: add claw chat history hydration * fix: stabilize claw chat viewport layout * fix: use conversation scroll base for claw chat * refactor: split claw chat controller responsibilities * fix: keep active agent turns in memory * fix: normalize openclaw chat sessions * refactor: use HTTP client for agent history instead of CLI client Replace the CLI-based getChatHistory() call in getAgentHistoryPage() with the HTTP client's getSessionHistory() from PR #795. This uses the direct HTTP transport to OpenClaw's /sessions/<key>/history endpoint instead of shelling out through the CLI. - Add filterHttpSessionHistoryMessages() for flat-string content format - Add normalizeHttpHistoryMessages() for OpenClawSessionHistoryMessage shape - Update getAgentHistoryPage() to call getSessionHistory() via httpClient - Remove unused getChatHistory(), filterOpenClawSystemMessages(), normalizeChatHistoryMessages(), and getTextContent() - Update test mocks from cliClient.getChatHistory to httpClient.getSessionHistory - Update MutableOpenClawService type: chatClient -> httpClient * fix: fetch all session messages by iterating OpenClaw pagination OpenClaw's HTTP history endpoint returns a limited page by default. When called without a limit, only the first ~27 messages were returned, causing all newer conversation messages to be silently dropped. Add fetchAllSessionMessages() that iterates through OpenClaw's cursor- based pagination (200 messages per page) until hasMore is false, then feeds the complete message list into the existing BrowserOS normalization and in-memory pagination layer. * refactor: migrate chat history from HTTP gateway to direct JSONL file reads Replace the HTTP-based chat history pipeline (BrowserOS server → OpenClaw gateway /sessions/:key/history pagination loop) with direct JSONL file reads from the host filesystem via Lima's virtiofs mount. - Add OpenClawJsonlReader that reads session JSONL files directly from ~/.browseros/vm/openclaw/.openclaw/agents/<id>/sessions/ - Replace fetchAllSessionMessages() HTTP pagination with single file read - Replace CLI-based listSessions() with sessions.json file reads - Make listSessions, resolveAgentSession, getAgentHistoryPage synchronous - Remove unused toBrowserOSSession, filterHttpSessionHistoryMessages, normalizeHttpHistoryMessages helpers - Update route handlers to drop unnecessary async/await - Update tests to use temp JSONL files instead of mocked HTTP/CLI clients * fix: restore async route handlers for test compatibility with mocked service * fix: address review feedback — path traversal guard, lazy reader, exists flag - Add safePath() to OpenClawJsonlReader that validates resolved paths stay within stateRoot, preventing path traversal via crafted agentId values - Use lazy initialization for jsonlReader (nulled on rebuildRuntimeClients) instead of creating a new instance per property access - Return exists: false from resolveSpecificAgentSession when no session matches instead of fabricating a ghost session with sessionId: '' * feat: add dashboard API and enrich home page agent cards Server: - Add summarizeToolActivity() that converts tool events into natural language descriptions ("Browsed 3 pages, took 2 screenshots") - Add getDashboard() to OpenClawService that aggregates per-agent stats from JSONL: latest message, activity summary, cost, session count - Add GET /claw/dashboard endpoint Client: - Add useAgentDashboard() React Query hook (10s refetch, 5s stale) - Rewrite useAgentCardData from async IndexedDB hook to pure buildAgentCardData() function merging agent entries with dashboard data - Add activity summary and cost to AgentCardExpanded footer - Add activitySummary and costUsd fields to AgentCardData type - Remove IndexedDB dependency from the home page * feat: add OpenClawObserver for real-time per-agent status via gateway WS - Add OpenClawObserver that connects to the OpenClaw gateway WebSocket control plane and subscribes to chat broadcast events - Track per-agent status in real time: working (streaming), idle (turn complete), error (run failed), with current tool name - Auto-connect when gateway control plane becomes available, auto- reconnect on disconnect with 5s backoff - Disconnect observer on stop/shutdown - Wire live status + currentTool into getDashboard() response - Update client: AgentOverview includes status + currentTool, card shows spinning loader + tool name when agent is working - Status resolution: per-agent WS status takes precedence over gateway- level status for working/error states * feat: add SSE dashboard stream for real-time agent status on home page Server: - Add GET /claw/dashboard/stream SSE endpoint that sends an initial snapshot then pushes per-agent status events as they arrive from the OpenClaw observer - Add onAgentStatusChange() to OpenClawService exposing the observer's listener for the route layer - Heartbeat every 15s to keep connections alive Client: - useAgentDashboard() now subscribes to EventSource at /claw/dashboard/stream - SSE snapshot event hydrates the React Query cache immediately - SSE status events patch individual agent status + currentTool in the cache without refetching — agent cards update instantly - Polling fallback raised to 30s since SSE handles real-time * fix: observer WS handshake — wait for challenge before sending connect The OpenClaw gateway sends a connect.challenge event before accepting the connect request. The observer was sending the connect request on ws.open which raced with the challenge. Now waits for the challenge event before sending the handshake. Also add dangerouslyDisableDeviceAuth to the gateway setup config batch so the observer can connect without device identity on new installs. * fix: JSONL reader falls back to most recent file when sessions.json is stale OpenClaw's sessions.json can record a Pi session ID that doesn't match the actual JSONL filename on disk. This happens after context compaction or session restart — the JSONL file gets a new UUID but sessions.json keeps the old one. Previously this caused history to silently disappear (the reader tried to open a non-existent file and returned empty). Now resolveJsonlPath() checks if the mapped file exists and, when it doesn't, scans the sessions directory for the most recently modified .jsonl file as a fallback. * feat: add ClawSession state machine for reliable per-agent status The OpenClawObserver only knows about status changes it witnesses via WS events. If an agent was already running when the observer connected, or after a reconnect, statuses were stuck at "unknown". ClawSession is an in-memory state machine that solves this: 1. Seeds from JSONL on first control plane call — reads the latest events for each agent and infers working/idle. A session is "working" if the last event is a user.message with no subsequent agent.message, or an agent.tool_use with no matching agent.tool_result. 2. Receives live transitions from the WS observer — the observer now delegates all state management to ClawSession instead of maintaining its own status map. 3. Applies a 5-minute staleness threshold — if the last JSONL event is older than 5 minutes, assume idle (handles agent crashes). Consumers (SSE stream, dashboard endpoint) read from ClawSession and get correct state from the first call — no "unknown" period. * fix: remove staleTime so dashboard refetches on every mount * fix: reset stale working status on WS disconnect, eliminate redundant JSONL reads - Observer resets all "working" agents to "unknown" when the WS closes, preventing agents from appearing stuck as Working indefinitely after a gateway restart. ClawSession re-seeds correct state on reconnect. - getDashboard() now derives latestAgentMessage and cost from the already-loaded events array for the latest session instead of calling latestAgentMessage() and getSessionStats() which each re-read the same JSONL file. Reduces file reads from 3x to 1x per agent.	2026-04-25 19:03:03 +05:30
Dani Akash	752f42d1fe	refactor: migrate chat history to direct JSONL file reads via Lima filesystem (#808) ... * feat: draft agent chat ui exploration * feat: refine agent chat ui draft * feat: remove outer frame from agent chat workspace * fix: offset agent chat for app sidebar * fix: simplify agent conversation shell * fix: remove redundant chat header actions * fix: unify agent conversation headers * fix: tighten agent chat spacing * fix: bound agent chat composer height * fix: remove agent chat page inset * fix: align agent header height with sidepanel * fix: center agent composer resting state * fix: anchor multiline composer controls * fix: remove focus grid from agent home * fix: remove redundant agent home header * fix: constrain home agent composer * fix: match home composer default posture * feat: add openclaw chat history APIs * feat: add claw chat history hydration * fix: stabilize claw chat viewport layout * fix: use conversation scroll base for claw chat * refactor: split claw chat controller responsibilities * fix: keep active agent turns in memory * fix: normalize openclaw chat sessions * refactor: use HTTP client for agent history instead of CLI client Replace the CLI-based getChatHistory() call in getAgentHistoryPage() with the HTTP client's getSessionHistory() from PR #795. This uses the direct HTTP transport to OpenClaw's /sessions/<key>/history endpoint instead of shelling out through the CLI. - Add filterHttpSessionHistoryMessages() for flat-string content format - Add normalizeHttpHistoryMessages() for OpenClawSessionHistoryMessage shape - Update getAgentHistoryPage() to call getSessionHistory() via httpClient - Remove unused getChatHistory(), filterOpenClawSystemMessages(), normalizeChatHistoryMessages(), and getTextContent() - Update test mocks from cliClient.getChatHistory to httpClient.getSessionHistory - Update MutableOpenClawService type: chatClient -> httpClient * fix: fetch all session messages by iterating OpenClaw pagination OpenClaw's HTTP history endpoint returns a limited page by default. When called without a limit, only the first ~27 messages were returned, causing all newer conversation messages to be silently dropped. Add fetchAllSessionMessages() that iterates through OpenClaw's cursor- based pagination (200 messages per page) until hasMore is false, then feeds the complete message list into the existing BrowserOS normalization and in-memory pagination layer. * refactor: migrate chat history from HTTP gateway to direct JSONL file reads Replace the HTTP-based chat history pipeline (BrowserOS server → OpenClaw gateway /sessions/:key/history pagination loop) with direct JSONL file reads from the host filesystem via Lima's virtiofs mount. - Add OpenClawJsonlReader that reads session JSONL files directly from ~/.browseros/vm/openclaw/.openclaw/agents/<id>/sessions/ - Replace fetchAllSessionMessages() HTTP pagination with single file read - Replace CLI-based listSessions() with sessions.json file reads - Make listSessions, resolveAgentSession, getAgentHistoryPage synchronous - Remove unused toBrowserOSSession, filterHttpSessionHistoryMessages, normalizeHttpHistoryMessages helpers - Update route handlers to drop unnecessary async/await - Update tests to use temp JSONL files instead of mocked HTTP/CLI clients * fix: restore async route handlers for test compatibility with mocked service * fix: address review feedback — path traversal guard, lazy reader, exists flag - Add safePath() to OpenClawJsonlReader that validates resolved paths stay within stateRoot, preventing path traversal via crafted agentId values - Use lazy initialization for jsonlReader (nulled on rebuildRuntimeClients) instead of creating a new instance per property access - Return exists: false from resolveSpecificAgentSession when no session matches instead of fabricating a ghost session with sessionId: ''	2026-04-24 13:19:46 +05:30
Nikhil	6e37742a5a	feat: reuse agent command chat for agents page (#803)	2026-04-23 17:09:49 -07:00
Dani Akash	e80ec467f4	feat: wire lazy monitoring to OpenClaw chat handoff (#768) ... * feat: add lazy monitoring substrate * feat: wire lazy monitoring to openclaw chat handoff * test: cover openclaw chat history handoff * fix: reject concurrent monitored chats	2026-04-20 21:52:03 +05:30
Dani Akash	0c96002cf5	fix: complete openclaw gateway recovery UX (#703) ... * fix: complete openclaw gateway recovery ui * fix: guard unknown gateway ui state * fix: guard unknown openclaw status badge	2026-04-14 18:22:47 +05:30
Nikhil	ce7c209ba6	feat: add OpenClaw agent command center and terminal (#692) ... * feat: agent command center new tab with OpenClaw conversation history * feat: add web terminal for Podman container shell access * feat: align agent command center with new tab * fix: simplify agent command center styling * style: polish agent terminal layout and theming * style: simplify agent terminal styling * fix: address PR review comments for OpenClaw routes * fix: handle OpenClaw client start and error states * fix: resolve remaining OpenClaw review comments	2026-04-13 17:06:48 -07:00