mirror of
https://github.com/browseros-ai/BrowserOS.git
synced 2026-05-18 11:06:19 +00:00
dev
164 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Nikhil
|
a59f96f657 | test(agent): tighten claude acp command assertion (#1022) | ||
|
Nikhil
|
4f3d83c1ff | chore: server update (#1021) | ||
|
Nikhil
|
4b2e887fbb | fix(agent): resolve claude codex acp commands (#1020) | ||
|
Nikhil
|
d097aa5648 | chore: bump version (#1012) | ||
|
Nikhil
|
2bd6c18dca |
feat: add window visibility tool (#1011)
* chore: update protocol json * feat: add window visibility tool * fix: address window visibility review comments |
||
|
Nikhil
|
628586ce51 | fix: remove unsupported CDP window visibility tool (#1007) | ||
|
shivammittal274
|
0026e7f41f |
feat(mcp): X-BrowserOS-Default-Window-Id header + set_window_visibility tool (#1004)
* feat(mcp): X-BrowserOS-Default-Window-Id header + set_window_visibility tool
Two related additions for host applications that want to deterministically
bind every browser tool call in an MCP session to a specific window.
1. Header `X-BrowserOS-Default-Window-Id` is honored in
`register-mcp.ts`: before `executeTool` runs, the handler wrap reads
the tool's zod input schema and, when it has a `windowId` field and
`args.windowId` is undefined, injects the header value. Schema-driven
— covers today's new_page / new_hidden_page / show_page / move_page,
and any future tool with a windowId field, with no per-tool list to
maintain. The agent's explicit `args.windowId` still wins when set.
2. New tool `set_window_visibility {windowId, visible}` — symmetric
show/hide on existing windows, delegating to
`cdp.Browser.showWindow`/`hideWindow`. Note: those CDP methods are in
the inspector_protocol_config include list and the generated client
types but the C++ implementation isn't wired yet (`is_hidden_` on
Browser is const). The tool returns the runtime error verbatim so
the host can degrade gracefully; once the Chromium patch lands, the
tool starts working with no changes here.
Companion changes on the agent-company side (separate PR) use the header
to per-thread-bind windows + PATCH /threads/:id browserVisibility calls
set_window_visibility on toggle.
* fix(mcp): require integer in X-BrowserOS-Default-Window-Id parser
Greptile flagged: `Number.isFinite` lets non-integer floats through
(e.g. header value `"1.5"` parses to `1.5`), and that float would be
injected as `args.windowId` into tools that forward it to CDP — which
expects an integer windowId and rejects with an opaque protocol error.
Swap to `Number.isInteger` to reject at the parse boundary instead.
|
||
|
Nikhil
|
0be59dccdd |
fix: revert recent agent/server changes (#995)
* Revert "fix(server): tolerate existing workspace dirs" This reverts commit |
||
|
shivammittal274
|
dad2331448 | refactor(agent): clean up hermes adapter structure (#994) | ||
|
Nikhil
|
d7e1125db3 |
fix(server): tolerate existing workspace dirs
Fixes #974 |
||
|
Nikhil
|
8b6483a633 |
fix(server): support Gemini computer use requests
Fixes #148 |
||
|
Nikhil
|
f54eff4543 |
feat(agent): add reset controls for sessions and memory
Fixes #418 |
||
|
Dani Akash
|
4e405681a7 |
feat(container): richen ManagedContainer — isImageCurrent + logs + sibling-exec (#968)
* feat(container): add isImageCurrent + getLogs + tailLogs + runOneShot to ManagedContainer
Four base-class additions ahead of the OpenClaw runtime migration so
the upcoming subclass doesn't have to re-implement them:
- isImageCurrent() — pure predicate comparing the existing container's
image ref to descriptor.defaultImage. Treats SHA-pinned variants as
matches. start() is unchanged; subclasses + service layers compose
the predicate where they want short-circuit behaviour.
- getLogs(tail) and tailLogs(onLine) — generic log primitives, thin
pass-throughs to ContainerCli.
- runOneShot(argv, opts) — sibling-container helper that spawns a
<name>-setup container with the same image+mounts+env (no ports/
health/restart), runs argv, force-removes after. Includes the
retry-on-name-collision behaviour previously bespoke to OpenClaw.
Hermes inherits unused surface only — no behavioural change. The
in-flight base-class tests cover all four primitives.
* fix(container): tighten getLogs error path + close runOneShot timeout-onLog leak; trim docstrings
- getLogs now distinguishes a missing container (returns []) from
other CLI failures (throws). Previously nerdctl's stderr ("Error:
no such container: …") leaked into the lines array as if it were
log output. isNoSuchContainer is exported from container-cli to
share the predicate.
- runWithOptionalTimeout wraps the caller's onLog so post-timeout
lines from the abandoned runCommand promise become no-ops; before
this, callers could see onLog fire after runOneShot had already
rejected, hitting state the caller may have torn down on the
timeout error.
- Tightens the new docstrings to one short line per the project
convention; drops a restating comment in the test file.
|
||
|
Dani Akash
|
b445615d61 |
refactor(claude+codex): migrate onto HostProcessAgentRuntime; collapse adapter-health (#967)
* feat(runtime): add ClaudeRuntime + CodexRuntime + factories * refactor(host-adapters): switch wire-up + dispatch + health to runtime registry main.ts registers ClaudeRuntime + CodexRuntime alongside Hermes. ACP runtime resolves all three via the registry; legacy host-process spawn is preserved as a fallback so unit tests that don't bootstrap runtimes keep working. AdapterHealthChecker now reads runtime snapshots through the registry — the embedded execAsync probe, ADAPTER_HEALTH_COMMANDS table, and friendlyProbeFailure mapper delete. As a side-effect this also fixes the Hermes "Unavailable" chip (Hermes was missing from ADAPTER_HEALTH_COMMANDS). Drops the standalone claude-code/prepare.ts and codex/prepare.ts modules (their bodies are exported from the runtime files now). * test(runtime): cover ClaudeRuntime + CodexRuntime descriptor + prep + factory * fix(runtime): coalesce concurrent host-process probes; expose probedAt on snapshot * fix(runtime): preserve acpx-core npx-wrapped spawn for claude + codex The host-process runtimes were resolving the ACP spawn command through their own getAcpExecSpec, which returned argv [claude] / [codex] — bare binaries. acpx-core's built-in registry actually resolves these adapters to npx wrappers around the official ACP-aware packages (claude-agent-acp, codex-acp), and the package version range is owned by acpx-core. The bare-binary spawn would fail because either the binary is missing or doesn't speak ACP. Spawn dispatch now goes through registry.resolve() + wrapCommandWithEnv for claude/codex (matching pre-#967 behaviour). The runtime registrations still drive health probing and per-turn prep — only the spawn-command source-of-truth stays in acpx-core. Drops the misleading getAcpExecSpec from the host-process runtime classes. Regression test asserts the spawn command contains the npx package name (claude-agent-acp / codex-acp) for each adapter. |
||
|
Dani Akash
|
d68e8905fe |
refactor(hermes): migrate Hermes onto ContainerAgentRuntime (#965)
* feat(runtime): add HermesContainerRuntime + factory * refactor(hermes): switch wire-up + dispatch to runtime registry main.ts and the agent route stack now resolve Hermes through `AgentRuntimeRegistry`. Drops the `hermesGateway` plumbing chain (server.ts → routes → harness → AcpxRuntime), the `HermesGatewayAccessor` interface, and `resolveHermesAcpCommand`. Removes `HermesContainerService`, `HermesContainer`, and `prepareHermesContext`'s standalone module — their behaviour is now owned by `HermesContainerRuntime`. * test(runtime): cover HermesContainerRuntime descriptor + lifecycle + factory * test(runtime): move registry reset to afterEach to survive assertion failures |
||
|
Dani Akash
|
e89fccd997 |
feat(runtime): introduce AgentRuntime abstraction (types, interface, registry, abstract bases) (#964)
* feat(runtime): introduce AgentRuntime types + interface + registry Foundation for the unified agent-runtime abstraction. No adapter migrates yet; the existing acpx-runtime, per-adapter prepare modules, OpenClawService, HermesContainerService, and adapter-health.ts all keep working unchanged. This commit adds the data layer of the abstraction: - `RuntimeDescriptor` discriminates the two kinds we ship today (`'container'` | `'host-process'`). UI components route on this. - `RuntimeState` is the union of both kinds' states — container flow `not_installed → installing → installed → starting → running → stopped`, host flow `cli_missing | cli_present | cli_unhealthy`, plus the shared `errored` and `unsupported_platform` terminals. - `RuntimeStatusSnapshot` carries a single `isReady: boolean` so the harness has one bit to read before spawning turns. - `RuntimeAction` is a typed discriminated union — required args (e.g. `agentId` for `'reset-wipe-agent'`) are compile-time enforced, removing the previous footgun of optional args on a string-keyed dispatch. - `RuntimeCapability` lists every action a runtime can advertise; `getCapabilities()` is the single switchboard the UI uses to decide which buttons to render. `AgentRuntime` interface declares the contract every runtime implements: status snapshot + subscriber, capability list, `executeAction(action)`, `buildExecArgv(spec)`, and per-agent home dir. `prepareTurnContext` is intentionally absent until the first adapter migrates so callers can't depend on a method that has no implementation. `AgentRuntimeRegistry` is a small class + module-level singleton — adapters register themselves at boot, the harness/UI look up by `adapterId`. `resetAgentRuntimeRegistry()` is for tests only. Two error classes round it out: `ActionNotSupportedError` (capability gate, mapped to HTTP 405 in a later phase) and `RuntimeNotReadyError` (state gate at the runtime layer, distinct from the container-layer's `ContainerNotReadyError`). * feat(runtime): add ContainerAgentRuntime + HostProcessAgentRuntime abstract bases * test(runtime): cover state translation, action dispatch, registry * fix(runtime): gate host-process executeAction on capabilities; only stamp probe cache after probe resolves |
||
|
Dani Akash
|
805ae8e607 |
feat(server): ManagedContainer abstraction — Hermes readiness gate + ACP layering fix (#962)
* feat(container): add waitForContainerRunning primitive + typed error
Adds `ContainerCli.waitForContainerRunning(name, opts)` polling
`inspectContainer().running === true` until either the container
reports running or the timeout expires. Distinct from the existing
`waitForContainerNameRelease` (which waits for *deletion*).
Used by the upcoming managed-container layer between
`nerdctl create + start` and "container is ready for exec" so the
harness never spawns a turn against a half-started container —
which is the root cause of the silent first-turn failure on Hermes
today (`hermes-container.ts:130-160` returns immediately after
start).
Defaults sized for cold-start: 30s budget at 500ms cadence.
Throws `ContainerNotRunningError` (new, in `lib/vm/errors.ts`) on
timeout — distinct from `ContainerNameReleaseTimeoutError` so
callers can branch on "didn't come up" vs "didn't get cleaned up".
* feat(container): add ManagedContainer abstract base + state machine
Introduces the abstract base every container-backed agent adapter
will subclass. Owns the canonical state machine (not_installed |
installing | installed | starting | running | stopped | errored),
the lifecycle lock (per-process promise chain + cross-process file
lock), the gated `execute*` family, and the host↔container path
translator.
Subclasses provide only what's actually adapter-specific:
- `descriptor` (image, container name, supported platforms)
- `buildContainerSpec()` for the `nerdctl create` args
- `readinessProbe()` after the container reaches running
- `mountRoots()` for the path translator
Three execute methods, all sharing one invariant — every entry
point gates on state == running:
- `execProcess(spec)` spawns a long-lived child process via Bun,
waits through `starting` up to 60s, throws typed
`ContainerNotReadyError` if the container is not_installed /
stopped / errored / timed out.
- `execOneShot(spec)` is a buffered convenience wrapper.
- `buildExecArgv(spec)` is the pure builder for callers (acpx-core)
that need a shell-command string. Single source of truth for the
`env LIMA_HOME=… limactl shell <vm> -- nerdctl exec -i …` chain
that today's ACP runtime hand-rolls in two places (`acpx-runtime
.ts:780-820` and `:823-870`).
`reset(level)` is on the API surface but throws
`ResetNotSupportedError` so the next PR can wire soft / wipe-agent
/ hard without revving the abstract class.
Path translator uses lexical containment against declared mount
roots; the realpath-based symlink-escape check lives one layer up
(in the file-attribution code that already shipped) since the
translator itself never reads from disk.
* feat(container): HermesContainer subclass + wrapper-service bridge
`HermesContainer` (lib/container/managed/) is the first concrete
adapter on the new `ManagedContainer` base. Provides the four bits
that are actually adapter-specific:
- `descriptor`: image, container name, supported platforms,
readiness-probe tuning.
- `mountRoots()`: host↔container path mapping for the harness dir.
- `buildContainerSpec()`: nerdctl create args (env, mounts,
add-hosts, entrypoint override).
- `readinessProbe()`: execs `hermes --version` inside the
freshly-started container; bypasses the state gate via
`cli.exec` since we're in `starting`, not `running`, when the
probe runs.
`HermesContainerService` (api/services/hermes/) is rewritten as a
thin wrapper that delegates `prewarm` / `start` / `stop` /
`restart` / `shutdown` to the underlying `HermesContainer`. Public
surface is preserved so `main.ts`, `server.ts`, and
`agent-harness-service` compile unchanged in this PR; `getAccessor()`
still returns the structural `HermesAccessor` the ACP runtime
expects today (the runtime swap is the next commit). The wrapper
also exposes `getContainer(): HermesContainer | null` for callers
that want the richer surface.
The user-visible bug — Hermes silent first-turn failure — is fixed
as a side effect: `start()` now waits through
`cli.waitForContainerRunning` and runs the `hermes --version`
readiness probe before transitioning to `running`. Subsequent
chat turns are gated on the container actually being ready, not
just on `nerdctl create + start` having returned.
* feat(agent): ACP runtime spawns Hermes via ManagedContainer.buildExecArgv
`resolveHermesAcpCommand` no longer hand-rolls the
`env LIMA_HOME=… limactl shell <vm> -- nerdctl exec -i …` chain.
It now delegates to `gateway.buildExecArgv`, which the wrapper
service routes to the underlying `ManagedContainer.buildExecArgv`.
The structural `HermesGatewayAccessor` type gains one method
(`buildExecArgv`) — keeps the existing four getters so any
test/legacy caller still works. The wrapper's `getAccessor()`
delegates `buildExecArgv` to its `HermesContainer`. Net effect:
the `limactl shell ... -- nerdctl exec ...` argv chain has
exactly one owner (`ManagedContainer.buildExecArgv` in the
container layer) instead of being duplicated across `acpx-runtime`
and the now-deleted hand-built chain.
The OpenClaw branch (`resolveOpenclawAcpCommand`) is untouched —
its migration to ManagedContainer is a separate, larger PR that
also has to model the gateway / control-plane surfaces.
Tests: the existing acpx-runtime test suite expected the four
old getters; updated the Hermes-container fixture to also
provide `buildExecArgv` (mirrors the production builder inline so
the test stays independent of the production class wiring). All
320 server tests pass.
* test(container): managed-container + hermes-container coverage
20 cases across two files in `tests/lib/container/managed/`.
ManagedContainer base (14 cases):
- State machine: start() walks installing → starting → running;
probe-false lands errored with lastError populated; stop()
force-transitions to stopped even from errored.
- execProcess gating: rejects ContainerNotReadyError with
reason='not_installed' when never started; reason='errored'
when in errored state (preserving lastError); resolves once
state flips to running while waiting; reason='timeout' when
starting never resolves.
- buildExecArgv: snapshot test pinning the exact canonical
`env LIMA_HOME=… limactl shell <vm> -- nerdctl exec -i …` string
for the Hermes-shaped invocation; -e flags omitted when env is
empty.
- reset(level): throws ResetNotSupportedError for all three
levels (Phase 1 stub).
- Path translation: round-trip host ↔ container under a declared
mount; mount-root itself translates without suffix; rejects
PathOutsideMountsError for /etc/passwd / /proc/cpuinfo.
- subscribeState fires every transition, stops after unsubscribe.
HermesContainer subclass (6 cases):
- Descriptor declares adapterId='hermes', the canonical container
name, image, and darwin platform support.
- start() happy path reaches running + invokes the
`hermes --version` probe via cli.exec.
- Probe-non-zero start() lands errored with the right error.
- ContainerSpec built with idle entrypoint, harness bind-mount
(source = /mnt/browseros/vm/hermes/harness, target =
HERMES_CONTAINER_HARNESS_DIR), and host.containers.internal
add-host pointing at the VM gateway.
- toContainerPath maps host harness paths to /data/agents/harness.
- buildExecArgv produces the canonical Hermes ACP spawn string
with LIMA_HOME, container name, hermes binary path, and -e env.
Pre-existing test in tests/lib/container/container-cli.test.ts
(`waits until a container name is no longer resolvable`) flakes
under parallel test load on dev; passes solo. Last touched in
|
||
|
shivammittal274
|
7a2a8e09bc |
feat(agent): add Hermes as 4th ACPX adapter (in-VM container, BrowserOS-managed providers) (#956)
* feat(agent): add Hermes as a 4th ACPX adapter (Phase A)
Adds Hermes Agent (NousResearch/hermes-agent) as a host-process ACPX
adapter, mirroring the Claude Code pattern.
- agent-types.ts: extend AgentAdapter union with 'hermes'
- agent-catalog.ts: add Hermes catalog entry
- lib/agents/hermes/prepare.ts (new): minimal prepare using prepareBrowserosManagedContext
- acpx-agent-adapter.ts: register the adapter
- acpx-runtime.ts: add 'hermes' branch returning 'hermes acp' (host)
- AdapterIcon.tsx: add Hermes icon
- db schema + supporting frontend types/literals updated for the new adapter
Phase A scope: host-process only. Phase A.5 swaps to nerdctl exec
into a Hermes container.
OpenClaw is untouched. Verified by all 6 POC spikes
(plans/features/claude-browseros-hermes-poc/findings.md).
* fix(agent): address Hermes adapter review issues
- NewAgentDialog: add 'hermes' to onValueChange guard so the dropdown
option actually wires through onRuntimeChange/onHarnessAdapterChange
(was a no-op before — selecting Hermes silently kept previous value)
- tests/acpx-runtime: add coverage for the new 'hermes' registry branch
- tests/acpx-agent-adapter: fold hermes prepare test into existing file,
matching the pattern used for claude/codex/openclaw
- Delete tests/lib/agents/hermes-prepare.test.ts (now redundant)
- Reconcile install-mechanism comment between acpx-runtime.ts and
agent-catalog.ts
* fix(agent): make Hermes adapter actually work end-to-end
Two surgical fixes uncovered while running the Phase A smoke test
through the BrowserOS chat HTTP API:
1. lib/agents/hermes/prepare.ts — seed per-agent HERMES_HOME from
the user's global ~/.hermes/ on first use. ensureAgentHome only
writes SOUL.md and MEMORY.md; without seeding config.yaml, .env,
and auth.json, hermes acp comes up unconfigured and either hangs
or errors with "No LLM provider configured." Copy is idempotent
(skip if dest exists) so subsequent prepare calls don't clobber
per-agent edits.
2. lib/agents/acpx-runtime.ts — wrap the hermes spawn in
`bash -c "exec hermes acp | tee /dev/null"` to bridge Bun's
socketpair-based child stdio with Python's asyncio.connect_write_pipe
(which only drains correctly to a real pipe(2)). Without it, hermes'
stdout never reaches the harness — verified by inspecting hermes
process FDs: Bun gives the child unix sockets, asyncio queues writes
that never become readable on Bun's end. With tee in the middle,
hermes writes to a real pipe and tee bridges the bytes through the
socket. Verified 2026-05-06 against hermes-agent 0.12.0 on macOS
arm64 + Bun 1.3.6.
Smoke-test result with both fixes:
- ACP session created end-to-end
- BrowserOS MCP wired (96 browser tools registered with hermes)
- Reasoning + text streamed back through /agents/:id/sidepanel/chat
- Final stream: text-delta "PONG", finishReason "stop"
Updates the existing acpx-runtime test to assert the new spawn shape
(bash -c, tee /dev/null bridge) so the workaround can't silently regress.
* feat(agent): run Hermes adapter in Lima container (Phase A.5)
Move Hermes ACPX adapter from host-process spawn to running inside
docker.io/nousresearch/hermes-agent:v2026.4.30 in the existing
BrowserOS Lima VM, mirroring the OpenClaw container pattern.
Container lifecycle (api/services/hermes/hermes-container.ts):
- prewarm: ensure VM ready, pull image (or skip if already in
containerd), start an idle container with /bin/sh -c "exec sleep
infinity" so the harness can nerdctl exec into it per turn
- Tini bypassed — tini 0.19.0 in upstream image getopt-parses any
-x token even after PROGRAM, breaking /bin/sh -c
- --add-host host.containers.internal:<vm-gateway> so hermes inside
the container can reach the BrowserOS HTTP MCP endpoint
- Bind-mount <browserosDir>/vm/hermes/harness onto /data/agents/harness
so per-agent HERMES_HOME dirs are visible to the container
Spawn (acpx-runtime.ts):
- HermesGatewayAccessor interface (mirrors OpenclawGatewayAccessor)
- resolveHermesAcpCommand builds:
env LIMA_HOME=... limactl shell --workdir / browseros-vm --
nerdctl exec -i -e PYTHONUNBUFFERED=1 -e HERMES_HOME=... <container>
/opt/hermes/.venv/bin/hermes acp
- Absolute path /opt/hermes/.venv/bin/hermes (not bare "hermes") since
upstream image's PATH is set by its entrypoint script which we
override to keep the container idle
- Falls back to host-process spawn when no HermesGatewayAccessor wired
(test path / dev fallback)
- Drops the host-mode bash+tee workaround — limactl/SSH/nerdctl pipe
chain is sufficient for asyncio's pipe writer
MCP wiring:
- New PreparedAcpxAgentContext.browserosMcpHost field threads through
prepare → getRuntime → createBrowserosMcpServers
- Hermes prepare sets browserosMcpHost='host.containers.internal' so
the URL injected into newSession.mcpServers resolves from inside
the container; other adapters keep '127.0.0.1' default
Per-agent home (lib/agents/hermes/prepare.ts):
- HERMES_HOME points at /data/agents/harness/<agentId>/home (in-container)
- Host-side seedHermesHomeFromGlobal still copies ~/.hermes/{config.yaml,
.env, auth.json} into the per-agent home; the volume mount makes them
visible inside the container
- New api/services/hermes/hermes-paths.ts holds host/container path helpers
End-to-end smoke tests against the dev server (clean Lima state):
- Plain text: PONG round-trip via /sidepanel/chat ✓
- Multi-turn context: RUBY-7421 stored + recalled ✓
- Multi-agent isolation: agent 2 doesn't see agent 1's secret ✓
- MCP tool execution: mcp_browseros_browseros_info fires ✓
- Image attachment via /chat: model identifies "Red" from a 128x128 PNG ✓
- Concurrent turns + 409 attachUrl: full attach streams the in-flight
Pacific Ocean essay turn cleanly ✓
- Cancel midstream + recovery turn: ALIVE response ✓
- Persistence across server restart: agents survive ✓
Companion knowledge doc:
plans/features/claude-browseros-hermes-acp-knowledge.md
* feat(agent): per-agent provider/key for Hermes adapter
Lets users create multiple Hermes agents each with its own provider,
model, and API key. NewAgentDialog now shows provider/model/key fields
inline when 'Hermes' is selected. On submit, the harness writes the
per-agent <browserosDir>/vm/hermes/harness/<agentId>/home/{config.yaml,
.env} directly so the agent has the right config from turn 1 — no
dependency on the user having run `hermes setup` outside BrowserOS.
The existing seedHermesHomeFromGlobal flow remains as a fallback for
agents created without provider fields (e.g. via direct API or with
an existing ~/.hermes/ install).
Backend:
- shared/constants/hermes.ts: HERMES_SUPPORTED_PROVIDERS registry
(openrouter, anthropic, openai, custom — bedrock follow-up)
- api/services/hermes/hermes-paths.ts: writeHermesPerAgentProvider
- agent-harness-service: writes per-agent config.yaml + .env in
createAgent when adapter=hermes and apiKey present
- routes/agents.ts: relax modelId catalog validation for adapter=hermes
(catalog has empty models[] by design; per-agent modelId is free-form)
- tests/agent-harness-service: cover write + skip paths
Frontend:
- HermesProviderFields.tsx (new): provider dropdown, model field, API
key + optional baseUrl when provider=custom
- NewAgentDialog: render the new fields when adapter=hermes
- agents-page-actions: thread fields through createHarnessAgent
- AgentsPage / agent-harness-types: minor pass-through edits
Smoke-tested end-to-end against the dev server (clean Hermes per-agent
home, no ~/.hermes/ seed): create agent with apiKey + modelId, files
written at the per-agent path with mode 0600, first chat returns the
expected response, all without touching ~/.hermes/.
* feat(agent): source Hermes provider config from BrowserOS LLM providers
Replace the Hermes-specific provider/model/API-key form in New Agent
with a chooser that pulls from the same global LLM providers OpenClaw
uses (Settings → BrowserOS AI). Backend rejects creation with a 400
when the selected provider is missing required fields (apiKey, modelId,
plus baseUrl for openai-compatible) or is not in the Hermes-supported
set; the ~/.hermes/ fallback is removed so Hermes agents always carry
their own per-agent config.
|
||
|
shivammittal274
|
6f8da5b7fb |
refactor(openclaw): TKT-788 cleanup (relanded, openclaw-only) — bump image, lock no-auth, delete observer + image bypass (#954)
* refactor(openclaw): TKT-788 cleanup — bump image, lock no-auth, delete observer + image bypass Re-lands the openclaw-only changes from #934 (reverted in #953 because the original PR's working tree had stale rollback content for `packages/browseros/tools/patch/`). This commit is the same openclaw diff with zero changes outside `packages/browseros-agent/`. What changes (TKT-788 work-streams A + B + C): WS-A — bundled gateway no-auth: - Bump image from `ghcr.io/openclaw/openclaw:2026.4.12` to `ghcr.io/browseros-ai/openclaw:2026.5.2-browseros.1` (BrowserOS- pinned variant with the no-auth contract baked in). - Configure gateway with `auth.mode: 'none'`; remove the device-auth bootstrap dance that the older binary required. - Delete the per-call token plumbing the http-client / observer / chat- client carried (340 LOC). The harness still passes a stable token in headers for backwards-compat with code that hasn't been re-pointed yet, but it is no longer required by the gateway. WS-C — delete the image-attachment bypass: - The HTTP `/v1/chat/completions` carve-out for OpenClaw image turns is gone. Image attachments now ride through ACP as image content blocks (which acpx 0.6.x supports natively for openclaw, claude, codex). - Delete `openclaw-gateway-chat-client.ts` (211 LOC) and `image-turn.ts` (219 LOC). - Drop `maybeHandleTurn` from the `AcpxAgentAdapter` interface and the openclaw entry. `AcpxAdapterTurnInput` removed. - Drop the corresponding 'diverts OpenClaw image turns to the gateway chat client' test from `acpx-runtime.test.ts`. WS-B — replace the WS observer with harness events: - Delete `openclaw-observer.ts` (276 LOC) — no more parallel WS subscription, no more `new OpenClawObserver`, no more `ensureObserverConnected` / `observer.disconnect()` plumbing. - Wire `AgentHarnessService` to receive turn-lifecycle events from the runtime stream itself (`turnLifecycleListeners`) and feed ClawSession from those, preserving the dashboard SSE shape. Net: 314 insertions / 1144 deletions, all under `packages/browseros-agent/`. Typecheck clean across all 6 packages. 946 server tests pass (1 unrelated CDP-dependent test skipped — same state as origin/dev). Reference: TKT-788. The patch-CLI rollback that was in the squash of #934 is intentionally NOT in this commit. * fix(openclaw): handle 2026.5.4 acp-cli envelope shapes (media + injected timestamp) + bump image OpenClaw 2026.5.4 (the BrowserOS-pinned image variant with the no-auth handshake bypass needed for cron tool calls from inside ACP) introduced two new envelope prefix shapes that the post-bypass-deletion path now surfaces in user-message text: [media attached: <internal-path> (<mime>)] [<weekday> <YYYY-MM-DD HH:MM> <TZ>] [Working directory: <path>] <BrowserOS role envelope> The previous cleaner only matched a leading [Working directory: ...] \n\n line. With media + timestamp prefixes ahead of it the anchor no longer matched, so image-attachment user turns rendered with 8+ lines of envelope leak in the chat panel. Replaces the single OPENCLAW_WORKDIR_PREFIX with three content-shape- anchored patterns chained through stripOpenClawAcpCliEnvelope(): 1. [media attached: <path> (<mime>)] ← repeats per attachment 2. [<weekday> <YYYY-MM-DD HH:MM> <TZ>] ← injectTimestamp 3. [Working directory: <path>] ← acp-cli prefixCwd Each is anchored on its content shape (media attached:, weekday abbrev + ISO date, Working directory:) rather than just '[…]', so user-typed lines that happen to start with brackets are not eaten. Also bumps OPENCLAW_IMAGE from 2026.5.2-browseros.1 to 2026.5.4-browseros.1. The 5.2 image refused tool-side WS connections with 'device identity required' even though gateway auth.mode=none — PR #6 in browseros-ai/openclaw added the OPENCLAW_GATEWAY_PRIVATE_INGRESS_NO_AUTH bypass that ships in 5.4. Without 5.4, the cron tool (and any other tool that opens a fresh gateway WS from inside the embedded runner) fails with 1008. Verified end-to-end with the BrowserOS chat endpoint: - Plain text turn: clean - Image attachment turn: clean (was leaking 8 envelope lines pre-fix) - One-shot kind:at cron fires, PING fire renders clean - Second openclaw agent creates, runs, history isolated 15/15 history-mapper unit tests pass; typecheck clean across all packages. |
||
|
shivammittal274
|
50cbe48558 |
Revert "refactor(openclaw): lock no-auth gateway, bump image, delete token pl…" (#953)
This reverts commit
|
||
|
shivammittal274
|
d81b99c8e3 |
refactor(openclaw): lock no-auth gateway, bump image, delete token plumbing (TKT-788 WS-A) (#934)
* fix: disable bundled OpenClaw gateway auth * refactor(openclaw): delete token plumbing now that auth is locked off Builds on the cherry-picked spike (#933). With gateway.auth.mode=none locked in as the only path the bundled gateway runs, the BrowserOS-side token machinery becomes dead weight. This commit deletes: - OpenClawService: token field, tokenLoaded, gatewayAuthMode state machine, getGatewayToken(), getGatewayHttpToken(), ensureTokenLoaded(), refreshGatewayAuthToken(), loadTokenFromConfig() and all six lifecycle call sites. - OpenclawGatewayAccessor.getGatewayToken interface field. - OpenClawHttpClient / OpenClawGatewayChatClient: optional getToken constructor arg and authHeaders() helpers. - OpenClawObserver: gatewayToken field/parameter and the auth.token branch in the connect frame. - GatewayContainerSpec.gatewayToken and the OPENCLAW_GATEWAY_TOKEN env wiring; the OPENCLAW_GATEWAY_PRIVATE_INGRESS_NO_AUTH=1 env is now always set rather than conditional. Test suites: dropped bearer-token assertions and the two persisted-token tests in openclaw-service that asserted deleted behavior. Net: -310 LOC across src + tests, with 118 openclaw + acpx tests still green. Typecheck and biome clean. Reference: TKT-788 (move OpenClaw integration to ACPX runtime), WS-A. * refactor(openclaw): delete gateway image bypass, route image turns via ACP (TKT-788 WS-C) (#935) * refactor(openclaw): delete gateway image bypass, route image turns through ACP The browseros-ai/openclaw ACP bridge accepts image content blocks natively (extractAttachmentsFromPrompt at openclaw/src/acp/event-mapper.ts:92, forwarded via chat.send attachments at translator.ts:295), so the BrowserOS-side carve-out that diverted image-bearing turns to the gateway HTTP /v1/chat/completions endpoint is no longer needed. Deletes: - apps/server/src/api/services/openclaw/openclaw-gateway-chat-client.ts - The corresponding test file - AcpxRuntime.sendOpenclawViaGateway, persistGatewayTurn, recordToOpenAIMessages helpers - The image-attachment carve-out branch in AcpxRuntime.send - openclawGatewayChat option from AcpxRuntime + AgentHarnessService + agent routes ctor wiring - The randomUUID import (only the deleted helper used it) - The acpx-runtime test for the deleted carve-out Net: 614 LOC removed, 0 added, all 142 openclaw + acpx + agent tests still green. Reference: TKT-788, WS-C. Stacked on WS-A (#934). * refactor(openclaw): delete WS observer, feed ClawSession from harness events (#936) The openclaw-observer.ts WebSocket observer was a second tap on the same gateway events the AcpxRuntime already sees as ACP session/update notifications. Replace it with a pull from the AgentHarnessService's turn lifecycle stream — keeping ClawSession and the /openclaw/dashboard SSE endpoint shape unchanged for the BrowserOS UI. Changes: - AgentHarnessService: emit `turn_started` / `turn_event` / `turn_ended` to subscribers via a new `onTurnLifecycle(listener)` API. Wired around the existing `notifyTurnStarted/Ended` calls and inside the per-event read loop. - agents route: forward an optional `onTurnLifecycle` dep into the service it constructs. - server.ts: subscribe and route OpenClaw-adapter events to `OpenClawService.recordAgentTurnEvent(agentId, sessionKey, event)`. - OpenClawService: new `recordAgentTurnEvent` method that maps stream events to ClawSession transitions (working/idle/error + currentTool from `tool_call` events). Keeps the existing `onAgentStatusChange` / `getAgentState` / `getDashboard` API. - Delete `openclaw-observer.ts` (276 LOC) and all observer wiring (`new OpenClawObserver`, `ensureObserverConnected`, three `observer.disconnect()` call sites, the import). Net: 276 LOC removed from the observer; ~130 LOC added across harness event plumbing + recorder method. -146 LOC overall, all 141 tests still green, typecheck clean, biome clean. Reference: TKT-788, WS-B (Path 1: keep ClawSession + dashboard SSE shape). Independent of WS-A (#934) and WS-C (#935); will rebase on top of whichever lands first. --------- Co-authored-by: Nikhil Sonti <nikhilsv92@gmail.com> |
||
|
shivammittal274
|
86cb03a1fc |
fix(openclaw): drop BrowserOS-envelope regexes in history mapper (#952)
* fix(openclaw): drop BrowserOS-envelope regexes in history mapper
Replace the four BrowserOS-side regex strips (`<role>`,
`<user_request>`, `<system-reminder>`, `[Working directory:]`)
in history-mapper with a single call to
`unwrapBrowserosAcpUserMessage`. That helper is the same exact-string
matcher acpx-runtime already uses for non-OpenClaw history paths
(chat history endpoint, listing's `lastUserMessage`); it anchors on
the exact constants `buildBrowserosAcpPrompt` writes, so matcher and
wrapper travel together.
Also drops two patterns that were defensive-only with no emit site in
the codebase (`[Working directory:]` prefix and trailing
`<system-reminder>` block), and updates the corresponding tests to
use the realistic envelope shape `buildBrowserosAcpPrompt` actually
produces.
The OpenClaw-injected scaffolding patterns (cron prefix, queued-
marker, subagent context) stay in place for now — replacing those
needs either a side-channel cache keyed on cron job id or a structured
`trigger` field on the gateway's history schema, tracked as a
follow-up.
* fix(openclaw): strip acp-cli's [Working directory:] prefix before BrowserOS unwrap
The previous commit incorrectly removed the workdir-prefix strip on the
assumption it was speculative defensive code with no live emit site.
Actually emitted by OpenClaw's acp-cli (`/app/dist/acp-cli-*.js` line
1361, `prefixCwd ? \`[Working directory: ${displayCwd}]\\n\\n...` style),
so live history rendering regressed: every user message surfaced with
a `[Working directory: /Users/...]\\n\\n<role>...` envelope intact.
Restore the strip as an exact-shape line match (`^\\[Working directory:
[^\\]]*\\]\\n\\n`) anchored on the closing bracket + double-newline so
path content is consumed without a content-shape regex. Apply it
ahead of `unwrapBrowserosAcpUserMessage` so the BrowserOS unwrap's
`^<role>` anchor can match the now-leading envelope.
Also fix the test fixture: the BrowserOS unwrap performs exact-prefix
match against the full `BROWSEROS_ACP_AGENT_INSTRUCTIONS` constant —
truncated `<role>...` test bodies didn't match. Tests now use the
verbatim constant text via a shared `ROLE_BLOCK` helper.
Verified live: 8/8 history entries render with no envelope leaks.
|
||
|
shivammittal274
|
7765d99c73 |
feat(openclaw): aggregate sub-session history into agent main session (#939)
* feat(openclaw): aggregate sub-session history into agent's main session
Cron-triggered (and hook/channel-triggered) runs land in their own
ephemeral session files under the parent agent's directory:
/home/node/.openclaw/agents/<agentId>/sessions/<runId>.jsonl
The chat panel reads agent:<id>:main, so autonomous runs were invisible
in history even though they fired and persisted on disk.
This change makes `getSessionHistory(agent:<id>:main)` enumerate every
session under that agent (via the existing `sessions.list` gateway RPC)
and merge their messages into one chronological response. Each merged
message is tagged with `source` (main / cron / hook / channel) and the
sub-session's key, so the UI can render section markers without
re-parsing.
Filesystem isolation is enforced upstream — `sessions.list({ agentId })`
resolves to that agent's directory only (browseros-ai/openclaw
src/config/sessions/combined-store-gateway.ts:90), so no cross-agent
leakage is possible.
Behavior:
- Main session keys (`^agent:[^:]+:main$`) → aggregate
- Any other key → existing single-session behavior
- Sub-session fetch failures → logged + dropped (partial timeline
preferable to a hard failure that hides main)
- `limit` applied post-merge across the unified timeline
- Streaming variant (`Accept: text/event-stream`) unchanged for now
Reuses the pre-existing `cliClient.listSessions` and
`httpClient.getSessionHistory` — no new gateway integration.
Validation:
- bun typecheck clean
- bunx biome check clean
- 44 openclaw service + route tests pass
* feat(openclaw): wire chat panel history through gateway aggregation
Adds the missing seam between the chat panel's history fetch and
OpenClawService's aggregated history.
Before this change:
- Chat panel calls GET /agents/<id>/sessions/main/history
- AgentHarnessService.getHistory delegates to AcpxRuntime.getHistory
- AcpxRuntime reads ~/.browseros-dev/agents/acpx/sessions/<key>.json
- That local file is only written by AcpxRuntime.send (user turns)
- Cron / hook / channel turns persist on the gateway side instead
- Panel sees user turns only; autonomous turns are invisible
After this change:
- OpenClawProvisioner gains optional getAgentHistory(agentId) method
- AgentHarnessService.getHistory branches on adapter — for openclaw,
routes through the provisioner instead of the runtime
- server.ts wires the provisioner method to call
OpenClawService.getSessionHistory("agent:<id>:main") which already
aggregates main + every sub-session
- New history-mapper.ts converts OpenClaw rich content blocks
(text/thinking/toolCall/toolResult) into AgentHistoryEntry shape
the chat panel consumes
Layering preserved:
- AcpxRuntime untouched, still generic, zero services/openclaw imports
- AgentHarnessService still talks only to abstract OpenClawProvisioner
- server.ts is the single concrete-binding seam (same place that
wires createAgent, removeAgent, getStatus)
- Other adapters (claude, codex) keep their existing local-file
history path — no behavior change for them
Tool-call pairing: assistant `toolCall` blocks are stored by
toolCallId; subsequent `toolResult` (role: 'tool') messages mutate the
same AgentHistoryToolCall reference to attach output / error, so the
UI renders complete tool entries instead of orphan inputs.
Net: +240 LOC, 1 new file, AcpxRuntime untouched, 117 tests still pass.
* feat(openclaw): paginate aggregated history + strip prompt scaffolding
Two follow-ups on the aggregation work, both required for the chat
panel to render OpenClaw history cleanly.
1. Compound-cursor pagination across sub-sessions
The previous aggregation always returned the full merged window with
cursor=null/hasMore=false, which broke "load more" in the chat panel
once an agent's history grew beyond a single page (every cron job
spawns a sub-session, so this hits quickly).
Per-session cursor support already exists on the gateway HTTP endpoint
(`session-history-state.ts:paginateSessionMessages`). The aggregator
now threads each session's cursor through and emits a compound cursor
encoding `{<sessionKey>: messageSeq | null}`, base64url JSON. A `null`
slot means the session is exhausted; subsequent pages skip it.
The gateway records the per-session monotonic seq inside the
`__openclaw.seq` extension envelope rather than the top-level
`messageSeq` field; the cursor reads from there. The wire-shape type
gains an optional `__openclaw?: { id?, seq? }` field reflecting that.
2. Strip OpenClaw + BrowserOS scaffolding from history user messages
Cron-fired user messages on the gateway side carry an OpenClaw
template:
[cron:<uuid> <name>] <payload>
Current time: ...
Use the message tool if you need to notify the user directly with an
explicit target. ...
BrowserOS-initiated turns carry the ACP system prefix:
[Working directory: ...]
<role>...</role>
<user_request>
<actual user text>
</user_request>
<system-reminder>...</system-reminder>
Both surface verbatim in the chat panel today. Add
`cleanHistoryUserText` (in history-mapper) which extracts:
- the cron payload (and drops the trailer)
- the user_request body (and drops the role / working-dir / system-
reminder envelopes)
Non-matching text falls through unchanged so future patterns we don't
recognize stay visible rather than getting silently dropped.
Verified end-to-end:
- /agents history endpoint now returns clean text per item
- Pagination cursor advances across pages with correct seq ordering
- Chat panel renders messages as `print('hello')`, `hey`, etc.
(no leaked envelopes or trailers)
- 8 new unit tests for cleanHistoryUserText + the converter, +
86 existing openclaw tests still pass
* feat(openclaw): handle queued-marker concatenation in history cleaner
When multiple cron prompts (or any prompts) arrive while a turn is
still active, BrowserOS's harness queue concatenates them into a
single user message joined by a marker line:
[Queued user message that arrived while the previous turn was still active]
That blob renders as one wall of text in the chat panel — and worse,
the cron-prompt cleaner doesn't fire because the message no longer
*starts* with `[cron:...]`. cleanHistoryUserText now splits on the
queued-marker line and runs each chunk through the per-message cleaner
(cron-prompt extraction or BrowserOS-prefix unwrap), then joins the
non-empty results with single newlines so each prompt renders as its
own visually distinct line.
Verified live: a 6926-char queued blob containing five concatenated
[cron:...] prompts now renders as five short `print('hello')` lines.
+ 2 unit tests covering split + leading-marker edge case.
* feat(openclaw): drop subagent context + reasoning-only assistant turns
Two new patterns surfaced during e2e cron testing.
1. [Subagent Context] prefix: when an OpenClaw agent invokes a nested
subagent, the subagent's session is seeded with a user message:
[Subagent Context] You are running as a subagent (depth N/M). ...
Begin. Your assigned task is in the system prompt under **Your Role**.
The actual task lives in the subagent's system prompt; the user
message body is pure scaffolding. cleanHistoryUserText now returns
empty for these so the converter drops the entry — no empty bubble.
2. Reasoning-only assistant turns: MiniMax with thinking:minimal often
returns content with only `thinking` blocks and no `text` block on
trivial prompts ("Print hello"). The empty text bubble plus dangling
reasoning collapsible reads as a broken UI. The converter now skips
any entry where text is empty AND there are no tool calls (regardless
of reasoning).
Trade-off: reasoning-only turns lose their reasoning collapsible. The
alternative (empty-bubble cards) is worse. If we want to preserve the
reasoning, surface it as the bubble's text — separate UI decision for
later.
+ 3 unit tests covering both patterns.
|
||
|
Dani Akash
|
db5e55a174 |
feat(agent-files): expose openclaw produced files inline + outputs rail (#946)
* feat(server): foundation for OpenClaw agent file-output attribution
Phase 1 of TKT-762 — surface files OpenClaw agents produce as
artifacts inline in chat + a per-agent Outputs rail. This commit
lays the storage + I/O foundation only; turn-lifecycle wiring,
HTTP routes, and UI follow in subsequent phases.
- New `produced_files` Drizzle table (FK→agent_definitions with
cascade, unique on (agent, path) so re-modifications upsert).
Migration 0002_chemical_whirlwind.sql. Adapter-agnostic schema
— V1 only enables the watcher for openclaw, V2 can plug Claude
/ Codex into the same table without migrating.
- `ProducedFilesStore` — snapshot/finalize-turn diff API plus
by-turn / by-agent queries and a path-resolver that enforces
workspace-root containment for the download / preview routes.
- `walkWorkspace` — bounded recursive workspace walker; skips
symlinks (no host-fs smuggling), excludes node_modules / .git /
.cache, hard-capped at 50k entries / depth 16.
- `file-preview` helper — extension + magic-byte MIME detection,
bounded text-snippet reader (1 MB cap), inline image base64
reader (4 MB cap). Streaming download path lives in the route
layer (next phase) — this module only handles the small
in-memory reads the preview UX needs.
* feat(server): attribute openclaw turn outputs to the harness layer
Phase 2 of TKT-762 — wire the per-turn workspace diff into the
single dispatch path that owns every turn's lifecycle. Two prior
wiring points the original plan named (the OpenClaw HTTP chat
route + OutboundQueueService.tryDispatch) were collapsed in dev
into agent-harness-service.runDetachedTurn — both direct sends
and queued sends route through it now, so a single hook covers
both. The old `OutboundQueueService` is gone; its successor
`message-queue.ts` re-enters runDetachedTurn for the queued
case, so we still only need to bracket once.
Changes:
- New `produced_files` variant on `AgentStreamEvent` so the
inline artifact card has a wire-format hook independent of the
REST API.
- `ProducedFilesStore` gains `resolveAgentDefinitionId` to bridge
gateway-side openclaw agent names to the harness's
`agent_definitions.id`, handling both the reconciled-row shape
(id == openclaw name) and the BrowserOS-created shape
(id = oc-<uuid>, name = openclaw display name).
- `AgentHarnessService.runDetachedTurn`: snapshot the openclaw
workspace before `runtime.send(...)`, finalize the diff in the
outer finally, push the resulting rows as a `produced_files`
event. Adapter-gated to openclaw only — Claude / Codex agents
write to the user's own filesystem and don't need
attribution.
- Skip attribution on user-cancel (`abort.signal.aborted`) so
the side effects of an aborted turn don't get surfaced as
"outputs you asked for." On runtime errors we still attribute,
because partial outputs are what the user is most likely to
want to recover.
- Lazy-init the store via `tryGetProducedFilesStore()` so tests
that swap in a fake `agentStore` don't trip the
process-wide `getDb()` initialisation guard.
- File attribution extracted into `attributeTurnFiles` helper to
keep `runDetachedTurn`'s cognitive complexity under the lint
ceiling.
Verifications:
- Server tsgo --noEmit clean for changed files.
- 162/162 server-api tests pass.
- Biome lint clean on all three changed files.
* feat(server): expose produced-files HTTP API for /agents
Phase 3 of TKT-762 — surface the rows Phase 2 attributes via four
read-only endpoints under the existing `/agents` router. Mounted
where the agents page already polls so the rail UI doesn't add
a second router/origin to its trust boundary.
Routes:
- GET /agents/:agentId/files
Outputs-rail data, grouped by the assistant turn that
produced each batch, newest first. `?limit=` clamps to N
rows server-side (default 200).
- GET /agents/:agentId/files/turn/:turnId
Per-turn refresh — used by the inline-card consumer to
rebuild metadata after the SSE `produced_files` event lands,
and by direct fetches that missed the live event.
- GET /agents/files/:fileId/preview
Discriminated `FilePreview` JSON: text snippet (≤1MB),
base64 image (≤4MB), pdf metadata, or `binary` placeholder
when neither preview path applies. 404 when the file id is
unknown OR the on-disk file disappeared after attribution.
- GET /agents/files/:fileId/download
Streams raw bytes via `Bun.file().stream()` with
`Content-Disposition: attachment` and the detected MIME
type. The fileId is opaque — the server resolves the agent
and on-disk path; the client never sees a path, so traversal
is impossible by construction.
Service layer:
- `AgentHarnessService` gains `listAgentFiles`,
`listAgentFilesForTurn`, `previewProducedFile`, and
`resolveProducedFileForDownload`. All four are no-ops for
claude / codex adapters (they return null/[]) so the route
contract stays uniform across adapters even though only
openclaw produces rows in v1.
- New `ProducedFileEntry` and `ProducedFilesRailGroup` DTOs —
trimmed wire shapes that strip `agentDefinitionId` and
`sessionKey` from the on-disk row.
Verifications:
- Server tsgo --noEmit clean for changed files (only pre-
existing `Bun` global warning).
- 162/162 server-api tests pass.
- Biome clean on both changed files.
Smoke-test instructions for the route shape live in the plan
under §6 and §8; full end-to-end smoke happens in Phase 6.
* feat(agent): client-side hooks + types for agent file outputs
Phase 4 of TKT-762 — frontend foundation for the inline artifact
card and the per-agent Outputs rail. UI components themselves
land in Phase 5; this commit only adds types, hooks, and shared
helpers so the wiring is in place when the components arrive.
New module: `apps/agent/lib/agent-files/`
- `types.ts` — `ProducedFile`, `ProducedFilesRailGroup`, and the
discriminated `FilePreview` union, mirrored from the server-side
DTOs in `apps/server/src/api/services/agents/agent-harness-service.ts`.
The `agentDefinitionId` / `sessionKey` columns on the on-disk
rows deliberately do NOT exist at the type boundary — clients
refer to files by opaque `id`.
- `file-helpers.ts` — pure helpers: `inferFileKind` (icon
routing), `formatFileSize`, `extensionOf`, `basenameOf`,
`buildFileDownloadUrl`. No React, no fetch, no DOM — anything
stateful belongs in the hooks.
- `useAgentOutputs.ts` — `useAgentOutputs(agentId)` for the rail,
`useAgentTurnFiles(agentId, turnId)` for the inline card,
`useInvalidateAgentOutputs()` for the chat-stream-completion
hook (Phase 5 will plumb this), and `useRefreshAgentOutputs()`
for the rail's manual refresh button.
- `useFilePreview.ts` — `useFilePreview(fileId)` with
`staleTime: Infinity` (previews are immutable for a given id;
no point refetching on focus). Always opt-in (`enabled`) — the
preview only loads when the user clicks a row.
- `index.ts` — barrel re-export so consumers import from one path.
Touched in `apps/agent/entrypoints/app/agents/`:
- `agent-harness-types.ts` — added `produced_files` variant + the
`HarnessProducedFile` type to `AgentHarnessStreamEvent`. Mirrors
the server-side change from Phase 2 so the client SSE consumer
type-narrows correctly.
- `useAgents.ts` — exported the previously-private `agentsFetch`
helper and the `AGENT_QUERY_KEYS` registry so the agent-files
hooks reuse them without duplicating fetch / key conventions.
Three new keys added: `agentOutputs`, `agentTurnFiles`,
`filePreview`.
Verifications:
- Agent tsgo --noEmit clean.
- Biome clean on all touched files.
* feat(agent): inline artifact card + per-agent outputs rail
Wires the chat surface to the produced-files API shipped earlier:
- Inline artifact card under each assistant turn that produced files,
populated by the live `produced_files` SSE event (resumes also stamp
`turnId` so a missed live event can fall back to the per-turn fetch).
- Collapsible right-side Outputs rail on the agent conversation page,
grouped by turn, with Refresh + per-agent open/close persistence in
localStorage. Gated to openclaw adapters in v1.
- Shared file preview Sheet branches on the FilePreview union: text
snippet (markdown for `.md`/`.mdx`, otherwise pre+code), image data
URL, and download-only fallback for pdf/binary/missing.
- Conversation hook invalidates the rail's React Query cache from its
finally block so newly attributed files appear without a manual
refresh.
* feat(agent-files): polish — symlink-safe paths + toast on failures
- `resolveFilePath` now rejects symlink-escapes from the workspace
by realpath-resolving both endpoints and re-checking containment.
Lexical traversal (`..` segments) still fails fast without
touching the filesystem.
- Added `produced-files-store.test.ts` with 6 path-resolution cases
including a symlink whose target lives outside the workspace
root — the prior string-only check would have allowed this.
- File preview Sheet: surfaces preview-load failures in a toast
(in addition to the inline error block, which is easy to miss
when the body has scrolled). Download button now intercepts the
click so a missing baseUrl shows a toast instead of silently
hiding the button.
- Outputs rail: refresh failures fire `toast.error` with the
underlying message.
* fix(agent-files): drop duplicate `/agents` prefix from client paths
`agentsFetch` / `buildAgentApiUrl` already prepend `/agents`, but
the file-output hooks were passing fully-qualified paths
(`/agents/<id>/files`, `/agents/files/<id>/preview`, etc.) which
resolved to `/agents/agents/...` and 404'd. Fixed the four call
sites to pass paths relative to the `/agents` root.
* fix(agents): strip openclaw role envelope from chat history
PR #924 introduced a second `<role>…</role>` prefix for openclaw
turns — a single-line block distinct from the multi-line BrowserOS
role TKT-774 wired the unwrap against. Because TKT-774's
`stripOuterRoleEnvelope` matched the BrowserOS prefix exactly, the
openclaw envelope sailed through unstripped and user messages on
openclaw agents rendered the full preamble in /sessions/main/history
responses.
Make the strip adapter-agnostic: any
`<role …>…</role>\n\n<user_request>\n…\n</user_request>` shape gets
unwrapped. Drops the now-unused BROWSEROS_ACP_AGENT_INSTRUCTIONS
constant and adds a regression test that uses the openclaw form
verbatim.
* feat(agent-files): inline file-card strip with rail deep-link
Replaces Phase 5's row-list ArtifactCard with a horizontal strip
of small file cards under any assistant turn that produced files.
Click a card → opens the FilePreviewSheet directly (preview +
download). Click View / +N → opens the per-agent Outputs rail and
scrolls / expands the matching turn group.
The card strip:
- Caps at 4 visible cards; remainder collapses into a +N pill that
shares the View handler.
- Owns its own FilePreviewSheet instance (parallel to the
deprecated ArtifactCard) so the per-card preview path doesn't
fight with the rail's Sheet.
- Hidden during streaming and absent when producedFiles is empty.
- Adapter-gated upstream: AgentCommandConversation only passes the
open-rail callback when adapter==='openclaw', so claude / codex
agents render no rail-opening affordance.
Rail changes:
- Accepts focusTurnId + onFocusTurnConsumed; the matching
RailTurnGroup expands and scrollIntoView's on focus, then fires
the consumed callback so the parent can drop the URL state.
- ?outputsTurn=<turnId> deep-links work: external nav opens the
rail, sets focusTurnId, and clears the param after consumption.
ArtifactCard is marked @deprecated; remove in a follow-up once
nothing imports it.
* fix(agent-files): keep file-card strip visible after history reload
After Phase 7 the inline FileCardStrip vanished as soon as a turn
finished: `filterTurnsPersistedInHistory` dropped the optimistic
turn once history reloaded, and history items don't carry
`producedFiles`. So the user could see a file produced inside an
assistant message but no card to open it.
Two fixes in tandem so the strip survives both the just-finished
case AND a fresh page load:
- New `selectStripOnlyTurns` keeps persisted turns that still
carry `producedFiles`. `ConversationMessage` learns a
`stripOnly` mode that renders only the trailing strip (no
duplicate user/assistant bubbles, since those are rendered by
`ClawChatMessage`).
- `AgentCommandConversation` now also calls `useAgentOutputs` and
passes `tailStripGroups` to `ClawChat`. Each rail group not
already covered by a live or strip-only turn renders as its own
tail `FileCardStrip` after history. Dedup keys on `turnId` so
the same turn never doubles up.
Adapter-gated upstream — claude / codex agents skip the
useAgentOutputs fetch entirely. The card click still opens the
preview Sheet directly; View / +N still deep-link to the rail at
the matching turn group.
* fix(agent-files): per-turn association + cache invalidation
Two fixes for the inline file-card strip:
1. Strips were stacking at the conversation tail because every
produced-files group rendered as a tail strip after history.
New `mapHistoryToProducedFilesGroups` matches each group to
the assistant history message that came from its turn — by
`group.turnPrompt` vs the first non-blank line of the
preceding user message — and ClawChat renders the strip
directly under that bubble. Groups that don't match any
history pair (orphans) still fall through to the tail.
2. `useInvalidateAgentOutputs` was passing `undefined` as the
baseUrl placeholder to `invalidateQueries({ queryKey })` —
react-query's positional partial-match doesn't treat
undefined as a wildcard, so the cache stayed stale until the
query refetched on its own (e.g. window focus). Switched to
predicate-based invalidation that matches by [agentOutputs
marker, agentId] regardless of baseUrl. Same for the per-turn
files key.
Net effect: send a turn that produces files → strip appears
under the just-finished assistant message; reload the page →
strips still appear under the right bubbles, not bunched at
the bottom.
* fix(agent-files): review feedback — name guard, RFC 5987, limit cap
Three review-flagged issues:
1. Path traversal via agent display name — `getHostWorkspaceDir`
accepted any string and `path.join`'d it, so a name like
`../../tmp` escaped `.openclaw`. The pre-turn snapshot would
then walk that escaped directory and attribute every file to
the new turn; resolveSafeWorkspacePath's containment check is
relative to the same escaped root so it would later serve
arbitrary host paths. Added `isAgentWorkspaceNameSafe` (rejects
`..`, separators, control chars, leading dots, empty); the
builder now throws on unsafe names plus a defensive
realpath-style containment check after the join. Harness
wraps the call so the path-traversal trip just disables file
attribution for the turn instead of failing the whole send.
Six-case regression test pinned.
2. `encodeRfc6266Filename` JSDoc claimed an RFC 5987
`filename*=UTF-8''<percent-encoded>` fallback but the impl
only stripped CRLFs/quotes. Now actually emits the fallback
when non-ASCII is present; helper returns the full
`filename="…"; filename*=UTF-8''…` attribute pair so the call
site doesn't have to wrap in quotes.
3. `/agents/:agentId/files` `?limit=` was forwarded to the DB
uncapped — extracted `parseAgentFilesLimit` that clamps to
[1, 500] before forwarding.
Also extracted `resolveSafeWorkspaceDir` + `snapshotWorkspaceForTurn`
helpers off `runDetachedTurn` so the new safety branch doesn't
push it past biome's cognitive-complexity cap.
|
||
|
Nikhil
|
d61d6fc8a9 |
feat: add ACPX agent runtime adapters (#924)
* feat: add acpx claude runtime paths * feat: add acpx adapter preparation * refactor: use acpx adapter preparation * refactor: move openclaw image turns to adapter * fix: keep openclaw independent of host cwd * fix: address acpx review feedback * fix: preserve claude host auth in acpx |
||
|
shivammittal274
|
d383b5e344 |
feat(eval): add claude-generated run report artifact (#892)
* feat(eval): add claude-generated run report artifact * fix(eval): install claude code cli for CI evals * fix(eval): bypass claude code tool permissions * Eval metrics configs (#932) * feat(eval): add agisdk comparison metrics configs * fix(eval): keep cdp crashes from aborting run |
||
|
Nikhil
|
0d56815cba |
fix: store server database under BrowserOS dir (#923)
* fix: store server database under browseros dir * fix: address PR review feedback for 923 |
||
|
Nikhil
|
c07d3d95d4 |
feat: add sqlite drizzle persistence (#919)
* feat: add drizzle agent schema * feat: run sqlite drizzle migrations * refactor: remove old sql identity dependency * feat: store harness agents in sqlite * build: package db migrations * refactor: remove sqlite oauth token store * feat: restore oauth token storage * fix: handle empty install id * chore: ignore server runtime state * fix: address review feedback for PR 919 |
||
|
Nikhil
|
1d42a973ea | refactor: extract acpx runtime templates (#918) | ||
|
Nikhil
|
921a797c5b |
feat: add ACPX agent soul and memory support (#917)
* feat: add acpx agent runtime context helpers * feat: add acpx runtime state store * feat: prepare acpx agent runtime context * feat: inject acpx agent command environment * feat: forward acpx agent chat cwd * fix: normalize acpx session record fallback * feat: improve acpx agent soul and memory prompts * fix: address PR review comments for memory-soul-acp * fix: satisfy acpx runtime deepscan checks |
||
|
Nikhil
|
d94597bbf9 |
fix(agent): add CLI model catalog entries (#915)
* fix(agent): add CLI model catalog entries * fix: address PR review comments for acpx-models |
||
|
Dani Akash
|
974e7e9b86 |
fix(agents): hide BrowserOS ACP envelope from chat history payloads (TKT-774) (#907)
* fix(agents): hide BrowserOS ACP envelope from chat history payloads (TKT-774) The user-message text persisted on the wire carried two nested envelopes — the outer `<role>You are BrowserOS…</role>` + `<user_request>…</user_request>` block from buildBrowserosAcpPrompt and the inner `## Browser Context` + `<selected_text>` + `<USER_QUERY>` block from formatUserMessage. PR #856 had unwrapped only the outer envelope on history reads, so the user bubble in the agent rail still rendered the inner envelope, and the LLM chat-service path leaked the wrapper all the way back to the sidepanel client through AI SDK's stream sync. Two surgical fixes, both server-only: 1) ACP path (acpx-runtime.ts) — replace unwrapBrowserosAcpPrompt with a comprehensive unwrapBrowserosAcpUserMessage that strips both layers and decodes the </>/& escapes the server applied via escapePromptTagText. Each step is independently defensive (anchors that don't match are skipped) so the helper is idempotent and tolerates partial / older / future-shape envelopes. Applied in userContentToText (history mapper) and inherited by extractLastUserMessage (listing's lastUserMessage). 2) LLM chat path (chat-service.ts) — split the persisted user message from the prompt-time copy. session.agent.appendUserMessage now stores the raw user text; a transient promptUiMessages array is built with the wrapped (formatUserMessage + context-change prefix) form and passed to createAgentUIStreamResponse for the model. onFinish restores the raw form before persisting, so the user-visible message and any future history reads see only the user's typed text. Tests: - acpx-runtime.test.ts: new dedicated unwrapBrowserosAcpUserMessage suite covering fully-wrapped messages, only-outer / only-inner inputs, selected_text blocks with attribute strings, idempotency, literal user-typed angle-bracket round-trip, and an integration test that round-trips the real formatUserMessage output through the unwrap to pin the writer/reader contract. - chat-service.test.ts: existing 'rebuilds a managed-app session' test updated for the new behaviour — asserts the persisted user message is the raw text and the prompt copy passed to the agent carries the Klavis context-change notice. * fix(agents): decode entity escapes before stripping inner envelope (TKT-774) The unwrap was running its inner-envelope strips against the literal-tag form (<USER_QUERY>, <selected_text>) but the persisted payload has those tags entity-escaped (<USER_QUERY>, <selected_text>) — buildBrowserosAcpPrompt runs escapePromptTagText over the entire formatUserMessage payload before adding the outer <role>+<user_request> envelope, so the inner anchors never matched against the on-disk text and the user was still seeing <USER_QUERY> in /agents/:id/sessions/main/history responses. Reorder unwrapBrowserosAcpUserMessage to: outer-strip → decode entities → inner-strips. Test fixtures updated to reflect the actual on-wire form (escaped inner tags); the round-trip test duplicates the escape rule inline so the contract between buildBrowserosAcpPrompt and the unwrap is pinned end-to-end. |
||
|
Nikhil
|
fd5aba249b |
fix: stabilize OpenClaw gateway startup (#888)
* feat(server): add shared process lock helper * feat(container): add container name reconciliation helpers * feat(openclaw): serialize lifecycle across processes * fix(openclaw): reconcile fixed gateway container startup * test(openclaw): cover lifecycle race recovery * fix(server): satisfy process lock error override * fix(openclaw): address review feedback * test(openclaw): align serialization mock with image check |
||
|
Nikhil
|
492f3fcdf2 |
feat(openclaw): prewarm ghcr image in vm (#887)
* feat(openclaw): add gateway image inspection * feat(openclaw): pull gateway image from registry * refactor(vm): decouple readiness from image cache * refactor(openclaw): remove vm cache from runtime factory * feat(openclaw): detect current gateway image * feat(openclaw): prewarm vm runtime and reuse current gateway * feat(openclaw): prewarm runtime on server startup * refactor(vm): remove browseros image cache runtime * refactor(build-tools): remove openclaw tarball pipeline * chore: self-review fixes * fix(openclaw): suppress prewarm pull progress logs * fix(openclaw): address review feedback * fix(openclaw): resolve review findings * fix(dev): stop stale watch supervisors |
||
|
Dani Akash
|
8712f89f18 |
feat(agents): durable per-agent chat message queue + composer Stop (#880)
* feat(agents): durable per-agent chat message queue + composer Stop button
* fix(agents): tighten queue UI — smaller Stop, drop empty indicator, live drain attach
User feedback round 1 on the message-queue UX:
1) The Stop button matched the send/voice mics at h-10 w-10 with a
solid destructive fill, which read as alarming. Shrunk to h-8 w-8,
ghost variant with a soft destructive/10 background, smaller
filled square glyph. Reads as a calm 'stop' affordance instead of
a panic button.
2) The QueueItem's leading <QueueItemIndicator> dot was decorative
only — no state, no interaction. Dropped it from QueuePanel along
with the import; queue items now render as a clean preview line
with the trailing X remove action.
3) When the server drained the queue and started the next turn, the
chat panel didn't pick up the live stream until the user
navigated away and back. The hook's resume effect previously
only fired on agent change, not on listing-observed activeTurnId
change. Surface activeTurnId from useHarnessAgents into
useAgentConversation; effect now re-runs when the id changes,
calls /chat/active, and attaches to the new turn — so a queued
message starts streaming the moment the server drain pops it.
* fix(agents): don't reset streaming state from the resume effect's no-op paths
The Stop button was disappearing while the agent was actively
streaming, even though events were still flowing into the chat. Root
cause: the resume effect's `finally` block reset `streaming`,
`turnIdRef`, and `lastSeqRef` unconditionally — including on the
early-return paths (no active turn, or another mechanism already
owns the stream).
Sequence that triggered it:
1) User sends a message → send() sets streamAbortRef + streaming=true
and starts consuming the SSE.
2) User enqueues another message → enqueue mutation invalidates the
listing query.
3) Listing refetches with the live activeTurnId → the resume
effect re-fires (deps include activeTurnIdDep).
4) attemptResume hits `if (streamAbortRef.current) return` because
send() owns it.
5) The finally clause fires anyway and calls setStreaming(false),
clobbering the live state set by send(). The SSE consumer keeps
running (refs are intact) so text keeps streaming, but the React
flag is wrong, so the Stop button gates off.
Fix: track whether *this* run actually started a stream
(`weStartedStream`). The finally only resets state when it does.
Early-return / no-active-turn paths now leave streaming/turnIdRef/
lastSeqRef alone for whoever does own them.
Also widens the Stop button's visibility (`canStop` prop on
ConversationInput) so it stays steady across the brief gap between
turns when a queue drain is mid-flight; the parent computes
`streaming || activeTurnId !== null || queue.length > 0`. The
visibility widening is independent of the streaming-state fix above
— both are now in place.
* revert: drop canStop widening — Stop only shows while streaming
Reverts the canStop prop on ConversationInput and the OR-with-queue
visibility from AgentCommandConversation. Stop is gated solely on
`streaming` again. Between turns (queue draining) the button stays
hidden — only the actively-streaming turn is interruptible from the
composer, which matches what the user actually expects.
* fix(agents): persist the kicking-off prompt on active turns so the resume placeholder isn't empty
When a queued message drained and started a new turn, the chat
panel's resume effect staged a placeholder turn with userText: ''
because the hook had no way to know what message kicked off the
turn — only the agent-side stream was visible, and the user bubble
above it was blank until the user navigated away and back (at which
point the session record's history loaded normally).
Fix: ActiveTurnRegistry.register now accepts an optional `prompt`
that's stashed on the turn and surfaced via describe() / the
ActiveTurnInfo response. AgentHarnessService.startTurn passes the
incoming message into register. /chat/active returns it. The chat
hook's resume effect uses active.prompt as the placeholder
turn's userText, so the user bubble shows the queued message text
the moment streaming begins. Falls back to '' for older clients
that haven't been refetched yet.
* fix(agents): always release streamAbortRef on resume cleanup, even when cancelled
Greptile P1 follow-up. The previous `weStartedStream` guard correctly
stopped the resume effect's no-op early-returns from clobbering an
in-flight `send()` stream — but it also stopped a *cancelled*
mid-stream resume from clearing its own `streamAbortRef`. When the
cleanup fires (e.g. the 5s listing poll captures a new queue-drain
turn id while the SSE for the prior turn is still finishing), the
next effect run hits the `if (streamAbortRef.current) return` guard
against the now-aborted controller and never reattaches, leaving
`streaming === true` with no live stream until the user navigates
away.
Split the finally block: always release `streamAbortRef` when we
owned the controller (so the next run can take over), but only
reset the streaming flag / turn id / lastSeq on a clean exit (the
new run will set those itself, so resetting on cancel would just
flicker).
|
||
|
Dani Akash
|
ba60bf466f |
feat(agents): rich command-center rows + home grid + dead-code sweep (#879)
* feat(agents): rich-info command center rows + pin/PATCH/adapter-health backbone
Splits AgentRowCard from a 271-line monolith into a shallow tree of
single-responsibility sub-components under `agent-row/`:
AgentTile, AdapterHealthDot, PinToggle, AgentTitleRow,
AgentSparkline, AgentSummaryChips, AgentLastMessage, CwdChip,
AgentTokenSummary, AgentMetaRow, AgentErrorPanel, AgentActions
Adds the data each row consumes:
- pinned: boolean field on AgentDefinition + FileAgentStore.update
+ new PATCH /agents/:id route. useUpdateHarnessAgent mutation
optimistically updates the listing cache so the star flips
instantly; rolls back on error.
- Listing payload extended with lastUserMessage, cwd, tokens
(cumulative + last7d shape — last7d zero-filled until the
activity ledger lands), turnsByDay/failedByDay (zero-filled),
lastError/lastErrorAt, activeTurnId. AcpxRuntime grows a
getRowSnapshot() that reads cwd + cumulative tokens + last user
message from the session record in one pass.
- Adapter health: in-memory AdapterHealthChecker probes
`claude --version` / `codex --version` with a 2s timeout and
caches results for 5 min. /adapters response carries
{ healthy, reason?, checkedAt }. Tile-corner dot exposes the
state via HoverCard; openclaw inherits health from the gateway
snapshot already on the page.
Sub-components are pure: card itself owns no state. Sort order
becomes pinned-first, then recency. HoverCard is the workhorse for
keeping rows compact while exposing depth (full message, token
breakdown, daily turn list, error stack, adapter reason).
* refactor(agents): tighten command-center row design + cut redundant affordances
User feedback round 1:
1) Two green dots on the tile (health + liveness) was confusing. Health
moves out of the tile entirely and surfaces as an inline 'Unavailable'
chip in the model line — silent when the adapter is healthy, with a
warning amber chip + HoverCard reason when not. The tile now shows
one signal: liveness.
2) The last-user-message HoverCard wasn't telegraphing intent. Drop the
HoverCard. The line is informational, italic, with a leading quote
glyph so the row reads like a conversation snippet. To see the full
message the user opens the chat (which is the action they want next
anyway).
3) Resume + Chat were duplicate CTAs. Single primary action per row:
Resume (filled, accent-orange, with a pulsing dot) replaces Chat
when there's an active turn. Both navigate to /agents/:id but the
row tells the user which action they're taking.
4) Tokens weren't visible because the row gated on last7d.requestCount,
which is zero until the activity ledger ships. Switch to lifetime
tokens (which we have today). Drop the '7d stats:' framing — talking
about a window we can't compute would be misleading. The HoverCard
surfaces input/output split + a footnote that per-window stats land
in a follow-up.
5) CWD was rendering the server's own running directory, which is
meaningless to users. Hide it from the row entirely. The cwd field
still rides in the listing payload for future surfaces (chat panel,
debug view) — only the row stops rendering it.
Aesthetic refinements while we're here:
- Whole card carries state, not just the tile: working rows get an
accent-orange tinted border with a soft glow, error rows tint
destructive, idle rows lift on hover.
- Pin star fades in on hover (group-hover) when unpinned and stays
solid amber when pinned — keeps the rail calm by default.
- Tabular-nums on token figures so columns visually align across rows.
- Drop CwdChip and AdapterHealthDot files: no callers left.
* fix(agents): align row title flush-left whether pinned or not
Pin star moved from leading the title to trailing the badges, and
hidden from layout entirely (`hidden group-hover:inline-flex`) when
unpinned. The previous `opacity-0` rule kept the star reserving its
`size-6` slot, which left every unpinned title indented relative to
the model / preview / meta lines underneath it. Title now flushes
left in both states; pinned star stays solid amber so the signal
isn't hidden, and unpinned reveals an outline star on row hover for
the toggle affordance.
* fix(agents): keep pin-toggle slot reserved so row height is constant
Switching the unpinned star from `hidden group-hover:inline-flex`
to `opacity-0 group-hover:opacity-100`. The hidden/show variant was
collapsing the title row's height when the star wasn't rendered,
which made every card below visibly shift on hover. Always rendering
the button (with opacity-only visibility) keeps the row's vertical
metrics constant; the title still flushes left because the slot is
trailing, not leading.
Card hover effect (-translate-y + shadow-md) restored — the layout
shift wasn't coming from the card hover; it was the pin slot
appearing and disappearing.
* fix(agents): quieten row hover — border-tint only, no lift, no shadow
Drop the `-translate-y-px` and `hover:shadow-md` from the row card
plus the working-state inner ring. The translate + shadow grow
combination was visibly noisy as the cursor moved through the rail —
each row 'lifted' as you passed over it. Hover now just tints the
border in accent-orange/30; working and error states keep their
distinct border colours but no inner ring. Card height and shadow
stay constant in every state, so the rail reads as a calm vertical
list of cards.
* feat(home): rich Recent Agents grid + dead-code sweep
The /home Recent Agents grid was a placeholder shell. Every 'rich'
field on the card (lastMessage, lastMessageTimestamp, activitySummary,
currentTool, costUsd) was wired to undefined because AgentCommandHome
called `buildAgentCardData(agents, status?.status, undefined)` — the
dashboard arg has been hard-coded undefined since the harness
migration. Repointing the grid at `useHarnessAgents` + `useAgentAdapters`
gives every card the same enriched data the rail uses.
What the new card shows per agent:
• Adapter glyph tile + liveness dot (working pulses; asleep is
hollow; error is red)
• Name + Working pill (when active)
• Adapter · model · reasoning summary line, with an inline
Unavailable chip + HoverCard reason when the adapter binary
isn't on $PATH
• Italic last-user-message preview (line-clamp-2, leading quote
glyph) — same visual language as the rail
• Footer: 'X ago' + state chip (Asleep / Attention) OR a Resume
button (orange, with pulsing dot) when activeTurnId is non-null
Sort on the home grid is active-turn → recency. Pinning is NOT a
sort key here (and there's no pin indicator on the card) — pinning
belongs to the rail at /agents; the home page is action-oriented
and trusts active-turn + recency to surface the right agent.
Dead code removed:
• useAgentDashboard.ts (96 lines, no callers; subscribed to the
dead /claw/dashboard/stream from the OpenClaw-only era)
• useAgentCardData.ts (the dashboard-merge shim; passed undefined
every call so all enriched fields landed as undefined)
• AgentCard.tsx (AgentCardExpanded replaced by HomeAgentCard;
AgentCardCompact had no callers — the dock's compact mode was
never used)
• AgentCardData interface dropped from lib/agent-conversations/
types.ts; the new card consumes HarnessAgent directly
Visual language stays continuous between rail and grid: same
<AgentTile>, same <LivenessDot>, same italic-quote message
preview, same orange Resume button with a pulsing dot.
|
||
|
shivammittal274
|
df0f45dd29 |
Feat: eval debug dev ci (#869)
* chore(eval): instrument server startup to root-cause dev CI health-check timeouts Three diagnostics + one config swap to investigate why the eval-weekly workflow has been failing on dev since 2026-04-25 with "Server health check timed out" (every worker, every retry). Background: - Last successful weekly eval on dev: 2026-04-18 (sha |
||
|
Nikhil
|
edfc5c751c |
fix: align OpenClaw gateway image with VM cache (#868)
* fix: load OpenClaw gateway image from VM cache * fix: use container port for OpenClaw ACP bridge * fix: address review feedback for PR #868 |
||
|
Nikhil
|
471256f31c | fix: stop passing native permission flags to ACP adapters (#867) | ||
|
Nikhil
|
4c90ca696b | fix(agents): connect OpenClaw ACP inside gateway container (#866) | ||
|
Nikhil
|
f2ac87d7c3 |
feat: show created agents in sidepanel (#865)
* feat(agent): list created agents in sidepanel target catalog * feat(agent): show created agents in sidepanel selector * feat(server): add sidepanel chat route for created agents * feat(agent): route sidepanel agent sends by agent id * chore(agent): retire virtual sidepanel acp targets * fix: address review feedback for PR #865 |
||
|
Dani Akash
|
a228c278c6 |
feat(agents): background-resilient chat — turns survive tab disconnect (#863)
* feat(agents): decouple chat turn lifecycle from SSE response
Introduce a per-process ActiveTurnRegistry that owns each agent turn's
lifecycle and a ring-buffered event stream, so chat tabs that close,
refresh, or navigate away no longer cancel the in-flight turn. New
endpoints:
POST /agents/:id/chat starts a turn (now returns 409 when
one is already running, with the
active turnId for attaching)
GET /agents/:id/chat/active reports the running turn for a UI
that just mounted
GET /agents/:id/chat/stream subscribes to a turn; supports
Last-Event-ID resume via per-event
seq ids
POST /agents/:id/chat/cancel explicit cancel — fetch abort no
longer affects the underlying turn
The chat hook now captures X-Turn-Id, tracks lastSeq from SSE id lines,
re-attaches on mount when the server still has an active turn, and
routes Stop through the cancel endpoint. The runtime call uses the
registry's per-turn AbortController instead of the HTTP request signal,
which is the core decoupling that lets turns outlive their initiator.
* feat(agents): add ActiveTurnRegistry primitive backing the new chat lifecycle
The previous commit referenced these files in tests and the harness
service but global gitignore swallowed them on the first add.
The registry owns the per-turn ring buffer (drop-oldest, terminal frame
preserved), the per-turn AbortController, and subscriber fan-out used
by /chat/stream resume.
|
||
|
Dani Akash
|
e2ec1991cf |
feat(agents): redesign the agent command center for multi-adapter use (#861)
* feat(agents): redesign agent rail to match the rest of the app
Reshape the `/agents` page so it reads as a sibling of `/scheduled`
and `/soul` and adapts to the multi-adapter world (OpenClaw, Claude
Code, Codex). Visual scaffolding only in this commit — per-agent
liveness state ships as `unknown` until the server-side activity
tracker lands.
- New `AgentsHeader` mirrors `SoulHeader`/`ScheduledTasksHeader`:
accent bot tile, title, descriptive subtitle, "+ New Agent"
button. Replaces the loose top toolbar that mixed page-level and
OpenClaw-lifecycle controls.
- New `GatewayStatusBar` collects the OpenClaw lifecycle pills
(running, control plane connected) plus the Terminal/Refresh
affordances into a single labeled bar that only renders when the
gateway is running AND there is at least one OpenClaw agent in
the merged list.
- New `AgentRowCard` per agent: adapter tile with liveness dot,
name + status badge, adapter/model/reasoning chips, last-used
relative time + truncated workspace path, primary "Chat" button,
overflow menu (Copy id / Rename* / Reset history* / Delete).
Rename + Reset are disabled with "coming soon" tooltips until
the corresponding endpoints ship; Delete is hidden for the
protected `main` agent.
- New `AgentsEmptyState` mirrors the scheduled-tasks empty card.
- New `AdapterIcon` + `LivenessDot` + `agent-display.helpers.ts`
keep the row card focused on layout; helpers cover display name
fallbacks for legacy `oc-<uuid>` titles, workspace label rules,
and a tiny relative-time formatter.
- `AgentList` now sorts by `lastUsedAt` desc with `null`s falling
to the bottom; the gateway's `main` agent is pinned to the top
only while it has zero turns so a fresh install has an obvious
starting point. The list also threads a per-agent activity map
so future commits can light up working/idle/asleep without
reshuffling the API.
- `AgentsPage` swaps to the standard `fade-in slide-in-from-bottom-5
animate-in space-y-6 duration-500` shell and threads a
`harnessAgentLookup` Map down to the row card so adapter chips
and reasoning effort render correctly without a re-fetch.
* feat(agents): wire per-agent liveness end-to-end into the rail
Closes the placeholder `unknown` dot from the redesign's first
commit. The rail now shows real working / idle / asleep / error
states per agent, with `lastUsedAt` driving the recency sort.
Server side:
- `AgentHarnessService` keeps an in-memory activity tracker keyed
by agentId. `notifyTurnStarted` flips an entry to `working`,
`notifyTurnEnded({ok})` either drops it (success) or pins it to
`error` (failure / error event).
- `send()` wraps the runtime stream so the lifecycle hook fires
exactly once on natural close, error event, downstream cancel,
or thrown setup. The runtime itself stays unchanged — fork is
contained at the harness layer.
- New `listAgentsWithActivity()` method enriches every agent with
`{ status, lastUsedAt }`. lastUsedAt is read from the acpx
session record's last persisted item via `runtime.getHistory`,
so it survives server restart even though the activity map
doesn't.
- Status derivation: `working`/`error` take precedence; otherwise
timestamp-based — `idle` until 15 min of silence, then `asleep`.
Never-used agents resolve to `idle` (asleep implies "was active,
went quiet").
- `GET /agents` returns the enriched shape.
Client side:
- `HarnessAgent` UI type extended with optional `status` +
`lastUsedAt` so older deployments still typecheck.
- `useHarnessAgents` flips on `refetchInterval: 5_000` (with
`refetchIntervalInBackground: false` so hidden tabs go quiet)
so the per-row dots and last-used copy stay fresh without a
websocket.
- `AgentsPage` builds an activity map from the harness listing
response and threads it into `AgentList` → `AgentRowCard`. The
sort by `lastUsedAt` desc (already in the row card) now has
real data to operate on.
Tests:
- New `marks an agent working while a turn streams and idle once
it ends` exercises the wrap; uses a held upstream stream so
the in-flight `working` state is observable.
- New `flips to error when a turn emits an error event`.
* fix(agents): dedupe agent rail when /claw/agents and /agents share an id
The agents page was rendering every OpenClaw agent twice — once from
the legacy `/claw/agents` listing (`useOpenClawAgents`) and once from
the harness `/agents` listing (`useHarnessAgents`). Post Step 9
backfill the harness store contains every gateway agent, so the
overlap is the rule, not the exception.
Mirror the dedup the chat-panel layout already does: when a gateway
agent's id appears in the harness listing, drop the legacy entry and
keep the harness one (it has adapter/model/reasoning/status/lastUsedAt
the chat path actually consumes).
* feat(agents): swap GatewayStatusBar refresh icon for a Restart Gateway button + tooltips
The manual refresh became redundant once `useHarnessAgents` and
`useOpenClawStatus` started polling on a 5s interval — every visible
field self-refreshes within seconds. The previous AgentsPageHeader
had a real Restart action that the redesign dropped; reinstate it on
the bar so a wedged gateway is one click away again.
- GatewayStatusBar: dropped the `RotateCcw` refresh icon and the
`onRefresh` prop. Added `onRestart` + `actionInProgress` props;
the button shows a spinner while a gateway lifecycle mutation is
in flight.
- Both Terminal and Restart Gateway buttons get tooltips explaining
what they do — Terminal as a power-user shell escape hatch,
Restart for unsticking a wedged gateway or after manual config
edits.
- AgentsPage: drop the now-unused `refreshAll` helper and the
`refetchStatus`/`refetchAdapters`/`refetchOpenClawAgents`
destructures it depended on. Wire `restartOpenClaw` (already
pulled from `useOpenClawMutations`) through
`runWithPageErrorHandling` like the legacy header did.
* feat(agents): consolidate gateway status into the /agents listing
Folds the gateway lifecycle snapshot into the harness listing so the
agents page polls one endpoint instead of two. Drops the dead
`/claw/status` call from the command center while keeping every UI
affordance the page already shipped (Running / Control plane
connected pills, GatewayStateCards setup/start prompts,
ControlPlaneAlert for degraded states).
Server side:
- `OpenClawProvisioner.getStatus()` (optional) — when wired, returns
the same `GatewayStatusSnapshot` shape `/claw/status` does.
- `AgentHarnessService.getGatewayStatus()` — best-effort wrapper
around the provisioner method; logs and swallows errors so a
transient gateway issue doesn't 500 the listing endpoint.
- `GET /agents` now returns `{agents, gateway}` in a single
`Promise.all`. Both fields are independent — agents enrichment
succeeds even if the gateway snapshot is null.
- `server.ts` wires `getOpenClawService().getStatus()` into the
provisioner accessor object alongside `createAgent` /
`removeAgent` / `listAgents`.
Client side:
- `useHarnessAgents` returns `{harnessAgents, gateway}` (plus the
legacy `agents` mapping). Same 5s `refetchInterval` as before —
one round-trip drives the per-row liveness AND the gateway pills.
- `AgentsPage` drops `useOpenClawStatus` entirely; `status` comes
from the harness query. Loader + error/lifecycle plumbing
rewired around the harness query's loading/error.
- `agents-page-utils.getInlineError` and `getAgentsLoading` lose
the now-redundant `statusError` / `statusLoading` /
`openClawAgentsEnabled` params.
The chat-panel layout (`agent-command-layout.tsx`) still consumes
`useOpenClawStatus(5000)` for now — left intact per the user's "only
the command center" scope. Folding that one in is a separate,
smaller pass once we're sure no regression slipped here.
* test(agents): teach the route fake service about the new listing shape
PR #861 CI surfaced two failures in tests/api/routes/agents.test.ts:
both call \`GET /agents\` and the route handler now invokes
\`service.listAgentsWithActivity()\` + \`service.getGatewayStatus()\`
which the fake created here didn't implement. Add both methods to
the fake (returning idle / null) and update the empty-list assertion
to expect the new \`{agents, gateway}\` envelope.
|
||
|
Dani Akash
|
0c84547e8f |
feat(agents): migrate OpenClaw chat onto the unified harness/ACP path (#859)
* chore(acp): smoke-test ACP capabilities against running gateway
Adds apps/server/scripts/acp-smoke.ts which spawns `openclaw acp`
inside the gateway container and exercises every method we plan to
depend on: initialize, newSession, prompt (text + image), cancel,
listSessions, loadSession.
SDK pinned to 0.19.1 (Bun's minimum-release-age policy blocks 0.20+
which were released < 7 days ago).
Findings (full notes in plan outcomes):
- promptCapabilities advertises image:true but the model does NOT see
image bytes — silently dropped at the bridge.
- sessionCapabilities advertises {list:{}} but session/list throws
"Method not found": stale capability advertising.
- loadSession works; replays user/assistant/thought text and
session_info/usage/commands updates. No tool_call replay, as
documented.
- cancel works end-to-end: stopReason=cancelled.
- closeSession/resumeSession are not on ClientSideConnection in
0.19.1; kill child to close, use loadSession for rebind.
Plan revisions triggered by spike are recorded in
plans/browseros-ai/BrowserOS/features/2026-04-28-2310-claude-code-acp-implementation-roadmap.md.
* chore(acp): re-run smoke on SDK 0.21.0 and add mode/config/auth scenarios
After bypassing Bun's minimum-release-age and upgrading the SDK to
0.21.0, restore the previously-skipped resume/close paths and add
three new scenarios: mode (setSessionMode), config (setSessionConfigOption,
correct configId field), and auth (authenticate noop).
Findings, all bridge-side (independent of SDK):
- session/list, session/resume, session/close all throw -32601 on
OpenClaw 2026.4.12 — capability advertising is stale.
- Image content blocks silently dropped; model never sees the bytes.
- setSessionMode and setSessionConfigOption work; latter requires
`configId` (not `optionId`) per the schema.
- loadSession replays user/assistant/thought text + session_info +
usage + available_commands; no tool_call replay (documented).
- authenticate is a noop on OpenClaw (no authMethods advertised).
Plan outcomes updated with full method-support matrix.
* chore(deps): promote @agentclientprotocol/sdk to a runtime dependency
The smoke script in apps/server/scripts/acp-smoke.ts used the SDK as
devDependency. The upcoming ACP bridge (apps/server/src/api/services/acp/)
needs it at runtime, not just for tooling. Move the entry from
devDependencies to dependencies, alphabetically first under @a*.
Pinned to 0.21.0 — same version the smoke script validated against.
README gains a small Dependencies note pointing at the future bridge
location.
No code changes yet. The bridge wiring lands in subsequent commits.
* fix(openclaw): wire LlmProvider.supportsImages through to OpenClaw model config
When BrowserOS sets up a custom OpenAI-compat provider on the gateway,
the agent UI's "Supports Image" flag (LlmProviderConfig.supportsImages)
was being dropped on the floor. As a result the persisted model entry
had no `input` field, OpenClaw defaulted it to ['text'], and image_url
content parts were silently stripped before the model saw them.
Fix:
- Extend OpenClawSetupInput / OpenClawAgentMutationInput on the agent
side (useOpenClaw.ts) and the route body schema + SetupInput +
createAgent input on the server side with `supportsImages?: boolean`.
- AgentsPage forwards `llmOption?.supportsImages` from the selected
LlmProviderConfig in both handleSetup and handleCreate.
- provider-map.resolveSupportedOpenClawProvider emits
`input: ['text', 'image']` on the model entry when the flag is
truthy; otherwise emits the explicit `['text']` so the value is
always pinned (avoids relying on OpenClaw's implicit default).
- applyBrowserosConfig adds `tools.media.image.enabled = true` to the
bootstrap batch so the gateway's image-understanding pipeline is
always wired up — per-model `input` still gates which models see
images, this just enables the global path.
ACP image content blocks are still dropped by the OpenClaw bridge —
that's a separate bridge bug, not addressed here. This commit
restores image support for the OpenAI-compat /v1/chat/completions
path that the upcoming ACP chat panel will use as a carve-out for
image-bearing prompts.
Existing custom-provider configs are NOT auto-migrated; users will
re-acquire image support either by re-running setup or by editing
their model entries' `input` field manually. A migration pass for
legacy installs is not in scope for this commit because the
"supportsImages" intent isn't recoverable from the persisted config
alone — the source of truth is the LlmProvider record on the agent
side.
* feat(agents): add OpenClaw to AgentAdapter union and catalog
Extends AgentAdapter to 'claude' | 'codex' | 'openclaw' and adds the
OpenClaw entry to AGENT_ADAPTER_CATALOG. The new entry has:
- defaultModelId: 'default' — OpenClaw's ACP bridge does not surface
per-session model selection (verified during the ACP spike), so
models live in the OpenClawService config, not in the adapter
catalog. AgentDefinition.modelId carries the gateway-side model
name for display only.
- models: [] — empty list signals "no per-session model picker" in
the UI; isSupportedAgentModel('openclaw', undefined|'default')
returns true via the existing fallback path.
- reasoningEfforts mirror OpenClaw's session-level `thought_level`
config option (off / minimal / low / medium / high / adaptive).
Also extends:
- isAgentAdapter type guard recognizes 'openclaw'
- HarnessAgentAdapter union on the extension side
- agents.test.ts createAgent fake type
- agent-catalog.test.ts asserts on the new entry, empty model list
passthrough behavior, and OpenClaw's reasoning effort set
Lockfile delta is the workspace SDK pin reconciling 0.20.0 (taken
from dev's lock) up to our package.json's 0.21.0 (added in
|
||
|
Nikhil
|
2ff5c12840 |
feat: add sidepanel ACP chat targets (#857)
* feat(agent): add sidepanel chat target catalog * feat(agent): show acp models in sidepanel selector * feat(server): adapt acp events to ui message streams * feat(server): add sidepanel acp chat route * feat(agent): route sidepanel chat through acp targets * chore: self-review fixes * fix: address review feedback for PR #857 |
||
|
Nikhil
|
d87422eea1 | fix: hide BrowserOS ACP wrapper in history (#856) | ||
|
Nikhil
|
1946ca0cf8 | chore: clean up unused agent sdk (#855) | ||
|
Nikhil
|
754f7d0e1d | test: cover terminal limactl resolver errors (#854) | ||
|
Nikhil
|
85bb3f7b42 | fix: avoid eager limactl resolution in server tests (#853) | ||
|
Nikhil
|
cb32b8191d |
fix: show rich ACP harness history from ACPX (#852)
* fix: load ACP harness history from ACPX * fix: address ACP history review comments |