Compare commits

...

3 Commits

Author SHA1 Message Date
Nikhil Sonti
239dcecb9d fix: address limactl entitlement test review 2026-04-26 13:34:48 -07:00
Nikhil Sonti
983faf5375 fix: sign limactl with VZ entitlement 2026-04-26 13:23:16 -07:00
Dani Akash
0035893f33 feat: dashboard API, JSONL reader, and OpenClaw observer for enriched home page (#810)
* feat: draft agent chat ui exploration

* feat: refine agent chat ui draft

* feat: remove outer frame from agent chat workspace

* fix: offset agent chat for app sidebar

* fix: simplify agent conversation shell

* fix: remove redundant chat header actions

* fix: unify agent conversation headers

* fix: tighten agent chat spacing

* fix: bound agent chat composer height

* fix: remove agent chat page inset

* fix: align agent header height with sidepanel

* fix: center agent composer resting state

* fix: anchor multiline composer controls

* fix: remove focus grid from agent home

* fix: remove redundant agent home header

* fix: constrain home agent composer

* fix: match home composer default posture

* feat: add openclaw chat history APIs

* feat: add claw chat history hydration

* fix: stabilize claw chat viewport layout

* fix: use conversation scroll base for claw chat

* refactor: split claw chat controller responsibilities

* fix: keep active agent turns in memory

* fix: normalize openclaw chat sessions

* refactor: use HTTP client for agent history instead of CLI client

Replace the CLI-based getChatHistory() call in getAgentHistoryPage()
with the HTTP client's getSessionHistory() from PR #795. This uses
the direct HTTP transport to OpenClaw's /sessions/<key>/history
endpoint instead of shelling out through the CLI.

- Add filterHttpSessionHistoryMessages() for flat-string content format
- Add normalizeHttpHistoryMessages() for OpenClawSessionHistoryMessage shape
- Update getAgentHistoryPage() to call getSessionHistory() via httpClient
- Remove unused getChatHistory(), filterOpenClawSystemMessages(),
  normalizeChatHistoryMessages(), and getTextContent()
- Update test mocks from cliClient.getChatHistory to httpClient.getSessionHistory
- Update MutableOpenClawService type: chatClient -> httpClient

* fix: fetch all session messages by iterating OpenClaw pagination

OpenClaw's HTTP history endpoint returns a limited page by default.
When called without a limit, only the first ~27 messages were returned,
causing all newer conversation messages to be silently dropped.

Add fetchAllSessionMessages() that iterates through OpenClaw's cursor-
based pagination (200 messages per page) until hasMore is false, then
feeds the complete message list into the existing BrowserOS normalization
and in-memory pagination layer.

* refactor: migrate chat history from HTTP gateway to direct JSONL file reads

Replace the HTTP-based chat history pipeline (BrowserOS server → OpenClaw
gateway /sessions/:key/history pagination loop) with direct JSONL file reads
from the host filesystem via Lima's virtiofs mount.

- Add OpenClawJsonlReader that reads session JSONL files directly from
  ~/.browseros/vm/openclaw/.openclaw/agents/<id>/sessions/
- Replace fetchAllSessionMessages() HTTP pagination with single file read
- Replace CLI-based listSessions() with sessions.json file reads
- Make listSessions, resolveAgentSession, getAgentHistoryPage synchronous
- Remove unused toBrowserOSSession, filterHttpSessionHistoryMessages,
  normalizeHttpHistoryMessages helpers
- Update route handlers to drop unnecessary async/await
- Update tests to use temp JSONL files instead of mocked HTTP/CLI clients

* fix: restore async route handlers for test compatibility with mocked service

* fix: address review feedback — path traversal guard, lazy reader, exists flag

- Add safePath() to OpenClawJsonlReader that validates resolved paths stay
  within stateRoot, preventing path traversal via crafted agentId values
- Use lazy initialization for jsonlReader (nulled on rebuildRuntimeClients)
  instead of creating a new instance per property access
- Return exists: false from resolveSpecificAgentSession when no session
  matches instead of fabricating a ghost session with sessionId: ''

* feat: add dashboard API and enrich home page agent cards

Server:
- Add summarizeToolActivity() that converts tool events into natural
  language descriptions ("Browsed 3 pages, took 2 screenshots")
- Add getDashboard() to OpenClawService that aggregates per-agent stats
  from JSONL: latest message, activity summary, cost, session count
- Add GET /claw/dashboard endpoint

Client:
- Add useAgentDashboard() React Query hook (10s refetch, 5s stale)
- Rewrite useAgentCardData from async IndexedDB hook to pure
  buildAgentCardData() function merging agent entries with dashboard data
- Add activity summary and cost to AgentCardExpanded footer
- Add activitySummary and costUsd fields to AgentCardData type
- Remove IndexedDB dependency from the home page

* feat: add OpenClawObserver for real-time per-agent status via gateway WS

- Add OpenClawObserver that connects to the OpenClaw gateway WebSocket
  control plane and subscribes to chat broadcast events
- Track per-agent status in real time: working (streaming), idle (turn
  complete), error (run failed), with current tool name
- Auto-connect when gateway control plane becomes available, auto-
  reconnect on disconnect with 5s backoff
- Disconnect observer on stop/shutdown
- Wire live status + currentTool into getDashboard() response
- Update client: AgentOverview includes status + currentTool, card shows
  spinning loader + tool name when agent is working
- Status resolution: per-agent WS status takes precedence over gateway-
  level status for working/error states

* feat: add SSE dashboard stream for real-time agent status on home page

Server:
- Add GET /claw/dashboard/stream SSE endpoint that sends an initial
  snapshot then pushes per-agent status events as they arrive from
  the OpenClaw observer
- Add onAgentStatusChange() to OpenClawService exposing the observer's
  listener for the route layer
- Heartbeat every 15s to keep connections alive

Client:
- useAgentDashboard() now subscribes to EventSource at /claw/dashboard/stream
- SSE snapshot event hydrates the React Query cache immediately
- SSE status events patch individual agent status + currentTool in the
  cache without refetching — agent cards update instantly
- Polling fallback raised to 30s since SSE handles real-time

* fix: observer WS handshake — wait for challenge before sending connect

The OpenClaw gateway sends a connect.challenge event before accepting
the connect request. The observer was sending the connect request on
ws.open which raced with the challenge. Now waits for the challenge
event before sending the handshake.

Also add dangerouslyDisableDeviceAuth to the gateway setup config
batch so the observer can connect without device identity on new
installs.

* fix: JSONL reader falls back to most recent file when sessions.json is stale

OpenClaw's sessions.json can record a Pi session ID that doesn't match
the actual JSONL filename on disk. This happens after context compaction
or session restart — the JSONL file gets a new UUID but sessions.json
keeps the old one.

Previously this caused history to silently disappear (the reader tried
to open a non-existent file and returned empty). Now resolveJsonlPath()
checks if the mapped file exists and, when it doesn't, scans the
sessions directory for the most recently modified .jsonl file as a
fallback.

* feat: add ClawSession state machine for reliable per-agent status

The OpenClawObserver only knows about status changes it witnesses via
WS events. If an agent was already running when the observer connected,
or after a reconnect, statuses were stuck at "unknown".

ClawSession is an in-memory state machine that solves this:

1. Seeds from JSONL on first control plane call — reads the latest
   events for each agent and infers working/idle. A session is "working"
   if the last event is a user.message with no subsequent agent.message,
   or an agent.tool_use with no matching agent.tool_result.

2. Receives live transitions from the WS observer — the observer now
   delegates all state management to ClawSession instead of maintaining
   its own status map.

3. Applies a 5-minute staleness threshold — if the last JSONL event
   is older than 5 minutes, assume idle (handles agent crashes).

Consumers (SSE stream, dashboard endpoint) read from ClawSession and
get correct state from the first call — no "unknown" period.

* fix: remove staleTime so dashboard refetches on every mount

* fix: reset stale working status on WS disconnect, eliminate redundant JSONL reads

- Observer resets all "working" agents to "unknown" when the WS closes,
  preventing agents from appearing stuck as Working indefinitely after
  a gateway restart. ClawSession re-seeds correct state on reconnect.

- getDashboard() now derives latestAgentMessage and cost from the
  already-loaded events array for the latest session instead of calling
  latestAgentMessage() and getSessionStats() which each re-read the
  same JSONL file. Reduces file reads from 3x to 1x per agent.
2026-04-25 19:03:03 +05:30
13 changed files with 1061 additions and 80 deletions

View File

@@ -1,4 +1,4 @@
import { Bot } from 'lucide-react'
import { Bot, Loader2, Wrench } from 'lucide-react'
import type { FC } from 'react'
import type { AgentCardData } from '@/lib/agent-conversations/types'
import { cn } from '@/lib/utils'
@@ -32,6 +32,11 @@ function getStatusTone(status: AgentCardData['status']): string {
return 'bg-emerald-500'
}
function formatCost(usd: number): string {
if (usd < 0.005) return `$${usd.toFixed(4)}`
return `$${usd.toFixed(2)}`
}
export const AgentCardExpanded: FC<AgentCardProps> = ({
agent,
onClick,
@@ -81,9 +86,26 @@ export const AgentCardExpanded: FC<AgentCardProps> = ({
</p>
</div>
<div className="mt-4 flex items-center justify-between gap-3 text-muted-foreground text-xs">
<span>{formatTimestamp(agent.lastMessageTimestamp)}</span>
<span>Open conversation</span>
<div className="mt-4 space-y-1.5 text-muted-foreground text-xs">
<div className="flex items-center justify-between gap-3">
<span>{formatTimestamp(agent.lastMessageTimestamp)}</span>
{agent.costUsd ? (
<span className="tabular-nums opacity-70">
{formatCost(agent.costUsd)}
</span>
) : null}
</div>
{agent.status === 'working' && agent.currentTool ? (
<div className="flex items-center gap-1.5 text-[var(--accent-orange)]/70">
<Loader2 className="size-3 shrink-0 animate-spin" />
<span className="truncate">{agent.currentTool}</span>
</div>
) : agent.activitySummary ? (
<div className="flex items-center gap-1.5 text-muted-foreground/60">
<Wrench className="size-3 shrink-0" />
<span className="truncate">{agent.activitySummary}</span>
</div>
) : null}
</div>
</button>
)

View File

@@ -8,10 +8,12 @@ import type { AgentEntry } from '@/entrypoints/app/agents/useOpenClaw'
import { ImportDataHint } from '@/entrypoints/newtab/index/ImportDataHint'
import { SignInHint } from '@/entrypoints/newtab/index/SignInHint'
import { useActiveHint } from '@/entrypoints/newtab/index/useActiveHint'
import type { AgentCardData } from '@/lib/agent-conversations/types'
import { AgentCardDock } from './AgentCardDock'
import { useAgentCommandData } from './agent-command-layout'
import { ConversationInput } from './ConversationInput'
import { useAgentCardData } from './useAgentCardData'
import { buildAgentCardData } from './useAgentCardData'
import { useAgentDashboard } from './useAgentDashboard'
function AgentCommandSetupState({
onOpenAgents,
@@ -95,7 +97,7 @@ function RecentThreads({
onSelectAgent,
}: {
activeAgentId?: string | null
agents: ReturnType<typeof useAgentCardData>
agents: AgentCardData[]
onOpenAgents: () => void
onSelectAgent: (agentId: string) => void
}) {
@@ -134,7 +136,8 @@ export const AgentCommandHome: FC = () => {
const activeHint = useActiveHint()
const { status, agents } = useAgentCommandData()
const [selectedAgentId, setSelectedAgentId] = useState<string | null>(null)
const cardData = useAgentCardData(agents, status?.status)
const { data: dashboard } = useAgentDashboard(status?.status === 'running')
const cardData = buildAgentCardData(agents, status?.status, dashboard?.agents)
useEffect(() => {
if (agents.length === 0) {

View File

@@ -1,69 +1,50 @@
import { useEffect, useState } from 'react'
import {
type AgentEntry,
getModelDisplayName,
type OpenClawStatus,
} from '@/entrypoints/app/agents/useOpenClaw'
import { getLatestConversation } from '@/lib/agent-conversations/storage'
import type { AgentCardData } from '@/lib/agent-conversations/types'
import type { AgentOverview } from './useAgentDashboard'
function getAgentStatusTone(
status: OpenClawStatus['status'] | undefined,
function resolveAgentStatus(
gatewayStatus: OpenClawStatus['status'] | undefined,
liveStatus: AgentOverview['status'] | undefined,
): AgentCardData['status'] {
if (status === 'error') return 'error'
if (status === 'starting') return 'working'
// Gateway-level errors take precedence
if (gatewayStatus === 'error') return 'error'
if (gatewayStatus === 'starting') return 'working'
// Per-agent live status from the WS observer
if (liveStatus === 'working') return 'working'
if (liveStatus === 'error') return 'error'
return 'idle'
}
async function getAgentCardData(
agent: AgentEntry,
status: OpenClawStatus['status'] | undefined,
): Promise<AgentCardData> {
const conversation = await getLatestConversation(agent.agentId)
const lastTurn = conversation?.turns[conversation.turns.length - 1]
const lastTextPart = lastTurn?.parts.findLast((part) => part.kind === 'text')
return {
agentId: agent.agentId,
name: agent.name,
model: getModelDisplayName(agent.model),
status: getAgentStatusTone(status),
lastMessage:
lastTextPart?.kind === 'text'
? lastTextPart.text.slice(0, 120)
: undefined,
lastMessageTimestamp: lastTurn?.timestamp,
}
}
export function useAgentCardData(
/**
* Build agent card display data by merging the raw agent entries from
* the gateway with enriched overview data from the dashboard API.
*
* Pure function — no hooks, no IndexedDB, no async.
*/
export function buildAgentCardData(
agents: AgentEntry[],
status: OpenClawStatus['status'] | undefined,
) {
const [cardData, setCardData] = useState<AgentCardData[]>([])
dashboard: AgentOverview[] | undefined,
): AgentCardData[] {
return agents.map((agent) => {
const overview = dashboard?.find((d) => d.agentId === agent.agentId)
useEffect(() => {
let active = true
const loadCardData = async () => {
const nextCardData = await Promise.all(
agents.map((agent) => getAgentCardData(agent, status)),
)
if (active) {
setCardData(nextCardData)
}
return {
agentId: agent.agentId,
name: agent.name,
model: getModelDisplayName(agent.model),
status: resolveAgentStatus(status, overview?.status),
lastMessage: overview?.latestMessage?.slice(0, 200) ?? undefined,
lastMessageTimestamp: overview?.latestMessageAt ?? undefined,
activitySummary: overview?.activitySummary ?? undefined,
currentTool: overview?.currentTool ?? undefined,
costUsd: overview?.totalCostUsd ?? undefined,
}
if (agents.length > 0) {
void loadCardData()
} else {
setCardData([])
}
return () => {
active = false
}
}, [agents, status])
return cardData
})
}

View File

@@ -0,0 +1,95 @@
import { useQuery, useQueryClient } from '@tanstack/react-query'
import { useEffect } from 'react'
import { useAgentServerUrl } from '@/lib/browseros/useBrowserOSProviders'
export interface AgentOverview {
agentId: string
status: 'working' | 'idle' | 'error' | 'unknown'
latestMessage: string | null
latestMessageAt: number | null
activitySummary: string | null
currentTool: string | null
totalCostUsd: number
sessionCount: number
}
export interface DashboardResponse {
agents: AgentOverview[]
summary: {
totalAgents: number
totalCostUsd: number
}
}
interface StatusEvent {
agentId: string
status: AgentOverview['status']
currentTool: string | null
error: string | null
timestamp: number
}
const DASHBOARD_QUERY_KEY = ['claw', 'dashboard']
export function useAgentDashboard(enabled: boolean) {
const { baseUrl, isLoading: urlLoading } = useAgentServerUrl()
const queryClient = useQueryClient()
const ready = enabled && Boolean(baseUrl) && !urlLoading
// Initial data load + periodic refresh as fallback
const query = useQuery<DashboardResponse>({
queryKey: [...DASHBOARD_QUERY_KEY, baseUrl],
queryFn: async () => {
const url = new URL('/claw/dashboard', baseUrl as string)
const response = await fetch(url.toString())
if (!response.ok) throw new Error('Failed to fetch dashboard')
return response.json()
},
enabled: ready,
})
// SSE subscription for real-time status patches
useEffect(() => {
if (!ready || !baseUrl) return
const streamUrl = new URL('/claw/dashboard/stream', baseUrl)
const eventSource = new EventSource(streamUrl.toString())
eventSource.addEventListener('snapshot', (event) => {
try {
const dashboard = JSON.parse(event.data) as DashboardResponse
queryClient.setQueryData([...DASHBOARD_QUERY_KEY, baseUrl], dashboard)
} catch {}
})
eventSource.addEventListener('status', (event) => {
try {
const status = JSON.parse(event.data) as StatusEvent
queryClient.setQueryData<DashboardResponse>(
[...DASHBOARD_QUERY_KEY, baseUrl],
(prev) => {
if (!prev) return prev
return {
...prev,
agents: prev.agents.map((agent) =>
agent.agentId === status.agentId
? {
...agent,
status: status.status,
currentTool: status.currentTool,
}
: agent,
),
}
},
)
} catch {}
})
return () => {
eventSource.close()
}
}, [ready, baseUrl, queryClient])
return query
}

View File

@@ -50,4 +50,7 @@ export interface AgentCardData {
status: 'idle' | 'working' | 'error'
lastMessage?: string
lastMessageTimestamp?: number
activitySummary?: string
currentTool?: string
costUsd?: number
}

View File

@@ -284,6 +284,72 @@ export function createOpenClawRoutes() {
}
})
.get('/dashboard', (c) => {
try {
const dashboard = getOpenClawService().getDashboard()
return c.json(dashboard)
} catch (err) {
const message = err instanceof Error ? err.message : String(err)
return c.json({ error: message }, 500)
}
})
.get('/dashboard/stream', (c) => {
c.header('Content-Type', 'text/event-stream')
c.header('Cache-Control', 'no-cache')
c.header('Connection', 'keep-alive')
return stream(c, async (s) => {
const encoder = new TextEncoder()
// Send initial snapshot
try {
const dashboard = getOpenClawService().getDashboard()
await s.write(
encoder.encode(
`event: snapshot\ndata: ${JSON.stringify(dashboard)}\n\n`,
),
)
} catch {}
// Subscribe to live status changes
const unsubscribe = getOpenClawService().onAgentStatusChange(
(agentId, entry) => {
const event = {
agentId,
status: entry.status,
currentTool: entry.currentTool,
error: entry.error,
timestamp: entry.lastEventAt,
}
s.write(
encoder.encode(
`event: status\ndata: ${JSON.stringify(event)}\n\n`,
),
).catch(() => {})
},
)
// Heartbeat every 15s to keep connection alive
const heartbeat = setInterval(() => {
s.write(
encoder.encode(
`event: heartbeat\ndata: ${JSON.stringify({ ts: Date.now() })}\n\n`,
),
).catch(() => {})
}, 15_000)
// Wait until client disconnects
try {
await new Promise<void>((resolve) => {
s.onAbort(() => resolve())
})
} finally {
unsubscribe()
clearInterval(heartbeat)
}
})
})
.post('/agents/:id/chat', async (c) => {
const { id } = c.req.param()
const body = await c.req.json<{

View File

@@ -0,0 +1,267 @@
/**
* @license
* Copyright 2025 BrowserOS
* SPDX-License-Identifier: AGPL-3.0-or-later
*
* In-memory state machine tracking the live status of every OpenClaw agent
* session. Acts as the single source of truth for "is agent X running?"
*
* Two data sources feed it:
* 1. JSONL files (seed) — on init, reads the latest events for each agent
* to infer whether a session is running or idle. This handles the case
* where an agent was already mid-task when BrowserOS started.
* 2. Gateway WS events (live) — the OpenClawObserver pipes chat broadcast
* events into this state machine for real-time transitions.
*
* Consumers (SSE streams, dashboard endpoint) read from this class and get
* correct state from the first call — no "unknown" period while waiting for
* the first WS event.
*/
import { logger } from '../../../lib/logger'
import type { ClawEvent, OpenClawJsonlReader } from './openclaw-jsonl-reader'
// ---------------------------------------------------------------------------
// Types
// ---------------------------------------------------------------------------
export type AgentLiveStatus = 'working' | 'idle' | 'error' | 'unknown'
export interface AgentSessionState {
status: AgentLiveStatus
sessionKey: string | null
lastEventAt: number
currentTool: string | null
error: string | null
}
export type SessionStateListener = (
agentId: string,
state: AgentSessionState,
) => void
// ---------------------------------------------------------------------------
// State machine
// ---------------------------------------------------------------------------
export class ClawSession {
private readonly states = new Map<string, AgentSessionState>()
private readonly listeners = new Set<SessionStateListener>()
private seeded = false
/**
* Seed the state machine from JSONL files. Call this once when the
* gateway becomes ready. For each agent, reads the latest session's
* events and infers whether the agent is currently working or idle.
*
* A session is considered "working" if:
* - The last message-type event is a user.message (agent hasn't replied yet)
* - The last event is an agent.tool_use without a matching agent.tool_result
*
* Otherwise it's "idle".
*/
seedFromJsonl(reader: OpenClawJsonlReader): void {
const agents = reader.listAgents()
for (const agentId of agents) {
const sessions = reader.listSessions(agentId)
if (sessions.length === 0) continue
const latestSession = sessions[0]
const events = reader.listBySession(agentId, latestSession.key)
const state = inferStateFromEvents(events, latestSession.key)
this.states.set(agentId, state)
if (state.status === 'working') {
logger.info('ClawSession seed: agent is working', {
agentId,
currentTool: state.currentTool,
})
}
}
this.seeded = true
logger.info('ClawSession seeded from JSONL', {
agentCount: agents.length,
working: [...this.states.values()].filter((s) => s.status === 'working')
.length,
})
}
/** Whether seedFromJsonl() has been called. */
isSeeded(): boolean {
return this.seeded
}
/** Get the current state of an agent. */
getState(agentId: string): AgentSessionState {
return (
this.states.get(agentId) ?? {
status: 'unknown',
sessionKey: null,
lastEventAt: 0,
currentTool: null,
error: null,
}
)
}
/** Get all tracked agent states. */
getAllStates(): Map<string, AgentSessionState> {
return this.states
}
/**
* Transition an agent's state. Called by the OpenClawObserver when
* a chat WS event arrives.
*/
transition(
agentId: string,
status: AgentLiveStatus,
update: {
sessionKey?: string | null
currentTool?: string | null
error?: string | null
} = {},
): void {
const prev = this.states.get(agentId)
const entry: AgentSessionState = {
status,
sessionKey: update.sessionKey ?? prev?.sessionKey ?? null,
lastEventAt: Date.now(),
currentTool:
status === 'working'
? (update.currentTool ?? prev?.currentTool ?? null)
: null,
error: status === 'error' ? (update.error ?? null) : null,
}
this.states.set(agentId, entry)
for (const listener of this.listeners) {
try {
listener(agentId, entry)
} catch {}
}
}
/** Subscribe to state changes. Returns unsubscribe function. */
onStateChange(listener: SessionStateListener): () => void {
this.listeners.add(listener)
return () => this.listeners.delete(listener)
}
}
// ---------------------------------------------------------------------------
// JSONL state inference
// ---------------------------------------------------------------------------
/**
* Infer the current session state from JSONL events.
*
* The key insight: if the last meaningful event in the JSONL is a
* user.message with no subsequent agent.message, the agent is still
* processing (working). Similarly, an agent.tool_use without a matching
* agent.tool_result means the agent is mid-tool-call.
*
* We also check event recency — if the last event was more than 5 minutes
* ago, we assume the session is idle regardless (handles cases where the
* agent crashed without writing a final event).
*/
function inferStateFromEvents(
events: ClawEvent[],
sessionKey: string,
): AgentSessionState {
if (events.length === 0) {
return {
status: 'idle',
sessionKey,
lastEventAt: 0,
currentTool: null,
error: null,
}
}
const lastEvent = events[events.length - 1]!
const lastEventAt = lastEvent.createdAt
// If the last event is older than 5 minutes, assume idle — the agent
// likely finished or crashed without writing a final event.
const STALE_THRESHOLD_MS = 5 * 60 * 1000
if (Date.now() - lastEventAt > STALE_THRESHOLD_MS) {
return {
status: 'idle',
sessionKey,
lastEventAt,
currentTool: null,
error: null,
}
}
// Walk backward to find the last meaningful event
let lastUserMessageIdx = -1
let lastAssistantMessageIdx = -1
let lastToolUseIdx = -1
let lastToolResultIdx = -1
for (let i = events.length - 1; i >= 0; i--) {
const e = events[i]!
if (e.type === 'user.message' && lastUserMessageIdx === -1) {
lastUserMessageIdx = i
}
if (e.type === 'agent.message' && lastAssistantMessageIdx === -1) {
lastAssistantMessageIdx = i
}
if (e.type === 'agent.tool_use' && lastToolUseIdx === -1) {
lastToolUseIdx = i
}
if (e.type === 'agent.tool_result' && lastToolResultIdx === -1) {
lastToolResultIdx = i
}
// Stop scanning once we've found all event types
if (
lastUserMessageIdx !== -1 &&
lastAssistantMessageIdx !== -1 &&
lastToolUseIdx !== -1 &&
lastToolResultIdx !== -1
) {
break
}
}
// Agent is working if the last user message came AFTER the last
// assistant message — the agent hasn't replied yet
if (
lastUserMessageIdx !== -1 &&
lastUserMessageIdx > lastAssistantMessageIdx
) {
return {
status: 'working',
sessionKey,
lastEventAt,
currentTool: null,
error: null,
}
}
// Agent is working if there's a tool_use without a subsequent tool_result
if (lastToolUseIdx !== -1 && lastToolUseIdx > lastToolResultIdx) {
const toolEvent = events[lastToolUseIdx]!
return {
status: 'working',
sessionKey,
lastEventAt,
currentTool: toolEvent.toolName ?? null,
error: null,
}
}
return {
status: 'idle',
sessionKey,
lastEventAt,
currentTool: null,
error: null,
}
}

View File

@@ -4,7 +4,7 @@
* SPDX-License-Identifier: AGPL-3.0-or-later
*/
import { readdirSync, readFileSync } from 'node:fs'
import { existsSync, readdirSync, readFileSync, statSync } from 'node:fs'
import { resolve } from 'node:path'
// ---------------------------------------------------------------------------
@@ -148,12 +148,19 @@ export class OpenClawJsonlReader {
}
}
/** Read and parse all events from a session's JSONL file. */
/**
* Read and parse all events from a session's JSONL file.
*
* Uses resolveJsonlPath() which handles a known OpenClaw quirk: the
* Pi session ID recorded in sessions.json can drift from the actual
* JSONL filename after context compaction or session restart. When the
* mapped ID doesn't match a file on disk, we fall back to the most
* recently modified JSONL in the agent's sessions directory.
*/
listBySession(agentId: string, sessionKey: string): ClawEvent[] {
const piSessionId = this.resolvePiSessionId(agentId, sessionKey)
if (!piSessionId) return []
const filePath = this.resolveJsonlPath(agentId, sessionKey)
if (!filePath) return []
const filePath = this.jsonlPath(agentId, piSessionId)
let raw: string
try {
raw = readFileSync(filePath, 'utf8')
@@ -255,31 +262,91 @@ export class OpenClawJsonlReader {
}
}
private resolvePiSessionId(
agentId: string,
sessionKey: string,
): string | undefined {
/**
* Resolve the path to a session's JSONL file. Tries the sessions.json
* mapping first (fast), then falls back to scanning the directory for
* the most recently modified JSONL file when the mapped ID doesn't
* match an actual file on disk.
*
* This fallback handles a known OpenClaw behavior where the Pi session
* ID in sessions.json can become stale after context compaction or
* session restart — the JSONL file on disk has a different UUID than
* what sessions.json records.
*/
private resolveJsonlPath(agentId: string, sessionKey: string): string | null {
const sessionsJson = this.readSessionsJson(agentId)
if (!sessionsJson) return undefined
if (!sessionsJson) return null
// Try exact key match first
// Try exact key match in sessions.json
let resolvedId: string | undefined
const entry = sessionsJson[sessionKey]
if (entry && typeof entry.sessionId === 'string') {
return entry.sessionId
resolvedId = entry.sessionId
}
// Try matching by scanning all keys (handles key format variations)
for (const [key, value] of Object.entries(sessionsJson)) {
if (key === sessionKey || key.endsWith(`:${sessionKey}`)) {
if (typeof value.sessionId === 'string') return value.sessionId
if (!resolvedId) {
for (const [key, value] of Object.entries(sessionsJson)) {
if (key === sessionKey || key.endsWith(`:${sessionKey}`)) {
if (typeof value.sessionId === 'string') {
resolvedId = value.sessionId
break
}
}
}
}
return undefined
// If we found a sessionId and the file exists, use it
if (resolvedId) {
const path = this.safePath(
'agents',
agentId,
'sessions',
`${resolvedId}.jsonl`,
)
if (existsSync(path)) return path
}
// Fallback: scan the sessions directory for the most recent JSONL
// file. This handles stale sessions.json entries where the Pi
// session ID doesn't match the actual file on disk.
return this.findMostRecentJsonl(agentId)
}
private jsonlPath(agentId: string, piSessionId: string): string {
return this.safePath('agents', agentId, 'sessions', `${piSessionId}.jsonl`)
/**
* Scan the sessions directory and return the path to the most recently
* modified JSONL file. Used as a fallback when sessions.json points to
* a non-existent file.
*/
private findMostRecentJsonl(agentId: string): string | null {
let sessionsDir: string
try {
sessionsDir = this.safePath('agents', agentId, 'sessions')
} catch {
return null
}
let names: string[]
try {
names = readdirSync(sessionsDir).filter(
(n): n is string => typeof n === 'string' && n.endsWith('.jsonl'),
)
} catch {
return null
}
let best: { path: string; mtime: number } | null = null
for (const name of names) {
const fullPath = this.safePath('agents', agentId, 'sessions', name)
try {
const st = statSync(fullPath)
if (!best || st.mtimeMs > best.mtime) {
best = { path: fullPath, mtime: st.mtimeMs }
}
} catch {}
}
return best?.path ?? null
}
}
@@ -439,3 +506,51 @@ function combineModel(
if (!model) return undefined
return provider ? `${provider}/${model}` : model
}
// ---------------------------------------------------------------------------
// Tool activity summary
// ---------------------------------------------------------------------------
const TOOL_DESCRIPTIONS: Record<string, (count: number) => string> = {
browser_navigate: (n) => `Browsed ${n} page${n !== 1 ? 's' : ''}`,
browser_take_screenshot: (n) => `Took ${n} screenshot${n !== 1 ? 's' : ''}`,
browser_click: (n) => `Clicked ${n} element${n !== 1 ? 's' : ''}`,
browser_fill: (n) => `Filled ${n} field${n !== 1 ? 's' : ''}`,
browser_type: (n) => `Typed in ${n} field${n !== 1 ? 's' : ''}`,
google_calendar_list_events: (n) =>
n > 1 ? `Checked calendar ${n} times` : 'Checked calendar',
gmail_search: (n) => (n > 1 ? `Searched email ${n} times` : 'Searched email'),
gmail_send: (n) => `Sent ${n} email${n !== 1 ? 's' : ''}`,
slack_post_message: (n) => `Sent ${n} Slack message${n !== 1 ? 's' : ''}`,
file_write: (n) => `Wrote ${n} file${n !== 1 ? 's' : ''}`,
file_read: (n) => `Read ${n} file${n !== 1 ? 's' : ''}`,
}
function defaultToolDescription(toolName: string, count: number): string {
const short = toolName
.replace(/^(browser_|google_|mcp_)/, '')
.replaceAll('_', ' ')
return count > 1 ? `Used ${short} ${count} times` : `Used ${short}`
}
/**
* Convert raw tool-use events into a human-readable activity summary.
*
* Example output: "Browsed 3 pages, took 2 screenshots"
*/
export function summarizeToolActivity(events: ClawEvent[]): string | null {
const toolCounts = new Map<string, number>()
for (const e of events) {
if (e.type === 'agent.tool_use' && e.toolName) {
toolCounts.set(e.toolName, (toolCounts.get(e.toolName) ?? 0) + 1)
}
}
if (toolCounts.size === 0) return null
const parts: string[] = []
for (const [tool, count] of toolCounts) {
const describe = TOOL_DESCRIPTIONS[tool]
parts.push(describe ? describe(count) : defaultToolDescription(tool, count))
}
return parts.join(', ')
}

View File

@@ -0,0 +1,276 @@
/**
* @license
* Copyright 2025 BrowserOS
* SPDX-License-Identifier: AGPL-3.0-or-later
*
* Connects to the OpenClaw gateway's WebSocket control plane and pipes
* chat broadcast events into a ClawSession state machine. The observer
* is a transport layer only — it handles the WS connection lifecycle
* (connect, handshake, reconnect) and delegates all state management
* to ClawSession.
*/
import WebSocket from 'ws'
import { logger } from '../../../lib/logger'
import type { ClawSession } from './claw-session'
// ---------------------------------------------------------------------------
// Protocol types (subset of OpenClaw gateway protocol v3)
// ---------------------------------------------------------------------------
const PROTOCOL_VERSION = 3
const HANDSHAKE_REQUEST_ID = 'connect'
const RECONNECT_DELAY_MS = 5_000
const CONNECT_TIMEOUT_MS = 10_000
interface RequestFrame {
type: 'req'
id: string
method: string
params: Record<string, unknown>
}
type IncomingFrame =
| { type: 'res'; id: string; ok: true; payload?: unknown }
| {
type: 'res'
id: string
ok: false
error: { code: string; message: string }
}
| { type: 'event'; event: string; payload?: unknown }
// ---------------------------------------------------------------------------
// Observer
// ---------------------------------------------------------------------------
export class OpenClawObserver {
private ws: WebSocket | null = null
private reconnectTimer: ReturnType<typeof setTimeout> | null = null
private connected = false
private closed = false
private gatewayUrl: string | null = null
private gatewayToken: string | null = null
constructor(private readonly session: ClawSession) {}
/** Start observing the gateway at the given URL with the given token. */
connect(gatewayUrl: string, token: string): void {
this.gatewayUrl = gatewayUrl
this.gatewayToken = token
this.closed = false
this.doConnect()
}
/** Stop observing and close the WebSocket. */
disconnect(): void {
this.closed = true
this.clearReconnect()
if (this.ws) {
try {
this.ws.close()
} catch {}
this.ws = null
}
this.connected = false
}
/** Whether the observer has an active WS connection. */
isConnected(): boolean {
return this.connected
}
// ── Private ─────────────────────────────────────────────────────────
private doConnect(): void {
if (this.closed || !this.gatewayUrl || !this.gatewayToken) return
const wsUrl = this.gatewayUrl
.replace(/^http:\/\//, 'ws://')
.replace(/^https:\/\//, 'wss://')
logger.debug('OpenClaw observer connecting', { url: wsUrl })
const ws = new WebSocket(wsUrl)
this.ws = ws
const connectTimeout = setTimeout(() => {
logger.warn('OpenClaw observer handshake timeout')
ws.terminate()
}, CONNECT_TIMEOUT_MS)
let handshakeSent = false
ws.on('message', (raw) => {
let frame: IncomingFrame
try {
frame = JSON.parse(raw.toString('utf8')) as IncomingFrame
} catch {
return
}
// The gateway sends a connect.challenge event before accepting
// the connect request. Send the handshake after receiving it.
if (
frame.type === 'event' &&
frame.event === 'connect.challenge' &&
!handshakeSent
) {
handshakeSent = true
const connectReq: RequestFrame = {
type: 'req',
id: HANDSHAKE_REQUEST_ID,
method: 'connect',
params: {
minProtocol: PROTOCOL_VERSION,
maxProtocol: PROTOCOL_VERSION,
client: {
id: 'openclaw-tui',
displayName: 'browseros-observer',
version: '1.0.0',
platform: 'node',
mode: 'ui',
},
role: 'operator',
scopes: ['operator.read'],
auth: { token: this.gatewayToken },
},
}
ws.send(JSON.stringify(connectReq))
return
}
// Handshake response
if (frame.type === 'res' && frame.id === HANDSHAKE_REQUEST_ID) {
clearTimeout(connectTimeout)
if (frame.ok) {
this.connected = true
logger.info('OpenClaw observer connected')
} else {
logger.warn('OpenClaw observer handshake failed', {
error: frame.error,
})
ws.close()
}
return
}
// Broadcast events (only process after handshake completes)
if (frame.type === 'event' && this.connected) {
this.handleEvent(frame.event, frame.payload)
}
})
ws.on('close', () => {
clearTimeout(connectTimeout)
this.connected = false
this.ws = null
// Reset any agents stuck in "working" to "unknown" — we missed
// the final/end event because the WS closed mid-task. The
// ClawSession will re-infer correct state from JSONL when the
// observer reconnects and ensureObserverConnected() re-seeds.
for (const [agentId, state] of this.session.getAllStates()) {
if (state.status === 'working') {
this.session.transition(agentId, 'unknown')
}
}
if (!this.closed) {
logger.debug('OpenClaw observer disconnected, scheduling reconnect')
this.scheduleReconnect()
}
})
ws.on('error', (err) => {
clearTimeout(connectTimeout)
logger.debug('OpenClaw observer WS error', {
message: err.message,
})
})
}
private handleEvent(eventName: string, payload: unknown): void {
if (eventName === 'chat') {
this.handleChatEvent(payload)
}
}
/**
* Parse a gateway chat broadcast event and transition the ClawSession
* state machine accordingly.
*/
private handleChatEvent(payload: unknown): void {
if (!payload || typeof payload !== 'object') return
const p = payload as Record<string, unknown>
const sessionKey = typeof p.sessionKey === 'string' ? p.sessionKey : null
const state = typeof p.state === 'string' ? p.state : null
if (!sessionKey || !state) return
const agentId = extractAgentId(sessionKey)
if (!agentId) return
if (state === 'delta' || state === 'streaming') {
this.session.transition(agentId, 'working', {
sessionKey,
currentTool: extractToolName(p),
})
} else if (state === 'final' || state === 'end') {
this.session.transition(agentId, 'idle', { sessionKey })
} else if (state === 'error') {
const errorMsg =
typeof p.errorMessage === 'string'
? p.errorMessage
: typeof p.error === 'string'
? p.error
: 'Unknown error'
this.session.transition(agentId, 'error', { sessionKey, error: errorMsg })
}
}
private scheduleReconnect(): void {
this.clearReconnect()
this.reconnectTimer = setTimeout(() => {
this.reconnectTimer = null
this.doConnect()
}, RECONNECT_DELAY_MS)
}
private clearReconnect(): void {
if (this.reconnectTimer) {
clearTimeout(this.reconnectTimer)
this.reconnectTimer = null
}
}
}
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
/**
* Extract agentId from an OpenClaw session key.
* Format: "agent:<agentId>:..." — we take the segment after "agent:".
*/
function extractAgentId(sessionKey: string): string | null {
if (!sessionKey.startsWith('agent:')) return null
const colonIdx = sessionKey.indexOf(':', 6)
if (colonIdx === -1) return sessionKey.slice(6)
return sessionKey.slice(6, colonIdx)
}
/**
* Try to extract a tool name from a chat event payload.
*/
function extractToolName(payload: Record<string, unknown>): string | null {
if (typeof payload.toolName === 'string') return payload.toolName
if (typeof payload.tool === 'string') return payload.tool
const content = payload.content
if (content && typeof content === 'object' && 'name' in content) {
const name = (content as Record<string, unknown>).name
if (typeof name === 'string') return name
}
return null
}

View File

@@ -18,6 +18,11 @@ import { DEFAULT_PORTS } from '@browseros/shared/constants/ports'
import { getOpenClawDir } from '../../../lib/browseros-dir'
import { logger } from '../../../lib/logger'
import type { MonitoringChatTurn } from '../../../monitoring/types'
import {
type AgentLiveStatus,
type AgentSessionState,
ClawSession,
} from './claw-session'
import type {
ContainerRuntime,
GatewayContainerSpec,
@@ -58,7 +63,12 @@ import {
type OpenClawSessionHistory,
type OpenClawSessionHistoryEvent,
} from './openclaw-http-client'
import { type ClawEvent, OpenClawJsonlReader } from './openclaw-jsonl-reader'
import {
type ClawEvent,
OpenClawJsonlReader,
summarizeToolActivity,
} from './openclaw-jsonl-reader'
import { OpenClawObserver } from './openclaw-observer'
import {
type ResolvedOpenClawProviderConfig,
resolveSupportedOpenClawProvider,
@@ -299,6 +309,14 @@ function jsonlEventsToHistoryItems(
return items
}
function sumCostFromEvents(events: ClawEvent[]): number {
let cost = 0
for (const e of events) {
if (e.type === 'agent.message' && e.costUsd) cost += e.costUsd
}
return cost
}
function encodeHistoryCursor(input: {
sessionKey: string
end: number
@@ -330,6 +348,25 @@ function decodeHistoryCursor(
}
}
export interface AgentOverview {
agentId: string
status: AgentLiveStatus
latestMessage: string | null
latestMessageAt: number | null
activitySummary: string | null
currentTool: string | null
totalCostUsd: number
sessionCount: number
}
export interface DashboardResponse {
agents: AgentOverview[]
summary: {
totalAgents: number
totalCostUsd: number
}
}
export class OpenClawService {
private runtime: ContainerRuntime
private cliClient: OpenClawCliClient
@@ -349,6 +386,8 @@ export class OpenClawService {
private lastRecoveryReason: OpenClawGatewayRecoveryReason | null = null
private stopLogTail: (() => void) | null = null
private lifecycleLock: Promise<void> = Promise.resolve()
private clawSession = new ClawSession()
private observer = new OpenClawObserver(this.clawSession)
private _jsonlReader: OpenClawJsonlReader | null = null
private get jsonlReader(): OpenClawJsonlReader {
@@ -418,6 +457,13 @@ export class OpenClawService {
return this.hostPort
}
/** Subscribe to real-time agent status changes from the ClawSession state machine. */
onAgentStatusChange(
listener: (agentId: string, state: AgentSessionState) => void,
): () => void {
return this.clawSession.onStateChange(listener)
}
// ── Lifecycle ────────────────────────────────────────────────────────
async setup(input: SetupInput, onLog?: (msg: string) => void): Promise<void> {
@@ -586,6 +632,7 @@ export class OpenClawService {
return this.withLifecycleLock('stop', async () => {
logger.info('Stopping OpenClaw service', { hostPort: this.hostPort })
this.controlPlaneStatus = 'disconnected'
this.observer.disconnect()
this.stopGatewayLogTail()
await this.runtime.stopGateway()
logger.info('OpenClaw container stopped')
@@ -659,6 +706,7 @@ export class OpenClawService {
async shutdown(): Promise<void> {
this.controlPlaneStatus = 'disconnected'
this.observer.disconnect()
this.stopGatewayLogTail()
try {
await this.runtime.stopGateway()
@@ -896,6 +944,75 @@ export class OpenClawService {
}
}
// ── Dashboard ──────────────────────────────────────────────────────
getDashboard(): DashboardResponse {
const agentIds = this.jsonlReader.listAgents()
const agentOverviews: AgentOverview[] = []
let totalCostUsd = 0
for (const agentId of agentIds) {
const liveStatus = this.clawSession.getState(agentId)
const sessions = this.jsonlReader.listSessions(agentId)
if (sessions.length === 0) {
agentOverviews.push({
agentId,
status: liveStatus.status,
latestMessage: null,
latestMessageAt: null,
activitySummary: null,
currentTool: liveStatus.currentTool,
totalCostUsd: 0,
sessionCount: 0,
})
continue
}
const latestSession = sessions[0]
// Read the latest session's JSONL once and derive everything from
// the loaded events array — avoids re-reading the same file for
// latestAgentMessage() and getSessionStats() individually.
const events = this.jsonlReader.listBySession(agentId, latestSession.key)
let latestMsg: ClawEvent | undefined
for (let i = events.length - 1; i >= 0; i--) {
if (events[i]?.type === 'agent.message') {
latestMsg = events[i]
break
}
}
// Accumulate cost: derive from the already-loaded events for the
// latest session, read remaining sessions separately.
let agentCost = sumCostFromEvents(events)
for (let i = 1; i < sessions.length; i++) {
const stats = this.jsonlReader.getSessionStats(
agentId,
sessions[i]!.key,
)
agentCost += stats.totalCostUsd
}
totalCostUsd += agentCost
agentOverviews.push({
agentId,
status: liveStatus.status,
latestMessage: latestMsg?.content?.slice(0, 200) ?? null,
latestMessageAt: latestMsg?.createdAt ?? latestSession.updatedAt,
activitySummary: summarizeToolActivity(events),
currentTool: liveStatus.currentTool,
totalCostUsd: agentCost,
sessionCount: sessions.length,
})
}
return {
agents: agentOverviews,
summary: { totalAgents: agentIds.length, totalCostUsd },
}
}
// ── Chat Stream (HTTP) ───────────────────────────────────────────────
async chatStream(
@@ -1266,6 +1383,7 @@ export class OpenClawService {
this.controlPlaneStatus = 'connected'
this.lastGatewayError = null
this.lastRecoveryReason = null
this.ensureObserverConnected()
return result
} catch (error) {
const message = error instanceof Error ? error.message : String(error)
@@ -1277,6 +1395,19 @@ export class OpenClawService {
}
}
private ensureObserverConnected(): void {
// Seed the ClawSession state machine from JSONL on first control plane
// call. This gives every agent a correct initial status (working/idle)
// before the WS observer has seen any events.
if (!this.clawSession.isSeeded()) {
this.clawSession.seedFromJsonl(this.jsonlReader)
}
if (this.observer.isConnected()) return
const url = `http://127.0.0.1:${this.hostPort}`
this.observer.connect(url, this.token)
}
private classifyControlPlaneError(
error: unknown,
): OpenClawGatewayRecoveryReason {
@@ -1346,6 +1477,10 @@ export class OpenClawService {
path: 'gateway.controlUi.allowInsecureAuth',
value: true,
},
{
path: 'gateway.controlUi.dangerouslyDisableDeviceAuth',
value: true,
},
{
path: 'gateway.controlUi.allowedOrigins',
value: [

View File

@@ -30,7 +30,7 @@ MACOS_SERVER_BINARIES: Dict[str, SignSpec] = {
),
"bun": SignSpec("bun", "runtime", "browseros-executable-entitlements.plist"),
"rg": SignSpec("rg", "runtime"),
"limactl": SignSpec("limactl", "runtime"),
"limactl": SignSpec("limactl", "runtime", "lima-vz-entitlements.plist"),
}

View File

@@ -1,6 +1,7 @@
#!/usr/bin/env python3
"""Tests for the shared server-binary sign table."""
import plistlib
import unittest
from pathlib import Path
@@ -33,6 +34,15 @@ class MacosServerBinariesTest(unittest.TestCase):
self.assertEqual(spec.identifier_suffix, "limactl")
self.assertIsNone(macos_sign_spec_for(Path("/x/not_a_known_binary")))
def test_limactl_uses_vz_entitlement(self):
entitlements_name = "lima-vz-entitlements.plist"
spec = macos_sign_spec_for(Path("/x/limactl"))
assert spec is not None
self.assertEqual(spec.entitlements, entitlements_name)
entitlements = plistlib.loads((ENTITLEMENTS_DIR / entitlements_name).read_bytes())
self.assertIs(entitlements.get("com.apple.security.virtualization"), True)
def test_matches_lima_bundle_layout(self):
keys = set(MACOS_SERVER_BINARIES.keys())
self.assertIn("limactl", keys)

View File

@@ -0,0 +1,8 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>com.apple.security.virtualization</key>
<true/>
</dict>
</plist>