Commit Graph

2457 Commits

Author SHA1 Message Date
Dani Akash
c7dde92960 feat: personalized onboarding (#468)
* feat: update onboarding steps

* chore: customize demo page

* fix: prompt display on onboarding

* fix: use styled scrollbar

* feat: show appicon on prompt

* fix: lint issues
2026-03-11 17:30:28 +05:30
Nikhil Sonti
32ce02b59f fix: hidden windows fix 2026-03-10 18:40:10 -07:00
Nikhil Sonti
7566f0ee82 fix: sidepanel request focus fix 2026-03-10 18:39:19 -07:00
Nikhil Sonti
ffe1f8a469 chore: server ota 2026-03-10 18:31:37 -07:00
Nikhil Sonti
a5e7c359e3 chore: Merge branch 'main' 2026-03-10 18:22:19 -07:00
Nikhil Sonti
3f4cccdf12 chore: bump PATCH and OFFSET 2026-03-10 18:22:15 -07:00
Nikhil
866fe88acd feat: fix hidden window and tab tools (#417) 2026-03-10 18:21:10 -07:00
Nikhil Sonti
385cf03227 chore: bump server version 2026-03-10 18:19:21 -07:00
Nikhil
a824078f6d fix: compaction config for small context windows (≤32K) (#466)
* fix: compaction config for small context windows (≤32K)

Raise COMPACTION_SMALL_CONTEXT_WINDOW from 16K to 32K so models like
Haiku 4.5 (30K context) use proportional 50% reserve instead of the
fixed 20K reserve. Also scale fixedOverhead for small contexts (capped
at 40% of context window) to prevent the doom loop where overhead alone
triggers compaction on every step.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add compaction tuning guidance to limits constants

Explain the relationship between SMALL_CONTEXT_WINDOW and
FIXED_OVERHEAD so devs know the 24K minimum constraint when
tweaking these values.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 18:12:20 -07:00
Nikhil
3e23796724 fix: auto-focus chat input when side panel opens (#465)
Add window focus listener in ChatFooter that focuses the textarea when
the side panel receives focus. Handles both initial open (via
document.hasFocus check on mount) and re-focus scenarios (via window
focus event). Guards against stealing focus from other interactive
elements.

Companion Chromium fix: side_panel_coordinator.cc now always calls
RequestFocus() in PopulateSidePanel(), not just when there's no
previous entry — ensuring the side panel WebContents receives focus
on every open/toggle.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 17:29:15 -07:00
Nikhil Sonti
ae49da6e09 fix: sidepanel request focus fix 2026-03-10 17:27:54 -07:00
Nikhil Sonti
bcd91a8e03 chore: Merge branch 'main' 2026-03-10 17:23:27 -07:00
Nikhil
2d6d08c9fe fix: move tool-result media normalization into agent (#460)
* fix: sanitize media during compaction

* fix: normalize content outputs in compaction helpers

* fix: move tool-result media normalization into agent

* chore: rename compaction orchestrator file
2026-03-10 17:21:09 -07:00
Nikhil Sonti
4472c2b890 chore: bump PATCH and OFFSET 2026-03-10 15:12:18 -07:00
Nikhil Sonti
2477063673 chore: bump server version 2026-03-10 15:08:17 -07:00
Felarof
146b9af17c Update README.md (#416) 2026-03-10 13:33:47 -07:00
Nikhil
de70525889 fix: grab handle size (#414) 2026-03-10 12:26:08 -07:00
Nikhil
f81e73f6a4 fix: avoid crashing on controller startup failure (#458)
* fix: avoid crashing on controller startup failure

* fix: address PR review comments for remove_controller_startup_crash
2026-03-10 11:53:11 -07:00
Nikhil
4fc68b5264 feat: use execution dir for tool temp output (#456)
* feat: use execution dir for tool temp output

* fix: harden execution dir temp staging

* refactor: use temp files for transient tool output
2026-03-10 10:57:00 -07:00
Nikhil
5b27933c63 feat: add 2-stage pruning to compaction pipeline (#455)
* feat: add 2-stage pruning to compaction pipeline before LLM summarization

Add two new lightweight stages to the compaction prepareStep pipeline that
recover context tokens cheaply before falling back to expensive LLM
summarization:

- Stage 2: Use AI SDK's pruneMessages to remove old tool call/result
  pairs beyond the last 6 messages entirely
- Stage 3: Replace remaining tool output values with short placeholders
  ("[Cleared — N chars]") while preserving tool call structure and IDs

Both stages re-estimate tokens from message content (not stale step
usage) after modifying messages. The existing LLM summarization and
sliding window fallback remain as Stage 4.

Also adds estimateTokensForThreshold() helper, clearToolOutputs()
function, and COMPACTION_PRUNE_KEEP_RECENT_MESSAGES /
COMPACTION_CLEAR_OUTPUT_MIN_CHARS constants.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: reorder compaction pipeline — truncate before clear, protect recent tools

- Stage 0: Check threshold, return untouched when under (no data loss)
- Stage 1: Prune old tool call/result pairs beyond last 6 messages
- Stage 2: Truncate large tool outputs to 15K chars (keeps partial content)
- Stage 3: Clear old tool outputs with placeholders, protect last 2
- Stage 4: LLM-based compaction with sliding window fallback

clearToolOutputs now accepts keepRecentCount parameter (default 2) to
skip the N most recent tool messages from clearing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: limits fixes

* fix: address review — preserve toKeep context, derive test values from constants

- When Stage 3 (clearToolOutputs) doesn't resolve overflow, pass
  truncated (not cleared) messages to Stage 4 so toKeep retains
  meaningful tool outputs for the agent's immediate context
- Add comment explaining intentional conservatism in post-prune
  token estimation (step usage is stale, must re-estimate safely)
- Refactor computeConfig tests to derive expected values from
  AGENT_LIMITS constants instead of hardcoding magic numbers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 10:41:34 -07:00
shivammittal274
d1937b3280 fix: replace stale browser_open_tab tool name with new_page in prompt (#454)
The system prompt referenced `browser_open_tab` which was renamed to
`new_page`. This caused models to infer a `browser_*` naming convention
and call non-existent tools like `browser_navigate`, resulting in
MCP error -32602.

Fixes TKT-540
2026-03-10 22:21:21 +05:30
Nikhil
15755a84d9 feat: use execution dir in browser tool context (#453) 2026-03-10 09:38:36 -07:00
Nikhil
7d20768d8e feat: persist large tool outputs to disk (#452)
* feat: persist large tool outputs to disk

* fix: address PR review comments for tool output limits

* chore: raise filesystem read line limit to 500
2026-03-10 09:25:19 -07:00
Felarof
cd6ca756c1 docs: rename Local Model Guide to Bring Your Local Model
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 08:42:54 -07:00
Felarof
da137cbb97 docs: wrap example prompts in accordion for clearer separation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 08:42:10 -07:00
Felarof
91995854fa docs: add OpenClaw as MCP client with connection instructions
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 08:34:10 -07:00
Felarof
e312f29138 docs: add v0.42.0 changelog entry with release images
Add changelog entry for BrowserOS v0.42.0 featuring SOUL.md, vertical tabs,
long-term memory, and Chromium 146 update. Include screenshots from the
GitHub release.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 08:30:45 -07:00
Nikhil Sonti
a2eb965759 chore: Merge branch 'main' of https://github.com/browseros-ai/BrowserOS-agent 2026-03-10 07:50:11 -07:00
Dani Akash
e1a9174de1 feat: setup docs for skills and nudges (#412) 2026-03-10 20:08:04 +05:30
Felarof
1e6b5ac7a8 chore: sync packages/browseros-agent submodule (to f35ac0d) 2026-03-10 12:20:28 +00:00
Dani Akash
f35ac0ddd3 feat: new onboarding tools (#385)
* feat: new tools for breadcrumbs

* feat: setup scheduled task card

* feat: added dismiss cooldown

* chore: update prompt

* fix: support api key tool

* fix: prompt text to limit nudges

* fix: scheduled tasks card

* fix: update nudges prompt

* feat: skip nudges when user dismisses nudge

* fix: ensure nudges only show if they are not dismissed

* Revert "fix: ensure nudges only show if they are not dismissed"

This reverts commit d825254698829b8e9941aae7873bd440027d0c74.

* Revert "feat: skip nudges when user dismisses nudge"

This reverts commit 12b552b454d10ec4209b88668fc48681423ff6fc.

* Revert "fix: update nudges prompt"

This reverts commit 80b7520b953b4d3cbed2ed477b9e508e39938dca.

* feat: update agent with mcp when new mcp connection is added

* feat: created connect apps option as a blocking card system

* feat: schedule tasks passive without dismiss

* fix: nudges and prompt texts

* fix: biome lint errors

* fix: review comments

* fix: resolve comments

* fix: review comments

* fix: review comments

* fix: auto resolve state

* fix: eliminate the race where the async delete could resolve after the
new session

* feat: track ignored apps list

* fix: empty response text object on message reply

* feat: sync previously connected mcps

* feat: sync integrations with klavis

* feat: account for unauthenticated connections

* fix: analytics events

* fix: typescript issues

* fix: klavis client issue

* fix: invalid mcps causing entire responses from failing

* fix: prompt with card for integrations when the integration fails

* fix: prompt structure to support declined apps

* fix: refresh session on mcp changes
2026-03-10 17:44:10 +05:30
shivammittal274
b6b45404ee feat: add agent skills system with catalog, loader, and UI (#450)
* feat: add agent skills system with catalog, loader, and UI

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: return 500 for server errors in PUT/DELETE skill routes

Previously both handlers returned 404 for all errors, masking filesystem
failures (disk full, permission denied) as "not found". Now only
"not found" errors return 404; everything else returns 500.

* fix: align SKILL.md format with agentskills.io spec

- Move `enabled` and `version` into `metadata` field (spec only allows
  name, description, license, compatibility, metadata, allowed-tools)
- Frontmatter `name` now matches directory name (lowercase kebab-case)
- Human-readable name stored in `metadata.display-name`
- Add index signature to SkillMetadata for arbitrary string keys
- Validate frontmatter with type guard in getSkill (remove unsafe cast)
- updateSkill now preserves existing frontmatter fields (license, etc.)
- Tighten buildSkillMd param from Record<string, unknown> to SkillFrontmatter

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 17:24:05 +05:30
Nikhil Sonti
ca777dd2fd chore: bump server version 2026-03-09 16:25:43 -07:00
Felarof
797c75baee chore: sync packages/browseros-agent submodule (to 44071cb) 2026-03-09 21:13:22 +00:00
shivammittal274
44071cb0f4 fix: fix compaction tool output truncation and token estimation (#448)
- truncateToolOutputs: handle all output.type variants (text, json,
  content) by checking output.value directly instead of branching on
  type. The old code missed type 'content' (array of content parts),
  causing 1M+ char tool results to pass through untouched.

- estimateTokens: change chars/4 to chars/3 — HTML/Markdown content
  tokenizes at ~3.14 chars/token empirically, not 4.

- COMPACTION_FIXED_OVERHEAD: 5K → 12K to account for system prompt
  (~2.5K tokens) + tool definitions as JSON Schema (~8-9K tokens).

- Apply truncateToolOutputs in prepareStep (Stage 0) before token
  estimation, not just during summarization.
2026-03-10 02:39:54 +05:30
Nikhil
b035278ad9 fix: OTA binary discovery for artifact-extracted structure (#411)
* fix: support artifact-extracted directory structure in OTA binary discovery

The download_resources system now extracts server binaries into
platform-specific subdirectories (e.g., darwin-arm64/resources/bin/),
but the OTA module only looked for flat binary names. This adds
find_server_binary() which checks both layouts, keeping backward
compatibility with --binaries while supporting the new structure.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: download server binaries from R2 instead of requiring --binaries

Remove the --binaries flag from `ota server release`. The module now
downloads artifact zips from artifacts/server/latest/ in R2, extracts
them, then signs and packages as before. This eliminates the need to
have mono build output locally.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 13:20:08 -07:00
Felarof
04ca38c93b chore: sync packages/browseros-agent submodule (to 09bd10c) 2026-03-09 19:20:33 +00:00
Nikhil Sonti
09bd10cb56 chore: bump server version 2026-03-09 12:04:20 -07:00
Nikhil Sonti
3e3ffb3f51 feat: vertical tabs docs 2026-03-09 09:34:54 -07:00
Felarof
93b59633c7 chore: sync packages/browseros-agent submodule (to 3808faf) v0.42.0 2026-03-09 09:24:07 +00:00
shivammittal274
3808faf94d fix: robust compaction with Pi-style token counting + overflow middle… (#444)
* fix: robust compaction with Pi-style token counting + overflow middleware

Root cause: getCurrentTokenCount() returned stale inputTokens from the
previous step, ignoring new tool results added to messages since that
step. A large tool output (DOM snapshot, page content) caused a token
jump that bypassed the compaction threshold check, leading to
context_length_exceeded errors (322K tokens sent, model max 262K).

Layer 1 — Accurate token counting (proactive):
- Adopt Pi coding agent's additive approach: base(inputTokens) +
  outputTokens + estimate(trailing tool results)
- Trailing tool results are estimated by walking backwards from end of
  messages array until a non-tool message is found
- Falls back to full estimation with safety multiplier when no real
  usage data is available (first step of a turn)

Layer 2 — Context overflow middleware (reactive):
- LanguageModelV3Middleware that wraps doGenerate/doStream
- Catches context_length_exceeded errors at the model call level
- Truncates prompt (keeps system messages + most recent non-system
  messages targeting 60% of context window)
- Retries the model call once

Verified end-to-end with real model (Gemini Flash Lite via OpenRouter)
on 16K context window: 4 compactions triggered correctly across 8
steps, no context_length_exceeded errors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: adopt Pi-style overflow detection patterns + fix truncation edge case

- Replace 6 generic substring matches with 17 provider-specific regex
  patterns from Pi coding agent (Anthropic, OpenAI, Google, xAI, Groq,
  OpenRouter, Bedrock, Copilot, llama.cpp, LM Studio, MiniMax, Kimi,
  Mistral, z.ai)
- Fix truncatePrompt edge case: when the last message alone exceeds the
  target, keepFrom was never updated → empty non-system messages. Now
  always keeps at least the most recent non-system message.
- Add runtime guard for LanguageModelV3 cast in ai-sdk-agent.ts
- Add tests for false-positive rejection and truncation edge case

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 14:22:35 +05:30
Felarof
a94d6d918c chore: sync packages/browseros-agent submodule (to eb208b0) 2026-03-08 18:11:51 +00:00
Felarof
eb208b0515 feat: update new tab placeholder copy (#441)
* feat: update new tab placeholder copy

* fix: simplify new tab placeholder logic
2026-03-08 11:06:41 -07:00
Felarof
c7990566d9 chore: sync packages/browseros-agent submodule (to 60a4167) 2026-03-08 03:06:57 +00:00
Felarof
60a4167a0e fix: update Kimi K2.5 context window from 128K to 256K (#440)
The Kimi K2.5 model supports a 256,000 token context window, not
128,000. Updated the provider template and model config to reflect
the correct value.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 17:58:52 -08:00
Felarof
6ce0fd35a0 chore: sync packages/browseros-agent submodule (to c8a674f) 2026-03-07 11:07:29 +00:00
shivammittal274
c8a674fe93 feat: return element coordinates in tool responses and DPR in screens… (#437)
* feat: return element coordinates in tool responses and DPR in screenshots

- click, hover, fill, drag now return resolved coordinates in response text
- take_screenshot returns devicePixelRatio for mapping coordinates to pixels
- Coordinates are in CSS pixels; multiply by DPR to get screenshot pixels

* fix: use Promise.allSettled in screenshot to prevent DPR eval from aborting capture

Runtime.evaluate for devicePixelRatio can fail on PDF pages or
chrome-extension pages. Using Promise.allSettled ensures the screenshot
still succeeds, falling back to DPR=1.
2026-03-07 16:29:13 +05:30
Nikhil Sonti
135fa65a2e chore: bump PATCH and OFFSET 2026-03-06 17:06:13 -08:00
Felarof
5c774501f3 chore: sync packages/browseros-agent submodule (to 2e79933) 2026-03-07 00:39:44 +00:00
Nikhil
65b5e74a75 fix: windows header (#407) 2026-03-06 16:08:51 -08:00