Compare commits

..

107 Commits

Author SHA1 Message Date
Nikhil Sonti
9a36c09577 fix: address review feedback for PR #602
- Make loadProdEnv return empty map when .env.production is absent so
  pickEnv falls through to process.env in CI (Greptile P1)
- Add semver format validation for version string in install.sh and
  install.ps1 to guard against malformed CDN responses
- Pass inputs.version via env var instead of inline ${{ }} interpolation
  to prevent command injection in workflow shell
2026-03-27 11:36:49 -07:00
Nikhil Sonti
2ce6379f8c feat: upload CLI binaries to CDN during release and gate workflow to core team
- Extend scripts/build/cli/upload.ts with uploadCliRelease() that pushes
  archives + checksums to R2 under versioned (cli/v{VERSION}/) and latest
  (cli/latest/) paths, plus a version.txt for lightweight latest resolution
- Update scripts/build/cli.ts entry point with --release/--version/--binaries-dir
  flags (existing no-args behavior preserved for upload:cli-installers)
- Rewrite install.sh and install.ps1 to fetch from cdn.browseros.com instead of
  GitHub releases API — eliminates rate limits and API dependency
- Add environment: release-core to release-cli.yml for core-team gating via
  GitHub environment protection rules
- Add Bun setup + CDN upload step to the workflow between build and GitHub release
2026-03-27 11:36:49 -07:00
Nikhil
1c5ffdf878 fix: harden cli installer bootstrap (#601)
* fix: harden cli installer bootstrap

* refactor: rework 0327-harden_cli_installers based on feedback
2026-03-27 11:24:16 -07:00
Nikhil
39a7d49c25 feat: add workspace-centric bdev cli (#585)
* fix: clean-up bdev

* feat: add workspace-centric bdev cli

* fix: address review comments for 0326-bdev_cli_redesign

* fix: address review feedback for PR #585

* fix: address review feedback for PR #585
2026-03-27 08:48:23 -07:00
shivammittal274
ed948f4b59 Feat/cli launch ready v2 (#600)
* test: temporarily allow release workflow on any branch

* fix(cli): restore main-only guard, remove goreleaser dependency

Replaces GoReleaser (Pro-only monorepo feature) with plain go build.
Tested: RC release created successfully on branch with all 6 binaries.

* fix(cli): fix hdiutil mount detection, update README with install/launch/init flow
2026-03-27 20:20:17 +05:30
shivammittal274
aad5bc16fd Feat/cli launch ready v2 (#599)
* test: temporarily allow release workflow on any branch

* fix(cli): restore main-only guard, remove goreleaser dependency

Replaces GoReleaser (Pro-only monorepo feature) with plain go build.
Tested: RC release created successfully on branch with all 6 binaries.

* fix(cli): remove -quiet from hdiutil so mount point is detected
2026-03-27 20:17:13 +05:30
Dani Akash
cee318a40b fix: improve chat history freshness and reduce query payload (#598)
* fix: add refresh indicator to chat history when fetching latest conversations

Show a non-blocking "Fetching latest conversations" indicator at the top
of the history list while the cached data is being refreshed. Users can
still interact with the cached conversation list during the refresh.

* perf: reduce chat history query payload — fetch last 2 messages instead of 5

The conversation list only displays the last user message as a preview.
Fetching 5 messages per conversation was wasteful — each message contains
the full UIMessage object (tool calls, reasoning, etc.) multiplied by
50 conversations per page. Reduced to last 2 which is sufficient to
find the last user message in a user→assistant exchange.

* perf: use first+DESC instead of last+ASC to push LIMIT down to SQL

PostGraphile's `last: N` doesn't map to SQL LIMIT — it uses a padded
LIMIT 10 and slices in application code. Changing to `first: 2` with
ORDER_INDEX_DESC generates a true SQL LIMIT 2, reducing rows scanned
from 500 to 100 per page (50 conversations × 2 vs 10 messages each).

No UX impact — extractLastUserMessage() filters by role regardless
of message order.

* chore: update react query packages

* feat: replace localforage with idb-keyval
2026-03-27 19:49:47 +05:30
Dani Akash
febaf58f91 fix: guard filesystem tools behind workspace selection and handle mid-conversation changes (#595)
* fix: remove filesystem tools when no workspace is selected

- Make workingDir optional on ResolvedAgentConfig
- Remove resolveSessionDir() fallback that always created a session dir,
  masking the no-workspace state and keeping filesystem tools available
- Gate buildFilesystemToolSet() on workingDir being defined
- Add workspace change detection mid-conversation — rebuilds the agent
  session when workspace is added, removed, or switched (same pattern
  as existing MCP server change detection)
- download_file falls back to tmpdir() when no workspace is set
- Memory/soul tools are unaffected — they use ~/BrowserOS/ paths

* fix: sanitize message history when session rebuilds with different tools

When a session is rebuilt due to workspace or MCP changes, the carried-over
message history may contain tool parts for tools that no longer exist in
the new session. The AI SDK validates messages against the current toolset
and rejects parts with no matching schema.

- Add toolNames getter to AiSdkAgent exposing registered tool names
- Add sanitizeMessagesForToolset() to strip tool parts referencing
  removed tools from carried-over messages
- Apply sanitization in both MCP and workspace session rebuilds

* fix: prepend tool-change context to user message on session rebuild

When workspace or MCP integrations change mid-conversation, prepend a
[Context: ...] block to the user's message explaining what changed.
This prevents the LLM from hallucinating tool usage based on patterns
in the carried-over conversation history.

Context messages vary by change type:
- Workspace removed: lists unavailable filesystem tools, suggests
  selecting a working directory
- Workspace added: confirms filesystem tools are available with path
- Workspace switched: notes the new working directory
- MCP changed: notes that some integration tools may have changed

Only fires on the first message after a rebuild. Invisible in the UI.

* fix: make MCP change context specific about which apps were added/removed

Diff the old and new MCP server keys to produce specific context like:
- "The following app integrations were disconnected: Gmail, Slack."
- "The following app integrations were connected: Linear."
instead of a generic "some tools may no longer be available" message.

* refactor: extract shared rebuildSession helper in ChatService

Eliminates the duplicated 20-line dispose→create→sanitize→store flow
that existed separately in both the MCP and workspace change-detection
blocks.

Co-authored-by: Dani Akash <DaniAkash@users.noreply.github.com>

* test: add sanitizeMessagesForToolset test suite

Tests for the message sanitization that runs when a session rebuilds
with a different toolset (workspace or MCP change mid-conversation):

- Preserves messages with no tool parts
- Preserves tool parts when tool is in the toolset
- Strips tool parts when tool is NOT in the toolset
- Strips multiple removed tool parts from same message
- Keeps browser tools while removing filesystem tools
- Removes messages that become empty after stripping
- Preserves non-tool parts (reasoning, step-start, file)
- Returns same references when no filtering needed
- Handles empty message array and empty toolset

* style: fix biome formatting in chat-service.ts

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
2026-03-27 18:30:25 +05:30
Dani Akash
aacb47f7ee feat: isolate new-tab agent navigation from origin tab (#593)
* feat: isolate new-tab agent navigation from origin tab

Add origin-aware navigation isolation so the agent never navigates
away from the new-tab chat UI. This is a two-layer defense:

1. Prompt adaptation: When origin is 'newtab', the system prompt's
   execution and tool-selection sections are rewritten to prohibit
   navigating the active tab and default all lookups to new_page.

2. Tool-level guards: navigate_page and close_page reject attempts
   to act on the origin tab when in newtab mode, returning an error
   that teaches the agent to self-correct.

The client now sends an `origin` field ('sidepanel' | 'newtab')
instead of injecting a soft NEWTAB_SYSTEM_PROMPT that LLMs could
ignore. Backwards compatible — defaults to 'sidepanel'.

Closes TKT-592, addresses TKT-564

* test: add newtab origin navigation guard tests

- 14 new prompt tests verifying the system prompt adapts correctly
  for newtab vs sidepanel origin (execution rules, tool selection table,
  absence of conflicting single-tab guidance)
- 6 new integration tests for navigate_page and close_page guards:
  rejects origin tab in newtab mode, allows non-origin tabs, allows
  all tabs in sidepanel mode, backwards compatible with no session
2026-03-27 12:06:32 +05:30
Dani Akash
b3003542d8 docs: overhaul READMEs across all major packages (#594)
* docs: overhaul READMEs across all major packages

- Root README: restructure with feature table, LLM provider table,
  comparison matrix, architecture map, and docs link
- New: packages/browseros/README.md (Chromium fork build system)
- New: apps/server/README.md (MCP server + agent loop)
- New: packages/cdp-protocol/README.md (CDP type bindings)
- Polish: agent-sdk (badges, prerequisites, multi-step example, links)
- Polish: cli (badges, install section, MCP server section, links)
- Polish: agent extension (badges, WXT mention, architecture context)
- Polish: eval (badges, paper links)

* fix: address review — consistent tool count and correct default port

- CLI README: "54 MCP tools" → "53+ MCP tools" to match root and server docs
- Agent SDK README: localhost:3000 → localhost:9100 to match documented default

* docs: add detailed comparison links to How We Compare section

* docs: update comparison table with verified competitor data

Research all 5 competitors via official websites and docs:
- Chrome: no AI agent, Gemini Nano only, MV3 weakening ad blocking
- Brave: BYOM feature, local models via BYOM, Shields ad blocking, MV2+MV3
- Dia: Skills-based AI, no BYOK, cloud AI, acquired by Atlassian
- Comet: full cloud-based agent, built-in ad blocking, extensions on desktop
- Atlas: standalone Chromium browser with Agent Mode, 30-day cloud memory

Renamed Arc/Dia column to just Dia (Arc is sunset).

* docs: simplify comparison table with clean checkmarks and key differentiators

* docs: update browseros-agent README — remove submodule note, add missing packages
2026-03-27 11:59:04 +05:30
Nikhil
aba7a10430 chore: server release (#592) 2026-03-26 19:13:56 -07:00
Nikhil
b7462aa042 fix(cli): move install instructions below What's Changed in release notes (#591)
The installer block was appearing above the changelog. Reorder so
What's Changed comes first and install instructions follow.
2026-03-26 18:16:23 -07:00
Nikhil
883bcc9670 fix: clean up README CLI wording and add Vertical Tabs feature (#590)
- Simplify CLI section: remove confusing MCP jargon, clarify it works
  from terminal and AI coding agents
- Replace "point the CLI at your MCP server" with plain language
- Add Vertical Tabs to the features list
2026-03-26 18:05:54 -07:00
Nikhil
279b41fdc4 feat(cli): add install commands to GitHub release notes (#589)
* feat(cli): add install commands to release notes

* fix(cli): add install header to release workflow
2026-03-26 18:04:58 -07:00
Nikhil
220577b41c feat: add CDN-hosted CLI installer flow (#588)
* feat: add CDN upload flow for cli installers

* fix: move cli install docs to top-level readme

* fix: bun.lock update
2026-03-26 17:41:03 -07:00
Nikhil
03b45013a6 feat(cli): add install scripts for macOS, Linux, and Windows (#587)
* feat(cli): add install scripts for macOS, Linux, and Windows

Bash script (install.sh) for macOS/Linux and PowerShell script
(install.ps1) for Windows. Both download the correct platform binary
from GitHub Releases with checksum verification, version resolution,
and PATH setup.

* fix(cli): address PR review comments for install scripts

- Add checksum verification to install.ps1 using Get-FileHash
- Add warnings on all checksum skip paths in install.sh
- Use grep -F (fixed-string) instead of regex for filename matching
- Add ?per_page=100 to GitHub API call in install.ps1
- Use random temp directory name in install.ps1 to avoid collisions

* fix(cli): address installer review feedback
2026-03-26 17:05:21 -07:00
shivammittal274
aa85907212 Feat/cli launch ready v2 (#582)
* fix(cli): use full path for dist artifacts in release step

* test: temporarily allow release workflow on any branch

* fix(cli): restore main-only guard, remove goreleaser dependency

Replaces GoReleaser (Pro-only monorepo feature) with plain go build.
Tested: RC release created successfully on branch with all 6 binaries.
2026-03-27 01:28:04 +05:30
Nikhil
085352a6f0 fix(ui): resolve MCP promo banner dismiss button overlapping with text (#581)
Move dismiss button from absolute positioning to inline flex child,
preventing it from overlapping with the "Set up" button.
2026-03-26 12:54:00 -07:00
shivammittal274
c0578d0e53 Feat/cli launch ready v2 (#580)
* fix(cli): update goreleaser tag_prefix to match browseros-cli-v* format

* fix(cli): replace goreleaser with plain go build for releases

GoReleaser free version cannot parse prefixed tags (browseros-cli-v*).
monorepo.tag_prefix is a Pro-only feature.

Replaced with direct go build + gh release create:
- Builds all 6 targets with go build (verified locally)
- Creates tar.gz/zip archives with checksums
- Uses gh release create to publish
- No external tool dependency
2026-03-27 01:12:25 +05:30
shivammittal274
663c18ee97 fix(cli): update goreleaser tag_prefix to match browseros-cli-v* format (#579) 2026-03-27 01:07:36 +05:30
Dani Akash
48727750b4 fix: change CLI tag format from cli/v* to browseros-cli-v* (#578)
GoReleaser free cannot parse slash-prefixed tags (cli/v0.0.1) as semver.
Switch to browseros-cli-v0.0.1 format which is valid semver after
stripping the prefix. Remove the monorepo config (GoReleaser Pro only).
2026-03-27 00:58:13 +05:30
Dani Akash
30a3a96a57 fix: add monorepo tag prefix for goreleaser to parse cli/ tags (#576) 2026-03-27 00:50:38 +05:30
shivammittal274
6773ce39da ci(cli): manual dispatch release workflow (#574)
* ci(cli): change release workflow to manual dispatch from main

- Trigger via Actions UI with a version input (e.g. "0.1.0")
- Only runs on main branch
- Creates git tag cli/v<version> automatically
- Then GoReleaser builds all 6 binaries and creates the GitHub Release

* feat: add scoped release notes, changelog PR, and idempotent tags to CLI workflow

- Add concurrency group to prevent parallel releases
- Add scoped release notes from commits touching the CLI directory
- Pass release notes to goreleaser via --release-notes flag
- Make tag creation idempotent for safe re-runs
- Tag the saved release SHA, not HEAD after branching
- Add CHANGELOG.md and auto-update via PR with auto-merge
- Add pull-requests: write permission

---------

Co-authored-by: Dani Akash <DaniAkash@users.noreply.github.com>
2026-03-27 00:41:08 +05:30
github-actions[bot]
342a3e4a07 docs: update agent extension changelog for v0.0.52 (#573)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-03-26 19:01:46 +00:00
Dani Akash
09406ea794 feat: add release workflow for agent extension (#572)
* feat: add release workflow for agent extension

Adds a workflow_dispatch workflow that builds the WXT extension,
creates a .zip for sideloading, generates scoped release notes with
contributors and PR links, creates a GitHub release with the zip
attached, and opens an auto-merge PR to update CHANGELOG.md.

* fix: correct API URL to api.browseros.com

* fix: remove duplicate PR numbers and contributors from extension release notes

Apply the same fixes from the agent-sdk workflow:
- Skip PR number if already in commit subject (squash merges)
- Remove custom Contributors section (GitHub auto-generates one)
- Clean up unused variables

* fix: use absolute path for extension zip in release upload

* fix: wxt zip already builds, use correct output path

- Remove separate build step since wxt zip runs the build internally
- Fix zip path from .output/*.zip to dist/*-chrome.zip

* fix: run codegen before wxt zip to generate graphql types
2026-03-27 00:29:47 +05:30
Dani Akash
1f00cbc9cc feat: add release workflow for agent extension (#566)
* feat: add release workflow for agent extension

Adds a workflow_dispatch workflow that builds the WXT extension,
creates a .zip for sideloading, generates scoped release notes with
contributors and PR links, creates a GitHub release with the zip
attached, and opens an auto-merge PR to update CHANGELOG.md.

* fix: correct API URL to api.browseros.com

* fix: remove duplicate PR numbers and contributors from extension release notes

Apply the same fixes from the agent-sdk workflow:
- Skip PR number if already in commit subject (squash merges)
- Remove custom Contributors section (GitHub auto-generates one)
- Clean up unused variables

* fix: use absolute path for extension zip in release upload

* fix: wxt zip already builds, use correct output path

- Remove separate build step since wxt zip runs the build internally
- Fix zip path from .output/*.zip to dist/*-chrome.zip
2026-03-27 00:23:04 +05:30
Dani Akash
422a829f5e fix: remove duplicate PR numbers and contributors from release notes (#571)
- Skip adding PR number if already present in the commit subject
  (squash merges include "(#123)" automatically)
- Remove custom Contributors section since GitHub auto-generates one
  with avatars at the bottom of every release
2026-03-27 00:07:13 +05:30
github-actions[bot]
ed109fcedf docs: update agent-sdk changelog for v0.0.7 (#570)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-03-26 18:31:39 +00:00
Dani Akash
19af96d08e chore: bump @browseros-ai/agent-sdk to 0.0.7 (#569) 2026-03-27 00:00:35 +05:30
Dani Akash
e0304b203c chore: bump @browseros-ai/agent-sdk to 0.0.6 (#568) 2026-03-26 23:53:35 +05:30
Nikhil
af65bdbcfb feat(build): add build:server:ci script with --compile-only flag (#567)
Add a compile-only mode to the server build pipeline for CI/CD
environments that don't have R2 credentials. The --compile-only flag
skips resource staging and upload, producing only compiled binaries.
2026-03-26 11:21:39 -07:00
Dani Akash
d79c2a4123 feat: create GitHub release with changelog on agent-sdk publish (#564)
* feat: create GitHub release with changelog on agent-sdk publish

After publishing to npm, the workflow now:
- Tags the commit as agent-sdk-v<version>
- Generates release notes from commits that modified the agent-sdk
  directory since the last agent-sdk release tag
- Creates a GitHub release with those notes

First release will show "Initial release" since no previous tag exists.

* feat: update CHANGELOG.md on agent-sdk release

Add a CHANGELOG.md for @browseros-ai/agent-sdk and update the release
workflow to prepend a versioned entry with the release notes before
creating the GitHub release. The changelog is committed to main
automatically.

* fix: address review issues in agent-sdk release workflow

- Add explicit permissions: contents: write
- Replace sed with head/tail for safe CHANGELOG insertion (fixes
  double-quote and backslash corruption in commit messages)
- Handle empty release notes with "No notable changes." fallback
- Make git tag idempotent for workflow reruns (2>/dev/null || true)

* fix: use PR with auto-merge for changelog updates

Direct push to main fails due to branch protection requiring PRs.
Instead, create a branch, open a PR, and auto-merge via squash.

* feat: add contributors and PR links to agent-sdk release notes

Release notes now include PR numbers (linked automatically by GitHub),
GitHub usernames for each commit author, and a contributors section
at the bottom. All scoped to commits that modified the agent-sdk path.

* fix: reorder release steps and fix tag/idempotency issues

- Capture release SHA before any branching so the tag always points
  to the main commit that was built and published to npm
- Reorder: generate notes → publish → tag/release → changelog PR
  (changelog is lowest-stakes, runs last)
- Make tag push and release create idempotent for safe re-runs
  (fall back to gh release edit if release already exists)
- Add || true to gh pr merge --auto in case auto-merge is not enabled
- Explicit git checkout main before creating changelog branch

* fix: explicit error handling for tag/release and contributor dedup

- Replace silent || true guards with explicit checks that log what's
  happening (tag exists, remote tag exists, release exists) so errors
  are visible instead of swallowed
- Fix contributor dedup: use grep -qw (word match) instead of grep -qF
  (substring match) so "dan" isn't excluded when "dansmith" exists

* fix: exclude current version tag when finding previous release

On re-runs, the current version's tag already exists on the remote, so
PREV_TAG resolves to it and git log produces empty output. Filter it
out so release notes are generated against the actual previous version.

* ci: prevent concurrent agent-sdk release runs

Add concurrency group so multiple dispatches queue instead of racing
on the same tag/release/PR.
2026-03-26 23:38:14 +05:30
shivammittal274
e3d57e5347 feat(cli): production-ready CLI with auto-launch, install, and cross-platform builds (#555)
* feat(cli): production-ready CLI with auto-launch, install, and cross-platform builds

- init: accept URL argument and --auto flag for non-interactive setup
- install: new command to download BrowserOS app for current platform
- launch: auto-detect and launch BrowserOS when server is not running
- discovery: prefer server.json (live) over config.yaml (may be stale)
- errors: actionable messages guiding users to init/install
- goreleaser: cross-platform builds for 6 targets (darwin/linux/windows × amd64/arm64)
- ci: GitHub Actions workflow to release CLI binaries on cli/v* tag push

* fix(cli): check health status code and add progress dots during launch

- Health check in newClient() now verifies HTTP 200, not just no error
- waitForServer prints dots during the 30s poll so users know it's working

* refactor(cli): make launch an explicit command, remove auto-launch from newClient

- launch: new explicit command to find and open BrowserOS app
- launch: probes server.json, config, and common ports before launching
- launch: if already running, reports URL instead of launching again
- init --auto: uses port probing to find running servers
- install --deb: errors on non-Linux instead of silently downloading DMG
- error messages: guide users to launch/install/init explicitly
- removed: auto-launch from newClient() — CLI never does something surprising

* fix(cli): platform-native detection, launch, and install for all OSes

Detection (isBrowserOSInstalled):
- macOS: uses `open -Ra` to query Launch Services (no hardcoded paths)
- Linux: checks /usr/bin/browseros (.deb), browseros.desktop, AppImage search
- Windows: checks %LOCALAPPDATA%\BrowserOS\Application\BrowserOS.exe
  and HKCU/HKLM uninstall registry keys

Launch (startBrowserOS):
- macOS: `open -b com.browseros.BrowserOS` (bundle ID, not path)
- Linux: `browseros` binary, AppImage, or `gtk-launch browseros`
  (fixed: was using xdg-open which opens by MIME type, not desktop files)
- Windows: runs BrowserOS.exe from known Chromium per-user install path
  (fixed: was using `cmd /c start BrowserOS` which doesn't resolve)

Install (runPostInstall):
- macOS: hdiutil attach → cp -R to /Applications → hdiutil detach
- Linux: chmod +x for AppImage, dpkg -i instruction for .deb
- Windows: launches installer exe
- --deb flag now errors on non-Linux platforms

Removed auto-launch from newClient() — CLI never does surprising things.

Sources verified from:
- packages/browseros/build/common/context.py (binary names per platform)
- packages/browseros/build/modules/package/linux.py (.deb structure, .desktop file)
- packages/browseros/chromium_patches/chrome/install_static/chromium_install_modes.h
  (Windows base_app_name="BrowserOS", registry GUID, install paths)
- /Applications/BrowserOS.app/Contents/Info.plist (bundle ID)
2026-03-26 23:12:55 +05:30
Dani Akash
392312f203 ci: only run PR title validation on open and edit (#565)
Remove synchronize and reopened triggers since this workflow only
validates the PR title, which doesn't change on new commits or reopen.
2026-03-26 23:06:11 +05:30
Dani Akash
0f193055c7 fix: broaden connection error detection for main page and sidepanel (#563)
* fix: broaden connection error detection for main page and sidepanel

The connection error check required both "Failed to fetch" AND
"127.0.0.1" in the error message. On the main page, the browser
only produces "Failed to fetch" without the IP, so users saw a
generic "Something went wrong" instead of the troubleshooting link.

Broaden detection to also match "localhost" and bare "Failed to fetch"
errors that don't contain an external URL. Also pass providerType in
NewTabChat so provider-specific errors render correctly.

Closes #526

* fix: simplify connection error detection

All chat requests go through the local BrowserOS agent server, so any
"Failed to fetch" error is always a local connection issue. Remove the
unnecessary 127.0.0.1/localhost/URL checks.

* fix: pass providerType to agentUrlError ChatError instances
2026-03-26 20:55:40 +05:30
Dani Akash
f45cb58889 fix: stop sending port-in-use errors to Sentry (#558)
Port conflicts are expected — Chromium retries with a different port.
These errors were flooding Sentry (14k+ events) without user impact.

- handleStartupError: move Sentry.captureException below the
  port-in-use check so it only fires for unexpected startup errors
- handleControllerStartupError: skip Sentry capture for port errors
- index.ts: exit early for port errors before Sentry capture
2026-03-26 09:32:18 +05:30
shivammittal274
37ead6d129 fix: add cursor-pointer to credit badge in sidepanel (#554) 2026-03-26 00:09:58 +05:30
Nikhil
5ea9463030 fix: widen scheduled task results dialog and add horizontal scroll for tables (#549)
- Change dialog width from sm:max-w-2xl (672px) to sm:w-[70vw] sm:max-w-4xl
  so it takes 70% of viewport width, capped at 896px
- Add overflow-x-auto on table wrappers so wide tables scroll horizontally
  instead of being clipped

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 16:27:46 -07:00
shivammittal274
dde35ccbd5 feat: integrate models.dev for dynamic LLM provider/model data (#547)
* feat: integrate models.dev for dynamic LLM provider/model data (#TKT-657)

Replace hardcoded model lists with data sourced from models.dev so new
providers and models appear automatically when the community adds them.

- Add build script (scripts/generate-models.ts) that fetches models.dev/api.json
  and outputs a compact JSON with 10 providers and 520 models
- Replace hardcoded MODELS_DATA (50 models) with dynamic models.dev lookups
- Add searchable model combobox (Popover + Command) replacing plain Select dropdown
- Enrich provider templates with models.dev metadata (context window, image support)
- Keep chatgpt-pro, qwen-code, browseros, openai-compatible as hardcoded providers

* fix: address review — remove ollama-cloud mapping, fix default models, remove dead code

- Remove ollama from PROVIDER_MAP (ollama-cloud has cloud models, not local)
- Add ollama to CUSTOM_PROVIDER_MODELS with empty list (users type custom IDs)
- Update defaultModelIds to ones that exist in models.dev data:
  openrouter → anthropic/claude-sonnet-4.5
  lmstudio → openai/gpt-oss-20b
  bedrock → anthropic.claude-sonnet-4-6
- Remove dead isCustomModel export
- Regenerate models-dev-data.json (9 providers, 486 models)

* fix: model suggestion list focus/dismiss behavior

- List only opens when input is focused or user types
- Clicking a model selects it and closes the list
- Clicking outside (blur) dismisses the list
- onMouseDown preventDefault on list items prevents blur race condition

* refactor: extract ModelPickerList component with proper open/close UX

- Collapsed state: Select-like trigger showing selected model + chevron
- Expanded state: search input + scrollable filtered list, inline
- Click outside or Escape to close, Enter to submit custom model
- Extracted as separate component (reduces dialog nesting, testable)
- No more setTimeout hacks for blur handling

* chore: remove plan doc from repo
2026-03-25 02:41:07 +05:30
Dani Akash
7f20319272 docs: add OAuth provider setup guides for ChatGPT Pro, GitHub Copilot, and Qwen Code (#545)
* docs: add setup guides for ChatGPT Pro, GitHub Copilot, and Qwen Code

Add individual OAuth setup guide pages with step-by-step screenshots
for each provider. Add "Use Your Existing Subscription" section to the
Bring Your Own LLM page with card links to each guide. Register pages
in docs navigation.

* docs: add ChatGPT Pro setup screenshots

* docs: use custom provider icons for OAuth setup cards

* docs: inline SVG icons in provider cards for dark mode support

* docs: place provider icons above card titles
2026-03-24 18:29:20 +05:30
shivammittal274
c8204efab6 feat: improve rate limit UX, usage page, and provider selector (#544)
* feat: improve rate limit UX, usage page, and provider selector

- Show "Add your own provider for unlimited usage" CTA when BrowserOS
  credits are exhausted or daily limit is reached
- Fix credit exhaustion detection to match actual error message
- Improve Usage page: remove disabled Add Credits button, add "Coming
  soon" badge, add "Want unlimited usage?" section linking to providers
- Add "+ Add Provider" button at bottom of chat provider selector dropdown

* fix: use asChild pattern for Button+anchor in usage page

Replace nested <a><Button> (invalid HTML) with Button asChild
pattern per shadcn/ui convention.
2026-03-24 18:01:42 +05:30
shivammittal274
fb5143b563 feat: UI improvements for OAuth dialog, provider badges, and events docs (#543)
* feat: UI improvements for OAuth dialog, provider badges, and events docs

- Replace OAuth device code toast with a proper Dialog showing the code
  prominently with a copy button (GitHub Copilot, Qwen Code, ChatGPT Pro)
- Add "New" badge on provider template cards for ChatGPT Plus/Pro,
  GitHub Copilot, and Qwen Code with orange border highlight
- Add events.md documenting all analytics events across the platform

* fix: add verificationUri to DeviceCodeDialog for popup-blocked fallback

Add verificationUri to PendingDeviceCode interface and pass it from
both handleClientAuth and handleServerAuth. Render a fallback "Open
verification page" link in DeviceCodeDialog so users can navigate
to the auth page if the popup was blocked.
2026-03-24 17:27:27 +05:30
Dani Akash
fe257cd8d1 feat: only parse browseros provider errors (#542) 2026-03-24 14:43:05 +05:30
shivammittal274
890d3406dd feat: promote BrowserOS as MCP with UI improvements (#541)
- Add MCP promo banner on AI providers page with "New" badge and
  "66+ tools" highlight, linking to /settings/mcp
- Add Quick Setup section on MCP settings page with copy-paste
  commands for Claude Code, Gemini CLI, Codex, Claude Desktop, OpenClaw
- Consolidate MCP settings: move restart button inline with server URL,
  remove separate MCP Server Settings card
- Add analytics event for promo banner clicks
2026-03-24 03:08:08 +05:30
shivammittal274
c316e09c11 feat: add source tag to tool_executed PostHog events (#538)
Add `source: 'mcp' | 'chat'` property to all `tool_executed` metrics
events so we can distinguish tool calls from external MCP clients
(Claude Code, Cursor) vs the built-in BrowserOS agent in PostHog.

- register-mcp.ts: source='mcp' (browser tools via MCP endpoint)
- register-klavis-mcp.ts: source='mcp' (Klavis tools via MCP endpoint)
- tool-adapter.ts: source='chat' (browser tools via chat agent)
- ai-sdk-agent.ts: source='chat' (Klavis/external MCP tools via chat agent, previously untracked)
- filesystem/utils.ts: source='chat' (filesystem tools via chat agent)
2026-03-24 02:03:18 +05:30
shivammittal274
65547c60c0 fix(eval): clean up eval configs and add test-clado-api script (#540)
Consolidate 13 configs down to 7 with uniform settings:
- 3 weekly (CI): browseros-agent, browseros-oe-agent, browseros-oe-clado
- 4 test (local): test_gemini-computer-use, test_yutori-navigator, test_webvoyager, test_mind2web
- All configs: headless=false, captcha block, full browseros ports, restart_server_per_task

Deleted: debug-test, mind2web-test, tool-loop-test, orchestrator-executor-test,
orchestrator-executor-clado-test, fireworks-minimax-m2, webvoyager-test

Added: test-clado-api.ts script, browseros-oe-agent-weekly.json (OE with AI SDK executor)
2026-03-24 01:28:05 +05:30
shivammittal274
0babc05077 feat(eval): NopeCHA CAPTCHA solver integration (#537)
* feat(eval): show mean score instead of pass/fail in report and viewer

* feat(eval): integrate NopeCHA CAPTCHA solver into eval pipeline

Add CAPTCHA detection and waiting so screenshots capture post-solve state.
Run headed with xvfb on CI since headless breaks extension content scripts.

- Add CaptchaWaiter module (detect reCAPTCHA/hCaptcha/Turnstile, poll until solved)
- Add optional `captcha` config block to EvalConfigSchema
- Wait for CAPTCHA solve before screenshot in single-agent and orchestrator-executor
- Patch NopeCHA manifest with API key before launching workers
- Fix CAPTCHA_EXT_DIR path (was pointing one level too high)
- Remove --incognito (extensions don't run in incognito; fresh user-data-dir isolates)
- CI: install xvfb, run headed via xvfb-run, pass NOPECHA_API_KEY secret
2026-03-24 00:14:16 +05:30
Nikhil
1270b5b55c feat: new manifest perms (#536)
* feat: new manifest perms

* fix: minor

* fix: minor
2026-03-23 09:31:07 -07:00
Nikhil
e97d8bc1cb fix: remove daily rate-limit middleware (#535)
* fix: remove daily rate-limit middleware

The daily conversation rate limit is no longer needed. Remove the
middleware, RateLimiter class, fetch-config, error type, shared
constants, DB schema table, and integration tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove unused getDb() method

No longer needed after rate-limiter removal.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 08:31:20 -07:00
Dani Akash
5109ca4347 feat: added scope in server error logs (#533)
* feat: added scope in server error logs

* fix: prevent double capture on chat request
2026-03-23 20:47:28 +05:30
shivammittal274
f14942c6f9 feat(eval): show mean score instead of pass/fail in report and viewer (#534) 2026-03-23 20:28:34 +05:30
Dani Akash
86ec88ed80 feat: sentry improvements (#532)
* feat: process request record from sentry locally

* feat: added analytics for logged in users
2026-03-23 19:45:28 +05:30
Dani Akash
4928b7e84b fix: no current window and sentry context (#531)
* fix: error reporting and better breadcrumbs

* fix: lint issues
2026-03-23 18:46:39 +05:30
shivammittal274
94a1a701f6 fix(eval): include browser context in agent prompt (#530)
The eval's single-agent was passing raw task.query as the prompt,
without browser context (active tab URL, title). The agent didn't
know which page it was on, causing it to ask "which website?" instead
of browsing.

Use formatUserMessage() (same as chat-service.ts) to include browser
context in the prompt. Re-export formatUserMessage from agent/tool-loop.
2026-03-23 17:42:03 +05:30
Dani Akash
ecf2efa857 fix: add unlimited storage permission to agent (#529) 2026-03-23 17:36:26 +05:30
shivammittal274
026c6a03a3 feat(eval): auto-trigger eval on agent/tools changes pushed to main (#528) 2026-03-23 16:52:30 +05:30
Nikhil
2b53daf641 fix: prevent deleted scheduled tasks from reappearing after sync (#518)
* fix: prevent deleted scheduled tasks from reappearing after sync

When a scheduled task was deleted, the sync function would see the
remote job missing locally and re-add it, undoing the delete. Fix by
tracking pending deletions in storage so the sync function deletes
them from the backend instead of re-adding them locally.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use read-modify-write for pending deletions to prevent concurrent clobber

Re-read pendingDeletionStorage before write-back and only remove
resolved IDs, preserving any new entries added by concurrent
removeJob calls during the sync's network I/O.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-21 11:31:57 -07:00
Nikhil
3cc946ded8 fix(ci): report test pass/fail status on PRs (#520)
The test workflow captured exit codes but never failed the job, so PR
checks always showed green even when tests failed. Exit with the
captured code in the summarize step so each suite properly reports
pass/fail. Not a required check, so failures remain non-blocking.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-21 11:31:23 -07:00
shivammittal274
70be5c5c21 fix(eval): log agent errors in task progress for CI visibility (#523) 2026-03-21 23:33:19 +05:30
shivammittal274
0f9d93058f chore(eval): remove unused env vars from workflow (OPENROUTER, OPENAI) (#522) 2026-03-21 23:22:03 +05:30
shivammittal274
cafed57832 fix(eval): use CLAUDE_CODE_OAUTH_TOKEN for performance grader auth (#521) 2026-03-21 23:14:23 +05:30
shivammittal274
f157436e7d feat(eval): switch to Linux GitHub-hosted runner (#519)
* feat(eval): switch to ubuntu-latest runner, add OE-Clado config

- Switch workflow from self-hosted Mac Studio to ubuntu-latest
- Install BrowserOS Linux .deb in CI (no self-hosted runner needed)
- Add browseros-oe-clado-weekly.json config for orchestrator-executor
- Fix report chart to show date+time (not just date)
- Make BROWSEROS_BINARY configurable via env var

* feat(eval): add NopeCHA captcha solver extension to eval runs

- Auto-load NopeCHA extension in eval Chrome instances
- Works in incognito + headless mode
- CI workflow downloads NopeCHA before eval
- extensions/ directory gitignored (downloaded at runtime)

* feat(eval): per-config concurrency — different configs run in parallel

* feat(eval): remove concurrency limit — all runs execute in parallel
2026-03-21 23:04:45 +05:30
Nikhil
ba7892322b ci: run BrowserOS test suites on PRs (#514)
* ci: run browseros tests on pull requests

* refactor: rework 0320-github_action_for_tests based on feedback

* refactor: rework 0320-github_action_for_tests based on feedback

* chore: add CI artifacts to .gitignore

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove mikepenz/action-junit-report to fix check suite misattribution

The JUnit report action creates check runs that GitHub associates with the
CLA check suite instead of the Tests check suite, causing test reports to
appear under "CLA Assistant" in the PR checks UI.

Remove the action and rely on job status + step summary + artifact upload
for test result visibility.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-21 09:46:36 -07:00
shivammittal274
4e90b4561a feat(eval): weekly eval pipeline with R2 uploads and trend dashboard (#516)
* feat(eval): weekly eval pipeline with R2 uploads and trend dashboard

Add infrastructure for running weekly evaluations and tracking score
trends over time:

- Auto-generated output dirs: results/{config-name}/{timestamp}/
  Each eval run gets its own timestamped folder, nothing is overwritten.

- upload-run.ts: uploads eval results to Cloudflare R2. Supports
  uploading a specific run or all un-uploaded runs for a config.

- weekly-report.ts: generates an interactive HTML dashboard from R2
  data. Config dropdown, trend chart with hover tooltips, searchable
  runs table. Groups runs by config name.

- viewer.html: client-facing 3-column run viewer (task list,
  screenshots with autoplay, agent stream with messages.jsonl).
  Shows performance grader axis breakdown with per-axis scores.

- browseros-agent-weekly.json: weekly benchmark config (kimi-k2p5,
  webbench-2of4-50, 10 workers, performance grader, headless).

- eval-weekly.yml: GitHub Actions workflow with cron (Saturday 6am)
  and manual trigger. Runs on self-hosted Mac Studio runner.
  Concurrency group ensures only one eval runs at a time.

- Dashboard updates: load previous runs, messages.jsonl viewer,
  grade badges show percentages, async stream loading.

- Grader updates: timeout 30min, max turns 100, DOM content
  verification guidance for performance grader.

* fix(eval): address Greptile review — injection, nested dirs, escaping

- Fix script injection in eval-weekly.yml: pass github.event.inputs
  through env var instead of interpolating into shell
- Fix /api/runs to enumerate nested results/{config}/{timestamp}/ dirs
- Fix /api/load-run to allow single-slash run names (config/timestamp)
- Add HTML escaping for R2-sourced values in weekly-report.ts
- Escape axis names in viewer.html renderAxesBreakdown

* fix(eval): fix biome lint — non-null assertion, template literals

* fix(eval): fix biome errors — replace var with let, fix inner function declaration

* fix(eval): address Greptile P2 issues

- isRunDir: check all subdirs for metadata.json, not just first 3
- eval-runner: guard configPath for dashboard-driven runs (fallback to 'eval')
- load-run: default unknown termination_reason to 'failed' not 'completed'

* feat(eval): make BROWSEROS_BINARY configurable via env var
2026-03-21 22:12:52 +05:30
shivammittal274
86eed82350 fix: lazy OAuth callback server with cancel+retry (Codex CLI pattern) (#515)
The OAuth callback server on port 1455 was bound eagerly at startup,
crashing the server if another BrowserOS instance was already running.

Rewrite as a lazy class (OAuthCallbackServer) that:
- Only binds port 1455 when the user initiates a ChatGPT Pro login
- Sends GET /cancel to any existing server on the port first, then
  retries up to 5 times (follows Codex CLI's cancel+retry pattern)
- Exposes /cancel endpoint so other instances/tools can cancel us
- Releases the port after the OAuth callback arrives
- Device-code providers (GitHub Copilot, Qwen) never touch port 1455

This allows running eval, dev instances, and multiple BrowserOS
instances without port conflicts. OAuth login works on whichever
instance initiates it — the others continue without OAuth.
2026-03-21 16:44:03 +05:30
Nikhil
be6ed22af4 test: fix BrowserOS tool test harness regressions (#513)
* test: fix browseros tool test harness regressions

* test: align working directory naming in page action tests
2026-03-20 12:05:39 -07:00
Nikhil
149cde118d chore: bump server version, offset and patch for release (#512) 2026-03-20 11:45:12 -07:00
Nikhil
9bc5e666c4 feat: auto-discover server port via ~/.browseros/server.json (#504)
* feat: auto-discover server port via ~/.browseros/server.json

Server writes its port to ~/.browseros/server.json on startup so the CLI
can auto-discover the server URL without requiring `browseros-cli init`.

Discovery chain: BROWSEROS_URL env > config.yaml > server.json > error

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address review feedback for PR #504

- Use synchronous unlinkSync in stop() since process.exit() fires
  immediately after, abandoning any pending async operations
- Wrap writeServerConfig in try/catch so a write failure doesn't crash
  a healthy server for a convenience feature

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: type server discovery config and add version metadata

Add ServerDiscoveryConfig interface to @browseros/shared and enrich
server.json with server_version, browseros_version, and chromium_version.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: normalize URL from server.json for consistency

All other URL sources (env var, config.yaml) pass through
normalizeServerURL; apply the same to the server.json path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 11:37:00 -07:00
Nikhil
2271277b4d feat: add voice input to new tab search bar (#509)
* feat: add voice recording UI with waveform overlay to new tab search bar

Add a microphone button to the NewTab search bar that opens a fullscreen
recording overlay powered by react-voice-visualizer. The overlay shows a
real-time waveform visualization during recording, recording time, and a
stop button. On completion, the audio is transcribed via the existing
gateway endpoint and the transcript auto-navigates to inline chat.

Changes:
- Extract transcribeAudio() to shared lib/voice/transcribe-audio.ts
- Add VoiceRecordingOverlay component with react-voice-visualizer
- Add Mic button to NewTab search bar
- Track analytics via existing NEWTAB_VOICE_* events
- Handle cancel (backdrop click) vs submit (stop button) correctly

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address PR review comments for voice recording overlay

- Reset processingRef on transcription error to prevent stuck state
- Use stable callback refs to prevent useEffect re-runs from inline
  arrow function props (fixes timer reset and unnecessary re-processing)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: replace voice overlay with inline sidepanel-style voice UI

Remove react-voice-visualizer dependency and VoiceRecordingOverlay.
Instead use the same inline voice pattern as the sidepanel ChatInput:
- Waveform bars replace the search input during recording
- Mic/stop/loading button states in the search bar
- Transcript populates the search input on completion
- Voice error shown inline below the search bar

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 11:33:01 -07:00
Nikhil
f865d301a2 test: add build smoke test to catch compile failures (#511)
* test: add build smoke test to catch compile failures

Compiles the server binary (darwin-arm64) and verifies --version outputs
the correct version from package.json. Uses an empty resource manifest
and stub env vars so the test runs without R2 access or real secrets.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address review feedback for PR #511

- Derive build target from process.platform/arch for CI portability
- Include binary stderr in --version assertion for better diagnostics

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 11:16:57 -07:00
Nikhil
6f398f0b36 fix: replace sharp with jimp to fix compiled binary crash (#510)
sharp is a native C module (libvips) whose .node binaries can't be
embedded in Bun compiled executables. It was imported at the top level
in copilot-fetch.ts, crashing the entire server at startup.

Replace with jimp (pure JavaScript, zero native deps) which bundles
cleanly into compiled binaries. Same resize algorithm preserved.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 11:06:05 -07:00
shivammittal274
8548bcf50a feat: credit-based tracking for BrowserOS provider (#489)
* feat: add credit-based tracking for BrowserOS provider

Send X-BrowserOS-ID header on all LLM requests through the BrowserOS
gateway for per-installation credit tracking. Handle 429 CREDITS_EXHAUSTED
as non-retryable. Add GET/PUT /credits endpoints to check and manage
credit balance.

* docs: add credits tracking UI design

Design for showing credit balance in side panel chat header (color-coded
badge) and a dedicated Usage & Billing settings page. Credits refresh
after each completed message turn or on exhaustion error.

* docs: add credits tracking UI implementation plan

8-task plan covering useCredits hook, CreditBadge component, ChatHeader
integration, message completion refresh, ChatError CREDITS_EXHAUSTED
handling, Usage & Billing settings page, and route/sidebar registration.

* feat: add useCredits React Query hook

* feat: add CreditBadge component with color thresholds

* feat: show credit badge in chat header for BrowserOS provider

* feat: refresh credits after chat message completion and on error

* feat: handle CREDITS_EXHAUSTED error in chat

* feat: add Usage & Billing settings page

* feat: register usage page route and sidebar entry

* fix: lint and formatting fixes for credit tracking UI

* fix: separate credits exhausted from Kimi rate limit in ChatError, redesign Usage page

* chore: remove PUT /credits endpoint and setCredits function

* fix: extract shared credit colors, add error state to UsagePage, use dailyLimit from gateway

* fix: make dailyLimit required in CreditsInfo (gateway always returns it)

* feat: gate credits UI behind CREDITS_SUPPORT feature flag (server >= 0.0.78)
2026-03-20 22:49:00 +05:30
shivammittal274
e3601bfdc1 feat: gate Qwen Code behind server version 0.0.77 (#508) 2026-03-20 20:07:39 +05:30
Dani Akash
2b4fdf1aad feat: improved multi tab agent workflow (#507)
* feat: updated multitab workflow

* fix: updated prompt with fix for test cases

* fix: active agent glow

* fix: review comments
2026-03-20 18:31:36 +05:30
shivammittal274
11d15d079f feat: alibaba qwen oauth (#506)
* feat: add Qwen Code as OAuth LLM provider with refactored OAuth hooks

Add Alibaba Qwen Code as a third OAuth provider using Device Code flow
with PKCE. Free tier: 2,000 requests/day, up to 1M token context.

Refactoring:
- Extract useOAuthProviderFlow hook (eliminates ~180 lines of duplicated
  OAuth logic from AISettingsPage for ChatGPT Pro + Copilot + Qwen)
- Extract resolveOAuthConfig in config.ts (shared resolver for all OAuth
  providers, parameterized by provider name, default model, refresh flag)
- Generalize token-manager device code flow to support PKCE
  (code_challenge/code_verifier) and form-urlencoded content type

New code:
- Qwen Code provider config with PKCE + form encoding flags
- Provider factories (both provider.ts and provider-factory.ts)
- Extension UI (template card, models, analytics, dialog)

* fix: use portal.qwen.ai as API base URL for OAuth tokens

DashScope (dashscope.aliyuncs.com) expects Alibaba Cloud API keys,
not OAuth tokens from chat.qwen.ai. The correct endpoint for OAuth
Bearer tokens is portal.qwen.ai/v1.

* fix: correct Qwen Code model IDs and context windows

- coder-model (1M context): virtual alias that routes to best model
- qwen3-coder-plus (1M): was incorrectly 131K
- qwen3-coder-flash (1M): new, speed-optimized variant
- qwen3.5-plus (1M): was incorrectly 1048576 (power-of-two vs decimal)
- Removed qwen3-coder-next (local/self-hosted, not available via OAuth)
- Default model changed to coder-model (auto-routes server-side)

* fix: move Qwen device code request to extension (bypasses WAF)

Alibaba WAF blocks server-side requests to chat.qwen.ai. Move the
initial device code request to the extension (browser context with
cookies), then hand off the deviceCode + codeVerifier to the server
for background polling via new POST /oauth/:provider/poll endpoint.

* fix: persist OAuth flow-started flag in sessionStorage

The flowStartedRef was lost when the component remounted (e.g. user
navigated to onboarding then back to settings). Use sessionStorage
to persist the flag so auto-create works after navigation.

* revert: remove sessionStorage for OAuth flow flag

Revert to simple useRef pattern matching the original ChatGPT Pro
implementation. The auto-create works when the user stays on the
AI settings page during auth.

* revert: move Qwen back to server-side device code flow

WAF block was temporary (rate-limiting), not permanent. Server-side
fetch to chat.qwen.ai now works. Reverted client-side device code
approach — Qwen now uses the same clean server-side flow as Copilot.

Removed: clientSideDeviceCode config, startClientSideDeviceCode(),
POST /oauth/:provider/poll endpoint, startDeviceCodePolling().

* feat: add WAF detection, rate-limit protection, and token storage endpoint

- Detect WAF captcha responses (HTML instead of JSON) in device code
  request and token polling, with user-friendly error messages
- Add 30s cooldown on "USE" button to prevent rapid clicks triggering WAF
- WAF-blocked poll requests silently retry instead of aborting
- Add POST /oauth/:provider/token endpoint for storing externally-provided
  tokens (useful for future fallback flows)
- Add storeTokens() method to OAuthTokenManager
- Pass server error messages through to extension toast notifications

* refactor: remove 30s cooldown, simplify OAuth hook

The hook is now identical for all providers — server handles retries
via activeDeviceFlows.delete(). Removed flowStartedAtRef cooldown
that was blocking legitimate retries.

* feat: client-side OAuth for Copilot and Qwen Code

Move device code OAuth flow to the extension for GitHub Copilot and
Qwen Code. The extension makes requests using Chrome's network stack,
which bypasses Alibaba WAF TLS fingerprint detection that blocks
server-side Bun/Node.js fetch.

New files:
- client-oauth.ts: Client-side device code + PKCE + token polling

Changes:
- useOAuthProviderFlow: handleClientAuth() for providers with clientAuth
  config, handleServerAuth() for others (ChatGPT Pro)
- AISettingsPage: clientAuth config for Copilot and Qwen Code
- WAF detection: opens provider site for captcha solving on block

Server-side device code flow preserved as fallback (token-manager.ts,
providers.ts). Token storage via POST /oauth/:provider/token endpoint.

* fix: export OAuthProviderFlowConfig type, fix typecheck errors

- Export OAuthProviderFlowConfig interface so AISettingsPage can use it
  instead of duplicating the type inline
- Fix string | null → string | undefined for agentServerUrl parameter
2026-03-20 17:46:48 +05:30
Nikhil
9257832acf feat: gate ChatGPT Pro and GitHub Copilot behind server version 0.0.77 (#503)
Add CHATGPT_PRO_SUPPORT and GITHUB_COPILOT_SUPPORT feature flags gated
on minServerVersion 0.0.77. Hide template cards and provider type
dropdown options when the server doesn't support the OAuth endpoints.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 16:43:09 -07:00
Nikhil
7bde0d59fa chore: bump chromium version (#502) 2026-03-19 16:22:13 -07:00
Nikhil
1c737b0f02 chore: bump server version (#501) 2026-03-19 16:17:50 -07:00
Nikhil
5d0a2b9bfe feat: add model selector to newtab search bar (#499)
* feat: add model selector to newtab search bar

Add AI provider/model selector button to the newtab homepage footer bar,
matching the existing button aesthetics (Workspace, Tabs, Apps). Reuses
ChatProviderSelector popover from sidepanel. Users can now see and change
their AI provider before starting a conversation from the newtab page.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: clean up newtab footer with icon-only buttons

Reduce visual clutter in the search bar footer by converting Provider,
Workspace, and Tabs buttons to compact icon-only buttons (8x8). Text
labels and chevron indicators are removed — native title tooltips
provide discoverability on hover. Apps button on the right keeps its
text label per user preference.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add hover-expand labels to newtab footer icon buttons

Replace static title tooltips with smooth hover-expand animation —
buttons show icon-only by default, text label slides out on hover
via max-w transition. Gives a clean compact look while keeping
labels discoverable.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: revert workspace/tabs to full text, keep provider hover-expand only

Restore full text labels for Workspace and Tabs buttons. Only the
provider selector uses the compact icon + hover-expand pattern.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: simplify provider selector to plain icon button

Remove hover-expand animation, use a simple icon-only button with
native title tooltip for the provider selector.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 16:14:15 -07:00
shivammittal274
720baaed3e feat: add GitHub Copilot as OAuth LLM provider (#500)
* feat: add GitHub Copilot as OAuth-based LLM provider

Add GitHub Copilot as a second OAuth provider using the Device Code flow
(RFC 8628). Users authenticate via github.com/login/device, and the server
polls for token completion. Supports 25+ models through a single Copilot
subscription.

Key changes:
- Device Code OAuth flow in token manager (poll with safety margin)
- Custom fetch wrapper injecting Copilot headers + vision detection
- Provider factory using createOpenAICompatible for Chat Completions API
- Extension UI with template card, auto-create on auth, and disconnect

* fix: address PR review comments for GitHub Copilot OAuth

- Validate device code response for error fields (GitHub can return 200
  with error payload)
- Store empty refreshToken instead of access token for GitHub tokens
- Add closeButton to Toaster for dismissing device code toast

* fix: add github-copilot to agent provider factory

The chat route uses a separate provider-factory.ts (agent layer) from the
test-provider route (llm/provider.ts). Added createGitHubCopilotFactory
to the agent factory so chat works with GitHub Copilot.

* fix: add github-copilot to provider icons, models, and dialog

- Add Github icon from lucide-react to providerIcons map
- Add 8 Copilot models (GPT-4o, Claude, Gemini, Grok) to models.ts
- Add github-copilot to NewProviderDialog zod enum, validation skip,
  canTest check, and OAuth credential message

* fix: reorder copilot models with free-tier models first

Put models available on Copilot Free at the top (gpt-4o, gpt-4.1,
gpt-5-mini, claude-haiku-4.5, grok-code-fast-1), followed by
premium models that require paid Copilot subscription.

* fix: set correct 64K context window for Copilot models

Copilot API enforces a 64K input token limit regardless of the
underlying model's native context window. Updated all model entries
and the default template to 64000 so compaction triggers correctly.

* fix: use actual per-model prompt limits from Copilot /models API

Queried api.githubcopilot.com/models for real max_prompt_tokens values.
GPT-4o/4.1 have 64K, Claude/gpt-5-mini have 128K, GPT-5.x have 272K.
Also updated model list to match what's actually available on the API
(e.g. claude-sonnet-4.6 instead of 4.5, added gpt-5.4/5.2-codex).

* feat: resize images for Copilot using VS Code's algorithm

Large screenshots cause 413 errors on Copilot's API. Resize images
following VS Code's approach: max 2048px longest side, 768px shortest
side, re-encode as JPEG at 75% quality. Uses sharp for server-side
image processing.

* fix: address all Greptile P1 review comments

- Add .catch() on fire-and-forget pollDeviceCode to prevent unhandled
  rejection crashes (Node 15+)
- Add deduplication guard (activeDeviceFlows Set) to prevent concurrent
  device code flows for the same provider
- Add runtime validation of server response in frontend before calling
  window.open() and showing toast
- Remove dead GITHUB_DEVICE_VERIFICATION constant from urls.ts

* fix: upgrade biome to 2.4.8, fix all lint errors, and address review bugs

- Upgrade biome from 2.4.5 to 2.4.8 (matches CI) and migrate configs
- Fix image resize: only re-encode when dimensions actually change
- Fix device code polling: retry on transient network errors instead of aborting
- Allow restarting device code flow (clear old flow instead of throwing 500)
- Fix pre-existing noNonNullAssertion and noExplicitAny lint errors globally

* fix: address Greptile P2 review — image resize and config guard

- Fix early-return guard: check max/min sides against their respective
  limits (MAX_LONG_SIDE/MAX_SHORT_SIDE) instead of both against SHORT
- Preserve PNG alpha: detect hasAlpha and keep PNG format instead of
  unconditionally converting to lossy JPEG
- Keep browserosId guard in resolveGitHubCopilotConfig consistent with
  ChatGPT Pro pattern (safety check that caller context is valid)

* feat: update Copilot models to full list from pricing page, default to gpt-5-mini

Added all 23 models from GitHub Copilot pricing page. Ordered with
free-tier models first (gpt-5-mini, claude-haiku-4.5), then premium.
Changed default from gpt-4o to gpt-5-mini since it's unlimited on
Pro plan and has 128K context (vs gpt-4o's 64K limit).
2026-03-20 02:33:09 +05:30
shivammittal274
cee9c764b1 fix(skills): read-only view mode for built-in skills (#494)
* fix(skills): read-only view mode for built-in skills

- SkillCard shows Eye icon + "View" for built-in, Pencil + "Edit" for user
- SkillDialog in read-only mode: disabled fields, no toolbar on markdown
  editor, "View Skill" title, "Close" button, no "Update Skill"
- Hide tip section in read-only mode

* fix(skills): use react-markdown for read-only skill view

Replace MDXEditor with react-markdown for viewing built-in skills.
MDXEditor chokes on code fences, angle brackets, and image syntax
causing content truncation. react-markdown handles standard markdown
correctly with no rendering issues.
2026-03-19 23:48:51 +05:30
Nikhil
7bdeeb85d5 fix: revert: convert settings to popup dialog (#477) (#498)
* Revert "feat: convert settings to popup dialog (#477)"

This reverts commit 42aa0ff1ef.

* fix: address review feedback for PR #498

- Remove erroneous SETTINGS_PAGE_VIEWED_EVENT tracking from SidebarLayout
  (was firing on every non-settings page navigation)
- Fix mobile settings sidebar not closing on route change by merging
  setMobileOpen(false) into the pathname-dependent analytics useEffect

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 11:13:14 -07:00
Dani Akash
19069cb9c4 fix: newtab layout (#497) 2026-03-19 20:40:38 +05:30
Dani Akash
5bb6143373 feat: display selected text from page in sidepanel (#496)
* feat: select text and pass to sidepanel

* fix: lint issues

* fix: persist selection across tabs

* fix: review comments

* fix: change when the selection is cleared

* feat: sanitize url
2026-03-19 20:21:31 +05:30
Dani Akash
f4d4b73a24 fix: improved memory tools (#495)
* fix: new prompt update tool

* fix: memory search tool

* fix: all review comments

* chore: remove dead code
2026-03-19 19:01:25 +05:30
Dani Akash
d965698905 fix: biome & tsc setup across repo (#493)
* fix: biome lint issues

* fix: code quality workflow

* fix: all lint issues

* chore: test lefthook pre-commit hook

* chore: test lefthook with agent file

* chore: revert test comment from lefthook verification

* feat: setup tsgo for typechecking agent

* fix: typecheck cli command

* fix: early return to prevent errors
2026-03-19 18:18:24 +05:30
shivammittal274
50b2f45590 fix(skills): UI section separation and fix find-alternatives rendering (#492)
* fix(skills): UI section separation and fix find-alternatives rendering

- Split skills page into "My Skills" (user) and "BrowserOS Skills" (built-in) sections
- Fix find-alternatives SKILL.md — replace angle bracket placeholders with curly
  braces to prevent MDXEditor from parsing them as JSX and rendering empty content

* fix(skills): bump find-alternatives to v1.1 for CDN sync
2026-03-19 17:38:28 +05:30
Dani Akash
1b88ade021 feat: updated homepage chat (#481)
* feat: updated chat ui from homepage

* fix: vertical scroll

* fix: horizontal scroll issue

* fix: lint issues

* fix: header width

* fix: message input from home to chat

* feat: created sidebar header support in new tab chat

* fix: remove history from new tab chat

* fix: remove the shared element transition

* fix: lint issues

* fix: review comments

* fix: defer the sendMessage callback

* fix: all code concerns

* fix: preserve state of chat on homepage

* fix: review comments
2026-03-19 15:24:05 +05:30
shivammittal274
079a254fa4 fix(skills): separate built-in and user skills into distinct directories (#487)
* fix(skills): separate built-in and user skills into distinct directories

- Move built-in skills to ~/.browseros/skills/builtin/, user skills stay in root
- Unify seed + sync into single syncBuiltinSkills() function, delete seed.ts
- Preserve user's enabled/disabled state during remote sync version updates
- Add catalog reconciliation — remove built-in skills dropped from remote catalog
- Fallback to bundled defaults per-skill when remote sync fails
- One-time migration moves existing default skills from root to builtin/
- Add builtIn field to SkillMeta, determined by directory (not metadata)
- UI shows "Built-in" badge, hides delete button for built-in skills
- Reject deletion of built-in skills in service layer
- Check both dirs for ID collision on skill creation

* fix(skills): address review — dedup by id, guard applyEnabled regex

- loader.ts: deduplication now keys on skill.id (directory slug) not
  skill.name (display name), preventing silent drops on name collision
- remote-sync.ts: applyEnabled checks if regex matched before writing,
  logs warning if remote content lacks an enabled field

* fix(skills): reconciliation preserves bundled defaults, delete returns 403

- reconcileRemovedSkills now keeps DEFAULT_SKILLS IDs in the safe set,
  preventing delete-then-reinstall cycle that lost enabled:false state
- DELETE /skills/:id returns 403 for built-in skills instead of 500

* refactor(skills): simplify syncBuiltinSkills to single clean pass

Build content map (bundled + remote), iterate once, preserve enabled,
reconcile deletions. Removes 7 helper functions, 70 lines of code.

* refactor(skills): extract syncOneSkill, patch content before writing

- syncBuiltinSkills is now 15 lines: build map, iterate, clean up
- syncOneSkill: flat, patches enabled state before writing (single write)
- setEnabled: pure function for content patching
- removeObsoleteSkills: extracted from inline block
2026-03-19 13:35:47 +05:30
Felarof
42aa0ff1ef feat: convert settings to popup dialog (#477)
* feat: convert settings page to popup dialog, move workflows to main nav

Replace the dedicated settings page layout (SettingsSidebarLayout) with a
modal dialog (SettingsDialog) that opens on top of the current page. Settings
are now accessible via a dialog triggered from the main sidebar, eliminating
the confusing dual-sidebar navigation pattern.

- Create SettingsDialog with tabbed left panel and content area
- Move Workflows into main sidebar navigation (feature-gated)
- Remove /settings/* routes (except /settings/survey)
- Delete SettingsSidebarLayout and SettingsSidebar components
- Update backward compatibility redirects

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: setup new urls for the dialog box

* fix: dialog close button

* fix: settings analytics

* fix: address review comments

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Dani Akash <DaniAkash@users.noreply.github.com>
2026-03-18 23:26:13 +05:30
shivammittal274
4000f094f6 Feat/chatgpt pro polish (#484)
* fix: ChatGPT Pro UI polish — fix undefined display and add icon

- Fix "gpt-5.3-codex · undefined" — hide baseUrl when not set
- Add OpenAI icon for chatgpt-pro provider in icon map

* chore: rename ChatGPT Pro to ChatGPT Plus/Pro (supports both plans)

* chore: remove accidentally committed files
2026-03-18 22:51:22 +05:30
shivammittal274
151be81cee fix: ChatGPT Pro UI polish — fix undefined display and add icon (#483)
- Fix "gpt-5.3-codex · undefined" — hide baseUrl when not set
- Add OpenAI icon for chatgpt-pro provider in icon map
2026-03-18 22:23:28 +05:30
shivammittal274
46a8326140 feat: add ChatGPT Pro OAuth as LLM provider (#476)
* feat: add ChatGPT Pro OAuth as LLM provider

Adds OAuth 2.0 (Authorization Code + PKCE) flow so users can authenticate
with their ChatGPT Pro subscription to power BrowserOS's agent, matching
the pattern used by Codex CLI, OpenCode, and Pi.

Server:
- OAuth token lifecycle (PKCE, exchange, refresh, SQLite storage)
- Dedicated callback server on port 1455 (Codex client ID registration)
- Codex fetch wrapper routing API calls to chatgpt.com/backend-api
- Config resolution + provider factories for all code paths (chat, test, refine)

Extension:
- ChatGPT Pro template card with OAuth flow trigger
- Status polling hook + auto-create provider on auth success
- Model list with Codex-supported models (gpt-5.x-codex family)

* fix: address Greptile PR review comments

- Wire OAuth callback server stop handle into onShutdown (P1: port 1455 leak)
- Guard against missing refresh token + clear stale tokens on failed refresh (P1)
- Add logger.warn to silent catch in codex-fetch body mutation
- Document JWT trust assumption in parseAccessTokenClaims
- Source model ID from provider template instead of hard-coding

* simplify: remove unnecessary OAuth shutdown wiring and useCallback

- Revert OAuthHandle interface — callback server port releases on process exit
- Remove stopCallbackServer from shutdown flow (dead code)
- Remove all useCallback from useOAuthStatus per CLAUDE.md guidance

* style: add readonly modifiers and braces per TS style guide

* docs: add E2E test screenshots for ChatGPT Pro OAuth

* fix: strip item IDs from Codex requests to fix multi-turn conversations

* fix: preserve function_call_output IDs in Codex requests

* fix: resolve Codex store=false + tool-use incompatibility

- Pass providerOptions { openai: { store: false } } to ToolLoopAgent
  so the AI SDK inlines content instead of using item_reference
- Strip item IDs and previous_response_id in codex-fetch (safety net)
- Use .responses() model (Codex only speaks Responses API format)

* fix: remove non-Codex model gpt-5.2 from chatgpt-pro model list

* fix: strip unsupported Codex params and update model list

- Strip temperature, max_tokens, top_p from Codex requests (unsupported)
- Add all available Codex models including gpt-5.4, gpt-5.2, gpt-5.1

* chore: remove screenshots containing email

* feat: enable reasoning events for ChatGPT Pro Codex models

* chore: set reasoning effort to high for ChatGPT Pro

* feat: add configurable reasoning effort and summary for ChatGPT Pro

- Add reasoningEffort (none/low/medium/high) and reasoningSummary
  (auto/concise/detailed) dropdowns in the Edit Provider dialog
- Pass through extension → chat request → agent config → providerOptions
- Defaults: effort=high, summary=auto

* fix: strip max_output_tokens from Codex requests (fixes compaction)

* fix: address Greptile P1 issues

- Fix default model fallback: gpt-4o → gpt-5.3-codex (Codex endpoint)
- Clear stale tokens on refresh failure (prevents infinite retry loop)
- Only auto-create provider after explicit OAuth flow, not on page load
- Add catch block to auto-create effect with error toast
2026-03-18 22:07:43 +05:30
Dani Akash
4b18723a21 fix: undo shortcut in rewrite button (#472)
* fix: undo shortcut in rewrite button

* fix: address reviews
2026-03-18 07:04:48 +05:30
Nikhil
4909927c03 chore: bump PATCH and OFFSET (#479) 2026-03-17 17:41:45 -07:00
Nikhil
22c5e85707 chore: bump server version (#478) 2026-03-17 17:12:23 -07:00
shivammittal274
59b00a6837 feat: remote skill download and auto-sync (#468)
* feat: add remote skill download and auto-sync

Download default skills from remote catalog on first setup with
bundled fallback when offline. Background sync every 45 minutes
checks for new/updated skills without overwriting user-customized
ones. Tracks installed defaults via content hashes in a local
manifest file.

* feat: make skills catalog URL configurable and add generation script

Add SKILLS_CATALOG_URL env var (following CODEGEN_SERVICE_URL pattern)
with fallback to the default constant. Add script to generate
catalog.json from bundled defaults for static hosting.

* feat: add R2 upload script and use cdn.browseros.com for catalog URL

Add upload-skills-catalog.ts that generates and uploads catalog.json
to Cloudflare R2 (same infra as existing build artifacts). Update
default catalog URL to cdn.browseros.com/skills/v1/catalog.json.

* test: add E2E tests for remote skill sync against live CDN

* fix: address code review findings — security, validation, DRY

- Add path traversal protection via safeSkillDir in writeSkillFile
  and readSkillContent (reuses existing validation from service.ts)
- Add runtime type guards for catalog JSON and manifest JSON parsing
- Fix seedFromRemote to return false on partial failure so bundled
  fallback kicks in
- Add per-skill error handling in syncRemoteSkills so one bad skill
  doesn't crash the entire sync
- Wire stopSkillSync into Application.stop() shutdown path
- Extract version from frontmatter in seedFromBundled instead of
  hardcoding '1.0'
- Consolidate duplicated logic: reuse installSkill/writeSkillFile/
  contentHash/saveManifest from remote-sync.ts in seed.ts
- Extract shared catalog generation into scripts/catalog-utils.ts

* test: add flow tests for all four sync scenarios against live CDN

* refactor: remove redundant scripts and inline catalog generation

Drop generate-skills-catalog.ts, catalog-utils.ts, and
e2e-remote-sync.test.ts (covered by flows.test.ts). Inline
catalog generation into upload-skills-catalog.ts.

* test: add full E2E server flow test against live CDN

Tests all 7 steps of the real server lifecycle: fresh seed from CDN,
no-op sync, user edit preservation, skill reinstall, custom skill
protection, background timer firing, and second startup skip.

* chore: remove e2e-server-flow test

* fix: address Greptile review — entry validation, size limit, DRY, no-op saves

- Validate individual skill entries in catalog (id, version, content
  must all be strings) not just the top-level shape
- Add 1MB response size limit on catalog fetch to prevent resource
  exhaustion from compromised/misconfigured CDN
- Skip manifest save when sync cycle had no changes (avoids
  unnecessary disk I/O every 45 minutes)
- Share extractVersion via remote-sync.ts export, remove duplicate
  from seed.ts

* fix: prevent bundled fallback from overwriting partial remote seeds

When seedFromRemote partially fails, the bundled fallback now skips
skills already in the manifest (installed by the partial remote
seed). Also adds Content-Length early check before downloading the
full catalog response body.

* fix: run sync immediately on startup, not just on interval

Previously the first sync fired 45 minutes after boot. Now
startSkillSync runs one sync immediately so returning users
get skill updates right away.

* refactor: simplify sync — remote always wins, remove manifest

Remote catalog is the source of truth. If a skill exists in the
catalog, its version is compared against local frontmatter and
overwritten when newer. No manifest file, no content hashes.

User-created skills (IDs not in catalog) are never touched.

* fix: skip bundled skills already installed by partial remote seed

* chore: remove unreliable Content-Length check

* chore: remove size limit checks, fetch timeout is sufficient
2026-03-17 21:40:45 +05:30
Nikhil
44af9aea6d fix: clean-up old scripts (#474)
* fix: remove old scripts

* fix: remove vscode
2026-03-17 08:56:55 -07:00
Nikhil
1779e1e7bd fix: create user-data dir if missing (#473) 2026-03-17 08:30:39 -07:00
shivammittal274
2597cdbc70 feat: add Rewrite with AI for scheduled task prompts (#465)
* feat: add "Rewrite with AI" prompt refinement for scheduled tasks

Add a lightweight /refine-prompt endpoint that uses generateText to
rewrite rough scheduled task prompts into clear, actionable instructions.
The UI adds a sparkle-icon button next to the Prompt label in the
NewScheduledTaskDialog with loading state, undo support, and disabled
state when the textarea is empty.

* fix: clear stale undo ref on dialog re-open and pass providerId to refinePrompt

- Reset originalPromptRef when dialog opens and on form submit to
  prevent stale "Undo rewrite" button on re-open
- Accept optional providerId in refinePrompt() so the form's selected
  provider is used for refinement instead of always the system default

* fix: hide undo rewrite link while refinement is in flight

* fix: reset isRefining state on dialog re-open

* fix: ignore stale refine-prompt responses after dialog re-open

Use a request generation counter so that if the dialog is closed and
re-opened while a rewrite is in flight, the stale response is silently
discarded instead of overwriting the fresh form state.

* fix: invalidate stale refine requests on dialog reopen and rename to kebab-case

- Increment refineRequestIdRef on dialog open so in-flight requests
  from a previous session are discarded when they complete
- Rename refinePrompt.ts to refine-prompt.ts per CLAUDE.md file naming
2026-03-17 19:40:56 +05:30
shivammittal274
515ad44826 fix: resolve biome v2 config and lint errors (#471)
Migrate `files.ignore` to `files.includes` for Biome v2 compatibility,
fix forEach callback return value, unused variable, import ordering,
and formatting violations.
2026-03-17 19:14:01 +05:30
Dani Akash
2a6848bc1d feat: improved system prompt (#466)
* feat: added ai-sdk dev tools

* feat: new system prompt section

* feat: tests to maintain prompt integrity

* feat: update mcp sync to use react query

* fix: refetch logic for sync

* chore: remove limits on fetching integrations

* fix: refetch integrations on delete

* fix: review comment

* chore: update tests

* fix: improved memory classification

* fix: lint issues

* fix: core memory prompts

* fix: handle scenario where soul file is empty
2026-03-17 19:01:10 +05:30
Dani Akash
74f6a2dff1 fix: issue with fill tool (#469) 2026-03-17 18:58:17 +05:30
Dani Akash
58adac17db feat: new workflows (#470) 2026-03-17 18:56:55 +05:30
shivammittal274
e67c17a0f8 feat: add voice input to agent chat sidebar (#467)
* feat: add voice input to agent chat sidebar

Allow users to record voice and transcribe to text in the chat input.
Mic button shows when input is empty, waveform visualizer during recording,
transcription via OpenAI (llm.browseros.com/api/transcribe).

- Extract shared useVoiceInput hook to lib/voice/
- Time-domain waveform bars that bounce per-frequency-band
- Bar height capped to fit input container
- Analytics events for recording lifecycle

* fix: address review — add fetch timeout, await stopRecording, deduplicate VoiceInputState

- Add AbortSignal.timeout(30s) to transcription fetch
- Await stopRecording() and track analytics after completion
- Export VoiceInputState from useVoiceInput, import in consumers

* fix: await startRecording before tracking, narrow SurveyChat effect deps

- Await startRecording() so analytics only fires after mic permission granted
- Narrow SurveyChat useEffect dependency from [voice] to [voice.transcript, voice.isTranscribing]

* fix: analytics only tracks on success, clean up stream on failure, type API response

- startRecording returns boolean; track(RECORDING_STARTED) only fires on success
- Catch block cleans up MediaStream tracks and AudioContext on partial failure
- Type transcription API response with TranscribeResponse interface

* fix: keep mic button always visible alongside send button

Mic and send are now separate buttons, both always visible.
Mic is disabled while AI is streaming. Send is disabled during
recording/transcribing. Buttons are no longer absolutely positioned
inside the textarea — they sit beside it in the flex row.

* fix: keep mic button always visible inside input alongside send

Both mic and send buttons are always visible inside the input field,
positioned on the right side (ChatGPT-style). Mic is disabled while
AI is streaming. Send is disabled during recording/transcribing.

* fix: remove unreachable CSS branch in recording waveform div
2026-03-17 18:28:19 +05:30
shivammittal274
94e3f99adb feat: add test-ui skill for visual testing of agent extension via CDP (#464)
* feat: add CDP UI inspector script for dev self-testing

* fix: address code review feedback for inspect-ui script

- Use Delete key (not Backspace) to match server's keyboard.ts clearField
- Add windowId resolution to open-sidepanel (chrome.sidePanel.open requires it)
- Make target matching case-insensitive
- Replace process.exit(1) in eval with thrown error for proper cleanup
- Add comment referencing DEV_PORTS source of truth

* docs: add self-testing workflow for UI changes via CDP inspector

* fix: runtime fixes for inspect-ui discovered during live testing

- Remove Input.enable (domain has no enable method)
- Add DOM.getDocument before DOM operations (required by protocol)
- Use BrowserOS-specific sidePanel.browserosToggle API instead of
  standard chrome.sidePanel.open (side panel starts disabled)
- Enable side panel with setOptions before toggling

* feat: add test-ui skill for visual testing of agent extension UI

Adds a Claude Code skill that lets the agent visually test both
surfaces of the BrowserOS extension:
- New tab page (app.html) — left sidebar with Home, Scheduled Tasks,
  Settings, Skills, Memory, Soul, Connect Apps
- Right side panel (sidepanel.html) — chat interface

Includes all gotchas discovered through real testing: randomized ports,
fresh profile onboarding redirect, stale element IDs after navigation,
BrowserOS-specific sidePanel APIs, DOM.getDocument requirement.

* feat: add press_key, scroll, hover, select_option, wait_for to inspect-ui

Brings inspect-ui.ts to parity with server's MCP input tools:
- press_key: key combos like Enter, Control+A, Meta+Shift+P
  (ported from keyboard.ts pressCombo)
- scroll: up/down/left/right with configurable amount
- hover: hover over element by ID for tooltip/hover state testing
- select_option: select dropdown option by value or visible text
  (ported from browser.ts selectOption)
- wait_for: poll for text or CSS selector with 10s timeout

Updated skill documentation with new commands and examples.

* docs: prefer snapshot over screenshot, add holistic debugging guidance

- Add snapshot vs screenshot guidance table — prefer snapshot for
  structural checks, screenshot only for visual/layout verification
- Add server log checking instructions ([agent], [server], [build] tags)
- Add JS error checking via eval
- Add API connectivity verification
- Add common issues troubleshooting table
- Update all examples to use snapshot as default verification

* fix: address Greptile review feedback

- Replace process.exit(1) with process.exitCode + return in cmdWaitFor
  to allow async CDP cleanup in finally blocks
- Fix cmdScroll enabling Runtime instead of Page domain
- Add BROWSEROS_EXTENSION_ID env var override for extension ID
- Align CLAUDE.md dev server command with SKILL.md canonical command
2026-03-17 15:18:00 +05:30
Nikhil
e2069bc999 chore: bump server version (#459) 2026-03-16 16:42:54 -07:00
393 changed files with 27822 additions and 10713 deletions

View File

@@ -9,6 +9,9 @@ on:
jobs:
security-audit:
runs-on: ubuntu-latest
defaults:
run:
working-directory: packages/browseros-agent
steps:
- name: Checkout code

View File

@@ -1,11 +1,11 @@
name: 'CLA Assistant'
name: CLA Assistant
on:
issue_comment:
types: [created]
pull_request_target:
types: [opened, closed, synchronize]
# Explicitly configure permissions
permissions:
actions: write
contents: write
@@ -13,47 +13,46 @@ permissions:
statuses: write
jobs:
CLAAssistant:
cla:
runs-on: ubuntu-latest
if: |
(github.event_name == 'pull_request_target') ||
(github.event_name == 'issue_comment' && github.event.issue.pull_request &&
(github.event.comment.body == 'recheck' ||
github.event.comment.body == 'I have read the CLA Document and I hereby sign the CLA'))
steps:
- name: 'CLA Assistant'
if: (github.event.comment.body == 'recheck' || github.event.comment.body == 'I have read the CLA Document and I hereby sign the CLA') || github.event_name == 'pull_request_target'
- name: CLA Assistant
uses: contributor-assistant/github-action@v2.6.1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PERSONAL_ACCESS_TOKEN: ${{ secrets.CLA_SIGNATURES_TOKEN }}
with:
# Path where signatures will be stored
path-to-signatures: 'signatures/version1/cla.json'
# Path to your CLA document
path-to-document: 'https://github.com/browseros-ai/BrowserOS/blob/main/CLA.md'
# Branch to store signatures (should not be protected)
path-to-signatures: 'cla-signatures.json'
path-to-document: 'https://github.com/${{ github.repository }}/blob/main/CLA.md'
branch: 'main'
# Allowlist for users who don't need to sign (bots, core team members)
allowlist: shadowfax92,felarof99,dependabot[bot],renovate[bot],github-actions[bot]
# Optional: Custom messages
remote-organization-name: 'browseros-ai'
remote-repository-name: 'cla-signatures'
allowlist: 'shadowfax92,felarof99,bot*,*[bot],dependabot,renovate,github-actions,snyk-bot,imgbot,greenkeeper,semantic-release-bot,allcontributors'
lock-pullrequest-aftermerge: false
custom-notsigned-prcomment: |
**CLA Assistant Lite bot** Thank you for your submission! We require contributors to sign our [Contributor License Agreement](https://github.com/browseros-ai/BrowserOS/blob/main/CLA.md) before we can accept your contribution.
Thank you for your contribution! Before we can merge this PR, we need you to sign our [Contributor License Agreement](https://github.com/${{ github.repository }}/blob/main/CLA.md).
By signing the CLA, you confirm that:
- You have read and agree to the AGPL-3.0 license terms
- Your contribution is your original work
- You grant us the rights to use your contribution under the AGPL-3.0 license
**To sign the CLA**, please add a comment to this PR with the following text:
**To sign the CLA, please comment on this PR with:**
`I have read the CLA Document and I hereby sign the CLA`
```
I have read the CLA Document and I hereby sign the CLA
```
You only need to sign once. After signing, this check will pass automatically.
---
<details>
<summary>Troubleshooting</summary>
- **Already signed but still failing?** Comment `recheck` to trigger a re-verification.
- **Signed with a different email?** Make sure your commit email matches your GitHub account email, or add your commit email to your GitHub account.
</details>
custom-pr-sign-comment: 'I have read the CLA Document and I hereby sign the CLA'
custom-allsigned-prcomment: |
**CLA Assistant Lite bot** ✅ All contributors have signed the CLA. Thank you for helping make BrowserOS better!
# Lock PR after merge to prevent signature tampering
lock-pullrequest-aftermerge: true
# Custom commit messages
create-file-commit-message: 'docs: Create CLA signatures file'
signed-commit-message: 'docs: $contributorName signed the CLA in $owner/$repo#$pullRequestNo'
All contributors have signed the CLA. Thank you!

View File

@@ -22,11 +22,11 @@ jobs:
(github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude')))
runs-on: ubuntu-latest
permissions:
contents: write # Can push branches and create commits
pull-requests: write # Can create and update PRs
contents: write
pull-requests: write
issues: read
id-token: write
actions: read # Required for Claude to read CI results on PRs
actions: read
steps:
- name: Checkout repository
uses: actions/checkout@v6
@@ -38,11 +38,5 @@ jobs:
uses: anthropics/claude-code-action@v1
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
# This is an optional setting that allows Claude to read CI results on PRs
additional_permissions: |
actions: read
# Allow all tools - branch protection rules at repo level prevent direct pushes to main/master
# Omitting --allowedTools means all tools are available by default

View File

@@ -4,11 +4,16 @@ on:
pull_request:
branches:
- main
paths:
- "packages/browseros-agent/**"
jobs:
biome:
name: runner / Biome
runs-on: ubuntu-latest
defaults:
run:
working-directory: packages/browseros-agent
permissions:
contents: read
steps:
@@ -28,6 +33,9 @@ jobs:
typecheck:
name: runner / Typecheck
runs-on: ubuntu-latest
defaults:
run:
working-directory: packages/browseros-agent
permissions:
contents: read
steps:
@@ -42,6 +50,9 @@ jobs:
- name: Install dependencies
run: bun ci
- name: Prepare wxt
run: VITE_PUBLIC_BROWSEROS_API=http://localhost:3000 bun run --cwd apps/agent wxt prepare
- name: Run codegen
run: bun run --cwd apps/agent codegen

98
.github/workflows/eval-weekly.yml vendored Normal file
View File

@@ -0,0 +1,98 @@
name: Weekly Eval
on:
schedule:
# Every Saturday at 06:00 UTC
- cron: '0 6 * * 6'
push:
branches: [main]
paths:
- 'packages/browseros-agent/apps/server/src/agent/**'
- 'packages/browseros-agent/apps/server/src/tools/**'
workflow_dispatch:
inputs:
config:
description: 'Eval config file (relative to apps/eval/)'
required: false
default: 'configs/browseros-agent-weekly.json'
permissions:
contents: read
jobs:
eval:
runs-on: ubuntu-latest
timeout-minutes: 360
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Install BrowserOS
run: |
wget -q https://github.com/browseros-ai/BrowserOS/releases/download/v0.44.0.1/BrowserOS_v0.44.0.1_amd64.deb
sudo dpkg -i BrowserOS_v0.44.0.1_amd64.deb
browseros --version || echo "BrowserOS installed at $(which browseros)"
- name: Install Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install dependencies
working-directory: packages/browseros-agent
run: bun install --ignore-scripts && bun run build:agent-sdk
- name: Install xvfb
run: sudo apt-get update && sudo apt-get install -y xvfb
- name: Install captcha solver extension
working-directory: packages/browseros-agent/apps/eval
run: |
mkdir -p extensions
curl -sL -o /tmp/nopecha.zip https://github.com/NopeCHALLC/nopecha-extension/releases/latest/download/chromium_automation.zip
unzip -qo /tmp/nopecha.zip -d extensions/nopecha
- name: Run eval
working-directory: packages/browseros-agent/apps/eval
env:
FIREWORKS_API_KEY: ${{ secrets.FIREWORKS_API_KEY }}
CLAUDE_CODE_OAUTH_TOKEN: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
NOPECHA_API_KEY: ${{ secrets.NOPECHA_API_KEY }}
BROWSEROS_BINARY: /usr/bin/browseros
EVAL_CONFIG: ${{ github.event.inputs.config || 'configs/browseros-agent-weekly.json' }}
run: |
echo "Running eval with config: $EVAL_CONFIG"
xvfb-run --auto-servernum --server-args="-screen 0 1440x900x24" bun run src/index.ts -c "$EVAL_CONFIG"
- name: Upload runs to R2
if: success()
working-directory: packages/browseros-agent/apps/eval
env:
EVAL_R2_ACCOUNT_ID: ${{ secrets.EVAL_R2_ACCOUNT_ID }}
EVAL_R2_ACCESS_KEY_ID: ${{ secrets.EVAL_R2_ACCESS_KEY_ID }}
EVAL_R2_SECRET_ACCESS_KEY: ${{ secrets.EVAL_R2_SECRET_ACCESS_KEY }}
EVAL_R2_BUCKET: ${{ secrets.EVAL_R2_BUCKET }}
EVAL_R2_CDN_BASE_URL: ${{ secrets.EVAL_R2_CDN_BASE_URL }}
EVAL_CONFIG: ${{ github.event.inputs.config || 'configs/browseros-agent-weekly.json' }}
run: |
CONFIG_NAME=$(basename "$EVAL_CONFIG" .json)
bun scripts/upload-run.ts "results/$CONFIG_NAME"
- name: Generate trend report
if: success()
working-directory: packages/browseros-agent
env:
EVAL_R2_ACCOUNT_ID: ${{ secrets.EVAL_R2_ACCOUNT_ID }}
EVAL_R2_ACCESS_KEY_ID: ${{ secrets.EVAL_R2_ACCESS_KEY_ID }}
EVAL_R2_SECRET_ACCESS_KEY: ${{ secrets.EVAL_R2_SECRET_ACCESS_KEY }}
EVAL_R2_BUCKET: ${{ secrets.EVAL_R2_BUCKET }}
EVAL_R2_CDN_BASE_URL: ${{ secrets.EVAL_R2_CDN_BASE_URL }}
run: bun apps/eval/scripts/weekly-report.ts /tmp/eval-report.html
- name: Upload report as artifact
if: success()
uses: actions/upload-artifact@v4
with:
name: eval-report-${{ github.run_id }}
path: /tmp/eval-report.html

View File

@@ -2,12 +2,12 @@ name: PR Conventional Commit Validation
on:
pull_request:
types: [opened, synchronize, reopened, edited]
types: [opened, edited]
permissions:
pull-requests: write # Read PR details and add labels
issues: write # Labels are managed via issues API
contents: read # Read repository content
pull-requests: write
issues: write
contents: read
jobs:
validate-pr-title:

View File

@@ -0,0 +1,148 @@
name: Release Agent Extension
on:
workflow_dispatch:
concurrency:
group: release-agent-extension
cancel-in-progress: false
jobs:
release:
if: github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
defaults:
run:
working-directory: packages/browseros-agent/apps/agent
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0
- uses: oven-sh/setup-bun@v2
- name: Install dependencies
run: bun ci
working-directory: packages/browseros-agent
- name: Build and zip extension
run: bun run codegen && bun run zip
env:
VITE_PUBLIC_BROWSEROS_API: https://api.browseros.com
- name: Get version and zip path
id: version
run: |
echo "version=$(node -p "require('./package.json').version")" >> "$GITHUB_OUTPUT"
echo "release_sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
ZIP_FILE=$(ls "$(pwd)/dist/"*-chrome.zip | head -n 1)
echo "zip_path=$ZIP_FILE" >> "$GITHUB_OUTPUT"
echo "zip_name=$(basename "$ZIP_FILE")" >> "$GITHUB_OUTPUT"
- name: Generate release notes
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
AGENT_PATH="packages/browseros-agent/apps/agent"
CURRENT_TAG="agent-extension-v${{ steps.version.outputs.version }}"
PREV_TAG=$(git tag -l "agent-extension-v*" --sort=-v:refname | grep -v "^${CURRENT_TAG}$" | head -n 1)
if [ -z "$PREV_TAG" ]; then
echo "Initial release" > /tmp/release-notes.md
else
COMMITS=$(git log "$PREV_TAG"..HEAD --pretty=format:"%H" -- "$AGENT_PATH")
if [ -z "$COMMITS" ]; then
echo "No notable changes." > /tmp/release-notes.md
else
echo "## What's Changed" > /tmp/release-notes.md
echo "" >> /tmp/release-notes.md
while IFS= read -r SHA; do
SUBJECT=$(git log -1 --pretty=format:"%s" "$SHA")
PR_NUM=$(gh api "/repos/${{ github.repository }}/commits/${SHA}/pulls" --jq '.[0].number // empty' 2>/dev/null)
# Skip PR number if already in the commit subject (squash merges include it)
if [ -n "$PR_NUM" ] && ! echo "$SUBJECT" | grep -qF "(#${PR_NUM})"; then
echo "- ${SUBJECT} (#${PR_NUM})" >> /tmp/release-notes.md
else
echo "- ${SUBJECT}" >> /tmp/release-notes.md
fi
done <<< "$COMMITS"
fi
fi
working-directory: ${{ github.workspace }}
- name: Create GitHub release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
TAG="agent-extension-v${{ steps.version.outputs.version }}"
RELEASE_SHA="${{ steps.version.outputs.release_sha }}"
TITLE="BrowserOS Agent Extension v${{ steps.version.outputs.version }}"
if git rev-parse "$TAG" >/dev/null 2>&1; then
echo "Tag $TAG already exists, skipping tag creation"
else
git tag "$TAG" "$RELEASE_SHA"
fi
if git ls-remote --tags origin "$TAG" | grep -q "$TAG"; then
echo "Tag $TAG already on remote, skipping push"
else
git push origin "$TAG"
fi
if gh release view "$TAG" >/dev/null 2>&1; then
echo "Release $TAG already exists, updating"
gh release edit "$TAG" --title "$TITLE" --notes-file /tmp/release-notes.md
gh release upload "$TAG" "${{ steps.version.outputs.zip_path }}" --clobber
else
gh release create "$TAG" \
--title "$TITLE" \
--notes-file /tmp/release-notes.md \
"${{ steps.version.outputs.zip_path }}"
fi
working-directory: ${{ github.workspace }}
- name: Update CHANGELOG.md via PR
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
VERSION="${{ steps.version.outputs.version }}"
DATE=$(date -u +"%Y-%m-%d")
BRANCH="docs/agent-extension-changelog-v${VERSION}"
CHANGELOG="packages/browseros-agent/apps/agent/CHANGELOG.md"
git checkout main
{
head -n 1 "$CHANGELOG"
echo ""
echo "## v${VERSION} (${DATE})"
echo ""
cat /tmp/release-notes.md
echo ""
tail -n +2 "$CHANGELOG"
} > /tmp/new-changelog.md
mv /tmp/new-changelog.md "$CHANGELOG"
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git checkout -b "$BRANCH"
git add "$CHANGELOG"
git commit -m "docs: update agent extension changelog for v${VERSION}"
git push origin "$BRANCH"
gh pr create \
--title "docs: update agent extension changelog for v${VERSION}" \
--body "Auto-generated changelog update for BrowserOS Agent Extension v${VERSION}." \
--base main \
--head "$BRANCH"
gh pr merge "$BRANCH" --squash --auto || true
working-directory: ${{ github.workspace }}

168
.github/workflows/release-agent-sdk.yml vendored Normal file
View File

@@ -0,0 +1,168 @@
name: Release Agent SDK
on:
workflow_dispatch:
concurrency:
group: release-agent-sdk
cancel-in-progress: false
jobs:
publish:
if: github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
defaults:
run:
working-directory: packages/browseros-agent/packages/agent-sdk
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0
- uses: oven-sh/setup-bun@v2
- uses: actions/setup-node@v6
with:
node-version: "20"
registry-url: "https://registry.npmjs.org"
- name: Install dependencies
run: bun ci
working-directory: packages/browseros-agent
- name: Build
run: bun run build
- name: Test
run: bun test
- name: Get version
id: version
run: |
echo "version=$(node -p "require('./package.json').version")" >> "$GITHUB_OUTPUT"
echo "release_sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
- name: Generate release notes
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
SDK_PATH="packages/browseros-agent/packages/agent-sdk"
CURRENT_TAG="agent-sdk-v${{ steps.version.outputs.version }}"
# Find the previous tag, excluding the current version's tag
# (which may already exist from a prior failed run)
PREV_TAG=$(git tag -l "agent-sdk-v*" --sort=-v:refname | grep -v "^${CURRENT_TAG}$" | head -n 1)
if [ -z "$PREV_TAG" ]; then
echo "Initial release" > /tmp/release-notes.md
else
# Get commits scoped to the SDK directory
COMMITS=$(git log "$PREV_TAG"..HEAD --pretty=format:"%H" -- "$SDK_PATH")
if [ -z "$COMMITS" ]; then
echo "No notable changes." > /tmp/release-notes.md
else
echo "## What's Changed" > /tmp/release-notes.md
echo "" >> /tmp/release-notes.md
# For each commit, find the associated PR and format with author
CONTRIBUTORS=""
while IFS= read -r SHA; do
# Get commit subject and author
SUBJECT=$(git log -1 --pretty=format:"%s" "$SHA")
AUTHOR=$(git log -1 --pretty=format:"%an" "$SHA")
GITHUB_USER=$(gh api "/repos/${{ github.repository }}/commits/${SHA}" --jq '.author.login // empty' 2>/dev/null)
# Find associated PR number
PR_NUM=$(gh api "/repos/${{ github.repository }}/commits/${SHA}/pulls" --jq '.[0].number // empty' 2>/dev/null)
# Format line: skip PR number if already in the commit subject
# (squash merges include "(#123)" in the subject automatically)
if [ -n "$PR_NUM" ] && ! echo "$SUBJECT" | grep -qF "(#${PR_NUM})"; then
echo "- ${SUBJECT} (#${PR_NUM})" >> /tmp/release-notes.md
else
echo "- ${SUBJECT}" >> /tmp/release-notes.md
fi
done <<< "$COMMITS"
fi
fi
working-directory: ${{ github.workspace }}
- name: Publish
run: npm publish --access public
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
- name: Create GitHub release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
TAG="agent-sdk-v${{ steps.version.outputs.version }}"
RELEASE_SHA="${{ steps.version.outputs.release_sha }}"
TITLE="@browseros-ai/agent-sdk v${{ steps.version.outputs.version }}"
# Create or reuse tag (idempotent for re-runs)
if git rev-parse "$TAG" >/dev/null 2>&1; then
echo "Tag $TAG already exists, skipping tag creation"
else
git tag "$TAG" "$RELEASE_SHA"
fi
# Push tag (skip if already on remote)
if git ls-remote --tags origin "$TAG" | grep -q "$TAG"; then
echo "Tag $TAG already on remote, skipping push"
else
git push origin "$TAG"
fi
# Create or update release
if gh release view "$TAG" >/dev/null 2>&1; then
echo "Release $TAG already exists, updating"
gh release edit "$TAG" --title "$TITLE" --notes-file /tmp/release-notes.md
else
gh release create "$TAG" --title "$TITLE" --notes-file /tmp/release-notes.md
fi
working-directory: ${{ github.workspace }}
- name: Update CHANGELOG.md via PR
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
VERSION="${{ steps.version.outputs.version }}"
DATE=$(date -u +"%Y-%m-%d")
BRANCH="docs/agent-sdk-changelog-v${VERSION}"
CHANGELOG="packages/browseros-agent/packages/agent-sdk/CHANGELOG.md"
# Return to main before branching
git checkout main
# Use head/tail to safely insert without sed quoting issues
{
head -n 1 "$CHANGELOG"
echo ""
echo "## v${VERSION} (${DATE})"
echo ""
cat /tmp/release-notes.md
echo ""
tail -n +2 "$CHANGELOG"
} > /tmp/new-changelog.md
mv /tmp/new-changelog.md "$CHANGELOG"
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git checkout -b "$BRANCH"
git add "$CHANGELOG"
git commit -m "docs: update agent-sdk changelog for v${VERSION}"
git push origin "$BRANCH"
gh pr create \
--title "docs: update agent-sdk changelog for v${VERSION}" \
--body "Auto-generated changelog update for @browseros-ai/agent-sdk v${VERSION}." \
--base main \
--head "$BRANCH"
gh pr merge "$BRANCH" --squash --auto || true
working-directory: ${{ github.workspace }}

167
.github/workflows/release-cli.yml vendored Normal file
View File

@@ -0,0 +1,167 @@
name: Release CLI
on:
workflow_dispatch:
inputs:
version:
description: "Release version (e.g. 0.1.0)"
required: true
type: string
concurrency:
group: release-cli
cancel-in-progress: false
jobs:
release:
if: github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
environment: release-core
permissions:
contents: write
pull-requests: write
defaults:
run:
working-directory: packages/browseros-agent/apps/cli
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0
- uses: actions/setup-go@v5
with:
go-version-file: packages/browseros-agent/apps/cli/go.mod
- uses: oven-sh/setup-bun@v2
with:
bun-version: "1.3.6"
- name: Run tests
run: go test ./... -v
- name: Run vet
run: go vet ./...
- name: Build all platforms
run: |
VERSION="${{ inputs.version }}"
LDFLAGS="-s -w -X main.version=${VERSION}"
DIST="dist"
mkdir -p "$DIST"
for pair in darwin/amd64 darwin/arm64 linux/amd64 linux/arm64 windows/amd64 windows/arm64; do
OS="${pair%/*}"
ARCH="${pair#*/}"
BIN="browseros-cli"
EXT=""
if [ "$OS" = "windows" ]; then EXT=".exe"; fi
echo "Building ${OS}/${ARCH}..."
GOOS=$OS GOARCH=$ARCH CGO_ENABLED=0 go build -trimpath -ldflags "$LDFLAGS" -o "${DIST}/${BIN}${EXT}" .
ARCHIVE="browseros-cli_${VERSION}_${OS}_${ARCH}"
if [ "$OS" = "windows" ]; then
(cd "$DIST" && zip "${ARCHIVE}.zip" "${BIN}${EXT}")
else
(cd "$DIST" && tar czf "${ARCHIVE}.tar.gz" "${BIN}")
fi
rm "${DIST}/${BIN}${EXT}"
done
(cd "$DIST" && sha256sum *.tar.gz *.zip > checksums.txt)
echo "=== Built artifacts ==="
ls -lh "$DIST"
- name: Install dependencies
run: bun install
working-directory: packages/browseros-agent
- name: Upload to CDN
env:
R2_ACCOUNT_ID: ${{ secrets.R2_ACCOUNT_ID }}
R2_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
R2_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
R2_BUCKET: ${{ secrets.R2_BUCKET }}
R2_UPLOAD_PREFIX: cli
CLI_VERSION: ${{ inputs.version }}
run: |
bun scripts/build/cli.ts \
--release \
--version "$CLI_VERSION" \
--binaries-dir apps/cli/dist
working-directory: packages/browseros-agent
- name: Generate release notes
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
CLI_PATH="packages/browseros-agent/apps/cli"
TAG="browseros-cli-v${{ inputs.version }}"
CHANGELOG_FILE="/tmp/release-changelog.md"
PREV_TAG=$(git tag -l "browseros-cli-v*" --sort=-v:refname | grep -v "^${TAG}$" | head -n 1)
if [ -z "$PREV_TAG" ]; then
echo "Initial release of browseros-cli." > "$CHANGELOG_FILE"
else
COMMITS=$(git log "$PREV_TAG"..HEAD --pretty=format:"%H" -- "$CLI_PATH")
if [ -z "$COMMITS" ]; then
echo "No notable changes." > "$CHANGELOG_FILE"
else
echo "## What's Changed" > "$CHANGELOG_FILE"
echo "" >> "$CHANGELOG_FILE"
while IFS= read -r SHA; do
SUBJECT=$(git log -1 --pretty=format:"%s" "$SHA")
PR_NUM=$(gh api "/repos/${{ github.repository }}/commits/${SHA}/pulls" --jq '.[0].number // empty' 2>/dev/null)
if [ -n "$PR_NUM" ] && ! echo "$SUBJECT" | grep -qF "(#${PR_NUM})"; then
echo "- ${SUBJECT} (#${PR_NUM})" >> "$CHANGELOG_FILE"
else
echo "- ${SUBJECT}" >> "$CHANGELOG_FILE"
fi
done <<< "$COMMITS"
fi
fi
cat "$CHANGELOG_FILE" > /tmp/release-notes.md
cat >> /tmp/release-notes.md <<'EOF'
## Install `browseros-cli`
### macOS / Linux
```bash
curl -fsSL https://cdn.browseros.com/cli/install.sh | bash
```
### Windows
```powershell
irm https://cdn.browseros.com/cli/install.ps1 | iex
```
After install, run `browseros-cli init` to point the CLI at your BrowserOS MCP server.
EOF
working-directory: ${{ github.workspace }}
- name: Create tag and release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
TAG="browseros-cli-v${{ inputs.version }}"
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
if ! git rev-parse "$TAG" >/dev/null 2>&1; then
git tag -a "$TAG" -m "browseros-cli v${{ inputs.version }}"
git push origin "$TAG"
fi
CLI_DIST="packages/browseros-agent/apps/cli/dist"
gh release create "$TAG" \
--title "browseros-cli v${{ inputs.version }}" \
--notes-file /tmp/release-notes.md \
${CLI_DIST}/*
working-directory: ${{ github.workspace }}

141
.github/workflows/test.yml vendored Normal file
View File

@@ -0,0 +1,141 @@
name: Tests
on:
pull_request:
types:
- opened
- synchronize
- reopened
- ready_for_review
paths:
- .github/workflows/test.yml
- packages/browseros-agent/**
workflow_dispatch:
permissions:
contents: read
env:
BROWSEROS_APPIMAGE_URL: https://files.browseros.com/download/BrowserOS.AppImage
jobs:
test:
name: Tests / ${{ matrix.suite }}
runs-on: ubuntu-latest
timeout-minutes: 20
defaults:
run:
working-directory: packages/browseros-agent
strategy:
fail-fast: false
matrix:
include:
- suite: tools
test_path: tests/tools
junit_path: test-results/tools.xml
- suite: integration
test_path: tests/server.integration.test.ts
junit_path: test-results/integration.xml
- suite: sdk
test_path: tests/sdk
junit_path: test-results/sdk.xml
steps:
- name: Checkout code
uses: actions/checkout@v6
- name: Setup Bun
uses: oven-sh/setup-bun@v2
- name: Install dependencies
run: bun ci
- name: Resolve BrowserOS cache key
id: browseros-cache-key
run: |
set -euo pipefail
headers="$(curl -fsSI "$BROWSEROS_APPIMAGE_URL")"
etag="$(printf '%s\n' "$headers" | awk 'BEGIN{IGNORECASE=1} /^etag:/ {sub(/\r$/, "", $2); gsub(/"/, "", $2); print $2; exit}')"
last_modified="$(printf '%s\n' "$headers" | awk 'BEGIN{IGNORECASE=1} /^last-modified:/ {$1=""; sub(/^ /, ""); sub(/\r$/, ""); print; exit}')"
raw_key="${etag:-$last_modified}"
if [ -z "$raw_key" ]; then
raw_key="$BROWSEROS_APPIMAGE_URL"
fi
cache_key="$(printf '%s' "$raw_key" | shasum -a 256 | awk '{print $1}')"
echo "key=browseros-appimage-${{ runner.os }}-$cache_key" >> "$GITHUB_OUTPUT"
- name: Restore BrowserOS cache
id: browseros-cache
uses: actions/cache@v4
with:
path: packages/browseros-agent/.ci/bin/BrowserOS.AppImage
key: ${{ steps.browseros-cache-key.outputs.key }}
- name: Download BrowserOS
if: steps.browseros-cache.outputs.cache-hit != 'true'
run: |
mkdir -p .ci/bin
curl -fsSL "$BROWSEROS_APPIMAGE_URL" -o .ci/bin/BrowserOS.AppImage
chmod +x .ci/bin/BrowserOS.AppImage
- name: Prepare BrowserOS wrapper
run: |
mkdir -p .ci/bin
cat > .ci/bin/browseros <<'EOF'
#!/usr/bin/env bash
set -euo pipefail
export APPIMAGE_EXTRACT_AND_RUN=1
exec "$(dirname "$0")/BrowserOS.AppImage" "$@"
EOF
chmod +x .ci/bin/browseros
- name: Create server env file
working-directory: packages/browseros-agent/apps/server
run: cp .env.example .env.development
- name: Run ${{ matrix.suite }} tests
id: test
env:
BROWSEROS_BINARY: ${{ github.workspace }}/packages/browseros-agent/.ci/bin/browseros
BROWSEROS_TEST_HEADLESS: "true"
BROWSEROS_TEST_EXTRA_ARGS: --no-sandbox --disable-dev-shm-usage
run: |
set +e
mkdir -p test-results
cd apps/server
bun run test:cleanup
bun --env-file=.env.development test "${{ matrix.test_path }}" --reporter=junit --reporter-outfile="../../${{ matrix.junit_path }}"
exit_code=$?
cd ../..
if [ ! -f "${{ matrix.junit_path }}" ]; then
cat > "${{ matrix.junit_path }}" <<EOF
<?xml version="1.0" encoding="UTF-8"?>
<testsuites tests="1" failures="1">
<testsuite name="${{ matrix.suite }}" tests="1" failures="1">
<testcase classname="workflow" name="${{ matrix.suite }} setup">
<failure message="Test run failed before JUnit output was written">See workflow logs for details.</failure>
</testcase>
</testsuite>
</testsuites>
EOF
fi
echo "exit_code=$exit_code" >> "$GITHUB_OUTPUT"
- name: Upload JUnit XML
if: always()
uses: actions/upload-artifact@v4
with:
name: junit-${{ matrix.suite }}
path: packages/browseros-agent/${{ matrix.junit_path }}
- name: Summarize suite result
if: always()
run: |
if [ "${{ steps.test.outputs.exit_code }}" = "0" ]; then
echo "### :white_check_mark: ${{ matrix.suite }} suite passed" >> "$GITHUB_STEP_SUMMARY"
else
echo "### :x: ${{ matrix.suite }} suite failed (exit code ${{ steps.test.outputs.exit_code }})" >> "$GITHUB_STEP_SUMMARY"
echo "" >> "$GITHUB_STEP_SUMMARY"
echo "See the uploaded \`junit-${{ matrix.suite }}\` artifact for details." >> "$GITHUB_STEP_SUMMARY"
exit 1
fi

3
.gitignore vendored
View File

@@ -26,3 +26,6 @@ gclient.json
**/resources/binaries/
packages/browseros/build/tools/
# AI SDK DevTools traces
.devtools/

File diff suppressed because it is too large Load Diff

4
.vscode/settings.json vendored Normal file
View File

@@ -0,0 +1,4 @@
{
"terminal.integrated.tabs.title": "${sequence} ${process}",
"terminal.integrated.tabs.description": "${cwd}"
}

214
README.md
View File

@@ -6,6 +6,7 @@
[![Slack](https://img.shields.io/badge/Slack-Join%20us-4A154B?logo=slack&logoColor=white)](https://dub.sh/browserOS-slack)
[![Twitter](https://img.shields.io/twitter/follow/browserOS_ai?style=social)](https://twitter.com/browseros_ai)
[![License: AGPL v3](https://img.shields.io/badge/License-AGPL%20v3-blue.svg)](LICENSE)
[![Docs](https://img.shields.io/badge/Docs-docs.browseros.com-blue)](https://docs.browseros.com)
<br></br>
<a href="https://files.browseros.com/download/BrowserOS.dmg">
<img src="https://img.shields.io/badge/Download-macOS-black?style=flat&logo=apple&logoColor=white" alt="Download for macOS (beta)" />
@@ -22,146 +23,183 @@
<br />
</div>
##
🌐 BrowserOS is an open-source Chromium fork that runs AI agents natively. **The privacy-first alternative to ChatGPT Atlas, Perplexity Comet, and Dia.**
BrowserOS is an open-source Chromium fork that runs AI agents natively. **The privacy-first alternative to ChatGPT Atlas, Perplexity Comet, and Dia.**
🔒 Use your own API keys or run local models with Ollama. Your data never leaves your machine.
Use your own API keys or run local models with Ollama. Your data never leaves your machine.
💡 Join our [Discord](https://discord.gg/YKwjt5vuKr) or [Slack](https://dub.sh/browserOS-slack) and help us build! Have feature requests? [Suggest here](https://github.com/browseros-ai/BrowserOS/issues/99).
> **[Documentation](https://docs.browseros.com)** · **[Discord](https://discord.gg/YKwjt5vuKr)** · **[Slack](https://dub.sh/browserOS-slack)** · **[Twitter](https://x.com/browserOS_ai)** · **[Feature Requests](https://github.com/browseros-ai/BrowserOS/issues/99)**
## Quick start
## Quick Start
1. Download and install BrowserOS:
- [macOS](https://files.browseros.com/download/BrowserOS.dmg)
- [Windows](https://files.browseros.com/download/BrowserOS_installer.exe)
- [Linux (AppImage)](https://files.browseros.com/download/BrowserOS.AppImage)
- [Linux (Debian)](https://cdn.browseros.com/download/BrowserOS.deb)
1. **Download and install** BrowserOS — [macOS](https://files.browseros.com/download/BrowserOS.dmg) · [Windows](https://files.browseros.com/download/BrowserOS_installer.exe) · [Linux (AppImage)](https://files.browseros.com/download/BrowserOS.AppImage) · [Linux (Debian)](https://cdn.browseros.com/download/BrowserOS.deb)
2. **Import your Chrome data** (optional) — bookmarks, passwords, extensions all carry over
3. **Connect your AI provider** — Claude, OpenAI, Gemini, ChatGPT Pro via OAuth, or local models via Ollama/LM Studio
2. Import your Chrome data (optional)
## Features
3. Connect your AI provider — use Claude, OpenAI, Gemini, or local models via Ollama and LMStudio.
4. Start automating!
## What makes BrowserOS special
- 🏠 Feels like home — same Chrome interface, all your extensions just work
- 🤖 AI agents that run on YOUR browser, not in the cloud
- 🔒 Privacy first — bring your own keys or run local models with Ollama. Your browsing history stays on your machine
- 🤝 [BrowserOS as MCP server](https://docs.browseros.com/features/use-with-claude-code) — control the browser from `claude-code`, `gemini-cli`, or any MCP client (31 tools)
- 🔄 [Workflows](https://docs.browseros.com/features/workflows) — build repeatable browser automations with a visual graph builder
- 📂 [Cowork](https://docs.browseros.com/features/cowork) — combine browser automation with local file operations. Research the web, save reports to your folder
- ⏰ [Scheduled Tasks](https://docs.browseros.com/features/scheduled-tasks) — run the agent on autopilot, daily or every few minutes
- 💬 [LLM Hub](https://docs.browseros.com/features/llm-chat-hub) — compare Claude, ChatGPT, and Gemini side-by-side on any page
- 🛡️ Built-in ad blocker — [10x more protection than Chrome](https://docs.browseros.com/features/ad-blocking) with uBlock Origin + Manifest V2 support
- 🚀 100% open source under AGPL-3.0
| Feature | Description | Docs |
|---------|-------------|------|
| **AI Agent** | 53+ browser automation tools — navigate, click, type, extract data, all with natural language | [Guide](https://docs.browseros.com/getting-started) |
| **MCP Server** | Control the browser from Claude Code, Gemini CLI, or any MCP client | [Setup](https://docs.browseros.com/features/use-with-claude-code) |
| **Workflows** | Build repeatable browser automations with a visual graph builder | [Docs](https://docs.browseros.com/features/workflows) |
| **Cowork** | Combine browser automation with local file operations — research the web, save reports to your folder | [Docs](https://docs.browseros.com/features/cowork) |
| **Scheduled Tasks** | Run agents on autopilot — daily, hourly, or every few minutes | [Docs](https://docs.browseros.com/features/scheduled-tasks) |
| **Memory** | Persistent memory across conversations — your assistant remembers context over time | [Docs](https://docs.browseros.com/features/memory) |
| **SOUL.md** | Define your AI's personality and instructions in a single markdown file | [Docs](https://docs.browseros.com/features/soul-md) |
| **LLM Hub** | Compare Claude, ChatGPT, and Gemini responses side-by-side on any page | [Docs](https://docs.browseros.com/features/llm-chat-hub) |
| **40+ App Integrations** | Gmail, Slack, GitHub, Linear, Notion, Figma, Salesforce, and more via MCP | [Docs](https://docs.browseros.com/features/connect-apps) |
| **Vertical Tabs** | Side-panel tab management — stay organized even with 100+ tabs open | [Docs](https://docs.browseros.com/features/vertical-tabs) |
| **Ad Blocking** | uBlock Origin + Manifest V2 support — [10x more protection](https://docs.browseros.com/features/ad-blocking) than Chrome | [Docs](https://docs.browseros.com/features/ad-blocking) |
| **Cloud Sync** | Sync browser config and agent history across devices | [Docs](https://docs.browseros.com/features/sync) |
| **Skills** | Custom instruction sets that shape how your AI assistant behaves | [Docs](https://docs.browseros.com/features/skills) |
| **Smart Nudges** | Contextual suggestions to connect apps and use features at the right moment | [Docs](https://docs.browseros.com/features/smart-nudges) |
## Demos
### 🤖 BrowserOS agent in action
### BrowserOS agent in action
[![BrowserOS agent in action](docs/videos/browserOS-agent-in-action.gif)](https://www.youtube.com/watch?v=SoSFev5R5dI)
<br/><br/>
### 🎇 Install [BrowserOS as MCP](https://docs.browseros.com/features/use-with-claude-code) and control it from `claude-code`
### Install [BrowserOS as MCP](https://docs.browseros.com/features/use-with-claude-code) and control it from `claude-code`
https://github.com/user-attachments/assets/c725d6df-1a0d-40eb-a125-ea009bf664dc
<br/><br/>
### 💬 Use BrowserOS to chat
### Use BrowserOS to chat
https://github.com/user-attachments/assets/726803c5-8e36-420e-8694-c63a2607beca
<br/><br/>
### Use BrowserOS to scrape data
### Use BrowserOS to scrape data
https://github.com/user-attachments/assets/9f038216-bc24-4555-abf1-af2adcb7ebc0
<br/><br/>
## Why We're Building BrowserOS
## Install `browseros-cli`
For the first time since Netscape pioneered the web in 1994, AI gives us the chance to completely reimagine the browser. We've seen tools like Cursor deliver 10x productivity gains for developers—yet everyday browsing remains frustratingly archaic.
Use `browseros-cli` to launch and control BrowserOS from the terminal or from AI coding agents like Claude Code.
You're likely juggling 70+ tabs, battling your browser instead of having it assist you. Routine tasks, like ordering something from amazon or filling a form should be handled seamlessly by AI agents.
**macOS / Linux:**
At BrowserOS, we're convinced that AI should empower you by automating tasks locally and securely—keeping your data private. We are building the best browser for this future!
```bash
curl -fsSL https://cdn.browseros.com/cli/install.sh | bash
```
## How we compare
**Windows:**
<details>
<summary><b>vs Chrome</b></summary>
<br>
While we're grateful for Google open-sourcing Chromium, but Chrome hasn't evolved much in 10 years. No AI features, no automation, no MCP support.
</details>
```powershell
irm https://cdn.browseros.com/cli/install.ps1 | iex
```
<details>
<summary><b>vs Brave</b></summary>
<br>
We love what Brave started, but they've spread themselves too thin with crypto, search, VPNs. We're laser-focused on AI-powered browsing.
</details>
After install, run `browseros-cli init` to connect the CLI to your running BrowserOS instance.
<details>
<summary><b>vs Arc/Dia</b></summary>
<br>
Many loved Arc, but it was closed source. When they abandoned users, there was no recourse. We're 100% open source - fork it anytime!
</details>
## LLM Providers
<details>
<summary><b>vs Perplexity Comet</b></summary>
<br>
They're a search/ad company. Your browser history becomes their product. We keep everything local.
</details>
BrowserOS works with any LLM. Bring your own keys, use OAuth, or run models locally.
<details>
<summary><b>vs ChatGPT Atlas</b></summary>
<br>
Your browsing data could be used for ads or to train their models. We keep your history and agent interactions strictly local.
</details>
| Provider | Type | Auth |
|----------|------|------|
| Kimi K2.5 | Cloud (default) | Built-in |
| ChatGPT Pro/Plus | Cloud | [OAuth](https://docs.browseros.com/features/chatgpt) |
| GitHub Copilot | Cloud | [OAuth](https://docs.browseros.com/features/github-copilot) |
| Qwen Code | Cloud | [OAuth](https://docs.browseros.com/features/qwen-code) |
| Claude (Anthropic) | Cloud | API key |
| GPT-4o / o3 (OpenAI) | Cloud | API key |
| Gemini (Google) | Cloud | API key |
| Azure OpenAI | Cloud | API key |
| AWS Bedrock | Cloud | IAM credentials |
| OpenRouter | Cloud | API key |
| Ollama | Local | [Setup](https://docs.browseros.com/features/ollama) |
| LM Studio | Local | [Setup](https://docs.browseros.com/features/lm-studio) |
## How We Compare
| | BrowserOS | Chrome | Brave | Dia | Comet | Atlas |
|---|:---:|:---:|:---:|:---:|:---:|:---:|
| Open Source | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ |
| AI Agent | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ |
| MCP Server | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ |
| Visual Workflows | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ |
| Cowork (files + browser) | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ |
| Scheduled Tasks | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ |
| Bring Your Own Keys | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ |
| Local Models (Ollama) | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ |
| Local-first Privacy | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ |
| Ad Blocking (MV2) | ✅ | ❌ | ✅ | ❌ | ✅ | ❌ |
**Detailed comparisons:**
- [BrowserOS vs Chrome DevTools MCP](https://docs.browseros.com/comparisons/chrome-devtools-mcp) — developer-focused comparison for browser automation
- [BrowserOS vs Claude Cowork](https://docs.browseros.com/comparisons/claude-cowork) — getting real work done with AI
- [BrowserOS vs OpenClaw](https://docs.browseros.com/comparisons/openclaw) — everyday AI assistance
## Architecture
BrowserOS is a monorepo with two main subsystems: the **browser** (Chromium fork) and the **agent platform** (TypeScript/Go).
```
BrowserOS/
├── packages/browseros/ # Chromium fork + build system (Python)
│ ├── chromium_patches/ # Patches applied to Chromium source
│ ├── build/ # Build CLI and modules
│ └── resources/ # Icons, entitlements, signing
├── packages/browseros-agent/ # Agent platform (TypeScript/Go)
│ ├── apps/
│ │ ├── server/ # MCP server + AI agent loop (Bun)
│ │ ├── agent/ # Browser extension UI (WXT + React)
│ │ ├── cli/ # CLI tool (Go)
│ │ ├── eval/ # Benchmark framework
│ │ └── controller-ext/ # Chrome API bridge extension
│ │
│ └── packages/
│ ├── agent-sdk/ # Node.js SDK (npm: @browseros-ai/agent-sdk)
│ ├── cdp-protocol/ # CDP type bindings
│ └── shared/ # Shared constants
```
| Package | What it does |
|---------|-------------|
| [`packages/browseros`](packages/browseros/) | Chromium fork — patches, build system, signing |
| [`apps/server`](packages/browseros-agent/apps/server/) | Bun server exposing 53+ MCP tools and running the AI agent loop |
| [`apps/agent`](packages/browseros-agent/apps/agent/) | Browser extension — new tab, side panel chat, onboarding, settings |
| [`apps/cli`](packages/browseros-agent/apps/cli/) | Go CLI — control BrowserOS from the terminal or AI coding agents |
| [`apps/eval`](packages/browseros-agent/apps/eval/) | Benchmark framework — WebVoyager, Mind2Web evaluation |
| [`agent-sdk`](packages/browseros-agent/packages/agent-sdk/) | Node.js SDK for browser automation with natural language |
| [`cdp-protocol`](packages/browseros-agent/packages/cdp-protocol/) | Type-safe Chrome DevTools Protocol bindings |
## Contributing
We'd love your help making BrowserOS better!
We'd love your help making BrowserOS better! See our [Contributing Guide](CONTRIBUTING.md) for details.
- 🐛 [Report bugs](https://github.com/browseros-ai/BrowserOS/issues)
- 💡 [Suggest features](https://github.com/browseros-ai/BrowserOS/issues/99)
- 💬 [Join Discord](https://discord.gg/YKwjt5vuKr)
- 🐦 [Follow on Twitter](https://x.com/browserOS_ai)
- [Report bugs](https://github.com/browseros-ai/BrowserOS/issues)
- [Suggest features](https://github.com/browseros-ai/BrowserOS/issues/99)
- [Join Discord](https://discord.gg/YKwjt5vuKr) · [Join Slack](https://dub.sh/browserOS-slack)
- [Follow on Twitter](https://x.com/browserOS_ai)
**Agent development** (TypeScript/Go) — see the [agent monorepo README](packages/browseros-agent/README.md) for setup instructions.
**Browser development** (C++/Python) — requires ~100GB disk space. See [`packages/browseros`](packages/browseros/) for build instructions.
## Credits
- [ungoogled-chromium](https://github.com/ungoogled-software/ungoogled-chromium) — BrowserOS uses some patches for enhanced privacy. Thanks to everyone behind this project!
- [The Chromium Project](https://www.chromium.org/) — at the core of BrowserOS, making it possible to exist in the first place.
## License
BrowserOS is open source under the [AGPL-3.0 license](LICENSE).
## Credits
- [ungoogled-chromium](https://github.com/ungoogled-software/ungoogled-chromium) - BrowserOS uses some patches for enhanced privacy. Thanks to everyone behind this project!
- [The Chromium Project](https://www.chromium.org/) - At the core of BrowserOS, making it possible to exist in the first place.
## Citation
If you use BrowserOS in your research or project, please cite:
```bibtex
@software{browseros2025,
author = {Sonti, Nithin and Sonti, Nikhil and {BrowserOS-team}},
title = {BrowserOS: The open-source Agentic browser},
url = {https://github.com/browseros-ai/BrowserOS},
year = {2025},
publisher = {GitHub},
license = {AGPL-3.0},
}
```
Copyright &copy; 2025 Felafax, Inc.
## Stargazers
Thank you to all our supporters!
[![Star History Chart](https://api.star-history.com/svg?repos=browseros-ai/BrowserOS&type=Date)](https://www.star-history.com/#browseros-ai/BrowserOS&Date)
##
<p align="center">
Built with ❤️ from San Francisco
</p>

View File

@@ -23,6 +23,9 @@
"group": "Core Features",
"pages": [
"features/bring-your-own-llm",
"features/chatgpt-pro-oauth",
"features/github-copilot-oauth",
"features/qwen-code-oauth",
"features/local-models",
"features/workflows",
"features/scheduled-tasks",

View File

@@ -13,6 +13,33 @@ See how to connect your own LLM in under a minute:
src="https://pub-80f8a01e6e8b4239ae53a7652ef85877.r2.dev/resources/feature-videos/1-bring-your-own-LLM.mov"
></video>
## Use Your Existing Subscription
Already paying for ChatGPT Pro, GitHub Copilot, or Qwen Code? Connect your existing account to BrowserOS with a single sign-in — no API keys, no extra cost.
<CardGroup cols={3}>
<Card href="/features/chatgpt-pro-oauth">
<svg fill="currentColor" fillRule="evenodd" height="24" width="24" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><path d="M9.205 8.658v-2.26c0-.19.072-.333.238-.428l4.543-2.616c.619-.357 1.356-.523 2.117-.523 2.854 0 4.662 2.212 4.662 4.566 0 .167 0 .357-.024.547l-4.71-2.759a.797.797 0 00-.856 0l-5.97 3.473zm10.609 8.8V12.06c0-.333-.143-.57-.429-.737l-5.97-3.473 1.95-1.118a.433.433 0 01.476 0l4.543 2.617c1.309.76 2.189 2.378 2.189 3.948 0 1.808-1.07 3.473-2.76 4.163zM7.802 12.703l-1.95-1.142c-.167-.095-.239-.238-.239-.428V5.899c0-2.545 1.95-4.472 4.591-4.472 1 0 1.927.333 2.712.928L8.23 5.067c-.285.166-.428.404-.428.737v6.898zM12 15.128l-2.795-1.57v-3.33L12 8.658l2.795 1.57v3.33L12 15.128zm1.796 7.23c-1 0-1.927-.332-2.712-.927l4.686-2.712c.285-.166.428-.404.428-.737v-6.898l1.974 1.142c.167.095.238.238.238.428v5.233c0 2.545-1.974 4.472-4.614 4.472zm-5.637-5.303l-4.544-2.617c-1.308-.761-2.188-2.378-2.188-3.948A4.482 4.482 0 014.21 6.327v5.423c0 .333.143.571.428.738l5.947 3.449-1.95 1.118a.432.432 0 01-.476 0zm-.262 3.9c-2.688 0-4.662-2.021-4.662-4.519 0-.19.024-.38.047-.57l4.686 2.71c.286.167.571.167.856 0l5.97-3.448v2.26c0 .19-.07.333-.237.428l-4.543 2.616c-.619.357-1.356.523-2.117.523zm5.899 2.83a5.947 5.947 0 005.827-4.756C22.287 18.339 24 15.84 24 13.296c0-1.665-.713-3.282-1.998-4.448.119-.5.19-.999.19-1.498 0-3.401-2.759-5.947-5.946-5.947-.642 0-1.26.095-1.88.31A5.962 5.962 0 0010.205 0a5.947 5.947 0 00-5.827 4.757C1.713 5.447 0 7.945 0 10.49c0 1.666.713 3.283 1.998 4.448-.119.5-.19 1-.19 1.499 0 3.401 2.759 5.946 5.946 5.946.642 0 1.26-.095 1.88-.309a5.96 5.96 0 004.162 1.713z"></path></svg>
**ChatGPT Pro / Plus**
Sign in with your OpenAI account. Access GPT-5 Codex, GPT-5.4, and the full Codex lineup with up to 400K context.
</Card>
<Card href="/features/github-copilot-oauth">
<svg fill="currentColor" fillRule="evenodd" height="24" width="24" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><path d="M19.245 5.364c1.322 1.36 1.877 3.216 2.11 5.817.622 0 1.2.135 1.592.654l.73.964c.21.278.323.61.323.955v2.62c0 .339-.173.669-.453.868C20.239 19.602 16.157 21.5 12 21.5c-4.6 0-9.205-2.583-11.547-4.258-.28-.2-.452-.53-.453-.868v-2.62c0-.345.113-.679.321-.956l.73-.963c.392-.517.974-.654 1.593-.654l.029-.297c.25-2.446.81-4.213 2.082-5.52 2.461-2.54 5.71-2.851 7.146-2.864h.198c1.436.013 4.685.323 7.146 2.864zm-7.244 4.328c-.284 0-.613.016-.962.05-.123.447-.305.85-.57 1.108-1.05 1.023-2.316 1.18-2.994 1.18-.638 0-1.306-.13-1.851-.464-.516.165-1.012.403-1.044.996a65.882 65.882 0 00-.063 2.884l-.002.48c-.002.563-.005 1.126-.013 1.69.002.326.204.63.51.765 2.482 1.102 4.83 1.657 6.99 1.657 2.156 0 4.504-.555 6.985-1.657a.854.854 0 00.51-.766c.03-1.682.006-3.372-.076-5.053-.031-.596-.528-.83-1.046-.996-.546.333-1.212.464-1.85.464-.677 0-1.942-.157-2.993-1.18-.266-.258-.447-.661-.57-1.108-.32-.032-.64-.049-.96-.05zm-2.525 4.013c.539 0 .976.426.976.95v1.753c0 .525-.437.95-.976.95a.964.964 0 01-.976-.95v-1.752c0-.525.437-.951.976-.951zm5 0c.539 0 .976.426.976.95v1.753c0 .525-.437.95-.976.95a.964.964 0 01-.976-.95v-1.752c0-.525.437-.951.976-.951zM7.635 5.087c-1.05.102-1.935.438-2.385.906-.975 1.037-.765 3.668-.21 4.224.405.394 1.17.657 1.995.657h.09c.649-.013 1.785-.176 2.73-1.11.435-.41.705-1.433.675-2.47-.03-.834-.27-1.52-.63-1.813-.39-.336-1.275-.482-2.265-.394zm6.465.394c-.36.292-.6.98-.63 1.813-.03 1.037.24 2.06.675 2.47.968.957 2.136 1.104 2.776 1.11h.044c.825 0 1.59-.263 1.995-.657.555-.556.765-3.187-.21-4.224-.45-.468-1.335-.804-2.385-.906-.99-.088-1.875.058-2.265.394zM12 7.615c-.24 0-.525.015-.84.044.03.16.045.336.06.526l-.001.159a2.94 2.94 0 01-.014.25c.225-.022.425-.027.612-.028h.366c.187 0 .387.006.612.028-.015-.146-.015-.277-.015-.409.015-.19.03-.365.06-.526a9.29 9.29 0 00-.84-.044z"></path></svg>
**GitHub Copilot**
Sign in with your GitHub account. Access 19+ models including Claude, GPT-5, and Gemini through one subscription.
</Card>
<Card href="/features/qwen-code-oauth">
<svg fill="currentColor" fillRule="evenodd" height="24" width="24" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><path d="M12.604 1.34c.393.69.784 1.382 1.174 2.075a.18.18 0 00.157.091h5.552c.174 0 .322.11.446.327l1.454 2.57c.19.337.24.478.024.837-.26.43-.513.864-.76 1.3l-.367.658c-.106.196-.223.28-.04.512l2.652 4.637c.172.301.111.494-.043.77-.437.785-.882 1.564-1.335 2.34-.159.272-.352.375-.68.37-.777-.016-1.552-.01-2.327.016a.099.099 0 00-.081.05 575.097 575.097 0 01-2.705 4.74c-.169.293-.38.363-.725.364-.997.003-2.002.004-3.017.002a.537.537 0 01-.465-.271l-1.335-2.323a.09.09 0 00-.083-.049H4.982c-.285.03-.553-.001-.805-.092l-1.603-2.77a.543.543 0 01-.002-.54l1.207-2.12a.198.198 0 000-.197 550.951 550.951 0 01-1.875-3.272l-.79-1.395c-.16-.31-.173-.496.095-.965.465-.813.927-1.625 1.387-2.436.132-.234.304-.334.584-.335a338.3 338.3 0 012.589-.001.124.124 0 00.107-.063l2.806-4.895a.488.488 0 01.422-.246c.524-.001 1.053 0 1.583-.006L11.704 1c.341-.003.724.032.9.34zm-3.432.403a.06.06 0 00-.052.03L6.254 6.788a.157.157 0 01-.135.078H3.253c-.056 0-.07.025-.041.074l5.81 10.156c.025.042.013.062-.034.063l-2.795.015a.218.218 0 00-.2.116l-1.32 2.31c-.044.078-.021.118.068.118l5.716.008c.046 0 .08.02.104.061l1.403 2.454c.046.081.092.082.139 0l5.006-8.76.783-1.382a.055.055 0 01.096 0l1.424 2.53a.122.122 0 00.107.062l2.763-.02a.04.04 0 00.035-.02.041.041 0 000-.04l-2.9-5.086a.108.108 0 010-.113l.293-.507 1.12-1.977c.024-.041.012-.062-.035-.062H9.2c-.059 0-.073-.026-.043-.077l1.434-2.505a.107.107 0 000-.114L9.225 1.774a.06.06 0 00-.053-.031zm6.29 8.02c.046 0 .058.02.034.06l-.832 1.465-2.613 4.585a.056.056 0 01-.05.029.058.058 0 01-.05-.029L8.498 9.841c-.02-.034-.01-.052.028-.054l.216-.012 6.722-.012z"></path></svg>
**Qwen Code**
Sign in with your Qwen account. Access Qwen 3 Coder with a 1 million token context window.
</Card>
</CardGroup>
---
## Which Model Should I Use?
| Mode | What works | Recommendation |

View File

@@ -0,0 +1,56 @@
---
title: "ChatGPT Pro / Plus"
description: "Use your ChatGPT subscription to power BrowserOS"
---
Connect your ChatGPT Pro or Plus subscription to BrowserOS and access GPT-5 Codex, GPT-5.4, and the full lineup of OpenAI's most advanced models — with up to 400K context. No API keys needed.
## Setup
**1.** Open BrowserOS and go to **Settings** (`chrome://browseros/settings`). You'll see the AI Providers section.
![AI Settings screen](/images/setting-up-chatgpt/llm-screen.png)
**2.** Click **USE** on the **ChatGPT Plus/Pro** card. You'll be prompted to sign in with your OpenAI account.
![Login screen](/images/setting-up-chatgpt/login-screen.png)
**3.** Sign in with the OpenAI account that has your ChatGPT Pro or Plus subscription active, and accept the authorization.
![Accept authorization](/images/setting-up-chatgpt/accept-screen.png)
**4.** Once authorized, ChatGPT will appear as a provider in your settings. Select a model and start using it.
## Available Models
| Model | Context Window |
|-------|---------------|
| `gpt-5.4` | 400K |
| `gpt-5.3-codex` | 400K |
| `gpt-5.2-codex` | 400K |
| `gpt-5.2` | 200K |
| `gpt-5.1-codex` | 400K |
| `gpt-5.1-codex-max` | 400K |
| `gpt-5.1-codex-mini` | 400K |
| `gpt-5.1` | 200K |
<Info>
ChatGPT Pro subscribers have access to the full model lineup. ChatGPT Plus subscribers can access a subset of models depending on their plan. The available models will be shown automatically after you connect.
</Info>
<Tip>
The Codex models (e.g., `gpt-5.3-codex`) are optimized for code and reasoning tasks — ideal for complex browser automation workflows that involve form filling, data extraction, and multi-step navigation.
</Tip>
## Reasoning Settings
ChatGPT Pro includes additional settings for models that support reasoning:
- **Reasoning Effort** — Control how much the model "thinks" before responding. Options: none, low, medium, high.
- **Reasoning Summary** — Choose how reasoning is displayed. Options: auto, concise, detailed.
These settings are available in the provider configuration after connecting.
## Disconnecting
To disconnect your OpenAI account, go to **Settings**, find the ChatGPT Plus/Pro provider, and click **Disconnect**. Your OAuth tokens will be immediately deleted from your machine.

View File

@@ -0,0 +1,60 @@
---
title: "GitHub Copilot"
description: "Use your GitHub Copilot subscription to power BrowserOS"
---
Connect your GitHub Copilot subscription to BrowserOS and access 19+ models — including Claude, GPT-5, and Gemini — through a single GitHub sign-in. No API keys needed.
<Info>
**Free tier** includes GPT-5 Mini, Claude Haiku 4.5, GPT-4o, and GPT-4.1. **Copilot Pro** ($10/month) unlocks Claude Sonnet 4.6, Claude Opus 4.6, Gemini 3 Pro, GPT-5.4, and more.
</Info>
## Setup
**1.** Open BrowserOS and go to **Settings** (`chrome://browseros/settings`). You'll see the AI Providers section.
![AI Settings screen](/images/setting-up-copilot/llm-screen.png)
**2.** Click **USE** on the **GitHub Copilot** card. A device code will appear — copy it, then click the link to open GitHub's device authorization page.
![Device code displayed](/images/setting-up-copilot/device-code.png)
**3.** Select your GitHub account to authorize.
![Select GitHub account](/images/setting-up-copilot/select-account.png)
**4.** Paste the device code and authorize BrowserOS to access your Copilot subscription.
![Authorize device](/images/setting-up-copilot/authorize-device.png)
**5.** Once authorized, GitHub Copilot will appear as a provider in your settings. Select a model and start using it.
## Available Models
### Free Tier
| Model | Context Window |
|-------|---------------|
| `gpt-5-mini` | 128K |
| `claude-haiku-4.5` | 128K |
| `gpt-4o` | 64K |
| `gpt-4.1` | 64K |
### Copilot Pro / Pro+
| Model | Context Window |
|-------|---------------|
| `claude-sonnet-4.6` | 200K |
| `claude-opus-4.6` | 200K |
| `gemini-2.5-pro` | 1M |
| `gemini-3-pro-preview` | 1M |
| `gpt-5.4` | 400K |
| `gpt-5.3-codex` | 400K |
| `gpt-5.2-codex` | 400K |
| `grok-code-fast-1` | 128K |
<Tip>
GitHub Copilot is the most versatile provider — one subscription gives you access to models from OpenAI, Anthropic, Google, and xAI. Great if you want to switch between models for different tasks.
</Tip>
## Disconnecting
To disconnect your GitHub account, go to **Settings**, find the GitHub Copilot provider, and click **Disconnect**. Your OAuth tokens will be immediately deleted from your machine.

View File

@@ -0,0 +1,39 @@
---
title: "Qwen Code"
description: "Use your Qwen Code account to power BrowserOS"
---
Connect your Qwen Code account to BrowserOS and access Alibaba's coding models with up to a **1 million token context window** — the largest of any provider we support. No API keys needed.
## Setup
**1.** Open BrowserOS and go to **Settings** (`chrome://browseros/settings`). You'll see the AI Providers section.
![AI Settings screen](/images/setting-up-qwen/llm-screen.png)
**2.** Click **USE** on the **Qwen Code** card. You'll be prompted to sign in with your Qwen account.
![Select Qwen Code](/images/setting-up-qwen/select-qwen.png)
**3.** Sign in with your Alibaba Cloud / Qwen account to authorize BrowserOS.
![Qwen sign in](/images/setting-up-qwen/qwen-signin.png)
**4.** Once authorized, Qwen Code will appear as a provider in your settings. Select a model and start using it.
## Available Models
| Model | Context Window |
|-------|---------------|
| `coder-model` | 1M |
| `qwen3-coder-plus` | 1M |
| `qwen3-coder-flash` | 1M |
| `qwen3.5-plus` | 1M |
<Tip>
Qwen Code's 1 million token context window is ideal for tasks that involve long documents, entire documentation sites, or working across many browser tabs simultaneously — the agent can hold everything in context at once.
</Tip>
## Disconnecting
To disconnect your Qwen account, go to **Settings**, find the Qwen Code provider, and click **Disconnect**. Your OAuth tokens will be immediately deleted from your machine.

View File

@@ -0,0 +1 @@
<svg fill="currentColor" fill-rule="evenodd" height="1em" style="flex:none;line-height:1" viewBox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><title>GithubCopilot</title><path d="M19.245 5.364c1.322 1.36 1.877 3.216 2.11 5.817.622 0 1.2.135 1.592.654l.73.964c.21.278.323.61.323.955v2.62c0 .339-.173.669-.453.868C20.239 19.602 16.157 21.5 12 21.5c-4.6 0-9.205-2.583-11.547-4.258-.28-.2-.452-.53-.453-.868v-2.62c0-.345.113-.679.321-.956l.73-.963c.392-.517.974-.654 1.593-.654l.029-.297c.25-2.446.81-4.213 2.082-5.52 2.461-2.54 5.71-2.851 7.146-2.864h.198c1.436.013 4.685.323 7.146 2.864zm-7.244 4.328c-.284 0-.613.016-.962.05-.123.447-.305.85-.57 1.108-1.05 1.023-2.316 1.18-2.994 1.18-.638 0-1.306-.13-1.851-.464-.516.165-1.012.403-1.044.996a65.882 65.882 0 00-.063 2.884l-.002.48c-.002.563-.005 1.126-.013 1.69.002.326.204.63.51.765 2.482 1.102 4.83 1.657 6.99 1.657 2.156 0 4.504-.555 6.985-1.657a.854.854 0 00.51-.766c.03-1.682.006-3.372-.076-5.053-.031-.596-.528-.83-1.046-.996-.546.333-1.212.464-1.85.464-.677 0-1.942-.157-2.993-1.18-.266-.258-.447-.661-.57-1.108-.32-.032-.64-.049-.96-.05zm-2.525 4.013c.539 0 .976.426.976.95v1.753c0 .525-.437.95-.976.95a.964.964 0 01-.976-.95v-1.752c0-.525.437-.951.976-.951zm5 0c.539 0 .976.426.976.95v1.753c0 .525-.437.95-.976.95a.964.964 0 01-.976-.95v-1.752c0-.525.437-.951.976-.951zM7.635 5.087c-1.05.102-1.935.438-2.385.906-.975 1.037-.765 3.668-.21 4.224.405.394 1.17.657 1.995.657h.09c.649-.013 1.785-.176 2.73-1.11.435-.41.705-1.433.675-2.47-.03-.834-.27-1.52-.63-1.813-.39-.336-1.275-.482-2.265-.394zm6.465.394c-.36.292-.6.98-.63 1.813-.03 1.037.24 2.06.675 2.47.968.957 2.136 1.104 2.776 1.11h.044c.825 0 1.59-.263 1.995-.657.555-.556.765-3.187-.21-4.224-.45-.468-1.335-.804-2.385-.906-.99-.088-1.875.058-2.265.394zM12 7.615c-.24 0-.525.015-.84.044.03.16.045.336.06.526l-.001.159a2.94 2.94 0 01-.014.25c.225-.022.425-.027.612-.028h.366c.187 0 .387.006.612.028-.015-.146-.015-.277-.015-.409.015-.19.03-.365.06-.526a9.29 9.29 0 00-.84-.044z"></path></svg>

After

Width:  |  Height:  |  Size: 2.0 KiB

View File

@@ -0,0 +1 @@
<svg fill="currentColor" fill-rule="evenodd" height="1em" style="flex:none;line-height:1" viewBox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><title>OpenAI</title><path d="M9.205 8.658v-2.26c0-.19.072-.333.238-.428l4.543-2.616c.619-.357 1.356-.523 2.117-.523 2.854 0 4.662 2.212 4.662 4.566 0 .167 0 .357-.024.547l-4.71-2.759a.797.797 0 00-.856 0l-5.97 3.473zm10.609 8.8V12.06c0-.333-.143-.57-.429-.737l-5.97-3.473 1.95-1.118a.433.433 0 01.476 0l4.543 2.617c1.309.76 2.189 2.378 2.189 3.948 0 1.808-1.07 3.473-2.76 4.163zM7.802 12.703l-1.95-1.142c-.167-.095-.239-.238-.239-.428V5.899c0-2.545 1.95-4.472 4.591-4.472 1 0 1.927.333 2.712.928L8.23 5.067c-.285.166-.428.404-.428.737v6.898zM12 15.128l-2.795-1.57v-3.33L12 8.658l2.795 1.57v3.33L12 15.128zm1.796 7.23c-1 0-1.927-.332-2.712-.927l4.686-2.712c.285-.166.428-.404.428-.737v-6.898l1.974 1.142c.167.095.238.238.238.428v5.233c0 2.545-1.974 4.472-4.614 4.472zm-5.637-5.303l-4.544-2.617c-1.308-.761-2.188-2.378-2.188-3.948A4.482 4.482 0 014.21 6.327v5.423c0 .333.143.571.428.738l5.947 3.449-1.95 1.118a.432.432 0 01-.476 0zm-.262 3.9c-2.688 0-4.662-2.021-4.662-4.519 0-.19.024-.38.047-.57l4.686 2.71c.286.167.571.167.856 0l5.97-3.448v2.26c0 .19-.07.333-.237.428l-4.543 2.616c-.619.357-1.356.523-2.117.523zm5.899 2.83a5.947 5.947 0 005.827-4.756C22.287 18.339 24 15.84 24 13.296c0-1.665-.713-3.282-1.998-4.448.119-.5.19-.999.19-1.498 0-3.401-2.759-5.947-5.946-5.947-.642 0-1.26.095-1.88.31A5.962 5.962 0 0010.205 0a5.947 5.947 0 00-5.827 4.757C1.713 5.447 0 7.945 0 10.49c0 1.666.713 3.283 1.998 4.448-.119.5-.19 1-.19 1.499 0 3.401 2.759 5.946 5.946 5.946.642 0 1.26-.095 1.88-.309a5.96 5.96 0 004.162 1.713z"></path></svg>

After

Width:  |  Height:  |  Size: 1.6 KiB

View File

@@ -0,0 +1 @@
<svg fill="currentColor" fill-rule="evenodd" height="1em" style="flex:none;line-height:1" viewBox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><title>Qwen</title><path d="M12.604 1.34c.393.69.784 1.382 1.174 2.075a.18.18 0 00.157.091h5.552c.174 0 .322.11.446.327l1.454 2.57c.19.337.24.478.024.837-.26.43-.513.864-.76 1.3l-.367.658c-.106.196-.223.28-.04.512l2.652 4.637c.172.301.111.494-.043.77-.437.785-.882 1.564-1.335 2.34-.159.272-.352.375-.68.37-.777-.016-1.552-.01-2.327.016a.099.099 0 00-.081.05 575.097 575.097 0 01-2.705 4.74c-.169.293-.38.363-.725.364-.997.003-2.002.004-3.017.002a.537.537 0 01-.465-.271l-1.335-2.323a.09.09 0 00-.083-.049H4.982c-.285.03-.553-.001-.805-.092l-1.603-2.77a.543.543 0 01-.002-.54l1.207-2.12a.198.198 0 000-.197 550.951 550.951 0 01-1.875-3.272l-.79-1.395c-.16-.31-.173-.496.095-.965.465-.813.927-1.625 1.387-2.436.132-.234.304-.334.584-.335a338.3 338.3 0 012.589-.001.124.124 0 00.107-.063l2.806-4.895a.488.488 0 01.422-.246c.524-.001 1.053 0 1.583-.006L11.704 1c.341-.003.724.032.9.34zm-3.432.403a.06.06 0 00-.052.03L6.254 6.788a.157.157 0 01-.135.078H3.253c-.056 0-.07.025-.041.074l5.81 10.156c.025.042.013.062-.034.063l-2.795.015a.218.218 0 00-.2.116l-1.32 2.31c-.044.078-.021.118.068.118l5.716.008c.046 0 .08.02.104.061l1.403 2.454c.046.081.092.082.139 0l5.006-8.76.783-1.382a.055.055 0 01.096 0l1.424 2.53a.122.122 0 00.107.062l2.763-.02a.04.04 0 00.035-.02.041.041 0 000-.04l-2.9-5.086a.108.108 0 010-.113l.293-.507 1.12-1.977c.024-.041.012-.062-.035-.062H9.2c-.059 0-.073-.026-.043-.077l1.434-2.505a.107.107 0 000-.114L9.225 1.774a.06.06 0 00-.053-.031zm6.29 8.02c.046 0 .058.02.034.06l-.832 1.465-2.613 4.585a.056.056 0 01-.05.029.058.058 0 01-.05-.029L8.498 9.841c-.02-.034-.01-.052.028-.054l.216-.012 6.722-.012z"></path></svg>

After

Width:  |  Height:  |  Size: 1.7 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 717 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 815 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 637 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 687 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.0 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 825 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 634 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 837 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 712 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 843 KiB

57
lefthook.yml Normal file
View File

@@ -0,0 +1,57 @@
commit-msg:
commands:
conventional:
run: |
msg=$(head -1 {1})
if [[ ! "$msg" =~ ^(feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert)(\(.+\))?\!?:\ .+ ]]; then
echo "Commit message must follow Conventional Commits format:"
echo " <type>(<optional scope>): <description>"
echo " Types: feat, fix, docs, style, refactor, perf, test, chore, ci, build, revert"
echo ""
echo "Examples:"
echo " feat(auth): add OAuth2 support"
echo " fix: resolve null pointer exception"
exit 1
fi
pre-commit:
commands:
biome-check:
root: "packages/browseros-agent/"
glob: "*.{js,ts,cjs,mjs,d.cts,d.mts,jsx,tsx,json,jsonc}"
run: npx @biomejs/biome check --write --no-errors-on-unmatched --files-ignore-unknown=true --colors=off {staged_files}
stage_fixed: true
file-length:
root: "packages/browseros-agent/"
glob: "*.{ts,tsx}"
exclude: "*.{test,spec,d}.ts|*.{test,spec}.tsx|**/__tests__/**|**/tests/**|**/*.generated.*"
run: |
for file in {staged_files}; do
if [[ -f "$file" ]]; then
lines=$(wc -l < "$file" | tr -d ' ')
if [[ $lines -gt 400 ]]; then
echo "⚠️ Warning: $file has $lines lines (threshold: 400)"
echo " Consider splitting this file if it has multiple responsibilities."
fi
fi
done
pre-push:
commands:
branch-name:
run: |
branch=$(git rev-parse --abbrev-ref HEAD)
if [[ "$branch" == "main" || "$branch" == "master" ]]; then
exit 0
fi
if [[ ! "$branch" =~ ^(feat|fix|bugfix|hotfix|release|docs|refactor|test|chore|experiment)/[a-z0-9-]+$ ]]; then
echo "⚠️ Warning: Branch name '$branch' doesn't match recommended format."
echo " Use: <type>/<short-description>"
echo " Types: feat, fix, bugfix, hotfix, release, docs, refactor, test, chore, experiment"
echo " Example: feat/add-auth, fix/login-crash"
echo ""
echo " To rename your branch:"
echo " git branch -m <new-name>"
echo " git push -u origin <new-name>"
fi

View File

@@ -0,0 +1,286 @@
---
name: test-ui
description: Test the BrowserOS agent extension UI by starting the dev environment and visually verifying changes via CDP. Covers the new tab page (left sidebar — Home, Scheduled Tasks, Settings, etc.) and the right side panel (chat interface). Use after making UI changes to apps/agent/.
argument-hint: [what to test, e.g. "verify the new settings page renders correctly"]
---
# Test Agent UI
Visually test the BrowserOS agent extension UI — both the new tab page (left sidebar) and the right side panel (chat) — by starting the dev environment and inspecting via CDP.
## When to use
After making code changes to `apps/agent/` (the Chrome extension), use this skill to:
- Verify new UI components render correctly
- Check navigation between views works
- Confirm layout/styling changes look right
- Test interactive elements (buttons, inputs, forms)
## Prerequisites
- **Go** must be installed (`brew install go`) — the dev tool is written in Go
- **BrowserOS.app** must be installed at `/Applications/BrowserOS.app/`
- The `scripts/dev/inspect-ui.ts` utility must exist (CDP inspector script)
## Step 1: Start the dev environment
```bash
bun run dev:watch -- --new
```
This single command handles everything:
- Builds the Go dev CLI tool
- Picks random available ports (avoids conflicts)
- Creates a fresh browser profile
- Builds controller-ext
- Runs GraphQL codegen if `apps/agent/generated/graphql/` doesn't exist
- Starts the agent extension with WXT HMR (hot module replacement)
- Waits for CDP to be ready
- Starts the MCP server
Run it in the background and **read the output to find the CDP port**:
```
[info] Ports: CDP=9552 Server=9065 Extension=9929
```
The CDP port is randomized. You MUST extract it from the output and set it for all subsequent commands:
```bash
export BROWSEROS_CDP_PORT=<port from output>
```
Wait for these messages before proceeding:
1. `[server] CDP ready`
2. `[server] HTTP server listening`
## Step 2: Discover targets
```bash
bun scripts/dev/inspect-ui.ts targets
```
You will see targets like:
- `[service_worker]` — extension background scripts (not directly testable for UI)
- `[page] chrome-extension://bflpfmnmnokmjhmgnolecpppdbdophmk/app.html#/...`**New tab page (left sidebar)**
- `[page] sidepanel.html`**Right side panel (chat)**
The two main testable surfaces:
- **`app.html`** — the new tab page with left sidebar (Home, Connect Apps, Scheduled Tasks, Skills, Memory, Soul, Settings)
- **`sidepanel.html`** — the right side panel chat interface
## Step 3: Navigate to the main UI
A fresh profile opens the **onboarding page** (`app.html#/onboarding`). Navigate to the home page first:
```bash
bun scripts/dev/inspect-ui.ts eval app.html "window.location.hash = '#/home'"
```
Verify with a snapshot (not screenshot — snapshot is faster and sufficient for structural checks):
```bash
bun scripts/dev/inspect-ui.ts snapshot app.html
```
## Snapshot vs Screenshot
**Prefer `snapshot` for most checks** — it's fast, text-based, and tells you what elements exist, their text, and their IDs. Use it after every navigation or interaction to verify state.
**Use `screenshot` only when you need visual verification** — layout changes, CSS/styling, colors, images, or a final "does it look right" check. Screenshots are expensive (capture → save → read image).
| Check | Use |
|-------|-----|
| Did the page navigate? | `snapshot` — look for new elements |
| Does my new component render? | `snapshot` — look for its text/role |
| Did a click change state? | `snapshot` — check element names/values |
| Is the layout correct? | `screenshot` — visual check needed |
| Do CSS changes look right? | `screenshot` — visual check needed |
| Final verification before committing | `screenshot` — one visual confirmation |
## Step 4: Test the new tab page (left sidebar)
### Get element IDs
```bash
bun scripts/dev/inspect-ui.ts snapshot app.html
```
Output shows interactive elements with IDs:
```
[52] link "Home"
[57] link "Connect Apps"
[65] link "Scheduled Tasks"
[74] link "Skills"
[103] link "Settings"
```
### Navigate via click or hash routing
**Click-based** (use element IDs from snapshot):
```bash
bun scripts/dev/inspect-ui.ts click app.html 65 # Click "Scheduled Tasks"
```
**Hash routing** (faster, no snapshot needed):
```bash
bun scripts/dev/inspect-ui.ts eval app.html "window.location.hash = '#/settings'"
bun scripts/dev/inspect-ui.ts eval app.html "window.location.hash = '#/scheduled-tasks'"
bun scripts/dev/inspect-ui.ts eval app.html "window.location.hash = '#/home'"
```
### Verify navigation
```bash
# Snapshot to confirm the page changed (fast, preferred)
bun scripts/dev/inspect-ui.ts snapshot app.html
# Screenshot only if you need to check visual layout
bun scripts/dev/inspect-ui.ts screenshot app.html /tmp/settings.png
```
### CRITICAL: Re-snapshot after every navigation
React re-renders change element IDs. **Always run snapshot again** before clicking/filling after navigating to a new view. Using stale IDs will fail.
## Step 5: Open and test the right side panel
The side panel starts **disabled** in a fresh profile. Open it using BrowserOS-specific APIs:
```bash
bun scripts/dev/inspect-ui.ts open-sidepanel
```
Wait 2 seconds for it to appear as a target, then:
```bash
bun scripts/dev/inspect-ui.ts screenshot sidepanel /tmp/panel.png
bun scripts/dev/inspect-ui.ts snapshot sidepanel
```
### Interact with the side panel
```bash
# Get element IDs
bun scripts/dev/inspect-ui.ts snapshot sidepanel
# Output: [37] textbox "What should I do?"
# [124] button "Send"
# [60] link "Chat history"
# [99] button "Agent Mode ON"
# Fill the chat input and press Enter to send
bun scripts/dev/inspect-ui.ts fill sidepanel 37 "Hello world"
bun scripts/dev/inspect-ui.ts press_key sidepanel Enter
# Or click the Send button
bun scripts/dev/inspect-ui.ts click sidepanel 124
# Wait for a response to appear
bun scripts/dev/inspect-ui.ts wait_for sidepanel text "response text"
# Scroll down to see more content
bun scripts/dev/inspect-ui.ts scroll sidepanel down 3
# Hover over an element to test hover states
bun scripts/dev/inspect-ui.ts hover sidepanel 99
# Snapshot to verify state changed (fast, preferred)
bun scripts/dev/inspect-ui.ts snapshot sidepanel
# Screenshot only for visual/layout verification
bun scripts/dev/inspect-ui.ts screenshot sidepanel /tmp/result.png
```
## Step 6: Verify and iterate
### The core loop
```
snapshot → identify element IDs → click/fill/press_key → snapshot → verify
```
Use `screenshot` only when visual layout verification is needed (CSS changes, final check).
### After making code changes
1. Fix the code in `apps/agent/`
2. WXT HMR will hot-reload the extension automatically (watch mode)
3. Wait 2-3 seconds for the reload to complete
4. **Re-snapshot** — element IDs WILL change after HMR reload
5. Verify the fix with snapshot (or screenshot if visual)
### Check server logs
The dev server output (running in background) contains useful diagnostics:
- `[agent]` — WXT build/HMR status, compilation errors
- `[server]` — MCP server logs, tool execution, errors
- `[build]` — Extension build output
If the UI isn't rendering, check for build errors in the `[agent]` output.
### Check for JavaScript errors
```bash
bun scripts/dev/inspect-ui.ts eval sidepanel "JSON.stringify(window.__errors || 'no errors')"
```
Or check the console for React errors:
```bash
bun scripts/dev/inspect-ui.ts eval app.html "document.querySelector('#root')?.innerHTML?.substring(0, 200)"
```
### Verify API connectivity
The extension talks to the MCP server. Verify the server is reachable:
```bash
bun scripts/dev/inspect-ui.ts eval sidepanel "fetch('http://127.0.0.1:<serverPort>/health').then(r => r.ok).catch(() => false)"
```
### Common issues
| Symptom | Cause | Fix |
|---------|-------|-----|
| Blank page after navigation | React render error | Check `eval` for JS errors |
| Element IDs don't match | Page re-rendered (HMR/navigation) | Re-run `snapshot` before interacting |
| `open-sidepanel` fails | Extension not fully loaded | Wait longer after dev server starts |
| Click does nothing | Element not visible (below fold) | Use `scroll` first, then re-snapshot |
| `wait_for` times out | Content hasn't loaded yet | Check server logs for API errors |
## Available commands reference
| Command | Description |
|---------|-------------|
| `targets` | List all CDP targets, marks extension pages with `[EXTENSION]` |
| `screenshot <target> [file]` | Capture PNG screenshot (default: `screenshot.png`) |
| `snapshot <target>` | Print accessibility tree with `[elementId] role "name"` |
| `click <target> <elementId>` | Click element by ID (3-tier coordinate fallback + JS click) |
| `fill <target> <elementId> <text>` | Focus element, clear, type text |
| `press_key <target> <key>` | Press key or combo: `Enter`, `Escape`, `Tab`, `Control+A`, `Meta+Shift+P` |
| `scroll <target> <dir> [amount]` | Scroll `up`/`down`/`left`/`right`, amount in ticks (default 3) |
| `hover <target> <elementId>` | Hover over element (for tooltips, hover states) |
| `select_option <target> <id> <val>` | Select dropdown option by value or visible text |
| `wait_for <target> text\|selector <v>` | Wait up to 10s for text or CSS selector to appear |
| `eval <target> <expression>` | Run JavaScript in the target's context |
| `open-sidepanel` | Enable and open the right side panel |
`<target>` is a URL substring (e.g., `sidepanel`, `app.html`) or numeric index from `targets` output.
## Known app.html routes
These can be used with `eval app.html "window.location.hash = '#/<route>'"`:
| Route | View |
|-------|------|
| `/home` | Home page with search bar and top sites |
| `/settings` | Settings (LLM providers, customization, workflows, MCP) |
| `/scheduled-tasks` | Scheduled Tasks management |
| `/onboarding` | Onboarding flow (first-run experience) |
## Gotchas learned from real testing
1. **Ports are randomized** with `--new` — always extract from dev server output
2. **Fresh profile = onboarding page** — navigate to `#/home` to see the main UI
3. **Element IDs change after navigation** — always re-snapshot before clicking
4. **Side panel starts disabled**`open-sidepanel` handles the BrowserOS-specific enable + toggle API
5. **`Input.enable` does not exist** — the CDP Input domain has no enable method (already handled in the script)
6. **`DOM.getDocument` required** — must be called before DOM operations like `pushNodesByBackendIdsToFrontend` (already handled in the script)
7. **Settings sub-navigation** — the settings page has its own left sidebar (BrowserOS AI, Chat & Council Provider, Search Provider, Customize BrowserOS, BrowserOS as MCP, Workflows) — use snapshot + click to navigate within settings

View File

@@ -1,41 +0,0 @@
version: 2
updates:
- package-ecosystem: bun
directory: /
schedule:
interval: weekly
day: 'sunday'
time: '02:00'
timezone: Europe/Berlin
open-pull-requests-limit: 10
groups:
dependencies:
applies-to: security-updates
dependency-type: production
exclude-patterns:
- 'puppeteer*'
patterns:
- '*'
dev-dependencies:
applies-to: security-updates
dependency-type: development
exclude-patterns:
- 'puppeteer*'
patterns:
- '*'
puppeteer:
patterns:
- 'puppeteer*'
- package-ecosystem: github-actions
directory: /
schedule:
interval: weekly
day: 'sunday'
time: '04:00'
timezone: Europe/Berlin
open-pull-requests-limit: 10
groups:
all:
applies-to: security-updates
patterns:
- '*'

View File

@@ -1,58 +0,0 @@
name: CLA Assistant
on:
issue_comment:
types: [created]
pull_request_target:
types: [opened, closed, synchronize]
permissions:
actions: write
contents: write
pull-requests: write
statuses: write
jobs:
cla:
runs-on: ubuntu-latest
if: |
(github.event_name == 'pull_request_target') ||
(github.event_name == 'issue_comment' && github.event.issue.pull_request &&
(github.event.comment.body == 'recheck' ||
github.event.comment.body == 'I have read the CLA Document and I hereby sign the CLA'))
steps:
- name: CLA Assistant
uses: contributor-assistant/github-action@v2.6.1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PERSONAL_ACCESS_TOKEN: ${{ secrets.CLA_SIGNATURES_TOKEN }}
with:
path-to-signatures: 'cla-signatures.json'
path-to-document: 'https://github.com/${{ github.repository }}/blob/main/CLA.md'
branch: 'main'
remote-organization-name: 'browseros-ai'
remote-repository-name: 'cla-signatures'
allowlist: 'bot*,*[bot],dependabot,renovate,github-actions,snyk-bot,imgbot,greenkeeper,semantic-release-bot,allcontributors'
lock-pullrequest-aftermerge: false
custom-notsigned-prcomment: |
Thank you for your contribution! Before we can merge this PR, we need you to sign our [Contributor License Agreement](https://github.com/${{ github.repository }}/blob/main/CLA.md).
**To sign the CLA**, please add a comment to this PR with the following text:
```
I have read the CLA Document and I hereby sign the CLA
```
You only need to sign once. After signing, this check will pass automatically.
---
<details>
<summary>Troubleshooting</summary>
- **Already signed but still failing?** Comment `recheck` to trigger a re-verification.
- **Signed with a different email?** Make sure your commit email matches your GitHub account email, or add your commit email to your GitHub account.
</details>
custom-pr-sign-comment: 'I have read the CLA Document and I hereby sign the CLA'
custom-allsigned-prcomment: |
All contributors have signed the CLA. Thank you!

View File

@@ -1,37 +0,0 @@
name: Release Agent SDK
on:
workflow_dispatch:
jobs:
publish:
if: github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
defaults:
run:
working-directory: packages/agent-sdk
steps:
- uses: actions/checkout@v6
- uses: oven-sh/setup-bun@v2
- uses: actions/setup-node@v6
with:
node-version: "20"
registry-url: "https://registry.npmjs.org"
- name: Install dependencies
run: bun ci
working-directory: .
- name: Build
run: bun run build
- name: Test
run: bun test
- name: Publish
run: npm publish --access public
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

View File

@@ -1,24 +0,0 @@
name: Tests
on: []
jobs:
test:
name: Run Tests
runs-on: macos-latest
timeout-minutes: 10
steps:
- name: 📥 Checkout code
uses: actions/checkout@v6
- name: 🧰 Setup Bun
uses: oven-sh/setup-bun@v2
- name: 📦 Install dependencies
run: bun ci
- name: 🧪 Run all tests
run: bun test:all
env:
PUPPETEER_EXECUTABLE_PATH: /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

View File

@@ -187,7 +187,12 @@ log.txt
# Testing iteration temp files
tmp/
# CI artifacts
.ci/
test-results/
# Coding agent artifacts
.agent/
.llm/
.grove/
docs/plans/2026-03-24-models-dev-integration.md

View File

@@ -81,6 +81,9 @@ bun run dev:server # Build server for development
bun run dev:ext # Build extension for development
bun run dist:server # Build server for production (all targets)
bun run dist:ext # Build extension for production
# Refresh models.dev data
bun run generate:models # Fetches latest from models.dev/api.json
```
## Architecture
@@ -165,3 +168,68 @@ Tests are in `apps/server/tests/`:
- `agent/` - Agent tests (compaction, rate limiter)
- `sdk/` - Agent SDK tests
- `__helpers__/` - Test utilities and fixtures
## Self-Testing UI Changes
After making UI changes to the agent extension (`apps/agent/`), you can visually verify them using the CDP inspector script. This connects directly to the browser via Chrome DevTools Protocol and can inspect extension pages (side panel, new tab, etc.) that the agent's own tools cannot see.
### Prerequisites
The dev server must be running:
```bash
bun run dev:watch -- --new
```
Read the output to find the randomized CDP port, then:
```bash
export BROWSEROS_CDP_PORT=<port from output>
```
### Workflow
1. **List all targets** to see what's available:
```bash
bun scripts/dev/inspect-ui.ts targets
```
2. **Open the side panel** if it's not already open:
```bash
bun scripts/dev/inspect-ui.ts open-sidepanel
```
3. **Take a screenshot** of the side panel:
```bash
bun scripts/dev/inspect-ui.ts screenshot sidepanel /tmp/panel.png
```
Then read `/tmp/panel.png` to view the result.
4. **Get the accessibility tree** for structural verification:
```bash
bun scripts/dev/inspect-ui.ts snapshot sidepanel
```
5. **Click an element** by its ID from the snapshot:
```bash
bun scripts/dev/inspect-ui.ts click sidepanel 142
```
6. **Fill a text input** by its ID from the snapshot:
```bash
bun scripts/dev/inspect-ui.ts fill sidepanel 85 "search query"
```
7. **Evaluate JavaScript** in the extension context:
```bash
bun scripts/dev/inspect-ui.ts eval sidepanel "document.title"
```
### Interaction workflow
The typical loop is: snapshot → identify element IDs → click/fill → screenshot to verify.
Element IDs come from the `[number]` in snapshot output (these are `backendDOMNodeId` values).
This uses the same element resolution as the server's MCP tools — no coordinate guessing.
### Target selection
The `<target>` argument can be:
- An **index** from the `targets` output (e.g., `3`)
- A **URL substring** (e.g., `sidepanel`, `newtab`, `chrome-extension://`)

View File

@@ -1,8 +1,6 @@
# BrowserOS Agent
Monorepo for the BrowserOS-agent -- contains 3 packages: agent-UI, server (which contains the agent loop) and controller-extension (which is used by the tools within the agent loop).
> **⚠️ NOTE:** This is only a submodule, the main project is at -- https://github.com/browseros-ai/BrowserOS
The agent platform powering [BrowserOS](https://github.com/browseros-ai/BrowserOS) — contains the MCP server, agent UI, CLI, evaluation framework, and SDK.
## Monorepo Structure
@@ -10,17 +8,25 @@ Monorepo for the BrowserOS-agent -- contains 3 packages: agent-UI, server (which
apps/
server/ # Bun server - MCP endpoints + agent loop
agent/ # Agent UI (Chrome extension)
cli/ # Go CLI for controlling BrowserOS from the terminal
eval/ # Evaluation framework for benchmarking agents
controller-ext/ # BrowserOS Controller (Chrome extension for chrome.* APIs)
packages/
agent-sdk/ # Node.js SDK (@browseros-ai/agent-sdk)
cdp-protocol/ # Type-safe Chrome DevTools Protocol bindings
shared/ # Shared constants (ports, timeouts, limits)
```
| Package | Description |
|---------|-------------|
| `apps/server` | Bun server exposing MCP tools and running the agent loop |
| `apps/agent` | Agent UI - Chrome extension for the chat interface |
| `apps/controller-ext` | BrowserOS Controller - Chrome extension that bridges `chrome.*` APIs (tabs, bookmarks, history) to the server via WebSocket |
| `apps/agent` | Agent UI Chrome extension for the chat interface |
| `apps/cli` | Go CLI — control BrowserOS from the terminal or AI coding agents |
| `apps/eval` | Benchmark framework — WebVoyager, Mind2Web evaluation |
| `apps/controller-ext` | BrowserOS Controller — bridges `chrome.*` APIs to the server via WebSocket |
| `packages/agent-sdk` | Node.js SDK for browser automation with natural language |
| `packages/cdp-protocol` | Auto-generated CDP type bindings used by the server |
| `packages/shared` | Shared constants used across packages |
## Architecture

View File

@@ -0,0 +1,6 @@
# BrowserOS Agent Extension
## v0.0.52 (2026-03-26)
Initial release

View File

@@ -1,16 +1,24 @@
# BrowserOS Agent Chrome Extension
# BrowserOS Agent Extension
The official Chrome extension for BrowserOS Agent, providing the UI layer for interacting with BrowserOS Core and Controllers. This extension enables intelligent browser automation, AI-powered search, and seamless integration with multiple LLM providers.
[![License: AGPL v3](https://img.shields.io/badge/License-AGPL%20v3-blue.svg)](../../../../LICENSE)
The built-in browser extension that powers BrowserOS's AI interface — new tab with unified search, side panel chat, onboarding, and settings. Built with [WXT](https://wxt.dev) and React.
> For user-facing feature documentation, see [docs.browseros.com](https://docs.browseros.com).
## Features
- **AI-Powered New Tab**: Custom new tab page with unified search across Google and AI assistants
- **Side Panel Chat**: Full-featured chat interface for interacting with BrowserOS Core
- **Side Panel Chat**: Full-featured chat interface for interacting with BrowserOS
- **Multi-Provider Support**: Connect to various LLM providers (OpenAI, Anthropic, Azure, Bedrock, and more)
- **MCP Integration**: Model Context Protocol support for extending AI capabilities
- **Visual Feedback**: Animated glow effect on tabs during AI agent operations
- **Privacy-First**: Local data handling with configurable provider settings
## How It Connects
The extension communicates with the [BrowserOS Server](../../apps/server/) running locally. The server handles the AI agent loop, MCP tools, and CDP connections — the extension provides the UI layer.
## Project Structure
```
@@ -80,47 +88,20 @@ Settings dashboard with multiple sections:
Content script that creates a visual indicator (pulsing orange glow) around the browser viewport when an AI agent is actively working on a tab.
## How Tools Are Used
### Bun
Bun is the exclusive runtime and package manager:
- All scripts use `bun run <script>` instead of npm
- Package installation via `bun install`
- Environment files automatically loaded (no dotenv needed)
- Enforced via `engines` field in `package.json`
```bash
bun install # Install dependencies
bun run dev # Development mode
bun run build # Production build
bun run lint # Run Biome linting
```
### Biome
Unified linter and formatter configured in `biome.json`:
- **Formatting**: 2-space indentation, single quotes, no semicolons
- **Linting**: Recommended rules plus custom rules for unused imports/variables
- **CSS Support**: Tailwind directives parsing enabled
- **Import Organization**: Automatic import sorting via assist actions
```bash
bun run lint # Check for issues
bun run lint:fix # Auto-fix issues
```
## Development
### Prerequisites
- [Bun](https://bun.sh) installed
- Chrome or Chromium-based browser
- BrowserOS Core running locally (for full functionality)
- BrowserOS Server running locally (for full functionality)
### Setup
```bash
# Copy environment file
cp .env.example .env.development
# Install dependencies
bun install
@@ -153,12 +134,30 @@ SENTRY_AUTH_TOKEN=your-token
### GraphQL Schema
Codegen requires a GraphQL schema. By default it uses the bundled `schema/schema.graphql`, so no extra setup is needed. If you have access to the original API source, you can set the following environment variable
Codegen requires a GraphQL schema. By default it uses the bundled `schema/schema.graphql`, so no extra setup is needed. If you have access to the original API source, you can set the following environment variable:
```env
GRAPHQL_SCHEMA_PATH=/path/to/api-repo/.../schema.graphql
```
## Development Tooling
### Bun
Bun is the exclusive runtime and package manager:
- All scripts use `bun run <script>` instead of npm
- Package installation via `bun install`
- Environment files automatically loaded (no dotenv needed)
- Enforced via `engines` field in `package.json`
### Biome
Unified linter and formatter configured in `biome.json`:
- **Formatting**: 2-space indentation, single quotes, no semicolons
- **Linting**: Recommended rules plus custom rules for unused imports/variables
- **CSS Support**: Tailwind directives parsing enabled
- **Import Organization**: Automatic import sorting via assist actions
## Scripts
| Script | Description |
@@ -169,4 +168,5 @@ GRAPHQL_SCHEMA_PATH=/path/to/api-repo/.../schema.graphql
| `bun run lint` | Run Biome linter |
| `bun run lint:fix` | Auto-fix linting issues |
| `bun run typecheck` | Run TypeScript type checking |
| `bun run codegen` | Generate GraphQL types |
| `bun run clean:cache` | Clear build caches |

View File

@@ -1,5 +1,5 @@
{
"$schema": "https://biomejs.dev/schemas/2.4.5/schema.json",
"$schema": "https://biomejs.dev/schemas/2.4.8/schema.json",
"root": false,
"extends": "//",
"vcs": {

View File

@@ -66,7 +66,7 @@ export const RunResultDialog: FC<RunResultDialogProps> = ({
return (
<Dialog open={!!run} onOpenChange={onOpenChange}>
<DialogContent className="sm:max-w-2xl">
<DialogContent className="sm:w-[70vw] sm:max-w-4xl">
<DialogHeader>
<DialogTitle className="flex items-center gap-2">
{run.status === 'completed' ? (
@@ -94,7 +94,7 @@ export const RunResultDialog: FC<RunResultDialogProps> = ({
<p className="text-destructive text-sm">{run.result}</p>
</div>
) : run.result ? (
<div className="prose prose-sm dark:prose-invert [&_[data-streamdown='code-block']]:!w-full [&_[data-streamdown='table-wrapper']]:!w-full max-w-none break-words rounded-lg border border-border bg-muted/50 p-4">
<div className="prose prose-sm dark:prose-invert [&_[data-streamdown='code-block']]:!w-full [&_[data-streamdown='table-wrapper']]:!w-full max-w-none break-words rounded-lg border border-border bg-muted/50 p-4 [&_[data-streamdown='table-wrapper']]:overflow-x-auto">
<MessageResponse>{run.result}</MessageResponse>
</div>
) : (

View File

@@ -1,4 +1,4 @@
import { Check } from 'lucide-react'
import { Check, Plus } from 'lucide-react'
import type { FC, PropsWithChildren } from 'react'
import { useState } from 'react'
import {
@@ -77,6 +77,19 @@ export const ChatProviderSelector: FC<
)
})}
</CommandGroup>
<div className="border-border border-t p-1">
<button
type="button"
className="flex w-full items-center gap-3 rounded-md p-2 text-muted-foreground text-sm transition-colors hover:bg-accent hover:text-foreground"
onClick={() => {
window.open('/app.html#/settings/ai', '_blank')
setOpen(false)
}}
>
<Plus className="h-4 w-4" />
Add Provider
</button>
</div>
</CommandList>
</Command>
</PopoverContent>

View File

@@ -0,0 +1,26 @@
import { Coins } from 'lucide-react'
import type { FC } from 'react'
import { getCreditTextColor } from '@/lib/credits/credit-colors'
import { cn } from '@/lib/utils'
interface CreditBadgeProps {
credits: number
onClick?: () => void
}
export const CreditBadge: FC<CreditBadgeProps> = ({ credits, onClick }) => {
return (
<button
type="button"
onClick={onClick}
className={cn(
'inline-flex cursor-pointer items-center gap-1 rounded-md px-1.5 py-0.5 font-medium text-xs transition-colors hover:bg-muted/50',
getCreditTextColor(credits),
)}
title={`${credits} credits remaining`}
>
<Coins className="h-3.5 w-3.5" />
<span>{credits}</span>
</button>
)
}

View File

@@ -3,6 +3,7 @@ import {
BookOpen,
Bot,
Compass,
CreditCard,
GitBranch,
MessageSquare,
Palette,
@@ -79,6 +80,12 @@ const primarySettingsSections: NavSection[] = [
feature: Feature.CUSTOMIZATION_SUPPORT,
},
{ name: 'BrowserOS as MCP', to: '/settings/mcp', icon: Server },
{
name: 'Usage & Billing',
to: '/settings/usage',
icon: CreditCard,
feature: Feature.CREDITS_SUPPORT,
},
{
name: 'Workflows',
to: '/workflows',

View File

@@ -176,14 +176,14 @@ function AlertDialogCancel({
export {
AlertDialog,
AlertDialogPortal,
AlertDialogOverlay,
AlertDialogTrigger,
AlertDialogContent,
AlertDialogHeader,
AlertDialogFooter,
AlertDialogTitle,
AlertDialogDescription,
AlertDialogAction,
AlertDialogCancel,
AlertDialogContent,
AlertDialogDescription,
AlertDialogFooter,
AlertDialogHeader,
AlertDialogOverlay,
AlertDialogPortal,
AlertDialogTitle,
AlertDialogTrigger,
}

View File

@@ -72,4 +72,4 @@ function AlertDescription({
)
}
export { Alert, AlertTitle, AlertDescription }
export { Alert, AlertDescription, AlertTitle }

View File

@@ -104,10 +104,10 @@ function CardFooter({ className, ...props }: React.ComponentProps<'div'>) {
export {
Card,
CardHeader,
CardFooter,
CardTitle,
CardAction,
CardDescription,
CardContent,
CardDescription,
CardFooter,
CardHeader,
CardTitle,
}

View File

@@ -251,10 +251,10 @@ function CarouselNext({
}
export {
type CarouselApi,
Carousel,
type CarouselApi,
CarouselContent,
CarouselItem,
CarouselPrevious,
CarouselNext,
CarouselPrevious,
}

View File

@@ -39,4 +39,4 @@ function CollapsibleContent({
)
}
export { Collapsible, CollapsibleTrigger, CollapsibleContent }
export { Collapsible, CollapsibleContent, CollapsibleTrigger }

View File

@@ -198,11 +198,11 @@ function CommandShortcut({
export {
Command,
CommandDialog,
CommandInput,
CommandList,
CommandEmpty,
CommandGroup,
CommandInput,
CommandItem,
CommandShortcut,
CommandList,
CommandSeparator,
CommandShortcut,
}

View File

@@ -283,18 +283,18 @@ function DropdownMenuSubContent({
export {
DropdownMenu,
DropdownMenuPortal,
DropdownMenuTrigger,
DropdownMenuCheckboxItem,
DropdownMenuContent,
DropdownMenuGroup,
DropdownMenuLabel,
DropdownMenuItem,
DropdownMenuCheckboxItem,
DropdownMenuLabel,
DropdownMenuPortal,
DropdownMenuRadioGroup,
DropdownMenuRadioItem,
DropdownMenuSeparator,
DropdownMenuShortcut,
DropdownMenuSub,
DropdownMenuSubTrigger,
DropdownMenuSubContent,
DropdownMenuSubTrigger,
DropdownMenuTrigger,
}

View File

@@ -179,12 +179,12 @@ function FormMessage({ className, ...props }: React.ComponentProps<'p'>) {
}
export {
useFormField,
Form,
FormItem,
FormLabel,
FormControl,
FormDescription,
FormMessage,
FormField,
FormItem,
FormLabel,
FormMessage,
useFormField,
}

View File

@@ -50,4 +50,4 @@ function HoverCardContent({
)
}
export { HoverCard, HoverCardTrigger, HoverCardContent }
export { HoverCard, HoverCardContent, HoverCardTrigger }

View File

@@ -184,7 +184,7 @@ export {
InputGroup,
InputGroupAddon,
InputGroupButton,
InputGroupText,
InputGroupInput,
InputGroupText,
InputGroupTextarea,
}

View File

@@ -55,4 +55,4 @@ function PopoverAnchor({
return <PopoverPrimitive.Anchor data-slot="popover-anchor" {...props} />
}
export { Popover, PopoverTrigger, PopoverContent, PopoverAnchor }
export { Popover, PopoverAnchor, PopoverContent, PopoverTrigger }

View File

@@ -49,4 +49,4 @@ function ResizableHandle({
)
}
export { ResizablePanelGroup, ResizablePanel, ResizableHandle }
export { ResizableHandle, ResizablePanel, ResizablePanelGroup }

View File

@@ -129,11 +129,11 @@ function SheetDescription({
export {
Sheet,
SheetTrigger,
SheetClose,
SheetContent,
SheetHeader,
SheetFooter,
SheetTitle,
SheetDescription,
SheetFooter,
SheetHeader,
SheetTitle,
SheetTrigger,
}

View File

@@ -18,6 +18,7 @@ const Toaster = ({ ...props }: ToasterProps) => {
<Sonner
theme={theme as ToasterProps['theme']}
className="toaster group"
closeButton
icons={{
success: <CircleCheckIcon className="size-4" />,
info: <InfoIcon className="size-4" />,

View File

@@ -86,4 +86,4 @@ function TabsContent({
)
}
export { Tabs, TabsList, TabsTrigger, TabsContent, tabsListVariants }
export { Tabs, TabsContent, TabsList, TabsTrigger, tabsListVariants }

View File

@@ -68,4 +68,4 @@ function TooltipContent({
)
}
export { Tooltip, TooltipTrigger, TooltipContent, TooltipProvider }
export { Tooltip, TooltipContent, TooltipProvider, TooltipTrigger }

View File

@@ -2,6 +2,7 @@ import type { FC } from 'react'
import { HashRouter, Navigate, Route, Routes, useParams } from 'react-router'
import { NewTab } from '../newtab/index/NewTab'
import { NewTabChat } from '../newtab/index/NewTabChat'
import { NewTabLayout } from '../newtab/layout/NewTabLayout'
import { Personalize } from '../newtab/personalize/Personalize'
import { OnboardingDemo } from '../onboarding/demo/OnboardingDemo'
@@ -27,6 +28,7 @@ import { ScheduledTasksPage } from './scheduled-tasks/ScheduledTasksPage'
import { SearchProviderPage } from './search-provider/SearchProviderPage'
import { SkillsPage } from './skills/SkillsPage'
import { SoulPage } from './soul/SoulPage'
import { UsagePage } from './usage/UsagePage'
import { WorkflowsPageWrapper } from './workflows/WorkflowsPageWrapper'
function getSurveyParams(): { maxTurns?: number; experimentId?: string } {
@@ -79,6 +81,7 @@ export const App: FC = () => {
{/* Home routes */}
<Route path="home" element={<NewTabLayout />}>
<Route index element={<NewTab />} />
<Route path="chat" element={<NewTabChat />} />
<Route path="personalize" element={<Personalize />} />
<Route path="soul" element={<SoulPage />} />
<Route path="skills" element={<SkillsPage />} />
@@ -101,6 +104,7 @@ export const App: FC = () => {
<Route path="customization" element={<CustomizationPage />} />
<Route path="search" element={<SearchProviderPage />} />
<Route path="survey" element={<SurveyPage {...surveyParams} />} />
<Route path="usage" element={<UsagePage />} />
</Route>
</Route>

View File

@@ -13,6 +13,17 @@ import {
} from '@/components/ui/alert-dialog'
import { useSessionInfo } from '@/lib/auth/sessionStorage'
import { useAgentServerUrl } from '@/lib/browseros/useBrowserOSProviders'
import {
CHATGPT_PRO_OAUTH_COMPLETED_EVENT,
CHATGPT_PRO_OAUTH_DISCONNECTED_EVENT,
CHATGPT_PRO_OAUTH_STARTED_EVENT,
GITHUB_COPILOT_OAUTH_COMPLETED_EVENT,
GITHUB_COPILOT_OAUTH_DISCONNECTED_EVENT,
GITHUB_COPILOT_OAUTH_STARTED_EVENT,
QWEN_CODE_OAUTH_COMPLETED_EVENT,
QWEN_CODE_OAUTH_DISCONNECTED_EVENT,
QWEN_CODE_OAUTH_STARTED_EVENT,
} from '@/lib/constants/analyticsEvents'
import { GetProfileIdByUserIdDocument } from '@/lib/conversations/graphql/uploadConversationDocument'
import { getQueryKeyFromDocument } from '@/lib/graphql/getQueryKeyFromDocument'
import { useGraphqlMutation } from '@/lib/graphql/useGraphqlMutation'
@@ -21,7 +32,13 @@ import type { ProviderTemplate } from '@/lib/llm-providers/providerTemplates'
import { testProvider } from '@/lib/llm-providers/testProvider'
import type { LlmProviderConfig } from '@/lib/llm-providers/types'
import { useLlmProviders } from '@/lib/llm-providers/useLlmProviders'
import {
type OAuthProviderFlowConfig,
useOAuthProviderFlow,
} from '@/lib/llm-providers/useOAuthProviderFlow'
import { track } from '@/lib/metrics/track'
import { ConfiguredProvidersList } from './ConfiguredProvidersList'
import { DeviceCodeDialog } from './DeviceCodeDialog'
import {
DeleteRemoteLlmProviderDocument,
GetRemoteLlmProvidersDocument,
@@ -29,9 +46,51 @@ import {
import type { IncompleteProvider } from './IncompleteProviderCard'
import { IncompleteProvidersList } from './IncompleteProvidersList'
import { LlmProvidersHeader } from './LlmProvidersHeader'
import { McpPromoBanner } from './McpPromoBanner'
import { NewProviderDialog } from './NewProviderDialog'
import { ProviderTemplatesSection } from './ProviderTemplatesSection'
// All OAuth providers share the same flow via useOAuthProviderFlow
const OAUTH_PROVIDERS_CONFIG: Record<string, OAuthProviderFlowConfig> = {
'chatgpt-pro': {
providerType: 'chatgpt-pro',
displayName: 'ChatGPT Plus/Pro',
startedEvent: CHATGPT_PRO_OAUTH_STARTED_EVENT,
completedEvent: CHATGPT_PRO_OAUTH_COMPLETED_EVENT,
disconnectedEvent: CHATGPT_PRO_OAUTH_DISCONNECTED_EVENT,
},
'github-copilot': {
providerType: 'github-copilot',
displayName: 'GitHub Copilot',
startedEvent: GITHUB_COPILOT_OAUTH_STARTED_EVENT,
completedEvent: GITHUB_COPILOT_OAUTH_COMPLETED_EVENT,
disconnectedEvent: GITHUB_COPILOT_OAUTH_DISCONNECTED_EVENT,
clientAuth: {
deviceCodeEndpoint: 'https://github.com/login/device/code',
tokenEndpoint: 'https://github.com/login/oauth/access_token',
clientId: 'Ov23li8tweQw6odWQebz',
scopes: 'read:user',
requiresPKCE: false,
contentType: 'json',
},
},
'qwen-code': {
providerType: 'qwen-code',
displayName: 'Qwen Code',
startedEvent: QWEN_CODE_OAUTH_STARTED_EVENT,
completedEvent: QWEN_CODE_OAUTH_COMPLETED_EVENT,
disconnectedEvent: QWEN_CODE_OAUTH_DISCONNECTED_EVENT,
clientAuth: {
deviceCodeEndpoint: 'https://chat.qwen.ai/api/v1/oauth2/device/code',
tokenEndpoint: 'https://chat.qwen.ai/api/v1/oauth2/token',
clientId: 'f0304373b74a44d2b584a3fb70ca9e56',
scopes: 'openid profile email model.completion',
requiresPKCE: true,
contentType: 'form',
},
},
}
/**
* AI Settings page for managing LLM providers
* @public
@@ -78,9 +137,7 @@ export const AISettingsPage: FC = () => {
const incompleteProviders = useMemo<IncompleteProvider[]>(() => {
if (!remoteProvidersData?.llmProviders?.nodes) return []
const localProviderIds = new Set(providers.map((p) => p.id))
return remoteProvidersData.llmProviders.nodes
.filter((node): node is NonNullable<typeof node> => node !== null)
.filter((node) => !localProviderIds.has(node.rowId))
@@ -101,12 +158,71 @@ export const AISettingsPage: FC = () => {
null,
)
// OAuth flows — shared hook eliminates per-provider duplication
const chatgptPro = useOAuthProviderFlow(
OAUTH_PROVIDERS_CONFIG['chatgpt-pro'],
providers,
saveProvider,
)
const copilot = useOAuthProviderFlow(
OAUTH_PROVIDERS_CONFIG['github-copilot'],
providers,
saveProvider,
)
const qwenCode = useOAuthProviderFlow(
OAUTH_PROVIDERS_CONFIG['qwen-code'],
providers,
saveProvider,
)
const activeDeviceCode =
chatgptPro.pendingDeviceCode ??
copilot.pendingDeviceCode ??
qwenCode.pendingDeviceCode
const clearActiveDeviceCode = () => {
chatgptPro.clearDeviceCode()
copilot.clearDeviceCode()
qwenCode.clearDeviceCode()
}
const oauthFlows: Record<
string,
{
startOAuthFlow: (url: string | undefined) => Promise<void>
disconnect: () => Promise<void>
disconnectedEvent: string
}
> = {
'chatgpt-pro': {
startOAuthFlow: chatgptPro.startOAuthFlow,
disconnect: chatgptPro.disconnect,
disconnectedEvent: CHATGPT_PRO_OAUTH_DISCONNECTED_EVENT,
},
'github-copilot': {
startOAuthFlow: copilot.startOAuthFlow,
disconnect: copilot.disconnect,
disconnectedEvent: GITHUB_COPILOT_OAUTH_DISCONNECTED_EVENT,
},
'qwen-code': {
startOAuthFlow: qwenCode.startOAuthFlow,
disconnect: qwenCode.disconnect,
disconnectedEvent: QWEN_CODE_OAUTH_DISCONNECTED_EVENT,
},
}
const handleAddProvider = () => {
setTemplateValues(undefined)
setIsNewDialogOpen(true)
}
const handleUseTemplate = (template: ProviderTemplate) => {
// OAuth providers: trigger OAuth flow
const oauthFlow = oauthFlows[template.id]
if (oauthFlow) {
oauthFlow.startOAuthFlow(agentServerUrl ?? undefined)
return
}
setTemplateValues({
type: template.id,
name: template.name,
@@ -129,11 +245,18 @@ export const AISettingsPage: FC = () => {
}
const confirmDeleteProvider = async () => {
if (providerToDelete) {
await deleteProvider(providerToDelete.id)
deleteRemoteProviderMutation.mutate({ rowId: providerToDelete.id })
setProviderToDelete(null)
if (!providerToDelete) return
// Clear OAuth tokens on server for OAuth-based providers
const oauthFlow = oauthFlows[providerToDelete.type]
if (oauthFlow) {
await oauthFlow.disconnect()
track(oauthFlow.disconnectedEvent)
}
await deleteProvider(providerToDelete.id)
deleteRemoteProviderMutation.mutate({ rowId: providerToDelete.id })
setProviderToDelete(null)
}
const handleAddKeysToIncomplete = (provider: IncompleteProvider) => {
@@ -236,6 +359,8 @@ export const AISettingsPage: FC = () => {
onAddProvider={handleAddProvider}
/>
<McpPromoBanner />
<ProviderTemplatesSection onUseTemplate={handleUseTemplate} />
<ConfiguredProvidersList
@@ -310,6 +435,11 @@ export const AISettingsPage: FC = () => {
</AlertDialogFooter>
</AlertDialogContent>
</AlertDialog>
<DeviceCodeDialog
deviceCode={activeDeviceCode}
onClose={clearActiveDeviceCode}
/>
</div>
)
}

View File

@@ -0,0 +1,81 @@
import { Check, Copy, ExternalLink } from 'lucide-react'
import { type FC, useState } from 'react'
import { Button } from '@/components/ui/button'
import {
Dialog,
DialogContent,
DialogDescription,
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog'
import type { PendingDeviceCode } from '@/lib/llm-providers/useOAuthProviderFlow'
interface DeviceCodeDialogProps {
deviceCode: PendingDeviceCode | null
onClose: () => void
}
export const DeviceCodeDialog: FC<DeviceCodeDialogProps> = ({
deviceCode,
onClose,
}) => {
const [copied, setCopied] = useState(false)
const handleCopy = async () => {
if (!deviceCode) return
try {
await navigator.clipboard.writeText(deviceCode.userCode)
setCopied(true)
setTimeout(() => setCopied(false), 2000)
} catch {
// Clipboard API failed
}
}
return (
<Dialog open={!!deviceCode} onOpenChange={(open) => !open && onClose()}>
<DialogContent className="sm:max-w-md">
<DialogHeader>
<DialogTitle>Connect to {deviceCode?.providerName}</DialogTitle>
<DialogDescription>
Paste this code on the {deviceCode?.providerName} page that just
opened in your browser.
</DialogDescription>
</DialogHeader>
<div className="flex flex-col items-center gap-4 py-4">
<div className="flex items-center gap-3 rounded-xl border-2 border-[var(--accent-orange)]/40 border-dashed bg-[var(--accent-orange)]/5 px-6 py-4">
<code className="font-bold font-mono text-2xl text-foreground tracking-widest">
{deviceCode?.userCode}
</code>
<Button
variant="ghost"
size="icon"
onClick={handleCopy}
className="shrink-0"
>
{copied ? (
<Check className="h-4 w-4 text-green-600" />
) : (
<Copy className="h-4 w-4" />
)}
</Button>
</div>
<p className="text-center text-muted-foreground text-xs">
This dialog will close automatically once authentication completes.
</p>
{deviceCode?.verificationUri && (
<a
href={deviceCode.verificationUri}
target="_blank"
rel="noopener noreferrer"
className="flex items-center gap-1 text-[var(--accent-orange)] text-xs transition-colors hover:underline"
>
Open verification page
<ExternalLink className="h-3 w-3" />
</a>
)}
</div>
</DialogContent>
</Dialog>
)
}

View File

@@ -0,0 +1,57 @@
import { ArrowRight, Server, X } from 'lucide-react'
import { type FC, useState } from 'react'
import { useNavigate } from 'react-router'
import { Button } from '@/components/ui/button'
import { MCP_PROMO_BANNER_CLICKED_EVENT } from '@/lib/constants/analyticsEvents'
import { track } from '@/lib/metrics/track'
export const McpPromoBanner: FC = () => {
const [dismissed, setDismissed] = useState(false)
const navigate = useNavigate()
if (dismissed) return null
const handleClick = () => {
track(MCP_PROMO_BANNER_CLICKED_EVENT)
navigate('/settings/mcp')
}
return (
<div className="flex items-center gap-4 rounded-xl border border-border bg-card p-4 shadow-sm transition-all hover:shadow-md">
<div className="flex h-10 w-10 shrink-0 items-center justify-center rounded-lg bg-[var(--accent-orange)]/10">
<Server className="h-5 w-5 text-[var(--accent-orange)]" />
</div>
<div className="min-w-0 flex-1">
<p className="flex items-center gap-2 font-semibold text-sm">
Use BrowserOS with Claude Code, Cursor & more
<span className="text-[var(--accent-orange)] text-xs">
(66+ tools)
</span>
<span className="inline-flex items-center gap-1 rounded-full bg-[var(--accent-orange)]/10 px-2.5 py-1 font-semibold text-[var(--accent-orange)] text-xs">
<span className="h-1.5 w-1.5 rounded-full bg-[var(--accent-orange)]" />
New
</span>
</p>
<p className="text-muted-foreground text-xs">
Connect your favorite coding tools to BrowserOS as an MCP server
</p>
</div>
<Button
variant="outline"
size="sm"
onClick={handleClick}
className="shrink-0 border-[var(--accent-orange)] bg-[var(--accent-orange)]/10 text-[var(--accent-orange)] hover:bg-[var(--accent-orange)]/20 hover:text-[var(--accent-orange)]"
>
Set up
<ArrowRight className="ml-1 h-3 w-3" />
</Button>
<button
type="button"
onClick={() => setDismissed(true)}
className="shrink-0 rounded-sm p-1 text-muted-foreground opacity-50 transition-opacity hover:opacity-100"
>
<X className="h-3.5 w-3.5" />
</button>
</div>
)
}

View File

@@ -1,6 +1,13 @@
import { zodResolver } from '@hookform/resolvers/zod'
import { CheckCircle2, ExternalLink, Loader2, XCircle } from 'lucide-react'
import { type FC, useEffect, useState } from 'react'
import {
CheckCircle2,
ChevronDown,
ExternalLink,
Loader2,
SearchIcon,
XCircle,
} from 'lucide-react'
import { type FC, useEffect, useRef, useState } from 'react'
import { useForm } from 'react-hook-form'
import { z } from 'zod/v3'
import { Button } from '@/components/ui/button'
@@ -47,7 +54,12 @@ import {
import { type TestResult, testProvider } from '@/lib/llm-providers/testProvider'
import type { LlmProviderConfig, ProviderType } from '@/lib/llm-providers/types'
import { track } from '@/lib/metrics/track'
import { getModelContextLength, getModelOptions } from './models'
import { cn } from '@/lib/utils'
import {
getModelContextLength,
getModelsForProvider,
type ModelInfo,
} from './models'
const providerTypeEnum = z.enum([
'moonshot',
@@ -61,6 +73,9 @@ const providerTypeEnum = z.enum([
'lmstudio',
'bedrock',
'browseros',
'chatgpt-pro',
'github-copilot',
'qwen-code',
])
/**
@@ -84,6 +99,9 @@ export const providerFormSchema = z
secretAccessKey: z.string().optional(),
region: z.string().optional(),
sessionToken: z.string().optional(),
// ChatGPT Pro (Codex)
reasoningEffort: z.enum(['none', 'low', 'medium', 'high']).optional(),
reasoningSummary: z.enum(['auto', 'concise', 'detailed']).optional(),
})
.superRefine((data, ctx) => {
// Azure: require either resourceName or baseUrl
@@ -127,6 +145,14 @@ export const providerFormSchema = z
})
}
}
// OAuth providers: no credentials needed (server-managed)
else if (
data.type === 'chatgpt-pro' ||
data.type === 'github-copilot' ||
data.type === 'qwen-code'
) {
// No validation needed — OAuth tokens are on the server
}
// Other providers: require baseUrl
else if (!data.baseUrl) {
ctx.addIssue({
@@ -149,6 +175,107 @@ export const providerFormSchema = z
*/
export type ProviderFormValues = z.infer<typeof providerFormSchema>
function formatContextWindow(tokens: number): string {
if (tokens >= 1000000)
return `${(tokens / 1000000).toFixed(tokens % 1000000 === 0 ? 0 : 1)}M`
if (tokens >= 1000) return `${Math.round(tokens / 1000)}K`
return `${tokens}`
}
function ModelPickerList({
models,
selectedModelId,
onSelect,
onCustomSubmit,
onClose,
}: {
models: ModelInfo[]
selectedModelId: string
onSelect: (modelId: string) => void
onCustomSubmit: (modelId: string) => void
onClose: () => void
}) {
const [search, setSearch] = useState('')
const inputRef = useRef<HTMLInputElement>(null)
const containerRef = useRef<HTMLDivElement>(null)
useEffect(() => {
inputRef.current?.focus()
}, [])
useEffect(() => {
const handleClickOutside = (e: MouseEvent) => {
if (
containerRef.current &&
!containerRef.current.contains(e.target as Node)
) {
onClose()
}
}
document.addEventListener('mousedown', handleClickOutside)
return () => document.removeEventListener('mousedown', handleClickOutside)
}, [onClose])
const query = search.toLowerCase()
const filtered = query
? models.filter((m) => m.modelId.toLowerCase().includes(query))
: models
const handleKeyDown = (e: React.KeyboardEvent) => {
if (e.key === 'Enter' && search) {
e.preventDefault()
onCustomSubmit(search)
}
if (e.key === 'Escape') {
onClose()
}
}
return (
<div ref={containerRef} className="rounded-md border">
<div className="flex items-center gap-2 border-b px-3">
<SearchIcon className="h-4 w-4 shrink-0 text-muted-foreground opacity-50" />
<input
ref={inputRef}
type="text"
value={search}
onChange={(e) => setSearch(e.target.value)}
onKeyDown={handleKeyDown}
placeholder="Search or type a custom model ID..."
className="flex h-9 w-full bg-transparent py-2 text-sm outline-none placeholder:text-muted-foreground"
/>
</div>
<div className="max-h-[200px] overflow-y-auto">
{filtered.length > 0 ? (
filtered.map((model) => {
const isSelected = selectedModelId === model.modelId
return (
<button
key={model.modelId}
type="button"
onClick={() => onSelect(model.modelId)}
className={cn(
'flex w-full items-center justify-between px-3 py-2 text-left text-sm transition-colors hover:bg-accent',
isSelected && 'bg-accent font-medium',
)}
>
<span className="truncate">{model.modelId}</span>
<span className="ml-2 shrink-0 rounded-md bg-muted px-1.5 py-0.5 font-mono text-[10px] text-muted-foreground">
{formatContextWindow(model.contextLength)}
</span>
</button>
)
})
) : (
<div className="px-3 py-6 text-center text-muted-foreground text-sm">
No models match. Press Enter to use &quot;{search}&quot;
</div>
)}
</div>
</div>
)
}
/**
* Props for NewProviderDialog
* @public
@@ -174,14 +301,19 @@ export const NewProviderDialog: FC<NewProviderDialogProps> = ({
initialValues,
onSave,
}) => {
const [isCustomModel, setIsCustomModel] = useState(false)
const [isTesting, setIsTesting] = useState(false)
const [testResult, setTestResult] = useState<TestResult | null>(null)
const [modelListOpen, setModelListOpen] = useState(false)
const { supports } = useCapabilities()
const { baseUrl: agentServerUrl } = useAgentServerUrl()
const kimiLaunch = useKimiLaunch()
const filteredProviderTypeOptions = providerTypeOptions.filter((opt) => {
if (opt.value === 'chatgpt-pro')
return supports(Feature.CHATGPT_PRO_SUPPORT)
if (opt.value === 'github-copilot')
return supports(Feature.GITHUB_COPILOT_SUPPORT)
if (opt.value === 'qwen-code') return supports(Feature.QWEN_CODE_SUPPORT)
if (opt.value === 'moonshot')
return kimiLaunch || initialValues?.type === 'moonshot'
if (opt.value === 'openai-compatible') {
@@ -209,6 +341,8 @@ export const NewProviderDialog: FC<NewProviderDialogProps> = ({
secretAccessKey: initialValues?.secretAccessKey || '',
region: initialValues?.region || '',
sessionToken: initialValues?.sessionToken || '',
reasoningEffort: initialValues?.reasoningEffort || 'high',
reasoningSummary: initialValues?.reasoningSummary || 'auto',
},
})
@@ -240,8 +374,7 @@ export const NewProviderDialog: FC<NewProviderDialogProps> = ({
watchedSessionToken,
])
// Get model options for current provider type
const modelOptions = getModelOptions(watchedType as ProviderType)
const modelInfoList = getModelsForProvider(watchedType as ProviderType)
// Handle provider type change (user-initiated via Select)
const handleTypeChange = (newType: ProviderType) => {
@@ -251,14 +384,13 @@ export const NewProviderDialog: FC<NewProviderDialogProps> = ({
form.setValue('baseUrl', defaultUrl)
}
form.setValue('modelId', '')
setIsCustomModel(false)
}
// Auto-fill context window when model changes (only for new providers)
useEffect(() => {
if (initialValues?.id) return
if (watchedModelId && watchedModelId !== 'custom') {
if (watchedModelId) {
const contextLength = getModelContextLength(
watchedType as ProviderType,
watchedModelId,
@@ -269,17 +401,6 @@ export const NewProviderDialog: FC<NewProviderDialogProps> = ({
}
}, [watchedModelId, watchedType, form, initialValues?.id])
// Handle model selection (including custom option)
const handleModelChange = (value: string) => {
if (value === 'custom') {
setIsCustomModel(true)
form.setValue('modelId', '')
} else {
setIsCustomModel(false)
form.setValue('modelId', value)
}
}
// Reset form when initialValues change
useEffect(() => {
if (initialValues) {
@@ -301,8 +422,9 @@ export const NewProviderDialog: FC<NewProviderDialogProps> = ({
secretAccessKey: initialValues.secretAccessKey || '',
region: initialValues.region || '',
sessionToken: initialValues.sessionToken || '',
reasoningEffort: initialValues.reasoningEffort || 'high',
reasoningSummary: initialValues.reasoningSummary || 'auto',
})
setIsCustomModel(false)
}
}, [initialValues, form])
@@ -326,8 +448,9 @@ export const NewProviderDialog: FC<NewProviderDialogProps> = ({
secretAccessKey: '',
region: '',
sessionToken: '',
reasoningEffort: 'high',
reasoningSummary: 'auto',
})
setIsCustomModel(false)
}
// Clear test result when dialog opens/closes
setTestResult(null)
@@ -363,6 +486,14 @@ export const NewProviderDialog: FC<NewProviderDialogProps> = ({
const canTest = (): boolean => {
if (!watchedModelId) return false
// OAuth providers: always testable (server has the OAuth token)
if (
watchedType === 'chatgpt-pro' ||
watchedType === 'github-copilot' ||
watchedType === 'qwen-code'
)
return true
if (watchedType === 'azure') {
return !!(watchedResourceName || watchedBaseUrl) && !!watchedApiKey
}
@@ -444,6 +575,85 @@ export const NewProviderDialog: FC<NewProviderDialogProps> = ({
}
const renderProviderSpecificFields = () => {
// OAuth-only providers (no API key needed)
if (watchedType === 'github-copilot' || watchedType === 'qwen-code') {
const name = watchedType === 'github-copilot' ? 'GitHub' : 'Qwen Code'
return (
<div className="rounded-lg border border-green-200 bg-green-50 p-3 text-green-700 text-sm dark:border-green-800 dark:bg-green-950 dark:text-green-300">
Credentials are managed via {name} OAuth. No API key needed.
</div>
)
}
// ChatGPT Pro: OAuth credentials + Codex reasoning settings
if (watchedType === 'chatgpt-pro') {
return (
<>
<div className="rounded-lg border border-green-200 bg-green-50 p-3 text-green-700 text-sm dark:border-green-800 dark:bg-green-950 dark:text-green-300">
Credentials are managed via OAuth. No API key needed.
</div>
<div className="grid gap-4 sm:grid-cols-2">
<FormField
control={form.control}
name="reasoningEffort"
render={({ field }) => (
<FormItem>
<FormLabel>Reasoning Effort</FormLabel>
<Select
onValueChange={field.onChange}
value={field.value || 'high'}
>
<FormControl>
<SelectTrigger className="w-full">
<SelectValue />
</SelectTrigger>
</FormControl>
<SelectContent>
<SelectItem value="none">None</SelectItem>
<SelectItem value="low">Low</SelectItem>
<SelectItem value="medium">Medium</SelectItem>
<SelectItem value="high">High</SelectItem>
</SelectContent>
</Select>
<FormDescription>
How much the model thinks before responding
</FormDescription>
<FormMessage />
</FormItem>
)}
/>
<FormField
control={form.control}
name="reasoningSummary"
render={({ field }) => (
<FormItem>
<FormLabel>Reasoning Summary</FormLabel>
<Select
onValueChange={field.onChange}
value={field.value || 'auto'}
>
<FormControl>
<SelectTrigger className="w-full">
<SelectValue />
</SelectTrigger>
</FormControl>
<SelectContent>
<SelectItem value="auto">Auto</SelectItem>
<SelectItem value="concise">Concise</SelectItem>
<SelectItem value="detailed">Detailed</SelectItem>
</SelectContent>
</Select>
<FormDescription>
Detail level of visible thinking steps
</FormDescription>
<FormMessage />
</FormItem>
)}
/>
</div>
</>
)
}
if (watchedType === 'azure') {
return (
<>
@@ -699,52 +909,51 @@ export const NewProviderDialog: FC<NewProviderDialogProps> = ({
control={form.control}
name="modelId"
render={({ field }) => (
<FormItem>
<FormItem className="flex flex-col">
<FormLabel>Model *</FormLabel>
{isCustomModel || modelOptions.length === 1 ? (
<>
<FormControl>
<Input
placeholder={
watchedType === 'azure'
? 'Enter your deployment name'
: watchedType === 'bedrock'
? 'e.g., anthropic.claude-3-5-sonnet-20241022-v2:0'
: 'Enter custom model ID'
}
{...field}
/>
</FormControl>
{modelOptions.length > 1 && (
<Button
type="button"
variant="link"
size="sm"
className="h-auto p-0 text-xs"
onClick={() => setIsCustomModel(false)}
>
Back to model list
</Button>
)}
</>
{modelInfoList.length === 0 ? (
<FormControl>
<Input
placeholder={
watchedType === 'azure'
? 'Enter your deployment name'
: watchedType === 'bedrock'
? 'e.g., anthropic.claude-3-5-sonnet-20241022-v2:0'
: 'Enter model ID'
}
{...field}
/>
</FormControl>
) : modelListOpen ? (
<ModelPickerList
models={modelInfoList}
selectedModelId={field.value}
onSelect={(modelId) => {
form.setValue('modelId', modelId)
setModelListOpen(false)
}}
onCustomSubmit={(modelId) => {
form.setValue('modelId', modelId)
setModelListOpen(false)
}}
onClose={() => setModelListOpen(false)}
/>
) : (
<Select
onValueChange={handleModelChange}
value={field.value}
<button
type="button"
onClick={() => setModelListOpen(true)}
className={cn(
'flex h-9 w-full items-center justify-between rounded-md border border-input bg-transparent px-3 py-1 text-sm shadow-xs',
field.value
? 'text-foreground'
: 'text-muted-foreground',
)}
>
<FormControl>
<SelectTrigger className="w-full">
<SelectValue placeholder="Select a model" />
</SelectTrigger>
</FormControl>
<SelectContent>
{modelOptions.map((modelId) => (
<SelectItem key={modelId} value={modelId}>
{modelId === 'custom' ? '+ Custom model' : modelId}
</SelectItem>
))}
</SelectContent>
</Select>
<span className="truncate">
{field.value || 'Select a model...'}
</span>
<ChevronDown className="ml-2 h-4 w-4 shrink-0 opacity-50" />
</button>
)}
<FormMessage />
</FormItem>

View File

@@ -103,8 +103,10 @@ export const ProviderCard: FC<ProviderCardProps> = ({
for better performance.
</>
)
) : (
) : provider.baseUrl ? (
`${provider.modelId}${provider.baseUrl}`
) : (
provider.modelId
)}
</p>
</div>

View File

@@ -7,12 +7,14 @@ import { cn } from '@/lib/utils'
interface ProviderTemplateCardProps {
template: ProviderTemplate
highlighted?: boolean
isNew?: boolean
onUseTemplate: (template: ProviderTemplate) => void
}
export const ProviderTemplateCard: FC<ProviderTemplateCardProps> = ({
template,
highlighted = false,
isNew = false,
onUseTemplate,
}) => {
return (
@@ -20,12 +22,19 @@ export const ProviderTemplateCard: FC<ProviderTemplateCardProps> = ({
type="button"
onClick={() => onUseTemplate(template)}
className={cn(
'group flex w-full items-center gap-3 rounded-lg border bg-background p-4 text-left transition-all hover:border-[var(--accent-orange)] hover:shadow-md',
'group relative flex w-full items-center gap-3 rounded-lg border bg-background p-4 text-left transition-all hover:border-[var(--accent-orange)] hover:shadow-md',
highlighted
? 'border-orange-300/80 bg-orange-50/30 shadow-sm ring-1 ring-orange-300/45 dark:bg-orange-500/5'
: 'border-border',
: isNew
? 'border-2 border-[var(--accent-orange)]/50'
: 'border-border',
)}
>
{isNew && (
<span className="absolute -top-2 left-3 rounded-full bg-[var(--accent-orange)] px-2 py-0.5 font-semibold text-[9px] text-white uppercase tracking-wider">
New
</span>
)}
<div className="flex min-w-0 flex-1 items-center gap-3">
<ProviderIcon
type={template.id}

View File

@@ -26,6 +26,11 @@ export const ProviderTemplatesSection: FC<ProviderTemplatesSectionProps> = ({
const kimiLaunch = useKimiLaunch()
const filteredTemplates = providerTemplates.filter((template) => {
if (template.id === 'chatgpt-pro')
return supports(Feature.CHATGPT_PRO_SUPPORT)
if (template.id === 'github-copilot')
return supports(Feature.GITHUB_COPILOT_SUPPORT)
if (template.id === 'qwen-code') return supports(Feature.QWEN_CODE_SUPPORT)
if (template.id === 'moonshot') return kimiLaunch
if (template.id === 'openai-compatible') {
return supports(Feature.OPENAI_COMPATIBLE_SUPPORT)
@@ -53,14 +58,21 @@ export const ProviderTemplatesSection: FC<ProviderTemplatesSectionProps> = ({
<CollapsibleContent>
<div className="mt-4 grid gap-3 sm:grid-cols-2 lg:grid-cols-3">
{filteredTemplates.map((template) => (
<ProviderTemplateCard
key={template.id}
template={template}
highlighted={template.id === 'moonshot'}
onUseTemplate={onUseTemplate}
/>
))}
{filteredTemplates.map((template) => {
const isNew =
template.id === 'chatgpt-pro' ||
template.id === 'github-copilot' ||
template.id === 'qwen-code'
return (
<ProviderTemplateCard
key={template.id}
template={template}
highlighted={template.id === 'moonshot'}
isNew={isNew}
onUseTemplate={onUseTemplate}
/>
)
})}
</div>
</CollapsibleContent>
</div>

View File

@@ -1,116 +1,56 @@
import {
getModelsDevModels,
type ModelsDevModel,
} from '@/lib/llm-providers/models-dev'
import type { ProviderType } from '@/lib/llm-providers/types'
/**
* Model information with context length
*/
export interface ModelInfo {
modelId: string
contextLength: number
supportsImages?: boolean
supportsReasoning?: boolean
supportsToolCall?: boolean
}
/**
* Models data organized by provider type (matches backend AIProvider enum)
*/
export interface ModelsData {
anthropic: ModelInfo[]
openai: ModelInfo[]
'openai-compatible': ModelInfo[]
google: ModelInfo[]
openrouter: ModelInfo[]
azure: ModelInfo[]
ollama: ModelInfo[]
lmstudio: ModelInfo[]
bedrock: ModelInfo[]
browseros: ModelInfo[]
moonshot: ModelInfo[]
}
/**
* Available models per provider with context lengths
* Based on: https://github.com/browseros-ai/BrowserOS-agent/blob/main/src/options/data/models.ts
*/
export const MODELS_DATA: ModelsData = {
moonshot: [{ modelId: 'kimi-k2.5', contextLength: 200000 }],
anthropic: [
{ modelId: 'claude-opus-4-5-20251101', contextLength: 200000 },
{ modelId: 'claude-haiku-4-5-20251001', contextLength: 200000 },
{ modelId: 'claude-sonnet-4-5-20250929', contextLength: 200000 },
{ modelId: 'claude-sonnet-4-20250514', contextLength: 200000 },
{ modelId: 'claude-opus-4-20250514', contextLength: 200000 },
{ modelId: 'claude-3-7-sonnet-20250219', contextLength: 200000 },
{ modelId: 'claude-3-5-haiku-20241022', contextLength: 200000 },
],
openai: [
{ modelId: 'gpt-5.2', contextLength: 200000 },
{ modelId: 'gpt-5.2-pro', contextLength: 200000 },
{ modelId: 'gpt-5', contextLength: 200000 },
{ modelId: 'gpt-5-mini', contextLength: 200000 },
{ modelId: 'gpt-5-nano', contextLength: 200000 },
{ modelId: 'gpt-4.1', contextLength: 200000 },
{ modelId: 'gpt-4.1-mini', contextLength: 200000 },
{ modelId: 'o4-mini', contextLength: 200000 },
{ modelId: 'o3-mini', contextLength: 200000 },
{ modelId: 'gpt-4o', contextLength: 128000 },
{ modelId: 'gpt-4o-mini', contextLength: 128000 },
],
'openai-compatible': [],
google: [
{ modelId: 'gemini-3-pro-preview', contextLength: 1048576 },
{ modelId: 'gemini-3-flash-preview', contextLength: 1048576 },
{ modelId: 'gemini-2.5-flash', contextLength: 1048576 },
{ modelId: 'gemini-2.5-pro', contextLength: 1048576 },
],
openrouter: [
{ modelId: 'google/gemini-3-pro-preview', contextLength: 1048576 },
{ modelId: 'google/gemini-3-flash-preview', contextLength: 1048576 },
{ modelId: 'google/gemini-2.5-flash', contextLength: 1048576 },
{ modelId: 'anthropic/claude-opus-4.5', contextLength: 200000 },
{ modelId: 'anthropic/claude-haiku-4.5', contextLength: 200000 },
{ modelId: 'anthropic/claude-sonnet-4.5', contextLength: 200000 },
{ modelId: 'anthropic/claude-sonnet-4', contextLength: 200000 },
{ modelId: 'anthropic/claude-3.7-sonnet', contextLength: 200000 },
{ modelId: 'openai/gpt-4o', contextLength: 128000 },
{ modelId: 'openai/gpt-oss-120b', contextLength: 128000 },
{ modelId: 'openai/gpt-oss-20b', contextLength: 128000 },
{ modelId: 'qwen/qwen3-14b', contextLength: 131072 },
{ modelId: 'qwen/qwen3-8b', contextLength: 131072 },
],
azure: [],
ollama: [
{ modelId: 'qwen3:4b', contextLength: 262144 },
{ modelId: 'qwen3:8b', contextLength: 40960 },
{ modelId: 'qwen3:14b', contextLength: 40960 },
{ modelId: 'gpt-oss:20b', contextLength: 128000 },
{ modelId: 'gpt-oss:120b', contextLength: 128000 },
],
lmstudio: [
{ modelId: 'openai/gpt-oss-20b', contextLength: 128000 },
{ modelId: 'openai/gpt-oss-120b', contextLength: 128000 },
{ modelId: 'qwen/qwen3-vl-8b', contextLength: 131072 },
],
bedrock: [],
const CUSTOM_PROVIDER_MODELS: Partial<Record<ProviderType, ModelInfo[]>> = {
browseros: [{ modelId: 'browseros-auto', contextLength: 200000 }],
'openai-compatible': [],
ollama: [],
'chatgpt-pro': [
{ modelId: 'gpt-5.4', contextLength: 400000 },
{ modelId: 'gpt-5.3-codex', contextLength: 400000 },
{ modelId: 'gpt-5.2-codex', contextLength: 400000 },
{ modelId: 'gpt-5.2', contextLength: 200000 },
{ modelId: 'gpt-5.1-codex', contextLength: 400000 },
{ modelId: 'gpt-5.1-codex-max', contextLength: 400000 },
{ modelId: 'gpt-5.1-codex-mini', contextLength: 400000 },
{ modelId: 'gpt-5.1', contextLength: 200000 },
],
'qwen-code': [
{ modelId: 'coder-model', contextLength: 1000000 },
{ modelId: 'qwen3-coder-plus', contextLength: 1000000 },
{ modelId: 'qwen3-coder-flash', contextLength: 1000000 },
{ modelId: 'qwen3.5-plus', contextLength: 1000000 },
],
}
function fromModelsDevModel(m: ModelsDevModel): ModelInfo {
return {
modelId: m.id,
contextLength: m.contextWindow,
supportsImages: m.supportsImages,
supportsReasoning: m.supportsReasoning,
supportsToolCall: m.supportsToolCall,
}
}
/**
* Get models for a specific provider type
*/
export function getModelsForProvider(providerType: ProviderType): ModelInfo[] {
return MODELS_DATA[providerType] || []
const custom = CUSTOM_PROVIDER_MODELS[providerType]
if (custom !== undefined) return custom
return getModelsDevModels(providerType).map(fromModelsDevModel)
}
/**
* Get model options for select dropdown (model IDs + custom option)
*/
export function getModelOptions(providerType: ProviderType): string[] {
const models = getModelsForProvider(providerType)
const modelIds = models.map((m) => m.modelId)
return modelIds.length > 0 ? [...modelIds, 'custom'] : ['custom']
}
/**
* Get context length for a specific model
*/
export function getModelContextLength(
providerType: ProviderType,
modelId: string,
@@ -119,14 +59,3 @@ export function getModelContextLength(
const model = models.find((m) => m.modelId === modelId)
return model?.contextLength
}
/**
* Check if model ID is a custom (user-entered) value
*/
export function isCustomModel(
providerType: ProviderType,
modelId: string,
): boolean {
const models = getModelsForProvider(providerType)
return !models.some((m) => m.modelId === modelId)
}

View File

@@ -156,6 +156,7 @@ export const ConnectMCP: FC = () => {
})
if (response.success) {
removeServer(id)
mutateUserIntegrations()
} else {
failedToRemoveMcp(name, 'Success not returned from server')
}

View File

@@ -1,4 +1,4 @@
import useSWR from 'swr'
import { useQuery } from '@tanstack/react-query'
import { useAgentServerUrl } from '@/lib/browseros/useBrowserOSProviders'
interface UserMCPIntegrationsList {
@@ -9,7 +9,11 @@ interface UserMCPIntegrationsList {
count: number
}
const getUserMCPIntegrations = async ([hostUrl]: [hostUrl: string]) => {
export const INTEGRATIONS_QUERY_KEY = 'klavis-user-integrations'
const getUserMCPIntegrations = async (
hostUrl: string,
): Promise<UserMCPIntegrationsList> => {
const response = await fetch(`${hostUrl}/klavis/user-integrations`)
const data = (await response.json()) as UserMCPIntegrationsList
return data
@@ -18,12 +22,19 @@ const getUserMCPIntegrations = async ([hostUrl]: [hostUrl: string]) => {
export const useGetUserMCPIntegrations = () => {
const { baseUrl: agentServerUrl } = useAgentServerUrl()
return useSWR(
agentServerUrl ? [agentServerUrl, 'klavis/user-integrations'] : null,
getUserMCPIntegrations,
{
keepPreviousData: true,
revalidateOnFocus: true,
},
)
const query = useQuery({
queryKey: [INTEGRATIONS_QUERY_KEY, agentServerUrl],
// biome-ignore lint/style/noNonNullAssertion: guarded by enabled
queryFn: () => getUserMCPIntegrations(agentServerUrl!),
enabled: !!agentServerUrl,
refetchOnWindowFocus: true,
})
return {
data: query.data,
isLoading: query.isLoading,
isFetching: query.isFetching,
isSuccess: query.isSuccess,
mutate: query.refetch,
}
}

View File

@@ -259,11 +259,23 @@ export const CreateGraph: FC = () => {
})
const onClickTest = async () => {
const backgroundWindow = await chrome.windows.create({
url: 'chrome://newtab',
focused: true,
type: 'normal',
})
let backgroundWindow: chrome.windows.Window | undefined
try {
backgroundWindow = await chrome.windows.create({
url: 'chrome://newtab',
focused: true,
type: 'normal',
})
} catch {
// Fallback when no window context is available (e.g. all windows closed)
const tab = await chrome.tabs.create({
url: 'chrome://newtab',
active: true,
})
if (tab.windowId) {
backgroundWindow = await chrome.windows.get(tab.windowId)
}
}
sendMessage({
text: 'Run a test of the graph you just created.',

View File

@@ -4,8 +4,8 @@ import { MessageResponse } from '@/components/ai-elements/message'
import { Button } from '@/components/ui/button'
import { Textarea } from '@/components/ui/textarea'
import { cn } from '@/lib/utils'
import { useVoiceInput } from '@/lib/voice/useVoiceInput'
import type { Message } from './useSurveyChat'
import { useVoiceInput } from './useVoiceInput'
import { VoiceInputButton } from './VoiceInputButton'
interface Props {
@@ -81,6 +81,7 @@ export const Chat: FC<Props> = ({
}, [messagesLength])
// Insert transcript into input when transcription completes
// biome-ignore lint/correctness/useExhaustiveDependencies: only trigger on transcript/transcribing change
useEffect(() => {
if (voice.transcript && !voice.isTranscribing) {
setInput((prev) => {
@@ -89,7 +90,7 @@ export const Chat: FC<Props> = ({
})
voice.clearTranscript()
}
}, [voice])
}, [voice.transcript, voice.isTranscribing])
const handleSubmit = (e: FormEvent) => {
e.preventDefault()

View File

@@ -17,11 +17,8 @@ export const SettingsSidebarLayout: FC = () => {
useEffect(() => {
track(SETTINGS_PAGE_VIEWED_EVENT, { page: location.pathname })
}, [location.pathname])
useEffect(() => {
setMobileOpen(false)
}, [])
}, [location.pathname])
if (isMobile) {
return (

View File

@@ -7,8 +7,6 @@ import { Button } from '@/components/ui/button'
import { Sheet, SheetContent } from '@/components/ui/sheet'
import { ShortcutsDialog } from '@/entrypoints/newtab/index/ShortcutsDialog'
import { useIsMobile } from '@/hooks/use-mobile'
import { SETTINGS_PAGE_VIEWED_EVENT } from '@/lib/constants/analyticsEvents'
import { track } from '@/lib/metrics/track'
import { RpcClientProvider } from '@/lib/rpc/RpcClientProvider'
const COLLAPSE_DELAY = 150
@@ -25,10 +23,6 @@ export const SidebarLayout: FC = () => {
setShortcutsDialogOpen(true)
}, [])
useEffect(() => {
track(SETTINGS_PAGE_VIEWED_EVENT, { page: location.pathname })
}, [location.pathname])
useEffect(() => {
setMobileOpen(false)
}, [])
@@ -103,11 +97,17 @@ export const SidebarLayout: FC = () => {
</div>
{/* Main content - full width, centered */}
<main className="min-h-screen overflow-y-auto">
<div className="mx-auto max-w-4xl px-4 py-8 sm:px-6 lg:px-8">
{location.pathname === '/home/chat' ? (
<main className="relative h-dvh overflow-hidden">
<Outlet />
</div>
</main>
</main>
) : (
<main className="min-h-screen overflow-y-auto">
<div className="mx-auto max-w-4xl px-4 py-8 sm:px-6 lg:px-8">
<Outlet />
</div>
</main>
)}
</div>
<ShortcutsDialog
open={shortcutsDialogOpen}

View File

@@ -1,5 +1,5 @@
import { useQueryClient } from '@tanstack/react-query'
import localforage from 'localforage'
import { clear } from 'idb-keyval'
import { Loader2 } from 'lucide-react'
import type { FC } from 'react'
import { useEffect } from 'react'
@@ -10,6 +10,7 @@ import {
CardHeader,
CardTitle,
} from '@/components/ui/card'
import { resetIdentity } from '@/lib/analytics/identify'
import { signOut } from '@/lib/auth/auth-client'
import { providersStorage } from '@/lib/llm-providers/storage'
import { scheduledJobStorage } from '@/lib/schedules/scheduleStorage'
@@ -24,8 +25,9 @@ export const LogoutPage: FC = () => {
await providersStorage.removeValue()
await scheduledJobStorage.removeValue()
queryClient.clear()
await localforage.clear()
await clear()
resetIdentity()
await signOut()
navigate('/home', { replace: true })
}

View File

@@ -1,31 +1,40 @@
import { Check, Copy, ExternalLink, Globe, Server } from 'lucide-react'
import { type FC, useState } from 'react'
import {
Check,
Copy,
ExternalLink,
Loader2,
RefreshCw,
Server,
} from 'lucide-react'
import { type FC, useCallback, useState } from 'react'
import { toast } from 'sonner'
import { Button } from '@/components/ui/button'
import { MCP_SERVER_RESTARTED_EVENT } from '@/lib/constants/analyticsEvents'
import { sendServerMessage } from '@/lib/messaging/server/serverMessages'
import { track } from '@/lib/metrics/track'
interface MCPServerHeaderProps {
serverUrl: string | null
isLoading: boolean
error: string | null
title?: string
description?: string
remoteAccessEnabled?: boolean
onServerRestart?: () => void
}
const DOCS_URL = 'https://docs.browseros.com/features/use-with-claude-code'
const HEALTH_CHECK_TIMEOUT_MS = 60_000
const HEALTH_CHECK_INTERVAL_MS = 2_000
export const MCPServerHeader: FC<MCPServerHeaderProps> = ({
serverUrl,
isLoading,
error,
title = 'BrowserOS MCP Server',
description = 'Connect BrowserOS to MCP clients like claude code, gemini and others.',
remoteAccessEnabled = false,
onServerRestart,
}) => {
const [isCopied, setIsCopied] = useState(false)
const [isRestarting, setIsRestarting] = useState(false)
const handleCopy = async () => {
if (!serverUrl) return
try {
await navigator.clipboard.writeText(serverUrl)
setIsCopied(true)
@@ -35,6 +44,57 @@ export const MCPServerHeader: FC<MCPServerHeaderProps> = ({
}
}
const checkServerHealth = useCallback(async (): Promise<boolean> => {
try {
const result = await sendServerMessage('checkHealth', undefined)
return result.healthy
} catch {
return false
}
}, [])
const handleRestart = async () => {
setIsRestarting(true)
try {
const { getBrowserOSAdapter } = await import('@/lib/browseros/adapter')
const { BROWSEROS_PREFS } = await import('@/lib/browseros/prefs')
const adapter = getBrowserOSAdapter()
await adapter.setPref(BROWSEROS_PREFS.RESTART_SERVER, true)
const startTime = Date.now()
const waitForHealth = (): Promise<boolean> =>
new Promise((resolve) => {
const check = async () => {
if (Date.now() - startTime >= HEALTH_CHECK_TIMEOUT_MS) {
resolve(false)
return
}
if (await checkServerHealth()) {
resolve(true)
return
}
setTimeout(check, HEALTH_CHECK_INTERVAL_MS)
}
setTimeout(check, HEALTH_CHECK_INTERVAL_MS)
})
const healthy = await waitForHealth()
if (healthy) {
track(MCP_SERVER_RESTARTED_EVENT)
toast.success('Server restarted successfully')
onServerRestart?.()
} else {
toast.error('Server did not respond. Try restarting the browser.')
}
} catch (err) {
toast.error(
err instanceof Error ? err.message : 'Failed to restart server',
)
} finally {
setIsRestarting(false)
}
}
return (
<div className="rounded-xl border border-border bg-card p-6 shadow-sm transition-all hover:shadow-md">
<div className="flex items-start gap-4">
@@ -43,18 +103,21 @@ export const MCPServerHeader: FC<MCPServerHeaderProps> = ({
</div>
<div className="flex-1">
<div className="mb-1 flex items-center justify-between">
<h2 className="font-semibold text-xl">{title}</h2>
<h2 className="font-semibold text-xl">BrowserOS MCP Server</h2>
<a
href={DOCS_URL}
target="_blank"
rel="noopener noreferrer"
className="flex items-center gap-1 text-muted-foreground text-sm transition-colors hover:text-[var(--accent-orange)]"
>
Setup a client
Docs
<ExternalLink className="h-3.5 w-3.5" />
</a>
</div>
<p className="mb-6 text-muted-foreground text-sm">{description}</p>
<p className="mb-6 text-muted-foreground text-sm">
Connect BrowserOS to MCP clients like Claude Code, Gemini CLI and
others.
</p>
<div className="flex flex-col gap-2 sm:flex-row sm:items-center">
<span className="whitespace-nowrap font-medium text-sm">
@@ -76,6 +139,7 @@ export const MCPServerHeader: FC<MCPServerHeaderProps> = ({
onClick={handleCopy}
disabled={!serverUrl || isLoading}
className="shrink-0"
title="Copy URL"
>
{isCopied ? (
<Check className="h-4 w-4 text-green-600" />
@@ -83,19 +147,22 @@ export const MCPServerHeader: FC<MCPServerHeaderProps> = ({
<Copy className="h-4 w-4" />
)}
</Button>
<Button
variant="outline"
size="icon"
onClick={handleRestart}
disabled={isLoading || isRestarting}
className="shrink-0"
title="Restart server"
>
{isRestarting ? (
<Loader2 className="h-4 w-4 animate-spin" />
) : (
<RefreshCw className="h-4 w-4" />
)}
</Button>
</div>
</div>
{remoteAccessEnabled && serverUrl && !isLoading && (
<div className="mt-3 flex items-start gap-2 rounded-lg bg-muted/50 px-3 py-2">
<Globe className="mt-0.5 h-3.5 w-3.5 shrink-0 text-muted-foreground" />
<p className="text-muted-foreground text-xs">
External access is enabled. To connect from another device,
replace <span className="font-mono">127.0.0.1</span> with this
machine's IP address.
</p>
</div>
)}
</div>
</div>
</div>

View File

@@ -4,7 +4,7 @@ import type { McpTool } from '@/lib/mcp/client'
import { sendServerMessage } from '@/lib/messaging/server/serverMessages'
import { MCPServerHeader } from './MCPServerHeader'
import { MCPToolsSection } from './MCPToolsSection'
import { ServerSettingsCard } from './ServerSettingsCard'
import { QuickSetupSection } from './QuickSetupSection'
/** @public */
export const MCPSettingsPage: FC = () => {
@@ -12,8 +12,6 @@ export const MCPSettingsPage: FC = () => {
const [urlLoading, setUrlLoading] = useState(true)
const [urlError, setUrlError] = useState<string | null>(null)
const [remoteAccessEnabled, setRemoteAccessEnabled] = useState(false)
const [tools, setTools] = useState<McpTool[]>([])
const [toolsLoading, setToolsLoading] = useState(false)
const [toolsError, setToolsError] = useState<string | null>(null)
@@ -82,13 +80,10 @@ export const MCPSettingsPage: FC = () => {
serverUrl={serverUrl}
isLoading={urlLoading}
error={urlError}
remoteAccessEnabled={remoteAccessEnabled}
onServerRestart={loadServerUrlAndTools}
/>
<ServerSettingsCard
onServerRestart={loadServerUrlAndTools}
onRemoteAccessChange={setRemoteAccessEnabled}
/>
<QuickSetupSection serverUrl={serverUrl} />
<MCPToolsSection
tools={tools}

View File

@@ -0,0 +1,162 @@
import { Check, Copy, Terminal } from 'lucide-react'
import { type FC, useState } from 'react'
import { Button } from '@/components/ui/button'
import { Tabs, TabsContent, TabsList, TabsTrigger } from '@/components/ui/tabs'
interface QuickSetupSectionProps {
serverUrl: string | null
}
interface ClientConfig {
id: string
name: string
type: 'command' | 'json'
getSnippet: (url: string) => string
fileName?: string
}
const clients: ClientConfig[] = [
{
id: 'claude-code',
name: 'Claude Code',
type: 'command',
getSnippet: (url) =>
`claude mcp add --transport http browseros ${url} --scope user`,
},
{
id: 'gemini-cli',
name: 'Gemini CLI',
type: 'command',
getSnippet: (url) =>
`gemini mcp add local-server ${url} --transport http --scope user`,
},
{
id: 'codex',
name: 'Codex',
type: 'command',
getSnippet: (url) => `codex mcp add browseros ${url}`,
},
{
id: 'claude-desktop',
name: 'Claude Desktop',
type: 'json',
fileName: 'claude_desktop_config.json',
getSnippet: (url) =>
JSON.stringify(
{
mcpServers: {
browserOS: {
command: 'npx',
args: ['mcp-remote', url],
},
},
},
null,
2,
),
},
{
id: 'openclaw',
name: 'OpenClaw',
type: 'json',
fileName: 'openclaw.json',
getSnippet: (url) =>
JSON.stringify(
{
mcpServers: {
browseros: { url },
},
},
null,
2,
),
},
]
const CopyButton: FC<{ text: string }> = ({ text }) => {
const [copied, setCopied] = useState(false)
const handleCopy = async () => {
try {
await navigator.clipboard.writeText(text)
setCopied(true)
setTimeout(() => setCopied(false), 2000)
} catch {
// Clipboard API failed
}
}
return (
<Button
variant="ghost"
size="icon-sm"
onClick={handleCopy}
className="shrink-0 text-muted-foreground hover:text-foreground"
>
{copied ? (
<Check className="h-3.5 w-3.5 text-green-600" />
) : (
<Copy className="h-3.5 w-3.5" />
)}
</Button>
)
}
export const QuickSetupSection: FC<QuickSetupSectionProps> = ({
serverUrl,
}) => {
if (!serverUrl) return null
return (
<div className="rounded-xl border border-border bg-card p-6 shadow-sm transition-all hover:shadow-md">
<div className="flex items-start gap-4">
<div className="flex h-12 w-12 shrink-0 items-center justify-center rounded-xl bg-[var(--accent-orange)]/10">
<Terminal className="h-6 w-6 text-[var(--accent-orange)]" />
</div>
<div className="flex-1">
<h2 className="mb-1 font-semibold text-xl">Quick Setup</h2>
<p className="mb-4 text-muted-foreground text-sm">
Copy and run the command for your tool
</p>
<Tabs defaultValue="claude-code">
<TabsList className="mb-3 flex-wrap">
{clients.map((client) => (
<TabsTrigger key={client.id} value={client.id}>
{client.name}
</TabsTrigger>
))}
</TabsList>
{clients.map((client) => {
const snippet = client.getSnippet(serverUrl)
return (
<TabsContent key={client.id} value={client.id}>
<div className="space-y-3">
{client.fileName && (
<p className="text-muted-foreground text-xs">
Add to{' '}
<code className="rounded bg-muted px-1 py-0.5 font-mono text-xs">
{client.fileName}
</code>
</p>
)}
<div className="flex items-start gap-2 rounded-lg border border-border bg-background px-3 py-2.5">
<pre className="flex-1 overflow-x-auto whitespace-pre-wrap break-all font-mono text-xs">
{client.type === 'command' && (
<span className="mr-1 text-muted-foreground">$</span>
)}
{snippet}
</pre>
<CopyButton text={snippet} />
</div>
</div>
</TabsContent>
)
})}
</Tabs>
</div>
</div>
</div>
)
}

View File

@@ -1,8 +1,9 @@
import { zodResolver } from '@hookform/resolvers/zod'
import { ChevronDown } from 'lucide-react'
import { ChevronDown, Loader2, Sparkles, Undo2 } from 'lucide-react'
import type { FC } from 'react'
import { useEffect, useState } from 'react'
import { useEffect, useRef, useState } from 'react'
import { useForm } from 'react-hook-form'
import { toast } from 'sonner'
import { z } from 'zod/v3'
import { ChatProviderSelector } from '@/components/chat/ChatProviderSelector'
import type { Provider } from '@/components/chat/chatComponentTypes'
@@ -34,12 +35,15 @@ import {
SelectValue,
} from '@/components/ui/select'
import { Textarea } from '@/components/ui/textarea'
import { SCHEDULED_TASK_PROMPT_REFINED_EVENT } from '@/lib/constants/analyticsEvents'
import { BrowserOSIcon, ProviderIcon } from '@/lib/llm-providers/providerIcons'
import {
defaultProviderIdStorage,
providersStorage,
} from '@/lib/llm-providers/storage'
import type { LlmProviderConfig, ProviderType } from '@/lib/llm-providers/types'
import { track } from '@/lib/metrics/track'
import { refinePrompt } from '@/lib/schedules/refine-prompt'
import type { ScheduledJob } from './types'
const formSchema = z
@@ -109,6 +113,11 @@ export const NewScheduledTaskDialog: FC<NewScheduledTaskDialogProps> = ({
const scheduleType = form.watch('scheduleType')
const selectedProviderId = form.watch('providerId')
const queryValue = form.watch('query')
const [isRefining, setIsRefining] = useState(false)
const originalPromptRef = useRef<string | null>(null)
const refineRequestIdRef = useRef(0)
const isProgrammaticChange = useRef(false)
// Load providers from storage
useEffect(() => {
@@ -124,6 +133,9 @@ export const NewScheduledTaskDialog: FC<NewScheduledTaskDialogProps> = ({
useEffect(() => {
if (open) {
refineRequestIdRef.current++
originalPromptRef.current = null
setIsRefining(false)
if (initialValues) {
form.reset({
name: initialValues.name,
@@ -168,6 +180,60 @@ export const NewScheduledTaskDialog: FC<NewScheduledTaskDialogProps> = ({
type: p.type,
}))
// Replace textarea content via execCommand so the browser's native undo
// stack (Cmd+Z / Ctrl+Z) records the change. Falls back to form.setValue
// if the textarea element can't be found.
const setQueryWithUndo = (value: string) => {
const textarea = document.querySelector(
'textarea[name="query"]',
) as HTMLTextAreaElement
if (textarea) {
isProgrammaticChange.current = true
textarea.focus()
textarea.select()
document.execCommand('insertText', false, value)
isProgrammaticChange.current = false
} else {
form.setValue('query', value)
}
}
const handleRefinePrompt = async () => {
const currentQuery = form.getValues('query').trim()
const currentName = form.getValues('name').trim()
if (!currentQuery) return
const requestId = ++refineRequestIdRef.current
setIsRefining(true)
originalPromptRef.current = currentQuery
try {
const refined = await refinePrompt({
prompt: currentQuery,
name: currentName || 'Untitled Task',
providerId: form.getValues('providerId'),
})
if (requestId !== refineRequestIdRef.current) return
setQueryWithUndo(refined)
track(SCHEDULED_TASK_PROMPT_REFINED_EVENT)
} catch {
if (requestId !== refineRequestIdRef.current) return
toast.error('Failed to rewrite prompt. Please try again.')
originalPromptRef.current = null
} finally {
if (requestId === refineRequestIdRef.current) {
setIsRefining(false)
}
}
}
const handleUndoRefine = () => {
if (originalPromptRef.current !== null) {
setQueryWithUndo(originalPromptRef.current)
originalPromptRef.current = null
}
}
const onSubmit = (values: FormValues) => {
onSave({
name: values.name.trim(),
@@ -181,6 +247,7 @@ export const NewScheduledTaskDialog: FC<NewScheduledTaskDialogProps> = ({
enabled: values.enabled,
})
form.reset()
originalPromptRef.current = null
onOpenChange(false)
}
@@ -218,17 +285,54 @@ export const NewScheduledTaskDialog: FC<NewScheduledTaskDialogProps> = ({
name="query"
render={({ field }) => (
<FormItem>
<FormLabel>Prompt</FormLabel>
<div className="flex items-center justify-between">
<FormLabel>Prompt</FormLabel>
<Button
type="button"
variant="ghost"
size="sm"
className="h-auto gap-1 px-2 py-1 text-muted-foreground text-xs"
disabled={!queryValue?.trim() || isRefining}
onClick={handleRefinePrompt}
>
{isRefining ? (
<Loader2 className="h-3 w-3 animate-spin" />
) : (
<Sparkles className="h-3 w-3" />
)}
{isRefining ? 'Rewriting...' : 'Rewrite with AI'}
</Button>
</div>
<FormControl>
<Textarea
placeholder="What should the agent do? e.g., Check my email and summarize important messages"
className="min-h-[100px] resize-none"
{...field}
onChange={(e) => {
field.onChange(e)
if (
!isProgrammaticChange.current &&
originalPromptRef.current !== null
) {
originalPromptRef.current = null
}
}}
/>
</FormControl>
<FormDescription>
The instruction that will be sent to the agent
</FormDescription>
{!isRefining && originalPromptRef.current !== null ? (
<button
type="button"
className="flex items-center gap-1 text-muted-foreground text-xs hover:text-foreground"
onClick={handleUndoRefine}
>
<Undo2 className="h-3 w-3" />
Undo rewrite
</button>
) : (
<FormDescription>
The instruction that will be sent to the agent
</FormDescription>
)}
<FormMessage />
</FormItem>
)}

View File

@@ -22,9 +22,7 @@ import {
SCHEDULED_TASK_TOGGLED_EVENT,
SCHEDULED_TASK_VIEW_RESULTS_EVENT,
} from '@/lib/constants/analyticsEvents'
import { useGraphqlMutation } from '@/lib/graphql/useGraphqlMutation'
import { track } from '@/lib/metrics/track'
import { DeleteScheduledJobDocument } from '@/lib/schedules/graphql/syncSchedulesDocument'
import {
scheduledJobRunStorage,
useScheduledJobRuns,
@@ -46,8 +44,6 @@ export const ScheduledTasksPage: FC = () => {
useScheduledJobs()
const { jobRuns, cancelJobRun } = useScheduledJobRuns()
const deleteRemoteJobMutation = useGraphqlMutation(DeleteScheduledJobDocument)
const [activeTab, setActiveTab] = useState<string | null>(null)
const [isDialogOpen, setIsDialogOpen] = useState(false)
const [editingJob, setEditingJob] = useState<ScheduledJob | null>(null)
@@ -102,7 +98,6 @@ export const ScheduledTasksPage: FC = () => {
const confirmDelete = async () => {
if (deleteJobId) {
await removeJob(deleteJobId)
deleteRemoteJobMutation.mutate({ rowId: deleteJobId })
setDeleteJobId(null)
track(SCHEDULED_TASK_DELETED_EVENT)
}

View File

@@ -1,5 +1,6 @@
import { AlertCircle, Pencil, Plus, Trash2, Wand2 } from 'lucide-react'
import { AlertCircle, Eye, Pencil, Plus, Trash2, Wand2 } from 'lucide-react'
import { type FC, useEffect, useState } from 'react'
import Markdown from 'react-markdown'
import { toast } from 'sonner'
import {
AlertDialog,
@@ -108,23 +109,19 @@ export const SkillsPage: FC = () => {
) : null}
{!isLoading && !error && skills.length > 0 ? (
<div className="grid grid-cols-1 gap-3 sm:grid-cols-2 xl:grid-cols-3">
{skills.map((skill) => (
<SkillCard
key={skill.id}
skill={skill}
onEdit={() => handleEdit(skill)}
onDelete={() => setSkillToDelete(skill)}
onToggle={(enabled) => handleToggle(skill, enabled)}
/>
))}
</div>
<SkillSections
skills={skills}
onEdit={handleEdit}
onDelete={(skill) => setSkillToDelete(skill)}
onToggle={handleToggle}
/>
) : null}
<SkillDialog
open={isDialogOpen}
onOpenChange={setIsDialogOpen}
editingSkill={editingSkill}
readOnly={editingSkill?.builtIn}
onSave={async (data) => {
try {
if (editingSkill) {
@@ -251,6 +248,50 @@ const EmptyState: FC<{ onCreateClick: () => void }> = ({ onCreateClick }) => (
</Card>
)
const SkillGrid: FC<{ children: React.ReactNode }> = ({ children }) => (
<div className="grid grid-cols-1 gap-3 sm:grid-cols-2 xl:grid-cols-3">
{children}
</div>
)
const SkillSections: FC<{
skills: SkillMeta[]
onEdit: (skill: SkillMeta) => void
onDelete: (skill: SkillMeta) => void
onToggle: (skill: SkillMeta, enabled: boolean) => void
}> = ({ skills, onEdit, onDelete, onToggle }) => {
const userSkills = skills.filter((s) => !s.builtIn)
const builtInSkills = skills.filter((s) => s.builtIn)
const renderCard = (skill: SkillMeta) => (
<SkillCard
key={skill.id}
skill={skill}
onEdit={() => onEdit(skill)}
onDelete={() => onDelete(skill)}
onToggle={(enabled) => onToggle(skill, enabled)}
/>
)
return (
<div className="space-y-6">
{userSkills.length > 0 ? (
<div className="space-y-3">
<h3 className="font-semibold text-sm">My Skills</h3>
<SkillGrid>{userSkills.map(renderCard)}</SkillGrid>
</div>
) : null}
{builtInSkills.length > 0 ? (
<div className="space-y-3">
<h3 className="font-semibold text-sm">BrowserOS Skills</h3>
<SkillGrid>{builtInSkills.map(renderCard)}</SkillGrid>
</div>
) : null}
</div>
)
}
const SkillCard: FC<{
skill: SkillMeta
onEdit: () => void
@@ -260,7 +301,14 @@ const SkillCard: FC<{
<Card className="h-full py-0 shadow-sm">
<CardContent className="flex h-full flex-col p-4">
<div className="flex items-start justify-between gap-3">
<h2 className="font-semibold text-sm leading-5">{skill.name}</h2>
<div className="flex items-center gap-2">
<h2 className="font-semibold text-sm leading-5">{skill.name}</h2>
{skill.builtIn ? (
<Badge variant="secondary" className="px-1.5 py-0 text-[10px]">
Built-in
</Badge>
) : null}
</div>
<Switch
checked={skill.enabled}
onCheckedChange={onToggle}
@@ -281,18 +329,29 @@ const SkillCard: FC<{
onClick={onEdit}
className="-ml-2 h-7 px-2 text-muted-foreground hover:bg-transparent hover:text-foreground"
>
<Pencil className="size-3.5" />
Edit
</Button>
<Button
variant="ghost"
size="icon-sm"
onClick={onDelete}
className="size-7 text-muted-foreground hover:bg-transparent hover:text-destructive"
aria-label={`Delete ${skill.name}`}
>
<Trash2 className="size-4" />
{skill.builtIn ? (
<>
<Eye className="size-3.5" />
View
</>
) : (
<>
<Pencil className="size-3.5" />
Edit
</>
)}
</Button>
{!skill.builtIn ? (
<Button
variant="ghost"
size="icon-sm"
onClick={onDelete}
className="size-7 text-muted-foreground hover:bg-transparent hover:text-destructive"
aria-label={`Delete ${skill.name}`}
>
<Trash2 className="size-4" />
</Button>
) : null}
</div>
</CardContent>
</Card>
@@ -302,12 +361,13 @@ const SkillDialog: FC<{
open: boolean
onOpenChange: (open: boolean) => void
editingSkill: SkillDetail | null
readOnly?: boolean
onSave: (data: {
name: string
description: string
content: string
}) => Promise<void>
}> = ({ open, onOpenChange, editingSkill, onSave }) => {
}> = ({ open, onOpenChange, editingSkill, readOnly, onSave }) => {
const [name, setName] = useState('')
const [description, setDescription] = useState('')
const [content, setContent] = useState('')
@@ -354,12 +414,18 @@ const SkillDialog: FC<{
<DialogContent className="flex max-h-[90vh] flex-col gap-0 overflow-hidden p-0 sm:max-w-5xl">
<DialogHeader className="border-b px-6 py-5">
<DialogTitle>
{editingSkill ? 'Edit Skill' : 'Create Skill'}
{readOnly
? 'View Skill'
: editingSkill
? 'Edit Skill'
: 'Create Skill'}
</DialogTitle>
<DialogDescription>
{editingSkill
? 'Refine when the agent should use this skill and how it should execute it.'
: 'Define a reusable instruction set your agent can apply when a request matches.'}
{readOnly
? 'This skill is managed by BrowserOS and updated automatically.'
: editingSkill
? 'Refine when the agent should use this skill and how it should execute it.'
: 'Define a reusable instruction set your agent can apply when a request matches.'}
</DialogDescription>
</DialogHeader>
@@ -373,6 +439,7 @@ const SkillDialog: FC<{
value={name}
onChange={(event) => setName(event.target.value)}
maxLength={100}
readOnly={readOnly}
/>
<p className="text-muted-foreground text-xs leading-5">
Keep it short and recognizable in the skills list.
@@ -388,19 +455,22 @@ const SkillDialog: FC<{
onChange={(event) => setDescription(event.target.value)}
maxLength={500}
className="min-h-28 resize-none bg-background"
readOnly={readOnly}
/>
<p className="text-muted-foreground text-xs leading-5">
This is the trigger summary the agent uses to pick the skill.
</p>
</div>
<div className="mt-auto rounded-lg border border-border/60 border-dashed bg-muted/30 px-3 py-2.5">
<p className="font-medium text-muted-foreground text-xs">Tip</p>
<ul className="mt-1.5 list-disc space-y-1 pl-4 text-muted-foreground text-xs leading-5">
<li>List the ordered steps the agent should follow.</li>
<li>Close with the output or formatting you expect back.</li>
</ul>
</div>
{!readOnly ? (
<div className="mt-auto rounded-lg border border-border/60 border-dashed bg-muted/30 px-3 py-2.5">
<p className="font-medium text-muted-foreground text-xs">Tip</p>
<ul className="mt-1.5 list-disc space-y-1 pl-4 text-muted-foreground text-xs leading-5">
<li>List the ordered steps the agent should follow.</li>
<li>Close with the output or formatting you expect back.</li>
</ul>
</div>
) : null}
</div>
<div className="flex min-h-0 flex-col px-6 py-5">
@@ -411,36 +481,52 @@ const SkillDialog: FC<{
</Badge>
</div>
<MarkdownEditor
id="skill-content"
value={content}
onChange={setContent}
onKeyDown={handleContentKeyDown}
placeholder="Write instructions for the agent. Use markdown for structure."
className="mt-4 min-h-[320px] flex-1 overflow-y-auto text-sm"
/>
{readOnly ? (
<div className="prose prose-sm dark:prose-invert mt-4 min-h-[320px] max-w-none flex-1 overflow-y-auto rounded-md border p-4 text-sm">
<Markdown>{content}</Markdown>
</div>
) : (
<MarkdownEditor
id="skill-content"
value={content}
onChange={setContent}
onKeyDown={handleContentKeyDown}
placeholder="Write instructions for the agent. Use markdown for structure."
className="mt-4 min-h-[320px] flex-1 overflow-y-auto text-sm"
/>
)}
</div>
</div>
<div className="flex flex-col gap-3 border-t px-6 py-4 sm:flex-row sm:items-center sm:justify-between">
<p className="text-muted-foreground text-xs">
Saved locally and available to your agent immediately.
{readOnly
? 'This skill is managed by BrowserOS and updated automatically.'
: 'Saved locally and available to your agent immediately.'}
</p>
<div className="flex flex-col-reverse gap-2 sm:flex-row">
<Button
variant="outline"
onClick={() => onOpenChange(false)}
disabled={saving}
>
Cancel
</Button>
<Button onClick={handleSubmit} disabled={!isValid || saving}>
{saving
? 'Saving...'
: editingSkill
? 'Update Skill'
: 'Create Skill'}
</Button>
{readOnly ? (
<Button variant="outline" onClick={() => onOpenChange(false)}>
Close
</Button>
) : (
<>
<Button
variant="outline"
onClick={() => onOpenChange(false)}
disabled={saving}
>
Cancel
</Button>
<Button onClick={handleSubmit} disabled={!isValid || saving}>
{saving
? 'Saving...'
: editingSkill
? 'Update Skill'
: 'Create Skill'}
</Button>
</>
)}
</div>
</div>
</DialogContent>

View File

@@ -7,6 +7,7 @@ export type SkillMeta = {
description: string
location: string
enabled: boolean
builtIn: boolean
}
export type SkillDetail = SkillMeta & {

View File

@@ -0,0 +1,147 @@
import { AlertCircle, Clock, Coins, CreditCard, Zap } from 'lucide-react'
import type { FC } from 'react'
import { Button } from '@/components/ui/button'
import {
getCreditBarColor,
getCreditTextColor,
} from '@/lib/credits/credit-colors'
import { useCredits } from '@/lib/credits/useCredits'
import { BrowserOSIcon } from '@/lib/llm-providers/providerIcons'
import { cn } from '@/lib/utils'
export const UsagePage: FC = () => {
const { data, isLoading, error } = useCredits()
if (isLoading) {
return (
<div className="flex items-center justify-center p-12 text-muted-foreground text-sm">
Loading usage data...
</div>
)
}
if (error) {
return (
<div className="space-y-6 p-6">
<div className="flex items-center gap-4 rounded-xl border p-5">
<BrowserOSIcon size={40} />
<div>
<h2 className="font-semibold text-lg">Usage & Billing</h2>
<p className="text-muted-foreground text-sm">
Monitor your BrowserOS AI credit usage
</p>
</div>
</div>
<div className="flex flex-col items-center gap-3 rounded-xl border border-destructive/30 bg-destructive/5 p-8">
<AlertCircle className="h-6 w-6 text-muted-foreground" />
<p className="text-muted-foreground text-sm">
Unable to load credit information
</p>
</div>
</div>
)
}
const credits = data?.credits ?? 0
const total = data?.dailyLimit ?? 100
const percentage = Math.min((credits / total) * 100, 100)
return (
<div className="space-y-6 p-6">
<div className="flex items-center gap-4 rounded-xl border p-5">
<BrowserOSIcon size={40} />
<div>
<h2 className="font-semibold text-lg">Usage & Billing</h2>
<p className="text-muted-foreground text-sm">
Monitor your BrowserOS AI credit usage
</p>
</div>
</div>
<div className="rounded-xl border p-5">
<div className="mb-4 flex items-center justify-between">
<div className="flex items-center gap-2">
<Coins className="h-5 w-5 text-muted-foreground" />
<span className="font-semibold text-sm">Daily Credits</span>
</div>
<span
className={cn('font-bold text-2xl', getCreditTextColor(credits))}
>
{credits}
<span className="ml-1 font-normal text-muted-foreground text-sm">
/ {total}
</span>
</span>
</div>
<div className="mb-5 h-2.5 w-full overflow-hidden rounded-full bg-muted">
<div
className={cn(
'h-full rounded-full transition-all duration-500',
getCreditBarColor(credits),
)}
style={{ width: `${percentage}%` }}
/>
</div>
<div className="grid grid-cols-2 gap-3">
<div className="flex items-center gap-2.5 rounded-lg bg-muted/50 px-3 py-2.5">
<Clock className="h-4 w-4 shrink-0 text-muted-foreground" />
<div>
<p className="font-medium text-xs">Resets daily</p>
<p className="text-muted-foreground text-xs">Midnight UTC</p>
</div>
</div>
<div className="flex items-center gap-2.5 rounded-lg bg-muted/50 px-3 py-2.5">
<Zap className="h-4 w-4 shrink-0 text-muted-foreground" />
<div>
<p className="font-medium text-xs">Credits used today</p>
<p className="text-muted-foreground text-xs">
{total - credits} of {total}
</p>
</div>
</div>
</div>
</div>
<div className="rounded-xl border p-5">
<div className="flex items-center gap-3">
<CreditCard className="h-5 w-5 text-muted-foreground" />
<div>
<p className="flex items-center gap-2 font-semibold text-sm">
Need more credits?
<span className="rounded-full bg-muted px-2 py-0.5 font-medium text-[10px] text-muted-foreground uppercase tracking-wide">
Coming soon
</span>
</p>
<p className="text-muted-foreground text-xs">
Additional credit packages will be available soon
</p>
</div>
</div>
</div>
<div className="rounded-xl border border-[var(--accent-orange)]/30 bg-[var(--accent-orange)]/5 p-5">
<div className="flex items-center justify-between">
<div className="flex items-center gap-3">
<Zap className="h-5 w-5 text-[var(--accent-orange)]" />
<div>
<p className="font-semibold text-sm">Want unlimited usage?</p>
<p className="text-muted-foreground text-xs">
Add your own LLM provider no credit limits
</p>
</div>
</div>
<Button
variant="outline"
size="sm"
className="border-[var(--accent-orange)] bg-[var(--accent-orange)]/10 text-[var(--accent-orange)] hover:bg-[var(--accent-orange)]/20"
asChild
>
<a href="/app.html#/settings/ai">Add Provider</a>
</Button>
</div>
</div>
</div>
)
}

View File

@@ -101,11 +101,23 @@ export const useRunWorkflow = () => {
setMessages([])
setWasCancelled(false)
const backgroundWindow = await chrome.windows.create({
url: 'chrome://newtab',
focused: true,
type: 'normal',
})
let backgroundWindow: chrome.windows.Window | undefined
try {
backgroundWindow = await chrome.windows.create({
url: 'chrome://newtab',
focused: true,
type: 'normal',
})
} catch {
// Fallback when no window context is available (e.g. all windows closed)
const tab = await chrome.tabs.create({
url: 'chrome://newtab',
active: true,
})
if (tab.windowId) {
backgroundWindow = await chrome.windows.get(tab.windowId)
}
}
sendMessage({
text: 'Run the workflow.',

View File

@@ -18,6 +18,7 @@ import {
syncScheduledJobs,
} from '@/lib/schedules/scheduleStorage'
import { searchActionsStorage } from '@/lib/search-actions/searchActionsStorage'
import { selectedTextStorage } from '@/lib/selected-text/selectedTextStorage'
import { stopAgentStorage } from '@/lib/stop-agent/stop-agent-storage'
import { scheduledJobRuns } from './scheduledJobRuns'
@@ -66,7 +67,12 @@ export default defineBackground(() => {
}
})
chrome.runtime.onMessage.addListener((message, sender) => {
chrome.runtime.onMessage.addListener((message, sender, sendResponse) => {
if (message?.type === 'get-tab-id') {
sendResponse({ tabId: sender.tab?.id })
return true
}
if (message?.type === 'AUTH_SUCCESS' && sender.tab?.id) {
const tabId = sender.tab.id
authRedirectPathStorage
@@ -93,6 +99,17 @@ export default defineBackground(() => {
}
})
// Clean up selected text storage when a tab is closed
chrome.tabs.onRemoved.addListener((tabId) => {
const key = String(tabId)
selectedTextStorage.getValue().then((map) => {
if (map[key]) {
const { [key]: _, ...rest } = map
selectedTextStorage.setValue(rest)
}
})
})
sessionStorage.watch(async (newSession) => {
if (newSession?.user?.id) {
try {

View File

@@ -5,12 +5,17 @@ import {
Folder,
Globe,
Layers,
Loader2,
Mic,
PlugZap,
Search,
Square,
X,
} from 'lucide-react'
import { AnimatePresence, motion } from 'motion/react'
import { useCallback, useEffect, useRef, useState } from 'react'
import { useNavigate } from 'react-router'
import { ChatProviderSelector } from '@/components/chat/ChatProviderSelector'
import { AppSelector } from '@/components/elements/AppSelector'
import {
GlowingBorder,
@@ -36,20 +41,26 @@ import {
import {
NEWTAB_AI_TRIGGERED_EVENT,
NEWTAB_APPS_OPENED_EVENT,
NEWTAB_CHAT_RESET_EVENT,
NEWTAB_CHAT_STARTED_EVENT,
NEWTAB_OPENED_EVENT,
NEWTAB_SEARCH_EXECUTED_EVENT,
NEWTAB_TAB_REMOVED_EVENT,
NEWTAB_TAB_TOGGLED_EVENT,
NEWTAB_TABS_OPENED_EVENT,
NEWTAB_VOICE_ERROR_EVENT,
NEWTAB_VOICE_RECORDING_STARTED_EVENT,
NEWTAB_VOICE_RECORDING_STOPPED_EVENT,
NEWTAB_VOICE_TRANSCRIPTION_COMPLETED_EVENT,
NEWTAB_WORKSPACE_OPENED_EVENT,
} from '@/lib/constants/analyticsEvents'
import { BrowserOSIcon, ProviderIcon } from '@/lib/llm-providers/providerIcons'
import type { ProviderType } from '@/lib/llm-providers/types'
import { useMcpServers } from '@/lib/mcp/mcpServerStorage'
import { useSyncRemoteIntegrations } from '@/lib/mcp/useSyncRemoteIntegrations'
import { openSidePanelWithSearch } from '@/lib/messaging/sidepanel/openSidepanelWithSearch'
import { track } from '@/lib/metrics/track'
import { cn } from '@/lib/utils'
import { useVoiceInput } from '@/lib/voice/useVoiceInput'
import { useWorkspace } from '@/lib/workspace/use-workspace'
import { ImportDataHint } from './ImportDataHint'
import type { SuggestionItem } from './lib/suggestions/types'
@@ -58,7 +69,6 @@ import {
useSuggestions,
} from './lib/suggestions/useSuggestions'
import { NewTabBranding } from './NewTabBranding'
import { NewTabChat } from './NewTabChat'
import { NewTabTip } from './NewTabTip'
import { ScheduleResults } from './ScheduleResults'
import { SearchSuggestions } from './SearchSuggestions'
@@ -78,13 +88,13 @@ interface MentionState {
*/
export const NewTab = () => {
const activeHint = useActiveHint()
const navigate = useNavigate()
const [inputValue, setInputValue] = useState('')
const [mounted, setMounted] = useState(false)
const inputRef = useRef<HTMLInputElement>(null)
const tabsDropdownRef = useRef<HTMLDivElement>(null)
const [selectedTabs, setSelectedTabs] = useState<chrome.tabs.Tab[]>([])
const [shortcutsDialogOpen, setShortcutsDialogOpen] = useState(false)
const [chatActive, setChatActive] = useState(false)
const [mentionState, setMentionState] = useState<MentionState>({
isOpen: false,
filterText: '',
@@ -92,12 +102,41 @@ export const NewTab = () => {
})
const { selectedFolder } = useWorkspace()
const { supports } = useCapabilities()
const { providers, selectedProvider, handleSelectProvider } =
useChatSessionContext()
const { servers: mcpServers } = useMcpServers()
const { data: userMCPIntegrations } = useGetUserMCPIntegrations()
useSyncRemoteIntegrations()
const { messages, sendMessage, setMode, resetConversation } =
useChatSessionContext()
const voice = useVoiceInput()
// Voice transcript → populate search input
// biome-ignore lint/correctness/useExhaustiveDependencies: only trigger on transcript/transcribing change
useEffect(() => {
if (voice.transcript && !voice.isTranscribing) {
setComboboxInputValue(voice.transcript)
track(NEWTAB_VOICE_TRANSCRIPTION_COMPLETED_EVENT)
voice.clearTranscript()
}
}, [voice.transcript, voice.isTranscribing])
useEffect(() => {
if (voice.error) {
track(NEWTAB_VOICE_ERROR_EVENT, { error: voice.error })
}
}, [voice.error])
const handleStartRecording = async () => {
const started = await voice.startRecording()
if (started) {
track(NEWTAB_VOICE_RECORDING_STARTED_EVENT)
}
}
const handleStopRecording = async () => {
await voice.stopRecording()
track(NEWTAB_VOICE_RECORDING_STOPPED_EVENT)
}
const connectedManagedServers = mcpServers.filter((s) => {
if (s.type !== 'managed' || !s.managedServerName) return false
@@ -275,17 +314,28 @@ export const NewTab = () => {
const startInlineChat = (
message: string,
mode: 'chat' | 'agent',
action?: ReturnType<
typeof createBrowserOSAction | typeof createAITabAction
>,
chatMode: 'chat' | 'agent',
aiTab?: { name: string; description: string },
) => {
track(NEWTAB_CHAT_STARTED_EVENT, { mode, tabs_count: selectedTabs.length })
setMode(mode)
setChatActive(true)
sendMessage({ text: message, action })
track(NEWTAB_CHAT_STARTED_EVENT, {
mode: chatMode,
tabs_count: selectedTabs.length,
})
const tabIds = selectedTabs
.map((t) => t.id)
.filter((id): id is number => id !== undefined)
reset()
setSelectedTabs([])
const params = new URLSearchParams({ q: message, mode: chatMode })
if (tabIds.length > 0) {
params.set('tabs', tabIds.join(','))
}
if (aiTab) {
params.set('actionType', 'ai-tab')
params.set('tabName', aiTab.name)
params.set('tabDescription', aiTab.description)
}
navigate(`/home/chat?${params.toString()}`)
}
const runSelectedAction = (item: SuggestionItem | undefined) => {
@@ -306,15 +356,18 @@ export const NewTab = () => {
mode: 'agent',
tabs_count: selectedTabs.length,
})
const action = createAITabAction({
name: item.name,
description: item.description,
tabs: selectedTabs,
})
const searchQuery = `${item.name}${item.description ? ` - ${item.description}` : ''}}`
if (supports(Feature.NEWTAB_CHAT_SUPPORT)) {
startInlineChat(searchQuery, 'agent', action)
startInlineChat(searchQuery, 'agent', {
name: item.name,
description: item.description,
})
} else {
const action = createAITabAction({
name: item.name,
description: item.description,
tabs: selectedTabs,
})
openSidePanelWithSearch('open', {
query: searchQuery,
mode: 'agent',
@@ -330,14 +383,14 @@ export const NewTab = () => {
mode: item.mode,
tabs_count: selectedTabs.length,
})
const action = createBrowserOSAction({
mode: item.mode,
message: item.message,
tabs: selectedTabs,
})
if (supports(Feature.NEWTAB_CHAT_SUPPORT)) {
startInlineChat(item.message, item.mode, action)
startInlineChat(item.message, item.mode)
} else {
const action = createBrowserOSAction({
mode: item.mode,
message: item.message,
tabs: selectedTabs,
})
openSidePanelWithSearch('open', {
query: item.message,
mode: item.mode,
@@ -351,12 +404,6 @@ export const NewTab = () => {
}
}
const handleBackToSearch = () => {
track(NEWTAB_CHAT_RESET_EVENT, { message_count: messages.length })
resetConversation()
setChatActive(false)
}
const isSuggestionsVisible =
!mentionState.isOpen &&
((isOpen && inputValue.length) ||
@@ -368,10 +415,6 @@ export const NewTab = () => {
track(NEWTAB_OPENED_EVENT)
}, [])
if (chatActive) {
return <NewTabChat onBackToSearch={handleBackToSearch} />
}
return (
<div className="pt-[max(25vh,16px)]">
{/* Main content */}
@@ -425,32 +468,89 @@ export const NewTab = () => {
anchorRef={inputRef}
side="bottom"
/>
<input
type="text"
placeholder={searchPlaceholder}
className="flex-1 border-none bg-transparent text-base text-foreground outline-none placeholder:text-muted-foreground"
{...getInputProps({
ref: inputRef,
onChange: (e) => handleInputChange(e.currentTarget.value),
onKeyDown: (e) => {
if (!mentionStateRef.current.isOpen) return
if (e.key === 'Tab') {
e.preventDefault()
closeMention()
}
},
})}
/>
{voice.isRecording ? (
<div className="flex min-h-[40px] flex-1 items-center justify-center gap-1.5">
{voice.audioLevels.map((level, i) => (
<div
key={i.toString()}
className="w-1.5 rounded-full bg-red-500 transition-all duration-75"
style={{
height: `${Math.max(6, Math.min(28, level * 0.7))}px`,
}}
/>
))}
</div>
) : (
<input
type="text"
placeholder={
voice.isTranscribing ? 'Transcribing...' : searchPlaceholder
}
disabled={voice.isTranscribing}
className="flex-1 border-none bg-transparent text-base text-foreground outline-none placeholder:text-muted-foreground disabled:opacity-60"
{...getInputProps({
ref: inputRef,
onChange: (e) => handleInputChange(e.currentTarget.value),
onKeyDown: (e) => {
if (!mentionStateRef.current.isOpen) return
if (e.key === 'Tab') {
e.preventDefault()
closeMention()
}
},
})}
/>
)}
<Button
onClick={handleSend}
size="icon"
className="h-10 w-10 flex-shrink-0 rounded-xl bg-primary text-primary-foreground hover:bg-primary/90"
>
<ArrowRight className="h-5 w-5" />
</Button>
<div className="flex items-center gap-1.5">
{voice.isRecording ? (
<Button
type="button"
size="icon"
onClick={handleStopRecording}
className="h-10 w-10 flex-shrink-0 rounded-xl bg-red-600 text-white hover:bg-red-700"
>
<Square className="h-4 w-4" />
</Button>
) : voice.isTranscribing ? (
<Button
type="button"
variant="ghost"
size="icon"
disabled
className="h-10 w-10 flex-shrink-0 rounded-xl"
>
<Loader2 className="h-5 w-5 animate-spin" />
</Button>
) : (
<Button
type="button"
variant="ghost"
size="icon"
onClick={handleStartRecording}
className="h-10 w-10 flex-shrink-0 rounded-xl text-muted-foreground transition-colors hover:text-foreground"
title="Voice input"
>
<Mic className="h-5 w-5" />
</Button>
)}
<Button
onClick={handleSend}
size="icon"
disabled={voice.isRecording || voice.isTranscribing}
className="h-10 w-10 flex-shrink-0 rounded-xl bg-primary text-primary-foreground hover:bg-primary/90"
>
<ArrowRight className="h-5 w-5" />
</Button>
</div>
</div>
{voice.error && (
<div className="px-5 pb-2 text-destructive text-xs">
{voice.error}
</div>
)}
<AnimatePresence>
{selectedTabs.length > 0 && (
<motion.div
@@ -524,6 +624,34 @@ export const NewTab = () => {
{mounted && (
<div className="flex items-center justify-between border-border/50 border-t px-5 py-3">
<div className="flex items-center gap-1">
{selectedProvider && (
<ChatProviderSelector
providers={providers}
selectedProvider={selectedProvider}
onSelectProvider={handleSelectProvider}
>
<Button
variant="ghost"
size="icon"
title={selectedProvider.name}
className={cn(
'h-8 w-8 rounded-lg transition-all',
'text-muted-foreground hover:bg-accent hover:text-accent-foreground',
'data-[state=open]:bg-accent',
)}
>
{selectedProvider.type === 'browseros' ? (
<BrowserOSIcon size={16} />
) : (
<ProviderIcon
type={selectedProvider.type as ProviderType}
size={16}
/>
)}
</Button>
</ChatProviderSelector>
)}
{supports(Feature.WORKSPACE_FOLDER_SUPPORT) && (
<WorkspaceSelector>
<Button

View File

@@ -1,35 +1,41 @@
import { Loader2 } from 'lucide-react'
import { type FC, useEffect, useState } from 'react'
import { type FC, useEffect, useRef } from 'react'
import { useSearchParams } from 'react-router'
import { ChatEmptyState } from '@/entrypoints/sidepanel/index/ChatEmptyState'
import { ChatError } from '@/entrypoints/sidepanel/index/ChatError'
import { ChatFooter } from '@/entrypoints/sidepanel/index/ChatFooter'
import { ChatHeader } from '@/entrypoints/sidepanel/index/ChatHeader'
import { ChatMessages } from '@/entrypoints/sidepanel/index/ChatMessages'
import type { ChatMode } from '@/entrypoints/sidepanel/index/chatTypes'
import { useChatSessionContext } from '@/entrypoints/sidepanel/layout/ChatSessionContext'
import { createBrowserOSAction } from '@/lib/chat-actions/types'
import {
createAITabAction,
createBrowserOSAction,
} from '@/lib/chat-actions/types'
import { useChatActions } from '@/lib/chat-actions/useChatActions'
import {
NEWTAB_AI_TRIGGERED_EVENT,
NEWTAB_CHAT_MODE_CHANGED_EVENT,
NEWTAB_CHAT_RESET_EVENT,
NEWTAB_CHAT_STOPPED_EVENT,
NEWTAB_CHAT_SUGGESTION_CLICKED_EVENT,
NEWTAB_TAB_REMOVED_EVENT,
NEWTAB_TAB_TOGGLED_EVENT,
NEWTAB_VOICE_ERROR_EVENT,
NEWTAB_VOICE_RECORDING_STARTED_EVENT,
NEWTAB_VOICE_RECORDING_STOPPED_EVENT,
NEWTAB_VOICE_TRANSCRIPTION_COMPLETED_EVENT,
} from '@/lib/constants/analyticsEvents'
import { track } from '@/lib/metrics/track'
import { NewTabChatHeader } from './NewTabChatHeader'
interface NewTabChatProps {
onBackToSearch: () => void
}
export const NewTabChat: FC = () => {
const [searchParams, setSearchParams] = useSearchParams()
const hasSentInitialRef = useRef(false)
export const NewTabChat: FC<NewTabChatProps> = ({ onBackToSearch }) => {
const {
mode,
setMode,
messages,
sendMessage,
status,
stop,
agentUrlError,
chatError,
getActionForMessage,
@@ -42,71 +48,80 @@ export const NewTabChat: FC<NewTabChatProps> = ({ onBackToSearch }) => {
selectedProvider,
handleSelectProvider,
resetConversation,
} = useChatSessionContext()
const [input, setInput] = useState('')
const [attachedTabs, setAttachedTabs] = useState<chrome.tabs.Tab[]>([])
const [mounted, setMounted] = useState(false)
input,
setInput,
attachedTabs,
mounted,
voiceState,
handleModeChange,
handleStop,
toggleTabSelection,
removeTab,
handleSubmit,
handleSuggestionClick,
} = useChatActions({
events: {
modeChanged: NEWTAB_CHAT_MODE_CHANGED_EVENT,
stopClicked: NEWTAB_CHAT_STOPPED_EVENT,
suggestionClicked: NEWTAB_CHAT_SUGGESTION_CLICKED_EVENT,
tabToggled: NEWTAB_TAB_TOGGLED_EVENT,
tabRemoved: NEWTAB_TAB_REMOVED_EVENT,
aiTriggered: NEWTAB_AI_TRIGGERED_EVENT,
voiceRecordingStarted: NEWTAB_VOICE_RECORDING_STARTED_EVENT,
voiceRecordingStopped: NEWTAB_VOICE_RECORDING_STOPPED_EVENT,
voiceTranscriptionCompleted: NEWTAB_VOICE_TRANSCRIPTION_COMPLETED_EVENT,
voiceError: NEWTAB_VOICE_ERROR_EVENT,
},
})
// Send the initial message from URL query params (from /home search bar).
// Guarded by ref to prevent double-fire in React Strict Mode.
// biome-ignore lint/correctness/useExhaustiveDependencies: must only run once on mount
useEffect(() => {
setMounted(true)
}, [])
if (hasSentInitialRef.current) return
const query = searchParams.get('q')
const chatMode = searchParams.get('mode')
const tabIdsParam = searchParams.get('tabs')
if (!query) return
const handleModeChange = (newMode: ChatMode) => {
track(NEWTAB_CHAT_MODE_CHANGED_EVENT, { from: mode, to: newMode })
setMode(newMode)
}
const handleStop = () => {
track(NEWTAB_CHAT_STOPPED_EVENT)
stop()
}
const toggleTabSelection = (tab: chrome.tabs.Tab) => {
setAttachedTabs((prev) => {
const isSelected = prev.some((t) => t.id === tab.id)
track(NEWTAB_TAB_TOGGLED_EVENT, {
action: isSelected ? 'removed' : 'added',
})
if (isSelected) {
return prev.filter((t) => t.id !== tab.id)
}
return [...prev, tab]
})
}
const removeTab = (tabId?: number) => {
track(NEWTAB_TAB_REMOVED_EVENT)
setAttachedTabs((prev) => prev.filter((t) => t.id !== tabId))
}
const executeMessage = (customMessageText?: string) => {
const messageText = customMessageText ? customMessageText : input.trim()
if (!messageText) return
if (attachedTabs.length) {
const action = createBrowserOSAction({
mode,
message: messageText,
tabs: attachedTabs,
})
sendMessage({ text: messageText, action })
} else {
sendMessage({ text: messageText })
hasSentInitialRef.current = true
if (chatMode === 'chat' || chatMode === 'agent') {
setMode(chatMode)
}
setInput('')
setAttachedTabs([])
}
setSearchParams({}, { replace: true })
const handleSubmit = (e: React.FormEvent) => {
e.preventDefault()
executeMessage()
}
const actionType = searchParams.get('actionType')
const tabName = searchParams.get('tabName')
const tabDescription = searchParams.get('tabDescription')
const handleSuggestionClick = (suggestion: string) => {
track(NEWTAB_CHAT_SUGGESTION_CLICKED_EVENT, { mode })
executeMessage(suggestion)
}
if (tabIdsParam) {
const tabIds = tabIdsParam.split(',').map(Number).filter(Boolean)
chrome.tabs.query({}).then((allTabs) => {
const matchedTabs = allTabs.filter(
(t) => t.id !== undefined && tabIds.includes(t.id),
)
if (matchedTabs.length > 0) {
const action =
actionType === 'ai-tab' && tabName
? createAITabAction({
name: tabName,
description: tabDescription ?? '',
tabs: matchedTabs,
})
: createBrowserOSAction({
mode: (chatMode as 'chat' | 'agent') ?? 'agent',
message: query,
tabs: matchedTabs,
})
sendMessage({ text: query, action })
} else {
sendMessage({ text: query })
}
})
} else {
sendMessage({ text: query })
}
}, [])
const handleNewConversation = () => {
track(NEWTAB_CHAT_RESET_EVENT, { message_count: messages.length })
@@ -116,17 +131,19 @@ export const NewTabChat: FC<NewTabChatProps> = ({ onBackToSearch }) => {
if (!selectedProvider) return null
return (
<div className="flex h-[calc(100vh-2rem)] flex-col">
<NewTabChatHeader
selectedProvider={selectedProvider}
providers={providers}
onSelectProvider={handleSelectProvider}
onNewConversation={handleNewConversation}
onBackToSearch={onBackToSearch}
hasMessages={messages.length > 0}
/>
<div className="absolute inset-0 flex flex-col overflow-hidden">
<div className="mx-auto w-full max-w-3xl">
<ChatHeader
selectedProvider={selectedProvider}
providers={providers}
onSelectProvider={handleSelectProvider}
onNewConversation={handleNewConversation}
hasMessages={messages.length > 0}
hideHistory
/>
</div>
<main className="mx-auto flex w-full max-w-3xl flex-1 flex-col space-y-4 overflow-y-auto px-4 pt-4">
<main className="styled-scrollbar [&_[data-streamdown='code-block']]:!max-w-full [&_[data-streamdown='code-block']]:!w-auto [&_[data-streamdown='table-wrapper']]:!max-w-full [&_[data-streamdown='table-wrapper']]:!w-auto mx-auto flex min-h-0 w-full max-w-3xl flex-1 flex-col space-y-4 overflow-y-auto overflow-x-hidden px-4 pt-4 [&_[data-streamdown='code-block']]:overflow-x-auto [&_[data-streamdown='table-wrapper']]:overflow-x-auto">
{isRestoringConversation ? (
<div className="flex flex-1 items-center justify-center">
<Loader2 className="h-6 w-6 animate-spin text-muted-foreground" />
@@ -152,11 +169,18 @@ export const NewTabChat: FC<NewTabChatProps> = ({ onBackToSearch }) => {
onDismissJtbdPopup={() => {}}
/>
)}
{agentUrlError && <ChatError error={agentUrlError} />}
{chatError && <ChatError error={chatError} />}
{agentUrlError && (
<ChatError
error={agentUrlError}
providerType={selectedProvider?.type}
/>
)}
{chatError && (
<ChatError error={chatError} providerType={selectedProvider?.type} />
)}
</main>
<div className="mx-auto w-full max-w-3xl px-4">
<div className="mx-auto w-full max-w-3xl flex-shrink-0 px-4 pb-2">
<ChatFooter
mode={mode}
onModeChange={handleModeChange}
@@ -168,6 +192,7 @@ export const NewTabChat: FC<NewTabChatProps> = ({ onBackToSearch }) => {
attachedTabs={attachedTabs}
onToggleTab={toggleTabSelection}
onRemoveTab={removeTab}
voice={voiceState}
/>
</div>
</div>

View File

@@ -1,78 +0,0 @@
import { ArrowLeft, Plus } from 'lucide-react'
import type { FC } from 'react'
import { ChatProviderSelector } from '@/components/chat/ChatProviderSelector'
import type { Provider } from '@/components/chat/chatComponentTypes'
import { BrowserOSIcon, ProviderIcon } from '@/lib/llm-providers/providerIcons'
import type { ProviderType } from '@/lib/llm-providers/types'
interface NewTabChatHeaderProps {
selectedProvider: Provider
providers: Provider[]
onSelectProvider: (provider: Provider) => void
onNewConversation: () => void
onBackToSearch: () => void
hasMessages: boolean
}
export const NewTabChatHeader: FC<NewTabChatHeaderProps> = ({
selectedProvider,
providers,
onSelectProvider,
onNewConversation,
onBackToSearch,
hasMessages,
}) => {
return (
<header className="flex items-center justify-between border-border/40 border-b bg-background/80 px-4 py-2.5 backdrop-blur-md">
<div className="flex items-center gap-2">
{/* Back to search */}
<button
type="button"
onClick={onBackToSearch}
className="cursor-pointer rounded-lg p-2 text-muted-foreground transition-colors hover:bg-muted/50 hover:text-foreground"
title="Back to search"
>
<ArrowLeft className="h-4 w-4" />
</button>
{/* Provider selector */}
<ChatProviderSelector
providers={providers}
selectedProvider={selectedProvider}
onSelectProvider={onSelectProvider}
>
<button
type="button"
className="group relative inline-flex cursor-pointer items-center gap-2 rounded-lg p-2 text-muted-foreground transition-colors hover:bg-muted/50 hover:text-foreground data-[state=open]:bg-accent"
title="Change AI Provider"
>
{selectedProvider.type === 'browseros' ? (
<BrowserOSIcon size={18} />
) : (
<ProviderIcon
type={selectedProvider.type as ProviderType}
size={18}
/>
)}
<span className="font-semibold text-base">
{selectedProvider.name}
</span>
</button>
</ChatProviderSelector>
</div>
<div className="flex items-center gap-1">
{hasMessages && (
<button
type="button"
onClick={onNewConversation}
className="cursor-pointer rounded-lg p-2 text-muted-foreground transition-colors hover:bg-muted/50 hover:text-foreground"
title="New conversation"
>
<Plus className="h-4 w-4" />
</button>
)}
</div>
</header>
)
}

View File

@@ -3,14 +3,19 @@ import { Outlet, useLocation } from 'react-router'
import { ChatSessionProvider } from '@/entrypoints/sidepanel/layout/ChatSessionContext'
import { NewTabFocusGrid } from './NewTabFocusGrid'
const HIDE_FOCUS_GRID_PATHS = new Set([
'/home/soul',
'/home/memory',
'/home/skills',
'/home/chat',
])
export const NewTabLayout: FC = () => {
const location = useLocation()
return (
<ChatSessionProvider origin="newtab">
{location.pathname !== '/home/soul' &&
location.pathname !== '/home/memory' &&
location.pathname !== '/home/skills' && <NewTabFocusGrid />}
{!HIDE_FOCUS_GRID_PATHS.has(location.pathname) && <NewTabFocusGrid />}
<Outlet />
</ChatSessionProvider>
)

View File

@@ -0,0 +1,42 @@
import { selectedTextStorage } from '@/lib/selected-text/selectedTextStorage'
const MAX_SELECTED_TEXT_LENGTH = 5000
export default defineContentScript({
matches: ['*://*/*'],
runAt: 'document_idle',
async main() {
const response = await chrome.runtime.sendMessage({ type: 'get-tab-id' })
const tabId: number | undefined = response?.tabId
if (!tabId) return
const key = String(tabId)
document.addEventListener('mouseup', () => {
const text = window.getSelection()?.toString().trim()
if (text && text.length > 0) {
selectedTextStorage.getValue().then((map) => {
selectedTextStorage.setValue({
...map,
[key]: {
text: text.slice(0, MAX_SELECTED_TEXT_LENGTH),
pageUrl: window.location.href,
pageTitle: document.title,
tabId,
timestamp: Date.now(),
},
})
})
} else {
// User clicked without selecting — clear this tab's entry only
selectedTextStorage.getValue().then((map) => {
if (map[key]) {
const { [key]: _, ...rest } = map
selectedTextStorage.setValue(rest)
}
})
}
})
},
})

View File

@@ -32,6 +32,7 @@ const RemoteChatHistory: FC<{ userId: string }> = ({ userId }) => {
const {
data: graphqlData,
isLoading: isLoadingConversations,
isFetching,
hasNextPage,
isFetchingNextPage,
fetchNextPage,
@@ -112,6 +113,7 @@ const RemoteChatHistory: FC<{ userId: string }> = ({ userId }) => {
hasNextPage={hasNextPage}
isFetchingNextPage={isFetchingNextPage}
onLoadMore={fetchNextPage}
isRefreshing={isFetching && !isLoadingConversations}
/>
)
}

View File

@@ -12,6 +12,7 @@ interface ConversationListProps {
hasNextPage?: boolean
isFetchingNextPage?: boolean
onLoadMore?: () => void
isRefreshing?: boolean
}
export const ConversationList: FC<ConversationListProps> = ({
@@ -21,6 +22,7 @@ export const ConversationList: FC<ConversationListProps> = ({
hasNextPage,
isFetchingNextPage,
onLoadMore,
isRefreshing,
}) => {
const loadMoreRef = useRef<HTMLDivElement>(null)
@@ -57,6 +59,12 @@ export const ConversationList: FC<ConversationListProps> = ({
return (
<main className="mt-4 flex h-full flex-1 flex-col space-y-4 overflow-y-auto">
<div className="w-full p-3">
{isRefreshing && (
<div className="flex items-center justify-center gap-2 pb-3 text-muted-foreground text-xs">
<Loader2 className="h-3 w-3 animate-spin" />
<span>Fetching latest conversations</span>
</div>
)}
{!hasConversations ? (
<div className="flex flex-col items-center justify-center py-12 text-center">
<MessageSquare className="mb-3 h-10 w-10 text-muted-foreground/50" />

View File

@@ -11,7 +11,7 @@ export const GetConversationsForHistoryDocument = graphql(`
nodes {
rowId
lastMessagedAt
conversationMessages(last: 5, orderBy: ORDER_INDEX_ASC) {
conversationMessages(first: 2, orderBy: ORDER_INDEX_DESC) {
nodes {
message
}

View File

@@ -8,9 +8,14 @@ import {
SIDEPANEL_SUGGESTION_CLICKED_EVENT,
SIDEPANEL_TAB_REMOVED_EVENT,
SIDEPANEL_TAB_TOGGLED_EVENT,
SIDEPANEL_VOICE_ERROR_EVENT,
SIDEPANEL_VOICE_RECORDING_STARTED_EVENT,
SIDEPANEL_VOICE_RECORDING_STOPPED_EVENT,
SIDEPANEL_VOICE_TRANSCRIPTION_COMPLETED_EVENT,
} from '@/lib/constants/analyticsEvents'
import { useJtbdPopup } from '@/lib/jtbd-popup/useJtbdPopup'
import { track } from '@/lib/metrics/track'
import { useVoiceInput } from '@/lib/voice/useVoiceInput'
import { useChatSessionContext } from '../layout/ChatSessionContext'
import { ChatEmptyState } from './ChatEmptyState'
import { ChatError } from './ChatError'
@@ -31,6 +36,7 @@ export const Chat = () => {
stop,
agentUrlError,
chatError,
selectedProvider,
getActionForMessage,
liked,
onClickLike,
@@ -48,6 +54,8 @@ export const Chat = () => {
onDismiss: onDismissJtbdPopup,
} = useJtbdPopup()
const voice = useVoiceInput()
const [input, setInput] = useState('')
const [attachedTabs, setAttachedTabs] = useState<chrome.tabs.Tab[]>([])
const [mounted, setMounted] = useState(false)
@@ -83,6 +91,26 @@ export const Chat = () => {
previousChatStatus.current = status
}, [status])
// Insert transcript into input when transcription completes
// biome-ignore lint/correctness/useExhaustiveDependencies: only trigger on transcript/transcribing change
useEffect(() => {
if (voice.transcript && !voice.isTranscribing) {
setInput((prev) => {
const separator = prev.trim() ? ' ' : ''
return prev + separator + voice.transcript
})
track(SIDEPANEL_VOICE_TRANSCRIPTION_COMPLETED_EVENT)
voice.clearTranscript()
}
}, [voice.transcript, voice.isTranscribing])
// Track voice errors
useEffect(() => {
if (voice.error) {
track(SIDEPANEL_VOICE_ERROR_EVENT, { error: voice.error })
}
}, [voice.error])
const handleModeChange = (newMode: ChatMode) => {
track(SIDEPANEL_MODE_CHANGED_EVENT, { from: mode, to: newMode })
setMode(newMode)
@@ -147,6 +175,27 @@ export const Chat = () => {
executeMessage(suggestion)
}
const handleStartRecording = async () => {
const started = await voice.startRecording()
if (started) {
track(SIDEPANEL_VOICE_RECORDING_STARTED_EVENT)
}
}
const handleStopRecording = async () => {
await voice.stopRecording()
track(SIDEPANEL_VOICE_RECORDING_STOPPED_EVENT)
}
const voiceState = {
isRecording: voice.isRecording,
isTranscribing: voice.isTranscribing,
audioLevels: voice.audioLevels,
error: voice.error,
onStartRecording: handleStartRecording,
onStopRecording: handleStopRecording,
}
return (
<>
<main className="mt-4 flex h-full flex-1 flex-col space-y-4 overflow-y-auto">
@@ -175,8 +224,15 @@ export const Chat = () => {
onDismissJtbdPopup={onDismissJtbdPopup}
/>
)}
{agentUrlError && <ChatError error={agentUrlError} />}
{chatError && <ChatError error={chatError} />}
{agentUrlError && (
<ChatError
error={agentUrlError}
providerType={selectedProvider?.type}
/>
)}
{chatError && (
<ChatError error={chatError} providerType={selectedProvider?.type} />
)}
</main>
<ChatFooter
@@ -190,6 +246,7 @@ export const Chat = () => {
attachedTabs={attachedTabs}
onToggleTab={toggleTabSelection}
onRemoveTab={removeTab}
voice={voiceState}
/>
</>
)

Some files were not shown because too many files have changed in this diff Show More