58 Commits

Author SHA1 Message Date
Dani Akash
dde403962f fix(server): tighten CORS allowlist for the agent server (#966)
* fix(server): tighten CORS allowlist for the agent server

Replace the permissive `origin || '*'` reflection in
`defaultCorsConfig` with an explicit allowlist composed of:

- a static list (empty by default)
- comma-separated origins from `BROWSEROS_TRUSTED_ORIGINS`

Add a small `requireTrustedOrigin` middleware that actively
rejects (403) any request whose `Origin` header is present and
not in the allowlist. The middleware is permissive when the
`Origin` header is absent — CLI tools, internal Node clients,
and some service-worker fetches legitimately omit it; the
threat model only covers cross-origin browser fetches, which
always carry `Origin` (it's on the Forbidden Header List, so
JS cannot suppress it).

Mount the middleware globally in `createHttpServer` after the
existing `cors()` layer. Document the new env var in
`.env.example`.

Tests cover allowlist parsing (empty, single, multi, trims,
case sensitivity, port match) and middleware behaviour
(missing Origin allowed, allowlisted Origin allowed, unknown
Origin rejected, "null" rejected, port mismatch rejected,
disallowed Origin doesn't reach the handler).

* fix(server): include published extension origin in default allowlist

Pin the published BrowserOS extension origin in the static
allowlist so the default install accepts the legitimate
extension without requiring `BROWSEROS_TRUSTED_ORIGINS` to be
populated. Additional origins (dev / alpha) keep working
through the env override.

* chore(server): trim .env.example comments

* chore(server): drop redundant comments from cors helpers
2026-05-08 11:22:54 +05:30
Felarof
1a2fe3a5bf feat(llm): Minimax LLM provider for Chinese and International Users (#756)
* feat(llm): Minimax Chinese and International Users providers

* fix(llm): Patch for p2 bugs

* fix(agent): correct MiniMax base URL handling and enforce API key validation

* fix(agent): add minimax entry to PROVIDER_DISPLAY_NAMES

The Record<ProviderType, string> map in ChatError.tsx was missing
the new minimax key added in this PR, causing a typecheck failure.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: krish-mm <112251957+krish-mm@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 13:40:50 -07:00
Felarof
b5bbbe1aff fix(credits): move credits fetch to extension side (#740)
* fix(credits): move credits fetch to extension side using install_id

Extension now reads `browseros.metrics_install_id` pref directly and fetches
credits from `llm.browseros.com` without going through the bundled server.
Unblocks the referral submit flow in prod without requiring a BrowserOS
binary release.

- Revert `/credits` route change that added `browserosId` to the response.
- Add `getOrCreateBrowserosId()` helper reading from BrowserOS prefs.
- Add `CREDITS_GATEWAY` to shared EXTERNAL_URLS.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(credits): drop fallback UUID, read install_id directly

Extension only runs inside BrowserOS, so the prefs API is always available.
The chrome.storage fallback was dead code that would generate a ghost ID
diverging from the server's install_id anyway. Rename the helper to match
its simpler contract.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(credits): guard against empty install_id pref

Address Greptile P1 — throw instead of silently fetching `/credits/null`
when `browseros.metrics_install_id` is unset. Fails loudly so the broken
state is observable rather than masquerading as a credits outage.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-16 19:27:21 -07:00
Felarof
b6d6d4eb1d feat: Twitter share referral UI for credit rewards (#729)
* feat: add Twitter share referral UI and expose browserosId

When credits are exhausted, users now see a "Share on Twitter" CTA with
a pre-filled tweet URL and an input to paste their tweet link. Reusable
ShareForCredits component used in both ChatError and UsagePage. Server's
GET /credits now includes browserosId for the extension to pass to the
referral service.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: rebuild chat session on provider change

* fix: address Greptile review comments

- Move referral service URL to EXTERNAL_URLS
- Guard submitReferral on !response.ok
- Remove stale TODO comment

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 15:25:04 -07:00
shivammittal274
1f2e783ab9 fix: enable agent interaction with elements inside iframes (#667)
* fix: enable agent interaction with elements inside iframes

Fetch accessibility trees from all frames via Page.getFrameTree() +
per-frame Accessibility.getFullAXTree(frameId), so iframe elements
appear in snapshots with valid backendNodeIds. Pages without iframes
take the original single-call path with zero overhead.

Update snapshot tree builders to walk multiple RootWebArea roots from
merged multi-frame trees. Extract same-origin iframe content in the
markdown walker; show [iframe: url] placeholder for cross-origin.

* fix: namespace AX nodeIds by frameId to prevent cross-frame collisions

CDP AXNodeId values are frame-scoped — each frame's accessibility tree
starts its own counter from 1. Prefix nodeId and childIds with frameId
before merging so the nodeMap in snapshot builders never overwrites
nodes from a different frame.
2026-04-09 23:14:53 +05:30
Nikhil
6712e1d321 chore: bump server and extension version (#659) 2026-04-08 10:18:24 -07:00
Dani Akash
94540d9e87 chore(agent): remove workflows feature (#656) 2026-04-08 08:42:22 +05:30
Nikhil
dee3086a48 feat(server): cache klavis createStrata to unblock /chat hot path (#654)
* feat(server): cache klavis createStrata to unblock /chat hot path

Conversation creation in /chat was blocking on a Worker-proxied
klavisClient.createStrata round-trip every time the user had any
managed Klavis app connected. The 5s KLAVIS_TIMEOUT_MS in the
ai-worker proxy existed specifically to bound this latency, but
the same cap also caused user-visible 504s on /klavis/servers/remove
since Strata DELETE operations routinely take >5s. Without caching
we couldn't raise the timeout without regressing chat creation.

This adds an in-process cache for Strata createStrata responses,
keyed by (browserosId, hashed sorted-server-set) and gated by a 1h
TTL. The cache stores only immutable JSON metadata (strataServerUrl,
strataId, addedServers); per-session MCP clients continue to be
opened and disposed by AiSdkAgent exactly as before, which keeps
the cache concurrency-safe by construction.

Cache invalidation has two layers: (a) the cache key embeds the
server set, so adding/removing apps naturally produces a different
key; (b) POST /klavis/servers/add and DELETE /klavis/servers/remove
explicitly call invalidate(browserosId) after their underlying
Klavis API call succeeds, as defense-in-depth.

Other changes:
- Consolidates klavis-related services into a new
  apps/server/src/api/services/klavis/ directory; moves
  register-klavis-mcp.ts -> strata-proxy.ts and adds strata-cache.ts
  there. lib/clients/klavis/ stays unchanged.
- Refactors KlavisClient.removeServer into a low-level
  deleteServersFromStrata(strataId, servers) primitive. The
  cache-lookup + delete + invalidate orchestration moves up into
  routes/klavis.ts where it belongs, eliminating the lib->api
  layering inversion the original removeServer would have introduced.
- Uses Bun.hash (xxhash64) for fixed-width 16-hex-char keys, with
  serverKey verified on read to make collision risk strictly zero.
- Dedupes concurrent fetches via in-flight Promise sharing, with
  identity-checks before delete to avoid races between invalidate()
  and a racing replacement insert.

Follow-up (separate PR): bump KLAVIS_TIMEOUT_MS to 30000 in
ai-worker/wrangler.toml so /klavis/servers/remove stops 504-ing.

* fix: address greptile review comments for klavis strata cache

- Drop dead `invalidated` field on InflightEntry. It was added to
  support a "discard post-resolution if invalidated" check that I
  later replaced with identity-checked deletes during self-review,
  but I forgot to remove the field and the misleading comment
  referencing it. Simplify Map<string, InflightEntry> to plain
  Map<string, Promise<CacheEntry>>.
- Lower cache miss log from info to debug. Misses fire on every new
  conversation; matching the existing debug-level for hits.
- Stop routing the /klavis/servers/remove handler through
  klavisStrataCache.getOrFetch. The chat hot path keys its cache by
  the user's full enabled-server set (e.g. hash('Gmail,Linear')),
  so a single-server lookup here (hash('Gmail')) is guaranteed to
  miss, write a spurious entry, and then have it immediately
  cleared by invalidate() on the next line. Call createStrata
  directly to recover the strataId, mirroring the original
  removeServer flow.
2026-04-07 11:11:41 -07:00
Nikhil
91be726381 refactor: remove --compile-only flag, consolidate into --ci (#646)
The --compile-only and --ci flags served overlapping purposes for CI
builds. Remove --compile-only entirely since --ci already handles the
CI use case (skip R2, skip prod env validation, local zip packaging)
and --no-upload covers the upload-skipping use case for full builds.
2026-04-03 14:58:52 -07:00
Nikhil
e5a852dd3d chore: update server version (#644) 2026-04-03 14:29:07 -07:00
Nikhil
000429277d fix: isolate server release packaging to ci mode (#629)
* fix: relax compile-only release env requirements

* refactor: add ci mode for server release builds
2026-03-31 20:57:44 -07:00
Nikhil
f0cbf77924 feat: add server release workflow (#627)
* feat: add server release workflow

* fix: address PR review comments for 0331-add_server_release_workflow

* refactor: rework 0331-add_server_release_workflow based on feedback

* refactor: rework 0331-add_server_release_workflow based on feedback
2026-03-31 17:37:06 -07:00
Nikhil
2bb432b0f2 feat: use hidden pages for scheduled tasks (#624)
* feat: use hidden pages for scheduled tasks

* refactor: rework 0331-use_hidden_pages_for_scheduled_tasks based on feedback
2026-03-31 16:02:47 -07:00
Nikhil
9bdb2413ec feat: clean-up - remove obsolete controller extension (#610)
* refactor(server): remove obsolete controller extension backend

* fix: address review feedback for PR #610
2026-03-27 17:01:04 -07:00
Nikhil
83a25ad301 fix: make SDK navigation tolerate unfocused startup tabs (#607) 2026-03-27 14:34:36 -07:00
Nikhil
517750e880 feat: add PostHog to CLI (#603)
* feat: add PostHog usage analytics to CLI

Add anonymous command-level analytics to browseros-cli using the PostHog
Go SDK. Tracks which commands are executed, their success/failure status,
and duration — no PII or person profiles.

- New analytics package with Init/Track/Close singleton
- Distinct ID resolves from server's browseros_id (server.json), falls
  back to CLI-generated UUID (~/.config/browseros-cli/install_id)
- API key injected at build time via ldflags (dev builds = silent no-op)
- Server now writes browseros_id into server.json for cross-surface
  identity correlation

* fix: address PR review feedback for #603

- Return "unknown" for unrecognized args in commandName to avoid
  sending arbitrary user input to PostHog
- Revert goreleaser to {{ .Env.POSTHOG_API_KEY }} (intentional hard
  fail — release builds must have the key set)
- go mod tidy to fix posthog-go direct/indirect marker
- Add POSTHOG_API_KEY to .env.production.example
2026-03-27 12:05:34 -07:00
Dani Akash
febaf58f91 fix: guard filesystem tools behind workspace selection and handle mid-conversation changes (#595)
* fix: remove filesystem tools when no workspace is selected

- Make workingDir optional on ResolvedAgentConfig
- Remove resolveSessionDir() fallback that always created a session dir,
  masking the no-workspace state and keeping filesystem tools available
- Gate buildFilesystemToolSet() on workingDir being defined
- Add workspace change detection mid-conversation — rebuilds the agent
  session when workspace is added, removed, or switched (same pattern
  as existing MCP server change detection)
- download_file falls back to tmpdir() when no workspace is set
- Memory/soul tools are unaffected — they use ~/BrowserOS/ paths

* fix: sanitize message history when session rebuilds with different tools

When a session is rebuilt due to workspace or MCP changes, the carried-over
message history may contain tool parts for tools that no longer exist in
the new session. The AI SDK validates messages against the current toolset
and rejects parts with no matching schema.

- Add toolNames getter to AiSdkAgent exposing registered tool names
- Add sanitizeMessagesForToolset() to strip tool parts referencing
  removed tools from carried-over messages
- Apply sanitization in both MCP and workspace session rebuilds

* fix: prepend tool-change context to user message on session rebuild

When workspace or MCP integrations change mid-conversation, prepend a
[Context: ...] block to the user's message explaining what changed.
This prevents the LLM from hallucinating tool usage based on patterns
in the carried-over conversation history.

Context messages vary by change type:
- Workspace removed: lists unavailable filesystem tools, suggests
  selecting a working directory
- Workspace added: confirms filesystem tools are available with path
- Workspace switched: notes the new working directory
- MCP changed: notes that some integration tools may have changed

Only fires on the first message after a rebuild. Invisible in the UI.

* fix: make MCP change context specific about which apps were added/removed

Diff the old and new MCP server keys to produce specific context like:
- "The following app integrations were disconnected: Gmail, Slack."
- "The following app integrations were connected: Linear."
instead of a generic "some tools may no longer be available" message.

* refactor: extract shared rebuildSession helper in ChatService

Eliminates the duplicated 20-line dispose→create→sanitize→store flow
that existed separately in both the MCP and workspace change-detection
blocks.

Co-authored-by: Dani Akash <DaniAkash@users.noreply.github.com>

* test: add sanitizeMessagesForToolset test suite

Tests for the message sanitization that runs when a session rebuilds
with a different toolset (workspace or MCP change mid-conversation):

- Preserves messages with no tool parts
- Preserves tool parts when tool is in the toolset
- Strips tool parts when tool is NOT in the toolset
- Strips multiple removed tool parts from same message
- Keeps browser tools while removing filesystem tools
- Removes messages that become empty after stripping
- Preserves non-tool parts (reasoning, step-start, file)
- Returns same references when no filtering needed
- Handles empty message array and empty toolset

* style: fix biome formatting in chat-service.ts

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
2026-03-27 18:30:25 +05:30
Dani Akash
aacb47f7ee feat: isolate new-tab agent navigation from origin tab (#593)
* feat: isolate new-tab agent navigation from origin tab

Add origin-aware navigation isolation so the agent never navigates
away from the new-tab chat UI. This is a two-layer defense:

1. Prompt adaptation: When origin is 'newtab', the system prompt's
   execution and tool-selection sections are rewritten to prohibit
   navigating the active tab and default all lookups to new_page.

2. Tool-level guards: navigate_page and close_page reject attempts
   to act on the origin tab when in newtab mode, returning an error
   that teaches the agent to self-correct.

The client now sends an `origin` field ('sidepanel' | 'newtab')
instead of injecting a soft NEWTAB_SYSTEM_PROMPT that LLMs could
ignore. Backwards compatible — defaults to 'sidepanel'.

Closes TKT-592, addresses TKT-564

* test: add newtab origin navigation guard tests

- 14 new prompt tests verifying the system prompt adapts correctly
  for newtab vs sidepanel origin (execution rules, tool selection table,
  absence of conflicting single-tab guidance)
- 6 new integration tests for navigate_page and close_page guards:
  rejects origin tab in newtab mode, allows non-origin tabs, allows
  all tabs in sidepanel mode, backwards compatible with no session
2026-03-27 12:06:32 +05:30
Dani Akash
b3003542d8 docs: overhaul READMEs across all major packages (#594)
* docs: overhaul READMEs across all major packages

- Root README: restructure with feature table, LLM provider table,
  comparison matrix, architecture map, and docs link
- New: packages/browseros/README.md (Chromium fork build system)
- New: apps/server/README.md (MCP server + agent loop)
- New: packages/cdp-protocol/README.md (CDP type bindings)
- Polish: agent-sdk (badges, prerequisites, multi-step example, links)
- Polish: cli (badges, install section, MCP server section, links)
- Polish: agent extension (badges, WXT mention, architecture context)
- Polish: eval (badges, paper links)

* fix: address review — consistent tool count and correct default port

- CLI README: "54 MCP tools" → "53+ MCP tools" to match root and server docs
- Agent SDK README: localhost:3000 → localhost:9100 to match documented default

* docs: add detailed comparison links to How We Compare section

* docs: update comparison table with verified competitor data

Research all 5 competitors via official websites and docs:
- Chrome: no AI agent, Gemini Nano only, MV3 weakening ad blocking
- Brave: BYOM feature, local models via BYOM, Shields ad blocking, MV2+MV3
- Dia: Skills-based AI, no BYOK, cloud AI, acquired by Atlassian
- Comet: full cloud-based agent, built-in ad blocking, extensions on desktop
- Atlas: standalone Chromium browser with Agent Mode, 30-day cloud memory

Renamed Arc/Dia column to just Dia (Arc is sunset).

* docs: simplify comparison table with clean checkmarks and key differentiators

* docs: update browseros-agent README — remove submodule note, add missing packages
2026-03-27 11:59:04 +05:30
Nikhil
aba7a10430 chore: server release (#592) 2026-03-26 19:13:56 -07:00
Dani Akash
f45cb58889 fix: stop sending port-in-use errors to Sentry (#558)
Port conflicts are expected — Chromium retries with a different port.
These errors were flooding Sentry (14k+ events) without user impact.

- handleStartupError: move Sentry.captureException below the
  port-in-use check so it only fires for unexpected startup errors
- handleControllerStartupError: skip Sentry capture for port errors
- index.ts: exit early for port errors before Sentry capture
2026-03-26 09:32:18 +05:30
shivammittal274
c316e09c11 feat: add source tag to tool_executed PostHog events (#538)
Add `source: 'mcp' | 'chat'` property to all `tool_executed` metrics
events so we can distinguish tool calls from external MCP clients
(Claude Code, Cursor) vs the built-in BrowserOS agent in PostHog.

- register-mcp.ts: source='mcp' (browser tools via MCP endpoint)
- register-klavis-mcp.ts: source='mcp' (Klavis tools via MCP endpoint)
- tool-adapter.ts: source='chat' (browser tools via chat agent)
- ai-sdk-agent.ts: source='chat' (Klavis/external MCP tools via chat agent, previously untracked)
- filesystem/utils.ts: source='chat' (filesystem tools via chat agent)
2026-03-24 02:03:18 +05:30
Nikhil
e97d8bc1cb fix: remove daily rate-limit middleware (#535)
* fix: remove daily rate-limit middleware

The daily conversation rate limit is no longer needed. Remove the
middleware, RateLimiter class, fetch-config, error type, shared
constants, DB schema table, and integration tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove unused getDb() method

No longer needed after rate-limiter removal.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 08:31:20 -07:00
Dani Akash
5109ca4347 feat: added scope in server error logs (#533)
* feat: added scope in server error logs

* fix: prevent double capture on chat request
2026-03-23 20:47:28 +05:30
Dani Akash
86ec88ed80 feat: sentry improvements (#532)
* feat: process request record from sentry locally

* feat: added analytics for logged in users
2026-03-23 19:45:28 +05:30
Dani Akash
4928b7e84b fix: no current window and sentry context (#531)
* fix: error reporting and better breadcrumbs

* fix: lint issues
2026-03-23 18:46:39 +05:30
shivammittal274
94a1a701f6 fix(eval): include browser context in agent prompt (#530)
The eval's single-agent was passing raw task.query as the prompt,
without browser context (active tab URL, title). The agent didn't
know which page it was on, causing it to ask "which website?" instead
of browsing.

Use formatUserMessage() (same as chat-service.ts) to include browser
context in the prompt. Re-export formatUserMessage from agent/tool-loop.
2026-03-23 17:42:03 +05:30
Nikhil
ba7892322b ci: run BrowserOS test suites on PRs (#514)
* ci: run browseros tests on pull requests

* refactor: rework 0320-github_action_for_tests based on feedback

* refactor: rework 0320-github_action_for_tests based on feedback

* chore: add CI artifacts to .gitignore

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove mikepenz/action-junit-report to fix check suite misattribution

The JUnit report action creates check runs that GitHub associates with the
CLA check suite instead of the Tests check suite, causing test reports to
appear under "CLA Assistant" in the PR checks UI.

Remove the action and rely on job status + step summary + artifact upload
for test result visibility.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-21 09:46:36 -07:00
shivammittal274
86eed82350 fix: lazy OAuth callback server with cancel+retry (Codex CLI pattern) (#515)
The OAuth callback server on port 1455 was bound eagerly at startup,
crashing the server if another BrowserOS instance was already running.

Rewrite as a lazy class (OAuthCallbackServer) that:
- Only binds port 1455 when the user initiates a ChatGPT Pro login
- Sends GET /cancel to any existing server on the port first, then
  retries up to 5 times (follows Codex CLI's cancel+retry pattern)
- Exposes /cancel endpoint so other instances/tools can cancel us
- Releases the port after the OAuth callback arrives
- Device-code providers (GitHub Copilot, Qwen) never touch port 1455

This allows running eval, dev instances, and multiple BrowserOS
instances without port conflicts. OAuth login works on whichever
instance initiates it — the others continue without OAuth.
2026-03-21 16:44:03 +05:30
Nikhil
be6ed22af4 test: fix BrowserOS tool test harness regressions (#513)
* test: fix browseros tool test harness regressions

* test: align working directory naming in page action tests
2026-03-20 12:05:39 -07:00
Nikhil
149cde118d chore: bump server version, offset and patch for release (#512) 2026-03-20 11:45:12 -07:00
Nikhil
9bc5e666c4 feat: auto-discover server port via ~/.browseros/server.json (#504)
* feat: auto-discover server port via ~/.browseros/server.json

Server writes its port to ~/.browseros/server.json on startup so the CLI
can auto-discover the server URL without requiring `browseros-cli init`.

Discovery chain: BROWSEROS_URL env > config.yaml > server.json > error

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address review feedback for PR #504

- Use synchronous unlinkSync in stop() since process.exit() fires
  immediately after, abandoning any pending async operations
- Wrap writeServerConfig in try/catch so a write failure doesn't crash
  a healthy server for a convenience feature

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: type server discovery config and add version metadata

Add ServerDiscoveryConfig interface to @browseros/shared and enrich
server.json with server_version, browseros_version, and chromium_version.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: normalize URL from server.json for consistency

All other URL sources (env var, config.yaml) pass through
normalizeServerURL; apply the same to the server.json path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 11:37:00 -07:00
Nikhil
f865d301a2 test: add build smoke test to catch compile failures (#511)
* test: add build smoke test to catch compile failures

Compiles the server binary (darwin-arm64) and verifies --version outputs
the correct version from package.json. Uses an empty resource manifest
and stub env vars so the test runs without R2 access or real secrets.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address review feedback for PR #511

- Derive build target from process.platform/arch for CI portability
- Include binary stderr in --version assertion for better diagnostics

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 11:16:57 -07:00
Nikhil
6f398f0b36 fix: replace sharp with jimp to fix compiled binary crash (#510)
sharp is a native C module (libvips) whose .node binaries can't be
embedded in Bun compiled executables. It was imported at the top level
in copilot-fetch.ts, crashing the entire server at startup.

Replace with jimp (pure JavaScript, zero native deps) which bundles
cleanly into compiled binaries. Same resize algorithm preserved.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 11:06:05 -07:00
shivammittal274
8548bcf50a feat: credit-based tracking for BrowserOS provider (#489)
* feat: add credit-based tracking for BrowserOS provider

Send X-BrowserOS-ID header on all LLM requests through the BrowserOS
gateway for per-installation credit tracking. Handle 429 CREDITS_EXHAUSTED
as non-retryable. Add GET/PUT /credits endpoints to check and manage
credit balance.

* docs: add credits tracking UI design

Design for showing credit balance in side panel chat header (color-coded
badge) and a dedicated Usage & Billing settings page. Credits refresh
after each completed message turn or on exhaustion error.

* docs: add credits tracking UI implementation plan

8-task plan covering useCredits hook, CreditBadge component, ChatHeader
integration, message completion refresh, ChatError CREDITS_EXHAUSTED
handling, Usage & Billing settings page, and route/sidebar registration.

* feat: add useCredits React Query hook

* feat: add CreditBadge component with color thresholds

* feat: show credit badge in chat header for BrowserOS provider

* feat: refresh credits after chat message completion and on error

* feat: handle CREDITS_EXHAUSTED error in chat

* feat: add Usage & Billing settings page

* feat: register usage page route and sidebar entry

* fix: lint and formatting fixes for credit tracking UI

* fix: separate credits exhausted from Kimi rate limit in ChatError, redesign Usage page

* chore: remove PUT /credits endpoint and setCredits function

* fix: extract shared credit colors, add error state to UsagePage, use dailyLimit from gateway

* fix: make dailyLimit required in CreditsInfo (gateway always returns it)

* feat: gate credits UI behind CREDITS_SUPPORT feature flag (server >= 0.0.78)
2026-03-20 22:49:00 +05:30
Dani Akash
2b4fdf1aad feat: improved multi tab agent workflow (#507)
* feat: updated multitab workflow

* fix: updated prompt with fix for test cases

* fix: active agent glow

* fix: review comments
2026-03-20 18:31:36 +05:30
shivammittal274
11d15d079f feat: alibaba qwen oauth (#506)
* feat: add Qwen Code as OAuth LLM provider with refactored OAuth hooks

Add Alibaba Qwen Code as a third OAuth provider using Device Code flow
with PKCE. Free tier: 2,000 requests/day, up to 1M token context.

Refactoring:
- Extract useOAuthProviderFlow hook (eliminates ~180 lines of duplicated
  OAuth logic from AISettingsPage for ChatGPT Pro + Copilot + Qwen)
- Extract resolveOAuthConfig in config.ts (shared resolver for all OAuth
  providers, parameterized by provider name, default model, refresh flag)
- Generalize token-manager device code flow to support PKCE
  (code_challenge/code_verifier) and form-urlencoded content type

New code:
- Qwen Code provider config with PKCE + form encoding flags
- Provider factories (both provider.ts and provider-factory.ts)
- Extension UI (template card, models, analytics, dialog)

* fix: use portal.qwen.ai as API base URL for OAuth tokens

DashScope (dashscope.aliyuncs.com) expects Alibaba Cloud API keys,
not OAuth tokens from chat.qwen.ai. The correct endpoint for OAuth
Bearer tokens is portal.qwen.ai/v1.

* fix: correct Qwen Code model IDs and context windows

- coder-model (1M context): virtual alias that routes to best model
- qwen3-coder-plus (1M): was incorrectly 131K
- qwen3-coder-flash (1M): new, speed-optimized variant
- qwen3.5-plus (1M): was incorrectly 1048576 (power-of-two vs decimal)
- Removed qwen3-coder-next (local/self-hosted, not available via OAuth)
- Default model changed to coder-model (auto-routes server-side)

* fix: move Qwen device code request to extension (bypasses WAF)

Alibaba WAF blocks server-side requests to chat.qwen.ai. Move the
initial device code request to the extension (browser context with
cookies), then hand off the deviceCode + codeVerifier to the server
for background polling via new POST /oauth/:provider/poll endpoint.

* fix: persist OAuth flow-started flag in sessionStorage

The flowStartedRef was lost when the component remounted (e.g. user
navigated to onboarding then back to settings). Use sessionStorage
to persist the flag so auto-create works after navigation.

* revert: remove sessionStorage for OAuth flow flag

Revert to simple useRef pattern matching the original ChatGPT Pro
implementation. The auto-create works when the user stays on the
AI settings page during auth.

* revert: move Qwen back to server-side device code flow

WAF block was temporary (rate-limiting), not permanent. Server-side
fetch to chat.qwen.ai now works. Reverted client-side device code
approach — Qwen now uses the same clean server-side flow as Copilot.

Removed: clientSideDeviceCode config, startClientSideDeviceCode(),
POST /oauth/:provider/poll endpoint, startDeviceCodePolling().

* feat: add WAF detection, rate-limit protection, and token storage endpoint

- Detect WAF captcha responses (HTML instead of JSON) in device code
  request and token polling, with user-friendly error messages
- Add 30s cooldown on "USE" button to prevent rapid clicks triggering WAF
- WAF-blocked poll requests silently retry instead of aborting
- Add POST /oauth/:provider/token endpoint for storing externally-provided
  tokens (useful for future fallback flows)
- Add storeTokens() method to OAuthTokenManager
- Pass server error messages through to extension toast notifications

* refactor: remove 30s cooldown, simplify OAuth hook

The hook is now identical for all providers — server handles retries
via activeDeviceFlows.delete(). Removed flowStartedAtRef cooldown
that was blocking legitimate retries.

* feat: client-side OAuth for Copilot and Qwen Code

Move device code OAuth flow to the extension for GitHub Copilot and
Qwen Code. The extension makes requests using Chrome's network stack,
which bypasses Alibaba WAF TLS fingerprint detection that blocks
server-side Bun/Node.js fetch.

New files:
- client-oauth.ts: Client-side device code + PKCE + token polling

Changes:
- useOAuthProviderFlow: handleClientAuth() for providers with clientAuth
  config, handleServerAuth() for others (ChatGPT Pro)
- AISettingsPage: clientAuth config for Copilot and Qwen Code
- WAF detection: opens provider site for captcha solving on block

Server-side device code flow preserved as fallback (token-manager.ts,
providers.ts). Token storage via POST /oauth/:provider/token endpoint.

* fix: export OAuthProviderFlowConfig type, fix typecheck errors

- Export OAuthProviderFlowConfig interface so AISettingsPage can use it
  instead of duplicating the type inline
- Fix string | null → string | undefined for agentServerUrl parameter
2026-03-20 17:46:48 +05:30
Nikhil
1c737b0f02 chore: bump server version (#501) 2026-03-19 16:17:50 -07:00
shivammittal274
720baaed3e feat: add GitHub Copilot as OAuth LLM provider (#500)
* feat: add GitHub Copilot as OAuth-based LLM provider

Add GitHub Copilot as a second OAuth provider using the Device Code flow
(RFC 8628). Users authenticate via github.com/login/device, and the server
polls for token completion. Supports 25+ models through a single Copilot
subscription.

Key changes:
- Device Code OAuth flow in token manager (poll with safety margin)
- Custom fetch wrapper injecting Copilot headers + vision detection
- Provider factory using createOpenAICompatible for Chat Completions API
- Extension UI with template card, auto-create on auth, and disconnect

* fix: address PR review comments for GitHub Copilot OAuth

- Validate device code response for error fields (GitHub can return 200
  with error payload)
- Store empty refreshToken instead of access token for GitHub tokens
- Add closeButton to Toaster for dismissing device code toast

* fix: add github-copilot to agent provider factory

The chat route uses a separate provider-factory.ts (agent layer) from the
test-provider route (llm/provider.ts). Added createGitHubCopilotFactory
to the agent factory so chat works with GitHub Copilot.

* fix: add github-copilot to provider icons, models, and dialog

- Add Github icon from lucide-react to providerIcons map
- Add 8 Copilot models (GPT-4o, Claude, Gemini, Grok) to models.ts
- Add github-copilot to NewProviderDialog zod enum, validation skip,
  canTest check, and OAuth credential message

* fix: reorder copilot models with free-tier models first

Put models available on Copilot Free at the top (gpt-4o, gpt-4.1,
gpt-5-mini, claude-haiku-4.5, grok-code-fast-1), followed by
premium models that require paid Copilot subscription.

* fix: set correct 64K context window for Copilot models

Copilot API enforces a 64K input token limit regardless of the
underlying model's native context window. Updated all model entries
and the default template to 64000 so compaction triggers correctly.

* fix: use actual per-model prompt limits from Copilot /models API

Queried api.githubcopilot.com/models for real max_prompt_tokens values.
GPT-4o/4.1 have 64K, Claude/gpt-5-mini have 128K, GPT-5.x have 272K.
Also updated model list to match what's actually available on the API
(e.g. claude-sonnet-4.6 instead of 4.5, added gpt-5.4/5.2-codex).

* feat: resize images for Copilot using VS Code's algorithm

Large screenshots cause 413 errors on Copilot's API. Resize images
following VS Code's approach: max 2048px longest side, 768px shortest
side, re-encode as JPEG at 75% quality. Uses sharp for server-side
image processing.

* fix: address all Greptile P1 review comments

- Add .catch() on fire-and-forget pollDeviceCode to prevent unhandled
  rejection crashes (Node 15+)
- Add deduplication guard (activeDeviceFlows Set) to prevent concurrent
  device code flows for the same provider
- Add runtime validation of server response in frontend before calling
  window.open() and showing toast
- Remove dead GITHUB_DEVICE_VERIFICATION constant from urls.ts

* fix: upgrade biome to 2.4.8, fix all lint errors, and address review bugs

- Upgrade biome from 2.4.5 to 2.4.8 (matches CI) and migrate configs
- Fix image resize: only re-encode when dimensions actually change
- Fix device code polling: retry on transient network errors instead of aborting
- Allow restarting device code flow (clear old flow instead of throwing 500)
- Fix pre-existing noNonNullAssertion and noExplicitAny lint errors globally

* fix: address Greptile P2 review — image resize and config guard

- Fix early-return guard: check max/min sides against their respective
  limits (MAX_LONG_SIDE/MAX_SHORT_SIDE) instead of both against SHORT
- Preserve PNG alpha: detect hasAlpha and keep PNG format instead of
  unconditionally converting to lossy JPEG
- Keep browserosId guard in resolveGitHubCopilotConfig consistent with
  ChatGPT Pro pattern (safety check that caller context is valid)

* feat: update Copilot models to full list from pricing page, default to gpt-5-mini

Added all 23 models from GitHub Copilot pricing page. Ordered with
free-tier models first (gpt-5-mini, claude-haiku-4.5), then premium.
Changed default from gpt-4o to gpt-5-mini since it's unlimited on
Pro plan and has 128K context (vs gpt-4o's 64K limit).
2026-03-20 02:33:09 +05:30
Dani Akash
5bb6143373 feat: display selected text from page in sidepanel (#496)
* feat: select text and pass to sidepanel

* fix: lint issues

* fix: persist selection across tabs

* fix: review comments

* fix: change when the selection is cleared

* feat: sanitize url
2026-03-19 20:21:31 +05:30
Dani Akash
f4d4b73a24 fix: improved memory tools (#495)
* fix: new prompt update tool

* fix: memory search tool

* fix: all review comments

* chore: remove dead code
2026-03-19 19:01:25 +05:30
Dani Akash
d965698905 fix: biome & tsc setup across repo (#493)
* fix: biome lint issues

* fix: code quality workflow

* fix: all lint issues

* chore: test lefthook pre-commit hook

* chore: test lefthook with agent file

* chore: revert test comment from lefthook verification

* feat: setup tsgo for typechecking agent

* fix: typecheck cli command

* fix: early return to prevent errors
2026-03-19 18:18:24 +05:30
shivammittal274
50b2f45590 fix(skills): UI section separation and fix find-alternatives rendering (#492)
* fix(skills): UI section separation and fix find-alternatives rendering

- Split skills page into "My Skills" (user) and "BrowserOS Skills" (built-in) sections
- Fix find-alternatives SKILL.md — replace angle bracket placeholders with curly
  braces to prevent MDXEditor from parsing them as JSX and rendering empty content

* fix(skills): bump find-alternatives to v1.1 for CDN sync
2026-03-19 17:38:28 +05:30
shivammittal274
079a254fa4 fix(skills): separate built-in and user skills into distinct directories (#487)
* fix(skills): separate built-in and user skills into distinct directories

- Move built-in skills to ~/.browseros/skills/builtin/, user skills stay in root
- Unify seed + sync into single syncBuiltinSkills() function, delete seed.ts
- Preserve user's enabled/disabled state during remote sync version updates
- Add catalog reconciliation — remove built-in skills dropped from remote catalog
- Fallback to bundled defaults per-skill when remote sync fails
- One-time migration moves existing default skills from root to builtin/
- Add builtIn field to SkillMeta, determined by directory (not metadata)
- UI shows "Built-in" badge, hides delete button for built-in skills
- Reject deletion of built-in skills in service layer
- Check both dirs for ID collision on skill creation

* fix(skills): address review — dedup by id, guard applyEnabled regex

- loader.ts: deduplication now keys on skill.id (directory slug) not
  skill.name (display name), preventing silent drops on name collision
- remote-sync.ts: applyEnabled checks if regex matched before writing,
  logs warning if remote content lacks an enabled field

* fix(skills): reconciliation preserves bundled defaults, delete returns 403

- reconcileRemovedSkills now keeps DEFAULT_SKILLS IDs in the safe set,
  preventing delete-then-reinstall cycle that lost enabled:false state
- DELETE /skills/:id returns 403 for built-in skills instead of 500

* refactor(skills): simplify syncBuiltinSkills to single clean pass

Build content map (bundled + remote), iterate once, preserve enabled,
reconcile deletions. Removes 7 helper functions, 70 lines of code.

* refactor(skills): extract syncOneSkill, patch content before writing

- syncBuiltinSkills is now 15 lines: build map, iterate, clean up
- syncOneSkill: flat, patches enabled state before writing (single write)
- setEnabled: pure function for content patching
- removeObsoleteSkills: extracted from inline block
2026-03-19 13:35:47 +05:30
shivammittal274
4000f094f6 Feat/chatgpt pro polish (#484)
* fix: ChatGPT Pro UI polish — fix undefined display and add icon

- Fix "gpt-5.3-codex · undefined" — hide baseUrl when not set
- Add OpenAI icon for chatgpt-pro provider in icon map

* chore: rename ChatGPT Pro to ChatGPT Plus/Pro (supports both plans)

* chore: remove accidentally committed files
2026-03-18 22:51:22 +05:30
shivammittal274
46a8326140 feat: add ChatGPT Pro OAuth as LLM provider (#476)
* feat: add ChatGPT Pro OAuth as LLM provider

Adds OAuth 2.0 (Authorization Code + PKCE) flow so users can authenticate
with their ChatGPT Pro subscription to power BrowserOS's agent, matching
the pattern used by Codex CLI, OpenCode, and Pi.

Server:
- OAuth token lifecycle (PKCE, exchange, refresh, SQLite storage)
- Dedicated callback server on port 1455 (Codex client ID registration)
- Codex fetch wrapper routing API calls to chatgpt.com/backend-api
- Config resolution + provider factories for all code paths (chat, test, refine)

Extension:
- ChatGPT Pro template card with OAuth flow trigger
- Status polling hook + auto-create provider on auth success
- Model list with Codex-supported models (gpt-5.x-codex family)

* fix: address Greptile PR review comments

- Wire OAuth callback server stop handle into onShutdown (P1: port 1455 leak)
- Guard against missing refresh token + clear stale tokens on failed refresh (P1)
- Add logger.warn to silent catch in codex-fetch body mutation
- Document JWT trust assumption in parseAccessTokenClaims
- Source model ID from provider template instead of hard-coding

* simplify: remove unnecessary OAuth shutdown wiring and useCallback

- Revert OAuthHandle interface — callback server port releases on process exit
- Remove stopCallbackServer from shutdown flow (dead code)
- Remove all useCallback from useOAuthStatus per CLAUDE.md guidance

* style: add readonly modifiers and braces per TS style guide

* docs: add E2E test screenshots for ChatGPT Pro OAuth

* fix: strip item IDs from Codex requests to fix multi-turn conversations

* fix: preserve function_call_output IDs in Codex requests

* fix: resolve Codex store=false + tool-use incompatibility

- Pass providerOptions { openai: { store: false } } to ToolLoopAgent
  so the AI SDK inlines content instead of using item_reference
- Strip item IDs and previous_response_id in codex-fetch (safety net)
- Use .responses() model (Codex only speaks Responses API format)

* fix: remove non-Codex model gpt-5.2 from chatgpt-pro model list

* fix: strip unsupported Codex params and update model list

- Strip temperature, max_tokens, top_p from Codex requests (unsupported)
- Add all available Codex models including gpt-5.4, gpt-5.2, gpt-5.1

* chore: remove screenshots containing email

* feat: enable reasoning events for ChatGPT Pro Codex models

* chore: set reasoning effort to high for ChatGPT Pro

* feat: add configurable reasoning effort and summary for ChatGPT Pro

- Add reasoningEffort (none/low/medium/high) and reasoningSummary
  (auto/concise/detailed) dropdowns in the Edit Provider dialog
- Pass through extension → chat request → agent config → providerOptions
- Defaults: effort=high, summary=auto

* fix: strip max_output_tokens from Codex requests (fixes compaction)

* fix: address Greptile P1 issues

- Fix default model fallback: gpt-4o → gpt-5.3-codex (Codex endpoint)
- Clear stale tokens on refresh failure (prevents infinite retry loop)
- Only auto-create provider after explicit OAuth flow, not on page load
- Add catch block to auto-create effect with error toast
2026-03-18 22:07:43 +05:30
Nikhil
22c5e85707 chore: bump server version (#478) 2026-03-17 17:12:23 -07:00
shivammittal274
59b00a6837 feat: remote skill download and auto-sync (#468)
* feat: add remote skill download and auto-sync

Download default skills from remote catalog on first setup with
bundled fallback when offline. Background sync every 45 minutes
checks for new/updated skills without overwriting user-customized
ones. Tracks installed defaults via content hashes in a local
manifest file.

* feat: make skills catalog URL configurable and add generation script

Add SKILLS_CATALOG_URL env var (following CODEGEN_SERVICE_URL pattern)
with fallback to the default constant. Add script to generate
catalog.json from bundled defaults for static hosting.

* feat: add R2 upload script and use cdn.browseros.com for catalog URL

Add upload-skills-catalog.ts that generates and uploads catalog.json
to Cloudflare R2 (same infra as existing build artifacts). Update
default catalog URL to cdn.browseros.com/skills/v1/catalog.json.

* test: add E2E tests for remote skill sync against live CDN

* fix: address code review findings — security, validation, DRY

- Add path traversal protection via safeSkillDir in writeSkillFile
  and readSkillContent (reuses existing validation from service.ts)
- Add runtime type guards for catalog JSON and manifest JSON parsing
- Fix seedFromRemote to return false on partial failure so bundled
  fallback kicks in
- Add per-skill error handling in syncRemoteSkills so one bad skill
  doesn't crash the entire sync
- Wire stopSkillSync into Application.stop() shutdown path
- Extract version from frontmatter in seedFromBundled instead of
  hardcoding '1.0'
- Consolidate duplicated logic: reuse installSkill/writeSkillFile/
  contentHash/saveManifest from remote-sync.ts in seed.ts
- Extract shared catalog generation into scripts/catalog-utils.ts

* test: add flow tests for all four sync scenarios against live CDN

* refactor: remove redundant scripts and inline catalog generation

Drop generate-skills-catalog.ts, catalog-utils.ts, and
e2e-remote-sync.test.ts (covered by flows.test.ts). Inline
catalog generation into upload-skills-catalog.ts.

* test: add full E2E server flow test against live CDN

Tests all 7 steps of the real server lifecycle: fresh seed from CDN,
no-op sync, user edit preservation, skill reinstall, custom skill
protection, background timer firing, and second startup skip.

* chore: remove e2e-server-flow test

* fix: address Greptile review — entry validation, size limit, DRY, no-op saves

- Validate individual skill entries in catalog (id, version, content
  must all be strings) not just the top-level shape
- Add 1MB response size limit on catalog fetch to prevent resource
  exhaustion from compromised/misconfigured CDN
- Skip manifest save when sync cycle had no changes (avoids
  unnecessary disk I/O every 45 minutes)
- Share extractVersion via remote-sync.ts export, remove duplicate
  from seed.ts

* fix: prevent bundled fallback from overwriting partial remote seeds

When seedFromRemote partially fails, the bundled fallback now skips
skills already in the manifest (installed by the partial remote
seed). Also adds Content-Length early check before downloading the
full catalog response body.

* fix: run sync immediately on startup, not just on interval

Previously the first sync fired 45 minutes after boot. Now
startSkillSync runs one sync immediately so returning users
get skill updates right away.

* refactor: simplify sync — remote always wins, remove manifest

Remote catalog is the source of truth. If a skill exists in the
catalog, its version is compared against local frontmatter and
overwritten when newer. No manifest file, no content hashes.

User-created skills (IDs not in catalog) are never touched.

* fix: skip bundled skills already installed by partial remote seed

* chore: remove unreliable Content-Length check

* chore: remove size limit checks, fetch timeout is sufficient
2026-03-17 21:40:45 +05:30
shivammittal274
2597cdbc70 feat: add Rewrite with AI for scheduled task prompts (#465)
* feat: add "Rewrite with AI" prompt refinement for scheduled tasks

Add a lightweight /refine-prompt endpoint that uses generateText to
rewrite rough scheduled task prompts into clear, actionable instructions.
The UI adds a sparkle-icon button next to the Prompt label in the
NewScheduledTaskDialog with loading state, undo support, and disabled
state when the textarea is empty.

* fix: clear stale undo ref on dialog re-open and pass providerId to refinePrompt

- Reset originalPromptRef when dialog opens and on form submit to
  prevent stale "Undo rewrite" button on re-open
- Accept optional providerId in refinePrompt() so the form's selected
  provider is used for refinement instead of always the system default

* fix: hide undo rewrite link while refinement is in flight

* fix: reset isRefining state on dialog re-open

* fix: ignore stale refine-prompt responses after dialog re-open

Use a request generation counter so that if the dialog is closed and
re-opened while a rewrite is in flight, the stale response is silently
discarded instead of overwriting the fresh form state.

* fix: invalidate stale refine requests on dialog reopen and rename to kebab-case

- Increment refineRequestIdRef on dialog open so in-flight requests
  from a previous session are discarded when they complete
- Rename refinePrompt.ts to refine-prompt.ts per CLAUDE.md file naming
2026-03-17 19:40:56 +05:30
Dani Akash
2a6848bc1d feat: improved system prompt (#466)
* feat: added ai-sdk dev tools

* feat: new system prompt section

* feat: tests to maintain prompt integrity

* feat: update mcp sync to use react query

* fix: refetch logic for sync

* chore: remove limits on fetching integrations

* fix: refetch integrations on delete

* fix: review comment

* chore: update tests

* fix: improved memory classification

* fix: lint issues

* fix: core memory prompts

* fix: handle scenario where soul file is empty
2026-03-17 19:01:10 +05:30