Browser tools (allBrowserTools) for tab/page interaction
Context providers for common browser data sources (tabs, bookmarks, history, current page, screenshots)
Storage adapters for extension environments (ChromeStorageAdapter, IndexedDBStorage)
A CDP-based automation layer (via chrome.debugger) and related helpers
Runtime contracts used by the AIPex extension (hosts, addons, omni action registry)

Note: These APIs depend on chrome.* and/or indexedDB. They are not meant to run in plain Node.js.

Why a separate runtime package?

AIPex is split into layers so each stays focused:

@aipexstudio/aipex-core: platform-agnostic agent + events + contexts + sessions
@aipexstudio/browser-runtime: Chrome/extension implementations (tools, providers, storage, automation)
@aipexstudio/aipex-react: React UI toolkit that depends only on core
browser-ext: the actual extension that wires everything together

Features

1) `allBrowserTools` (31 tools)

allBrowserTools is a curated bundle of FunctionTools that an agent can call. It includes:

Tab management: list/switch/open/duplicate/close, basic grouping helpers
UI operations: locate elements, click, hover, fill inputs/forms, wait
Page content: metadata, scrolling, highlighting
Screenshots: capture to data URL / clipboard
Downloads: save text/images from the agent workflow
Human-in-the-loop interventions: request/cancel interventions

Tool names included (strings used for tool-calling):

Tabs: get_all_tabs, get_current_tab, switch_to_tab, create_new_tab, get_tab_info, duplicate_tab, close_tab, organize_tabs, ungroup_tabs
UI ops: search_elements, click, fill_element_by_uid, get_editor_value, fill_form, hover_element_by_uid, wait
Page: get_page_metadata, scroll_to_element, highlight_element, highlight_text_inline
Screenshot: capture_screenshot, capture_tab_screenshot, capture_screenshot_to_clipboard
Download: download_text_as_markdown, download_image, download_chat_images, download_current_chat_images
Interventions: list_interventions, get_intervention_info, request_intervention, cancel_intervention

Note: take_snapshot exists but is intentionally not included in allBrowserTools because it is used internally.

2) Default browser context providers

This package includes providers and a convenience registration helper:

allBrowserProviders
registerDefaultBrowserContextProviders(manager)

Providers included by default:

CurrentPageProvider
TabsProvider
BookmarksProvider
ScreenshotProvider
HistoryProvider

3) Storage adapters

ChromeStorageAdapter<T>: implements KeyValueStorage<T> using chrome.storage.
- Supports "local" and "sync" areas
- Falls back to localStorage when chrome.storage is unavailable (useful for non-extension dev)
IndexedDBStorage<T extends { id: string }>: a small KeyValueStorage implementation backed by IndexedDB

4) CDP automation helpers

Under automation/, you’ll find building blocks for CDP-based workflows (via chrome.debugger):

DebuggerManager
SnapshotManager
SmartLocator / SmartElementHandle
Snapshot text search utilities (searchSnapshotText, parseSearchQuery, hasGlobPatterns)

Installation

npm install @aipexstudio/browser-runtime
# or
pnpm add @aipexstudio/browser-runtime

Peer dependencies:

@types/chrome (for TypeScript)
react (optional; only needed if you use React-related helpers)

Usage

1) Use the built-in browser tools with an agent

import { google } from "@ai-sdk/google";
import { AIPex, aisdk } from "@aipexstudio/aipex-core";
import { allBrowserTools } from "@aipexstudio/browser-runtime";

const agent = AIPex.create({
  instructions: "You can control the current browser tab.",
  model: aisdk(google("gemini-2.5-flash")),
  tools: allBrowserTools,
});

2) Register default browser context providers

import { ContextManager } from "@aipexstudio/aipex-core";
import { registerDefaultBrowserContextProviders } from "@aipexstudio/browser-runtime";

const contextManager = new ContextManager({ autoInitialize: true });
registerDefaultBrowserContextProviders(contextManager);

3) Use extension storage in UI packages

@aipexstudio/aipex-react accepts any KeyValueStorage implementation for persisting settings. In a Chrome extension, you can pass chromeStorageAdapter:

import { Chatbot } from "@aipexstudio/aipex-react";
import { chromeStorageAdapter } from "@aipexstudio/browser-runtime";

export function App({ agent }: { agent: any }) {
  return <Chatbot agent={agent} storageAdapter={chromeStorageAdapter} />;
}

4) Execute a function in the active tab

import { executeScriptInActiveTab } from "@aipexstudio/browser-runtime";

const title = await executeScriptInActiveTab(() => document.title, []);
console.log(title);

5) IndexedDB-backed storage

import { IndexedDBStorage } from "@aipexstudio/browser-runtime";

const storage = new IndexedDBStorage<{ id: string; value: string }>({
  dbName: "aipex",
  storeName: "sessions",
});

API reference

Tools

allBrowserTools: FunctionTool[]
interventionTools: FunctionTool[]
registerDefaultBrowserTools(registryLike)
getActiveTab(): Promise<chrome.tabs.Tab>
executeScriptInTab(tabId, func, args)
executeScriptInActiveTab(func, args)

Context providers

allBrowserProviders: ContextProvider[]
registerDefaultBrowserContextProviders(manager)
Provider classes: CurrentPageProvider, TabsProvider, BookmarksProvider, ScreenshotProvider, HistoryProvider

Storage

ChromeStorageAdapter<T>
chromeStorageAdapter: ChromeStorageAdapter
IndexedDBStorage<T>
IndexedDBConfig

Automation (CDP)

DebuggerManager, debuggerManager
SnapshotManager, snapshotManager
SmartLocator, SmartElementHandle
searchSnapshotText, parseSearchQuery, hasGlobPatterns

Runtime contracts

RuntimeAddon
NoopBrowserAutomationHost
InMemoryOmniActionRegistry
NullInterventionHost
NoopContextProvider

Development (monorepo)

From the repository root:

pnpm --filter @aipexstudio/browser-runtime build
pnpm --filter @aipexstudio/browser-runtime typecheck

License

MIT

README.md Unescape Escape

@aipexstudio/browser-runtime