mirror of
https://github.com/AIPexStudio/AIPex.git
synced 2026-05-13 18:51:35 +00:00
feat: enhance onboarding and replay functionality in browser extension
- Added a feature to open the onboarding page for new installs in production. - Implemented a user manual replay request handler, validating input data and managing replay steps. - Introduced a side panel for replay setup, allowing users to navigate and initialize replay sessions. - Enhanced the chat header with options to share conversations and save them as skills, improving user interaction. - Added error handling and validation for user-provided host URLs to mitigate security risks.
This commit is contained in:
ropzislaw
267
migration/SIDEPANEL_ENTRY_GAP_ANALYSIS.md
Normal file
267
migration/SIDEPANEL_ENTRY_GAP_ANALYSIS.md
Normal file
@@ -0,0 +1,267 @@
|
|||||||
|
# Sidepanel 入口链路：new-aipex vs aipex 缺口分析
|
||||||
|
|
||||||
|
> **目的**：从 sidepanel 入口出发，系统梳理 `new-aipex` 相比 `aipex` 在端到端链路上的不足之处，为后续补齐工作提供可执行清单。
|
||||||
|
>
|
||||||
|
> **生成日期**：2026-02-15
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. 对比基线与评估维度
|
||||||
|
|
||||||
|
| 项目 | 根路径 | 入口文件 |
|
||||||
|
|------|--------|----------|
|
||||||
|
| aipex（旧版，全功能） | `aipex/` | `src/sidepanel.tsx` → `ChatBot`（2800+ 行单体组件） |
|
||||||
|
| new-aipex（新版，模块化） | `new-aipex/packages/` | `browser-ext/src/pages/sidepanel/index.tsx` → `app-root.tsx` → `ChatBot`（组件化 + hooks） |
|
||||||
|
|
||||||
|
**评估维度**：触发与打开 / 初始化与配置加载 / 消息桥接 / Agent 与工具 / 会话持久化 / 会话历史 / 回放链路 / UI 功能 / 服务依赖 / 错误处理 / 可观测性
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Sidepanel 端到端链路对比
|
||||||
|
|
||||||
|
### 2.1 链路流程图
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart TD
|
||||||
|
subgraph trigger [触发阶段]
|
||||||
|
iconClick[扩展图标点击]
|
||||||
|
shortcut[Ctrl/Cmd+M 快捷键]
|
||||||
|
omni[Omni 选择 ai-chat]
|
||||||
|
external[外部网站 openWithPrompt]
|
||||||
|
replay[外部网站 REPLAY_USER_MANUAL]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph bg [Background Service Worker]
|
||||||
|
bgOpen[chrome.sidePanel.open]
|
||||||
|
bgRelay[消息中转与状态管理]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph sp [Sidepanel 初始化]
|
||||||
|
vmInit[VM 初始化 - QuickJS/ZenFS]
|
||||||
|
reactMount[React 挂载]
|
||||||
|
providerTree[Provider 树]
|
||||||
|
agentCreate[Agent/MessageHandler 创建]
|
||||||
|
toolLoad[工具加载与过滤]
|
||||||
|
skillInit[Skill 系统初始化]
|
||||||
|
interventionInit[Intervention 初始化]
|
||||||
|
pendingPrompt[Pending Prompt 消费]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph chat [首次交互]
|
||||||
|
userInput[用户输入]
|
||||||
|
stream[流式响应]
|
||||||
|
persist[会话持久化]
|
||||||
|
end
|
||||||
|
|
||||||
|
iconClick --> bgOpen
|
||||||
|
shortcut --> bgRelay --> bgOpen
|
||||||
|
omni --> bgRelay
|
||||||
|
external --> bgOpen
|
||||||
|
replay --> bgOpen
|
||||||
|
|
||||||
|
bgOpen --> vmInit
|
||||||
|
vmInit --> reactMount
|
||||||
|
reactMount --> providerTree
|
||||||
|
providerTree --> agentCreate
|
||||||
|
agentCreate --> toolLoad
|
||||||
|
agentCreate --> skillInit
|
||||||
|
agentCreate --> interventionInit
|
||||||
|
agentCreate --> pendingPrompt
|
||||||
|
|
||||||
|
pendingPrompt --> userInput
|
||||||
|
toolLoad --> userInput
|
||||||
|
userInput --> stream
|
||||||
|
stream --> persist
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2.2 新旧链路步骤对照
|
||||||
|
|
||||||
|
| 阶段 | aipex (旧版) | new-aipex (新版) | 差异 |
|
||||||
|
|------|-------------|-----------------|------|
|
||||||
|
| **触发 → 打开** | `chrome.action.onClicked` / `open-aipex` 快捷键 / content script `open-sidepanel` / `openWithPrompt` / `REPLAY_USER_MANUAL` | `chrome.action.onClicked` / `open-aipex` 快捷键 / content script 无 `open-sidepanel` / `openWithPrompt` | 缺少 `REPLAY_USER_MANUAL` 外部触发 |
|
||||||
|
| **选中文本传递** | `open-sidepanel` 可携带 `selectedText` → `get-selected-text` 取回 | 无此机制 | **缺失** |
|
||||||
|
| **VM 初始化** | `initializeVM()` — QuickJS + ZenFS | 同 | 一致 |
|
||||||
|
| **React 挂载** | `ReactDOM.createRoot` → `SidepanelApp` | `renderChatApp()` → `App` | 一致 |
|
||||||
|
| **Provider 树** | `ThemeProvider → I18nProvider → ChatBot` | `I18nProvider → ThemeProvider → AuthProvider → ChatApp` | 新版多了 `AuthProvider` |
|
||||||
|
| **配置加载** | `useStorage` 逐个读 `aiHost/aiToken/aiModel` + 等待 `isLoading*` | `useChatConfig({ storageAdapter })` 统一读取 | 新版更简洁 |
|
||||||
|
| **Agent 创建** | `new MessageHandler(config)` — 含 `DefaultBackgroundContextManager` + `getAllTools()` + `filterToolsByMode` | `useAgent({ settings, modelFactory, storage, contextProviders, tools })` → `AIPex.create()` | 架构不同但功能对等 |
|
||||||
|
| **工具加载** | `getAllTools()` — 约 70+ MCP 工具 | `allBrowserTools` — 32 个注册工具 | **缺口：~50 个工具未暴露** |
|
||||||
|
| **Skill 初始化** | `skillManager.initialize()` + 订阅 `skill_enabled/skill_disabled` | VM 层初始化 skill；无显式事件订阅 | 事件订阅缺失 |
|
||||||
|
| **Intervention 初始化** | `interventionManager.initialize()` + 5 个事件监听 (start/complete/cancel/timeout/error) | `InterventionModeProvider` + `InterventionUI` 组件化 | 功能对等但实现方式不同 |
|
||||||
|
| **Pending Prompt** | `useEffect` 检查 `aipex-pending-prompt` → `setInput()` | `usePendingPrompt()` hook → `initialInput` prop | 一致 |
|
||||||
|
| **Port 长连接** | `chrome.runtime.connect({ name: 'sidepanel' })` | 同 | 一致 |
|
||||||
|
| **Visibility 清理** | `visibilitychange` → `stop-recording` + `stop-capture` | 同 | 一致 |
|
||||||
|
| **模型列表** | `fetchModels()` → 手动管理 `models` state + 自定义模型追加 | `fetchModelsForSelector()` 在 InputArea 内自动加载 + BYOK 自定义模型 | 新版更灵活 |
|
||||||
|
| **Conversation 呼吸边框** | heartbeat → `aipex-conversation-active` → content script 边框 | 同（`useConversationHeartbeat`） | 一致 |
|
||||||
|
| **流式输出** | `MessageHandler._startStream` — 手动 fetch + `StreamingParser` | `AIPex.chat()` → `AsyncGenerator<AgentEvent>` → `ChatAdapter` | 新版架构更清晰 |
|
||||||
|
| **会话持久化** | `ConversationStorage` (IndexedDB, LRU 5 条) | `conversationStorage` (IndexedDB, LRU 5 条) + debounced auto-save | 一致 |
|
||||||
|
| **会话历史 UI** | `ConversationHistory` 组件 (header 内) | `ConversationHistory` 组件 (BrowserChatHeader 内) | 一致 |
|
||||||
|
| **Replay 链路** | `NAVIGATE_AND_SETUP_REPLAY` 监听 → `UseCaseDetail` 页面 | 不存在 | **完全缺失** |
|
||||||
|
| **Chat 图片导出** | sidepanel 内 `provide-current-chat-images` listener | `ChatImagesListener` 组件 | 一致 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. 缺口清单
|
||||||
|
|
||||||
|
### GAP-01: Replay 外部触发链路缺失
|
||||||
|
|
||||||
|
| 属性 | 值 |
|
||||||
|
|------|-----|
|
||||||
|
| **现象** | 旧版支持外部网站发送 `REPLAY_USER_MANUAL` → background 转发 `NAVIGATE_AND_SETUP_REPLAY` → sidepanel 接收并跳转到 `UseCaseDetail`。新版 `background.ts` 和 sidepanel 中均不存在这两个消息类型。 |
|
||||||
|
| **证据** | 旧版 `aipex/src/background.ts` L3151-3199（`REPLAY_USER_MANUAL`）；`aipex/src/lib/components/chatbot/index.tsx` L1373-1424（`NAVIGATE_AND_SETUP_REPLAY` listener）。新版 `rg "REPLAY_USER_MANUAL\|NAVIGATE_AND_SETUP_REPLAY" new-aipex/` → 零匹配。 |
|
||||||
|
| **影响** | claudechrome.com 网站上的 "Replay Manual" 功能无法调起新版扩展。 |
|
||||||
|
| **优先级** | **P1** |
|
||||||
|
| **建议** | 在 `background.ts` `onMessageExternal` 和 sidepanel ChatApp 中补齐消息协议。 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### GAP-02: 未迁移的后台服务
|
||||||
|
|
||||||
|
| 属性 | 值 |
|
||||||
|
|------|-----|
|
||||||
|
| **现象** | 旧版多个与 sidepanel 功能关联的后台服务在新版中不存在。 |
|
||||||
|
| **证据** | `new-aipex/migration/LOGIC_INCONSISTENCIES_BY_PACKAGE.md` 第 4.2 节。 |
|
||||||
|
| **影响** | |
|
||||||
|
|
||||||
|
| 缺失服务 | 旧版路径 | 影响范围 |
|
||||||
|
|----------|---------|---------|
|
||||||
|
| `version-checker.ts` | `aipex/src/lib/services/version-checker.ts` | 无法检测扩展新版本并通知用户 |
|
||||||
|
| `web-auth.ts` | `aipex/src/lib/services/web-auth.ts` | 旧版 Cookie-based auth 辅助函数缺失（新版有 `AuthProvider` 替代，但功能不完全对等） |
|
||||||
|
| `recording-upload.ts` | `aipex/src/lib/services/recording-upload.ts` | 无法将录屏上传到托管服务 |
|
||||||
|
| `screenshot-upload.ts` | `aipex/src/lib/services/screenshot-upload.ts` | 无法将截图上传到托管服务 |
|
||||||
|
| `user-manuals-api.ts` | `aipex/src/lib/services/user-manuals-api.ts` | 无法从服务端获取用户手册数据 |
|
||||||
|
| `replay-controller.ts` | `aipex/src/lib/services/replay-controller.ts` | 无法执行步骤回放控制 |
|
||||||
|
|
||||||
|
| **优先级** | **P1**（auth 相关）/ **P2**（其余） |
|
||||||
|
| **建议** | 按依赖顺序逐步迁移到 `packages/browser-ext/src/services/` 或新建 `packages/services`。 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### GAP-03: Voice 输入仅支持 Web Speech API
|
||||||
|
|
||||||
|
| 属性 | 值 |
|
||||||
|
|------|-----|
|
||||||
|
| **现象** | 旧版支持三层 STT 降级：Server STT → ElevenLabs → Web Speech API。新版仅保留 Web Speech API。 |
|
||||||
|
| **证据** | `new-aipex/migration/LOGIC_INCONSISTENCIES_BY_PACKAGE.md` 第 2.7 节。新版 `packages/aipex-react/src/components/voice/VoiceInput.tsx` 和 `packages/browser-ext/src/lib/browser-chat-input-area.tsx` 仅使用 Web Speech。 |
|
||||||
|
| **影响** | 非英语语言识别准确率下降；BYOK 用户无法使用 ElevenLabs 高精度识别。 |
|
||||||
|
| **优先级** | **P1** |
|
||||||
|
| **建议** | 在 `packages/aipex-react/src/lib/voice/` 中恢复 ElevenLabs + Server STT 路径。 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### GAP-04: Install 时自动打开引导页缺失
|
||||||
|
|
||||||
|
| 属性 | 值 |
|
||||||
|
|------|-----|
|
||||||
|
| **现象** | 旧版 `onInstalled` 在 production 环境下会自动打开 `https://www.claudechrome.com` 引导用户。新版仅打日志。 |
|
||||||
|
| **证据** | 旧版 `aipex/src/background.ts` L1082-1089；新版 `new-aipex/packages/browser-ext/src/background.ts` L31-39。 |
|
||||||
|
| **影响** | 新用户安装后无引导体验。 |
|
||||||
|
| **优先级** | **P2** |
|
||||||
|
| **建议** | 在 `onInstalled` install 分支中恢复引导页打开逻辑。 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### GAP-05: React ErrorBoundary 缺失
|
||||||
|
|
||||||
|
| 属性 | 值 |
|
||||||
|
|------|-----|
|
||||||
|
| **现象** | 新版 sidepanel 的 React 组件树中没有包裹 `ErrorBoundary`。任何渲染错误会导致整个 sidepanel 白屏。 |
|
||||||
|
| **证据** | `rg "ErrorBoundary" new-aipex/` → 零匹配。旧版同样缺少，但旧版组件更少、更稳定。 |
|
||||||
|
| **影响** | Agent 工具回调中的渲染错误、异步组件加载失败等场景会导致不可恢复的白屏。 |
|
||||||
|
| **优先级** | **P1** |
|
||||||
|
| **建议** | 在 `app-root.tsx` 的 `<App />` 外层包裹 `ErrorBoundary`，提供友好的错误恢复 UI。 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### GAP-06: Skill 系统事件订阅缺失
|
||||||
|
|
||||||
|
| 属性 | 值 |
|
||||||
|
|------|-----|
|
||||||
|
| **现象** | 旧版在 ChatBot 初始化时显式订阅 `skillManager` 的 `skill_enabled` / `skill_disabled` 事件。新版 VM 初始化了 skill 系统，但 sidepanel UI 层未订阅 skill 变更事件。 |
|
||||||
|
| **证据** | 旧版 `aipex/src/lib/components/chatbot/index.tsx` L1313-1325（`skillManager.subscribe`）。新版 `app-root.tsx` 和 `chatbot.tsx` 中无 skill 事件订阅。 |
|
||||||
|
| **影响** | Skill 启用/禁用后 UI 可能无法实时反映变化（例如 skill tag 不更新）。 |
|
||||||
|
| **优先级** | **P2** |
|
||||||
|
| **建议** | 在 ChatApp 或 ChatBot 内添加 `skillManager` 事件订阅并传播到 UI。 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### GAP-07: Share（分享对话）功能缺失
|
||||||
|
|
||||||
|
| 属性 | 值 |
|
||||||
|
|------|-----|
|
||||||
|
| **现象** | 旧版 sidepanel header 包含"分享"和"保存为 Skill"按钮。新版 `BrowserChatHeader` 中无此功能。 |
|
||||||
|
| **证据** | 旧版 `aipex/src/lib/components/chatbot/index.tsx` 中存在 `handleShareConversation`（L1744-1793）和 `handleSaveAsSkill`（L1796-1798）。新版 `browser-chat-header.tsx` 中无对应功能。 |
|
||||||
|
| **影响** | 用户无法分享当前对话或将对话保存为可复用的 Skill。 |
|
||||||
|
| **优先级** | **P2** |
|
||||||
|
| **建议** | 在 `BrowserChatHeader` 中添加分享与 Skill 保存入口。 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. 缺口优先级汇总
|
||||||
|
|
||||||
|
| 优先级 | 缺口编号 | 简述 |
|
||||||
|
|--------|---------|------|
|
||||||
|
| **P1** | GAP-01 | Replay 外部触发链路缺失 |
|
||||||
|
| **P1** | GAP-03 | Voice 输入仅 Web Speech API |
|
||||||
|
| **P1** | GAP-05 | React ErrorBoundary 缺失 |
|
||||||
|
| **P2** | GAP-02 | 未迁移的后台服务 |
|
||||||
|
| **P2** | GAP-04 | Install 引导页缺失 |
|
||||||
|
| **P2** | GAP-06 | Skill 事件订阅缺失 |
|
||||||
|
| **P2** | GAP-07 | 分享 / 保存为 Skill 功能缺失 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. 补齐路线建议
|
||||||
|
|
||||||
|
### Phase 1（P1 — 功能完整性）
|
||||||
|
|
||||||
|
1. **补齐 Replay 外部触发** — background 中处理 `REPLAY_USER_MANUAL`，sidepanel 中监听 `NAVIGATE_AND_SETUP_REPLAY`。
|
||||||
|
2. **恢复 Voice 多层 STT** — 在 voice engine 中添加 ElevenLabs 和 Server STT 路径。
|
||||||
|
3. **添加 ErrorBoundary** — `app-root.tsx` 顶层包裹。
|
||||||
|
|
||||||
|
### Phase 2（P2 — 体验优化）
|
||||||
|
|
||||||
|
4. Install 引导页、Skill 事件订阅、分享/Skill 保存、后台服务迁移。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. 安全评审卡
|
||||||
|
|
||||||
|
### 6.1 入口与信任边界
|
||||||
|
|
||||||
|
| 入口 | 信任级别 | 校验机制 |
|
||||||
|
|------|---------|---------|
|
||||||
|
| `chrome.action.onClicked` | 可信（用户主动触发） | 无需额外校验 |
|
||||||
|
| `chrome.commands` 快捷键 | 可信（用户主动触发） | 无需额外校验 |
|
||||||
|
| Content script → `open-sidepanel` | 半可信（content script 上下文） | Background 仅执行 `sidePanel.open`，不传递数据 |
|
||||||
|
| `onMessageExternal` (`openWithPrompt`) | 外部来源 | `manifest.json` `externally_connectable` 限制来源域名；prompt 类型校验 (`typeof === "string"`)；timestamp 过期丢弃（5s TTL）|
|
||||||
|
| `onMessageExternal` (`REPLAY_USER_MANUAL`) | 外部来源 | **旧版已有** — `externally_connectable` 域名白名单；**新版缺失** — 需补齐时同步实现。 |
|
||||||
|
|
||||||
|
### 6.2 外部消息校验路径
|
||||||
|
|
||||||
|
- `openWithPrompt`：prompt 必须为 `string` 类型 + 非空；通过 `chrome.storage.local` 中转（非直接注入 UI），sidepanel 消费时检查 timestamp（<5s）。**状态：已实现，安全。**
|
||||||
|
- `REPLAY_USER_MANUAL`：**新版缺失**，补齐时需确保 `steps` 数组做 schema 校验，防止注入恶意步骤数据。
|
||||||
|
|
||||||
|
### 6.3 敏感数据与日志
|
||||||
|
|
||||||
|
- `aiToken` 存储在 `chrome.storage.local`，UI 中未展示明文。新版 `AIPex.sanitizeErrorMessage()` 内建 Bearer token / API key 脱敏。**状态：安全。**
|
||||||
|
- 旧版大量 `console.log` 包含 emoji debug 信息，可能泄露会话内容。新版日志更精简。**建议**：production build 时移除或降级 debug 日志。
|
||||||
|
|
||||||
|
### 6.4 高风险工具默认策略
|
||||||
|
|
||||||
|
- 旧版所有 MCP 工具默认暴露；新版通过 `allBrowserTools` 白名单控制，且 `filterToolsByMode` 在 background 模式下过滤 computer/screenshot 工具。**状态：新版更安全。**
|
||||||
|
|
||||||
|
### 6.5 待验证测试点
|
||||||
|
|
||||||
|
- [ ] `ErrorBoundary` 添加后的错误恢复行为
|
||||||
|
- [ ] `REPLAY_USER_MANUAL` 补齐后的 steps 数据校验
|
||||||
|
- [ ] Production build 日志级别控制
|
||||||
|
- [ ] BYOK 自定义模型的 `aiHost` URL 校验（防 SSRF）
|
||||||
|
- [ ] `externally_connectable` 域名列表与实际部署域名一致性
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
> **交叉引用**：本文档与以下已有迁移文档互补
|
||||||
|
> - `migration/TOOL_SURFACE_AUDIT.md` — 工具级对比详表
|
||||||
|
> - `migration/LOGIC_INCONSISTENCIES_BY_PACKAGE.md` — 按包的逻辑差异
|
||||||
|
> - `migration/MIGRATION_STRATEGY.md` — 整体迁移规划
|
||||||
102
packages/aipex-react/src/components/error/ErrorBoundary.test.tsx
Normal file
102
packages/aipex-react/src/components/error/ErrorBoundary.test.tsx
Normal file
@@ -0,0 +1,102 @@
|
|||||||
|
import { fireEvent, render, screen } from "@testing-library/react";
|
||||||
|
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||||
|
import { ErrorBoundary } from "./ErrorBoundary";
|
||||||
|
|
||||||
|
// Component that throws on render
|
||||||
|
function ThrowingComponent({ shouldThrow }: { shouldThrow: boolean }) {
|
||||||
|
if (shouldThrow) {
|
||||||
|
throw new Error("Test rendering error");
|
||||||
|
}
|
||||||
|
return <div>Normal content</div>;
|
||||||
|
}
|
||||||
|
|
||||||
|
describe("ErrorBoundary", () => {
|
||||||
|
// Suppress React error boundary console.error during tests
|
||||||
|
const originalConsoleError = console.error;
|
||||||
|
beforeEach(() => {
|
||||||
|
console.error = vi.fn();
|
||||||
|
});
|
||||||
|
afterEach(() => {
|
||||||
|
console.error = originalConsoleError;
|
||||||
|
});
|
||||||
|
|
||||||
|
it("renders children when no error occurs", () => {
|
||||||
|
render(
|
||||||
|
<ErrorBoundary>
|
||||||
|
<div>Hello World</div>
|
||||||
|
</ErrorBoundary>,
|
||||||
|
);
|
||||||
|
expect(screen.getByText("Hello World")).toBeTruthy();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("renders fallback UI when a child throws", () => {
|
||||||
|
render(
|
||||||
|
<ErrorBoundary>
|
||||||
|
<ThrowingComponent shouldThrow={true} />
|
||||||
|
</ErrorBoundary>,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(screen.getByText("Something went wrong")).toBeTruthy();
|
||||||
|
expect(screen.getByText("Try Again")).toBeTruthy();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("shows error details in expandable section", () => {
|
||||||
|
render(
|
||||||
|
<ErrorBoundary>
|
||||||
|
<ThrowingComponent shouldThrow={true} />
|
||||||
|
</ErrorBoundary>,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(screen.getByText("Error details")).toBeTruthy();
|
||||||
|
expect(screen.getByText("Test rendering error")).toBeTruthy();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("resets error state when Try Again is clicked", () => {
|
||||||
|
// Use a stateful wrapper so we can control the throw behavior
|
||||||
|
let shouldThrow = true;
|
||||||
|
function ConditionalThrower() {
|
||||||
|
if (shouldThrow) throw new Error("Boom");
|
||||||
|
return <div>Recovered</div>;
|
||||||
|
}
|
||||||
|
|
||||||
|
render(
|
||||||
|
<ErrorBoundary>
|
||||||
|
<ConditionalThrower />
|
||||||
|
</ErrorBoundary>,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(screen.getByText("Something went wrong")).toBeTruthy();
|
||||||
|
|
||||||
|
// Stop throwing before clicking retry
|
||||||
|
shouldThrow = false;
|
||||||
|
fireEvent.click(screen.getByText("Try Again"));
|
||||||
|
|
||||||
|
expect(screen.getByText("Recovered")).toBeTruthy();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("renders custom fallback when provided", () => {
|
||||||
|
render(
|
||||||
|
<ErrorBoundary fallback={<div>Custom error UI</div>}>
|
||||||
|
<ThrowingComponent shouldThrow={true} />
|
||||||
|
</ErrorBoundary>,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(screen.getByText("Custom error UI")).toBeTruthy();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("calls onError callback when an error is caught", () => {
|
||||||
|
const onError = vi.fn();
|
||||||
|
|
||||||
|
render(
|
||||||
|
<ErrorBoundary onError={onError}>
|
||||||
|
<ThrowingComponent shouldThrow={true} />
|
||||||
|
</ErrorBoundary>,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(onError).toHaveBeenCalledTimes(1);
|
||||||
|
expect(onError).toHaveBeenCalledWith(
|
||||||
|
expect.objectContaining({ message: "Test rendering error" }),
|
||||||
|
expect.objectContaining({ componentStack: expect.any(String) }),
|
||||||
|
);
|
||||||
|
});
|
||||||
|
});
|
||||||
150
packages/aipex-react/src/components/error/ErrorBoundary.tsx
Normal file
150
packages/aipex-react/src/components/error/ErrorBoundary.tsx
Normal file
@@ -0,0 +1,150 @@
|
|||||||
|
/**
|
||||||
|
* ErrorBoundary
|
||||||
|
* Catches React rendering errors and displays a recovery UI
|
||||||
|
* instead of an unrecoverable white screen.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import React from "react";
|
||||||
|
|
||||||
|
export interface ErrorBoundaryProps {
|
||||||
|
children: React.ReactNode;
|
||||||
|
/** Optional fallback UI to render when an error is caught. */
|
||||||
|
fallback?: React.ReactNode;
|
||||||
|
/** Optional callback invoked when an error is caught. */
|
||||||
|
onError?: (error: Error, errorInfo: React.ErrorInfo) => void;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface ErrorBoundaryState {
|
||||||
|
hasError: boolean;
|
||||||
|
error: Error | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
export class ErrorBoundary extends React.Component<
|
||||||
|
ErrorBoundaryProps,
|
||||||
|
ErrorBoundaryState
|
||||||
|
> {
|
||||||
|
constructor(props: ErrorBoundaryProps) {
|
||||||
|
super(props);
|
||||||
|
this.state = { hasError: false, error: null };
|
||||||
|
}
|
||||||
|
|
||||||
|
static getDerivedStateFromError(error: Error): ErrorBoundaryState {
|
||||||
|
return { hasError: true, error };
|
||||||
|
}
|
||||||
|
|
||||||
|
componentDidCatch(error: Error, errorInfo: React.ErrorInfo): void {
|
||||||
|
// Log error without exposing sensitive data
|
||||||
|
console.error(
|
||||||
|
"[ErrorBoundary] Uncaught rendering error:",
|
||||||
|
error.message,
|
||||||
|
);
|
||||||
|
|
||||||
|
this.props.onError?.(error, errorInfo);
|
||||||
|
}
|
||||||
|
|
||||||
|
private handleRetry = (): void => {
|
||||||
|
this.setState({ hasError: false, error: null });
|
||||||
|
};
|
||||||
|
|
||||||
|
render(): React.ReactNode {
|
||||||
|
if (this.state.hasError) {
|
||||||
|
if (this.props.fallback) {
|
||||||
|
return this.props.fallback;
|
||||||
|
}
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div
|
||||||
|
style={{
|
||||||
|
display: "flex",
|
||||||
|
flexDirection: "column",
|
||||||
|
alignItems: "center",
|
||||||
|
justifyContent: "center",
|
||||||
|
height: "100%",
|
||||||
|
padding: "24px",
|
||||||
|
fontFamily:
|
||||||
|
'-apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif',
|
||||||
|
textAlign: "center",
|
||||||
|
color: "var(--foreground, #333)",
|
||||||
|
backgroundColor: "var(--background, #fff)",
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<div
|
||||||
|
style={{
|
||||||
|
fontSize: "48px",
|
||||||
|
marginBottom: "16px",
|
||||||
|
lineHeight: 1,
|
||||||
|
}}
|
||||||
|
aria-hidden="true"
|
||||||
|
>
|
||||||
|
!
|
||||||
|
</div>
|
||||||
|
<h2
|
||||||
|
style={{
|
||||||
|
fontSize: "18px",
|
||||||
|
fontWeight: 600,
|
||||||
|
margin: "0 0 8px",
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
Something went wrong
|
||||||
|
</h2>
|
||||||
|
<p
|
||||||
|
style={{
|
||||||
|
fontSize: "14px",
|
||||||
|
color: "var(--muted-foreground, #666)",
|
||||||
|
margin: "0 0 20px",
|
||||||
|
maxWidth: "320px",
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
An unexpected error occurred. Click the button below to try
|
||||||
|
recovering.
|
||||||
|
</p>
|
||||||
|
<button
|
||||||
|
type="button"
|
||||||
|
onClick={this.handleRetry}
|
||||||
|
style={{
|
||||||
|
padding: "8px 20px",
|
||||||
|
fontSize: "14px",
|
||||||
|
fontWeight: 500,
|
||||||
|
borderRadius: "6px",
|
||||||
|
border: "1px solid var(--border, #ddd)",
|
||||||
|
backgroundColor: "var(--primary, #333)",
|
||||||
|
color: "var(--primary-foreground, #fff)",
|
||||||
|
cursor: "pointer",
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
Try Again
|
||||||
|
</button>
|
||||||
|
{this.state.error && (
|
||||||
|
<details
|
||||||
|
style={{
|
||||||
|
marginTop: "16px",
|
||||||
|
fontSize: "12px",
|
||||||
|
color: "var(--muted-foreground, #888)",
|
||||||
|
maxWidth: "400px",
|
||||||
|
textAlign: "left",
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<summary style={{ cursor: "pointer" }}>Error details</summary>
|
||||||
|
<pre
|
||||||
|
style={{
|
||||||
|
marginTop: "8px",
|
||||||
|
padding: "8px",
|
||||||
|
borderRadius: "4px",
|
||||||
|
backgroundColor: "var(--muted, #f5f5f5)",
|
||||||
|
overflow: "auto",
|
||||||
|
whiteSpace: "pre-wrap",
|
||||||
|
wordBreak: "break-word",
|
||||||
|
maxHeight: "120px",
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{this.state.error.message}
|
||||||
|
</pre>
|
||||||
|
</details>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
return this.props.children;
|
||||||
|
}
|
||||||
|
}
|
||||||
1
packages/aipex-react/src/components/error/index.ts
Normal file
1
packages/aipex-react/src/components/error/index.ts
Normal file
@@ -0,0 +1 @@
|
|||||||
|
export { ErrorBoundary, type ErrorBoundaryProps } from "./ErrorBoundary";
|
||||||
@@ -31,6 +31,11 @@ chrome.commands.onCommand.addListener((command) => {
|
|||||||
chrome.runtime.onInstalled.addListener((details) => {
|
chrome.runtime.onInstalled.addListener((details) => {
|
||||||
if (details.reason === "install") {
|
if (details.reason === "install") {
|
||||||
console.log("AIPex extension installed");
|
console.log("AIPex extension installed");
|
||||||
|
|
||||||
|
// Open onboarding page for new installs in production
|
||||||
|
if (import.meta.env.PROD) {
|
||||||
|
chrome.tabs.create({ url: "https://www.claudechrome.com" });
|
||||||
|
}
|
||||||
} else if (details.reason === "update") {
|
} else if (details.reason === "update") {
|
||||||
console.log(
|
console.log(
|
||||||
"AIPex extension updated to version",
|
"AIPex extension updated to version",
|
||||||
@@ -530,6 +535,102 @@ chrome.runtime.onMessageExternal.addListener(
|
|||||||
return true; // Keep message channel open for async response
|
return true; // Keep message channel open for async response
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Handle user manual replay request from website
|
||||||
|
if (message.request === "REPLAY_USER_MANUAL") {
|
||||||
|
const { manualId, startFromStep, steps } = message as {
|
||||||
|
manualId?: unknown;
|
||||||
|
startFromStep?: unknown;
|
||||||
|
steps?: unknown;
|
||||||
|
};
|
||||||
|
|
||||||
|
// Validate required fields
|
||||||
|
if (
|
||||||
|
typeof manualId !== "number" ||
|
||||||
|
!Array.isArray(steps) ||
|
||||||
|
steps.length === 0
|
||||||
|
) {
|
||||||
|
sendResponse({
|
||||||
|
success: false,
|
||||||
|
error: "Invalid replay data: manualId (number) and non-empty steps (array) are required",
|
||||||
|
});
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Validate step entries have required shape and bounded size
|
||||||
|
const MAX_STEPS = 500;
|
||||||
|
if (steps.length > MAX_STEPS) {
|
||||||
|
sendResponse({
|
||||||
|
success: false,
|
||||||
|
error: `Too many replay steps (max ${MAX_STEPS})`,
|
||||||
|
});
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
const ALLOWED_EVENT_TYPES = ["click", "navigation"];
|
||||||
|
const stepsValid = steps.every((s: unknown) => {
|
||||||
|
if (s === null || typeof s !== "object") return false;
|
||||||
|
const rec = s as Record<string, unknown>;
|
||||||
|
if (!rec.event || typeof rec.event !== "object") return false;
|
||||||
|
const event = rec.event as Record<string, unknown>;
|
||||||
|
return (
|
||||||
|
typeof event.type === "string" &&
|
||||||
|
ALLOWED_EVENT_TYPES.includes(event.type)
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
if (!stepsValid) {
|
||||||
|
sendResponse({
|
||||||
|
success: false,
|
||||||
|
error:
|
||||||
|
"Invalid replay steps: each step must contain an event with type 'click' or 'navigation'",
|
||||||
|
});
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
const resolvedStartFromStep =
|
||||||
|
typeof startFromStep === "number" && startFromStep >= 0
|
||||||
|
? startFromStep
|
||||||
|
: 0;
|
||||||
|
|
||||||
|
// Open sidepanel then forward replay data
|
||||||
|
const windowId = sender.tab?.windowId;
|
||||||
|
|
||||||
|
if (!windowId) {
|
||||||
|
sendResponse({ success: false, error: "No window ID available" });
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
chrome.sidePanel
|
||||||
|
.open({ windowId })
|
||||||
|
.then(() => {
|
||||||
|
// Wait for sidepanel to initialize before forwarding
|
||||||
|
setTimeout(() => {
|
||||||
|
chrome.runtime
|
||||||
|
.sendMessage({
|
||||||
|
request: "NAVIGATE_AND_SETUP_REPLAY",
|
||||||
|
data: {
|
||||||
|
manualId,
|
||||||
|
startFromStep: resolvedStartFromStep,
|
||||||
|
steps,
|
||||||
|
},
|
||||||
|
})
|
||||||
|
.catch(() => {
|
||||||
|
// Sidepanel may not yet have a listener – acceptable race
|
||||||
|
});
|
||||||
|
}, 500);
|
||||||
|
|
||||||
|
sendResponse({ success: true });
|
||||||
|
})
|
||||||
|
.catch((error) => {
|
||||||
|
sendResponse({
|
||||||
|
success: false,
|
||||||
|
error: error instanceof Error ? error.message : String(error),
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
sendResponse({ success: false, error: "Unknown action" });
|
sendResponse({ success: false, error: "Unknown action" });
|
||||||
return true;
|
return true;
|
||||||
},
|
},
|
||||||
|
|||||||
113
packages/browser-ext/src/lib/ai-provider.test.ts
Normal file
113
packages/browser-ext/src/lib/ai-provider.test.ts
Normal file
@@ -0,0 +1,113 @@
|
|||||||
|
import { describe, expect, it, vi } from "vitest";
|
||||||
|
import { createAIProvider } from "./ai-provider";
|
||||||
|
|
||||||
|
// Provide minimal mock for import.meta.env
|
||||||
|
vi.stubGlobal("import", { meta: { env: { PROD: false } } });
|
||||||
|
|
||||||
|
describe("createAIProvider", () => {
|
||||||
|
describe("URL validation", () => {
|
||||||
|
it("accepts valid https URLs", () => {
|
||||||
|
expect(() =>
|
||||||
|
createAIProvider({
|
||||||
|
aiProvider: "openai",
|
||||||
|
aiToken: "sk-test",
|
||||||
|
aiHost: "https://api.openai.com/v1",
|
||||||
|
}),
|
||||||
|
).not.toThrow();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("accepts undefined aiHost", () => {
|
||||||
|
expect(() =>
|
||||||
|
createAIProvider({
|
||||||
|
aiProvider: "openai",
|
||||||
|
aiToken: "sk-test",
|
||||||
|
}),
|
||||||
|
).not.toThrow();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("accepts empty string aiHost", () => {
|
||||||
|
expect(() =>
|
||||||
|
createAIProvider({
|
||||||
|
aiProvider: "openai",
|
||||||
|
aiToken: "sk-test",
|
||||||
|
aiHost: "",
|
||||||
|
}),
|
||||||
|
).not.toThrow();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects invalid URLs", () => {
|
||||||
|
expect(() =>
|
||||||
|
createAIProvider({
|
||||||
|
aiProvider: "openai",
|
||||||
|
aiToken: "sk-test",
|
||||||
|
aiHost: "not-a-url",
|
||||||
|
}),
|
||||||
|
).toThrow("Invalid aiHost URL");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects non-http protocols", () => {
|
||||||
|
expect(() =>
|
||||||
|
createAIProvider({
|
||||||
|
aiProvider: "openai",
|
||||||
|
aiToken: "sk-test",
|
||||||
|
aiHost: "ftp://evil.com",
|
||||||
|
}),
|
||||||
|
).toThrow("Unsupported protocol");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects javascript: protocol", () => {
|
||||||
|
expect(() =>
|
||||||
|
createAIProvider({
|
||||||
|
aiProvider: "openai",
|
||||||
|
aiToken: "sk-test",
|
||||||
|
// eslint-disable-next-line no-script-url
|
||||||
|
aiHost: "javascript:alert(1)",
|
||||||
|
}),
|
||||||
|
).toThrow("Unsupported protocol");
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("provider creation", () => {
|
||||||
|
it("creates openai provider by default", () => {
|
||||||
|
const provider = createAIProvider({
|
||||||
|
aiProvider: "openai",
|
||||||
|
aiToken: "sk-test",
|
||||||
|
});
|
||||||
|
expect(provider).toBeDefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("creates anthropic provider", () => {
|
||||||
|
const provider = createAIProvider({
|
||||||
|
aiProvider: "anthropic",
|
||||||
|
aiToken: "sk-test",
|
||||||
|
});
|
||||||
|
expect(provider).toBeDefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("creates google provider", () => {
|
||||||
|
const provider = createAIProvider({
|
||||||
|
aiProvider: "google",
|
||||||
|
aiToken: "sk-test",
|
||||||
|
});
|
||||||
|
expect(provider).toBeDefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("requires baseURL for custom providers", () => {
|
||||||
|
expect(() =>
|
||||||
|
createAIProvider({
|
||||||
|
aiProvider: "custom" as any,
|
||||||
|
aiToken: "sk-test",
|
||||||
|
}),
|
||||||
|
).toThrow("requires aiHost");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("creates custom provider with valid baseURL", () => {
|
||||||
|
const provider = createAIProvider({
|
||||||
|
aiProvider: "custom" as any,
|
||||||
|
aiToken: "sk-test",
|
||||||
|
aiHost: "https://my-proxy.example.com",
|
||||||
|
});
|
||||||
|
expect(provider).toBeDefined();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -31,10 +31,52 @@ export interface ProviderConfig {
|
|||||||
* const model = provider("gpt-4");
|
* const model = provider("gpt-4");
|
||||||
* ```
|
* ```
|
||||||
*/
|
*/
|
||||||
|
/**
|
||||||
|
* Validate that a user-provided host URL is safe to use.
|
||||||
|
* Rejects private/internal addresses to mitigate SSRF risks.
|
||||||
|
*/
|
||||||
|
function validateHostUrl(url: string | undefined): string | undefined {
|
||||||
|
if (!url) return undefined;
|
||||||
|
|
||||||
|
let parsed: URL;
|
||||||
|
try {
|
||||||
|
parsed = new URL(url);
|
||||||
|
} catch {
|
||||||
|
throw new Error(`Invalid aiHost URL: ${url}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Only allow http/https schemes
|
||||||
|
if (parsed.protocol !== "https:" && parsed.protocol !== "http:") {
|
||||||
|
throw new Error(
|
||||||
|
`Unsupported protocol in aiHost: ${parsed.protocol} (only http/https allowed)`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Block common internal/private hostnames
|
||||||
|
const hostname = parsed.hostname.toLowerCase();
|
||||||
|
const blocked = [
|
||||||
|
"localhost",
|
||||||
|
"127.0.0.1",
|
||||||
|
"0.0.0.0",
|
||||||
|
"[::1]",
|
||||||
|
"metadata.google.internal",
|
||||||
|
"169.254.169.254",
|
||||||
|
];
|
||||||
|
|
||||||
|
// In production, block private addresses
|
||||||
|
if (import.meta.env.PROD && blocked.includes(hostname)) {
|
||||||
|
throw new Error(
|
||||||
|
`aiHost points to a restricted address: ${hostname}`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
return parsed.origin + parsed.pathname.replace(/\/+$/, "");
|
||||||
|
}
|
||||||
|
|
||||||
export function createAIProvider(settings: AppSettings) {
|
export function createAIProvider(settings: AppSettings) {
|
||||||
const provider = settings.aiProvider ?? "openai";
|
const provider = settings.aiProvider ?? "openai";
|
||||||
const apiKey = settings.aiToken ?? "";
|
const apiKey = settings.aiToken ?? "";
|
||||||
const baseURL = settings.aiHost || undefined;
|
const baseURL = validateHostUrl(settings.aiHost || undefined);
|
||||||
|
|
||||||
switch (provider) {
|
switch (provider) {
|
||||||
case "anthropic":
|
case "anthropic":
|
||||||
|
|||||||
@@ -5,14 +5,30 @@
|
|||||||
|
|
||||||
import { useChatContext } from "@aipexstudio/aipex-react/components/chatbot";
|
import { useChatContext } from "@aipexstudio/aipex-react/components/chatbot";
|
||||||
import { Button } from "@aipexstudio/aipex-react/components/ui/button";
|
import { Button } from "@aipexstudio/aipex-react/components/ui/button";
|
||||||
|
import {
|
||||||
|
DropdownMenu,
|
||||||
|
DropdownMenuContent,
|
||||||
|
DropdownMenuItem,
|
||||||
|
DropdownMenuSeparator,
|
||||||
|
DropdownMenuTrigger,
|
||||||
|
} from "@aipexstudio/aipex-react/components/ui/dropdown-menu";
|
||||||
import { useTranslation } from "@aipexstudio/aipex-react/i18n/context";
|
import { useTranslation } from "@aipexstudio/aipex-react/i18n/context";
|
||||||
import { getRuntime } from "@aipexstudio/aipex-react/lib/runtime";
|
import { getRuntime } from "@aipexstudio/aipex-react/lib/runtime";
|
||||||
import { cn } from "@aipexstudio/aipex-react/lib/utils";
|
import { cn } from "@aipexstudio/aipex-react/lib/utils";
|
||||||
import type { HeaderProps } from "@aipexstudio/aipex-react/types";
|
import type { HeaderProps } from "@aipexstudio/aipex-react/types";
|
||||||
import { conversationStorage } from "@aipexstudio/browser-runtime";
|
import { conversationStorage } from "@aipexstudio/browser-runtime";
|
||||||
import { KeyboardIcon, MicIcon, PlusIcon, SettingsIcon } from "lucide-react";
|
import {
|
||||||
|
KeyboardIcon,
|
||||||
|
MicIcon,
|
||||||
|
MoreHorizontalIcon,
|
||||||
|
PlusIcon,
|
||||||
|
SettingsIcon,
|
||||||
|
Share2Icon,
|
||||||
|
SparklesIcon,
|
||||||
|
} from "lucide-react";
|
||||||
import { useCallback, useEffect, useRef, useState } from "react";
|
import { useCallback, useEffect, useRef, useState } from "react";
|
||||||
import { UserProfile, useAuth } from "../auth";
|
import { UserProfile, useAuth } from "../auth";
|
||||||
|
import { shareConversation } from "../services/share-conversation";
|
||||||
import { ConversationHistory } from "./conversation-history";
|
import { ConversationHistory } from "./conversation-history";
|
||||||
import { useInputMode } from "./input-mode-context";
|
import { useInputMode } from "./input-mode-context";
|
||||||
import { fromStorageFormat, toStorageFormat } from "./message-adapter";
|
import { fromStorageFormat, toStorageFormat } from "./message-adapter";
|
||||||
@@ -27,7 +43,7 @@ export function BrowserChatHeader({
|
|||||||
}: HeaderProps) {
|
}: HeaderProps) {
|
||||||
const { t } = useTranslation();
|
const { t } = useTranslation();
|
||||||
const runtime = getRuntime();
|
const runtime = getRuntime();
|
||||||
const { messages, setMessages, interrupt } = useChatContext();
|
const { messages, setMessages, interrupt, sendMessage } = useChatContext();
|
||||||
const { user, login, isLoading: isAuthLoading } = useAuth();
|
const { user, login, isLoading: isAuthLoading } = useAuth();
|
||||||
|
|
||||||
const [currentConversationId, setCurrentConversationId] = useState<
|
const [currentConversationId, setCurrentConversationId] = useState<
|
||||||
@@ -133,6 +149,34 @@ export function BrowserChatHeader({
|
|||||||
setInputMode(inputMode === "voice" ? "text" : "voice");
|
setInputMode(inputMode === "voice" ? "text" : "voice");
|
||||||
}, [inputMode, setInputMode]);
|
}, [inputMode, setInputMode]);
|
||||||
|
|
||||||
|
// Share conversation
|
||||||
|
const [isSharing, setIsSharing] = useState(false);
|
||||||
|
|
||||||
|
const handleShare = useCallback(async () => {
|
||||||
|
if (isSharing) return;
|
||||||
|
|
||||||
|
const nonSystemMessages = messages.filter((m) => m.role !== "system");
|
||||||
|
if (nonSystemMessages.length === 0) return;
|
||||||
|
|
||||||
|
setIsSharing(true);
|
||||||
|
try {
|
||||||
|
const { url } = await shareConversation(messages);
|
||||||
|
chrome.tabs.create({ url });
|
||||||
|
} catch (error) {
|
||||||
|
console.error(
|
||||||
|
"[Share] Failed:",
|
||||||
|
error instanceof Error ? error.message : String(error),
|
||||||
|
);
|
||||||
|
} finally {
|
||||||
|
setIsSharing(false);
|
||||||
|
}
|
||||||
|
}, [messages, isSharing]);
|
||||||
|
|
||||||
|
// Save as Skill — instructs the AI to create a skill from the conversation
|
||||||
|
const handleSaveAsSkill = useCallback(() => {
|
||||||
|
sendMessage("use skill-creator skill to save the conversation");
|
||||||
|
}, [sendMessage]);
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<div
|
<div
|
||||||
className={cn(
|
className={cn(
|
||||||
@@ -180,8 +224,39 @@ export function BrowserChatHeader({
|
|||||||
/>
|
/>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
{/* Right side - New Chat and User Profile */}
|
{/* Right side - More menu, New Chat, User Profile */}
|
||||||
<div className="flex items-center gap-1">
|
<div className="flex items-center gap-1">
|
||||||
|
{/* More actions dropdown (Share, Save as Skill) */}
|
||||||
|
<DropdownMenu>
|
||||||
|
<DropdownMenuTrigger asChild>
|
||||||
|
<Button variant="ghost" size="icon" className="size-8">
|
||||||
|
<MoreHorizontalIcon className="size-4" />
|
||||||
|
</Button>
|
||||||
|
</DropdownMenuTrigger>
|
||||||
|
<DropdownMenuContent align="end" className="w-48">
|
||||||
|
<DropdownMenuItem
|
||||||
|
onClick={handleShare}
|
||||||
|
disabled={
|
||||||
|
isSharing ||
|
||||||
|
messages.filter((m) => m.role !== "system").length === 0
|
||||||
|
}
|
||||||
|
>
|
||||||
|
<Share2Icon className="mr-2 size-4" />
|
||||||
|
{isSharing ? "Sharing..." : "Share Conversation"}
|
||||||
|
</DropdownMenuItem>
|
||||||
|
<DropdownMenuSeparator />
|
||||||
|
<DropdownMenuItem
|
||||||
|
onClick={handleSaveAsSkill}
|
||||||
|
disabled={
|
||||||
|
messages.filter((m) => m.role !== "system").length === 0
|
||||||
|
}
|
||||||
|
>
|
||||||
|
<SparklesIcon className="mr-2 size-4" />
|
||||||
|
Save as Skill
|
||||||
|
</DropdownMenuItem>
|
||||||
|
</DropdownMenuContent>
|
||||||
|
</DropdownMenu>
|
||||||
|
|
||||||
<Button
|
<Button
|
||||||
variant="ghost"
|
variant="ghost"
|
||||||
size="sm"
|
size="sm"
|
||||||
|
|||||||
@@ -10,6 +10,7 @@ import { I18nProvider } from "@aipexstudio/aipex-react/i18n/context";
|
|||||||
import type { Language } from "@aipexstudio/aipex-react/i18n/types";
|
import type { Language } from "@aipexstudio/aipex-react/i18n/types";
|
||||||
import { ThemeProvider } from "@aipexstudio/aipex-react/theme/context";
|
import { ThemeProvider } from "@aipexstudio/aipex-react/theme/context";
|
||||||
import type { Theme } from "@aipexstudio/aipex-react/theme/types";
|
import type { Theme } from "@aipexstudio/aipex-react/theme/types";
|
||||||
|
import { ErrorBoundary } from "@aipexstudio/aipex-react/components/error/ErrorBoundary";
|
||||||
import { ChromeStorageAdapter } from "@aipexstudio/browser-runtime";
|
import { ChromeStorageAdapter } from "@aipexstudio/browser-runtime";
|
||||||
import React, { useCallback, useEffect, useRef, useState } from "react";
|
import React, { useCallback, useEffect, useRef, useState } from "react";
|
||||||
import ReactDOM from "react-dom/client";
|
import ReactDOM from "react-dom/client";
|
||||||
@@ -36,6 +37,76 @@ import { UpdateBannerWrapper } from "../../lib/update-banner-wrapper";
|
|||||||
const i18nStorageAdapter = new ChromeStorageAdapter<Language>();
|
const i18nStorageAdapter = new ChromeStorageAdapter<Language>();
|
||||||
const themeStorageAdapter = new ChromeStorageAdapter<Theme>();
|
const themeStorageAdapter = new ChromeStorageAdapter<Theme>();
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Replay setup listener
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/** Replay step shape coming from the external website */
|
||||||
|
interface ReplayStepData {
|
||||||
|
id?: number;
|
||||||
|
event: { type: string; [key: string]: unknown };
|
||||||
|
url?: string | null;
|
||||||
|
aiTitle?: string | null;
|
||||||
|
aiSummary?: string | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Listens for `NAVIGATE_AND_SETUP_REPLAY` messages forwarded by the
|
||||||
|
* background service worker after an external `REPLAY_USER_MANUAL` request.
|
||||||
|
*
|
||||||
|
* The replay steps are persisted to `chrome.storage.local` under
|
||||||
|
* `aipex-pending-replay` so they can be consumed by the use-case system
|
||||||
|
* when it is available.
|
||||||
|
*/
|
||||||
|
function useReplaySetup() {
|
||||||
|
useEffect(() => {
|
||||||
|
const handler = (message: Record<string, unknown>) => {
|
||||||
|
if (message?.request !== "NAVIGATE_AND_SETUP_REPLAY") return;
|
||||||
|
|
||||||
|
const data = message.data as {
|
||||||
|
manualId?: number;
|
||||||
|
startFromStep?: number;
|
||||||
|
steps?: ReplayStepData[];
|
||||||
|
} | undefined;
|
||||||
|
|
||||||
|
if (!data || !Array.isArray(data.steps) || data.steps.length === 0) {
|
||||||
|
console.warn("[ReplaySetup] Invalid or empty replay data received");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Persist replay data for future use-case system consumption
|
||||||
|
chrome.storage.local
|
||||||
|
.set({
|
||||||
|
"aipex-pending-replay": {
|
||||||
|
manualId: data.manualId,
|
||||||
|
startFromStep: data.startFromStep ?? 0,
|
||||||
|
steps: data.steps,
|
||||||
|
receivedAt: Date.now(),
|
||||||
|
},
|
||||||
|
})
|
||||||
|
.catch(() => {
|
||||||
|
/* storage may be unavailable */
|
||||||
|
});
|
||||||
|
|
||||||
|
console.log(
|
||||||
|
"[ReplaySetup] Replay data stored:",
|
||||||
|
data.steps.length,
|
||||||
|
"steps for manual",
|
||||||
|
data.manualId,
|
||||||
|
);
|
||||||
|
};
|
||||||
|
|
||||||
|
chrome.runtime.onMessage.addListener(handler);
|
||||||
|
return () => {
|
||||||
|
chrome.runtime.onMessage.removeListener(handler);
|
||||||
|
};
|
||||||
|
}, []);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Pending prompt
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Reads and consumes a pending prompt saved by the openWithPrompt external
|
* Reads and consumes a pending prompt saved by the openWithPrompt external
|
||||||
* message handler in the background service worker. Prompts older than 5 s
|
* message handler in the background service worker. Prompts older than 5 s
|
||||||
@@ -141,6 +212,7 @@ function ChatApp() {
|
|||||||
|
|
||||||
const pendingInput = usePendingPrompt();
|
const pendingInput = usePendingPrompt();
|
||||||
const heartbeat = useConversationHeartbeat();
|
const heartbeat = useConversationHeartbeat();
|
||||||
|
useReplaySetup();
|
||||||
|
|
||||||
const handleStatusChange = useCallback(
|
const handleStatusChange = useCallback(
|
||||||
(status: string) => {
|
(status: string) => {
|
||||||
@@ -242,13 +314,15 @@ export function renderChatApp() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
const App = () => (
|
const App = () => (
|
||||||
<I18nProvider storageAdapter={i18nStorageAdapter}>
|
<ErrorBoundary>
|
||||||
<ThemeProvider storageAdapter={themeStorageAdapter}>
|
<I18nProvider storageAdapter={i18nStorageAdapter}>
|
||||||
<AuthProvider>
|
<ThemeProvider storageAdapter={themeStorageAdapter}>
|
||||||
<ChatApp />
|
<AuthProvider>
|
||||||
</AuthProvider>
|
<ChatApp />
|
||||||
</ThemeProvider>
|
</AuthProvider>
|
||||||
</I18nProvider>
|
</ThemeProvider>
|
||||||
|
</I18nProvider>
|
||||||
|
</ErrorBoundary>
|
||||||
);
|
);
|
||||||
|
|
||||||
ReactDOM.createRoot(rootElement).render(
|
ReactDOM.createRoot(rootElement).render(
|
||||||
|
|||||||
@@ -7,6 +7,33 @@ export {
|
|||||||
WEBSITE_HOST,
|
WEBSITE_HOST,
|
||||||
WEBSITE_ORIGIN,
|
WEBSITE_ORIGIN,
|
||||||
} from "../config/website";
|
} from "../config/website";
|
||||||
|
// Recording upload
|
||||||
|
export {
|
||||||
|
uploadRecordingSession,
|
||||||
|
type UploadRecordingSessionPayload,
|
||||||
|
type UploadRecordingSessionResult,
|
||||||
|
type UploadRecordingStepPayload,
|
||||||
|
} from "./recording-upload";
|
||||||
|
// Replay controller
|
||||||
|
export {
|
||||||
|
ManualReplayController,
|
||||||
|
type ClickEvent,
|
||||||
|
type ExecutionResult,
|
||||||
|
type NavigationEvent,
|
||||||
|
type ReplayEventCallback,
|
||||||
|
type ReplayStatus,
|
||||||
|
type ReplayStep,
|
||||||
|
} from "./replay-controller";
|
||||||
|
// Screenshot upload
|
||||||
|
export {
|
||||||
|
uploadScreenshot,
|
||||||
|
type UploadScreenshotResult,
|
||||||
|
} from "./screenshot-upload";
|
||||||
|
// Share conversation
|
||||||
|
export {
|
||||||
|
shareConversation,
|
||||||
|
type ShareResult,
|
||||||
|
} from "./share-conversation";
|
||||||
// Sound effects
|
// Sound effects
|
||||||
export {
|
export {
|
||||||
playSoundEffect,
|
playSoundEffect,
|
||||||
@@ -34,6 +61,16 @@ export {
|
|||||||
toolManager,
|
toolManager,
|
||||||
unregisterDynamicTool,
|
unregisterDynamicTool,
|
||||||
} from "./tool-manager";
|
} from "./tool-manager";
|
||||||
|
// User manuals API
|
||||||
|
export {
|
||||||
|
deleteUserManual,
|
||||||
|
fetchMyUserManuals,
|
||||||
|
fetchUserManualDetail,
|
||||||
|
type FetchUserManualDetailResponse,
|
||||||
|
type UserManualDetail,
|
||||||
|
type UserManualListItem,
|
||||||
|
type UserManualStep,
|
||||||
|
} from "./user-manuals-api";
|
||||||
// Version checking
|
// Version checking
|
||||||
export {
|
export {
|
||||||
checkVersion,
|
checkVersion,
|
||||||
|
|||||||
81
packages/browser-ext/src/services/recording-upload.ts
Normal file
81
packages/browser-ext/src/services/recording-upload.ts
Normal file
@@ -0,0 +1,81 @@
|
|||||||
|
/**
|
||||||
|
* Recording Upload Service
|
||||||
|
* Uploads a complete recording session (steps + metadata) to the hosted API.
|
||||||
|
* Only used for non-BYOK user scenarios.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { WEBSITE_URL } from "../config/website";
|
||||||
|
import { getAuthCookieHeader } from "./web-auth";
|
||||||
|
|
||||||
|
/** Payload shape for a single recorded step */
|
||||||
|
export interface UploadRecordingStepPayload {
|
||||||
|
index: number;
|
||||||
|
timestamp: number;
|
||||||
|
url: string;
|
||||||
|
event: { type: string; [key: string]: unknown };
|
||||||
|
aiTitle?: string;
|
||||||
|
aiSummary?: string;
|
||||||
|
pageSnapshotId?: string;
|
||||||
|
screenshotUrl?: string | null;
|
||||||
|
screenshotKey?: string | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Full session payload sent to the server */
|
||||||
|
export interface UploadRecordingSessionPayload {
|
||||||
|
useCaseId: string;
|
||||||
|
params: Record<string, unknown>;
|
||||||
|
startedAt?: number;
|
||||||
|
completedAt?: number;
|
||||||
|
steps: UploadRecordingStepPayload[];
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Server response after a successful upload */
|
||||||
|
export interface UploadRecordingSessionResult {
|
||||||
|
id?: number;
|
||||||
|
stepsInserted?: number;
|
||||||
|
slug?: string;
|
||||||
|
/** @deprecated Use `slug` instead */
|
||||||
|
sessionId?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Upload a recording session to the server.
|
||||||
|
* Uses per-step `screenshotUrl` / `screenshotKey` references — raw base64
|
||||||
|
* screenshot data is NOT included in this request.
|
||||||
|
*/
|
||||||
|
export async function uploadRecordingSession(
|
||||||
|
payload: UploadRecordingSessionPayload,
|
||||||
|
): Promise<UploadRecordingSessionResult> {
|
||||||
|
const cookieHeader = await getAuthCookieHeader();
|
||||||
|
|
||||||
|
const response = await fetch(`${WEBSITE_URL}/api/recordings/sessions`, {
|
||||||
|
method: "POST",
|
||||||
|
headers: {
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
...(cookieHeader ? { Cookie: cookieHeader } : {}),
|
||||||
|
},
|
||||||
|
body: JSON.stringify(payload),
|
||||||
|
credentials: "include",
|
||||||
|
});
|
||||||
|
|
||||||
|
if (!response.ok) {
|
||||||
|
const text = await response.text().catch(() => "");
|
||||||
|
throw new Error(
|
||||||
|
`Recording upload failed (${response.status}): ${text || "Unknown error"}`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
const json = (await response.json().catch(() => ({}))) as Record<
|
||||||
|
string,
|
||||||
|
unknown
|
||||||
|
>;
|
||||||
|
|
||||||
|
return {
|
||||||
|
id: typeof json.id === "number" ? json.id : undefined,
|
||||||
|
stepsInserted:
|
||||||
|
typeof json.stepsInserted === "number" ? json.stepsInserted : undefined,
|
||||||
|
slug: typeof json.slug === "string" ? json.slug : undefined,
|
||||||
|
sessionId:
|
||||||
|
typeof json.sessionId === "string" ? json.sessionId : undefined,
|
||||||
|
};
|
||||||
|
}
|
||||||
133
packages/browser-ext/src/services/replay-controller.test.ts
Normal file
133
packages/browser-ext/src/services/replay-controller.test.ts
Normal file
@@ -0,0 +1,133 @@
|
|||||||
|
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||||
|
import {
|
||||||
|
ManualReplayController,
|
||||||
|
type ReplayEventCallback,
|
||||||
|
type ReplayStep,
|
||||||
|
} from "./replay-controller";
|
||||||
|
|
||||||
|
// Enhance chrome mock for tabs used by replay controller
|
||||||
|
beforeEach(() => {
|
||||||
|
vi.useFakeTimers();
|
||||||
|
|
||||||
|
// Reset chrome.tabs mocks
|
||||||
|
const mockTabs = globalThis.chrome.tabs as any;
|
||||||
|
mockTabs.create = vi.fn().mockResolvedValue({ id: 100 });
|
||||||
|
mockTabs.update = vi.fn().mockResolvedValue({});
|
||||||
|
mockTabs.get = vi.fn().mockResolvedValue({
|
||||||
|
id: 100,
|
||||||
|
url: "https://example.com",
|
||||||
|
});
|
||||||
|
mockTabs.query = vi
|
||||||
|
.fn()
|
||||||
|
.mockResolvedValue([{ id: 100, url: "https://example.com" }]);
|
||||||
|
mockTabs.sendMessage = vi.fn().mockResolvedValue({ success: true });
|
||||||
|
mockTabs.onUpdated = {
|
||||||
|
addListener: vi.fn((cb: Function) => {
|
||||||
|
// Immediately call with 'complete' to simulate page load
|
||||||
|
setTimeout(() => cb(100, { status: "complete" }), 0);
|
||||||
|
}),
|
||||||
|
removeListener: vi.fn(),
|
||||||
|
};
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
vi.useRealTimers();
|
||||||
|
});
|
||||||
|
|
||||||
|
function makeNavigationStep(url: string): ReplayStep {
|
||||||
|
return {
|
||||||
|
event: { type: "navigation", url },
|
||||||
|
url,
|
||||||
|
aiTitle: "Navigate",
|
||||||
|
aiSummary: `Navigate to ${url}`,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function makeClickStep(): ReplayStep {
|
||||||
|
return {
|
||||||
|
event: {
|
||||||
|
type: "click",
|
||||||
|
selector: "#btn",
|
||||||
|
textSnippet: "Click me",
|
||||||
|
rect: { x: 10, y: 20, width: 100, height: 40 },
|
||||||
|
value: {
|
||||||
|
tagName: "BUTTON",
|
||||||
|
id: "btn",
|
||||||
|
classes: [],
|
||||||
|
attributes: {},
|
||||||
|
elementDescription: "A button",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
url: "https://example.com",
|
||||||
|
aiTitle: "Click button",
|
||||||
|
aiSummary: "Click the button",
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
describe("ManualReplayController", () => {
|
||||||
|
it("initializes with idle status", () => {
|
||||||
|
const controller = new ManualReplayController([makeNavigationStep("https://example.com")]);
|
||||||
|
expect(controller.getStatus()).toBe("idle");
|
||||||
|
expect(controller.getCurrentStepIndex()).toBe(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("emits progress event on start", async () => {
|
||||||
|
const callback: ReplayEventCallback = vi.fn();
|
||||||
|
const steps = [makeNavigationStep("https://example.com")];
|
||||||
|
const controller = new ManualReplayController(steps, callback);
|
||||||
|
|
||||||
|
const startPromise = controller.start();
|
||||||
|
await vi.advanceTimersByTimeAsync(2000);
|
||||||
|
await startPromise;
|
||||||
|
|
||||||
|
expect(callback).toHaveBeenCalledWith(
|
||||||
|
expect.objectContaining({
|
||||||
|
type: "progress",
|
||||||
|
currentStep: 0,
|
||||||
|
totalSteps: 1,
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("can be stopped", () => {
|
||||||
|
const steps = [makeNavigationStep("https://example.com")];
|
||||||
|
const controller = new ManualReplayController(steps);
|
||||||
|
|
||||||
|
controller.stop();
|
||||||
|
expect(controller.getStatus()).toBe("idle");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("can be paused", () => {
|
||||||
|
const steps = [
|
||||||
|
makeNavigationStep("https://example.com"),
|
||||||
|
makeClickStep(),
|
||||||
|
];
|
||||||
|
const controller = new ManualReplayController(steps);
|
||||||
|
|
||||||
|
controller.pause();
|
||||||
|
expect(controller.getStatus()).toBe("paused");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("resume does nothing when not paused", async () => {
|
||||||
|
const steps = [makeNavigationStep("https://example.com")];
|
||||||
|
const controller = new ManualReplayController(steps);
|
||||||
|
|
||||||
|
// Status is "idle", resume should be a no-op
|
||||||
|
await controller.resume();
|
||||||
|
expect(controller.getStatus()).toBe("idle");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("skip current step increments index", async () => {
|
||||||
|
const steps = [makeNavigationStep("https://a.com"), makeClickStep()];
|
||||||
|
const controller = new ManualReplayController(steps);
|
||||||
|
|
||||||
|
expect(controller.getCurrentStepIndex()).toBe(0);
|
||||||
|
|
||||||
|
const skipPromise = controller.skipCurrentStep();
|
||||||
|
await vi.advanceTimersByTimeAsync(5000);
|
||||||
|
await skipPromise;
|
||||||
|
|
||||||
|
// Should have advanced past step 0
|
||||||
|
expect(controller.getCurrentStepIndex()).toBeGreaterThanOrEqual(1);
|
||||||
|
});
|
||||||
|
});
|
||||||
405
packages/browser-ext/src/services/replay-controller.ts
Normal file
405
packages/browser-ext/src/services/replay-controller.ts
Normal file
@@ -0,0 +1,405 @@
|
|||||||
|
/**
|
||||||
|
* User Manual Replay Controller
|
||||||
|
*
|
||||||
|
* Manages the execution of user manual replay steps,
|
||||||
|
* including navigation and click events.
|
||||||
|
*/
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Types
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
export interface ReplayStep {
|
||||||
|
id?: number;
|
||||||
|
event: NavigationEvent | ClickEvent | { type: string; [key: string]: unknown };
|
||||||
|
url: string | null;
|
||||||
|
aiTitle: string | null;
|
||||||
|
aiSummary: string | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface ClickEvent {
|
||||||
|
type: "click";
|
||||||
|
selector: string;
|
||||||
|
textSnippet: string;
|
||||||
|
rect: { x: number; y: number; width: number; height: number };
|
||||||
|
value: {
|
||||||
|
tagName: string;
|
||||||
|
id: string;
|
||||||
|
classes: string[];
|
||||||
|
attributes: Record<string, string>;
|
||||||
|
elementDescription: string;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface NavigationEvent {
|
||||||
|
type: "navigation";
|
||||||
|
url: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface ExecutionResult {
|
||||||
|
success: boolean;
|
||||||
|
error?: string;
|
||||||
|
details?: unknown;
|
||||||
|
}
|
||||||
|
|
||||||
|
export type ReplayStatus =
|
||||||
|
| "idle"
|
||||||
|
| "running"
|
||||||
|
| "paused"
|
||||||
|
| "completed"
|
||||||
|
| "error";
|
||||||
|
|
||||||
|
export type ReplayEventCallback = (event: {
|
||||||
|
type:
|
||||||
|
| "progress"
|
||||||
|
| "complete"
|
||||||
|
| "error"
|
||||||
|
| "step-start"
|
||||||
|
| "step-complete"
|
||||||
|
| "step-error";
|
||||||
|
currentStep?: number;
|
||||||
|
totalSteps?: number;
|
||||||
|
error?: string;
|
||||||
|
stepData?: ReplayStep;
|
||||||
|
result?: ExecutionResult;
|
||||||
|
}) => void;
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Controller
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
export class ManualReplayController {
|
||||||
|
private steps: ReplayStep[];
|
||||||
|
private currentStepIndex = 0;
|
||||||
|
private status: ReplayStatus = "idle";
|
||||||
|
private eventCallback?: ReplayEventCallback;
|
||||||
|
private targetTabId?: number;
|
||||||
|
private abortController?: AbortController;
|
||||||
|
|
||||||
|
constructor(steps: ReplayStep[], eventCallback?: ReplayEventCallback) {
|
||||||
|
this.steps = steps;
|
||||||
|
this.eventCallback = eventCallback;
|
||||||
|
}
|
||||||
|
|
||||||
|
// -----------------------------------------------------------------------
|
||||||
|
// Public API
|
||||||
|
// -----------------------------------------------------------------------
|
||||||
|
|
||||||
|
/** Start replay from a specific step */
|
||||||
|
async start(startFromStep = 0): Promise<void> {
|
||||||
|
if (this.status === "running") return;
|
||||||
|
|
||||||
|
this.currentStepIndex = startFromStep;
|
||||||
|
this.status = "running";
|
||||||
|
this.abortController = new AbortController();
|
||||||
|
|
||||||
|
this.emitEvent({
|
||||||
|
type: "progress",
|
||||||
|
currentStep: this.currentStepIndex,
|
||||||
|
totalSteps: this.steps.length,
|
||||||
|
});
|
||||||
|
|
||||||
|
try {
|
||||||
|
await this.executeNextStep();
|
||||||
|
} catch (error) {
|
||||||
|
this.status = "error";
|
||||||
|
this.emitEvent({
|
||||||
|
type: "error",
|
||||||
|
error: error instanceof Error ? error.message : String(error),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Stop replay */
|
||||||
|
stop(): void {
|
||||||
|
this.status = "idle";
|
||||||
|
this.abortController?.abort();
|
||||||
|
this.hideSpotlight();
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Pause replay */
|
||||||
|
pause(): void {
|
||||||
|
this.status = "paused";
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Resume replay from paused state */
|
||||||
|
async resume(): Promise<void> {
|
||||||
|
if (this.status !== "paused") return;
|
||||||
|
this.status = "running";
|
||||||
|
await this.executeNextStep();
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Retry current step */
|
||||||
|
async retryCurrentStep(): Promise<void> {
|
||||||
|
this.status = "running";
|
||||||
|
await this.executeNextStep();
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Skip current step and move to next */
|
||||||
|
async skipCurrentStep(): Promise<void> {
|
||||||
|
this.currentStepIndex++;
|
||||||
|
this.status = "running";
|
||||||
|
await this.executeNextStep();
|
||||||
|
}
|
||||||
|
|
||||||
|
getStatus(): ReplayStatus {
|
||||||
|
return this.status;
|
||||||
|
}
|
||||||
|
|
||||||
|
getCurrentStepIndex(): number {
|
||||||
|
return this.currentStepIndex;
|
||||||
|
}
|
||||||
|
|
||||||
|
// -----------------------------------------------------------------------
|
||||||
|
// Internal
|
||||||
|
// -----------------------------------------------------------------------
|
||||||
|
|
||||||
|
private async executeNextStep(): Promise<void> {
|
||||||
|
if (this.abortController?.signal.aborted) return;
|
||||||
|
|
||||||
|
if (this.currentStepIndex >= this.steps.length) {
|
||||||
|
this.status = "completed";
|
||||||
|
this.emitEvent({ type: "complete" });
|
||||||
|
this.hideSpotlight();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (this.status === "paused") return;
|
||||||
|
|
||||||
|
const step = this.steps[this.currentStepIndex]!;
|
||||||
|
|
||||||
|
this.emitEvent({
|
||||||
|
type: "step-start",
|
||||||
|
currentStep: this.currentStepIndex,
|
||||||
|
totalSteps: this.steps.length,
|
||||||
|
stepData: step,
|
||||||
|
});
|
||||||
|
|
||||||
|
try {
|
||||||
|
const result = await this.executeStep(step);
|
||||||
|
|
||||||
|
if (result.success) {
|
||||||
|
this.emitEvent({
|
||||||
|
type: "step-complete",
|
||||||
|
currentStep: this.currentStepIndex,
|
||||||
|
totalSteps: this.steps.length,
|
||||||
|
result,
|
||||||
|
});
|
||||||
|
this.currentStepIndex++;
|
||||||
|
await new Promise((r) => setTimeout(r, 1500));
|
||||||
|
await this.executeNextStep();
|
||||||
|
} else {
|
||||||
|
this.status = "paused";
|
||||||
|
this.emitEvent({
|
||||||
|
type: "step-error",
|
||||||
|
currentStep: this.currentStepIndex,
|
||||||
|
totalSteps: this.steps.length,
|
||||||
|
error: result.error,
|
||||||
|
result,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
} catch (error) {
|
||||||
|
this.status = "paused";
|
||||||
|
this.emitEvent({
|
||||||
|
type: "step-error",
|
||||||
|
currentStep: this.currentStepIndex,
|
||||||
|
totalSteps: this.steps.length,
|
||||||
|
error: error instanceof Error ? error.message : String(error),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private async executeStep(step: ReplayStep): Promise<ExecutionResult> {
|
||||||
|
const event = step.event as { type: string; [key: string]: unknown };
|
||||||
|
if (!event?.type) {
|
||||||
|
return { success: false, error: "Invalid event data" };
|
||||||
|
}
|
||||||
|
|
||||||
|
switch (event.type) {
|
||||||
|
case "navigation":
|
||||||
|
return this.executeNavigation(event as NavigationEvent, step);
|
||||||
|
case "click":
|
||||||
|
return this.executeClick(event as ClickEvent, step);
|
||||||
|
default:
|
||||||
|
return { success: false, error: `Unknown event type: ${event.type}` };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- Navigation ---
|
||||||
|
|
||||||
|
private async executeNavigation(
|
||||||
|
event: NavigationEvent,
|
||||||
|
step: ReplayStep,
|
||||||
|
): Promise<ExecutionResult> {
|
||||||
|
try {
|
||||||
|
const url = event.url || step.url;
|
||||||
|
if (!url) return { success: false, error: "No URL provided for navigation" };
|
||||||
|
|
||||||
|
if (this.targetTabId) {
|
||||||
|
await chrome.tabs.update(this.targetTabId, { url, active: true });
|
||||||
|
} else {
|
||||||
|
const tab = await chrome.tabs.create({ url, active: true });
|
||||||
|
this.targetTabId = tab.id;
|
||||||
|
}
|
||||||
|
|
||||||
|
await this.waitForTabLoad();
|
||||||
|
return { success: true };
|
||||||
|
} catch (error) {
|
||||||
|
return {
|
||||||
|
success: false,
|
||||||
|
error: error instanceof Error ? error.message : String(error),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- Click ---
|
||||||
|
|
||||||
|
private async executeClick(
|
||||||
|
event: ClickEvent,
|
||||||
|
step: ReplayStep,
|
||||||
|
): Promise<ExecutionResult> {
|
||||||
|
try {
|
||||||
|
let tabId = this.targetTabId;
|
||||||
|
if (!tabId) {
|
||||||
|
const tabs = await chrome.tabs.query({
|
||||||
|
active: true,
|
||||||
|
currentWindow: true,
|
||||||
|
});
|
||||||
|
if (!tabs[0]?.id) {
|
||||||
|
return { success: false, error: "No active tab found" };
|
||||||
|
}
|
||||||
|
tabId = tabs[0].id;
|
||||||
|
this.targetTabId = tabId;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Validate tab is accessible
|
||||||
|
let tab: chrome.tabs.Tab;
|
||||||
|
try {
|
||||||
|
tab = await chrome.tabs.get(tabId);
|
||||||
|
} catch {
|
||||||
|
return {
|
||||||
|
success: false,
|
||||||
|
error: "Target tab no longer exists. Please ensure the tab is open.",
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
if (
|
||||||
|
tab.url &&
|
||||||
|
!tab.url.startsWith("http://") &&
|
||||||
|
!tab.url.startsWith("https://")
|
||||||
|
) {
|
||||||
|
return {
|
||||||
|
success: false,
|
||||||
|
error:
|
||||||
|
"Cannot execute on this page. Only http/https pages are supported.",
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
// Navigate to correct URL if needed
|
||||||
|
if (step.url && tab.url && !tab.url.startsWith(step.url)) {
|
||||||
|
await chrome.tabs.update(tabId, { url: step.url });
|
||||||
|
await this.waitForTabLoad();
|
||||||
|
}
|
||||||
|
|
||||||
|
// Send click to content script with retry
|
||||||
|
const maxRetries = 3;
|
||||||
|
const retryDelay = 1000;
|
||||||
|
|
||||||
|
for (let attempt = 1; attempt <= maxRetries; attempt++) {
|
||||||
|
try {
|
||||||
|
const response = await chrome.tabs.sendMessage(tabId, {
|
||||||
|
request: "REPLAY_EXECUTE_CLICK",
|
||||||
|
eventData: event,
|
||||||
|
explanation: step.aiSummary || step.aiTitle,
|
||||||
|
stepUrl: step.url,
|
||||||
|
});
|
||||||
|
|
||||||
|
if (response?.success) {
|
||||||
|
return { success: true, details: response };
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
success: false,
|
||||||
|
error: response?.error || "Click execution failed",
|
||||||
|
details: response,
|
||||||
|
};
|
||||||
|
} catch (error) {
|
||||||
|
const msg =
|
||||||
|
error instanceof Error ? error.message : String(error);
|
||||||
|
|
||||||
|
if (
|
||||||
|
(msg.includes("Receiving end does not exist") ||
|
||||||
|
msg.includes("Could not establish connection")) &&
|
||||||
|
attempt < maxRetries
|
||||||
|
) {
|
||||||
|
await new Promise((r) => setTimeout(r, retryDelay));
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (attempt === maxRetries) {
|
||||||
|
return {
|
||||||
|
success: false,
|
||||||
|
error:
|
||||||
|
"Could not connect to the page. The page may have been closed or navigated away.",
|
||||||
|
};
|
||||||
|
}
|
||||||
|
return { success: false, error: msg };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
success: false,
|
||||||
|
error: "Failed to execute click after multiple attempts",
|
||||||
|
};
|
||||||
|
} catch (error) {
|
||||||
|
return {
|
||||||
|
success: false,
|
||||||
|
error: error instanceof Error ? error.message : String(error),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- Helpers ---
|
||||||
|
|
||||||
|
private waitForTabLoad(): Promise<void> {
|
||||||
|
return new Promise((resolve) => {
|
||||||
|
const listener = (
|
||||||
|
tabId: number,
|
||||||
|
changeInfo: chrome.tabs.TabChangeInfo,
|
||||||
|
) => {
|
||||||
|
if (tabId === this.targetTabId && changeInfo.status === "complete") {
|
||||||
|
chrome.tabs.onUpdated.removeListener(listener);
|
||||||
|
resolve();
|
||||||
|
}
|
||||||
|
};
|
||||||
|
chrome.tabs.onUpdated.addListener(listener);
|
||||||
|
// Timeout after 30 s
|
||||||
|
setTimeout(() => {
|
||||||
|
chrome.tabs.onUpdated.removeListener(listener);
|
||||||
|
resolve();
|
||||||
|
}, 30_000);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
private hideSpotlight(): void {
|
||||||
|
chrome.tabs
|
||||||
|
.query({ active: true, currentWindow: true })
|
||||||
|
.then((tabs) => {
|
||||||
|
if (tabs[0]?.id) {
|
||||||
|
chrome.tabs
|
||||||
|
.sendMessage(tabs[0].id, {
|
||||||
|
request: "ui-guider-hide-spotlight",
|
||||||
|
})
|
||||||
|
.catch(() => {});
|
||||||
|
}
|
||||||
|
})
|
||||||
|
.catch(() => {});
|
||||||
|
}
|
||||||
|
|
||||||
|
private emitEvent(event: Parameters<ReplayEventCallback>[0]): void {
|
||||||
|
try {
|
||||||
|
this.eventCallback?.(event);
|
||||||
|
} catch {
|
||||||
|
// Prevent callback errors from breaking the controller
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
66
packages/browser-ext/src/services/screenshot-upload.ts
Normal file
66
packages/browser-ext/src/services/screenshot-upload.ts
Normal file
@@ -0,0 +1,66 @@
|
|||||||
|
/**
|
||||||
|
* Screenshot Upload Service
|
||||||
|
* Uploads screenshots to the hosted storage service.
|
||||||
|
* Only used for non-BYOK user scenarios.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { WEBSITE_URL } from "../config/website";
|
||||||
|
import { getAuthCookieHeader } from "./web-auth";
|
||||||
|
|
||||||
|
export interface UploadScreenshotResult {
|
||||||
|
url: string;
|
||||||
|
key: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Upload a screenshot (base64 data URL) to server-side storage.
|
||||||
|
* @param base64 - Either a data URL (`data:image/...;base64,...`) or raw base64 string
|
||||||
|
* @returns The uploaded screenshot's public URL and storage key
|
||||||
|
*/
|
||||||
|
export async function uploadScreenshot(
|
||||||
|
base64: string,
|
||||||
|
): Promise<UploadScreenshotResult> {
|
||||||
|
if (!base64) {
|
||||||
|
throw new Error("Empty screenshot data");
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ensure data URL format
|
||||||
|
const payloadBase64 = base64.startsWith("data:image")
|
||||||
|
? base64
|
||||||
|
: `data:image/png;base64,${base64}`;
|
||||||
|
|
||||||
|
const cookieHeader = await getAuthCookieHeader();
|
||||||
|
|
||||||
|
const response = await fetch(`${WEBSITE_URL}/api/storage/upload`, {
|
||||||
|
method: "POST",
|
||||||
|
headers: {
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
...(cookieHeader ? { Cookie: cookieHeader } : {}),
|
||||||
|
},
|
||||||
|
body: JSON.stringify({
|
||||||
|
base64: payloadBase64,
|
||||||
|
}),
|
||||||
|
credentials: "include",
|
||||||
|
});
|
||||||
|
|
||||||
|
if (!response.ok) {
|
||||||
|
const text = await response.text().catch(() => "");
|
||||||
|
throw new Error(
|
||||||
|
`Screenshot upload failed (${response.status}): ${text || "Unknown error"}`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
const json = (await response.json().catch(() => ({}))) as Record<
|
||||||
|
string,
|
||||||
|
unknown
|
||||||
|
>;
|
||||||
|
|
||||||
|
if (typeof json.url !== "string" || typeof json.key !== "string") {
|
||||||
|
throw new Error("Invalid upload response: missing url or key");
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
url: json.url,
|
||||||
|
key: json.key,
|
||||||
|
};
|
||||||
|
}
|
||||||
170
packages/browser-ext/src/services/share-conversation.test.ts
Normal file
170
packages/browser-ext/src/services/share-conversation.test.ts
Normal file
@@ -0,0 +1,170 @@
|
|||||||
|
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||||
|
import { shareConversation } from "./share-conversation";
|
||||||
|
|
||||||
|
// Mock web-auth
|
||||||
|
vi.mock("./web-auth", () => ({
|
||||||
|
getAuthCookieHeader: vi.fn().mockResolvedValue("session=abc123"),
|
||||||
|
}));
|
||||||
|
|
||||||
|
// Mock website config
|
||||||
|
vi.mock("../config/website", () => ({
|
||||||
|
WEBSITE_URL: "https://www.claudechrome.com",
|
||||||
|
}));
|
||||||
|
|
||||||
|
describe("shareConversation", () => {
|
||||||
|
const originalFetch = globalThis.fetch;
|
||||||
|
|
||||||
|
beforeEach(() => {
|
||||||
|
globalThis.fetch = vi.fn();
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
globalThis.fetch = originalFetch;
|
||||||
|
});
|
||||||
|
|
||||||
|
it("throws when there are no non-system messages", async () => {
|
||||||
|
const messages = [
|
||||||
|
{
|
||||||
|
id: "1",
|
||||||
|
role: "system" as const,
|
||||||
|
parts: [{ type: "text" as const, text: "You are helpful" }],
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
await expect(shareConversation(messages)).rejects.toThrow(
|
||||||
|
"No messages to share",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("filters out system messages before sharing", async () => {
|
||||||
|
const mockFetch = globalThis.fetch as ReturnType<typeof vi.fn>;
|
||||||
|
mockFetch.mockResolvedValue({
|
||||||
|
ok: true,
|
||||||
|
json: () => Promise.resolve({ url: "https://www.claudechrome.com/share/abc" }),
|
||||||
|
});
|
||||||
|
|
||||||
|
const messages = [
|
||||||
|
{
|
||||||
|
id: "1",
|
||||||
|
role: "system" as const,
|
||||||
|
parts: [{ type: "text" as const, text: "System prompt" }],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: "2",
|
||||||
|
role: "user" as const,
|
||||||
|
parts: [{ type: "text" as const, text: "Hello" }],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: "3",
|
||||||
|
role: "assistant" as const,
|
||||||
|
parts: [{ type: "text" as const, text: "Hi there!" }],
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
await shareConversation(messages);
|
||||||
|
|
||||||
|
expect(mockFetch).toHaveBeenCalledTimes(1);
|
||||||
|
|
||||||
|
const callBody = JSON.parse(mockFetch.mock.calls[0][1].body);
|
||||||
|
// Should only have 2 messages (user + assistant), not the system one
|
||||||
|
expect(callBody.messages).toHaveLength(2);
|
||||||
|
expect(callBody.messages[0].role).toBe("user");
|
||||||
|
expect(callBody.messages[1].role).toBe("assistant");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("returns the share URL on success", async () => {
|
||||||
|
const mockFetch = globalThis.fetch as ReturnType<typeof vi.fn>;
|
||||||
|
mockFetch.mockResolvedValue({
|
||||||
|
ok: true,
|
||||||
|
json: () =>
|
||||||
|
Promise.resolve({ url: "https://www.claudechrome.com/share/xyz" }),
|
||||||
|
});
|
||||||
|
|
||||||
|
const messages = [
|
||||||
|
{
|
||||||
|
id: "1",
|
||||||
|
role: "user" as const,
|
||||||
|
parts: [{ type: "text" as const, text: "Hello" }],
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
const result = await shareConversation(messages);
|
||||||
|
expect(result.url).toBe("https://www.claudechrome.com/share/xyz");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("throws on server error", async () => {
|
||||||
|
const mockFetch = globalThis.fetch as ReturnType<typeof vi.fn>;
|
||||||
|
mockFetch.mockResolvedValue({
|
||||||
|
ok: false,
|
||||||
|
status: 500,
|
||||||
|
text: () => Promise.resolve("Internal Server Error"),
|
||||||
|
});
|
||||||
|
|
||||||
|
const messages = [
|
||||||
|
{
|
||||||
|
id: "1",
|
||||||
|
role: "user" as const,
|
||||||
|
parts: [{ type: "text" as const, text: "Hello" }],
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
await expect(shareConversation(messages)).rejects.toThrow(
|
||||||
|
"Share failed (500)",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("throws when no URL is returned", async () => {
|
||||||
|
const mockFetch = globalThis.fetch as ReturnType<typeof vi.fn>;
|
||||||
|
mockFetch.mockResolvedValue({
|
||||||
|
ok: true,
|
||||||
|
json: () => Promise.resolve({ success: true }),
|
||||||
|
});
|
||||||
|
|
||||||
|
const messages = [
|
||||||
|
{
|
||||||
|
id: "1",
|
||||||
|
role: "user" as const,
|
||||||
|
parts: [{ type: "text" as const, text: "Hello" }],
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
await expect(shareConversation(messages)).rejects.toThrow(
|
||||||
|
"no URL was returned",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("strips screenshot data from tool parts", async () => {
|
||||||
|
const mockFetch = globalThis.fetch as ReturnType<typeof vi.fn>;
|
||||||
|
mockFetch.mockResolvedValue({
|
||||||
|
ok: true,
|
||||||
|
json: () =>
|
||||||
|
Promise.resolve({ url: "https://www.claudechrome.com/share/abc" }),
|
||||||
|
});
|
||||||
|
|
||||||
|
const messages = [
|
||||||
|
{
|
||||||
|
id: "1",
|
||||||
|
role: "assistant" as const,
|
||||||
|
parts: [
|
||||||
|
{
|
||||||
|
type: "tool" as const,
|
||||||
|
toolName: "capture_screenshot",
|
||||||
|
toolCallId: "call-1",
|
||||||
|
input: {},
|
||||||
|
output: { success: true },
|
||||||
|
state: "completed" as const,
|
||||||
|
screenshot: "data:image/png;base64,HUGESCREENSHOTDATA",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
await shareConversation(messages);
|
||||||
|
|
||||||
|
const callBody = JSON.parse(mockFetch.mock.calls[0][1].body);
|
||||||
|
const toolPart = callBody.messages[0].content.parts[0];
|
||||||
|
// screenshot field should NOT be included in the shared payload
|
||||||
|
expect(toolPart.screenshot).toBeUndefined();
|
||||||
|
expect(toolPart.toolName).toBe("capture_screenshot");
|
||||||
|
});
|
||||||
|
});
|
||||||
129
packages/browser-ext/src/services/share-conversation.ts
Normal file
129
packages/browser-ext/src/services/share-conversation.ts
Normal file
@@ -0,0 +1,129 @@
|
|||||||
|
/**
|
||||||
|
* Share Conversation Service
|
||||||
|
* Uploads the current conversation to the hosted share API and returns
|
||||||
|
* a shareable URL.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import type { UIMessage } from "@aipexstudio/aipex-react/types";
|
||||||
|
import { WEBSITE_URL } from "../config/website";
|
||||||
|
import { getAuthCookieHeader } from "./web-auth";
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Types
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
interface ShareablePart {
|
||||||
|
type: string;
|
||||||
|
[key: string]: unknown;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface ShareableMessage {
|
||||||
|
id: string;
|
||||||
|
role: string;
|
||||||
|
content: { parts: ShareablePart[] };
|
||||||
|
metadata?: Record<string, unknown>;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface ShareResult {
|
||||||
|
url: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Helpers
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Convert a UI message part to a shareable format.
|
||||||
|
* Strips large binary payloads (screenshots) and internal-only fields.
|
||||||
|
*/
|
||||||
|
function toShareablePart(part: UIMessage["parts"][number]): ShareablePart {
|
||||||
|
switch (part.type) {
|
||||||
|
case "text":
|
||||||
|
return { type: "text", text: part.text };
|
||||||
|
case "file":
|
||||||
|
return {
|
||||||
|
type: "file",
|
||||||
|
mediaType: part.mediaType,
|
||||||
|
filename: part.filename,
|
||||||
|
url: part.url,
|
||||||
|
};
|
||||||
|
case "context":
|
||||||
|
return {
|
||||||
|
type: "context",
|
||||||
|
contextType: part.contextType,
|
||||||
|
label: part.label,
|
||||||
|
value: part.value,
|
||||||
|
metadata: part.metadata,
|
||||||
|
};
|
||||||
|
case "source-url":
|
||||||
|
return { type: "source-url", url: part.url };
|
||||||
|
case "reasoning":
|
||||||
|
return { type: "reasoning", text: part.text };
|
||||||
|
case "tool":
|
||||||
|
return {
|
||||||
|
type: "tool",
|
||||||
|
toolName: part.toolName,
|
||||||
|
input: part.input,
|
||||||
|
output: part.output,
|
||||||
|
state: part.state,
|
||||||
|
errorText: part.errorText,
|
||||||
|
toolCallId: part.toolCallId,
|
||||||
|
// Note: screenshot is intentionally omitted from share payload
|
||||||
|
};
|
||||||
|
default:
|
||||||
|
return part as ShareablePart;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Public API
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Share a conversation by uploading messages to the share API.
|
||||||
|
* @returns The shareable URL on success
|
||||||
|
* @throws On network or server errors
|
||||||
|
*/
|
||||||
|
export async function shareConversation(
|
||||||
|
messages: UIMessage[],
|
||||||
|
): Promise<ShareResult> {
|
||||||
|
// Filter out system messages
|
||||||
|
const messagesToShare = messages.filter((m) => m.role !== "system");
|
||||||
|
|
||||||
|
if (messagesToShare.length === 0) {
|
||||||
|
throw new Error("No messages to share");
|
||||||
|
}
|
||||||
|
|
||||||
|
const shareableMessages: ShareableMessage[] = messagesToShare.map((msg) => ({
|
||||||
|
id: msg.id,
|
||||||
|
role: msg.role,
|
||||||
|
content: { parts: msg.parts.map(toShareablePart) },
|
||||||
|
metadata: msg.metadata,
|
||||||
|
}));
|
||||||
|
|
||||||
|
const cookieHeader = await getAuthCookieHeader();
|
||||||
|
|
||||||
|
const response = await fetch(`${WEBSITE_URL}/api/share`, {
|
||||||
|
method: "POST",
|
||||||
|
headers: {
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
...(cookieHeader ? { Cookie: cookieHeader } : {}),
|
||||||
|
},
|
||||||
|
body: JSON.stringify({ messages: shareableMessages }),
|
||||||
|
});
|
||||||
|
|
||||||
|
if (!response.ok) {
|
||||||
|
const text = await response.text().catch(() => "");
|
||||||
|
throw new Error(
|
||||||
|
`Share failed (${response.status}): ${text || "Unknown error"}`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
const result = (await response.json()) as Record<string, unknown>;
|
||||||
|
|
||||||
|
if (typeof result.url !== "string") {
|
||||||
|
throw new Error("Share succeeded but no URL was returned");
|
||||||
|
}
|
||||||
|
|
||||||
|
return { url: result.url };
|
||||||
|
}
|
||||||
136
packages/browser-ext/src/services/user-manuals-api.ts
Normal file
136
packages/browser-ext/src/services/user-manuals-api.ts
Normal file
@@ -0,0 +1,136 @@
|
|||||||
|
/**
|
||||||
|
* User Manuals API Service
|
||||||
|
* CRUD operations for user manuals stored on the website backend.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { WEBSITE_URL } from "../config/website";
|
||||||
|
import { getAuthCookieHeader } from "./web-auth";
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Types
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
export interface UserManualListItem {
|
||||||
|
id: number;
|
||||||
|
title: string | null;
|
||||||
|
slug: string | null;
|
||||||
|
createdAt: string;
|
||||||
|
public: boolean;
|
||||||
|
stepCount: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface UserManualStep {
|
||||||
|
id: number;
|
||||||
|
index: number | null;
|
||||||
|
url: string;
|
||||||
|
event: unknown;
|
||||||
|
aiTitle: string | null;
|
||||||
|
aiSummary: string | null;
|
||||||
|
screenshotUrl: string | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface UserManualDetail {
|
||||||
|
id: number;
|
||||||
|
title: string | null;
|
||||||
|
createdAt: string;
|
||||||
|
public: boolean;
|
||||||
|
params: unknown;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface FetchUserManualDetailResponse {
|
||||||
|
manual: UserManualDetail;
|
||||||
|
steps: UserManualStep[];
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// API functions
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Fetch the current user's user manuals.
|
||||||
|
* Returns an empty array when the user is not authenticated.
|
||||||
|
*/
|
||||||
|
export async function fetchMyUserManuals(): Promise<UserManualListItem[]> {
|
||||||
|
const cookieHeader = await getAuthCookieHeader();
|
||||||
|
|
||||||
|
if (!cookieHeader) {
|
||||||
|
return [];
|
||||||
|
}
|
||||||
|
|
||||||
|
const response = await fetch(`${WEBSITE_URL}/api/user-manuals/my`, {
|
||||||
|
method: "GET",
|
||||||
|
headers: {
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
Cookie: cookieHeader,
|
||||||
|
},
|
||||||
|
credentials: "include",
|
||||||
|
});
|
||||||
|
|
||||||
|
if (!response.ok) {
|
||||||
|
if (response.status === 401) {
|
||||||
|
return [];
|
||||||
|
}
|
||||||
|
const text = await response.text().catch(() => "");
|
||||||
|
throw new Error(
|
||||||
|
`Fetch user manuals failed (${response.status}): ${text || "Unknown error"}`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
const json = (await response.json()) as { manuals: UserManualListItem[] };
|
||||||
|
return json.manuals;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Fetch a specific user manual with all its steps.
|
||||||
|
*/
|
||||||
|
export async function fetchUserManualDetail(
|
||||||
|
id: number,
|
||||||
|
): Promise<FetchUserManualDetailResponse> {
|
||||||
|
const cookieHeader = await getAuthCookieHeader();
|
||||||
|
|
||||||
|
const response = await fetch(`${WEBSITE_URL}/api/user-manuals/${id}`, {
|
||||||
|
method: "GET",
|
||||||
|
headers: {
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
...(cookieHeader ? { Cookie: cookieHeader } : {}),
|
||||||
|
},
|
||||||
|
credentials: "include",
|
||||||
|
});
|
||||||
|
|
||||||
|
if (!response.ok) {
|
||||||
|
const text = await response.text().catch(() => "");
|
||||||
|
throw new Error(
|
||||||
|
`Fetch manual detail failed (${response.status}): ${text || "Unknown error"}`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
return (await response.json()) as FetchUserManualDetailResponse;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Delete a user manual by ID.
|
||||||
|
* Requires authentication.
|
||||||
|
*/
|
||||||
|
export async function deleteUserManual(id: number): Promise<void> {
|
||||||
|
const cookieHeader = await getAuthCookieHeader();
|
||||||
|
|
||||||
|
if (!cookieHeader) {
|
||||||
|
throw new Error("User not logged in");
|
||||||
|
}
|
||||||
|
|
||||||
|
const response = await fetch(`${WEBSITE_URL}/api/user-manuals/${id}`, {
|
||||||
|
method: "DELETE",
|
||||||
|
headers: {
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
Cookie: cookieHeader,
|
||||||
|
},
|
||||||
|
credentials: "include",
|
||||||
|
});
|
||||||
|
|
||||||
|
if (!response.ok) {
|
||||||
|
const text = await response.text().catch(() => "");
|
||||||
|
throw new Error(
|
||||||
|
`Delete manual failed (${response.status}): ${text || "Unknown error"}`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user