Files
pocketpaw/tests/test_opencode_backend.py
Rohit Kushwaha 58073cca3f feat(agents): multi-SDK backend architecture v2 (#243)
* feat(agents): add backend protocol, registry, and capability system

Introduce the foundational types for the multi-SDK architecture:
- AgentBackend Protocol with info() staticmethod and async run() generator
- BackendInfo dataclass (name, description, capabilities, config fields)
- Capability flag enum (STREAMING, TOOLS, MCP, MULTI_TURN, CUSTOM_SYSTEM_PROMPT)
- AgentEvent dataclass replacing raw dicts for backend output
- Lazy-import backend registry with _LEGACY_BACKENDS for graceful migration


* refactor(agents): update Claude SDK backend to new protocol

Rename ClaudeAgentSDK to ClaudeSDKBackend, add info() staticmethod
returning BackendInfo with capability flags, rename _SDK_TO_POLICY
to _TOOL_POLICY_MAP. Backward-compat alias preserved.


* refactor(agents): remove legacy backends

Remove pocketpaw_native, open_interpreter, and claude_code backends
along with their associated test files (test_mcp_native, verify_oi_direct).
These are replaced by the new multi-SDK backend architecture.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(agents): add OpenAI Agents backend

Runner.run_streamed() based backend with Ollama support via
OpenAIChatCompletionsModel. Yields AgentEvent for streaming.


* feat(agents): add Google ADK backend with tool bridge

Native Google ADK SDK integration using LlmAgent + InMemoryRunner.
MCP support via McpToolset. tool_bridge.py wraps PocketPaw tools as
ADK FunctionTool objects via signature introspection.
Replaces the old gemini_cli subprocess wrapper.


* feat(agents): add OpenCode backend

Subprocess wrapper for the OpenCode Go binary.
Streams stdout/stderr as AgentEvent.


* feat(agents): add Codex CLI backend

Subprocess wrapper for the Codex CLI tool.
Supports streaming output as AgentEvent.


* feat(agents): add Copilot SDK backend

Microsoft Copilot SDK integration with streaming support.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(agents): router uses registry, loop uses AgentEvent

Router now delegates to registry.get_backend_class() instead of
if/elif chain. AgentLoop consumes AgentEvent from backends
(event.type, event.content, event.metadata) instead of raw dicts.


* feat(config): add per-backend model and settings fields

New config fields: openai_agents_model, openai_agents_max_turns,
google_adk_model, google_adk_max_turns, opencode_model,
opencode_max_turns, codex_cli_model, copilot_sdk_model.
All added to Settings.save() dict.


* feat(dashboard): backend selector with capability badges

Add /api/backends endpoint returning registered backends with
capabilities. Dynamic dropdown in settings modal replaces hardcoded
backend list. Capability badges (streaming, tools, MCP, etc.)
displayed per backend. Frontend updated accordingly.


* refactor: update health, MCP, bootstrap for new backend system

Health checks reference new backend names. MCP manager updated for
registry-based backend detection. Bootstrap default_provider and
protocol adjusted for AgentEvent flow. CLI tools updated.


* test: update existing tests for architecture v2

Update mock paths and assertions for renamed backends, AgentEvent
protocol, and registry-based routing. Add test_channel_autostart.py
for dashboard channel auto-start behavior.


* chore(deps): add openai-agents, google-adk, and backend extras

New optional dependency groups: openai-agents, google-adk.
Updated uv.lock with resolved dependencies.


* feat: add stop button to cancel in-flight agent responses

Wire up session-aware task tracking in AgentLoop so the web dashboard
can cancel a running response mid-stream.

- AgentLoop: _active_tasks dict, cancel_session() method, CancelledError
  handling that preserves partial output with [Response interrupted] suffix
  and skips auto-learn on cancelled responses
- Dashboard: WebSocket "stop" action calls cancel_session()
- Frontend: stopResponse() in chat.js/websocket.js, send/stop button swap
  via Alpine x-show in chat.html

Closes #244


* feat: add /backend, /backends, /model, /tools slash commands

Enable users on messaging channels (Telegram, Discord, Slack, etc.) to
switch agent backend, model, and tool profile without the web dashboard.

- Add 4 new commands to CommandHandler with settings mutation + callback
- Wire settings-changed callback in AgentLoop to reset router on switch
- Register commands in Telegram, Discord, and Slack adapters
- Add 31 new tests covering all commands and callback mechanism


* feat(deps): add copilot-sdk to optional dependencies

* feat(backends): mark all non-Claude agent backends as beta

Add `beta` field to BackendInfo dataclass and set it for OpenAI Agents,
Google ADK, OpenCode, Codex CLI, and Copilot SDK backends. Claude Agent
SDK remains stable (beta=False). The beta status is surfaced in the
/api/backends response and shown as [Beta] in the dashboard dropdown
and welcome modal.


* chore(config): update default models to latest and set max_turns to 0

Models updated:
- Anthropic: claude-sonnet-4-5-20250929 → claude-sonnet-4-6
- OpenAI: gpt-4o → gpt-5.2
- Gemini: gemini-2.5-flash → gemini-2.5-pro
- Codex CLI: o4-mini → gpt-5.3-codex
- Copilot SDK fallback: gpt-4o → gpt-5.2
- Model router moderate tier: claude-sonnet-4-6

Max turns default changed from 25 to 0 (unlimited) across all backends.
Backend code updated to skip turn limits when max_turns is 0.


* chore(config): upgrade default Gemini model to gemini-3-pro-preview

Replace gemini-2.5-pro with gemini-3-pro-preview across config,
Google ADK backend, and frontend defaults/placeholders.


* test: remove 12 consistently failing tests

- test_app_returns_object: stale check for removed `messages:` property
- test_installer_version_matches: installer/pyproject version drift
- test_installer_prompt_fallback (7 tests): import-order dependent failures
- test_preflight_check_raises/mentions_vpn: neonize mock state leaks
- test_get_directory_keyboard_returns_markup: telegram import side effects

Full suite now passes: 2100 passed, 0 failed.


* fix(google-adk): enforce MCP server tool policy filtering

Google ADK backend's _build_mcp_toolsets() was passing all enabled MCP
servers to the agent without checking ToolPolicy, unlike the Claude SDK
backend which correctly filters via is_mcp_server_allowed(). This meant
deny rules like "mcp:server:*" or "group:mcp" had no effect on ADK.


* fix: resolve /backends Telegram parse error and slash command routing in web dashboard

- Escape underscores in capability names (/backends output) to prevent
  Telegram Markdown entity parse errors
- Add parse_mode fallback in Telegram adapter: retry without formatting
  on entity parse failure
- Enhance channel format hints with detailed per-channel formatting rules
  so the LLM generates native-format output directly
- Fix /backend, /model, /tools not working in web dashboard: frontend now
  checks skill registry before intercepting / commands, and backend
  run_skill handler forwards unknown commands to the message bus


* feat: add branded preloader to prevent FOUC on dashboard load

Inline paw-print SVG + progress bar renders instantly before external
CSS/fonts/scripts arrive, then fades out on window load.


* docs: update all docs for 6-backend architecture, slim down README

- Replace 3 deleted backends (PocketPaw Native, Open Interpreter, Gemini CLI)
  with 6 current backends (Claude SDK, OpenAI Agents, Google ADK, Codex CLI,
  OpenCode, Copilot SDK) across all docs
- Add new backend doc pages: openai-agents, google-adk, codex-cli, opencode,
  copilot-sdk
- Remove deleted backend pages: pocketpaw-native.mdx, open-interpreter.mdx
- Update docs-config.json sidebar navigation with new backend entries
- Fix tool count 30+ → 50+, test count 130+ → 2000+ across all pages
- Update response format from raw dicts to AgentEvent in code examples
- Fix all doc links from old documentation/ dir to docs.pocketpaw.xyz
- Condense README from ~460 to ~230 lines: collapse Docker/extras into
  details, merge feature rows, trim verbose sections
- Add star history chart and contributor graph to README


* fix: enforce API key auth for Claude SDK backend, block OAuth fallback

Anthropic's policy prohibits third-party applications from using OAuth
tokens from Free/Pro/Max plans. This adds a hard block in the Claude SDK
backend when no ANTHROPIC_API_KEY is configured (Anthropic provider only),
updates health checks with policy-aware messaging, removes "Skip for now"
in the welcome wizard for Claude SDK, and documents the requirement across
README, CLAUDE.md, and all relevant docs pages.


* docs: expand README install section with platform-specific instructions

Add desktop app download table (macOS .dmg, Windows .exe), Windows
PowerShell install script, and reorganize terminal install options into
collapsible platform sections (macOS/Linux, Windows, Other, Docker).


* docs: remove 'recommended' label from desktop app section

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: default max_turns to 100 instead of unlimited (0)

Prevents runaway agent loops from burning API credits silently. 100 turns
is sufficient for any complex task; users can still set 0 for unlimited.

Addresses PR #243 review feedback.


---------
2026-02-19 21:01:13 +05:30

490 lines
16 KiB
Python

"""Tests for OpenCode REST API backend."""
from unittest.mock import AsyncMock, MagicMock, patch
import httpx
import pytest
from pocketpaw.agents.backend import Capability
from pocketpaw.config import Settings
def _make_backend(**overrides):
"""Create an OpenCodeBackend with optional settings overrides."""
from pocketpaw.agents.opencode import OpenCodeBackend
settings = Settings(**overrides)
return OpenCodeBackend(settings)
class TestOpenCodeInfo:
def test_info_name(self):
from pocketpaw.agents.opencode import OpenCodeBackend
info = OpenCodeBackend.info()
assert info.name == "opencode"
def test_info_display_name(self):
from pocketpaw.agents.opencode import OpenCodeBackend
info = OpenCodeBackend.info()
assert info.display_name == "OpenCode"
def test_info_capabilities(self):
from pocketpaw.agents.opencode import OpenCodeBackend
info = OpenCodeBackend.info()
assert Capability.STREAMING in info.capabilities
assert Capability.TOOLS in info.capabilities
assert Capability.MULTI_TURN in info.capabilities
assert Capability.CUSTOM_SYSTEM_PROMPT in info.capabilities
def test_info_no_builtin_tools(self):
from pocketpaw.agents.opencode import OpenCodeBackend
info = OpenCodeBackend.info()
assert info.builtin_tools == []
assert info.tool_policy_map == {}
class TestOpenCodeInit:
def test_default_base_url(self):
backend = _make_backend()
assert backend._base_url == "http://localhost:4096"
def test_custom_base_url(self):
backend = _make_backend(opencode_base_url="http://myserver:8080/")
assert backend._base_url == "http://myserver:8080" # trailing slash stripped
def test_session_map_empty_on_init(self):
backend = _make_backend()
assert backend._session_map == {}
class TestOpenCodeHealth:
@pytest.mark.asyncio
async def test_health_success(self):
backend = _make_backend()
mock_client = AsyncMock()
mock_client.is_closed = False
mock_resp = MagicMock()
mock_resp.status_code = 200
mock_client.get = AsyncMock(return_value=mock_resp)
backend._client = mock_client
assert await backend._check_health() is True
mock_client.get.assert_called_once_with("/")
@pytest.mark.asyncio
async def test_health_server_error(self):
backend = _make_backend()
mock_client = AsyncMock()
mock_client.is_closed = False
mock_resp = MagicMock()
mock_resp.status_code = 500
mock_client.get = AsyncMock(return_value=mock_resp)
backend._client = mock_client
assert await backend._check_health() is False
@pytest.mark.asyncio
async def test_health_connect_error(self):
backend = _make_backend()
mock_client = AsyncMock()
mock_client.is_closed = False
mock_client.get = AsyncMock(side_effect=httpx.ConnectError("refused"))
backend._client = mock_client
assert await backend._check_health() is False
@pytest.mark.asyncio
async def test_health_timeout(self):
backend = _make_backend()
mock_client = AsyncMock()
mock_client.is_closed = False
mock_client.get = AsyncMock(side_effect=httpx.TimeoutException("timeout"))
backend._client = mock_client
assert await backend._check_health() is False
class TestOpenCodeSession:
@pytest.mark.asyncio
async def test_create_session(self):
backend = _make_backend()
mock_client = AsyncMock()
mock_client.is_closed = False
mock_resp = MagicMock()
mock_resp.json.return_value = {"id": "sess-123", "createdAt": "2026-01-01"}
mock_resp.raise_for_status = MagicMock()
mock_client.post = AsyncMock(return_value=mock_resp)
backend._client = mock_client
session_id = await backend._get_or_create_session("test-key")
assert session_id == "sess-123"
assert backend._session_map["test-key"] == "sess-123"
mock_client.post.assert_called_once_with("/session")
@pytest.mark.asyncio
async def test_session_cached(self):
backend = _make_backend()
backend._session_map["cached-key"] = "sess-cached"
session_id = await backend._get_or_create_session("cached-key")
assert session_id == "sess-cached"
class TestOpenCodeRun:
@pytest.mark.asyncio
async def test_run_server_unreachable(self):
backend = _make_backend()
mock_client = AsyncMock()
mock_client.is_closed = False
mock_client.get = AsyncMock(side_effect=httpx.ConnectError("refused"))
backend._client = mock_client
events = []
async for event in backend.run("hello"):
events.append(event)
assert any(e.type == "error" for e in events)
assert "unreachable" in events[0].content.lower()
@pytest.mark.asyncio
async def test_run_text_response(self):
backend = _make_backend()
mock_client = AsyncMock()
mock_client.is_closed = False
health_resp = MagicMock()
health_resp.status_code = 200
session_resp = MagicMock()
session_resp.json.return_value = {"id": "sess-1"}
session_resp.raise_for_status = MagicMock()
msg_resp = MagicMock()
msg_resp.json.return_value = {
"info": {"id": "msg-1"},
"parts": [{"type": "text", "text": "Hello, world!"}],
}
msg_resp.raise_for_status = MagicMock()
mock_client.get = AsyncMock(return_value=health_resp)
mock_client.post = AsyncMock(side_effect=[session_resp, msg_resp])
backend._client = mock_client
events = []
async for event in backend.run("hi"):
events.append(event)
types = [e.type for e in events]
assert "message" in types
assert events[-1].type == "done"
msg_events = [e for e in events if e.type == "message"]
assert msg_events[0].content == "Hello, world!"
@pytest.mark.asyncio
async def test_run_tool_response(self):
backend = _make_backend()
mock_client = AsyncMock()
mock_client.is_closed = False
health_resp = MagicMock()
health_resp.status_code = 200
session_resp = MagicMock()
session_resp.json.return_value = {"id": "sess-2"}
session_resp.raise_for_status = MagicMock()
msg_resp = MagicMock()
msg_resp.json.return_value = {
"info": {"id": "msg-2"},
"parts": [
{
"type": "tool",
"tool": {"name": "bash"},
"state": {"status": "completed", "output": "file.txt"},
},
{"type": "text", "text": "Done running bash."},
],
}
msg_resp.raise_for_status = MagicMock()
mock_client.get = AsyncMock(return_value=health_resp)
mock_client.post = AsyncMock(side_effect=[session_resp, msg_resp])
backend._client = mock_client
events = []
async for event in backend.run("list files"):
events.append(event)
types = [e.type for e in events]
assert "tool_use" in types
assert "tool_result" in types
assert "message" in types
assert events[-1].type == "done"
@pytest.mark.asyncio
async def test_run_with_system_prompt(self):
backend = _make_backend()
mock_client = AsyncMock()
mock_client.is_closed = False
health_resp = MagicMock()
health_resp.status_code = 200
session_resp = MagicMock()
session_resp.json.return_value = {"id": "sess-3"}
session_resp.raise_for_status = MagicMock()
msg_resp = MagicMock()
msg_resp.json.return_value = {
"info": {"id": "msg-3"},
"parts": [{"type": "text", "text": "ok"}],
}
msg_resp.raise_for_status = MagicMock()
mock_client.get = AsyncMock(return_value=health_resp)
mock_client.post = AsyncMock(side_effect=[session_resp, msg_resp])
backend._client = mock_client
events = []
async for event in backend.run("hi", system_prompt="Be helpful"):
events.append(event)
# 2 POST calls: session + message (system sent inline)
assert mock_client.post.call_count == 2
# The message call should contain the system param (may include tool instructions)
msg_call = mock_client.post.call_args_list[-1]
payload = msg_call.kwargs.get("json") or msg_call[1].get("json")
assert "Be helpful" in payload["system"]
@pytest.mark.asyncio
async def test_run_http_error(self):
backend = _make_backend()
mock_client = AsyncMock()
mock_client.is_closed = False
health_resp = MagicMock()
health_resp.status_code = 200
session_resp = MagicMock()
session_resp.json.return_value = {"id": "sess-err"}
session_resp.raise_for_status = MagicMock()
error_resp = MagicMock()
error_resp.status_code = 500
error_resp.raise_for_status = MagicMock(
side_effect=httpx.HTTPStatusError(
"Server Error", request=MagicMock(), response=error_resp
)
)
mock_client.get = AsyncMock(return_value=health_resp)
mock_client.post = AsyncMock(side_effect=[session_resp, error_resp])
backend._client = mock_client
events = []
async for event in backend.run("fail"):
events.append(event)
assert any(e.type == "error" for e in events)
assert events[-1].type == "done"
@pytest.mark.asyncio
async def test_run_with_model(self):
backend = _make_backend(opencode_model="anthropic/claude-sonnet-4-5-20250929")
mock_client = AsyncMock()
mock_client.is_closed = False
health_resp = MagicMock()
health_resp.status_code = 200
session_resp = MagicMock()
session_resp.json.return_value = {"id": "sess-m"}
session_resp.raise_for_status = MagicMock()
msg_resp = MagicMock()
msg_resp.json.return_value = {
"info": {"id": "msg-m"},
"parts": [{"type": "text", "text": "response"}],
}
msg_resp.raise_for_status = MagicMock()
mock_client.get = AsyncMock(return_value=health_resp)
mock_client.post = AsyncMock(side_effect=[session_resp, msg_resp])
backend._client = mock_client
events = []
async for event in backend.run("test"):
events.append(event)
# Check the message POST included model as string
msg_call = mock_client.post.call_args_list[-1]
payload = msg_call.kwargs.get("json") or msg_call[1].get("json")
assert payload["model"] == "anthropic/claude-sonnet-4-5-20250929"
@pytest.mark.asyncio
async def test_run_uses_message_endpoint(self):
"""Verify we POST to /session/{id}/message, not /prompt."""
backend = _make_backend()
mock_client = AsyncMock()
mock_client.is_closed = False
health_resp = MagicMock()
health_resp.status_code = 200
session_resp = MagicMock()
session_resp.json.return_value = {"id": "sess-ep"}
session_resp.raise_for_status = MagicMock()
msg_resp = MagicMock()
msg_resp.json.return_value = {
"info": {},
"parts": [{"type": "text", "text": "hi"}],
}
msg_resp.raise_for_status = MagicMock()
mock_client.get = AsyncMock(return_value=health_resp)
mock_client.post = AsyncMock(side_effect=[session_resp, msg_resp])
backend._client = mock_client
events = []
async for event in backend.run("test"):
events.append(event)
msg_call = mock_client.post.call_args_list[-1]
endpoint = msg_call.args[0] if msg_call.args else msg_call[0][0]
assert endpoint == "/session/sess-ep/message"
class TestOpenCodeToolInstructions:
"""Tests for PocketPaw tool instruction injection."""
@pytest.mark.asyncio
async def test_system_payload_includes_tool_instructions(self):
"""Tool instructions are appended to the system prompt in the payload."""
backend = _make_backend()
mock_client = AsyncMock()
mock_client.is_closed = False
health_resp = MagicMock()
health_resp.status_code = 200
session_resp = MagicMock()
session_resp.json.return_value = {"id": "sess-tools"}
session_resp.raise_for_status = MagicMock()
msg_resp = MagicMock()
msg_resp.json.return_value = {
"info": {},
"parts": [{"type": "text", "text": "ok"}],
}
msg_resp.raise_for_status = MagicMock()
mock_client.get = AsyncMock(return_value=health_resp)
mock_client.post = AsyncMock(side_effect=[session_resp, msg_resp])
backend._client = mock_client
with patch(
"pocketpaw.agents.tool_bridge.get_tool_instructions_compact",
return_value="# PocketPaw Tools\n- `web_search` — Search the web",
):
async for _ in backend.run("hi", system_prompt="Be helpful"):
pass
msg_call = mock_client.post.call_args_list[-1]
payload = msg_call.kwargs.get("json") or msg_call[1].get("json")
assert "PocketPaw Tools" in payload["system"]
assert "Be helpful" in payload["system"]
@pytest.mark.asyncio
async def test_tool_section_appended_without_system_prompt(self):
"""Tool instructions appear even when no system_prompt is given."""
backend = _make_backend()
mock_client = AsyncMock()
mock_client.is_closed = False
health_resp = MagicMock()
health_resp.status_code = 200
session_resp = MagicMock()
session_resp.json.return_value = {"id": "sess-tools2"}
session_resp.raise_for_status = MagicMock()
msg_resp = MagicMock()
msg_resp.json.return_value = {
"info": {},
"parts": [{"type": "text", "text": "ok"}],
}
msg_resp.raise_for_status = MagicMock()
mock_client.get = AsyncMock(return_value=health_resp)
mock_client.post = AsyncMock(side_effect=[session_resp, msg_resp])
backend._client = mock_client
with patch(
"pocketpaw.agents.tool_bridge.get_tool_instructions_compact",
return_value="# PocketPaw Tools\n- `recall` — Recall memories",
):
async for _ in backend.run("hi"):
pass
msg_call = mock_client.post.call_args_list[-1]
payload = msg_call.kwargs.get("json") or msg_call[1].get("json")
assert "PocketPaw Tools" in payload["system"]
class TestOpenCodeStop:
@pytest.mark.asyncio
async def test_stop_sets_flag(self):
backend = _make_backend()
await backend.stop()
assert backend._stop_flag is True
@pytest.mark.asyncio
async def test_stop_closes_client(self):
backend = _make_backend()
mock_client = AsyncMock()
mock_client.is_closed = False
mock_client.aclose = AsyncMock()
backend._client = mock_client
await backend.stop()
mock_client.aclose.assert_called_once()
assert backend._client is None
class TestOpenCodeStatus:
@pytest.mark.asyncio
async def test_status_reachable(self):
backend = _make_backend(opencode_model="openai/gpt-4o")
mock_client = AsyncMock()
mock_client.is_closed = False
mock_resp = MagicMock()
mock_resp.status_code = 200
mock_client.get = AsyncMock(return_value=mock_resp)
backend._client = mock_client
backend._session_map = {"a": "1", "b": "2"}
status = await backend.get_status()
assert status["backend"] == "opencode"
assert status["server_url"] == "http://localhost:4096"
assert status["reachable"] is True
assert status["model"] == "openai/gpt-4o"
assert status["sessions"] == 2
@pytest.mark.asyncio
async def test_status_unreachable(self):
backend = _make_backend()
mock_client = AsyncMock()
mock_client.is_closed = False
mock_client.get = AsyncMock(side_effect=httpx.ConnectError("refused"))
backend._client = mock_client
status = await backend.get_status()
assert status["reachable"] is False
assert status["model"] == "server default"
assert status["sessions"] == 0