Fixes Copilot image understanding by exchanging OAuth tokens for Copilot API tokens, routing Copilot Gemini image requests through Chat Completions, and sending the prompt in user content with Copilot vision headers.
Real behavior proof:
- Old Responses route with real Copilot key reproduced `400 model gemini-3.1-pro-preview does not support Responses API`.
- Fixed route with the same real Copilot key returned `Cat`.
- Final CLI live smoke returned `ok: true` and `text: Cat` for `github-copilot/gemini-3.1-pro-preview`.
Verification:
- pnpm test src/media-understanding/image.test.ts extensions/github-copilot/models.test.ts extensions/github-copilot/stream.test.ts src/agents/pi-hooks/compaction-safeguard.test.ts -- --reporter=verbose
- pnpm check:changed via Blacksmith Testbox tbx_01krgt56pqmft8txekt017wke6, Actions run https://github.com/openclaw/openclaw/actions/runs/25803926150, exit 0.
Refs #80393, #80442.
Co-authored-by: Yang Haoyu <150496764+afunnyhy@users.noreply.github.com>
The plugin's `catalog.run` hook already exchanged a GitHub OAuth token
for a short-lived Copilot API token and resolved the per-account baseUrl,
but it returned `models: []` and the bundled openclaw runtime relied
entirely on the static manifest catalog. That meant:
- Static `contextWindow` values were a conservative 128k for every
model, far below reality (gpt-5.4/5.5 are 400k, claude-opus-4.6/4.7
internal variants are 1M, claude-sonnet-4 is 200k, etc.).
- Newly published Copilot models (gpt-5.5, gpt-5.1*, gemini-3-pro-preview,
the claude-opus-*-1m internal variants, etc.) didn't appear at all
until the manifest was patched.
- Per-account entitlement was invisible — every user saw the same
hardcoded 22-model list regardless of plan.
Wire it up:
- Add `fetchCopilotModelCatalog` in `extensions/github-copilot/models.ts`.
Calls `${baseUrl}/models` with the resolved Copilot API token and the
same Editor-Version / Copilot-Integration-Id headers used elsewhere in
the plugin. Maps each entry to a `ModelDefinitionConfig`:
- `contextWindow` ← `capabilities.limits.max_context_window_tokens`
- `maxTokens` ← `capabilities.limits.max_output_tokens`
- `input` ← `["text", "image"]` if `supports.vision`, else `["text"]`
- `reasoning` ← `Array.isArray(supports.reasoning_effort) && supports.reasoning_effort.length > 0`
- `api` ← `anthropic-messages` for Anthropic vendor or claude*
ids; otherwise `openai-responses`
Filters out non-chat objects (embeddings) and internal routers
(`accounts/...` ids). Dedupes by id. 10s default timeout.
- Update the `catalog.run` hook in `extensions/github-copilot/index.ts`
to call the new function after token-exchange and return the live
results. On any HTTP/parse failure it falls back to `models: []`,
which preserves the static manifest catalog as the visible fallback —
no behavior regression for users with `discovery.enabled: false` or
in offline scenarios.
- Bump `modelCatalog.discovery."github-copilot"` from `"static"` to
`"refreshable"` in `openclaw.plugin.json` so the catalog hook is
actually invoked at runtime. Without this the discovery infrastructure
treats the provider as static-only and never calls `catalog.run`.
- Add `gpt-5.5` to the static manifest catalog and `DEFAULT_MODEL_IDS`
with the correct values from the API (`contextWindow: 400000`,
`maxTokens: 128000`, `reasoning: true`, multimodal). This means users
on `discovery.enabled: false` still get gpt-5.5 visible without
needing to override `models.providers.github-copilot.models` in their
config.
Tests added (5, all passing alongside the existing 24):
- `fetchCopilotModelCatalog` maps a representative `/models` response
(chat models incl. an internal 1M-context Anthropic variant, a router,
an embedding) to the right `ModelDefinitionConfig` shape with real
context windows.
- baseUrl trailing slash is normalized.
- Duplicate ids in the API response are deduped (first wins).
- Non-2xx HTTP raises so the caller can fall back to the static catalog.
- Empty token / baseUrl reject synchronously without calling fetch.
Targeted run: `pnpm test extensions/github-copilot/models.test.ts` →
29/29 pass. `pnpm exec oxfmt --check extensions/github-copilot/` clean.
`pnpm tsgo:core` clean.
Real-world proof:
Built locally and dropped the resulting tarball into a downstream
container with `gh auth login --hostname github.com` (Copilot
subscription on the linked account). Before this change,
`openclaw models list --provider github-copilot` returned the 22-entry
static catalog with every entry showing 128k context. After this change,
the same command (with `--refresh`) returns 30 entries with API-accurate
context windows including the new gpt-5.1 family, the claude-opus-*-1m
variants, and the corrected `gemini-3*-preview` ids.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
