ci: split release docker integration chunks

This commit is contained in:
Peter Steinberger
2026-04-27 13:24:13 +01:00
parent 750c180a6c
commit 98b441edb1
7 changed files with 115 additions and 40 deletions

View File

@@ -286,15 +286,19 @@ generated inside GitHub artifacts include `package_artifact_run_id`,
exact tarball and prepared images from the failed run. When the fix changes
package contents, omit those reuse inputs so the workflow packs a new tarball.
Live-only targeted reruns skip the E2E images and build only the live-test
image. Release-path normal mode remains max three Docker chunk jobs:
image. Release-path normal mode fans out into four Docker chunk jobs:
- `core`
- `package-update`
- `plugins-integrations`
- `plugins-runtime`
- `bundled-channels`
OpenWebUI is folded into `plugins-integrations` for full release-path coverage
and keeps a standalone `openwebui` chunk only for OpenWebUI-only dispatches.
The bundled-channel runtime-dependency coverage inside `plugins-integrations`
OpenWebUI is folded into `plugins-runtime` for full release-path coverage and
keeps a standalone `openwebui` chunk only for OpenWebUI-only dispatches. The
legacy `plugins-integrations` chunk still works as an aggregate alias for manual
reruns, but the release workflow uses the split chunks so plugin runtime checks
and bundled-channel checks can run on separate machines. The bundled-channel
runtime-dependency coverage inside `bundled-channels`
uses the split `bundled-channel-*` and `bundled-channel-update-*` lanes rather
than the serial `bundled-channel-deps` lane, so failures produce cheap targeted
reruns for the exact channel/update scenario. The bundled plugin
@@ -447,7 +451,7 @@ gh workflow run openclaw-live-and-e2e-checks-reusable.yml \
That path still runs the prepare job, so it creates a new tarball for `<sha>`.
If the SHA-tagged GHCR bare/functional image already exists, CI skips rebuilding
that image and only uploads the fresh package artifact before the targeted lane
job. Do not rerun the full three-chunk release path unless the failed lane list
job. Do not rerun the full release path unless the failed lane list
or touched surface really requires it.
## Docker Expected Timings

View File

@@ -435,8 +435,11 @@ jobs:
- chunk_id: package-update
label: package/update
timeout_minutes: 180
- chunk_id: plugins-integrations
label: plugins/integrations
- chunk_id: plugins-runtime
label: plugins/runtime
timeout_minutes: 180
- chunk_id: bundled-channels
label: bundled channels
timeout_minutes: 180
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}

File diff suppressed because one or more lines are too long

View File

@@ -688,7 +688,7 @@ The live-model Docker runners also bind-mount only the needed CLI auth homes (or
Set `OPENCLAW_PLUGINS_E2E_CLAWHUB=0` to skip the live ClawHub block, or override the default package with `OPENCLAW_PLUGINS_E2E_CLAWHUB_SPEC` and `OPENCLAW_PLUGINS_E2E_CLAWHUB_ID`.
- Plugin update unchanged smoke: `pnpm test:docker:plugin-update` (script: `scripts/e2e/plugin-update-unchanged-docker.sh`)
- Config reload metadata smoke: `pnpm test:docker:config-reload` (script: `scripts/e2e/config-reload-source-docker.sh`)
- Bundled plugin runtime deps: `pnpm test:docker:bundled-channel-deps` builds a small Docker runner image by default, builds and packs OpenClaw once on the host, then mounts that tarball into each Linux install scenario. Reuse the image with `OPENCLAW_SKIP_DOCKER_BUILD=1`, skip the host rebuild after a fresh local build with `OPENCLAW_BUNDLED_CHANNEL_HOST_BUILD=0`, or point at an existing tarball with `OPENCLAW_CURRENT_PACKAGE_TGZ=/path/to/openclaw-*.tgz`. The full Docker aggregate and release-path `plugins-integrations` chunk pre-pack this tarball once, then shard bundled channel checks into independent lanes, including separate update lanes for Telegram, Discord, Slack, Feishu, memory-lancedb, and ACPX. Use `OPENCLAW_BUNDLED_CHANNELS=telegram,slack` to narrow the channel matrix when running the bundled lane directly, or `OPENCLAW_BUNDLED_CHANNEL_UPDATE_TARGETS=telegram,acpx` to narrow the update scenario. The lane also verifies that `channels.<id>.enabled=false` and `plugins.entries.<id>.enabled=false` suppress doctor/runtime-dependency repair.
- Bundled plugin runtime deps: `pnpm test:docker:bundled-channel-deps` builds a small Docker runner image by default, builds and packs OpenClaw once on the host, then mounts that tarball into each Linux install scenario. Reuse the image with `OPENCLAW_SKIP_DOCKER_BUILD=1`, skip the host rebuild after a fresh local build with `OPENCLAW_BUNDLED_CHANNEL_HOST_BUILD=0`, or point at an existing tarball with `OPENCLAW_CURRENT_PACKAGE_TGZ=/path/to/openclaw-*.tgz`. The full Docker aggregate and release-path `bundled-channels` chunk pre-pack this tarball once, then shard bundled channel checks into independent lanes, including separate update lanes for Telegram, Discord, Slack, Feishu, memory-lancedb, and ACPX. The legacy `plugins-integrations` chunk remains an aggregate alias for manual reruns. Use `OPENCLAW_BUNDLED_CHANNELS=telegram,slack` to narrow the channel matrix when running the bundled lane directly, or `OPENCLAW_BUNDLED_CHANNEL_UPDATE_TARGETS=telegram,acpx` to narrow the update scenario. The lane also verifies that `channels.<id>.enabled=false` and `plugins.entries.<id>.enabled=false` suppress doctor/runtime-dependency repair.
- Narrow bundled plugin runtime deps while iterating by disabling unrelated scenarios, for example:
`OPENCLAW_BUNDLED_CHANNEL_SCENARIOS=0 OPENCLAW_BUNDLED_CHANNEL_UPDATE_SCENARIO=0 OPENCLAW_BUNDLED_CHANNEL_ROOT_OWNED_SCENARIO=0 OPENCLAW_BUNDLED_CHANNEL_SETUP_ENTRY_SCENARIO=0 pnpm test:docker:bundled-channel-deps`.

View File

@@ -320,11 +320,11 @@ Release Docker coverage includes:
- full install smoke with the slow Bun global install smoke enabled
- repository E2E lanes
- release-path Docker chunks: `core`, `package-update`, and
`plugins-integrations`
- OpenWebUI coverage inside the `plugins-integrations` chunk when requested
- split bundled-channel dependency lanes inside `plugins-integrations` instead
of the serial all-in-one bundled-channel lane
- release-path Docker chunks: `core`, `package-update`, `plugins-runtime`, and
`bundled-channels`
- OpenWebUI coverage inside the `plugins-runtime` chunk when requested
- split bundled-channel dependency lanes in their own `bundled-channels` chunk
instead of the serial all-in-one bundled-channel lane
- split bundled plugin install/uninstall lanes
`bundled-plugin-install-uninstall-0` through
`bundled-plugin-install-uninstall-7`

View File

@@ -351,6 +351,32 @@ export const tailLanes = [
),
];
const releasePathPluginRuntimeLanes = [
lane("plugins", "OPENCLAW_SKIP_DOCKER_BUILD=1 pnpm test:docker:plugins", {
resources: ["npm", "service"],
weight: 6,
}),
...bundledPluginInstallUninstallLanes,
serviceLane(
"cron-mcp-cleanup",
"OPENCLAW_SKIP_DOCKER_BUILD=1 pnpm test:docker:cron-mcp-cleanup",
{
resources: ["npm"],
weight: 3,
},
),
serviceLane(
"openai-web-search-minimal",
"OPENCLAW_SKIP_DOCKER_BUILD=1 pnpm test:docker:openai-web-search-minimal",
{ timeoutMs: 8 * 60 * 1000 },
),
];
const releasePathBundledChannelLanes = [
npmLane("plugin-update", "OPENCLAW_SKIP_DOCKER_BUILD=1 pnpm test:docker:plugin-update"),
...bundledScenarioLanes,
];
const releasePathChunks = {
core: [
lane("qr", "OPENCLAW_SKIP_DOCKER_BUILD=1 pnpm test:docker:qr"),
@@ -398,31 +424,15 @@ const releasePathChunks = {
},
),
],
"plugins-integrations": [
lane("plugins", "OPENCLAW_SKIP_DOCKER_BUILD=1 pnpm test:docker:plugins", {
resources: ["npm", "service"],
weight: 6,
}),
...bundledPluginInstallUninstallLanes,
npmLane("plugin-update", "OPENCLAW_SKIP_DOCKER_BUILD=1 pnpm test:docker:plugin-update"),
...bundledScenarioLanes,
serviceLane(
"cron-mcp-cleanup",
"OPENCLAW_SKIP_DOCKER_BUILD=1 pnpm test:docker:cron-mcp-cleanup",
{
resources: ["npm"],
weight: 3,
},
),
serviceLane(
"openai-web-search-minimal",
"OPENCLAW_SKIP_DOCKER_BUILD=1 pnpm test:docker:openai-web-search-minimal",
{ timeoutMs: 8 * 60 * 1000 },
),
],
"plugins-runtime": releasePathPluginRuntimeLanes,
"bundled-channels": releasePathBundledChannelLanes,
openwebui: [],
};
const legacyReleasePathChunks = {
"plugins-integrations": [...releasePathPluginRuntimeLanes, ...releasePathBundledChannelLanes],
};
function openWebUILane() {
return serviceLane("openwebui", "OPENCLAW_SKIP_DOCKER_BUILD=1 pnpm test:docker:openwebui", {
timeoutMs: OPENWEBUI_TIMEOUT_MS,
@@ -431,16 +441,22 @@ function openWebUILane() {
}
export function releasePathChunkLanes(chunk, options = {}) {
const base = releasePathChunks[chunk];
const base = releasePathChunks[chunk] ?? legacyReleasePathChunks[chunk];
if (!base) {
throw new Error(
`OPENCLAW_DOCKER_ALL_CHUNK must be one of: ${Object.keys(releasePathChunks).join(", ")}. Got: ${JSON.stringify(chunk)}`,
`OPENCLAW_DOCKER_ALL_CHUNK must be one of: ${[
...Object.keys(releasePathChunks),
...Object.keys(legacyReleasePathChunks),
].join(", ")}. Got: ${JSON.stringify(chunk)}`,
);
}
if (chunk === "openwebui") {
return options.includeOpenWebUI ? [openWebUILane()] : [];
}
if (chunk !== "plugins-integrations" || !options.includeOpenWebUI) {
if (
(chunk !== "plugins-runtime" && chunk !== "plugins-integrations") ||
!options.includeOpenWebUI
) {
return base;
}
return [...base, openWebUILane()];

View File

@@ -68,6 +68,58 @@ describe("scripts/lib/docker-e2e-plan", () => {
expect(withOpenWebUI.lanes.map((lane) => lane.name)).toContain("openwebui");
});
it("splits the old plugins/integrations release chunk across plugin and bundled-channel chunks", () => {
const pluginsRuntime = planFor({
includeOpenWebUI: true,
profile: RELEASE_PATH_PROFILE,
releaseChunk: "plugins-runtime",
});
const bundledChannels = planFor({
includeOpenWebUI: true,
profile: RELEASE_PATH_PROFILE,
releaseChunk: "bundled-channels",
});
expect(pluginsRuntime.lanes.map((lane) => lane.name)).toEqual(
expect.arrayContaining([
"plugins",
"bundled-plugin-install-uninstall-0",
"bundled-plugin-install-uninstall-7",
"cron-mcp-cleanup",
"openai-web-search-minimal",
"openwebui",
]),
);
expect(pluginsRuntime.lanes.map((lane) => lane.name)).not.toContain("bundled-channel-telegram");
expect(bundledChannels.lanes.map((lane) => lane.name)).toEqual(
expect.arrayContaining([
"plugin-update",
"bundled-channel-telegram",
"bundled-channel-update-acpx",
]),
);
expect(bundledChannels.lanes.map((lane) => lane.name)).not.toContain("plugins");
expect(bundledChannels.lanes.map((lane) => lane.name)).not.toContain("openwebui");
});
it("keeps the legacy plugins-integrations release chunk as an aggregate alias", () => {
const legacy = planFor({
includeOpenWebUI: true,
profile: RELEASE_PATH_PROFILE,
releaseChunk: "plugins-integrations",
});
expect(legacy.lanes.map((lane) => lane.name)).toEqual(
expect.arrayContaining([
"plugins",
"bundled-plugin-install-uninstall-0",
"plugin-update",
"bundled-channel-update-acpx",
"openwebui",
]),
);
});
it("plans a live-only selected lane without package e2e images", () => {
const plan = planFor({ selectedLaneNames: ["live-models"] });