mirror of
https://github.com/moltbot/moltbot.git
synced 2026-05-13 15:47:28 +00:00
ci: split release docker integration chunks
This commit is contained in:
@@ -286,15 +286,19 @@ generated inside GitHub artifacts include `package_artifact_run_id`,
|
||||
exact tarball and prepared images from the failed run. When the fix changes
|
||||
package contents, omit those reuse inputs so the workflow packs a new tarball.
|
||||
Live-only targeted reruns skip the E2E images and build only the live-test
|
||||
image. Release-path normal mode remains max three Docker chunk jobs:
|
||||
image. Release-path normal mode fans out into four Docker chunk jobs:
|
||||
|
||||
- `core`
|
||||
- `package-update`
|
||||
- `plugins-integrations`
|
||||
- `plugins-runtime`
|
||||
- `bundled-channels`
|
||||
|
||||
OpenWebUI is folded into `plugins-integrations` for full release-path coverage
|
||||
and keeps a standalone `openwebui` chunk only for OpenWebUI-only dispatches.
|
||||
The bundled-channel runtime-dependency coverage inside `plugins-integrations`
|
||||
OpenWebUI is folded into `plugins-runtime` for full release-path coverage and
|
||||
keeps a standalone `openwebui` chunk only for OpenWebUI-only dispatches. The
|
||||
legacy `plugins-integrations` chunk still works as an aggregate alias for manual
|
||||
reruns, but the release workflow uses the split chunks so plugin runtime checks
|
||||
and bundled-channel checks can run on separate machines. The bundled-channel
|
||||
runtime-dependency coverage inside `bundled-channels`
|
||||
uses the split `bundled-channel-*` and `bundled-channel-update-*` lanes rather
|
||||
than the serial `bundled-channel-deps` lane, so failures produce cheap targeted
|
||||
reruns for the exact channel/update scenario. The bundled plugin
|
||||
@@ -447,7 +451,7 @@ gh workflow run openclaw-live-and-e2e-checks-reusable.yml \
|
||||
That path still runs the prepare job, so it creates a new tarball for `<sha>`.
|
||||
If the SHA-tagged GHCR bare/functional image already exists, CI skips rebuilding
|
||||
that image and only uploads the fresh package artifact before the targeted lane
|
||||
job. Do not rerun the full three-chunk release path unless the failed lane list
|
||||
job. Do not rerun the full release path unless the failed lane list
|
||||
or touched surface really requires it.
|
||||
|
||||
## Docker Expected Timings
|
||||
|
||||
@@ -435,8 +435,11 @@ jobs:
|
||||
- chunk_id: package-update
|
||||
label: package/update
|
||||
timeout_minutes: 180
|
||||
- chunk_id: plugins-integrations
|
||||
label: plugins/integrations
|
||||
- chunk_id: plugins-runtime
|
||||
label: plugins/runtime
|
||||
timeout_minutes: 180
|
||||
- chunk_id: bundled-channels
|
||||
label: bundled channels
|
||||
timeout_minutes: 180
|
||||
env:
|
||||
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -688,7 +688,7 @@ The live-model Docker runners also bind-mount only the needed CLI auth homes (or
|
||||
Set `OPENCLAW_PLUGINS_E2E_CLAWHUB=0` to skip the live ClawHub block, or override the default package with `OPENCLAW_PLUGINS_E2E_CLAWHUB_SPEC` and `OPENCLAW_PLUGINS_E2E_CLAWHUB_ID`.
|
||||
- Plugin update unchanged smoke: `pnpm test:docker:plugin-update` (script: `scripts/e2e/plugin-update-unchanged-docker.sh`)
|
||||
- Config reload metadata smoke: `pnpm test:docker:config-reload` (script: `scripts/e2e/config-reload-source-docker.sh`)
|
||||
- Bundled plugin runtime deps: `pnpm test:docker:bundled-channel-deps` builds a small Docker runner image by default, builds and packs OpenClaw once on the host, then mounts that tarball into each Linux install scenario. Reuse the image with `OPENCLAW_SKIP_DOCKER_BUILD=1`, skip the host rebuild after a fresh local build with `OPENCLAW_BUNDLED_CHANNEL_HOST_BUILD=0`, or point at an existing tarball with `OPENCLAW_CURRENT_PACKAGE_TGZ=/path/to/openclaw-*.tgz`. The full Docker aggregate and release-path `plugins-integrations` chunk pre-pack this tarball once, then shard bundled channel checks into independent lanes, including separate update lanes for Telegram, Discord, Slack, Feishu, memory-lancedb, and ACPX. Use `OPENCLAW_BUNDLED_CHANNELS=telegram,slack` to narrow the channel matrix when running the bundled lane directly, or `OPENCLAW_BUNDLED_CHANNEL_UPDATE_TARGETS=telegram,acpx` to narrow the update scenario. The lane also verifies that `channels.<id>.enabled=false` and `plugins.entries.<id>.enabled=false` suppress doctor/runtime-dependency repair.
|
||||
- Bundled plugin runtime deps: `pnpm test:docker:bundled-channel-deps` builds a small Docker runner image by default, builds and packs OpenClaw once on the host, then mounts that tarball into each Linux install scenario. Reuse the image with `OPENCLAW_SKIP_DOCKER_BUILD=1`, skip the host rebuild after a fresh local build with `OPENCLAW_BUNDLED_CHANNEL_HOST_BUILD=0`, or point at an existing tarball with `OPENCLAW_CURRENT_PACKAGE_TGZ=/path/to/openclaw-*.tgz`. The full Docker aggregate and release-path `bundled-channels` chunk pre-pack this tarball once, then shard bundled channel checks into independent lanes, including separate update lanes for Telegram, Discord, Slack, Feishu, memory-lancedb, and ACPX. The legacy `plugins-integrations` chunk remains an aggregate alias for manual reruns. Use `OPENCLAW_BUNDLED_CHANNELS=telegram,slack` to narrow the channel matrix when running the bundled lane directly, or `OPENCLAW_BUNDLED_CHANNEL_UPDATE_TARGETS=telegram,acpx` to narrow the update scenario. The lane also verifies that `channels.<id>.enabled=false` and `plugins.entries.<id>.enabled=false` suppress doctor/runtime-dependency repair.
|
||||
- Narrow bundled plugin runtime deps while iterating by disabling unrelated scenarios, for example:
|
||||
`OPENCLAW_BUNDLED_CHANNEL_SCENARIOS=0 OPENCLAW_BUNDLED_CHANNEL_UPDATE_SCENARIO=0 OPENCLAW_BUNDLED_CHANNEL_ROOT_OWNED_SCENARIO=0 OPENCLAW_BUNDLED_CHANNEL_SETUP_ENTRY_SCENARIO=0 pnpm test:docker:bundled-channel-deps`.
|
||||
|
||||
|
||||
@@ -320,11 +320,11 @@ Release Docker coverage includes:
|
||||
|
||||
- full install smoke with the slow Bun global install smoke enabled
|
||||
- repository E2E lanes
|
||||
- release-path Docker chunks: `core`, `package-update`, and
|
||||
`plugins-integrations`
|
||||
- OpenWebUI coverage inside the `plugins-integrations` chunk when requested
|
||||
- split bundled-channel dependency lanes inside `plugins-integrations` instead
|
||||
of the serial all-in-one bundled-channel lane
|
||||
- release-path Docker chunks: `core`, `package-update`, `plugins-runtime`, and
|
||||
`bundled-channels`
|
||||
- OpenWebUI coverage inside the `plugins-runtime` chunk when requested
|
||||
- split bundled-channel dependency lanes in their own `bundled-channels` chunk
|
||||
instead of the serial all-in-one bundled-channel lane
|
||||
- split bundled plugin install/uninstall lanes
|
||||
`bundled-plugin-install-uninstall-0` through
|
||||
`bundled-plugin-install-uninstall-7`
|
||||
|
||||
@@ -351,6 +351,32 @@ export const tailLanes = [
|
||||
),
|
||||
];
|
||||
|
||||
const releasePathPluginRuntimeLanes = [
|
||||
lane("plugins", "OPENCLAW_SKIP_DOCKER_BUILD=1 pnpm test:docker:plugins", {
|
||||
resources: ["npm", "service"],
|
||||
weight: 6,
|
||||
}),
|
||||
...bundledPluginInstallUninstallLanes,
|
||||
serviceLane(
|
||||
"cron-mcp-cleanup",
|
||||
"OPENCLAW_SKIP_DOCKER_BUILD=1 pnpm test:docker:cron-mcp-cleanup",
|
||||
{
|
||||
resources: ["npm"],
|
||||
weight: 3,
|
||||
},
|
||||
),
|
||||
serviceLane(
|
||||
"openai-web-search-minimal",
|
||||
"OPENCLAW_SKIP_DOCKER_BUILD=1 pnpm test:docker:openai-web-search-minimal",
|
||||
{ timeoutMs: 8 * 60 * 1000 },
|
||||
),
|
||||
];
|
||||
|
||||
const releasePathBundledChannelLanes = [
|
||||
npmLane("plugin-update", "OPENCLAW_SKIP_DOCKER_BUILD=1 pnpm test:docker:plugin-update"),
|
||||
...bundledScenarioLanes,
|
||||
];
|
||||
|
||||
const releasePathChunks = {
|
||||
core: [
|
||||
lane("qr", "OPENCLAW_SKIP_DOCKER_BUILD=1 pnpm test:docker:qr"),
|
||||
@@ -398,31 +424,15 @@ const releasePathChunks = {
|
||||
},
|
||||
),
|
||||
],
|
||||
"plugins-integrations": [
|
||||
lane("plugins", "OPENCLAW_SKIP_DOCKER_BUILD=1 pnpm test:docker:plugins", {
|
||||
resources: ["npm", "service"],
|
||||
weight: 6,
|
||||
}),
|
||||
...bundledPluginInstallUninstallLanes,
|
||||
npmLane("plugin-update", "OPENCLAW_SKIP_DOCKER_BUILD=1 pnpm test:docker:plugin-update"),
|
||||
...bundledScenarioLanes,
|
||||
serviceLane(
|
||||
"cron-mcp-cleanup",
|
||||
"OPENCLAW_SKIP_DOCKER_BUILD=1 pnpm test:docker:cron-mcp-cleanup",
|
||||
{
|
||||
resources: ["npm"],
|
||||
weight: 3,
|
||||
},
|
||||
),
|
||||
serviceLane(
|
||||
"openai-web-search-minimal",
|
||||
"OPENCLAW_SKIP_DOCKER_BUILD=1 pnpm test:docker:openai-web-search-minimal",
|
||||
{ timeoutMs: 8 * 60 * 1000 },
|
||||
),
|
||||
],
|
||||
"plugins-runtime": releasePathPluginRuntimeLanes,
|
||||
"bundled-channels": releasePathBundledChannelLanes,
|
||||
openwebui: [],
|
||||
};
|
||||
|
||||
const legacyReleasePathChunks = {
|
||||
"plugins-integrations": [...releasePathPluginRuntimeLanes, ...releasePathBundledChannelLanes],
|
||||
};
|
||||
|
||||
function openWebUILane() {
|
||||
return serviceLane("openwebui", "OPENCLAW_SKIP_DOCKER_BUILD=1 pnpm test:docker:openwebui", {
|
||||
timeoutMs: OPENWEBUI_TIMEOUT_MS,
|
||||
@@ -431,16 +441,22 @@ function openWebUILane() {
|
||||
}
|
||||
|
||||
export function releasePathChunkLanes(chunk, options = {}) {
|
||||
const base = releasePathChunks[chunk];
|
||||
const base = releasePathChunks[chunk] ?? legacyReleasePathChunks[chunk];
|
||||
if (!base) {
|
||||
throw new Error(
|
||||
`OPENCLAW_DOCKER_ALL_CHUNK must be one of: ${Object.keys(releasePathChunks).join(", ")}. Got: ${JSON.stringify(chunk)}`,
|
||||
`OPENCLAW_DOCKER_ALL_CHUNK must be one of: ${[
|
||||
...Object.keys(releasePathChunks),
|
||||
...Object.keys(legacyReleasePathChunks),
|
||||
].join(", ")}. Got: ${JSON.stringify(chunk)}`,
|
||||
);
|
||||
}
|
||||
if (chunk === "openwebui") {
|
||||
return options.includeOpenWebUI ? [openWebUILane()] : [];
|
||||
}
|
||||
if (chunk !== "plugins-integrations" || !options.includeOpenWebUI) {
|
||||
if (
|
||||
(chunk !== "plugins-runtime" && chunk !== "plugins-integrations") ||
|
||||
!options.includeOpenWebUI
|
||||
) {
|
||||
return base;
|
||||
}
|
||||
return [...base, openWebUILane()];
|
||||
|
||||
@@ -68,6 +68,58 @@ describe("scripts/lib/docker-e2e-plan", () => {
|
||||
expect(withOpenWebUI.lanes.map((lane) => lane.name)).toContain("openwebui");
|
||||
});
|
||||
|
||||
it("splits the old plugins/integrations release chunk across plugin and bundled-channel chunks", () => {
|
||||
const pluginsRuntime = planFor({
|
||||
includeOpenWebUI: true,
|
||||
profile: RELEASE_PATH_PROFILE,
|
||||
releaseChunk: "plugins-runtime",
|
||||
});
|
||||
const bundledChannels = planFor({
|
||||
includeOpenWebUI: true,
|
||||
profile: RELEASE_PATH_PROFILE,
|
||||
releaseChunk: "bundled-channels",
|
||||
});
|
||||
|
||||
expect(pluginsRuntime.lanes.map((lane) => lane.name)).toEqual(
|
||||
expect.arrayContaining([
|
||||
"plugins",
|
||||
"bundled-plugin-install-uninstall-0",
|
||||
"bundled-plugin-install-uninstall-7",
|
||||
"cron-mcp-cleanup",
|
||||
"openai-web-search-minimal",
|
||||
"openwebui",
|
||||
]),
|
||||
);
|
||||
expect(pluginsRuntime.lanes.map((lane) => lane.name)).not.toContain("bundled-channel-telegram");
|
||||
expect(bundledChannels.lanes.map((lane) => lane.name)).toEqual(
|
||||
expect.arrayContaining([
|
||||
"plugin-update",
|
||||
"bundled-channel-telegram",
|
||||
"bundled-channel-update-acpx",
|
||||
]),
|
||||
);
|
||||
expect(bundledChannels.lanes.map((lane) => lane.name)).not.toContain("plugins");
|
||||
expect(bundledChannels.lanes.map((lane) => lane.name)).not.toContain("openwebui");
|
||||
});
|
||||
|
||||
it("keeps the legacy plugins-integrations release chunk as an aggregate alias", () => {
|
||||
const legacy = planFor({
|
||||
includeOpenWebUI: true,
|
||||
profile: RELEASE_PATH_PROFILE,
|
||||
releaseChunk: "plugins-integrations",
|
||||
});
|
||||
|
||||
expect(legacy.lanes.map((lane) => lane.name)).toEqual(
|
||||
expect.arrayContaining([
|
||||
"plugins",
|
||||
"bundled-plugin-install-uninstall-0",
|
||||
"plugin-update",
|
||||
"bundled-channel-update-acpx",
|
||||
"openwebui",
|
||||
]),
|
||||
);
|
||||
});
|
||||
|
||||
it("plans a live-only selected lane without package e2e images", () => {
|
||||
const plan = planFor({ selectedLaneNames: ["live-models"] });
|
||||
|
||||
|
||||
Reference in New Issue
Block a user