ci(release): split release soak validation

This commit is contained in:
Peter Steinberger
2026-05-04 23:24:43 +01:00
parent ac3cd1a0ca
commit 358cd87ff3
6 changed files with 131 additions and 53 deletions

View File

@@ -114,11 +114,13 @@ the maintainer-only release runbook.
- Run the manual `Full Release Validation` workflow before release approval to
kick off all pre-release test boxes from one entrypoint. It accepts a branch,
tag, or full commit SHA, dispatches manual `CI`, and dispatches
`OpenClaw Release Checks` for install smoke, package acceptance, Docker
release-path suites, live/E2E, OpenWebUI, QA Lab parity, Matrix, and Telegram
lanes. With `release_profile=full` and `rerun_group=all`, it also runs package
Telegram E2E against the `release-package-under-test` artifact from release
checks. Provide `npm_telegram_package_spec` after publishing when the same
`OpenClaw Release Checks` for install smoke, package acceptance, cross-OS
package checks, QA Lab parity, Matrix, and Telegram lanes. Stable/default runs
keep exhaustive live/E2E and Docker release-path soak behind
`run_release_soak=true`; `release_profile=full` forces soak on. With
`release_profile=full` and `rerun_group=all`, it also runs package Telegram
E2E against the `release-package-under-test` artifact from release checks.
Provide `npm_telegram_package_spec` after publishing when the same
Telegram E2E should prove the published npm package too. Provide
`package_acceptance_package_spec` after publishing when Package Acceptance
should run its package/update matrix against the shipped npm package instead
@@ -293,8 +295,8 @@ parent `release-package-under-test` artifact for package-facing checks, and
dispatches standalone package Telegram E2E when `release_profile=full` with
`rerun_group=all` or when `npm_telegram_package_spec` is set. `OpenClaw Release
Checks` then fans out install smoke, cross-OS release checks, live/E2E Docker
release-path coverage, Package Acceptance with Telegram package QA, QA Lab
parity, live Matrix, and live Telegram. A full run is only acceptable when the
release-path coverage when soak is enabled, Package Acceptance with Telegram
package QA, QA Lab parity, live Matrix, and live Telegram. A full run is only acceptable when the
`Full Release Validation`
summary shows `normal_ci` and `release_checks` as successful. In full/all mode,
the `npm_telegram` child must also be successful; outside full/all it is skipped
@@ -318,10 +320,15 @@ Use `release_profile` to select live/provider breadth:
- `stable`: minimum plus stable provider/backend coverage for release approval
- `full`: stable plus broad advisory provider/media coverage
Use `run_release_soak=true` with `stable` when the release-blocking lanes are
green and you want the exhaustive live/E2E, Docker release-path, and
all-since-2026.4.23 upgrade-survivor sweep before promotion. `full` implies
`run_release_soak=true`.
`OpenClaw Release Checks` uses the trusted workflow ref to resolve the target
ref once as `release-package-under-test` and reuses that artifact in both
release-path Docker checks and Package Acceptance. This keeps all
package-facing boxes on the same bytes and avoids repeated package builds.
ref once as `release-package-under-test` and reuses that artifact in cross-OS,
Package Acceptance, and release-path Docker checks when soak runs. This keeps
all package-facing boxes on the same bytes and avoids repeated package builds.
The cross-OS OpenAI install smoke uses `OPENCLAW_CROSS_OS_OPENAI_MODEL` when the
repo/org variable is set, otherwise `openai/gpt-5.4`, because this lane is
proving package install, onboarding, gateway startup, and one live agent turn
@@ -474,11 +481,12 @@ Supported candidate sources:
`OpenClaw Release Checks` runs Package Acceptance with `source=artifact`, the
prepared release package artifact, `suite_profile=custom`,
`docker_lanes=doctor-switch update-channel-switch upgrade-survivor published-upgrade-survivor plugins-offline plugin-update`,
`published_upgrade_survivor_baselines=all-since-2026.4.23`,
`published_upgrade_survivor_scenarios=reported-issues`, and
`telegram_mode=mock-openai`. Package Acceptance keeps migration, update, stale
plugin dependency cleanup, offline plugin fixtures, plugin update, and Telegram
package QA against the same resolved tarball. The upgrade matrix covers every stable npm-published baseline from `2026.4.23` through `latest`; use
package QA against the same resolved tarball. Blocking release checks use the
default latest published package baseline; `run_release_soak=true` or
`release_profile=full` expands to every stable npm-published baseline from
`2026.4.23` through `latest` plus reported-issue fixtures. Use
Package Acceptance with `source=npm` for an already shipped candidate, or
`source=ref`/`source=artifact` for a SHA-backed local npm tarball before
publish. It is the GitHub-native
@@ -615,6 +623,9 @@ OpenClaw package must not be published.
- `ref`: branch, tag, or full commit SHA to validate. Secret-bearing checks
require the resolved commit to be reachable from an OpenClaw branch or
release tag.
- `run_release_soak`: opt into exhaustive live/E2E, Docker release-path, and
all-since upgrade-survivor soak on stable/default release checks. It is forced
on by `release_profile=full`.
Rules: