refactor: replace 156k-line generated baselines with SHA-256 hash files

Config and Plugin SDK drift detection now compares SHA-256 hashes instead
of full JSON content. The .sha256 files (6 lines total) are tracked in git;
the full JSON baselines are gitignored and generated locally for inspection.

Same CI guarantee, zero repo churn on schema changes.
This commit is contained in:
Peter Steinberger
2026-04-04 16:46:07 +09:00
parent b4e9802ef3
commit b5265a07d7
17 changed files with 124 additions and 156374 deletions

4
.gitignore vendored
View File

@@ -136,6 +136,10 @@ ui/src/ui/views/__screenshots__
ui/.vitest-attachments
docs/superpowers
# Generated docs baseline artifacts (locally generated, only hashes tracked)
docs/.generated/*.json
docs/.generated/*.jsonl
# Deprecated changelog fragment workflow
changelog/fragments/

View File

@@ -130,10 +130,10 @@
- Formatting gate: the pre-commit hook runs `pnpm format` before `pnpm check`. If you want a formatting-only preflight locally, run `pnpm format` explicitly.
- If you need a fast commit loop, `FAST_COMMIT=1 git commit ...` skips the hooks repo-wide `pnpm format` and `pnpm check`; use that only when you are deliberately covering the touched surface some other way.
- Tests: `pnpm test` (vitest); coverage: `pnpm test:coverage`
- Generated baseline artifacts live together under `docs/.generated/`.
- Generated baseline drift detection uses SHA-256 hash files under `docs/.generated/` (`.sha256` files tracked in git; full JSON baselines are gitignored, generated locally for inspection).
- Config schema drift uses `pnpm config:docs:gen` / `pnpm config:docs:check`.
- Plugin SDK API drift uses `pnpm plugin-sdk:api:gen` / `pnpm plugin-sdk:api:check`.
- If you change config schema/help or the public Plugin SDK surface, update the matching baseline artifact and keep the two drift-check flows adjacent in scripts/workflows/docs guidance rather than inventing a third pattern.
- If you change config schema/help or the public Plugin SDK surface, run the matching gen command and commit the updated `.sha256` hash file. Keep the two drift-check flows adjacent in scripts/workflows/docs guidance rather than inventing a third pattern.
- For narrowly scoped changes, prefer narrowly scoped tests that directly validate the touched behavior. If no meaningful scoped test exists, say so explicitly and use the next most direct validation available.
- Verification modes for work on `main`:
- Default mode: `main` is relatively stable. Count pre-commit hook coverage when it already verified the current tree, avoid rerunning the exact same checks just for ceremony, and prefer keeping CI/main green before landing.

View File

@@ -1,14 +1,21 @@
# Generated Docs Artifacts
These baseline artifacts are generated from the repo-owned OpenClaw config schema and bundled channel/plugin metadata.
SHA-256 hash files are the tracked drift-detection artifacts. The full JSON
baselines are generated locally (gitignored) for inspection only.
- Do not edit `config-baseline.json` by hand.
- Do not edit `config-baseline.core.json` by hand.
- Do not edit `config-baseline.channel.json` by hand.
- Do not edit `config-baseline.plugin.json` by hand.
- Do not edit `plugin-sdk-api-baseline.json` by hand.
- Do not edit `plugin-sdk-api-baseline.jsonl` by hand.
- Regenerate config baseline artifacts with `pnpm config:docs:gen`.
- Validate config baseline artifacts in CI or locally with `pnpm config:docs:check`.
- Regenerate Plugin SDK API baseline artifacts with `pnpm plugin-sdk:api:gen`.
- Validate Plugin SDK API baseline artifacts in CI or locally with `pnpm plugin-sdk:api:check`.
**Tracked (committed to git):**
- `config-baseline.sha256` — hashes of config baseline JSON artifacts.
- `plugin-sdk-api-baseline.sha256` — hashes of Plugin SDK API baseline artifacts.
**Local only (gitignored):**
- `config-baseline.json`, `config-baseline.core.json`, `config-baseline.channel.json`, `config-baseline.plugin.json`
- `plugin-sdk-api-baseline.json`, `plugin-sdk-api-baseline.jsonl`
Do not edit any of these files by hand.
- Regenerate config baseline: `pnpm config:docs:gen`
- Validate config baseline: `pnpm config:docs:check`
- Regenerate Plugin SDK API baseline: `pnpm plugin-sdk:api:gen`
- Validate Plugin SDK API baseline: `pnpm plugin-sdk:api:check`

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,4 @@
ad87e3ff267b151ae163402f3cb52503e10641e332bcfbb6a574bbd7087a2484 config-baseline.json
03ff4a3e314f17dd8851aed3653269294bc62412bee05a6804dce840bd3d7551 config-baseline.core.json
73b57f395a2ad983f1660112d0b2b998342f1ddbe3089b440d7f73d0665de739 config-baseline.channel.json
9d5cb864e70768b66c1ecd881a9a584b7696ef2e5b32df686cfdc3fa21ddabbe config-baseline.plugin.json

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,2 @@
80a8238588cca3c6e0f89115f4271834b6e18f3497f70bc91dccc9203539cdd9 plugin-sdk-api-baseline.json
68ed97a285d82f995f377db7823104976e8ebab5c6d0750332acb1d3d79288ef plugin-sdk-api-baseline.jsonl

View File

@@ -19,24 +19,14 @@ const result = await writeConfigDocBaselineArtifacts({
if (checkOnly) {
if (!result.changed) {
console.log(
[
`OK ${path.relative(repoRoot, result.jsonPaths.combined)}`,
`OK ${path.relative(repoRoot, result.jsonPaths.core)}`,
`OK ${path.relative(repoRoot, result.jsonPaths.channel)}`,
`OK ${path.relative(repoRoot, result.jsonPaths.plugin)}`,
].join("\n"),
);
console.log(`OK ${path.relative(repoRoot, result.hashPath)}`);
process.exit(0);
}
console.error(
[
"Config baseline drift detected.",
`Expected current: ${path.relative(repoRoot, result.jsonPaths.combined)}`,
`Expected current: ${path.relative(repoRoot, result.jsonPaths.core)}`,
`Expected current: ${path.relative(repoRoot, result.jsonPaths.channel)}`,
`Expected current: ${path.relative(repoRoot, result.jsonPaths.plugin)}`,
"If this config-surface change is intentional, run `pnpm config:docs:gen` and commit the updated baseline files.",
`Hash mismatch: ${path.relative(repoRoot, result.hashPath)}`,
"If this config-surface change is intentional, run `pnpm config:docs:gen` and commit the updated hash file.",
"If not intentional, treat this as docs drift or a possible breaking config change and fix the schema/help changes first.",
].join("\n"),
);
@@ -45,9 +35,10 @@ if (checkOnly) {
console.log(
[
`Wrote ${path.relative(repoRoot, result.jsonPaths.combined)}`,
`Wrote ${path.relative(repoRoot, result.jsonPaths.core)}`,
`Wrote ${path.relative(repoRoot, result.jsonPaths.channel)}`,
`Wrote ${path.relative(repoRoot, result.jsonPaths.plugin)}`,
`Wrote ${path.relative(repoRoot, result.hashPath)}`,
`Wrote ${path.relative(repoRoot, result.jsonPaths.combined)} (gitignored, local only)`,
`Wrote ${path.relative(repoRoot, result.jsonPaths.core)} (gitignored, local only)`,
`Wrote ${path.relative(repoRoot, result.jsonPaths.channel)} (gitignored, local only)`,
`Wrote ${path.relative(repoRoot, result.jsonPaths.plugin)} (gitignored, local only)`,
].join("\n"),
);

View File

@@ -24,23 +24,21 @@ async function main(): Promise<void> {
console.error(
[
"Plugin SDK API baseline drift detected.",
`Expected current: ${path.relative(repoRoot, result.jsonPath)}`,
`Expected current: ${path.relative(repoRoot, result.statefilePath)}`,
"If this Plugin SDK surface change is intentional, run `pnpm plugin-sdk:api:gen` and commit the updated baseline files.",
`Hash mismatch: ${path.relative(repoRoot, result.hashPath)}`,
"If this Plugin SDK surface change is intentional, run `pnpm plugin-sdk:api:gen` and commit the updated hash file.",
"If not intentional, treat this as API drift and fix the plugin-sdk exports or metadata first.",
].join("\n"),
);
process.exit(1);
}
console.log(
`OK ${path.relative(repoRoot, result.jsonPath)} ${path.relative(repoRoot, result.statefilePath)}`,
);
console.log(`OK ${path.relative(repoRoot, result.hashPath)}`);
return;
}
console.log(
[
`Wrote ${path.relative(repoRoot, result.jsonPath)}`,
`Wrote ${path.relative(repoRoot, result.statefilePath)}`,
`Wrote ${path.relative(repoRoot, result.hashPath)}`,
`Wrote ${path.relative(repoRoot, result.jsonPath)} (gitignored, local only)`,
`Wrote ${path.relative(repoRoot, result.statefilePath)} (gitignored, local only)`,
].join("\n"),
);
}

View File

@@ -3,9 +3,7 @@ import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import {
type ConfigDocBaseline,
type ConfigDocBaselineEntry,
type ConfigDocBaselineArtifacts,
flattenConfigDocBaselineEntries,
renderConfigDocBaselineArtifacts,
writeConfigDocBaselineArtifacts,
@@ -13,41 +11,19 @@ import {
describe("config doc baseline integration", () => {
const tempRoots: string[] = [];
const generatedBaselinePaths = {
combined: path.resolve(process.cwd(), "docs/.generated/config-baseline.json"),
core: path.resolve(process.cwd(), "docs/.generated/config-baseline.core.json"),
channel: path.resolve(process.cwd(), "docs/.generated/config-baseline.channel.json"),
plugin: path.resolve(process.cwd(), "docs/.generated/config-baseline.plugin.json"),
} satisfies Record<keyof ConfigDocBaselineArtifacts, string>;
let sharedBaselinePromise: Promise<ConfigDocBaseline> | null = null;
let sharedRenderedPromise: Promise<
Awaited<ReturnType<typeof renderConfigDocBaselineArtifacts>>
> | null = null;
const sharedGeneratedJsonPromises: Partial<
Record<keyof ConfigDocBaselineArtifacts, Promise<string>>
> = {};
let sharedByPathPromise: Promise<Map<string, ConfigDocBaselineEntry>> | null = null;
function getSharedBaseline() {
sharedBaselinePromise ??= fs
.readFile(generatedBaselinePaths.combined, "utf8")
.then((raw) => JSON.parse(raw) as ConfigDocBaseline);
return sharedBaselinePromise;
}
function getSharedRendered() {
sharedRenderedPromise ??= renderConfigDocBaselineArtifacts(getSharedBaseline());
sharedRenderedPromise ??= renderConfigDocBaselineArtifacts();
return sharedRenderedPromise;
}
function getGeneratedJson(kind: keyof ConfigDocBaselineArtifacts) {
sharedGeneratedJsonPromises[kind] ??= fs.readFile(generatedBaselinePaths[kind], "utf8");
return sharedGeneratedJsonPromises[kind];
}
function getSharedByPath() {
sharedByPathPromise ??= getSharedBaseline().then(
(baseline) =>
sharedByPathPromise ??= getSharedRendered().then(
({ baseline }) =>
new Map(flattenConfigDocBaselineEntries(baseline).map((entry) => [entry.path, entry])),
);
return sharedByPathPromise;
@@ -62,7 +38,7 @@ describe("config doc baseline integration", () => {
});
it("is deterministic across repeated runs", async () => {
const baseline = await getSharedBaseline();
const { baseline } = await getSharedRendered();
const first = await renderConfigDocBaselineArtifacts(baseline);
const second = await renderConfigDocBaselineArtifacts(baseline);
@@ -72,22 +48,6 @@ describe("config doc baseline integration", () => {
expect(second.json.plugin).toBe(first.json.plugin);
});
it("matches the checked-in generated baseline artifacts", async () => {
const [rendered, generatedCombined, generatedCore, generatedChannel, generatedPlugin] =
await Promise.all([
getSharedRendered(),
getGeneratedJson("combined"),
getGeneratedJson("core"),
getGeneratedJson("channel"),
getGeneratedJson("plugin"),
]);
expect(rendered.json.combined).toBe(generatedCombined);
expect(rendered.json.core).toBe(generatedCore);
expect(rendered.json.channel).toBe(generatedChannel);
expect(rendered.json.plugin).toBe(generatedPlugin);
});
it("includes core, channel, and plugin config metadata", async () => {
const byPath = await getSharedByPath();
@@ -160,44 +120,33 @@ describe("config doc baseline integration", () => {
expect(byPath.get("bindings.*.match.peer.id")).toBeDefined();
});
it("supports check mode for stale generated artifacts", async () => {
it("supports check mode for stale hash files", async () => {
const tempRoot = await fs.mkdtemp(path.join(os.tmpdir(), "openclaw-config-doc-baseline-"));
tempRoots.push(tempRoot);
const rendered = getSharedRendered();
const initial = await writeConfigDocBaselineArtifacts({
repoRoot: tempRoot,
combinedPath: "docs/.generated/config-baseline.json",
corePath: "docs/.generated/config-baseline.core.json",
channelPath: "docs/.generated/config-baseline.channel.json",
pluginPath: "docs/.generated/config-baseline.plugin.json",
rendered,
});
expect(initial.wrote).toBe(true);
const current = await writeConfigDocBaselineArtifacts({
repoRoot: tempRoot,
combinedPath: "docs/.generated/config-baseline.json",
corePath: "docs/.generated/config-baseline.core.json",
channelPath: "docs/.generated/config-baseline.channel.json",
pluginPath: "docs/.generated/config-baseline.plugin.json",
check: true,
rendered,
});
expect(current.changed).toBe(false);
// Corrupt the hash file to simulate drift
await fs.writeFile(
path.join(tempRoot, "docs/.generated/config-baseline.json"),
'{"generatedBy":"broken","entries":[]}\n',
path.join(tempRoot, "docs/.generated/config-baseline.sha256"),
"0000000000000000000000000000000000000000000000000000000000000000 config-baseline.json\n",
"utf8",
);
const stale = await writeConfigDocBaselineArtifacts({
repoRoot: tempRoot,
combinedPath: "docs/.generated/config-baseline.json",
corePath: "docs/.generated/config-baseline.core.json",
channelPath: "docs/.generated/config-baseline.channel.json",
pluginPath: "docs/.generated/config-baseline.plugin.json",
check: true,
rendered,
});

View File

@@ -1,3 +1,4 @@
import { createHash } from "node:crypto";
import fsSync from "node:fs";
import os from "node:os";
import path from "node:path";
@@ -78,6 +79,7 @@ export type ConfigDocBaselineArtifactsWriteResult = {
changed: boolean;
wrote: boolean;
jsonPaths: ConfigDocBaselineArtifactPaths;
hashPath: string;
};
const GENERATED_BY = "scripts/generate-config-doc-baseline.ts" as const;
@@ -85,6 +87,7 @@ const DEFAULT_COMBINED_OUTPUT = "docs/.generated/config-baseline.json";
const DEFAULT_CORE_OUTPUT = "docs/.generated/config-baseline.core.json";
const DEFAULT_CHANNEL_OUTPUT = "docs/.generated/config-baseline.channel.json";
const DEFAULT_PLUGIN_OUTPUT = "docs/.generated/config-baseline.plugin.json";
const DEFAULT_HASH_OUTPUT = "docs/.generated/config-baseline.sha256";
let cachedConfigDocBaselinePromise: Promise<ConfigDocBaseline> | null = null;
let cachedDocBaselineRuntimePromise: Promise<typeof import("./doc-baseline.runtime.js")> | null =
null;
@@ -587,7 +590,7 @@ export async function renderConfigDocBaselineArtifacts(
};
}
async function readIfExists(filePath: string): Promise<string | null> {
function readFileIfExists(filePath: string): string | null {
try {
return fsSync.readFileSync(filePath, "utf8");
} catch {
@@ -595,14 +598,24 @@ async function readIfExists(filePath: string): Promise<string | null> {
}
}
async function writeIfChanged(filePath: string, next: string): Promise<boolean> {
const current = await readIfExists(filePath);
if (current === next) {
return false;
}
function writeFileAtomic(filePath: string, content: string): void {
fsSync.mkdirSync(path.dirname(filePath), { recursive: true });
fsSync.writeFileSync(filePath, next, "utf8");
return true;
fsSync.writeFileSync(filePath, content, "utf8");
}
function sha256(content: string): string {
return createHash("sha256").update(content, "utf8").digest("hex");
}
/** Build the sha256 hash file content for all config baseline artifacts. */
export function computeConfigBaselineHashFileContent(json: ConfigDocBaselineArtifacts): string {
const lines = [
`${sha256(json.combined)} config-baseline.json`,
`${sha256(json.core)} config-baseline.core.json`,
`${sha256(json.channel)} config-baseline.channel.json`,
`${sha256(json.plugin)} config-baseline.plugin.json`,
];
return `${lines.join("\n")}\n`;
}
function resolveBaselineArtifactPaths(
@@ -629,29 +642,24 @@ export async function writeConfigDocBaselineArtifacts(params?: {
corePath?: string;
channelPath?: string;
pluginPath?: string;
hashPath?: string;
rendered?: ConfigDocBaselineArtifactsRender | Promise<ConfigDocBaselineArtifactsRender>;
}): Promise<ConfigDocBaselineArtifactsWriteResult> {
const start = Date.now();
logConfigDocBaselineDebug("write artifacts start");
const repoRoot = params?.repoRoot ?? resolveRepoRoot();
const jsonPaths = resolveBaselineArtifactPaths(repoRoot, params);
const hashPath = path.resolve(repoRoot, params?.hashPath ?? DEFAULT_HASH_OUTPUT);
const rendered = params?.rendered
? await params.rendered
: await renderConfigDocBaselineArtifacts();
logConfigDocBaselineDebug(`render artifacts done elapsedMs=${Date.now() - start}`);
const current = await Promise.all(
Object.entries(jsonPaths).map(async ([key, filePath]) => [key, await readIfExists(filePath)]),
);
const currentByKey = Object.fromEntries(current) as Record<
keyof ConfigDocBaselineArtifacts,
string | null
>;
const changed = (Object.keys(jsonPaths) as Array<keyof ConfigDocBaselineArtifacts>).some(
(key) => currentByKey[key] !== rendered.json[key],
);
const nextHashContent = computeConfigBaselineHashFileContent(rendered.json);
const currentHashContent = readFileIfExists(hashPath);
const changed = currentHashContent !== nextHashContent;
logConfigDocBaselineDebug(
`compare artifacts done changed=${changed} elapsedMs=${Date.now() - start}`,
`compare hashes done changed=${changed} elapsedMs=${Date.now() - start}`,
);
if (params?.check) {
@@ -659,18 +667,23 @@ export async function writeConfigDocBaselineArtifacts(params?: {
changed,
wrote: false,
jsonPaths,
hashPath,
};
}
const wroteResults = await Promise.all(
(Object.keys(jsonPaths) as Array<keyof ConfigDocBaselineArtifacts>).map((key) =>
writeIfChanged(jsonPaths[key], rendered.json[key]),
),
);
// Write the hash file (tracked in git)
writeFileAtomic(hashPath, nextHashContent);
// Write full JSON artifacts locally (gitignored, useful for inspection)
for (const key of Object.keys(jsonPaths) as Array<keyof ConfigDocBaselineArtifacts>) {
writeFileAtomic(jsonPaths[key], rendered.json[key]);
}
return {
changed,
wrote: wroteResults.some(Boolean),
wrote: true,
jsonPaths,
hashPath,
};
}

View File

@@ -3,7 +3,7 @@ import path from "node:path";
import { fileURLToPath } from "node:url";
import { describe, expect, it } from "vitest";
import {
type ConfigDocBaseline,
buildConfigDocBaseline,
flattenConfigDocBaselineEntries,
normalizeConfigDocBaselineHelpPath,
} from "./doc-baseline.js";
@@ -20,11 +20,9 @@ function readRepoFile(relativePath: string): string {
}
describe("talk silence timeout defaults", () => {
it("keeps help text and docs aligned with the policy", () => {
it("keeps help text and docs aligned with the policy", async () => {
const defaultsDescription = describeTalkSilenceTimeoutDefaults();
const baseline = JSON.parse(
readRepoFile("docs/.generated/config-baseline.json"),
) as ConfigDocBaseline;
const baseline = await buildConfigDocBaseline();
const talkEntry = flattenConfigDocBaselineEntries(baseline).find(
(entry) => entry.path === normalizeConfigDocBaselineHelpPath("talk.silenceTimeoutMs"),
);

View File

@@ -1,3 +1,4 @@
import { createHash } from "node:crypto";
import fs from "node:fs/promises";
import path from "node:path";
import { fileURLToPath } from "node:url";
@@ -57,11 +58,13 @@ export type PluginSdkApiBaselineWriteResult = {
wrote: boolean;
jsonPath: string;
statefilePath: string;
hashPath: string;
};
const GENERATED_BY = "scripts/generate-plugin-sdk-api-baseline.ts" as const;
const DEFAULT_JSON_OUTPUT = "docs/.generated/plugin-sdk-api-baseline.json";
const DEFAULT_STATEFILE_OUTPUT = "docs/.generated/plugin-sdk-api-baseline.jsonl";
const DEFAULT_HASH_OUTPUT = "docs/.generated/plugin-sdk-api-baseline.sha256";
function assert(condition: unknown, message: string): asserts condition {
if (!condition) {
@@ -446,6 +449,21 @@ async function loadCurrentFile(filePath: string): Promise<string | null> {
}
}
function sha256(content: string): string {
return createHash("sha256").update(content, "utf8").digest("hex");
}
/** Build the sha256 hash file content for plugin SDK API baseline artifacts. */
export function computePluginSdkApiBaselineHashFileContent(
rendered: PluginSdkApiBaselineRender,
): string {
const lines = [
`${sha256(rendered.json)} plugin-sdk-api-baseline.json`,
`${sha256(rendered.jsonl)} plugin-sdk-api-baseline.jsonl`,
];
return `${lines.join("\n")}\n`;
}
function validateMetadata(): void {
const canonicalEntrypoints = new Set<string>(pluginSdkEntrypoints);
const metadataEntrypoints = new Set<string>(Object.keys(pluginSdkDocMetadata));
@@ -463,14 +481,17 @@ export async function writePluginSdkApiBaselineStatefile(params?: {
check?: boolean;
jsonPath?: string;
statefilePath?: string;
hashPath?: string;
}): Promise<PluginSdkApiBaselineWriteResult> {
const repoRoot = params?.repoRoot ?? resolveRepoRoot();
const jsonPath = path.resolve(repoRoot, params?.jsonPath ?? DEFAULT_JSON_OUTPUT);
const statefilePath = path.resolve(repoRoot, params?.statefilePath ?? DEFAULT_STATEFILE_OUTPUT);
const hashPath = path.resolve(repoRoot, params?.hashPath ?? DEFAULT_HASH_OUTPUT);
const rendered = await renderPluginSdkApiBaseline({ repoRoot });
const currentJson = await loadCurrentFile(jsonPath);
const currentJsonl = await loadCurrentFile(statefilePath);
const changed = currentJson !== rendered.json || currentJsonl !== rendered.jsonl;
const nextHashContent = computePluginSdkApiBaselineHashFileContent(rendered);
const currentHashContent = await loadCurrentFile(hashPath);
const changed = currentHashContent !== nextHashContent;
if (params?.check) {
return {
@@ -478,9 +499,15 @@ export async function writePluginSdkApiBaselineStatefile(params?: {
wrote: false,
jsonPath,
statefilePath,
hashPath,
};
}
// Write the hash file (tracked in git)
await fs.mkdir(path.dirname(hashPath), { recursive: true });
await fs.writeFile(hashPath, nextHashContent, "utf8");
// Write full JSON/JSONL artifacts locally (gitignored, useful for inspection)
await fs.mkdir(path.dirname(jsonPath), { recursive: true });
await fs.writeFile(jsonPath, rendered.json, "utf8");
await fs.writeFile(statefilePath, rendered.jsonl, "utf8");
@@ -490,5 +517,6 @@ export async function writePluginSdkApiBaselineStatefile(params?: {
wrote: true,
jsonPath,
statefilePath,
hashPath,
};
}