Compare commits

..

8 Commits

Author SHA1 Message Date
Nikhil Sonti
e3fa82f441 fix: address greptile review comments for klavis strata cache
- Drop dead `invalidated` field on InflightEntry. It was added to
  support a "discard post-resolution if invalidated" check that I
  later replaced with identity-checked deletes during self-review,
  but I forgot to remove the field and the misleading comment
  referencing it. Simplify Map<string, InflightEntry> to plain
  Map<string, Promise<CacheEntry>>.
- Lower cache miss log from info to debug. Misses fire on every new
  conversation; matching the existing debug-level for hits.
- Stop routing the /klavis/servers/remove handler through
  klavisStrataCache.getOrFetch. The chat hot path keys its cache by
  the user's full enabled-server set (e.g. hash('Gmail,Linear')),
  so a single-server lookup here (hash('Gmail')) is guaranteed to
  miss, write a spurious entry, and then have it immediately
  cleared by invalidate() on the next line. Call createStrata
  directly to recover the strataId, mirroring the original
  removeServer flow.
2026-04-07 11:00:14 -07:00
Nikhil Sonti
60273ee514 feat(server): cache klavis createStrata to unblock /chat hot path
Conversation creation in /chat was blocking on a Worker-proxied
klavisClient.createStrata round-trip every time the user had any
managed Klavis app connected. The 5s KLAVIS_TIMEOUT_MS in the
ai-worker proxy existed specifically to bound this latency, but
the same cap also caused user-visible 504s on /klavis/servers/remove
since Strata DELETE operations routinely take >5s. Without caching
we couldn't raise the timeout without regressing chat creation.

This adds an in-process cache for Strata createStrata responses,
keyed by (browserosId, hashed sorted-server-set) and gated by a 1h
TTL. The cache stores only immutable JSON metadata (strataServerUrl,
strataId, addedServers); per-session MCP clients continue to be
opened and disposed by AiSdkAgent exactly as before, which keeps
the cache concurrency-safe by construction.

Cache invalidation has two layers: (a) the cache key embeds the
server set, so adding/removing apps naturally produces a different
key; (b) POST /klavis/servers/add and DELETE /klavis/servers/remove
explicitly call invalidate(browserosId) after their underlying
Klavis API call succeeds, as defense-in-depth.

Other changes:
- Consolidates klavis-related services into a new
  apps/server/src/api/services/klavis/ directory; moves
  register-klavis-mcp.ts -> strata-proxy.ts and adds strata-cache.ts
  there. lib/clients/klavis/ stays unchanged.
- Refactors KlavisClient.removeServer into a low-level
  deleteServersFromStrata(strataId, servers) primitive. The
  cache-lookup + delete + invalidate orchestration moves up into
  routes/klavis.ts where it belongs, eliminating the lib->api
  layering inversion the original removeServer would have introduced.
- Uses Bun.hash (xxhash64) for fixed-width 16-hex-char keys, with
  serverKey verified on read to make collision risk strictly zero.
- Dedupes concurrent fetches via in-flight Promise sharing, with
  identity-checks before delete to avoid races between invalidate()
  and a racing replacement insert.

Follow-up (separate PR): bump KLAVIS_TIMEOUT_MS to 30000 in
ai-worker/wrangler.toml so /klavis/servers/remove stops 504-ing.
2026-04-07 10:25:11 -07:00
Nikhil
8de2bf984f feat: build linux x64 + arm64 in a single invocation (#652)
`release.linux.yaml` now declares `architecture: [x64, arm64]` and the
runner loops the entire pipeline once per architecture. depot_tools
fetches both Linux sysroots automatically — `git_setup` idempotently
ensures `target_cpus = ['x64', 'arm64']` is in `.gclient` before
`gclient sync`, so cross-compiling arm64 from an x64 host just works.

The resolver returns `List[Context]` (single-element for the common
single-arch case), and `build/cli/build.py` loops `execute_pipeline` over
the per-arch contexts. Modules stay 100% arch-agnostic — no new
orchestration module, no new YAML schema beyond the list form.

Also fix a cross-compile bug in `build/modules/package/linux.py`: the
appimagetool binary must match the BUILD machine's arch (it executes
locally), not the target arch. Split into a host-keyed
`LINUX_HOST_APPIMAGETOOL` lookup vs the existing target-keyed
`LINUX_ARCHITECTURE_CONFIG`. Target arch is still passed to appimagetool
via the `ARCH` env var.

- build/common/resolver.py: scalar OR list `architecture` -> List[Context]
- build/cli/build.py: loop pipeline per arch, log multi-arch headers
- build/config/release.linux.yaml: `architecture: [x64, arm64]`
- build/modules/setup/git.py: idempotent `target_cpus` edit on Linux
- build/modules/package/linux.py: host vs target appimagetool split
- build/modules/package/linux_test.py: cover the host/target split
2026-04-06 13:08:06 -07:00
Nikhil
1b8720740c feat: add linux arm64 release support (#651)
* feat: support linux arm64 release artifacts

* fix: address PR review comments for 0406-linux_arm64_support
2026-04-06 10:20:38 -07:00
Nikhil
91be726381 refactor: remove --compile-only flag, consolidate into --ci (#646)
The --compile-only and --ci flags served overlapping purposes for CI
builds. Remove --compile-only entirely since --ci already handles the
CI use case (skip R2, skip prod env validation, local zip packaging)
and --no-upload covers the upload-skipping use case for full builds.
2026-04-03 14:58:52 -07:00
Nikhil
ff5386a24a fix: agent storage issue on update (#643)
* fix: agent storage erase issue fix

* fix: remove the guard against remote
2026-04-03 14:50:14 -07:00
Nikhil
a5f3c4da65 fix: skip windows exe patching in ci mode to avoid wine dependency (#645)
The server release CI workflow fails on ubuntu-latest because
patch-windows-exe.ts requires Wine to run rcedit. Thread the existing
--ci flag through compileServerBinaries so Windows PE metadata patching
is skipped in CI mode with a warning log.
2026-04-03 14:46:33 -07:00
Nikhil
e5a852dd3d chore: update server version (#644) 2026-04-03 14:29:07 -07:00
31 changed files with 881 additions and 146 deletions

View File

@@ -1,6 +1,6 @@
{
"name": "@browseros/server",
"version": "0.0.80",
"version": "0.0.81",
"description": "BrowserOS server",
"type": "module",
"main": "./src/index.ts",

View File

@@ -2,6 +2,7 @@ import { createMCPClient } from '@ai-sdk/mcp'
import { TIMEOUTS } from '@browseros/shared/constants/timeouts'
import type { BrowserContext } from '@browseros/shared/schemas/browser-context'
import type { ToolSet } from 'ai'
import { klavisStrataCache } from '../api/services/klavis/strata-cache'
import type { KlavisClient } from '../lib/clients/klavis/klavis-client'
import { logger } from '../lib/logger'
import {
@@ -40,7 +41,8 @@ export async function buildMcpServerSpecs(
deps.browserContext?.enabledMcpServers?.length
) {
try {
const result = await deps.klavisClient.createStrata(
const result = await klavisStrataCache.getOrFetch(
deps.klavisClient,
deps.browserosId,
deps.browserContext.enabledMcpServers,
)

View File

@@ -10,6 +10,7 @@ import { z } from 'zod'
import { KlavisClient } from '../../lib/clients/klavis/klavis-client'
import { OAUTH_MCP_SERVERS } from '../../lib/clients/klavis/oauth-mcp-servers'
import { logger } from '../../lib/logger'
import { klavisStrataCache } from '../services/klavis/strata-cache'
const ServerNameSchema = z.object({
serverName: z.string().min(1),
@@ -125,6 +126,7 @@ export function createKlavisRoutes(deps: KlavisRouteDeps) {
logger.info('Adding server to strata', { serverName })
const result = await klavisClient.createStrata(browserosId, [serverName])
klavisStrataCache.invalidate(browserosId)
return c.json({
success: true,
@@ -184,7 +186,17 @@ export function createKlavisRoutes(deps: KlavisRouteDeps) {
logger.info('Removing server from strata', { serverName })
await klavisClient.removeServer(browserosId, serverName)
// The chat hot path keys its cache by the user's full enabled set,
// so a single-server lookup here would always miss and immediately
// be cleared by invalidate() below — call createStrata directly
// to recover the strataId, mirroring the original removeServer flow.
const strata = await klavisClient.createStrata(browserosId, [
serverName,
])
await klavisClient.deleteServersFromStrata(strata.strataId, [
serverName,
])
klavisStrataCache.invalidate(browserosId)
return c.json({
success: true,

View File

@@ -11,8 +11,8 @@ import { logger } from '../../lib/logger'
import { metrics } from '../../lib/metrics'
import { Sentry } from '../../lib/sentry'
import type { ToolRegistry } from '../../tools/tool-registry'
import type { KlavisProxyHandle } from '../services/klavis/strata-proxy'
import { createMcpServer } from '../services/mcp/mcp-server'
import type { KlavisProxyHandle } from '../services/mcp/register-klavis-mcp'
import type { Env } from '../types'
interface McpRouteDeps {

View File

@@ -38,7 +38,7 @@ import { createStatusRoute } from './routes/status'
import {
connectKlavisProxy,
type KlavisProxyHandle,
} from './services/mcp/register-klavis-mcp'
} from './services/klavis/strata-proxy'
import type { Env, HttpServerConfig } from './types'
import { defaultCorsConfig } from './utils/cors'

View File

@@ -0,0 +1,145 @@
/**
* @license
* Copyright 2025 BrowserOS
* SPDX-License-Identifier: AGPL-3.0-or-later
*
* In-process cache for Klavis Strata `createStrata` responses.
*
* Conversation creation in `/chat` was blocking on a Worker-proxied
* `klavisClient.createStrata` round-trip every time the user had any
* managed Klavis app connected. This cache stores the (immutable) JSON
* metadata returned by `createStrata` so that subsequent chats with the
* same `(browserosId, enabled-server-set)` skip the round-trip entirely.
*
* It does NOT cache live MCP client connections — only URL/id metadata.
* Per-session MCP clients continue to be opened and closed by
* `AiSdkAgent.create` / `dispose` exactly as before, which makes the
* cache safe across concurrent chats by construction.
*/
import type {
KlavisClient,
StrataCreateResponse,
} from '../../../lib/clients/klavis/klavis-client'
import { logger } from '../../../lib/logger'
const DEFAULT_TTL_MS = 60 * 60 * 1000
interface CacheEntry {
strataServerUrl: string
strataId: string
addedServers: string[]
serverKey: string
expiresAt: number
}
function normalizeServers(servers: readonly string[]): string {
return [...new Set(servers)].sort().join(',')
}
function keyOf(browserosId: string, normalized: string): string {
// xxhash64 → 16 hex chars, fixed width. Birthday-bound collision risk
// for our scale (<10k entries) is ~5e-15; we additionally verify
// serverKey on read so collisions cannot affect correctness.
const hash = Bun.hash(normalized).toString(16).padStart(16, '0')
return `${browserosId}|${hash}`
}
export class KlavisStrataCache {
private entries = new Map<string, Promise<CacheEntry>>()
constructor(private ttlMs: number = DEFAULT_TTL_MS) {}
async getOrFetch(
client: KlavisClient,
browserosId: string,
servers: readonly string[],
): Promise<StrataCreateResponse> {
const normalized = normalizeServers(servers)
const key = keyOf(browserosId, normalized)
const existing = this.entries.get(key)
if (existing) {
const resolved = await existing.catch(() => null)
if (
resolved &&
resolved.serverKey === normalized &&
Date.now() < resolved.expiresAt
) {
logger.debug('Klavis strata cache hit', { key })
return this.toResponse(resolved)
}
// Stale/collision/rejected — evict, but only if we're the rightful
// evictor (a racing caller may have already replaced this slot).
if (this.entries.get(key) === existing) {
this.entries.delete(key)
}
}
logger.debug('Klavis strata cache miss', {
key,
serverCount: servers.length,
})
const inflight = this.fetch(client, browserosId, servers, normalized)
this.entries.set(key, inflight)
try {
return this.toResponse(await inflight)
} catch (err) {
// Identity-check: only drop OUR entry. A racing invalidate() may have
// already removed it, or a racing miss may have inserted a new one
// that we must not clobber.
if (this.entries.get(key) === inflight) {
this.entries.delete(key)
}
throw err
}
}
invalidate(browserosId: string): void {
const prefix = `${browserosId}|`
let dropped = 0
for (const key of this.entries.keys()) {
if (key.startsWith(prefix)) {
this.entries.delete(key)
dropped++
}
}
if (dropped > 0) {
logger.debug('Klavis strata cache invalidated', {
browserosId: browserosId.slice(0, 12),
dropped,
})
}
}
clear(): void {
this.entries.clear()
}
private async fetch(
client: KlavisClient,
browserosId: string,
servers: readonly string[],
normalized: string,
): Promise<CacheEntry> {
const result = await client.createStrata(browserosId, [...servers])
return {
strataServerUrl: result.strataServerUrl,
strataId: result.strataId,
addedServers: result.addedServers,
serverKey: normalized,
expiresAt: Date.now() + this.ttlMs,
}
}
private toResponse(entry: CacheEntry): StrataCreateResponse {
return {
strataServerUrl: entry.strataServerUrl,
strataId: entry.strataId,
addedServers: entry.addedServers,
}
}
}
export const klavisStrataCache = new KlavisStrataCache()

View File

@@ -14,6 +14,7 @@ import type { KlavisClient } from '../../../lib/clients/klavis/klavis-client'
import { OAUTH_MCP_SERVERS } from '../../../lib/clients/klavis/oauth-mcp-servers'
import { logger } from '../../../lib/logger'
import { metrics } from '../../../lib/metrics'
import { klavisStrataCache } from './strata-cache'
function withTimeout<T>(promise: Promise<T>, label: string): Promise<T> {
let timerId: ReturnType<typeof setTimeout> | undefined
@@ -49,7 +50,8 @@ export async function connectKlavisProxy(
// even unauthenticated ones (Klavis handles auth prompts on call)
const allServers = OAUTH_MCP_SERVERS.map((s) => s.name)
const strata = await deps.klavisClient.createStrata(
const strata = await klavisStrataCache.getOrFetch(
deps.klavisClient,
deps.browserosId,
allServers,
)

View File

@@ -8,11 +8,11 @@ import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js'
import { SetLevelRequestSchema } from '@modelcontextprotocol/sdk/types.js'
import type { Browser } from '../../../browser/browser'
import type { ToolRegistry } from '../../../tools/tool-registry'
import { MCP_INSTRUCTIONS } from './mcp-prompt'
import {
type KlavisProxyHandle,
registerKlavisTools,
} from './register-klavis-mcp'
} from '../klavis/strata-proxy'
import { MCP_INSTRUCTIONS } from './mcp-prompt'
import { registerTools } from './register-mcp'
export interface McpServiceDeps {

View File

@@ -145,16 +145,14 @@ export class KlavisClient {
})
}
/**
* Remove a server from a Strata instance
* Flow: createStrata(server) to get strataId → DELETE /strata/{strataId}/servers?servers=X
*/
async removeServer(userId: string, serverName: string): Promise<void> {
// createStrata to get strataId (passing same server ensures it exists)
const strata = await this.createStrata(userId, [serverName])
async deleteServersFromStrata(
strataId: string,
servers: string[],
): Promise<void> {
const query = servers.map(encodeURIComponent).join(',')
await this.request(
'DELETE',
`/mcp-server/strata/${strata.strataId}/servers?servers=${encodeURIComponent(serverName)}`,
`/mcp-server/strata/${strataId}/servers?servers=${query}`,
)
}
}

View File

@@ -3,12 +3,17 @@
* Copyright 2025 BrowserOS
*/
import { afterEach, describe, it } from 'bun:test'
import { afterEach, beforeEach, describe, it } from 'bun:test'
import assert from 'node:assert'
import { createKlavisRoutes } from '../../../src/api/routes/klavis'
import { klavisStrataCache } from '../../../src/api/services/klavis/strata-cache'
const originalFetch = globalThis.fetch
beforeEach(() => {
klavisStrataCache.clear()
})
afterEach(() => {
globalThis.fetch = originalFetch
})

View File

@@ -0,0 +1,163 @@
/**
* @license
* Copyright 2025 BrowserOS
*/
import { describe, expect, it } from 'bun:test'
import { KlavisStrataCache } from '../../../../src/api/services/klavis/strata-cache'
import type {
KlavisClient,
StrataCreateResponse,
} from '../../../../src/lib/clients/klavis/klavis-client'
class StubKlavisClient {
callCount = 0
delayMs = 0
shouldThrowOnce = false
lastServers: string[] | null = null
async createStrata(
userId: string,
servers: string[],
): Promise<StrataCreateResponse> {
this.callCount++
this.lastServers = servers
if (this.shouldThrowOnce) {
this.shouldThrowOnce = false
throw new Error('boom')
}
if (this.delayMs > 0) {
await new Promise((r) => setTimeout(r, this.delayMs))
}
return {
strataServerUrl: `https://strata.test/${userId}/${servers.join('-')}`,
strataId: `strata_${userId}`,
addedServers: servers,
}
}
}
const asClient = (stub: StubKlavisClient): KlavisClient =>
stub as unknown as KlavisClient
describe('KlavisStrataCache', () => {
it('cache hit returns the same value without re-calling the client', async () => {
const cache = new KlavisStrataCache()
const client = new StubKlavisClient()
const a = await cache.getOrFetch(asClient(client), 'user1', ['Gmail'])
const b = await cache.getOrFetch(asClient(client), 'user1', ['Gmail'])
expect(client.callCount).toBe(1)
expect(a.strataServerUrl).toBe(b.strataServerUrl)
expect(a.strataId).toBe(b.strataId)
})
it('normalizes server order — [Gmail, Linear] === [Linear, Gmail]', async () => {
const cache = new KlavisStrataCache()
const client = new StubKlavisClient()
await cache.getOrFetch(asClient(client), 'u', ['Gmail', 'Linear'])
await cache.getOrFetch(asClient(client), 'u', ['Linear', 'Gmail'])
expect(client.callCount).toBe(1)
})
it('dedupes duplicate server names within one call', async () => {
const cache = new KlavisStrataCache()
const client = new StubKlavisClient()
await cache.getOrFetch(asClient(client), 'u', ['Gmail', 'Gmail'])
await cache.getOrFetch(asClient(client), 'u', ['Gmail'])
expect(client.callCount).toBe(1)
})
it('different user gets a separate cache entry', async () => {
const cache = new KlavisStrataCache()
const client = new StubKlavisClient()
await cache.getOrFetch(asClient(client), 'userA', ['Gmail'])
await cache.getOrFetch(asClient(client), 'userB', ['Gmail'])
expect(client.callCount).toBe(2)
})
it('different server set (same user) gets a separate cache entry', async () => {
const cache = new KlavisStrataCache()
const client = new StubKlavisClient()
await cache.getOrFetch(asClient(client), 'u', ['Gmail'])
await cache.getOrFetch(asClient(client), 'u', ['Gmail', 'Linear'])
expect(client.callCount).toBe(2)
})
it('concurrent misses share a single in-flight Promise', async () => {
const cache = new KlavisStrataCache()
const client = new StubKlavisClient()
client.delayMs = 30
const [a, b, c] = await Promise.all([
cache.getOrFetch(asClient(client), 'u', ['Gmail']),
cache.getOrFetch(asClient(client), 'u', ['Gmail']),
cache.getOrFetch(asClient(client), 'u', ['Gmail']),
])
expect(client.callCount).toBe(1)
expect(a.strataId).toBe(b.strataId)
expect(b.strataId).toBe(c.strataId)
})
it('TTL expiry triggers a fresh fetch', async () => {
const cache = new KlavisStrataCache(10) // 10 ms TTL
const client = new StubKlavisClient()
await cache.getOrFetch(asClient(client), 'u', ['Gmail'])
await new Promise((r) => setTimeout(r, 25))
await cache.getOrFetch(asClient(client), 'u', ['Gmail'])
expect(client.callCount).toBe(2)
})
it('invalidate(userA) drops only userA entries', async () => {
const cache = new KlavisStrataCache()
const client = new StubKlavisClient()
await cache.getOrFetch(asClient(client), 'userA', ['Gmail'])
await cache.getOrFetch(asClient(client), 'userB', ['Gmail'])
cache.invalidate('userA')
await cache.getOrFetch(asClient(client), 'userA', ['Gmail'])
await cache.getOrFetch(asClient(client), 'userB', ['Gmail'])
expect(client.callCount).toBe(3) // userA: cold + cold, userB: cold + hit
})
it('invalidate while a fetch is in flight does not store the result', async () => {
const cache = new KlavisStrataCache()
const client = new StubKlavisClient()
client.delayMs = 30
const inflight = cache.getOrFetch(asClient(client), 'u', ['Gmail'])
cache.invalidate('u')
const result = await inflight
expect(result.strataId).toBe('strata_u')
// Next call should not see the post-invalidate write — must re-fetch.
await cache.getOrFetch(asClient(client), 'u', ['Gmail'])
expect(client.callCount).toBe(2)
})
it('rejected fetches do not poison the cache', async () => {
const cache = new KlavisStrataCache()
const client = new StubKlavisClient()
client.shouldThrowOnce = true
await expect(
cache.getOrFetch(asClient(client), 'u', ['Gmail']),
).rejects.toThrow('boom')
await cache.getOrFetch(asClient(client), 'u', ['Gmail'])
expect(client.callCount).toBe(2)
})
it('clear() drops all entries', async () => {
const cache = new KlavisStrataCache()
const client = new StubKlavisClient()
await cache.getOrFetch(asClient(client), 'userA', ['Gmail'])
await cache.getOrFetch(asClient(client), 'userB', ['Linear'])
cache.clear()
await cache.getOrFetch(asClient(client), 'userA', ['Gmail'])
await cache.getOrFetch(asClient(client), 'userB', ['Linear'])
expect(client.callCount).toBe(4)
})
it('passes a defensive copy of the servers array to the client', async () => {
const cache = new KlavisStrataCache()
const client = new StubKlavisClient()
const input: readonly string[] = ['Gmail', 'Linear']
await cache.getOrFetch(asClient(client), 'u', input)
expect(client.lastServers).not.toBe(input)
expect(client.lastServers).toEqual(['Gmail', 'Linear'])
})
})

View File

@@ -156,28 +156,6 @@ describe('server build', () => {
assert.strictEqual(versionOutput.trim(), expectedVersion)
}, 300_000)
it('keeps compile-only on the strict production validation path', async () => {
resetProdEnvToTemplate()
const build = Bun.spawn(
['bun', buildScript, `--target=${target.id}`, '--compile-only'],
{
cwd: rootDir,
stdout: 'pipe',
stderr: 'pipe',
env: buildEnv({}, PROD_SECRET_KEYS),
},
)
const buildExit = await build.exited
const stderr = await new Response(build.stderr).text()
assert.notStrictEqual(buildExit, 0, 'Compile-only build should fail')
assert.match(stderr, /Production build requires variables:/)
assert.match(stderr, /CODEGEN_SERVICE_URL/)
assert.match(stderr, /POSTHOG_API_KEY/)
assert.match(stderr, /SENTRY_DSN/)
}, 300_000)
it('archives CI builds without R2 config or production env secrets', async () => {
resetProdEnvToTemplate()
rmSync(zipPath, { force: true })

View File

@@ -152,7 +152,7 @@
},
"apps/server": {
"name": "@browseros/server",
"version": "0.0.80",
"version": "0.0.81",
"bin": {
"browseros-server": "./src/index.ts",
},

View File

@@ -21,10 +21,6 @@ export function parseBuildArgs(argv: string[]): BuildArgs {
)
.option('--upload', 'Upload artifact zips to R2')
.option('--no-upload', 'Skip zip upload to R2')
.option(
'--compile-only',
'Compile binaries only (skip artifact packaging, R2 staging, and upload)',
)
.option(
'--ci',
'Build local release zip artifacts for CI without R2 and without requiring production env secrets',
@@ -34,15 +30,10 @@ export function parseBuildArgs(argv: string[]): BuildArgs {
target: string
manifest: string
upload: boolean
compileOnly: boolean
ci: boolean
}>()
const compileOnly = options.compileOnly ?? false
const ci = options.ci ?? false
if (ci && compileOnly) {
throw new Error('--ci cannot be combined with --compile-only')
}
if (ci && options.upload) {
throw new Error('--ci cannot be combined with --upload')
}
@@ -50,8 +41,7 @@ export function parseBuildArgs(argv: string[]): BuildArgs {
return {
targets: resolveTargets(options.target),
manifestPath: options.manifest,
upload: ci || compileOnly ? false : (options.upload ?? true),
compileOnly,
upload: ci ? false : (options.upload ?? true),
ci,
}
}

View File

@@ -75,7 +75,6 @@ function validateProductionEnv(envVars: Record<string, string>): void {
}
export interface LoadBuildConfigOptions {
compileOnly?: boolean
ci?: boolean
}
@@ -95,7 +94,7 @@ export function loadBuildConfig(
...process.env,
}
if (options.compileOnly || options.ci) {
if (options.ci) {
return { version: readServerVersion(rootDir), envVars, processEnv }
}

View File

@@ -10,11 +10,8 @@ import { getTargetRules, loadManifest } from './manifest'
import { createR2Client } from './r2'
import { stageCompiledArtifact, stageTargetArtifact } from './stage'
function buildModeLabel(argv: { compileOnly: boolean; ci: boolean }): string {
if (argv.ci) {
return 'ci'
}
return argv.compileOnly ? 'compile-only' : 'full'
function buildModeLabel(ci: boolean): string {
return ci ? 'ci' : 'full'
}
export async function runProdResourceBuild(argv: string[]): Promise<void> {
@@ -23,14 +20,11 @@ export async function runProdResourceBuild(argv: string[]): Promise<void> {
const args = parseBuildArgs(argv)
const buildConfig = loadBuildConfig(rootDir, {
compileOnly: args.compileOnly,
ci: args.ci,
})
const buildConfig = loadBuildConfig(rootDir, { ci: args.ci })
log.header(`Building BrowserOS server artifacts v${buildConfig.version}`)
log.info(`Targets: ${args.targets.map((target) => target.id).join(', ')}`)
log.info(`Mode: ${buildModeLabel(args)}`)
log.info(`Mode: ${buildModeLabel(args.ci)}`)
const compiled = await compileServerBinaries(
args.targets,
@@ -64,14 +58,6 @@ export async function runProdResourceBuild(argv: string[]): Promise<void> {
return
}
if (args.compileOnly) {
log.done('Compile-only build completed')
for (const binary of compiled) {
log.info(`${binary.target.id}: ${binary.binaryPath}`)
}
return
}
const manifestPath = resolve(rootDir, args.manifestPath)
if (!existsSync(manifestPath)) {
throw new Error(`Manifest not found: ${manifestPath}`)

View File

@@ -21,7 +21,6 @@ export interface BuildArgs {
targets: BuildTarget[]
manifestPath: string
upload: boolean
compileOnly: boolean
ci: boolean
}

View File

@@ -402,9 +402,11 @@ def main(
"upload": upload,
}
# Resolve build context (CONFIG mode or DIRECT mode)
# Resolve build context (CONFIG mode or DIRECT mode).
# Returns one Context per architecture — single-element for normal
# builds, multi-element when YAML declares `architecture: [x64, arm64]`.
try:
ctx = resolve_config(cli_args, config_data)
arch_ctxs = resolve_config(cli_args, config_data)
except ValueError as e:
log_error(str(e))
raise typer.Exit(1)
@@ -459,20 +461,40 @@ def main(
os.environ["DEPOT_TOOLS_WIN_TOOLCHAIN"] = "0"
log_info("Set DEPOT_TOOLS_WIN_TOOLCHAIN=0 for Windows build")
# Print build summary using the first context — versions and paths
# are identical across per-arch contexts. Architecture is logged again
# inside the loop below for multi-arch runs.
summary_ctx = arch_ctxs[0]
log_info(f"📍 Root: {root_dir}")
log_info(f"📍 Chromium: {ctx.chromium_src}")
log_info(f"📍 Architecture: {ctx.architecture}")
log_info(f"📍 Build type: {ctx.build_type}")
log_info(f"📍 Output: {ctx.out_dir}")
log_info(f"📍 Semantic version: {ctx.semantic_version}")
log_info(f"📍 Chromium version: {ctx.chromium_version}")
log_info(f"📍 Build offset: {ctx.browseros_build_offset}")
log_info(f"📍 Chromium: {summary_ctx.chromium_src}")
if len(arch_ctxs) > 1:
log_info(
f"📍 Architectures: {[c.architecture for c in arch_ctxs]} (multi-arch loop)"
)
else:
log_info(f"📍 Architecture: {summary_ctx.architecture}")
log_info(f"📍 Build type: {summary_ctx.build_type}")
log_info(f"📍 Semantic version: {summary_ctx.semantic_version}")
log_info(f"📍 Chromium version: {summary_ctx.chromium_version}")
log_info(f"📍 Build offset: {summary_ctx.browseros_build_offset}")
log_info(f"📍 Pipeline: {''.join(pipeline)}")
log_info("=" * 70)
# Set notification context for OS and architecture
os_name = "macOS" if IS_MACOS() else "Windows" if IS_WINDOWS() else "Linux"
set_build_context(os_name, ctx.architecture)
# Execute pipeline
execute_pipeline(ctx, pipeline, AVAILABLE_MODULES, pipeline_name="build")
# Execute the pipeline once per architecture. Modules see a normal
# single-arch ctx; the runner is the only thing that knows about the
# multi-arch loop.
for i, arch_ctx in enumerate(arch_ctxs, start=1):
if len(arch_ctxs) > 1:
log_info("\n" + "#" * 70)
log_info(
f"# Architecture {i}/{len(arch_ctxs)}: {arch_ctx.architecture}"
)
log_info(f"# Output: {arch_ctx.out_dir}")
log_info("#" * 70)
set_build_context(os_name, arch_ctx.architecture)
execute_pipeline(
arch_ctx, pipeline, AVAILABLE_MODULES, pipeline_name="build"
)

View File

@@ -26,11 +26,13 @@ from .context import Context
from .env import EnvConfig
from .utils import get_platform_arch, log_info
VALID_ARCHITECTURES = {"x64", "arm64", "universal"}
def resolve_config(
cli_args: Dict[str, Any],
yaml_config: Optional[Dict[str, Any]] = None,
) -> Context:
) -> List[Context]:
"""Resolve build configuration - single entry point.
Args:
@@ -38,7 +40,9 @@ def resolve_config(
yaml_config: Optional YAML configuration (triggers CONFIG mode)
Returns:
Fully resolved Context object
List of fully resolved Context objects. Single-element for the
common single-arch case; multi-element when YAML declares
`architecture: [x64, arm64]` (Linux multi-arch).
Raises:
ValueError: If required fields missing or invalid
@@ -59,7 +63,7 @@ def resolve_config(
def _resolve_config_mode(
yaml_config: Dict[str, Any], cli_args: Dict[str, Any]
) -> Context:
) -> List[Context]:
"""CONFIG MODE: YAML is base, CLI can override.
Args:
@@ -67,7 +71,7 @@ def _resolve_config_mode(
cli_args: CLI arguments (can override YAML values)
Returns:
Context with values from YAML, optionally overridden by CLI
List of Contexts. One per architecture when YAML provides a list.
Raises:
ValueError: If required fields missing from both YAML and CLI
@@ -94,41 +98,66 @@ def _resolve_config_mode(
f"Expected directory with Chromium source code"
)
# architecture: CLI override > YAML > platform default
architecture = (
cli_args.get("arch")
or build_section.get("architecture")
or build_section.get("arch")
)
arch_source = "cli" if cli_args.get("arch") else "yaml"
if not architecture:
architecture = get_platform_arch()
# architecture: CLI override > YAML > platform default.
# YAML may be a string OR a list (e.g. [x64, arm64]) — list form runs
# the entire pipeline once per arch.
cli_arch = cli_args.get("arch")
yaml_arch = build_section.get("architecture") or build_section.get("arch")
if cli_arch:
architectures = [cli_arch]
arch_source = "cli"
elif yaml_arch is not None:
architectures = yaml_arch if isinstance(yaml_arch, list) else [yaml_arch]
arch_source = "yaml"
else:
architectures = [get_platform_arch()]
arch_source = "default"
log_info(f"CONFIG MODE: Using platform default architecture: {architecture}")
log_info(
f"CONFIG MODE: Using platform default architecture: {architectures[0]}"
)
for arch in architectures:
if arch not in VALID_ARCHITECTURES:
raise ValueError(
f"CONFIG MODE: invalid architecture '{arch}'. "
f"Valid: {sorted(VALID_ARCHITECTURES)}"
)
# build_type: CLI override > YAML > debug
build_type = cli_args.get("build_type") or build_section.get("type", "debug")
build_type_source = "cli" if cli_args.get("build_type") else "yaml"
log_info(f"✓ CONFIG MODE: chromium_src={chromium_src} ({chromium_src_source})")
log_info(f"✓ CONFIG MODE: architecture={architecture} ({arch_source})")
if len(architectures) > 1:
log_info(
f"✓ CONFIG MODE: architectures={architectures} ({arch_source}, multi-arch loop)"
)
else:
log_info(
f"✓ CONFIG MODE: architecture={architectures[0]} ({arch_source})"
)
log_info(f"✓ CONFIG MODE: build_type={build_type} ({build_type_source})")
return Context(
chromium_src=chromium_src,
architecture=architecture,
build_type=build_type,
)
return [
Context(
chromium_src=chromium_src,
architecture=arch,
build_type=build_type,
)
for arch in architectures
]
def _resolve_direct_mode(cli_args: Dict[str, Any]) -> Context:
def _resolve_direct_mode(cli_args: Dict[str, Any]) -> List[Context]:
"""DIRECT MODE: CLI > Env > Defaults.
Args:
cli_args: CLI arguments (None if not provided by user)
Returns:
Context with resolved values
Single-element list with the resolved Context. DIRECT mode is
always single-arch (CLI --arch is a scalar).
Raises:
ValueError: If chromium_src not provided
@@ -160,6 +189,12 @@ def _resolve_direct_mode(cli_args: Dict[str, Any]) -> Context:
architecture = get_platform_arch()
log_info(f"DIRECT MODE: Using platform default architecture: {architecture}")
if architecture not in VALID_ARCHITECTURES:
raise ValueError(
f"DIRECT MODE: invalid architecture '{architecture}'. "
f"Valid: {sorted(VALID_ARCHITECTURES)}"
)
# build_type: CLI > Default
build_type = cli_args.get("build_type") or "debug"
@@ -167,11 +202,13 @@ def _resolve_direct_mode(cli_args: Dict[str, Any]) -> Context:
log_info(f"✓ DIRECT MODE: architecture={architecture} (cli/env/default)")
log_info(f"✓ DIRECT MODE: build_type={build_type} (cli/default)")
return Context(
chromium_src=chromium_src,
architecture=architecture,
build_type=build_type,
)
return [
Context(
chromium_src=chromium_src,
architecture=architecture,
build_type=build_type,
)
]
def resolve_pipeline(

View File

@@ -3,7 +3,10 @@
# This config packages an already-built Linux application.
# Use this when you have a pre-built app and only need to package it.
#
# Expects: out/Default/chrome (Linux binary)
# Expects: out/Default_<arch>/browseros
# Invoke with:
# browseros build --config build/config/package.linux.yaml --arch x64
# browseros build --config build/config/package.linux.yaml --arch arm64
#
# Environment Variables:
# Use !env tag to reference environment variables:
@@ -11,7 +14,6 @@
build:
type: release
architecture: x64 # Linux x64
gn_flags:
file: build/config/gn/flags.linux.release.gn

View File

@@ -1,17 +1,24 @@
# BrowserOS Linux Release Build Configuration
#
# Builds both x64 and arm64 in a single invocation on a Linux x64 host.
# The runner loops the entire pipeline once per architecture; depot_tools
# fetches the matching sysroots automatically (see git_setup module).
#
# Run:
# browseros build --config build/config/release.linux.yaml
#
# Environment Variables:
# Use !env tag to reference environment variables:
# Example: chromium_src: !env CHROMIUM_SRC
build:
type: release
architecture: x64 # Linux x64
architecture: [x64, arm64] # Builds both arches sequentially in one run
gn_flags:
file: build/config/gn/flags.linux.release.gn
# Explicit module execution order
# Explicit module execution order. Runs once per architecture above.
modules:
# Phase 1: Setup
- clean

View File

@@ -17,10 +17,64 @@ from ...common.utils import (
run_command,
safe_rmtree,
join_paths,
get_platform_arch,
IS_LINUX,
)
from ...common.notify import get_notifier, COLOR_GREEN
# Target-arch packaging metadata. These describe the artifact we're
# producing, not the build machine. `appimage_arch` is passed to
# appimagetool via the ARCH env var; `deb_arch` is written into the
# .deb control file.
LINUX_ARCHITECTURE_CONFIG = {
"x64": {
"appimage_arch": "x86_64",
"deb_arch": "amd64",
},
"arm64": {
"appimage_arch": "aarch64",
"deb_arch": "arm64",
},
}
# Host-arch tool selection. appimagetool is a normal binary that runs on
# the build machine — when cross-compiling arm64 from an x64 host, we
# still need the x86_64 tool to actually execute. Keyed on
# get_platform_arch() (BUILD machine arch), NOT ctx.architecture.
LINUX_HOST_APPIMAGETOOL = {
"x64": (
"appimagetool-x86_64.AppImage",
"https://github.com/AppImage/AppImageKit/releases/download/continuous/appimagetool-x86_64.AppImage",
),
"arm64": (
"appimagetool-aarch64.AppImage",
"https://github.com/AppImage/AppImageKit/releases/download/continuous/appimagetool-aarch64.AppImage",
),
}
def get_linux_architecture_config(architecture: str) -> dict[str, str]:
config = LINUX_ARCHITECTURE_CONFIG.get(architecture)
if not config:
supported = ", ".join(sorted(LINUX_ARCHITECTURE_CONFIG))
raise ValueError(
f"Unsupported Linux architecture: {architecture}. Supported: {supported}"
)
return config
def get_host_appimagetool() -> tuple[str, str]:
"""Return (filename, url) for the appimagetool binary that runs on
the current build machine. Critical for cross-compile correctness."""
host_arch = get_platform_arch()
tool = LINUX_HOST_APPIMAGETOOL.get(host_arch)
if not tool:
supported = ", ".join(sorted(LINUX_HOST_APPIMAGETOOL))
raise ValueError(
f"No appimagetool binary for host arch '{host_arch}'. Supported: {supported}"
)
return tool
class LinuxPackageModule(CommandModule):
produces = ["appimage", "deb"]
@@ -30,6 +84,10 @@ class LinuxPackageModule(CommandModule):
def validate(self, ctx: Context) -> None:
if not IS_LINUX():
raise ValidationError("Linux packaging requires Linux")
try:
get_linux_architecture_config(ctx.architecture)
except ValueError as exc:
raise ValidationError(str(exc)) from exc
out_dir = join_paths(ctx.chromium_src, ctx.out_dir)
chrome_binary = join_paths(out_dir, ctx.BROWSEROS_APP_NAME)
@@ -73,7 +131,7 @@ class LinuxPackageModule(CommandModule):
artifacts.append(deb_path.name)
notifier.notify(
"📦 Package Created",
f"Linux packages created successfully",
"Linux packages created successfully",
{
"Artifacts": ", ".join(artifacts),
"Version": ctx.semantic_version,
@@ -284,25 +342,30 @@ export CHROME_WRAPPER="${{THIS}}"
def download_appimagetool(ctx: Context) -> Optional[Path]:
"""Download appimagetool if not available"""
"""Download the appimagetool binary that runs on the build machine.
Note: this is keyed on the HOST arch, not ctx.architecture. When
cross-compiling arm64 packages from an x64 host, we still need the
x86_64 appimagetool because the tool executes locally; the target
arch is communicated via the ARCH env var in create_appimage().
"""
tool_dir = Path(join_paths(ctx.root_dir, "build", "tools"))
tool_dir.mkdir(exist_ok=True)
tool_path = Path(join_paths(tool_dir, "appimagetool-x86_64.AppImage"))
tool_filename, url = get_host_appimagetool()
tool_path = Path(join_paths(tool_dir, tool_filename))
if tool_path.exists():
log_info("✓ appimagetool already available")
log_info(f"✓ appimagetool already available ({tool_filename})")
return tool_path
log_info("📥 Downloading appimagetool...")
url = "https://github.com/AppImage/AppImageKit/releases/download/continuous/appimagetool-x86_64.AppImage"
log_info(f"📥 Downloading {tool_filename}...")
cmd = ["wget", "-O", str(tool_path), url]
result = run_command(cmd, check=False)
if result.returncode == 0:
tool_path.chmod(0o755)
log_success("✓ Downloaded appimagetool")
log_success(f"✓ Downloaded {tool_filename}")
return tool_path
else:
log_error("Failed to download appimagetool")
@@ -312,6 +375,7 @@ def download_appimagetool(ctx: Context) -> Optional[Path]:
def create_appimage(ctx: Context, appdir: Path, output_path: Path) -> bool:
"""Create AppImage from AppDir"""
log_info("📦 Creating AppImage...")
arch_config = get_linux_architecture_config(ctx.architecture)
# Download appimagetool if needed
appimagetool = download_appimagetool(ctx)
@@ -319,7 +383,7 @@ def create_appimage(ctx: Context, appdir: Path, output_path: Path) -> bool:
return False
# Set architecture environment variable (required by appimagetool)
arch = "x86_64" if ctx.architecture == "x64" else "aarch64"
arch = arch_config["appimage_arch"]
# Create AppImage with ARCH env var set for this command only
cmd = [
@@ -384,7 +448,7 @@ def create_control_file(ctx: Context, debian_dir: Path) -> None:
version = version.lstrip("v").replace(" ", "").replace("_", ".")
# Architecture mapping
deb_arch = "amd64" if ctx.architecture == "x64" else "arm64"
deb_arch = get_linux_architecture_config(ctx.architecture)["deb_arch"]
control_content = f"""Package: browseros
Version: {version}
@@ -653,7 +717,9 @@ def package_appimage(ctx: Context, package_dir: Path) -> Optional[Path]:
"""
log_info("🖼️ Building AppImage...")
appdir = Path(join_paths(package_dir, f"{ctx.BROWSEROS_APP_BASE_NAME}.AppDir"))
appdir = Path(
join_paths(package_dir, f"{ctx.BROWSEROS_APP_BASE_NAME}-{ctx.architecture}.AppDir")
)
if appdir.exists():
safe_rmtree(appdir)
@@ -683,7 +749,9 @@ def package_deb(ctx: Context, package_dir: Path) -> Optional[Path]:
"""
log_info("📦 Building .deb package...")
debdir = Path(join_paths(package_dir, f"{ctx.BROWSEROS_APP_BASE_NAME}_deb"))
debdir = Path(
join_paths(package_dir, f"{ctx.BROWSEROS_APP_BASE_NAME}_{ctx.architecture}_deb")
)
if debdir.exists():
safe_rmtree(debdir)
@@ -703,6 +771,8 @@ def package_deb(ctx: Context, package_dir: Path) -> Optional[Path]:
return output_path
return None
def package_universal(contexts: List[Context]) -> bool:
"""Linux doesn't support universal binaries"""
log_warning("Universal binaries are not supported on Linux")

View File

@@ -0,0 +1,63 @@
#!/usr/bin/env python3
"""Tests for Linux packaging architecture helpers."""
import unittest
from unittest.mock import patch
from build.modules.package.linux import (
LINUX_HOST_APPIMAGETOOL,
get_host_appimagetool,
get_linux_architecture_config,
)
class LinuxArchitectureConfigTest(unittest.TestCase):
def test_returns_x64_packaging_config(self) -> None:
config = get_linux_architecture_config("x64")
self.assertEqual(config["appimage_arch"], "x86_64")
self.assertEqual(config["deb_arch"], "amd64")
def test_returns_arm64_packaging_config(self) -> None:
config = get_linux_architecture_config("arm64")
self.assertEqual(config["appimage_arch"], "aarch64")
self.assertEqual(config["deb_arch"], "arm64")
def test_rejects_unsupported_architecture(self) -> None:
with self.assertRaisesRegex(ValueError, "Unsupported Linux architecture"):
get_linux_architecture_config("universal")
class HostAppImageToolTest(unittest.TestCase):
"""The appimagetool binary must match the BUILD machine's arch, not
the target arch — otherwise cross-compiling arm64 packages from an x64
host fails because the aarch64 tool can't execute on x64."""
def test_x64_host_picks_x86_64_tool(self) -> None:
with patch(
"build.modules.package.linux.get_platform_arch", return_value="x64"
):
filename, url = get_host_appimagetool()
self.assertEqual(filename, "appimagetool-x86_64.AppImage")
self.assertIn("x86_64", url)
def test_arm64_host_picks_aarch64_tool(self) -> None:
with patch(
"build.modules.package.linux.get_platform_arch", return_value="arm64"
):
filename, url = get_host_appimagetool()
self.assertEqual(filename, "appimagetool-aarch64.AppImage")
self.assertIn("aarch64", url)
def test_host_lookup_independent_of_target(self) -> None:
# Both architectures must be present in the host lookup so cross
# builds work in either direction.
self.assertIn("x64", LINUX_HOST_APPIMAGETOOL)
self.assertIn("arm64", LINUX_HOST_APPIMAGETOOL)
if __name__ == "__main__":
unittest.main()

View File

@@ -6,7 +6,6 @@ from datetime import datetime
from typing import Dict, List, Optional
from ...common.env import EnvConfig
from ...common.utils import log_warning
from ..storage import get_release_json, get_r2_client, BOTO3_AVAILABLE
PLATFORMS = ["macos", "win", "linux"]
@@ -24,6 +23,8 @@ DOWNLOAD_PATH_MAPPING = {
"linux": {
"x64_appimage": "download/BrowserOS.AppImage",
"x64_deb": "download/BrowserOS.deb",
"arm64_appimage": "download/BrowserOS-arm64.AppImage",
"arm64_deb": "download/BrowserOS-arm64.deb",
},
}

View File

@@ -1,12 +1,24 @@
#!/usr/bin/env python3
"""Git operations module for BrowserOS build system"""
import re
import subprocess
import tarfile
import urllib.request
from typing import List
from ...common.module import CommandModule, ValidationError
from ...common.context import Context
from ...common.utils import run_command, log_info, log_error, log_success, IS_WINDOWS, safe_rmtree
from ...common.utils import (
run_command,
log_info,
log_warning,
log_error,
log_success,
IS_LINUX,
IS_WINDOWS,
safe_rmtree,
)
class GitSetupModule(CommandModule):
@@ -32,6 +44,12 @@ class GitSetupModule(CommandModule):
log_info(f"🔀 Checking out tag: {ctx.chromium_version}")
run_command(["git", "checkout", f"tags/{ctx.chromium_version}"], cwd=ctx.chromium_src)
# On Linux, depot_tools fetches per-arch sysroots automatically when
# `.gclient` declares `target_cpus`. Ensure both x64 and arm64 are
# listed before sync so cross-compilation just works on x64 hosts.
if IS_LINUX():
self._ensure_gclient_target_cpus(ctx, ["x64", "arm64"])
log_info("📥 Syncing dependencies (this may take a while)...")
if IS_WINDOWS():
run_command(["gclient.bat", "sync", "-D", "--no-history", "--shallow"], cwd=ctx.chromium_src)
@@ -40,6 +58,49 @@ class GitSetupModule(CommandModule):
log_success("Git setup complete")
def _ensure_gclient_target_cpus(self, ctx: Context, required: List[str]) -> None:
"""Idempotently add `target_cpus` to .gclient so depot_tools fetches
the matching Linux sysroots for cross-compilation.
depot_tools convention: .gclient lives one directory above
chromium_src (i.e. ../.gclient). It is a Python file with a list
of solution dicts followed by optional top-level assignments.
We append a `target_cpus = [...]` line if missing or merge in any
archs that aren't already present.
"""
gclient_path = ctx.chromium_src.parent / ".gclient"
if not gclient_path.exists():
log_warning(
f"⚠️ .gclient not found at {gclient_path}; "
f"skipping target_cpus bootstrap. "
f"Cross-arch builds may fail until you run `fetch chromium`."
)
return
content = gclient_path.read_text()
match = re.search(r"^\s*target_cpus\s*=\s*\[([^\]]*)\]", content, re.MULTILINE)
if match:
existing = re.findall(r"['\"]([^'\"]+)['\"]", match.group(1))
missing = [arch for arch in required if arch not in existing]
if not missing:
log_info(f"✓ .gclient target_cpus already includes {required}")
return
merged = sorted(set(existing) | set(required))
new_line = f"target_cpus = {merged!r}"
content = (
content[: match.start()] + new_line + content[match.end() :]
)
log_info(
f"📝 Updating .gclient target_cpus: {existing}{merged}"
)
else:
new_line = f"\ntarget_cpus = {required!r}\n"
content = content.rstrip() + "\n" + new_line
log_info(f"📝 Adding target_cpus = {required} to .gclient")
gclient_path.write_text(content)
def _verify_tag_exists(self, ctx: Context) -> None:
result = subprocess.run(
["git", "tag", "-l", ctx.chromium_version],

View File

@@ -4,7 +4,7 @@
import json
from datetime import datetime, timezone
from pathlib import Path
from typing import Dict, List, Optional, Tuple
from typing import Any, Dict, List, Optional, Tuple, cast
from ...common.module import CommandModule, ValidationError
from ...common.context import Context
@@ -21,6 +21,7 @@ from ...common.notify import get_notifier, COLOR_GREEN
from .r2 import (
BOTO3_AVAILABLE,
get_r2_client,
get_release_json,
upload_file_to_r2,
)
@@ -58,7 +59,10 @@ class UploadModule(CommandModule):
log_info("\nUploading package artifacts to R2...")
extra_metadata = {}
sparkle_signatures = ctx.artifacts.get("sparkle_signatures")
sparkle_signatures = cast(
Optional[dict[str, tuple[str, int]]],
ctx.artifacts.get("sparkle_signatures"),
)
if sparkle_signatures:
for filename, (sig, length) in sparkle_signatures.items():
extra_metadata[filename] = {
@@ -120,6 +124,36 @@ def generate_release_json(
return release_data
def merge_release_metadata(existing: Optional[Dict], new: Dict) -> Dict:
if not existing:
return new
merged = dict(existing)
merged.update({key: value for key, value in new.items() if key != "artifacts"})
artifacts = dict(existing.get("artifacts", {}))
artifacts.update(new.get("artifacts", {}))
merged["artifacts"] = artifacts
return merged
def _get_linux_artifact_key(filename: str) -> Optional[str]:
lower = filename.lower()
if ".appimage" in lower:
if "arm64" in lower or "aarch64" in lower:
return "arm64_appimage"
if "x64" in lower or "x86_64" in lower:
return "x64_appimage"
elif ".deb" in lower:
if "arm64" in lower or "aarch64" in lower:
return "arm64_deb"
if "amd64" in lower or "x64" in lower or "x86_64" in lower:
return "x64_deb"
return None
def _get_artifact_key(filename: str, platform: str) -> str:
"""Get artifact key name from filename
@@ -147,10 +181,10 @@ def _get_artifact_key(filename: str, platform: str) -> str:
return "x64_zip"
elif platform == "linux":
if ".appimage" in lower:
return "x64_appimage"
elif ".deb" in lower:
return "x64_deb"
artifact_key = _get_linux_artifact_key(filename)
if artifact_key:
return artifact_key
log_warning(f"Unrecognized Linux artifact name: {filename}; using stem key")
return Path(filename).stem
@@ -181,7 +215,7 @@ def detect_artifacts(ctx: Context) -> List[Path]:
def upload_release_artifacts(
ctx: Context,
extra_metadata: Optional[Dict[str, Dict[str, any]]] = None,
extra_metadata: Optional[Dict[str, Dict[str, Any]]] = None,
) -> Tuple[bool, Optional[Dict]]:
"""Upload release artifacts to R2 and generate release.json
@@ -240,6 +274,13 @@ def upload_release_artifacts(
artifact_metadata.append(metadata)
release_data = generate_release_json(ctx, artifact_metadata, platform)
if platform == "linux":
# Linux x64 and arm64 release jobs must be sequenced. A parallel
# fetch-merge-upload flow can still race and drop one architecture.
existing_release_data = get_release_json(
ctx.get_semantic_version(), platform, env
)
release_data = merge_release_metadata(existing_release_data, release_data)
release_json_path = ctx.get_dist_dir() / "release.json"
release_json_path.write_text(json.dumps(release_data, indent=2))
@@ -248,7 +289,7 @@ def upload_release_artifacts(
return False, None
log_success(f"\nSuccessfully uploaded {len(artifacts)} artifact(s) to R2")
log_info(f"\nRelease metadata:")
log_info("\nRelease metadata:")
log_info(f" Version: {release_data['version']}")
if platform == "macos":
log_info(f" Sparkle version: {release_data.get('sparkle_version', 'N/A')}")

View File

@@ -0,0 +1,85 @@
#!/usr/bin/env python3
"""Tests for release artifact upload metadata helpers."""
import unittest
from build.modules.storage.upload import _get_artifact_key, merge_release_metadata
class UploadMetadataTest(unittest.TestCase):
def test_linux_x64_artifacts_use_x64_keys(self) -> None:
self.assertEqual(
_get_artifact_key("BrowserOS_v1.2.3_x64.AppImage", "linux"),
"x64_appimage",
)
self.assertEqual(
_get_artifact_key("BrowserOS_v1.2.3_amd64.deb", "linux"),
"x64_deb",
)
def test_linux_arm64_artifacts_use_arm64_keys(self) -> None:
self.assertEqual(
_get_artifact_key("BrowserOS_v1.2.3_arm64.AppImage", "linux"),
"arm64_appimage",
)
self.assertEqual(
_get_artifact_key("BrowserOS_v1.2.3_arm64.deb", "linux"),
"arm64_deb",
)
self.assertEqual(
_get_artifact_key("BrowserOS_v1.2.3_aarch64.deb", "linux"),
"arm64_deb",
)
def test_merge_release_metadata_preserves_existing_artifacts(self) -> None:
existing = {
"platform": "linux",
"version": "1.2.3",
"build_date": "old",
"artifacts": {
"x64_appimage": {"filename": "BrowserOS_v1.2.3_x64.AppImage"},
"x64_deb": {"filename": "BrowserOS_v1.2.3_amd64.deb"},
},
}
new = {
"platform": "linux",
"version": "1.2.3",
"build_date": "new",
"artifacts": {
"arm64_appimage": {"filename": "BrowserOS_v1.2.3_arm64.AppImage"},
"arm64_deb": {"filename": "BrowserOS_v1.2.3_arm64.deb"},
},
}
merged = merge_release_metadata(existing, new)
self.assertEqual(merged["build_date"], "new")
self.assertEqual(
sorted(merged["artifacts"]),
["arm64_appimage", "arm64_deb", "x64_appimage", "x64_deb"],
)
def test_merge_release_metadata_overwrites_matching_artifact_keys(self) -> None:
existing = {
"platform": "linux",
"version": "1.2.3",
"artifacts": {
"x64_appimage": {"filename": "old.AppImage", "size": 1},
},
}
new = {
"platform": "linux",
"version": "1.2.3",
"artifacts": {
"x64_appimage": {"filename": "new.AppImage", "size": 2},
},
}
merged = merge_release_metadata(existing, new)
self.assertEqual(merged["artifacts"]["x64_appimage"]["filename"], "new.AppImage")
self.assertEqual(merged["artifacts"]["x64_appimage"]["size"], 2)
if __name__ == "__main__":
unittest.main()

View File

@@ -1,9 +1,9 @@
diff --git a/chrome/browser/browseros/extensions/browseros_extension_loader.cc b/chrome/browser/browseros/extensions/browseros_extension_loader.cc
new file mode 100644
index 0000000000000..e61b45d08b7e2
index 0000000000000..fdb6be443f25b
--- /dev/null
+++ b/chrome/browser/browseros/extensions/browseros_extension_loader.cc
@@ -0,0 +1,226 @@
@@ -0,0 +1,269 @@
+// Copyright 2024 The Chromium Authors
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
@@ -89,10 +89,53 @@ index 0000000000000..e61b45d08b7e2
+ extension_ids_.merge(result.extension_ids);
+ last_config_ = std::move(result.config);
+
+ LoadFinished(std::move(result.prefs));
+ base::DictValue prefs_to_load = std::move(result.prefs);
+
+ if (prefs_to_load.empty()) {
+ LOG(WARNING) << "browseros: Install returned empty prefs, "
+ << "reconstructing from installed extensions";
+ prefs_to_load = ReconstructPrefsFromInstalledExtensions();
+ LOG(INFO) << "browseros: Reconstructed prefs for "
+ << prefs_to_load.size() << " installed extensions";
+ }
+
+ LoadFinished(std::move(prefs_to_load));
+ OnStartupComplete(result.from_bundled);
+}
+
+base::DictValue
+BrowserOSExtensionLoader::ReconstructPrefsFromInstalledExtensions() {
+ base::DictValue prefs;
+
+ extensions::ExtensionRegistry* registry =
+ extensions::ExtensionRegistry::Get(profile_);
+ if (!registry) {
+ return prefs;
+ }
+
+ const std::string update_url =
+ base::FeatureList::IsEnabled(features::kBrowserOsAlphaFeatures)
+ ? kBrowserOSAlphaUpdateUrl
+ : kBrowserOSUpdateUrl;
+
+ for (const std::string& id : GetBrowserOSExtensionIds()) {
+ const extensions::Extension* ext = registry->GetInstalledExtension(id);
+ if (!ext) {
+ continue;
+ }
+
+ base::DictValue ext_pref;
+ ext_pref.Set(extensions::ExternalProviderImpl::kExternalUpdateUrl,
+ update_url);
+ prefs.Set(id, std::move(ext_pref));
+
+ LOG(INFO) << "browseros: Reconstructed pref for installed extension "
+ << id << " v" << ext->version().GetString();
+ }
+
+ return prefs;
+}
+
+const base::FilePath BrowserOSExtensionLoader::GetBaseCrxFilePath() {
+ return bundled_crx_base_path_;
+}

View File

@@ -1,9 +1,9 @@
diff --git a/chrome/browser/browseros/extensions/browseros_extension_loader.h b/chrome/browser/browseros/extensions/browseros_extension_loader.h
new file mode 100644
index 0000000000000..2a14e9068156e
index 0000000000000..ea2c856556f5f
--- /dev/null
+++ b/chrome/browser/browseros/extensions/browseros_extension_loader.h
@@ -0,0 +1,81 @@
@@ -0,0 +1,86 @@
+// Copyright 2024 The Chromium Authors
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
@@ -62,6 +62,11 @@ index 0000000000000..2a14e9068156e
+ // Convergence point for both startup paths.
+ void OnStartupComplete(bool from_bundled);
+
+ // Reconstructs minimal prefs from already-installed BrowserOS extensions.
+ // Used as a fallback when both bundled CRX and remote config fail,
+ // preventing orphan detection from uninstalling existing extensions.
+ base::DictValue ReconstructPrefsFromInstalledExtensions();
+
+ // Installs remote extensions immediately via PendingExtensionManager + updater.
+ void InstallRemoteExtensionsNow(base::DictValue config);
+

View File

@@ -1,6 +1,6 @@
diff --git a/chrome/browser/browseros/extensions/browseros_extension_maintainer.cc b/chrome/browser/browseros/extensions/browseros_extension_maintainer.cc
new file mode 100644
index 0000000000000..bb33ae5d3b156
index 0000000000000..5804d54696e8f
--- /dev/null
+++ b/chrome/browser/browseros/extensions/browseros_extension_maintainer.cc
@@ -0,0 +1,395 @@

View File

@@ -1,5 +1,5 @@
diff --git a/chrome/browser/extensions/chrome_extension_registrar_delegate.cc b/chrome/browser/extensions/chrome_extension_registrar_delegate.cc
index 6eec0585e8925..55c2a73647527 100644
index adfb4e4d49fa4..409e26fa1cb1b 100644
--- a/chrome/browser/extensions/chrome_extension_registrar_delegate.cc
+++ b/chrome/browser/extensions/chrome_extension_registrar_delegate.cc
@@ -12,6 +12,7 @@
@@ -10,7 +10,26 @@ index 6eec0585e8925..55c2a73647527 100644
#include "chrome/browser/extensions/component_loader.h"
#include "chrome/browser/extensions/corrupted_extension_reinstaller.h"
#include "chrome/browser/extensions/data_deleter.h"
@@ -317,6 +318,13 @@ bool ChromeExtensionRegistrarDelegate::CanDisableExtension(
@@ -256,7 +257,17 @@ void ChromeExtensionRegistrarDelegate::PostUninstallExtension(
}
}
- DataDeleter::StartDeleting(profile_, extension.get(), subtask_done_callback);
+ // Preserve chrome.storage.local data for BrowserOS extensions. These may be
+ // transiently uninstalled during update cycles (e.g., when both bundled CRX
+ // and remote config fail on startup). User configuration must survive.
+ if (browseros::IsBrowserOSExtension(extension->id())) {
+ LOG(INFO) << "browseros: Preserving storage for extension "
+ << extension->id();
+ subtask_done_callback.Run();
+ } else {
+ DataDeleter::StartDeleting(profile_, extension.get(),
+ subtask_done_callback);
+ }
}
void ChromeExtensionRegistrarDelegate::DoLoadExtensionForReload(
@@ -322,6 +333,13 @@ bool ChromeExtensionRegistrarDelegate::CanDisableExtension(
return true;
}