Benchmarks
The app's benchmarks run with strobe. Each compares two builds in pairs of fresh launches (A B, B A, …) at normal speed, decides one primary metric per case with an interval, and refuses setups it cannot defend (a hidden window, a software GPU, a throttled CPU, a busy machine). The web cases run the production build on the e2e mock server, so they need no backend.
| File | Cases | Workload |
|---|---|---|
timeline.bench.ts |
scroll history · type in composer · find in session · expand an edit · drag the review's edge | A 120-turn session of markdown, code, tools and diffs. |
stream.bench.ts |
stream a reply · into a tool-heavy history · with review open · with Find open · wait on a running session | 160 markdown deltas at 40 a second from a server that never waits. |
tabs.bench.ts |
switch tab (warm, review open, after a resize, 1,200 models) · away and back with review open · open an unvisited tab (± review) | Two tabs of 200 long markdown exchanges each, loaded in full. |
navigation.bench.ts |
Home → session · close the last tab · new session (Home, titlebar) · new session page in and out · child session · restore 8 tabs · activate a restored tab · command palette (session, Home, scroll) | The stress fixture, paginated, from a fresh document every run. |
home.bench.ts |
load home · update a session in the background | 500, 5,000 and 10,000 root sessions across 12 folders. |
review.bench.ts |
open review · filter files · next file | 100 and 10,000 changed files of 100 lines each. |
diff.bench.ts |
open a long diff · open a huge diff · return to a long diff | A 1,559-line diff, highlighted; a 15,599-line one, plain. |
composer.bench.ts |
type in a large draft · move the cursor in it · recall a prompt from history | A 44,000-character saved draft; 100 prompts with screenshots. |
docks.bench.ts |
open and close request docks | A permission dock and an MCP elicitation over a 20-turn session. |
question.bench.ts |
switch questions | The new question dock with its motion on: five questions. |
desktop.bench.ts |
idle | The packaged desktop app, every process, doing nothing for 5 s. |
Every case decides main.work (renderer main-thread task time per run) unless it says otherwise: it is continuous,
while durations that end on a presented frame move in whole vsyncs. Durations, interaction latency, frames and the
main thread's split into script, style and layout are reported beside it, and some cases count what should not
happen, compared exactly:
| Count | Case | Expected |
|---|---|---|
net.requests{/api/session/*/message} |
switch tab (warm) | 0: a cached transcript stays |
net.requests{/api/session/*/message} |
restore 8 tabs | 1: only the active tab's |
count.draft.encode |
type in a large draft | about 25 for 51 keystrokes: batched |
count.draft.encode |
move the cursor in a large draft | 0: a caret move saves no text |
layout.shift |
switch tab, open an unvisited tab | 0: nothing on screen jumps |
Other packages keep their benchmarks next to their code:
| File | Cases | Decides |
|---|---|---|
packages/desktop/perf/startup.bench.ts |
launch to Home · to a restored tab · with the service stopped · first launch · dev start to Home | launch.ready, from process spawn |
packages/core/perf/location.bench.ts |
load a location, cached and dropped · keep a location graph · keep a models.dev plugin instance | cpu.work, js.heap.retained |
packages/core/perf/snapshot.bench.ts |
capture (nothing changed, a file edited or added, 100 edited) · list and diff 100 changed files · restore 100 files · agent step loop · pack | run.duration, with count.git |
packages/core/perf/session.bench.ts |
an agent session of 15 steps, snapshots on and off | run.duration |
packages/ui/perf/motion.bench.ts |
1,000 springs for 60 frames | cpu.work |
A script case loads the package's harness module (location.ts, snapshot.ts, motion.ts) in a fresh Bun process
per block and calls its exports. Both builds run the same harness: in a git ref's worktree strobe places this
checkout's copy, so its imports reach that build's code.
Run
# Did my change help? Compare dev with this worktree.
strobe bench packages/app/perf/timeline.bench.ts --base origin/dev --head .
# Does the file still work after a change to the app? Each case once, pass or fail with a screenshot.
strobe bench packages/app/perf/tabs.bench.ts --check
# One case, one parameter value, or the whole sweep (Home and review run two sizes unless asked).
strobe bench packages/app/perf/tabs.bench.ts --base origin/dev --head . --filter "open an unvisited*"
strobe bench packages/app/perf/home.bench.ts --base origin/dev --head . --params sessions=5000
strobe bench packages/app/perf/review.bench.ts --base origin/dev --head . --params all
# Is the method sound on this machine? The same build on both sides must come out "same".
strobe bench packages/app/perf/timeline.bench.ts --aa --head .
# Two installed desktop channels, or the desktop's launch (packages/desktop/README.md).
strobe bench packages/app/perf/desktop.bench.ts --base app:opencode --head app:opencode-beta
strobe bench packages/desktop/perf/startup.bench.ts --target desktop --base app:opencode --head app:opencode-beta
# Core and UI code, in a Bun process. The snapshot fixtures are generated once (a minute or two).
bun packages/core/perf/snapshot.ts medium opencode
strobe bench packages/core/perf/snapshot.bench.ts --base origin/dev --head .
# The interactive report, and the results block in pull request <n> (see "In a pull request").
strobe report experiment <exp_…> --open
strobe report experiment <exp_…> --format pr --update-pr <n>
In a pull request
Write the PR's own text: what strobe showed that led to the change (the command and recording id), and what the change
does. Then strobe report experiment <exp_…> --format pr --update-pr <n> adds strobe's generated results block to PR
n's body, or replaces that experiment's block after a rerun, and leaves the rest of the body alone. Don't edit inside
the block (strobe refuses to update a hand-edited one); put your reading of the numbers above or below it. Without
--update-pr, the block goes to stdout for pasting.
Every page target expects the OpenCode title (expect): a block that lands on another page (an error page, a blank
document) is refused WrongScreen instead of timed, and the block's On screen: line says what each block showed.
bunx strobe runs the strobe version pinned in the root catalog (@hona/strobe), which ships builds for Windows x64,
macOS (arm64, x64) and Linux (x64, arm64); a strobe on your PATH also works. Script cases (core, ui) run the same on
every OS. Page and desktop cases measure the most on Windows:
- On macOS and Linux, frames come from Chromium's own trace only: the OS's present events (
--os-frames) are Windows only. A software GPU is refused, so a VM or a server without a GPU cannot run page cases. - macOS has no CPU affinity API, so runs are not pinned. Window pixels (
launch.painted,launch.settled) need the Screen Recording permission. A desktop case launches only a build whose bundle id this account has never run: an isolated launch cannot move a Cocoa app's~/Library, so strobe refuses rather than write into yours. - Linux finds windows through X11 (XWayland works; a native Wayland window is invisible to other apps).
Set OPENCODE_PERF_SESSION to a session export to add two cases to tabs.bench.ts on your own session: open it cold,
and switch to it warm.
Why did it change?
A verdict that moved adds one profiled launch per side after the timed pairs (--why always forces it). The report's
lower half reads them, and so do these views:
strobe query <exp_…> insights # what moved most between base and head, and what stands out in head
strobe query <rec_…> hotspots # CPU self time by function, at the original file:line:column, with stacks
strobe query <rec_…> layout # forced layouts and style recalculations, by the line that forced them
strobe query <rec_…> alloc # bytes allocated by function
strobe query <rec_…> gc # collections and GC time per run
strobe query <rec_…> heap # live objects by class, retained sizes, retainer paths
strobe query <base rec>..<head rec> heap # which classes grew or shrank
strobe query <rec_…> memory # memory by process; --by layer or --by allocator for Chromium's accounting
strobe query <rec_…> shifts # the largest layout shifts and the elements that moved
Is something leaking?
A leak run repeats one case (10 runs to a middle snapshot, 10 more to the last) with a census after every run, and says how much the JS heap and DOM grow per run, with an interval. It tells the app's objects from V8's compiled code, which grows while the JIT warms up:
strobe bench packages/app/perf/tabs.bench.ts --filter "switch tab with a large model catalog" --head . --probes memory --repeat 10
strobe query <rec_…> leaks # objects from the first half still alive at the end, and what holds them
strobe record app:opencode --probes memory --for 60s # the same over time, for an app left running
Which commit moved it?
strobe bench packages/app/perf/timeline.bench.ts --filter "scroll*" --bisect <good>..<bad>
--bisect walks the first-parent line, one full experiment per step.
Write a case
- A file's
prepareruns in Playwright before the first page load and mocks the server under the variant's own origin (new URL(url).origin);ctx.caselets one file give each case its own fixture. What it returns ispreparedinsetup,runandreset. - Mock on
cached(playwright.page, mock)(fixtures/mock.ts), never onplaywright.pageitself: Playwright's routing turns the browser's HTTP cache off for the whole page, so every reload would refetch every script.cachedanswers the same routes through strobe'smock, which pauses only the API, and the e2e mock server runs on it unchanged. runis measured;resetputs the state back between runs and is not. A case that should pay a first visit every run reloads inreset; park the pointer withpage.hover(...)afterwards, because the app prefetches a tab's transcript when the pointer rests on it, and the pointer stays where the last click left it.- End a run on what a person waits for: a selector that is visible (strobe ignores
visibility: hiddenandopacity: 0) or{ selector, text }, with>>>to reach into a shadow root such as the diff viewer's. End withpage.idle()when work continues after the last change on screen. - Deliver events with
eventSource(fixtures/events.ts): a real SSE endpoint whose pace never depends on the renderer, so delivery costs the page nothing. Stop it in the file'steardown. - Count what should not happen with
metrics:net.requests{<path glob>}for requests,layout.shift, andcount.<name>fromctx.countor a counting init script (countDraftEncodesinfixtures/composer.ts). The app itself never counts for a benchmark. - Do not throttle the CPU or the network: make the workload bigger instead. A parameter that sweeps sizes runs two by
default (
{ run, more });--params allruns the curve. - Fixtures shared with the e2e suites live in
e2e/utils; ones only benchmarks use live infixtures/.bun run typecheck:e2echecks both and every bench file, against@hona/strobe's types; strobe runs its own copy of that package. Validate a new or changed file with--checkbefore a long run.
strobe.json
buildandserveare how strobe builds and serves the web app for each variant. A variant is a directory or a git ref; a git ref becomes a detached worktree in strobe's own build folder.appsare the desktop profiles. Strobe gives every launch a private home; a profile adds what the desktop needs on top: its own database and config, a private service port, a file where the updater keeps its download cache (otherwise an older channel downloads the next release mid-run and looks like it idles at 10% CPU), the user's service registration as guarded state (strobe stops if it changes), and theopencode://key, which strobe saves and restores around each launch. The Dev channel has no profile on purpose. A source build is packed withOPENCODE_CHANNEL=betaorprodand passed asexe:<path>.
Not benchmarks
- Layout continuity. No blank frames, jumps or replaced rows under stress is a correctness contract, owned by the
regression suites and by
e2e/stability(bun run test:stability), not a benchmark. - Snapshot robustness. A zeroed index with a stale lock, concurrent captures from separate processes and a
..scopefolder arepackages/core/test/snapshot.test.ts; packing keeps every object inpackages/core/test/git.test.ts.