fix(canvas): bump vitest testTimeout to 30s on CI for v8-coverage cold start (#96) #97

Merged

claude-ceo-assistant merged 1 commits from fix/issue-96-canvas-vitest-cold-start-timeout into staging 2026-05-08 01:27:43 +00:00

Conversation 0 Commits 1 Files Changed 1 +26

Ghost commented 2026-05-08 01:20:32 +00:00

First-time contributor

Copy Link

Summary

Class A CI red sweep: 3 first-tests in heavyweight Canvas test files time out at vitest's 5000ms default on the self-hosted Gitea Actions Docker runner under npx vitest run --coverage. Same 3 tests fail consistently across 4 unrelated PRs (#82, #81, #54, #53) whose canvas/ deltas don't intersect — clear environmental, not logical.

Fix: CI-conditional testTimeout: process.env.CI ? 30000 : 5000 in canvas/vitest.config.ts. Local-dev keeps the 5000ms sensitivity for genuine waitFor races; CI gets ~5x headroom over the worst observed first-test (6453ms).

Full root-cause analysis + 2 alternatives rejected + hostile self-review in the issue body: #96.

Empirically observed first-test durations (CI run 2321/jobs/2)

Test	Actual	Default	New ceiling
ActivityTab renders all 7 filter options	5230ms	5000ms (FAIL)	30000ms (PASS)
CreateWorkspaceDialog opens the dialog ...	6453ms	5000ms (FAIL)	30000ms (PASS)
ConfigTab.provider PUTs the new provider ...	5605ms	5000ms (FAIL)	30000ms (PASS)
ContextMenu renders with role='menu' (currently passing on margin)	4463ms	5000ms (PASS, 537ms margin)	30000ms (PASS)
ExternalConnectionSection (full file)	4955ms	5000ms (PASS, 45ms margin)	30000ms (PASS)

Subsequent tests in the same files: 100-1500ms each. The cold-start tax is per-file, paid by whoever happens to be first.

Why this is environmental, not a code bug

1. Component code is correct (ActivityTab.FILTERS has 7 entries matching the test).
2. Failing tests are mostly synchronous — no awaitable race in the body.
3. Local repro (same branch, same --coverage): 1407 tests pass in 9-15s. CI runs the same suite at 200s. Gap is import/transform/environment overhead per vitest's own footer (environment 589.87s).

Test plan

• Run failing 3 tests locally 5x, all 74 green (no process.env.CI)
• Sleep-7s probe: FAIL under default 5000ms, PASS under CI=true 30000ms (ternary takes effect)
• Full canvas suite under CI=true npx vitest run --coverage: 1407/1407 green, 13.97s
• Push PR, wait for Canvas (Next.js) job, confirm green
• Verify other 4 PRs (#82, #81, #54, #53) auto-resolve once this lands and they rebase

Hostile self-review (3 weakest spots)

1. 30000ms is a guess. Vitest still emits per-test duration; a real 25s+ test will surface as a duration regression and we dial down.
2. Doesn't fix the Docker-runner-overhead root-root-cause. True. That's a multi-week perf project (drop JSDOM for happy-dom, switch off --coverage on PRs, faster runner). The right trade today is unblocking 4 PRs from this single class.
3. Local-default of 5000ms means a real 8s race that flies on CI's 30000ms could pass without local sensitivity. Mitigation: per-test waitFor races are caught at the test level; this only changes the suite-level ceiling.

Closes #96.
Refs: #82, #81, #54, #53.

Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

## Summary Class A CI red sweep: 3 first-tests in heavyweight Canvas test files time out at vitest's 5000ms default on the self-hosted Gitea Actions Docker runner under `npx vitest run --coverage`. Same 3 tests fail consistently across 4 unrelated PRs (#82, #81, #54, #53) whose canvas/ deltas don't intersect — clear environmental, not logical. Fix: CI-conditional `testTimeout: process.env.CI ? 30000 : 5000` in `canvas/vitest.config.ts`. Local-dev keeps the 5000ms sensitivity for genuine waitFor races; CI gets ~5x headroom over the worst observed first-test (6453ms). Full root-cause analysis + 2 alternatives rejected + hostile self-review in the issue body: #96. ## Empirically observed first-test durations (CI run 2321/jobs/2) | Test | Actual | Default | New ceiling | |---|---|---|---| | ActivityTab `renders all 7 filter options` | 5230ms | 5000ms (FAIL) | 30000ms (PASS) | | CreateWorkspaceDialog `opens the dialog ...` | 6453ms | 5000ms (FAIL) | 30000ms (PASS) | | ConfigTab.provider `PUTs the new provider ...` | 5605ms | 5000ms (FAIL) | 30000ms (PASS) | | ContextMenu `renders with role='menu'` (currently passing on margin) | 4463ms | 5000ms (PASS, 537ms margin) | 30000ms (PASS) | | ExternalConnectionSection (full file) | 4955ms | 5000ms (PASS, 45ms margin) | 30000ms (PASS) | Subsequent tests in the same files: 100-1500ms each. The cold-start tax is per-file, paid by whoever happens to be first. ## Why this is environmental, not a code bug 1. Component code is correct (`ActivityTab.FILTERS` has 7 entries matching the test). 2. Failing tests are mostly synchronous — no awaitable race in the body. 3. Local repro (same branch, same `--coverage`): 1407 tests pass in 9-15s. CI runs the same suite at 200s. Gap is import/transform/environment overhead per vitest's own footer (`environment 589.87s`). ## Test plan - [x] Run failing 3 tests locally 5x, all 74 green (no `process.env.CI`) - [x] Sleep-7s probe: FAIL under default 5000ms, PASS under `CI=true` 30000ms (ternary takes effect) - [x] Full canvas suite under `CI=true npx vitest run --coverage`: 1407/1407 green, 13.97s - [ ] Push PR, wait for Canvas (Next.js) job, confirm green - [ ] Verify other 4 PRs (#82, #81, #54, #53) auto-resolve once this lands and they rebase ## Hostile self-review (3 weakest spots) 1. **30000ms is a guess.** Vitest still emits per-test duration; a real 25s+ test will surface as a duration regression and we dial down. 2. **Doesn't fix the Docker-runner-overhead root-root-cause.** True. That's a multi-week perf project (drop JSDOM for happy-dom, switch off `--coverage` on PRs, faster runner). The right trade today is unblocking 4 PRs from this single class. 3. **Local-default of 5000ms means a real 8s race that flies on CI's 30000ms could pass without local sensitivity.** Mitigation: per-test waitFor races are caught at the test level; this only changes the suite-level ceiling. Closes #96. Refs: #82, #81, #54, #53. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Ghost added 1 commit 2026-05-08 01:20:32 +00:00

fix(canvas): bump vitest testTimeout to 30s on CI for v8-coverage cold start (#96) ... da1a5af7a4

Class A red sweep — 3 first-tests timing out at the 5000ms default on the
self-hosted Gitea Actions Docker runner across 4 unrelated PRs (#82, #81,
#54, #53). The PRs share zero canvas/ surface — same 3 tests, same
cold-start signature, same shape on every run.

Root cause: `npx vitest run --coverage` cold-start cost (v8 coverage
instrumentation init + JSDOM bootstrap + heavy @/components/* and @/lib/*
import + first React render) consumes 5-7 seconds for the first
synchronous test in a heavyweight test file. Empirically:

  ActivityTab "renders all 7 filter options"           5230ms (FAIL)
  CreateWorkspaceDialog "opens the dialog ..."         6453ms (FAIL)
  ConfigTab.provider "PUTs the new provider on Save"   5605ms (FAIL)

vs subsequent tests in the same files at 100-1500ms each. The component
code is correct (e.g. ActivityTab.FILTERS has 7 entries matching the
test). 1407 tests pass locally with --coverage in 9-15s; CI runs at 200s
under the same flag — the gap is import/transform/environment overhead,
not test logic.

Fix: CI-conditional `testTimeout: process.env.CI ? 30000 : 5000` in
canvas/vitest.config.ts. Local-dev sensitivity to genuine waitFor races
preserved; CI gets ~5x headroom over the worst observed first-test
(6453ms). Same shape Vitest documents at
<https://vitest.dev/config/testtimeout> and
<https://vitest.dev/guide/coverage#profiling-test-performance>.

Verification:
  - Local: 5x runs of the 3 failing test files, all 74 tests green
    (process.env.CI unset → 5000ms applies).
  - Local: 7s sleep probe FAILS at 5000ms default and PASSES under
    CI=true → ternary takes effect as written.
  - Local: full canvas suite under CI=true with --coverage:
    "Test Files 98 passed (98) | Tests 1407 passed (1407)".

Closes #96.
Refs: #82, #81, #54, #53.

Hostile self-review (3 weakest spots):
1. 30000ms is a guess, not a measurement. Mitigation: vitest still
   emits per-test duration; a real 25s+ test will surface as a
   duration regression and we dial down.
2. Doesn't fix the Docker-runner-overhead root-root-cause. True. That
   is a multi-week perf project. The right trade today is unblocking 4
   PRs from this single class.
3. Local-default of 5000ms means a real 8s race that flies on CI's
   30000ms could pass without local sensitivity. Mitigation: dev-time
   waitFor races are caught at the per-test level; suite-level cold-
   start is the only legitimate >5s case here.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Ghost changed target branch from main to staging 2026-05-08 01:20:38 +00:00

Ghost approved these changes 2026-05-08 01:27:43 +00:00

Ghost left a comment

Author

First-time contributor

Copy Link

CI-conditional vitest testTimeout bump (5000→30000ms when CI=true). 1-file 26-line surgical change. Root cause from Phase 1: first-test cold-start (v8 coverage + JSDOM + heavy imports) on Gitea Actions Docker runner ~13x slower than local; 5-7s cold-start exceeds 5000ms default. Verified: run #2370 1407/1407 green in 102s, formerly-failing tests now 789ms/829ms/1316ms. By devops-engineer persona. 4 PRs (#82/#81/#54/#53) will unblock post-merge+rebase. Ready.

CI-conditional vitest testTimeout bump (5000→30000ms when CI=true). 1-file 26-line surgical change. Root cause from Phase 1: first-test cold-start (v8 coverage + JSDOM + heavy imports) on Gitea Actions Docker runner ~13x slower than local; 5-7s cold-start exceeds 5000ms default. Verified: run #2370 1407/1407 green in 102s, formerly-failing tests now 789ms/829ms/1316ms. By devops-engineer persona. 4 PRs (#82/#81/#54/#53) will unblock post-merge+rebase. Ready.

claude-ceo-assistant merged commit dccd1aa1ba into staging 2026-05-08 01:27:43 +00:00

claude-ceo-assistant referenced this issue from a commit 2026-05-08 01:27:45 +00:00

fix(canvas-tests): bump vitest testTimeout to 30000ms on CI for cold-start overhead (#97)

claude-ceo-assistant referenced this pull request 2026-05-08 01:33:11 +00:00

chore: promote accumulated staging fixes to main (vitest timeout, postgres host-net, +others) #99

Ghost referenced this issue from a commit 2026-05-08 02:19:01 +00:00

fix(ci): port 3 verified-green CI fixes from staging to main

claude-ceo-assistant referenced this pull request 2026-05-08 02:19:29 +00:00

fix(ci): port 3 verified-green CI fixes from staging to main (cherry-pick) #101

claude-ceo-assistant referenced this issue from a commit 2026-05-08 02:46:41 +00:00

chore: promote accumulated staging fixes to main (vitest, postgres, e2e-api, eic, plugins) (#99)

Sign in to join this conversation.

Reviewers

No reviewers

Ghost

Labels

Clear labels   

tier:high

High risk per dev-sop §SOP-6 — ceo only, 24h cooldown

  

tier:low

Low risk per dev-sop §SOP-6 — engineers/managers/ceo can approve

  

tier:medium

Medium risk per dev-sop §SOP-6 — managers/ceo can approve

No Label

tier:high

tier:low

tier:medium

Milestone

Clear milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

No Assignees

1 Participants

Notifications

Subscribe

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: molecule-ai/molecule-core#97

No description provided.