fix(canvas): bump vitest testTimeout to 30s on CI for v8-coverage cold start (#96)
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (go) (pull_request) Successful in 1s
CodeQL / Analyze (${{ matrix.language }}) (javascript-typescript) (pull_request) Successful in 1s
CodeQL / Analyze (${{ matrix.language }}) (python) (pull_request) Successful in 1s
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 13s
Retarget main PRs to staging / Retarget to staging (pull_request) Successful in 9s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 10s
Harness Replays / detect-changes (pull_request) Successful in 11s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 12s
E2E API Smoke Test / detect-changes (pull_request) Successful in 12s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 11s
CI / Detect changes (pull_request) Successful in 14s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 12s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 4s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 4s
CI / Platform (Go) (pull_request) Successful in 4s
CI / Python Lint & Test (pull_request) Successful in 4s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 3s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 4s
Harness Replays / Harness Replays (pull_request) Failing after 43s
CI / Canvas (Next.js) (pull_request) Successful in 3m44s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 4m35s
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (go) (pull_request) Successful in 1s
CodeQL / Analyze (${{ matrix.language }}) (javascript-typescript) (pull_request) Successful in 1s
CodeQL / Analyze (${{ matrix.language }}) (python) (pull_request) Successful in 1s
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 13s
Retarget main PRs to staging / Retarget to staging (pull_request) Successful in 9s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 10s
Harness Replays / detect-changes (pull_request) Successful in 11s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 12s
E2E API Smoke Test / detect-changes (pull_request) Successful in 12s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 11s
CI / Detect changes (pull_request) Successful in 14s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 12s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 4s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 4s
CI / Platform (Go) (pull_request) Successful in 4s
CI / Python Lint & Test (pull_request) Successful in 4s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 3s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 4s
Harness Replays / Harness Replays (pull_request) Failing after 43s
CI / Canvas (Next.js) (pull_request) Successful in 3m44s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 4m35s
Class A red sweep — 3 first-tests timing out at the 5000ms default on the self-hosted Gitea Actions Docker runner across 4 unrelated PRs (#82, #81, #54, #53). The PRs share zero canvas/ surface — same 3 tests, same cold-start signature, same shape on every run. Root cause: `npx vitest run --coverage` cold-start cost (v8 coverage instrumentation init + JSDOM bootstrap + heavy @/components/* and @/lib/* import + first React render) consumes 5-7 seconds for the first synchronous test in a heavyweight test file. Empirically: ActivityTab "renders all 7 filter options" 5230ms (FAIL) CreateWorkspaceDialog "opens the dialog ..." 6453ms (FAIL) ConfigTab.provider "PUTs the new provider on Save" 5605ms (FAIL) vs subsequent tests in the same files at 100-1500ms each. The component code is correct (e.g. ActivityTab.FILTERS has 7 entries matching the test). 1407 tests pass locally with --coverage in 9-15s; CI runs at 200s under the same flag — the gap is import/transform/environment overhead, not test logic. Fix: CI-conditional `testTimeout: process.env.CI ? 30000 : 5000` in canvas/vitest.config.ts. Local-dev sensitivity to genuine waitFor races preserved; CI gets ~5x headroom over the worst observed first-test (6453ms). Same shape Vitest documents at <https://vitest.dev/config/testtimeout> and <https://vitest.dev/guide/coverage#profiling-test-performance>. Verification: - Local: 5x runs of the 3 failing test files, all 74 tests green (process.env.CI unset → 5000ms applies). - Local: 7s sleep probe FAILS at 5000ms default and PASSES under CI=true → ternary takes effect as written. - Local: full canvas suite under CI=true with --coverage: "Test Files 98 passed (98) | Tests 1407 passed (1407)". Closes #96. Refs: #82, #81, #54, #53. Hostile self-review (3 weakest spots): 1. 30000ms is a guess, not a measurement. Mitigation: vitest still emits per-test duration; a real 25s+ test will surface as a duration regression and we dial down. 2. Doesn't fix the Docker-runner-overhead root-root-cause. True. That is a multi-week perf project. The right trade today is unblocking 4 PRs from this single class. 3. Local-default of 5000ms means a real 8s race that flies on CI's 30000ms could pass without local sensitivity. Mitigation: dev-time waitFor races are caught at the per-test level; suite-level cold- start is the only legitimate >5s case here. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
068c968206
commit
da1a5af7a4
@ -7,6 +7,32 @@ export default defineConfig({
|
||||
test: {
|
||||
environment: 'node',
|
||||
exclude: ['e2e/**', 'node_modules/**', '**/dist/**'],
|
||||
// CI-conditional test timeout (issue #96).
|
||||
//
|
||||
// Vitest's 5000ms default is too tight for the first test in any
|
||||
// file under our CI shape: `npx vitest run --coverage` on the
|
||||
// self-hosted Gitea Actions Docker runner. The cold-start cost
|
||||
// (v8 coverage instrumentation init + JSDOM bootstrap + module-
|
||||
// graph import for @/components/* and @/lib/* + first React
|
||||
// render) consistently consumes 5-7 seconds for the first
|
||||
// synchronous test in heavyweight component files
|
||||
// (ActivityTab.test.tsx, CreateWorkspaceDialog.test.tsx,
|
||||
// ConfigTab.provider.test.tsx) — even though every subsequent
|
||||
// test in the same file completes in 100-1500ms.
|
||||
//
|
||||
// Empirically the worst observed first-test was 6453ms in a
|
||||
// single file (CreateWorkspaceDialog). 30000ms gives ~5x
|
||||
// headroom over that on CI; we still keep 5000ms locally so
|
||||
// genuine waitFor races / hung promises stay sensitive in dev.
|
||||
//
|
||||
// Same vitest pattern documented at:
|
||||
// https://vitest.dev/config/testtimeout
|
||||
// https://vitest.dev/guide/coverage#profiling-test-performance
|
||||
//
|
||||
// Per-test duration is still emitted to the CI log; if a test
|
||||
// ever silently approaches 25-30s under this raised ceiling that
|
||||
// will surface as a duration regression and we revisit.
|
||||
testTimeout: process.env.CI ? 30000 : 5000,
|
||||
// Coverage is instrumented but NOT yet a CI gate — first land
|
||||
// observability so we can see the baseline, then dial in
|
||||
// thresholds + a hard gate in a follow-up PR (#1815). Today's
|
||||
|
||||
Loading…
Reference in New Issue
Block a user