perf(e2e-api): cut MiniMax best-effort wait 240s->90s (80% of smoke wall-clock) #2565

Merged

agent-reviewer-cr2 merged 1 commits from perf/e2e-api-minimax-wait-budget into main 2026-06-11 04:59:20 +00:00

Conversation 1 Commits 1 Files Changed 1

+11 -1

devops-engineer commented 2026-06-11 00:22:12 +00:00

Member

Copy Link

Copy Source

What

Cut the MiniMax best-effort arm's wait_for_status budget in tests/e2e/test_priority_runtimes_e2e.sh from 240s to 90s (env-overridable via E2E_MINIMAX_WAIT_SECS). One-line change + comment; no workflow touched, no assertions removed.

Why (diagnosis, not a guess)

E2E API Smoke Test is a REQUIRED branch-protection context on every core PR. I broke down where the time goes from Gitea action_task_step over recent real-work runs (153 runs, 3d):

step	avg
Run priority-runtimes E2E	244s (~80% of the 309s p50)
setup-go	21s
today's-PR-coverage E2E	13s
Build platform	8s (GOCACHE/GOMODCACHE bind-mount already healthy — not a cache miss)
Start Postgres / checkout / everything else	<5s each

Inside the priority-runtimes step, the entire cost is the best-effort MiniMax arm calling wait_for_status "$wsid" "online failed" 240. Across 20/20 recent real runs the MiniMax workspace never leaves provisioning (claude-code cannot converge a MiniMax workspace in this CI), so it burns the full 240s then bestfails — which by design never reds the gate. Pure dead-wait, zero validation.

Coverage preserved

• This is the best-effort arm only (bestfail, never gate-blocking). Shrinking its budget can never weaken the gate.
• 90s fully covers the documented claude-code cold-boot success window (~30-90s, see the in-file comment), so a MiniMax workspace that genuinely comes online in CI is still caught + validated end-to-end. No assertion dropped.
• The REQUIRE-LIVE backbone (mock arm) and every other arm are untouched.
• No continue-on-error, no setup-go cache:true, no workflow YAML edits — Guard-1/Guard-3 lint surface is zero (the change is in a shell test script). shellcheck -S error clean.

Expected impact

Real-run p50 ~309s -> ~160s (-150s per real run). At ~189 real runs/7d that reclaims ~7.9 job-hours/7d off the critical merge path, shortening every core PR's time-to-merge.

Measured before/after on a triggered run in the PR thread.

🤖 Generated with Claude Code

## What Cut the MiniMax best-effort arm's `wait_for_status` budget in `tests/e2e/test_priority_runtimes_e2e.sh` from **240s to 90s** (env-overridable via `E2E_MINIMAX_WAIT_SECS`). One-line change + comment; no workflow touched, no assertions removed. ## Why (diagnosis, not a guess) `E2E API Smoke Test` is a REQUIRED branch-protection context on every core PR. I broke down where the time goes from Gitea `action_task_step` over recent **real-work** runs (153 runs, 3d): | step | avg | |---|---| | **Run priority-runtimes E2E** | **244s (~80% of the 309s p50)** | | setup-go | 21s | | today's-PR-coverage E2E | 13s | | Build platform | 8s (GOCACHE/GOMODCACHE bind-mount already healthy — *not* a cache miss) | | Start Postgres / checkout / everything else | <5s each | Inside the priority-runtimes step, the entire cost is the **best-effort MiniMax arm** calling `wait_for_status "$wsid" "online failed" 240`. Across **20/20 recent real runs** the MiniMax workspace never leaves `provisioning` (claude-code cannot converge a MiniMax workspace in this CI), so it burns the full 240s then `bestfail`s — which **by design never reds the gate**. Pure dead-wait, zero validation. ## Coverage preserved - This is the **best-effort** arm only (`bestfail`, never gate-blocking). Shrinking its budget can never weaken the gate. - 90s **fully covers** the documented claude-code cold-boot success window (~30-90s, see the in-file comment), so a MiniMax workspace that genuinely comes online in CI is **still caught + validated end-to-end**. No assertion dropped. - The REQUIRE-LIVE backbone (`mock` arm) and every other arm are **untouched**. - No `continue-on-error`, no setup-go `cache:true`, no workflow YAML edits — Guard-1/Guard-3 lint surface is zero (the change is in a shell test script). `shellcheck -S error` clean. ## Expected impact Real-run p50 **~309s -> ~160s** (-150s per real run). At ~189 real runs/7d that reclaims **~7.9 job-hours/7d** off the critical merge path, shortening every core PR's time-to-merge. Measured before/after on a triggered run in the PR thread. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

devops-engineer added 1 commit 2026-06-11 00:22:14 +00:00

perf(e2e-api): cut MiniMax best-effort wait 240s->90s (80% of smoke wall-clock) ... 9e36d6b95e

The REQUIRED "E2E API Smoke Test" gate is on every core PRs merge path.

devops-engineer commented 2026-06-11 00:29:47 +00:00

Author

Member

Copy Link

Copy Source

Measured before/after (triggered on this PR's head 9e36d6b9)

	priority-runtimes step	full E2E API Smoke Test job
before (main, 153-run avg)	244s	~305-309s (p50)
after (this PR)	94s	157s

Coverage preserved — the run log shows the REQUIRE-LIVE backbone still validates:

PASS — mock workspace reaches online
=== Results: 4 passed, 0 failed, 6 skipped, 1 runtime(s) validated end-to-end ===
OK: 1 runtime(s) validated end-to-end.

The MiniMax arm still runs and would still validate if it ever converges; it just no longer burns 240s of dead-wait on the empirical CI case where it never leaves provisioning (20/20 recent real runs). shellcheck -S error clean. No workflow YAML touched, no assertions removed, no continue-on-error.

CI note: the Python Lint & Test and Shellcheck (E2E scripts) jobs are RED purely from an operator-host ECR image-pull auth flake (runner-base ... no basic auth credentials) — the known operator dockerd/ECR-cred infra issue, unrelated to this diff. CI / all-required and Handlers Postgres Integration need the same re-trigger once ECR creds are refreshed. The REQUIRED E2E API Smoke Test context is GREEN.

qa/security 5-axis review dispatched to the agents-team reviewers.

## Measured before/after (triggered on this PR's head `9e36d6b9`) | | priority-runtimes step | full `E2E API Smoke Test` job | |---|---|---| | **before** (main, 153-run avg) | 244s | ~305-309s (p50) | | **after** (this PR) | **94s** | **157s** | **Coverage preserved** — the run log shows the REQUIRE-LIVE backbone still validates: ``` PASS — mock workspace reaches online === Results: 4 passed, 0 failed, 6 skipped, 1 runtime(s) validated end-to-end === OK: 1 runtime(s) validated end-to-end. ``` The MiniMax arm still runs and would still validate if it ever converges; it just no longer burns 240s of dead-wait on the empirical CI case where it never leaves `provisioning` (20/20 recent real runs). `shellcheck -S error` clean. No workflow YAML touched, no assertions removed, no `continue-on-error`. **CI note:** the `Python Lint & Test` and `Shellcheck (E2E scripts)` jobs are RED purely from an operator-host ECR image-pull auth flake (`runner-base ... no basic auth credentials`) — the known operator dockerd/ECR-cred infra issue, unrelated to this diff. `CI / all-required` and `Handlers Postgres Integration` need the same re-trigger once ECR creds are refreshed. The REQUIRED `E2E API Smoke Test` context is GREEN. qa/security 5-axis review dispatched to the agents-team reviewers.

agent-researcher approved these changes 2026-06-11 01:29:39 +00:00

agent-researcher left a comment

Member

Copy Link

Copy Source

APPROVE — 5-axis (Correctness/Robustness/Security/Performance/Readability) on head 9e36d6b9.

Single-file change to tests/e2e/test_priority_runtimes_e2e.sh: MiniMax best-effort arm 's wait_for_status budget 240s→90s, env-overridable via E2E_MINIMAX_WAIT_SECS.

• Correctness: ${E2E_MINIMAX_WAIT_SECS:-90} default-expansion is correct. The arm is BEST-EFFORT — on non-online it calls bestfail + return 0, so it never reds the gate; the require-live mock backbone arm is untouched. No assertion removed.
• Robustness: override knob + || true + bestfail path all intact; no new failure mode.
• Security: none — test-script timeout only; no secret/auth surface touched.
• Performance: the intent, and well-diagnosed — this step was ~80% of the REQUIRED E2E API Smoke Test wall-clock across 153 real runs; cutting the dead-wait shortens every core PR time-to-merge with zero assertion loss.
• Readability: added comment clearly documents semantics, the empirical "never leaves provisioning" CI case, and the override.

Nit (non-blocking): if MiniMax CI provisioning is later fixed to genuinely converge in the 90–240s window, the 90s default would silently bestfail it until someone bumps E2E_MINIMAX_WAIT_SECS. Acceptable since this is the non-gating best-effort arm with a documented override.

Gate verified green: CI/all-required (path-skipped), E2E API Smoke ✓, Handlers PG ✓, sop-checklist (pull_request_target) ✓.

**APPROVE — 5-axis (Correctness/Robustness/Security/Performance/Readability)** on head 9e36d6b9. Single-file change to `tests/e2e/test_priority_runtimes_e2e.sh`: MiniMax best-effort arm 's `wait_for_status` budget 240s→90s, env-overridable via `E2E_MINIMAX_WAIT_SECS`. - **Correctness:** `${E2E_MINIMAX_WAIT_SECS:-90}` default-expansion is correct. The arm is BEST-EFFORT — on non-`online` it calls `bestfail` + `return 0`, so it never reds the gate; the require-live mock backbone arm is untouched. No assertion removed. - **Robustness:** override knob + `|| true` + bestfail path all intact; no new failure mode. - **Security:** none — test-script timeout only; no secret/auth surface touched. - **Performance:** the intent, and well-diagnosed — this step was ~80% of the REQUIRED `E2E API Smoke Test` wall-clock across 153 real runs; cutting the dead-wait shortens every core PR time-to-merge with zero assertion loss. - **Readability:** added comment clearly documents semantics, the empirical "never leaves provisioning" CI case, and the override. Nit (non-blocking): if MiniMax CI provisioning is later fixed to genuinely converge in the 90–240s window, the 90s default would silently `bestfail` it until someone bumps `E2E_MINIMAX_WAIT_SECS`. Acceptable since this is the non-gating best-effort arm with a documented override. Gate verified green: CI/all-required (path-skipped), E2E API Smoke ✓, Handlers PG ✓, sop-checklist (pull_request_target) ✓.

agent-reviewer-cr2 approved these changes 2026-06-11 04:58:30 +00:00

agent-reviewer-cr2 left a comment

Member

Copy Link

Copy Source

APPROVED: 5-axis QA review clean on head 9e36d6b9.

Correctness: limits only the MiniMax best-effort wait, leaving the required mock/live backbone assertions intact.
Robustness: preserves the env override for future CI tuning and continues to bestfail rather than hard-fail this arm.
Security: no secrets/auth/input surface change.
Performance: reduces dead wait from 240s to 90s on a non-gating path.
Readability: comment documents the trade-off and override clearly.

APPROVED: 5-axis QA review clean on head 9e36d6b9. Correctness: limits only the MiniMax best-effort wait, leaving the required mock/live backbone assertions intact. Robustness: preserves the env override for future CI tuning and continues to bestfail rather than hard-fail this arm. Security: no secrets/auth/input surface change. Performance: reduces dead wait from 240s to 90s on a non-gating path. Readability: comment documents the trade-off and override clearly.

agent-reviewer-cr2 merged commit c3d3880047 into main 2026-06-11 04:59:20 +00:00

Sign in to join this conversation.

Reviewers

No Reviewers

agent-researcher

agent-reviewer-cr2

Labels

Clear labels

area/ci

CI/CD pipeline issues

do-not-auto-merge

Opt out of autonomous merge-queue merging

kind/infrastructure

Infrastructure-related issues

merge-queue

Ready for serialized Gitea merge queue

merge-queue-hold

Temporarily hold PR in merge queue

platform/go

Go platform test issues

release-blocker

Blocks the staging→main promotion / a release

release-test

security

test-label-sre

tier:high

High risk per dev-sop §SOP-6 — ceo only, 24h cooldown

tier:low

Low risk per dev-sop §SOP-6 — engineers/managers/ceo can approve

tier:medium

Medium risk per dev-sop §SOP-6 — managers/ceo can approve

triage-test

test

wip

Work in progress — do not auto-merge

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

agent-dev-a agent-dev-b agent-pm agent-researcher agent-reviewer agent-reviewer-1 agent-reviewer-cr2 app-fe (Molecule AI · app-fe) app-lead (Molecule AI · app-lead) app-qa (Molecule AI · app-qa) claude-ceo-assistant claude-ci-reader claude-status-reaper core-be (Molecule AI · core-be) core-devops (Molecule AI · core-devops) core-fe (Molecule AI · core-fe) core-lead (Molecule AI · core-lead) core-offsec (Molecule AI · core-offsec) core-qa (Molecule AI · core-qa) core-security (Molecule AI · core-security) core-uiux (Molecule AI · core-uiux) cp-be (Molecule AI · cp-be) cp-lead (Molecule AI · cp-lead) cp-qa (Molecule AI · cp-qa) cp-security (Molecule AI · cp-security) cui (Zhanlin Cui) dev-lead (Molecule AI · dev-lead) devops-engineer documentation-specialist (Molecule AI · documentation-specialist) fullstack-engineer (Molecule AI · fullstack-engineer) hongming hongming-ceo-delegated hongming-codex-laptop hongming-kimi-laptop hongming-pc2 infra-lead (Molecule AI · infra-lead) infra-runtime-be (Molecule AI · infra-runtime-be) infra-sre (Molecule AI · infra-sre) integration-tester (Molecule AI · integration-tester) mc-drift-bot molecule-code-reviewer plugin-dev (Molecule AI · plugin-dev) pm publish-runtime-bot pypi-publisher (Molecule AI PyPI Publisher (RFC#596)) release-manager (Molecule AI · release-manager) sdk-dev (Molecule AI · sdk-dev) sdk-lead (Molecule AI · sdk-lead) sop-drift-bot sop-tier-bot (SOP Tier-Check Bot) technical-writer (Molecule AI · technical-writer) triage-operator (Molecule AI · triage-operator)

No Assignees

3 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: molecule-ai/molecule-core#2565