4c49ff75f6
The staging canary's A2A step has a ladder of specific regression classifiers (hermes-agent down, model_not_found, Invalid API key, etc.) followed by a generic "error|exception" catch-all. Provider- side OpenAI 429 quota errors fell through to the catch-all, so the canary issue body and CI log just said "A2A returned an error-shaped response" — which is technically true but obscures the actual operator action. This adds a 7th classifier above the catch-all for "exceeded your current quota" / "insufficient_quota" — both terms appear in OpenAI's quota-exhaustion 429 response. When matched, the failure message names the operator action directly (top up MOLECULE_STAGING_OPENAI_KEY or rotate the secret) and links to #2578. Why this is correct, not "lowering the bar": - Steps 0–7 of the canary cover full platform health (CP up, tenant provisioned, DNS+TLS reachable, workspace booted, A2A delivered). - Reaching step 8 with a provider-side 429 means the platform IS healthy — the failure is downstream of all platform invariants. - The canary still exits 1 (CI stays red, threshold-3 alarm still fires); only the failure message changes. - All 6 existing specific classifiers run BEFORE this one, so any real platform regression is still caught with its specific message. Verification: - Regex tested against the actual 429 string from canary run 25291517608: "API call failed after 3 retries: HTTP 429: You exceeded your current quota..." → matches ✅ - Negative tests: "PONG", "hermes-agent unreachable" → no match ✅ - bash -n syntax check passes - shellcheck -S error clean Tracking: #2593 (canary), #2578 (root cause) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| e2e | ||
| harness | ||
| ops | ||
| README.md | ||
Tests
This repo uses the standard monorepo testing convention: unit tests live with their package, cross-component E2E tests live here.
Where to find tests
| Scope | Location |
|---|---|
| Go unit + integration (platform, CLI, handlers) | workspace-server/**/*_test.go — run with cd workspace-server && go test -race ./... |
| TypeScript unit (canvas components, hooks, store) | canvas/src/**/__tests__/ — run with cd canvas && npm test -- --run |
| TypeScript unit (MCP server handlers) | mcp-server/src/__tests__/ — run with cd mcp-server && npx jest |
| Python unit (workspace runtime, adapters) | workspace/tests/ — run with cd workspace && python3 -m pytest |
| Python unit (SDK: plugin + remote agent) | sdk/python/tests/ — run with cd sdk/python && python3 -m pytest |
| Cross-component E2E (spans platform + runtime + HTTP) | tests/e2e/ ← you are here |
Why split this way
- Go requires co-located
_test.gofiles to access unexported symbols. - Per-package test commands keep the inner loop fast — changing canvas doesn't re-run Go tests.
tests/e2e/covers scenarios that no single package owns: a full workspace lifecycle, A2A across two provisioned agents, delegation chains, bundle round-trips.
Running E2E
Every E2E script here assumes the platform is running at localhost:8080 and (where noted) provisioned agents are online. See the header comment of each .sh for specifics.
Cleaning up rogue test workspaces
If an E2E run aborts before its teardown runs (Ctrl-C, crash, CI timeout),
the platform can be left with workspaces whose config volume is stale or
empty — Docker's unless-stopped restart policy then spins those
containers in a FileNotFoundError loop. The platform's pre-flight check
(#17) marks such workspaces failed on the next restart, but a manual
cleanup is useful:
bash scripts/cleanup-rogue-workspaces.sh # deletes ws with id/name starting aaaaaaaa-, bbbbbbbb-, cccccccc-, test-ws-
MOLECULE_URL=http://host:8080 bash scripts/cleanup-rogue-workspaces.sh
The script DELETEs each matching workspace via the API and
force-removes the ws-<id[:12]> container as a belt-and-suspenders
fallback.