test(e2e): forbid dev token path in staging peer visibility #1650

Merged

hongming merged 1 commits from fix/staging-token-diagnostic into main 2026-05-21 20:26:34 +00:00

Conversation 0 Commits 1 Files Changed 3

+58 -11

hongming commented 2026-05-21 20:22:21 +00:00

Owner

Copy Link

Copy Source

What

Removes the dev-only /admin/workspaces/:id/test-token fallback from staging peer-visibility E2E and adds a focused staging token diagnostic for hermes vs claude-code.

Why

Staging/production E2E must prove production behavior. /admin/workspaces/:id/test-token is intentionally disabled in production-like envs, so staging tests must use the production-safe admin route POST /admin/workspaces/:id/tokens or fail.

The diagnostic run also proved the original failure was not Hermes-specific: both Hermes and Claude Code failed on the stale staging tenant image, then both passed after staging CP was pointed at platform-tenant:staging-latest.

Brief-falsification log

[H1] Hermes-specific token issue.
Verification: Ran PV_RUNTIMES='hermes claude-code' bash tests/e2e/test_peer_visibility_token_mint_staging.sh against staging.
Result: falsified. Both Hermes and Claude Code failed when staging pulled stale build a93c4ce.

[H2] Shared auth route missing because tenant image was stale.
Verification: Fresh tenant buildinfo showed a93c4ce, which predates 07457ad fix(core): add admin workspace token mint route; staging ECR latest pointed to staging-a93c4ce while staging-latest pointed to current c3806cd.
Result: supported. After switching Railway staging TENANT_IMAGE/STAGING_TENANT_IMAGE to :staging-latest and redeploying CP, fresh tenant buildinfo showed c3806cd and token diagnostic passed for both Hermes and Claude Code.

Comprehensive testing performed

• bash -n tests/e2e/test_peer_visibility_mcp_staging.sh && bash -n tests/e2e/test_peer_visibility_token_mint_staging.sh && bash -n tests/e2e/test_peer_visibility_mcp_local.sh
• if rg -n '/admin/workspaces/.*/test-token|test-token' tests/e2e/test_*staging*.sh; then exit 1; fi -> no matches
• Live staging diagnostic before env fix: tenant buildinfo a93c4ce, both hermes and claude-code failed POST /admin/workspaces/:id/tokens with Next.js 404
• Live staging diagnostic after env fix + CP redeploy: tenant buildinfo c3806cd, token diagnostic passed for hermes claude-code

Local-postgres E2E run

N/A for this PR: staging E2E harness change plus live staging diagnostic. Local peer-visibility script syntax was checked; no local platform behavior changed.

Staging-smoke verified or pending

Verified with PV_RUNTIMES='hermes claude-code' bash tests/e2e/test_peer_visibility_token_mint_staging.sh against https://staging-api.moleculesai.app. Result: passed after staging CP env moved to platform-tenant:staging-latest.

Root-cause not symptom

Root cause was staging tenant image tag drift: fresh staging tenants used stale ECR tag latest (a93c4ce) instead of SSOT-published staging-latest (c3806cd), so the production-safe admin token route did not exist in the tenant image.

Five-Axis review walked

Correctness: staging tests now use only production-safe token route and classify all requested runtimes before failing. Readability: diagnostic wrapper is intentionally thin. Architecture: keeps the full MCP assertion shared; only adds a stop-after-token mode. Security: removes dev-only test-token use from staging. Performance: diagnostic mode exits before workspace online/MCP checks.

No backwards-compat shim / dead code added

Yes. No compatibility shim; the dev-only fallback was removed from staging tests.

Memory/saved-feedback consulted

No task-specific saved memory was used for this change. Applied current AGENTS/SOP and live Gitea/Railway/ECR evidence.

Verification

• bash -n tests/e2e/test_peer_visibility_mcp_staging.sh && bash -n tests/e2e/test_peer_visibility_token_mint_staging.sh && bash -n tests/e2e/test_peer_visibility_mcp_local.sh -> staging-no-dev-token-ok
• Staging live token diagnostic after env fix -> ✅ token diagnostic passed for runtimes: hermes claude-code

Coverage ledger

file/function	branch/condition	test name	red/green evidence	drift caught
tests/e2e/test_peer_visibility_mcp_staging.sh	managed runtime create returns no auth_token; admin route must mint	test_peer_visibility_token_mint_staging.sh	Red: both Hermes and Claude Code got Next.js 404 on stale image; Green: both minted tokens on c3806cd	Stale tenant image missing production-safe admin token route
.gitea/workflows/e2e-peer-visibility.yml::pr-validate	staging script references dev-only test-token	PR validation grep	Green: rg found no test-token in test_*staging*.sh	Dev-only token fallback drifting back into staging E2E
tests/e2e/test_peer_visibility_token_mint_staging.sh	runtime classification before full MCP wait	live staging diagnostic	Green: reports Hermes and Claude Code token acquisition independently	False attribution of shared auth route failures to Hermes

Idempotency notes

The diagnostic uses the same scoped throwaway org teardown as the full staging peer-visibility gate. No cluster-wide cleanup.

Loki query

N/A — route/image drift was proven through tenant buildinfo, Gitea commit history, ECR tags, and live E2E output.

Tier

tier:low

## What Removes the dev-only `/admin/workspaces/:id/test-token` fallback from staging peer-visibility E2E and adds a focused staging token diagnostic for `hermes` vs `claude-code`. ## Why Staging/production E2E must prove production behavior. `/admin/workspaces/:id/test-token` is intentionally disabled in production-like envs, so staging tests must use the production-safe admin route `POST /admin/workspaces/:id/tokens` or fail. The diagnostic run also proved the original failure was not Hermes-specific: both Hermes and Claude Code failed on the stale staging tenant image, then both passed after staging CP was pointed at `platform-tenant:staging-latest`. ## Brief-falsification log [H1] Hermes-specific token issue. Verification: Ran `PV_RUNTIMES='hermes claude-code' bash tests/e2e/test_peer_visibility_token_mint_staging.sh` against staging. Result: falsified. Both Hermes and Claude Code failed when staging pulled stale build `a93c4ce`. [H2] Shared auth route missing because tenant image was stale. Verification: Fresh tenant buildinfo showed `a93c4ce`, which predates `07457ad fix(core): add admin workspace token mint route`; staging ECR `latest` pointed to `staging-a93c4ce` while `staging-latest` pointed to current `c3806cd`. Result: supported. After switching Railway staging `TENANT_IMAGE`/`STAGING_TENANT_IMAGE` to `:staging-latest` and redeploying CP, fresh tenant buildinfo showed `c3806cd` and token diagnostic passed for both Hermes and Claude Code. ## Comprehensive testing performed - `bash -n tests/e2e/test_peer_visibility_mcp_staging.sh && bash -n tests/e2e/test_peer_visibility_token_mint_staging.sh && bash -n tests/e2e/test_peer_visibility_mcp_local.sh` - `if rg -n '/admin/workspaces/.*/test-token|test-token' tests/e2e/test_*staging*.sh; then exit 1; fi` -> no matches - Live staging diagnostic before env fix: tenant buildinfo `a93c4ce`, both `hermes` and `claude-code` failed `POST /admin/workspaces/:id/tokens` with Next.js 404 - Live staging diagnostic after env fix + CP redeploy: tenant buildinfo `c3806cd`, token diagnostic passed for `hermes claude-code` ## Local-postgres E2E run N/A for this PR: staging E2E harness change plus live staging diagnostic. Local peer-visibility script syntax was checked; no local platform behavior changed. ## Staging-smoke verified or pending Verified with `PV_RUNTIMES='hermes claude-code' bash tests/e2e/test_peer_visibility_token_mint_staging.sh` against `https://staging-api.moleculesai.app`. Result: passed after staging CP env moved to `platform-tenant:staging-latest`. ## Root-cause not symptom Root cause was staging tenant image tag drift: fresh staging tenants used stale ECR tag `latest` (`a93c4ce`) instead of SSOT-published `staging-latest` (`c3806cd`), so the production-safe admin token route did not exist in the tenant image. ## Five-Axis review walked Correctness: staging tests now use only production-safe token route and classify all requested runtimes before failing. Readability: diagnostic wrapper is intentionally thin. Architecture: keeps the full MCP assertion shared; only adds a stop-after-token mode. Security: removes dev-only test-token use from staging. Performance: diagnostic mode exits before workspace online/MCP checks. ## No backwards-compat shim / dead code added Yes. No compatibility shim; the dev-only fallback was removed from staging tests. ## Memory/saved-feedback consulted No task-specific saved memory was used for this change. Applied current AGENTS/SOP and live Gitea/Railway/ECR evidence. ## Verification - `bash -n tests/e2e/test_peer_visibility_mcp_staging.sh && bash -n tests/e2e/test_peer_visibility_token_mint_staging.sh && bash -n tests/e2e/test_peer_visibility_mcp_local.sh` -> `staging-no-dev-token-ok` - Staging live token diagnostic after env fix -> `✅ token diagnostic passed for runtimes: hermes claude-code` ## Coverage ledger | file/function | branch/condition | test name | red/green evidence | drift caught | |---|---|---|---|---| | `tests/e2e/test_peer_visibility_mcp_staging.sh` | managed runtime create returns no `auth_token`; admin route must mint | `test_peer_visibility_token_mint_staging.sh` | Red: both Hermes and Claude Code got Next.js 404 on stale image; Green: both minted tokens on `c3806cd` | Stale tenant image missing production-safe admin token route | | `.gitea/workflows/e2e-peer-visibility.yml::pr-validate` | staging script references dev-only `test-token` | PR validation grep | Green: `rg` found no `test-token` in `test_*staging*.sh` | Dev-only token fallback drifting back into staging E2E | | `tests/e2e/test_peer_visibility_token_mint_staging.sh` | runtime classification before full MCP wait | live staging diagnostic | Green: reports Hermes and Claude Code token acquisition independently | False attribution of shared auth route failures to Hermes | ## Idempotency notes The diagnostic uses the same scoped throwaway org teardown as the full staging peer-visibility gate. No cluster-wide cleanup. ## Loki query N/A — route/image drift was proven through tenant buildinfo, Gitea commit history, ECR tags, and live E2E output. ## Tier tier:low

hongming added 1 commit 2026-05-21 20:22:21 +00:00

test(e2e): forbid dev token path in staging peer visibility 119743d0de

hongming merged commit ff2557d899 into main 2026-05-21 20:26:34 +00:00

hongming referenced this issue from a commit 2026-05-21 20:26:34 +00:00

Merge pull request 'test(e2e): forbid dev token path in staging peer visibility' (#1650) from fix/staging-token-diagnostic into main

hongming referenced this pull request 2026-05-21 20:31:50 +00:00

[main-red] molecule-ai/molecule-core: def18f28fa #1638

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

Clear labels

area/ci

CI/CD pipeline issues

do-not-auto-merge

Opt out of autonomous merge-queue merging

kind/infrastructure

Infrastructure-related issues

merge-queue

Ready for serialized Gitea merge queue

merge-queue-hold

Temporarily hold PR in merge queue

platform/go

Go platform test issues

release-blocker

Blocks the staging→main promotion / a release

release-test

security

test-label-sre

tier:high

High risk per dev-sop §SOP-6 — ceo only, 24h cooldown

tier:low

Low risk per dev-sop §SOP-6 — engineers/managers/ceo can approve

tier:medium

Medium risk per dev-sop §SOP-6 — managers/ceo can approve

triage-test

test

wip

Work in progress — do not auto-merge

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

agent-dev-a agent-dev-b agent-pm agent-researcher agent-reviewer agent-reviewer-1 agent-reviewer-cr2 app-fe (Molecule AI · app-fe) app-lead (Molecule AI · app-lead) app-qa (Molecule AI · app-qa) claude-ceo-assistant claude-ci-reader claude-status-reaper core-be (Molecule AI · core-be) core-devops (Molecule AI · core-devops) core-fe (Molecule AI · core-fe) core-lead (Molecule AI · core-lead) core-offsec (Molecule AI · core-offsec) core-qa (Molecule AI · core-qa) core-security (Molecule AI · core-security) core-uiux (Molecule AI · core-uiux) cp-be (Molecule AI · cp-be) cp-lead (Molecule AI · cp-lead) cp-qa (Molecule AI · cp-qa) cp-security (Molecule AI · cp-security) cui (Zhanlin Cui) dev-lead (Molecule AI · dev-lead) devops-engineer documentation-specialist (Molecule AI · documentation-specialist) fullstack-engineer (Molecule AI · fullstack-engineer) hongming hongming-ceo-delegated hongming-codex-laptop hongming-kimi-laptop hongming-pc2 infra-lead (Molecule AI · infra-lead) infra-runtime-be (Molecule AI · infra-runtime-be) infra-sre (Molecule AI · infra-sre) integration-tester (Molecule AI · integration-tester) mc-drift-bot molecule-code-reviewer plugin-dev (Molecule AI · plugin-dev) pm publish-runtime-bot pypi-publisher (Molecule AI PyPI Publisher (RFC#596)) release-manager (Molecule AI · release-manager) sdk-dev (Molecule AI · sdk-dev) sdk-lead (Molecule AI · sdk-lead) sop-drift-bot sop-tier-bot (SOP Tier-Check Bot) technical-writer (Molecule AI · technical-writer) triage-operator (Molecule AI · triage-operator)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: molecule-ai/molecule-core#1650