molecule-ai/molecule-core
49
0
Fork 2
Code Issues 145 Pull Requests 157 Actions 5 Packages Projects Releases Wiki Activity

Staging peer-visibility E2E cannot mint MCP bearer for managed runtimes #1644

New Issue
Open
opened 2026-05-21 18:47:03 +00:00 by hongming · 1 comment
hongming commented 2026-05-21 18:47:03 +00:00
Owner
Copy Link
Copy Source

Observed on molecule-core main merge commit da4b86a159, run 91206 / task 142513. Local peer visibility E2E passed, but staging peer visibility failed while provisioning Hermes because the workspace create response returned no auth_token and both token fallbacks failed: POST /admin/workspaces/:id/tokens returned HTML 404, GET /admin/workspaces/:id/test-token returned {"error":"not found"}. This looks separate from the MCP delegate platform-path fix; the tenant route/image or staging admin-token surface is drifting from the test's expected bearer-mint path. Need determine whether the staging tenant image lacks the admin token route, the route is intentionally disabled in production-shaped tenants, or the E2E should use a different SSOT bearer-mint surface.

Observed on molecule-core main merge commit da4b86a1593d555ecd6282d39db6be29ee047a61, run 91206 / task 142513. Local peer visibility E2E passed, but staging peer visibility failed while provisioning Hermes because the workspace create response returned no auth_token and both token fallbacks failed: POST /admin/workspaces/:id/tokens returned HTML 404, GET /admin/workspaces/:id/test-token returned {"error":"not found"}. This looks separate from the MCP delegate platform-path fix; the tenant route/image or staging admin-token surface is drifting from the test's expected bearer-mint path. Need determine whether the staging tenant image lacks the admin token route, the route is intentionally disabled in production-shaped tenants, or the E2E should use a different SSOT bearer-mint surface.
agent-researcher commented 2026-05-24 07:41:55 +00:00
Member
Copy Link
Copy Source

MECHANISM: Current main still has the peer-visibility staging failure class tracked here, now isolated from the earlier Platform-Go/Handlers-Postgres main break. On main 406d73ff (fix(templates): revert templates.go change from #1781), CI / Platform (Go) and Handlers Postgres Integration are green, but E2E Peer Visibility (literal MCP list_peers) / E2E Peer Visibility (push) is red. The failing path is the staging job in .gitea/workflows/e2e-peer-visibility.yml:298-345: it uses staging CP, verifies admin/LLM keys, then runs tests/e2e/test_peer_visibility_mcp_staging.sh to provision runtimes and call literal MCP list_peers.

EVIDENCE: Gitea status API for main 406d73ff reports CI / Platform (Go) success, CI / all-required success, and Handlers Postgres Integration success, while E2E Peer Visibility (literal MCP list_peers) / E2E Peer Visibility (push) fails after 2m33s at /molecule-ai/molecule-core/actions/runs/84418/jobs/2. This matches the existing #1644 symptom class: local peer visibility can pass, but staging fails before/around managed-runtime bearer minting and literal MCP visibility. Workflow comments at .gitea/workflows/e2e-peer-visibility.yml:28-35 confirm this is intentionally not a registry/proxy check; it exercises the real /workspaces/:id/mcp list_peers call.

RECOMMENDED FIX SHAPE: Responsible repo/files are molecule-ai/molecule-core/.gitea/workflows/e2e-peer-visibility.yml and molecule-ai/molecule-core/tests/e2e/test_peer_visibility_mcp_staging.sh, plus the staging CP admin token/bearer-mint route they call. Infra should inspect run 84418 job 2 logs and confirm whether the failure is still missing/404 token minting versus a later MCP list_peers authorization failure, then align the staging test with the production-safe bearer mint surface rather than weakening the gate.

MECHANISM: Current main still has the peer-visibility staging failure class tracked here, now isolated from the earlier Platform-Go/Handlers-Postgres main break. On main `406d73ff` (`fix(templates): revert templates.go change from #1781`), `CI / Platform (Go)` and `Handlers Postgres Integration` are green, but `E2E Peer Visibility (literal MCP list_peers) / E2E Peer Visibility (push)` is red. The failing path is the staging job in `.gitea/workflows/e2e-peer-visibility.yml:298-345`: it uses staging CP, verifies admin/LLM keys, then runs `tests/e2e/test_peer_visibility_mcp_staging.sh` to provision runtimes and call literal MCP `list_peers`. EVIDENCE: Gitea status API for main `406d73ff` reports `CI / Platform (Go)` success, `CI / all-required` success, and `Handlers Postgres Integration` success, while `E2E Peer Visibility (literal MCP list_peers) / E2E Peer Visibility (push)` fails after 2m33s at `/molecule-ai/molecule-core/actions/runs/84418/jobs/2`. This matches the existing #1644 symptom class: local peer visibility can pass, but staging fails before/around managed-runtime bearer minting and literal MCP visibility. Workflow comments at `.gitea/workflows/e2e-peer-visibility.yml:28-35` confirm this is intentionally not a registry/proxy check; it exercises the real `/workspaces/:id/mcp` list_peers call. RECOMMENDED FIX SHAPE: Responsible repo/files are `molecule-ai/molecule-core/.gitea/workflows/e2e-peer-visibility.yml` and `molecule-ai/molecule-core/tests/e2e/test_peer_visibility_mcp_staging.sh`, plus the staging CP admin token/bearer-mint route they call. Infra should inspect run 84418 job 2 logs and confirm whether the failure is still missing/404 token minting versus a later MCP `list_peers` authorization failure, then align the staging test with the production-safe bearer mint surface rather than weakening the gate.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
area/ci
CI/CD pipeline issues
 kind/infrastructure
Infrastructure-related issues
 merge-queue
Ready for serialized Gitea merge queue
 merge-queue-hold
Temporarily hold PR in merge queue
 platform/go
Go platform test issues
 release-blocker
Blocks the staging→main promotion / a release
 release-test
security
test-label-sre
tier:high
High risk per dev-sop §SOP-6 — ceo only, 24h cooldown
 tier:low
Low risk per dev-sop §SOP-6 — engineers/managers/ceo can approve
 tier:medium
Medium risk per dev-sop §SOP-6 — managers/ceo can approve
 triage-test
test
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
agent-dev-a agent-dev-b agent-pm agent-researcher agent-reviewer app-fe (Molecule AI · app-fe) app-lead (Molecule AI · app-lead) app-qa (Molecule AI · app-qa) claude-ceo-assistant claude-ci-reader claude-status-reaper core-be (Molecule AI · core-be) core-devops (Molecule AI · core-devops) core-fe (Molecule AI · core-fe) core-lead (Molecule AI · core-lead) core-offsec (Molecule AI · core-offsec) core-qa (Molecule AI · core-qa) core-security (Molecule AI · core-security) core-uiux (Molecule AI · core-uiux) cp-be (Molecule AI · cp-be) cp-lead (Molecule AI · cp-lead) cp-qa (Molecule AI · cp-qa) cp-security (Molecule AI · cp-security) cui (Zhanlin Cui) dev-lead (Molecule AI · dev-lead) devops-engineer documentation-specialist (Molecule AI · documentation-specialist) fullstack-engineer (Molecule AI · fullstack-engineer) hongming hongming-ceo-delegated hongming-codex-laptop hongming-kimi-laptop hongming-pc2 infra-lead (Molecule AI · infra-lead) infra-runtime-be (Molecule AI · infra-runtime-be) infra-sre (Molecule AI · infra-sre) integration-tester (Molecule AI · integration-tester) mc-drift-bot molecule-code-reviewer plugin-dev (Molecule AI · plugin-dev) pm publish-runtime-bot pypi-publisher (Molecule AI PyPI Publisher (RFC#596)) release-manager (Molecule AI · release-manager) sdk-dev (Molecule AI · sdk-dev) sdk-lead (Molecule AI · sdk-lead) sop-drift-bot sop-tier-bot (SOP Tier-Check Bot) technical-writer (Molecule AI · technical-writer) triage-operator (Molecule AI · triage-operator)
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: molecule-ai/molecule-core#1644
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?