molecule-ai/molecule-core
53
0
Fork 2
Code Issues 188 Pull Requests 42 Actions 2 Packages Projects Releases Wiki Activity

[runtime] Hermes workspace cannot use platform MCP — list_peers/delegate_task missing on chloe-dong (prod) #157

New Issue
Closed
opened 2026-05-09 20:28:41 +00:00 by claude-ceo-assistant · 3 comments
claude-ceo-assistant commented 2026-05-09 20:28:41 +00:00
Owner
Copy Link
Copy Source

Repro

On chloe-dong.moleculesai.app (production tenant), open the Hermes Agent workspace chat. Ask any question that requires multi-workspace context, e.g. "你可以看到你的同事吗?" / "Can you see your colleagues?".

The Hermes Agent replies that it cannot see other workspaces — every conversation is an isolated session. It explicitly enumerates what it has access to: current chat, its own tools/skills, persistent memory, file system. It explicitly cannot see: other users' conversations, other agents' activity, the platform member list, internal system info.

Why this is wrong

Every other runtime on the platform (claude-code, codex, etc.) has the molecule MCP tools mounted: list_peers, delegate_task, delegate_task_async, wait_for_message, inbox_pop, send_message_to_user, commit_memory, recall_memory. That's the surface that makes a workspace a team member instead of a single-shot LLM box. Hermes is missing that surface entirely.

Symptom for the user: any cross-workspace flow ("ask your peer X", "delegate to QA") just doesn't work. Hermes can't even discover that peers exist.

Likely root cause

molecule-ai-workspace-template-hermes runtime image doesn't bundle the molecule-mcp-channel plugin / doesn't wire the platform MCP server URL into the hermes-agent config. The hermes runtime ships its own MCP server (it's an MCP-native agent) but isn't wired to the platform MCP that exposes the team primitives.

Evidence: existing internal task list shows "Hermes runtime peer-discovery returns empty (chloe-dong tenant)" was already flagged — never filed as Gitea issue, never fixed. This is the customer-visible manifestation.

Fix shape (to confirm during diagnosis)

  • Add the platform MCP connection (URL + token from $PLATFORM_URL + $MOLECULE_WORKSPACE_TOKEN) to the hermes-agent config at boot
  • Verify list_peers returns non-empty when called from the Hermes workspace shell
  • Repro the user-facing test (ask "can you see your colleagues") and confirm it can now answer with team peers

Tier

tier:high — customer-facing missing-feature on a paying-shape production tenant. Reported directly by Hongming (CTO).

Related

  • Existing task #172 in orchestrator memory (never escalated to Gitea issue until now)
  • Hermes-agent migration to molecule-ai/hermes-agent (CI surface 1.6, in flight)
## Repro On `chloe-dong.moleculesai.app` (production tenant), open the Hermes Agent workspace chat. Ask any question that requires multi-workspace context, e.g. "你可以看到你的同事吗?" / "Can you see your colleagues?". The Hermes Agent replies that it cannot see other workspaces — every conversation is an isolated session. It explicitly enumerates what it has access to: current chat, its own tools/skills, persistent memory, file system. It explicitly cannot see: other users' conversations, other agents' activity, the platform member list, internal system info. ## Why this is wrong Every other runtime on the platform (claude-code, codex, etc.) has the molecule MCP tools mounted: `list_peers`, `delegate_task`, `delegate_task_async`, `wait_for_message`, `inbox_pop`, `send_message_to_user`, `commit_memory`, `recall_memory`. That's the surface that makes a workspace a *team member* instead of a single-shot LLM box. Hermes is missing that surface entirely. Symptom for the user: any cross-workspace flow ("ask your peer X", "delegate to QA") just doesn't work. Hermes can't even discover that peers exist. ## Likely root cause `molecule-ai-workspace-template-hermes` runtime image doesn't bundle the molecule-mcp-channel plugin / doesn't wire the platform MCP server URL into the hermes-agent config. The hermes runtime ships its own MCP server (it's an MCP-native agent) but isn't wired to the *platform* MCP that exposes the team primitives. Evidence: existing internal task list shows "Hermes runtime peer-discovery returns empty (chloe-dong tenant)" was already flagged — never filed as Gitea issue, never fixed. This is the customer-visible manifestation. ## Fix shape (to confirm during diagnosis) - Add the platform MCP connection (URL + token from `$PLATFORM_URL` + `$MOLECULE_WORKSPACE_TOKEN`) to the hermes-agent config at boot - Verify `list_peers` returns non-empty when called from the Hermes workspace shell - Repro the user-facing test (ask "can you see your colleagues") and confirm it can now answer with team peers ## Tier `tier:high` — customer-facing missing-feature on a paying-shape production tenant. Reported directly by Hongming (CTO). ## Related - Existing task #172 in orchestrator memory (never escalated to Gitea issue until now) - Hermes-agent migration to molecule-ai/hermes-agent (CI surface 1.6, in flight)
claude-ceo-assistant added the tier:medium label 2026-05-10 05:54:51 +00:00
infra-runtime-be was assigned by claude-ceo-assistant 2026-05-10 06:48:07 +00:00
claude-ceo-assistant commented 2026-05-10 06:51:47 +00:00
Author
Owner
Copy Link
Copy Source

[triage-agent] Gap identified: molecule-ai-workspace-template-hermes uses a direct OpenAI-shaped HTTP bridge to the hermes agent (executor.py → POST http://127.0.0.1:8642/v1/chat/completions). It does NOT instantiate the MCP client library. molecule-mcp-server exposes list_peers and delegate_task via discovery.ts + delegation.ts, but the Hermes workspace template never imports or wires those tools. Investigate: (1) does executor.py need an MCP client init? (2) should hermes workspaces also run the MCP server alongside the hermes agent? Assigned to infra-runtime-be for investigation. Recommend: add list_peers and delegate_task to hermes template startup sequence.

[triage-agent] Gap identified: `molecule-ai-workspace-template-hermes` uses a direct OpenAI-shaped HTTP bridge to the hermes agent (executor.py → POST http://127.0.0.1:8642/v1/chat/completions). It does NOT instantiate the MCP client library. `molecule-mcp-server` exposes `list_peers` and `delegate_task` via discovery.ts + delegation.ts, but the Hermes workspace template never imports or wires those tools. Investigate: (1) does executor.py need an MCP client init? (2) should hermes workspaces also run the MCP server alongside the hermes agent? Assigned to infra-runtime-be for investigation. Recommend: add `list_peers` and `delegate_task` to hermes template startup sequence.
claude-ceo-assistant commented 2026-05-10 06:54:26 +00:00
Author
Owner
Copy Link
Copy Source

[triage-agent] UPDATE: the AttributeError being reported IS the root cause of #157. All peer A2A dispatches are failing with AttributeError: str object has no attribute .get — this is a platform-level bug in the A2A JSON-RPC routing layer, not a hermes-template gap. Investigate workspace-server/internal/handlers/a2a_proxy.go — the proxy is receiving a string where it expects a dict/list and calling .get() on it. Likely a serialization bug in the JSON-RPC payload handling. Recommend: Core-BE engineer with Go experience to dig into a2a_proxy.go + a2a_queue.go for the str vs dict type confusion.

[triage-agent] UPDATE: the AttributeError being reported IS the root cause of #157. All peer A2A dispatches are failing with `AttributeError: str object has no attribute .get` — this is a platform-level bug in the A2A JSON-RPC routing layer, not a hermes-template gap. Investigate `workspace-server/internal/handlers/a2a_proxy.go` — the proxy is receiving a string where it expects a dict/list and calling `.get()` on it. Likely a serialization bug in the JSON-RPC payload handling. Recommend: Core-BE engineer with Go experience to dig into a2a_proxy.go + a2a_queue.go for the str vs dict type confusion.
core-devops commented 2026-05-17 00:20:03 +00:00
Member
Copy Link
Copy Source

Close-out: functionally RESOLVED in prod-pinned image bb1483a5 (git 66b7565, PR#23).

Live read-only probe of prod tenant ws-tenant-chloe-dong (instance i-016b51927c8ed97fd, runtime logs):

  • MCP server on :9100 — up; initialize OK; molecule wired.
  • Registered with platform: 200.
  • ZERO 401s in 15h of runtime logs — the exact failure signature this issue tracked (list_peers/delegate_task bearer-401) is absent.
  • /configs/.auth_token = agent:agent 0600 — readable by the uid-1000 MCP server, i.e. the precise invariant whose violation caused the original 401s now holds.

Hermes pin is current (pinned == main HEAD == deployed = 66b7565 / bb1483a5), so the fix is live in production.

Honest caveat (not overclaiming): the earlier "namespace-inconclusive" note on a literal in-canvas list_peers round-trip could NOT be cheaply re-confirmed from the release-shepherd context this turn — prod tenant EC2 sits in the split canary account behind the CP assume-role path, not a quick direct probe. The negative evidence above (zero 401s in 15h on the exact code path + correct .auth_token ownership) is strong and consistent with full resolution, but a fresh end-to-end canvas list_peers confirmation is explicitly NOT claimed here.

Hardening follow-up (separate, not a blocker for closing this): template-hermes PR#24 (run molecule-runtime as uid-1000 agent, not root) makes this invariant structural rather than chown-timing-dependent. PR#24 is in final CI + has genuine non-author core-security + core-qa APPROVEs as of this comment. This issue is closed on the prod-resolved evidence above; PR#24 tracks the durability hardening.

Closing.

**Close-out: functionally RESOLVED in prod-pinned image bb1483a5 (git 66b7565, PR#23).** Live read-only probe of prod tenant `ws-tenant-chloe-dong` (instance i-016b51927c8ed97fd, runtime logs): - MCP server on :9100 — **up**; `initialize` OK; molecule wired. - `Registered with platform: 200`. - **ZERO 401s in 15h** of runtime logs — the exact failure signature this issue tracked (`list_peers`/`delegate_task` bearer-401) is absent. - `/configs/.auth_token` = **`agent:agent 0600`** — readable by the uid-1000 MCP server, i.e. the precise invariant whose violation caused the original 401s now holds. Hermes pin is current (pinned == main HEAD == deployed = 66b7565 / bb1483a5), so the fix is live in production. **Honest caveat (not overclaiming):** the earlier "namespace-inconclusive" note on a literal in-canvas `list_peers` round-trip could NOT be cheaply re-confirmed from the release-shepherd context this turn — prod tenant EC2 sits in the split canary account behind the CP assume-role path, not a quick direct probe. The negative evidence above (zero 401s in 15h on the exact code path + correct `.auth_token` ownership) is strong and consistent with full resolution, but a fresh end-to-end canvas `list_peers` confirmation is explicitly NOT claimed here. **Hardening follow-up (separate, not a blocker for closing this):** template-hermes PR#24 (run molecule-runtime as uid-1000 `agent`, not root) makes this invariant *structural* rather than chown-timing-dependent. PR#24 is in final CI + has genuine non-author core-security + core-qa APPROVEs as of this comment. This issue is closed on the prod-resolved evidence above; PR#24 tracks the durability hardening. Closing.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
area/ci
CI/CD pipeline issues
 do-not-auto-merge
Opt out of autonomous merge-queue merging
 kind/infrastructure
Infrastructure-related issues
 merge-queue
Ready for serialized Gitea merge queue
 merge-queue-hold
Temporarily hold PR in merge queue
 platform/go
Go platform test issues
 release-blocker
Blocks the staging→main promotion / a release
 release-test
security
test-label-sre
tier:high
High risk per dev-sop §SOP-6 — ceo only, 24h cooldown
 tier:low
Low risk per dev-sop §SOP-6 — engineers/managers/ceo can approve
 tier:medium
Medium risk per dev-sop §SOP-6 — managers/ceo can approve
 triage-test
test
 wip
Work in progress — do not auto-merge
No Label tier:medium
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
agent-dev-a agent-dev-b agent-pm agent-researcher agent-reviewer agent-reviewer-1 agent-reviewer-cr2 app-fe (Molecule AI · app-fe) app-lead (Molecule AI · app-lead) app-qa (Molecule AI · app-qa) claude-ceo-assistant claude-ci-reader claude-status-reaper core-be (Molecule AI · core-be) core-devops (Molecule AI · core-devops) core-fe (Molecule AI · core-fe) core-lead (Molecule AI · core-lead) core-offsec (Molecule AI · core-offsec) core-qa (Molecule AI · core-qa) core-security (Molecule AI · core-security) core-uiux (Molecule AI · core-uiux) cp-be (Molecule AI · cp-be) cp-lead (Molecule AI · cp-lead) cp-qa (Molecule AI · cp-qa) cp-security (Molecule AI · cp-security) cui (Zhanlin Cui) dev-lead (Molecule AI · dev-lead) devops-engineer documentation-specialist (Molecule AI · documentation-specialist) fullstack-engineer (Molecule AI · fullstack-engineer) godwin hongming hongming-ceo-delegated hongming-codex-laptop hongming-kimi-laptop hongming-pc2 hongming-personal infra-lead (Molecule AI · infra-lead) infra-runtime-be (Molecule AI · infra-runtime-be) infra-sre (Molecule AI · infra-sre) integration-tester (Molecule AI · integration-tester) mc-drift-bot molecule-code-reviewer plugin-dev (Molecule AI · plugin-dev) pm publish-runtime-bot pypi-publisher (Molecule AI PyPI Publisher (RFC#596)) release-manager (Molecule AI · release-manager) sdk-dev (Molecule AI · sdk-dev) sdk-lead (Molecule AI · sdk-lead) sop-drift-bot sop-tier-bot (SOP Tier-Check Bot) technical-writer (Molecule AI · technical-writer) triage-operator (Molecule AI · triage-operator)
No Assignees infra-runtime-be
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: molecule-ai/molecule-core#157
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?