molecule-ai/molecule-core
53
0
Fork 2
Code Issues 183 Pull Requests 41 Actions Packages Projects Releases Wiki Activity

[a2a] successful delegation responses rendered as error + Restart-workspace suggestion → retry storms + data-loss risk #159

New Issue
Closed
opened 2026-05-09 20:28:41 +00:00 by claude-ceo-assistant · 0 comments
claude-ceo-assistant commented 2026-05-09 20:28:41 +00:00
Owner
Copy Link
Copy Source

Symptom (real customer-impacting bug)

When workspace A delegates to workspace B and B successfully produces a response, the canvas often renders the result as:

⚠  Core-DevOps returned an error
TASK: <full successful diagnosis text — agent did the work>
Underlying error
(no detail returned)
The remote agent returned no error detail (the underlying httpx exception had an empty message — typically a connection-reset or silent timeout). A workspace restart is the safe first move.
[Restart Core-DevOps]  [Open Core-DevOps]

Real example from this morning: Core-DevOps successfully produced "PR #149: tier-check fails NO REVIEWS (author needs engineers/managers/ceo approval); PR #140: pm approved but pm not in managers/ceo team; Harness Replays #141 known-broken." That's a correct, complete diagnosis. But the canvas marked the response as an error and suggested restarting the workspace.

Why this matters

  1. Restarting a workspace as suggested = data loss (kills the agent's in-progress reasoning + any unsaved tool-call state)
  2. The PM sees "error" and retries the delegation 4–5 times, each retry fires a fresh delegation that lands on the busy adapter and bounces — a retry storm
  3. The actual diagnosis is sitting in plain sight inside the error banner but the framing trains PMs to ignore it

This is the root cause of the retry-storm pattern observed across Core-Lead → Core-DevOps comms in the last hour (4+ retries of an already-completed diagnosis).

Likely root cause

A2A response delivery uses sync HTTP. When the target adapter is busy, the platform receives the agent's response body but the delivery transport returns a connection-reset / empty-body error to the canvas event consumer. Canvas then renders activity_type=delegation with status=error even though the response body was received.

Fix shape (to confirm during diagnosis)

  • Decouple delegation completed from delivery transport — the response body should be persisted as soon as the agent produces it, regardless of whether the sync HTTP delivery to the requester succeeded
  • Canvas event renderer: if the activity row has a non-empty response body, render as completed with the body; only mark error when body is genuinely empty
  • Drop the "Restart workspace" suggestion when the body is non-empty — it's actively harmful

Tier

tier:high — destroys A2A reliability; trains operators to retry-storm; suggests destructive recovery action

Reporter

Hongming (CTO), 2026-05-09 ~20:30 UTC, after seeing the Core-DevOps retry-storm pattern in screenshot.

## Symptom (real customer-impacting bug) When workspace A delegates to workspace B and B *successfully* produces a response, the canvas often renders the result as: ``` ⚠ Core-DevOps returned an error TASK: <full successful diagnosis text — agent did the work> Underlying error (no detail returned) The remote agent returned no error detail (the underlying httpx exception had an empty message — typically a connection-reset or silent timeout). A workspace restart is the safe first move. [Restart Core-DevOps] [Open Core-DevOps] ``` Real example from this morning: Core-DevOps successfully produced "PR #149: tier-check fails NO REVIEWS (author needs engineers/managers/ceo approval); PR #140: pm approved but pm not in managers/ceo team; Harness Replays #141 known-broken." That's a *correct, complete* diagnosis. But the canvas marked the response as an error and suggested restarting the workspace. ## Why this matters 1. Restarting a workspace as suggested = **data loss** (kills the agent's in-progress reasoning + any unsaved tool-call state) 2. The PM sees "error" and retries the delegation 4–5 times, each retry fires a fresh delegation that lands on the busy adapter and bounces — a retry storm 3. The actual diagnosis is sitting in plain sight inside the error banner but the framing trains PMs to ignore it This is the root cause of the retry-storm pattern observed across Core-Lead → Core-DevOps comms in the last hour (4+ retries of an already-completed diagnosis). ## Likely root cause A2A response delivery uses sync HTTP. When the target adapter is busy, the platform receives the agent's response body but the delivery transport returns a connection-reset / empty-body error to the canvas event consumer. Canvas then renders activity_type=delegation with status=error even though the response body was received. ## Fix shape (to confirm during diagnosis) - Decouple `delegation completed` from delivery transport — the response body should be persisted as soon as the agent produces it, regardless of whether the sync HTTP delivery to the requester succeeded - Canvas event renderer: if the activity row has a non-empty response body, render as `completed` with the body; only mark error when body is genuinely empty - Drop the "Restart workspace" suggestion when the body is non-empty — it's actively harmful ## Tier `tier:high` — destroys A2A reliability; trains operators to retry-storm; suggests destructive recovery action ## Reporter Hongming (CTO), 2026-05-09 ~20:30 UTC, after seeing the Core-DevOps retry-storm pattern in screenshot.
core-be referenced this issue from a commit 2026-05-09 21:52:16 +00:00
[core-be-agent]
core-be referenced this issue from a commit 2026-05-09 22:10:58 +00:00
[core-be-agent]
core-be referenced this issue from a commit 2026-05-09 22:12:04 +00:00
[core-be-agent]
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
area/ci
CI/CD pipeline issues
 do-not-auto-merge
Opt out of autonomous merge-queue merging
 kind/infrastructure
Infrastructure-related issues
 merge-queue
Ready for serialized Gitea merge queue
 merge-queue-hold
Temporarily hold PR in merge queue
 platform/go
Go platform test issues
 release-blocker
Blocks the staging→main promotion / a release
 release-test
security
test-label-sre
tier:high
High risk per dev-sop §SOP-6 — ceo only, 24h cooldown
 tier:low
Low risk per dev-sop §SOP-6 — engineers/managers/ceo can approve
 tier:medium
Medium risk per dev-sop §SOP-6 — managers/ceo can approve
 triage-test
test
 wip
Work in progress — do not auto-merge
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
agent-dev-a agent-dev-b agent-pm agent-researcher agent-reviewer agent-reviewer-1 agent-reviewer-cr2 app-fe (Molecule AI · app-fe) app-lead (Molecule AI · app-lead) app-qa (Molecule AI · app-qa) claude-ceo-assistant claude-ci-reader claude-status-reaper core-be (Molecule AI · core-be) core-devops (Molecule AI · core-devops) core-fe (Molecule AI · core-fe) core-lead (Molecule AI · core-lead) core-offsec (Molecule AI · core-offsec) core-qa (Molecule AI · core-qa) core-security (Molecule AI · core-security) core-uiux (Molecule AI · core-uiux) cp-be (Molecule AI · cp-be) cp-lead (Molecule AI · cp-lead) cp-qa (Molecule AI · cp-qa) cp-security (Molecule AI · cp-security) cui (Zhanlin Cui) dev-lead (Molecule AI · dev-lead) devops-engineer documentation-specialist (Molecule AI · documentation-specialist) fullstack-engineer (Molecule AI · fullstack-engineer) godwin hongming hongming-ceo-delegated hongming-codex-laptop hongming-kimi-laptop hongming-pc2 hongming-personal infra-lead (Molecule AI · infra-lead) infra-runtime-be (Molecule AI · infra-runtime-be) infra-sre (Molecule AI · infra-sre) integration-tester (Molecule AI · integration-tester) mc-drift-bot molecule-code-reviewer plugin-dev (Molecule AI · plugin-dev) pm publish-runtime-bot pypi-publisher (Molecule AI PyPI Publisher (RFC#596)) release-manager (Molecule AI · release-manager) sdk-dev (Molecule AI · sdk-dev) sdk-lead (Molecule AI · sdk-lead) sop-drift-bot sop-tier-bot (SOP Tier-Check Bot) technical-writer (Molecule AI · technical-writer) triage-operator (Molecule AI · triage-operator)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: molecule-ai/molecule-core#159
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?