fix(a2a): avoid false failure on busy queue fallback #1751

Merged

hongming merged 1 commits from fix/codex-scheduled-a2a-timeout into main 2026-05-23 23:29:01 +00:00

Conversation 1 Commits 1 Files Changed 3

+100 -11

hongming commented 2026-05-23 23:21:36 +00:00

Owner

Copy Link

Copy Source

Summary

• raise the A2A response-header budget from 180s to 5m so scheduled Codex turns get the same first-response budget as the scheduler fire timeout
• when an upstream busy/timeout attempt is successfully durably queued, log the A2A receive as queued/ok instead of recording a false failure row
• add a regression test for the live Researcher shape: response-header timeout -> queue success -> 202 + activity status ok

Live evidence

• Root-Cause Researcher was changed from poll to push; the scheduler then reached the runtime instead of queueing for poll
• the push attempt hit timeout awaiting response headers at 180002ms, then a2a_queue item 79496d0a-b00b-4e8c-aa55-39fc10a5b18b completed on heartbeat drain

Verification

• go test ./internal/handlers -run 'TestHandleA2ADispatchError_BusyEnqueueLogsQueuedNotFailure|TestHandleA2ADispatchError_ContextDeadline|TestA2AClientResponseHeaderTimeout|TestHandleA2ADispatchError_NativeSession_NowEnqueues|TestHandleA2ADispatchError_NoNativeSession_StillEnqueues'
• go test ./internal/handlers

## Summary - raise the A2A response-header budget from 180s to 5m so scheduled Codex turns get the same first-response budget as the scheduler fire timeout - when an upstream busy/timeout attempt is successfully durably queued, log the A2A receive as queued/ok instead of recording a false failure row - add a regression test for the live Researcher shape: response-header timeout -> queue success -> 202 + activity status ok ## Live evidence - Root-Cause Researcher was changed from poll to push; the scheduler then reached the runtime instead of queueing for poll - the push attempt hit `timeout awaiting response headers` at 180002ms, then a2a_queue item `79496d0a-b00b-4e8c-aa55-39fc10a5b18b` completed on heartbeat drain ## Verification - `go test ./internal/handlers -run 'TestHandleA2ADispatchError_BusyEnqueueLogsQueuedNotFailure|TestHandleA2ADispatchError_ContextDeadline|TestA2AClientResponseHeaderTimeout|TestHandleA2ADispatchError_NativeSession_NowEnqueues|TestHandleA2ADispatchError_NoNativeSession_StillEnqueues'` - `go test ./internal/handlers`

hongming added 1 commit 2026-05-23 23:21:37 +00:00

fix(a2a): avoid false failure on busy queue fallback 691d341fbb

agent-reviewer approved these changes 2026-05-23 23:27:27 +00:00

agent-reviewer left a comment

Member

Copy Link

Copy Source

5-axis review on 691d341:

Correctness: APPROVED. Busy/timeout dispatches that successfully enqueue now return the existing 202 queued response and record the activity as queued/ok rather than logging a false failure, while real container-dead/enqueue-failed/other errors still log failures. The response-header budget now matches the scheduler's longer first-response window.
Robustness: The new regression test covers the live shape: timeout/busy error, durable enqueue, 202 response, and status=ok activity logging. Existing enqueue failure fallback remains intact.
Security: No auth or secret surface changed.
Performance: Timeout budget increase can hold a connection longer, but aligns with scheduled turn expectations; no hot-loop or N+1 concern.
Readability: The new helper names and comments make the queued-vs-failed distinction clear. Functional CI is green; red statuses are review gates.

5-axis review on 691d341: Correctness: APPROVED. Busy/timeout dispatches that successfully enqueue now return the existing 202 queued response and record the activity as queued/ok rather than logging a false failure, while real container-dead/enqueue-failed/other errors still log failures. The response-header budget now matches the scheduler's longer first-response window. Robustness: The new regression test covers the live shape: timeout/busy error, durable enqueue, 202 response, and status=ok activity logging. Existing enqueue failure fallback remains intact. Security: No auth or secret surface changed. Performance: Timeout budget increase can hold a connection longer, but aligns with scheduled turn expectations; no hot-loop or N+1 concern. Readability: The new helper names and comments make the queued-vs-failed distinction clear. Functional CI is green; red statuses are review gates.

hongming merged commit 4d32736e25 into main 2026-05-23 23:29:01 +00:00

hongming referenced this issue from a commit 2026-05-23 23:29:01 +00:00

Merge pull request 'fix(a2a): avoid false failure on busy queue fallback' (#1751) from fix/codex-scheduled-a2a-timeout into main

hongming referenced this pull request 2026-05-24 00:23:35 +00:00

[main-red] molecule-ai/molecule-core: def18f28fa #1638

agent-researcher referenced this pull request 2026-05-24 01:46:15 +00:00

[RCA] Post-merge regression scan 2026-05-24 #1761

agent-researcher referenced this pull request 2026-05-24 03:29:18 +00:00

RCA: recent escalation-marker sweep shows CI/A2A false-positive clusters #1767

agent-researcher referenced this pull request 2026-05-26 01:03:54 +00:00

[main-red] molecule-ai/molecule-core: 4d32736e25 #1757

Sign in to join this conversation.

Reviewers

No Reviewers

agent-reviewer

Labels

Clear labels

area/ci

CI/CD pipeline issues

kind/infrastructure

Infrastructure-related issues

merge-queue

Ready for serialized Gitea merge queue

merge-queue-hold

Temporarily hold PR in merge queue

platform/go

Go platform test issues

release-blocker

Blocks the staging→main promotion / a release

release-test

security

test-label-sre

tier:high

High risk per dev-sop §SOP-6 — ceo only, 24h cooldown

tier:low

Low risk per dev-sop §SOP-6 — engineers/managers/ceo can approve

tier:medium

Medium risk per dev-sop §SOP-6 — managers/ceo can approve

triage-test

test

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

agent-dev-a agent-dev-b agent-pm agent-researcher agent-reviewer app-fe (Molecule AI · app-fe) app-lead (Molecule AI · app-lead) app-qa (Molecule AI · app-qa) claude-ceo-assistant claude-ci-reader claude-status-reaper core-be (Molecule AI · core-be) core-devops (Molecule AI · core-devops) core-fe (Molecule AI · core-fe) core-lead (Molecule AI · core-lead) core-offsec (Molecule AI · core-offsec) core-qa (Molecule AI · core-qa) core-security (Molecule AI · core-security) core-uiux (Molecule AI · core-uiux) cp-be (Molecule AI · cp-be) cp-lead (Molecule AI · cp-lead) cp-qa (Molecule AI · cp-qa) cp-security (Molecule AI · cp-security) cui (Zhanlin Cui) dev-lead (Molecule AI · dev-lead) devops-engineer documentation-specialist (Molecule AI · documentation-specialist) fullstack-engineer (Molecule AI · fullstack-engineer) hongming hongming-ceo-delegated hongming-codex-laptop hongming-kimi-laptop hongming-pc2 infra-lead (Molecule AI · infra-lead) infra-runtime-be (Molecule AI · infra-runtime-be) infra-sre (Molecule AI · infra-sre) integration-tester (Molecule AI · integration-tester) mc-drift-bot molecule-code-reviewer plugin-dev (Molecule AI · plugin-dev) pm publish-runtime-bot pypi-publisher (Molecule AI PyPI Publisher (RFC#596)) release-manager (Molecule AI · release-manager) sdk-dev (Molecule AI · sdk-dev) sdk-lead (Molecule AI · sdk-lead) sop-drift-bot sop-tier-bot (SOP Tier-Check Bot) technical-writer (Molecule AI · technical-writer) triage-operator (Molecule AI · triage-operator)

No Assignees

2 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: molecule-ai/molecule-core#1751