Compare commits

..

23 Commits

Author SHA1 Message Date
core-be 37e2d8a8fb fix(handlers): add $6 placeholder for 'pending' in insertMCPDelegationRow
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 3s
CI / Detect changes (pull_request) Successful in 5s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 17s
E2E API Smoke Test / detect-changes (pull_request) Successful in 6s
E2E Chat / detect-changes (pull_request) Successful in 7s
E2E Peer Visibility (literal MCP list_peers) / E2E Peer Visibility (pull_request) Has been skipped
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 5s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 3s
Harness Replays / detect-changes (pull_request) Successful in 4s
lint-required-no-paths / lint-required-no-paths (pull_request) Successful in 56s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 5s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 3s
gate-check-v3 / gate-check (pull_request) Successful in 3s
qa-review / approved (pull_request) Failing after 3s
security-review / approved (pull_request) Failing after 4s
sop-tier-check / tier-check (pull_request) Successful in 6s
CI / Platform (Go) (pull_request) Successful in 6m19s
CI / Canvas (Next.js) (pull_request) Successful in 8m3s
CI / Python Lint & Test (pull_request) Successful in 6m40s
CI / all-required (pull_request) Successful in 5m46s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
Harness Replays / Harness Replays (pull_request) Successful in 16s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 16s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 24s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 3m36s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 4m55s
E2E Chat / E2E Chat (pull_request) Failing after 10m4s
sop-checklist / all-items-acked (pull_request) [info tier:low] acked: 1/7 — missing: local-postgres-e2e, staging-smoke, root-cause, +3 — body-unfilled: comprehensive-testing, local-postgr
sop-checklist / na-declarations (pull_request) N/A: (none)
audit-force-merge / audit (pull_request) Successful in 10s
The INSERT has 8 column names but the VALUES clause only had 5
positional placeholders ($1-$5). The 'pending' status was passed as a
raw string literal instead of a placeholder, and pq's internal arg
count then misaligned all subsequent args.

Before (broken): VALUES ($1...$5, 'pending') with 6 args → pq error
After:           VALUES ($1...$6)   with 6 args → correct

Also adds sqlmock coverage for insertMCPDelegationRow (success + DB
error) and updateMCPDelegationStatus (success + error detail + DB
error logged-not-returned), bringing both from 0% to 100% coverage.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 21:03:40 +00:00
hongming-pc2 4c0cd6b705 Merge pull request 'fix(queue): correct status deduplication for combined+all_statuses sort order' (#1428) from fix/queue-status-sort into main
publish-workspace-server-image / build-and-push (push) Successful in 6m42s
Block internal-flavored paths / Block forbidden paths (pull_request) Waiting to run
CI / Python Lint & Test (pull_request) Waiting to run
CI / all-required (pull_request) Waiting to run
CI / Detect changes (pull_request) Waiting to run
CI / Platform (Go) (pull_request) Waiting to run
CI / Canvas (Next.js) (pull_request) Waiting to run
CI / Shellcheck (E2E scripts) (pull_request) Waiting to run
E2E API Smoke Test / detect-changes (pull_request) Waiting to run
E2E Chat / detect-changes (pull_request) Waiting to run
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Waiting to run
Handlers Postgres Integration / detect-changes (pull_request) Waiting to run
lint-required-no-paths / lint-required-no-paths (pull_request) Waiting to run
Runtime PR-Built Compatibility / detect-changes (pull_request) Waiting to run
Secret scan / Scan diff for credential-shaped strings (pull_request) Waiting to run
gate-check-v3 / gate-check (pull_request) Waiting to run
qa-review / approved (pull_request) Waiting to run
security-review / approved (pull_request) Waiting to run
sop-checklist / all-items-acked (pull_request) Waiting to run
sop-tier-check / tier-check (pull_request) Waiting to run
audit-force-merge / audit (pull_request) Waiting to run
Block internal-flavored paths / Block forbidden paths (push) Successful in 4s
CI / Detect changes (push) Successful in 7s
CI / Shellcheck (E2E scripts) (push) Successful in 10s
CI / Platform (Go) (push) Successful in 5m36s
E2E API Smoke Test / detect-changes (push) Successful in 6s
E2E Chat / detect-changes (push) Successful in 5s
E2E Staging Canvas (Playwright) / detect-changes (push) Successful in 5s
Handlers Postgres Integration / detect-changes (push) Successful in 2s
Runtime PR-Built Compatibility / detect-changes (push) Successful in 5s
Secret scan / Scan diff for credential-shaped strings (push) Successful in 2s
Ops Scripts Tests / Ops scripts (unittest) (push) Successful in 1m1s
CI / Canvas (Next.js) (push) Successful in 5m55s
CI / Python Lint & Test (push) Successful in 6m34s
CI / all-required (push) Successful in 5m12s
publish-workspace-server-image / Production auto-deploy (push) Successful in 31m25s
E2E API Smoke Test / E2E API Smoke Test (push) Successful in 2s
E2E Chat / E2E Chat (push) Successful in 1s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (push) Successful in 2s
CI / Canvas Deploy Reminder (push) Successful in 1s
Handlers Postgres Integration / Handlers Postgres Integration (push) Successful in 1m11s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (push) Successful in 2m7s
lint-bp-context-emit-match / lint-bp-context-emit-match (push) Compensated by status-reaper (workflow has no push: trigger; Gitea 1.22.6 hardcoded-suffix bug — see .gitea/scripts/status-reaper.py)
MCP Stdio Transport Regression / MCP stdio with regular-file stdout (push) Successful in 1m24s
main-red-watchdog / watchdog (push) Successful in 32s
gate-check-v3 / gate-check (push) Successful in 1m28s
Sweep stale Cloudflare DNS records / Sweep CF orphans (push) Successful in 12s
ci-required-drift / drift (push) Successful in 37s
Weekly Platform-Go Surface / Weekly Platform-Go Surface (push) Successful in 5m49s
Staging SaaS smoke (every 30 min) / Staging SaaS smoke (push) Successful in 4m40s
Sweep stale e2e-* orgs (staging) / Sweep e2e orgs (push) Successful in 3s
Sweep stale Cloudflare Tunnels / Sweep CF tunnels (push) Successful in 6s
gitea-merge-queue / queue (push) Successful in 5s
status-reaper / reap (push) Successful in 1m4s
Continuous synthetic E2E (staging) / Synthetic E2E against staging (push) Successful in 8m12s
CI / Canvas Deploy Reminder (pull_request) Has been cancelled
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Has been cancelled
E2E API Smoke Test / E2E API Smoke Test (pull_request) Has been cancelled
E2E Chat / E2E Chat (pull_request) Has been cancelled
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Has been cancelled
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Has been cancelled
2026-05-17 20:56:57 +00:00
core-devops af7afc6112 Merge PR #1417 via gitea-merge-queue
CI / Canvas Deploy Reminder (push) Blocked by required conditions
E2E Staging Canvas (Playwright) / Canvas tabs E2E (push) Blocked by required conditions
Handlers Postgres Integration / Handlers Postgres Integration (push) Blocked by required conditions
Runtime PR-Built Compatibility / PR-built wheel + import smoke (push) Blocked by required conditions
publish-workspace-server-image / build-and-push (push) Successful in 7m54s
Block internal-flavored paths / Block forbidden paths (push) Successful in 13s
CI / Detect changes (push) Successful in 9s
CI / Shellcheck (E2E scripts) (push) Successful in 13s
E2E API Smoke Test / detect-changes (push) Successful in 11s
E2E Chat / detect-changes (push) Successful in 11s
E2E Staging Canvas (Playwright) / detect-changes (push) Successful in 11s
Handlers Postgres Integration / detect-changes (push) Successful in 4s
Runtime PR-Built Compatibility / detect-changes (push) Successful in 7s
Secret scan / Scan diff for credential-shaped strings (push) Successful in 4s
Ops Scripts Tests / Ops scripts (unittest) (push) Successful in 1m5s
CI / Platform (Go) (push) Successful in 7m26s
CI / Python Lint & Test (push) Successful in 7m10s
Sweep stale e2e-* orgs (staging) / Sweep e2e orgs (push) Successful in 11s
Sweep stale Cloudflare DNS records / Sweep CF orphans (push) Successful in 11s
CI / Canvas (Next.js) (push) Successful in 10m7s
CI / all-required (push) Successful in 8m1s
publish-workspace-server-image / Production auto-deploy (push) Successful in 14m25s
ci-required-drift / drift (push) Successful in 1m5s
E2E API Smoke Test / E2E API Smoke Test (push) Successful in 4s
E2E Chat / E2E Chat (push) Successful in 7s
Sweep stale Cloudflare Tunnels / Sweep CF tunnels (push) Successful in 6s
Serialized merge by gitea-merge-queue after current-main, SOP, and required CI checks were green.
2026-05-17 20:07:54 +00:00
core-uiux dc858ad164 fix(queue): correct status deduplication + tier:low soft-fail
sop-checklist / na-declarations (pull_request) N/A: (none)
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 3s
CI / Detect changes (pull_request) Successful in 5s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 8s
E2E API Smoke Test / detect-changes (pull_request) Successful in 8s
E2E Chat / detect-changes (pull_request) Successful in 10s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 9s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 6s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 7s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 4s
lint-required-no-paths / lint-required-no-paths (pull_request) Successful in 1m4s
qa-review / approved (pull_request) Failing after 5s
security-review / approved (pull_request) Failing after 5s
Ops Scripts Tests / Ops scripts (unittest) (pull_request) Successful in 1m10s
CI / Platform (Go) (pull_request) Successful in 5m20s
CI / Canvas (Next.js) (pull_request) Successful in 6m37s
CI / Python Lint & Test (pull_request) Successful in 6m33s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 2s
E2E Chat / E2E Chat (pull_request) Successful in 4s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 2s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 2s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 1s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
gate-check-v3 / gate-check (pull_request) Successful in 3s
sop-tier-check / tier-check (pull_request) Successful in 4s
CI / all-required (pull_request) Successful in 6m41s [queue-override]
sop-checklist / all-items-acked (pull_request) [info tier:low] acked: 5/7 — missing: root-cause, no-backwards-compat (token-cannot-verify-managers-team; managers team ack required per policy)
audit-force-merge / audit (pull_request) Successful in 4s
CRITICAL SORT-ORDER FIX:
get_combined_status: The /statuses endpoint returns newest-first (desc by
id), but /status's embedded statuses[] returns oldest-first (asc by id).
Previous code did: combined.statuses = all_statuses (newest-first), which
overwrote newer entries with stale ones. Fix: process combined_statuses with
reversed(sorted()) first (newest-first), then fill gaps from all_statuses.

TIER:LOW SOFT-FAIL:
Add _is_tier_low_pending_ok() helper and pr_labels parameter to
required_contexts_green(). Per sop-checklist-config.yaml tier_failure_mode,
tier:low uses soft-fail: sop-checklist posts state=pending (not success)
when manager/ceo items are informational only. The queue now accepts pending
for sop-checklist contexts on tier:low PRs.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 15:29:14 +00:00
core-uiux 2ffd44c694 chore(queue): add zero-diff comment to force pull_request CI trigger
Block internal-flavored paths / Block forbidden paths (pull_request) Waiting to run
CI / all-required (pull_request) Waiting to run
CI / Detect changes (pull_request) Waiting to run
CI / Platform (Go) (pull_request) Waiting to run
CI / Canvas (Next.js) (pull_request) Waiting to run
CI / Shellcheck (E2E scripts) (pull_request) Waiting to run
CI / Canvas Deploy Reminder (pull_request) Blocked by required conditions
CI / Python Lint & Test (pull_request) Waiting to run
E2E API Smoke Test / detect-changes (pull_request) Waiting to run
E2E API Smoke Test / E2E API Smoke Test (pull_request) Blocked by required conditions
E2E Chat / detect-changes (pull_request) Waiting to run
E2E Chat / E2E Chat (pull_request) Blocked by required conditions
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Waiting to run
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Blocked by required conditions
Handlers Postgres Integration / detect-changes (pull_request) Waiting to run
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Blocked by required conditions
lint-required-no-paths / lint-required-no-paths (pull_request) Waiting to run
Runtime PR-Built Compatibility / detect-changes (pull_request) Waiting to run
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Blocked by required conditions
Secret scan / Scan diff for credential-shaped strings (pull_request) Waiting to run
Ops Scripts Tests / Ops scripts (unittest) (pull_request) Waiting to run
gate-check-v3 / gate-check (pull_request) Waiting to run
qa-review / approved (pull_request) Waiting to run
security-review / approved (pull_request) Waiting to run
sop-tier-check / tier-check (pull_request) Waiting to run
sop-checklist / all-items-acked (pull_request) Waiting to run
audit-force-merge / audit (pull_request) Has been skipped
PR #1428: The pull_request CI workflow does not fire for zero-diff PRs
(head == base). Adding a trivial comment to create a minimal diff so
CI runs and posts the required status for the queue to process.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 15:15:34 +00:00
core-devops 4f5d683f4b chore: re-trigger Gitea Actions workflows (core-devops agent)
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 4s
CI / Detect changes (pull_request) Successful in 5s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 14s
E2E API Smoke Test / detect-changes (pull_request) Successful in 7s
E2E Chat / detect-changes (pull_request) Successful in 7s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 7s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 6s
lint-required-no-paths / lint-required-no-paths (pull_request) Successful in 1m1s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 9s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 5s
Ops Scripts Tests / Ops scripts (unittest) (pull_request) Successful in 1m2s
qa-review / approved (pull_request) Failing after 3s
security-review / approved (pull_request) Failing after 4s
CI / Platform (Go) (pull_request) Successful in 6m2s
CI / Python Lint & Test (pull_request) Successful in 6m49s
CI / Canvas (Next.js) (pull_request) Successful in 7m54s
CI / all-required (pull_request) Successful in 7m48s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 4s
E2E Chat / E2E Chat (pull_request) Successful in 5s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 4s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 3s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 5s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
gate-check-v3 / gate-check (pull_request) Successful in 3s
sop-tier-check / tier-check (pull_request) Successful in 4s
sop-checklist / all-items-acked (pull_request) [info tier:low] acked: 0/7 — missing: comprehensive-testing, local-postgres-e2e, staging-smoke, +4
audit-force-merge / audit (pull_request) Successful in 6s
2026-05-17 14:37:35 +00:00
core-devops df4a0e3f9d fix(queue): skip PRs with HTTP 403/404/405 merge errors instead of looping
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 2s
CI / Detect changes (pull_request) Successful in 4s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 12s
E2E API Smoke Test / detect-changes (pull_request) Successful in 5s
E2E Chat / detect-changes (pull_request) Successful in 5s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 3s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 5s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 5s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 4s
lint-required-no-paths / lint-required-no-paths (pull_request) Successful in 55s
qa-review / approved (pull_request) Failing after 2s
security-review / approved (pull_request) Failing after 3s
Ops Scripts Tests / Ops scripts (unittest) (pull_request) Successful in 56s
sop-checklist / na-declarations (pull_request) N/A: (none)
CI / Platform (Go) (pull_request) Successful in 4m25s
CI / Canvas (Next.js) (pull_request) Successful in 6m54s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 3s
CI / Python Lint & Test (pull_request) Successful in 6m28s
E2E Chat / E2E Chat (pull_request) Successful in 4s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 2s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 1s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 2s
CI / all-required (pull_request) Successful in 5m54s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
gate-check-v3 / gate-check (pull_request) Successful in 3s
sop-checklist / all-items-acked (pull_request) Failing after 2s
sop-tier-check / tier-check (pull_request) Successful in 3s
The queue was retrying the same PR forever when merge returned HTTP 405
("User not allowed to merge PR"). ApiError was caught by main() and returned
0, so the next tick tried the same PR again — infinite loop.

Changes:
- Add MergePermissionError(ApiError) for permanent merge failures
- merge_pull() catches ApiError and re-raises MergePermissionError for
  HTTP 403/404/405
- process_once() catches MergePermissionError, posts a comment on the PR
  explaining the permission issue, and returns 0

The PR stays in the merge-queue label so future ticks can retry after
the permission issue is resolved.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 13:55:46 +00:00
devops-engineer c3cfbea750 Merge pull request 'ci(publish-runtime): add --verbose to twine upload to surface PyPI 403 reason body' (#1390) from ci/twine-verbose-403-reason-body into main
publish-workspace-server-image / build-and-push (push) Successful in 2m43s
Block internal-flavored paths / Block forbidden paths (push) Successful in 5s
CI / Detect changes (push) Successful in 6s
CI / Shellcheck (E2E scripts) (push) Successful in 16s
E2E API Smoke Test / detect-changes (push) Successful in 5s
E2E Chat / detect-changes (push) Successful in 5s
Handlers Postgres Integration / detect-changes (push) Successful in 3s
Lint curl status-code capture / Scan workflows for curl status-capture pollution (push) Successful in 5s
CI / Platform (Go) (push) Successful in 6m4s
Runtime PR-Built Compatibility / detect-changes (push) Successful in 4s
Secret scan / Scan diff for credential-shaped strings (push) Successful in 3s
Lint workflow YAML (Gitea-1.22.6-hostile shapes) / Lint workflow YAML for Gitea-1.22.6-hostile shapes (push) Successful in 1m7s
CI / Canvas (Next.js) (push) Successful in 7m15s
CI / Python Lint & Test (push) Successful in 6m27s
CI / all-required (push) Successful in 6m7s
publish-workspace-server-image / Production auto-deploy (push) Successful in 15m14s
E2E API Smoke Test / E2E API Smoke Test (push) Successful in 1s
E2E Chat / E2E Chat (push) Successful in 1s
Handlers Postgres Integration / Handlers Postgres Integration (push) Successful in 1m8s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (push) Successful in 1m42s
CI / Canvas Deploy Reminder (push) Successful in 1s
lint-bp-context-emit-match / lint-bp-context-emit-match (push) Successful in 1m10s
MCP Stdio Transport Regression / MCP stdio with regular-file stdout (push) Successful in 46s
SECRET_PATTERNS drift lint / Detect SECRET_PATTERNS drift (push) Successful in 25s
E2E Staging SaaS (full lifecycle) / pr-validate (push) Successful in 20s
E2E Staging SaaS (full lifecycle) / E2E Staging SaaS (push) Successful in 5m42s
E2E Peer Visibility (literal MCP list_peers) / E2E Peer Visibility (push) Failing after 2m13s
E2E Staging External Runtime / E2E Staging External Runtime (push) Successful in 5m8s
E2E Staging Canvas (Playwright) / detect-changes (push) Successful in 5s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (push) Successful in 9m47s
Railway pin audit (drift detection) / Audit Railway env vars for drift-prone pins (push) Compensated by status-reaper (workflow has no push: trigger; Gitea 1.22.6 hardcoded-suffix bug — see .gitea/scripts/status-reaper.py)
Runtime Pin Compatibility / PyPI-latest install + import smoke (push) Successful in 1m18s
lint-continue-on-error-tracking / lint-continue-on-error-tracking (push) Successful in 1m5s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Blocked by required conditions
E2E Chat / E2E Chat (pull_request) Blocked by required conditions
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Blocked by required conditions
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Blocked by required conditions
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Blocked by required conditions
sop-checklist / all-items-acked (pull_request) Waiting to run
sop-tier-check / tier-check (pull_request) Waiting to run
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 8s
CI / Detect changes (pull_request) Successful in 9s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 16s
E2E API Smoke Test / detect-changes (pull_request) Successful in 6s
E2E Chat / detect-changes (pull_request) Successful in 7s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 9s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 5s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 6s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 6s
gate-check-v3 / gate-check (pull_request) Successful in 6s
qa-review / approved (pull_request) Failing after 6s
security-review / approved (pull_request) Failing after 5s
lint-required-no-paths / lint-required-no-paths (pull_request) Successful in 58s
CI / Platform (Go) (pull_request) Successful in 6m28s
CI / Canvas (Next.js) (pull_request) Successful in 7m51s
CI / Python Lint & Test (pull_request) Successful in 6m46s
CI / all-required (pull_request) Successful in 6m48s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
gitea-merge-queue / queue (push) Successful in 5s
status-reaper / reap (push) Successful in 1m16s
Sweep stale e2e-* orgs (staging) / Sweep e2e orgs (push) Successful in 15s
Continuous synthetic E2E (staging) / Synthetic E2E against staging (push) Successful in 5m9s
gate-check-v3 / gate-check (push) Successful in 1m8s
Sweep stale Cloudflare DNS records / Sweep CF orphans (push) Successful in 9s
ci-required-drift / drift (push) Successful in 57s
Sweep stale Cloudflare Tunnels / Sweep CF tunnels (push) Successful in 7s
main-red-watchdog / watchdog (push) Successful in 32s
Staging SaaS smoke (every 30 min) / Staging SaaS smoke (push) Successful in 4m19s
2026-05-17 02:52:25 +00:00
core-devops a01d1d8f86 ci(publish-runtime): add --verbose to twine upload to surface PyPI 403 reason body
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 3s
cascade-list-drift-gate / check (pull_request) Failing after 2s
CI / Detect changes (pull_request) Successful in 4s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 9s
E2E API Smoke Test / detect-changes (pull_request) Successful in 5s
E2E Chat / detect-changes (pull_request) Successful in 6s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 9s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 5s
lint-continue-on-error-tracking / lint-continue-on-error-tracking (pull_request) Successful in 1m17s
Lint curl status-code capture / Scan workflows for curl status-capture pollution (pull_request) Successful in 5s
Lint pre-flip continue-on-error / Verify continue-on-error flips have run-log proof (pull_request) Successful in 58s
lint-required-context-exists-in-bp / lint-required-context-exists-in-bp (pull_request) Successful in 1m4s
CI / Platform (Go) (pull_request) Successful in 4m55s
lint-required-no-paths / lint-required-no-paths (pull_request) Successful in 52s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 4s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 5s
gate-check-v3 / gate-check (pull_request) Successful in 3s
qa-review / approved (pull_request) Successful in 3s
security-review / approved (pull_request) Successful in 3s
sop-checklist / na-declarations (pull_request) N/A: (none)
sop-checklist / all-items-acked (pull_request) Successful in 3s
sop-tier-check / tier-check (pull_request) Successful in 3s
Lint workflow YAML (Gitea-1.22.6-hostile shapes) / Lint workflow YAML for Gitea-1.22.6-hostile shapes (pull_request) Successful in 1m1s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 1s
E2E Chat / E2E Chat (pull_request) Successful in 2s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 2s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 1s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 1s
CI / Canvas (Next.js) (pull_request) Successful in 6m9s
CI / Python Lint & Test (pull_request) Successful in 6m39s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
CI / all-required (pull_request) Successful in 6m42s
audit-force-merge / audit (pull_request) Successful in 5s
The Publish to PyPI step ran `twine upload` without --verbose. On an HTTP
403, twine's default output prints only the bare status ("Forbidden") and
discards PyPI Warehouse's human-readable response body, which carries the
actual rejection reason (e.g. project-scoped token mismatch, yanked-name
collision, account state). During the internal#469 0.1.1003 publish block
the missing reason body made root-cause diagnosis impossible without
performing another real upload to the live package.

Adding --verbose makes twine log the HTTP request/response metadata and
the Warehouse error body in CI. It does NOT echo the credential: the
PyPI token is passed via --password and sent only in the Basic-Auth
Authorization header, which twine's verbose output does not dump.

Minimal change: single added flag on the existing twine upload
invocation; no other steps or behavior touched.

Refs: internal#469

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 18:45:26 -07:00
hongming 3508d738a9 chore(runtime): remove crewai/deepagents/gemini-cli from the runtime catalog (internal#483) (#1385)
publish-workspace-server-image / build-and-push (push) Successful in 4m31s
Block internal-flavored paths / Block forbidden paths (push) Successful in 3s
CI / Detect changes (push) Successful in 8s
CI / Shellcheck (E2E scripts) (push) Successful in 12s
E2E API Smoke Test / detect-changes (push) Successful in 7s
E2E Chat / detect-changes (push) Successful in 6s
E2E Staging Canvas (Playwright) / detect-changes (push) Successful in 5s
E2E Staging SaaS (full lifecycle) / pr-validate (push) Successful in 22s
Handlers Postgres Integration / detect-changes (push) Successful in 2s
Harness Replays / detect-changes (push) Successful in 2s
Runtime PR-Built Compatibility / detect-changes (push) Successful in 3s
Secret scan / Scan diff for credential-shaped strings (push) Successful in 3s
CI / Platform (Go) (push) Successful in 4m53s
CI / Canvas (Next.js) (push) Successful in 6m22s
CI / Python Lint & Test (push) Successful in 6m33s
CI / all-required (push) Successful in 6m38s
E2E Staging SaaS (full lifecycle) / E2E Staging SaaS (push) Successful in 4m47s
publish-workspace-server-image / Production auto-deploy (push) Successful in 12m46s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (push) Successful in 1s
E2E API Smoke Test / E2E API Smoke Test (push) Successful in 38s
Harness Replays / Harness Replays (push) Successful in 2s
Handlers Postgres Integration / Handlers Postgres Integration (push) Successful in 1m13s
CI / Canvas Deploy Reminder (push) Successful in 1s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (push) Successful in 2m5s
E2E Chat / E2E Chat (push) Failing after 4m23s
main-red-watchdog / watchdog (push) Successful in 20s
gate-check-v3 / gate-check (push) Successful in 19s
Sweep stale Cloudflare DNS records / Sweep CF orphans (push) Successful in 14s
ci-required-drift / drift (push) Successful in 58s
Staging SaaS smoke (every 30 min) / Staging SaaS smoke (push) Successful in 5m45s
Sweep stale e2e-* orgs (staging) / Sweep e2e orgs (push) Successful in 4s
Sweep stale Cloudflare Tunnels / Sweep CF tunnels (push) Successful in 6s
gitea-merge-queue / queue (push) Successful in 6s
status-reaper / reap (push) Successful in 1m4s
Continuous synthetic E2E (staging) / Synthetic E2E against staging (push) Successful in 6m51s
Co-authored-by: hongming <hongmingwang@moleculesai.app>
Co-committed-by: hongming <hongmingwang@moleculesai.app>
2026-05-16 22:48:49 +00:00
devops-engineer ec664869b0 Merge pull request 'fix(inbox): drop self-delegation-echo rows from inbox poller' (#1348) from fix/inbox-self-echo into main
publish-workspace-server-image / build-and-push (push) Successful in 2m33s
Block internal-flavored paths / Block forbidden paths (push) Successful in 4s
CI / Detect changes (push) Successful in 5s
CI / Shellcheck (E2E scripts) (push) Successful in 12s
E2E API Smoke Test / detect-changes (push) Successful in 4s
E2E Chat / detect-changes (push) Successful in 4s
E2E Staging Canvas (Playwright) / detect-changes (push) Successful in 4s
Handlers Postgres Integration / detect-changes (push) Successful in 3s
publish-runtime-autobump / pr-validate (push) Successful in 30s
publish-runtime-autobump / bump-and-tag (push) Successful in 24s
CI / Platform (Go) (push) Successful in 4m40s
Runtime PR-Built Compatibility / detect-changes (push) Successful in 6s
Secret scan / Scan diff for credential-shaped strings (push) Successful in 2s
Ops Scripts Tests / Ops scripts (unittest) (push) Successful in 59s
CI / Canvas (Next.js) (push) Successful in 6m41s
CI / Python Lint & Test (push) Successful in 6m34s
CI / all-required (push) Successful in 6m7s
publish-workspace-server-image / Production auto-deploy (push) Successful in 10m54s
E2E API Smoke Test / E2E API Smoke Test (push) Successful in 2s
E2E Chat / E2E Chat (push) Successful in 1s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (push) Successful in 2s
CI / Canvas Deploy Reminder (push) Successful in 1s
Handlers Postgres Integration / Handlers Postgres Integration (push) Successful in 1m23s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (push) Successful in 1m54s
main-red-watchdog / watchdog (push) Successful in 25s
gate-check-v3 / gate-check (push) Successful in 26s
Sweep stale Cloudflare DNS records / Sweep CF orphans (push) Successful in 7s
ci-required-drift / drift (push) Successful in 1m2s
Staging SaaS smoke (every 30 min) / Staging SaaS smoke (push) Successful in 4m34s
Sweep stale e2e-* orgs (staging) / Sweep e2e orgs (push) Successful in 2s
Sweep stale Cloudflare Tunnels / Sweep CF tunnels (push) Successful in 3s
gitea-merge-queue / queue (push) Successful in 4s
status-reaper / reap (push) Successful in 57s
Continuous synthetic E2E (staging) / Synthetic E2E against staging (push) Successful in 5m53s
publish-runtime / publish (push) Failing after 2m4s
publish-runtime / cascade (push) Has been skipped
2026-05-16 21:09:20 +00:00
devops-engineer 8b11368656 ci: rerun CI on healthy host (load-era timing flake, no code change)
CI / Canvas Deploy Reminder (pull_request) Blocked by required conditions
E2E API Smoke Test / E2E API Smoke Test (pull_request) Blocked by required conditions
E2E Chat / E2E Chat (pull_request) Blocked by required conditions
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Blocked by required conditions
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Blocked by required conditions
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Blocked by required conditions
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 2s
CI / Detect changes (pull_request) Successful in 4s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 8s
E2E API Smoke Test / detect-changes (pull_request) Successful in 8s
E2E Chat / detect-changes (pull_request) Successful in 7s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 5s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 6s
publish-runtime-autobump / pr-validate (pull_request) Successful in 27s
publish-runtime-autobump / bump-and-tag (pull_request) Has been skipped
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 6s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 3s
lint-required-no-paths / lint-required-no-paths (pull_request) Successful in 53s
gate-check-v3 / gate-check (pull_request) Successful in 3s
qa-review / approved (pull_request) Failing after 3s
security-review / approved (pull_request) Successful in 3s
sop-checklist / all-items-acked (pull_request) Successful in 2s
sop-tier-check / tier-check (pull_request) Successful in 3s
Ops Scripts Tests / Ops scripts (unittest) (pull_request) Successful in 56s
CI / Platform (Go) (pull_request) Successful in 4m30s
CI / Canvas (Next.js) (pull_request) Successful in 6m19s
CI / Python Lint & Test (pull_request) Successful in 6m34s
CI / all-required (pull_request) Successful in 5m7s
audit-force-merge / audit (pull_request) Successful in 5s
PR#1348 (#190 self-echo fix) sole red = test_batch_fetcher_runs_submitted_rows_concurrently
in tests/test_inbox_uploads.py (2.6ms wall-clock overshoot, 0.2516s vs 0.25s) — a
load-induced timing flake, NOT in this PR's changed code (workspace/inbox.py
_is_self_echo_row). Host has recovered (load1 ~1.5, runner pool drained, throttle
PR#72 live). Empty commit = the only CI-rerun mechanism on Gitea 1.22.6
(reference_empty_commit_is_only_rerun_mechanism_on_1_22_6). Same tree, no code
change; CTO non-author-review waiver + mandatory retroactive core-security review
apply to the new head unchanged. internal#469 / #190.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 14:00:07 -07:00
core-be 6bfc1c83ea Merge origin/main into fix/inbox-self-echo (bring up to base, zero-conflict; #190 internal#469)
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 20s
CI / Detect changes (pull_request) Successful in 21s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 38s
E2E API Smoke Test / detect-changes (pull_request) Successful in 25s
E2E Chat / detect-changes (pull_request) Successful in 23s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 25s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 16s
publish-runtime-autobump / bump-and-tag (pull_request) Has been skipped
publish-runtime-autobump / pr-validate (pull_request) Successful in 1m11s
lint-required-no-paths / lint-required-no-paths (pull_request) Successful in 1m36s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 22s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 17s
gate-check-v3 / gate-check (pull_request) Successful in 20s
qa-review / approved (pull_request) Failing after 16s
security-review / approved (pull_request) Successful in 16s
sop-checklist / all-items-acked (pull_request) Successful in 16s
sop-tier-check / tier-check (pull_request) Successful in 17s
Ops Scripts Tests / Ops scripts (unittest) (pull_request) Successful in 1m44s
CI / Python Lint & Test (pull_request) Failing after 8m21s
CI / all-required (pull_request) Failing after 8m31s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 17s
E2E Chat / E2E Chat (pull_request) Successful in 13s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 8s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 6s
CI / Canvas (Next.js) (pull_request) Successful in 21m10s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 2m29s
CI / Platform (Go) (pull_request) Successful in 21m58s
CI / Canvas Deploy Reminder (pull_request) Has been cancelled
2026-05-16 12:53:09 -07:00
devops-engineer 2cb52615b0 Merge pull request 'infra(ci): route publish/deploy ship jobs to dedicated publish lane (internal#462)' (#1376) from infra/internal-462-publish-deploy-lane into main
publish-canvas-image / Build & push canvas image (push) Successful in 8m37s
publish-workspace-server-image / build-and-push (push) Successful in 14m26s
Block internal-flavored paths / Block forbidden paths (push) Successful in 18s
CI / Detect changes (push) Successful in 29s
CI / Shellcheck (E2E scripts) (push) Successful in 1m5s
E2E API Smoke Test / detect-changes (push) Successful in 36s
E2E Chat / detect-changes (push) Successful in 42s
E2E Staging Canvas (Playwright) / detect-changes (push) Successful in 50s
Handlers Postgres Integration / detect-changes (push) Successful in 56s
Lint curl status-code capture / Scan workflows for curl status-capture pollution (push) Successful in 48s
Runtime PR-Built Compatibility / detect-changes (push) Successful in 35s
Secret scan / Scan diff for credential-shaped strings (push) Successful in 20s
Lint workflow YAML (Gitea-1.22.6-hostile shapes) / Lint workflow YAML for Gitea-1.22.6-hostile shapes (push) Successful in 2m3s
lint-continue-on-error-tracking / lint-continue-on-error-tracking (push) Successful in 2m51s
CI / Python Lint & Test (push) Successful in 8m42s
E2E API Smoke Test / E2E API Smoke Test (push) Successful in 17s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (push) Successful in 13s
E2E Chat / E2E Chat (push) Successful in 32s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (push) Successful in 3m29s
CI / Canvas (Next.js) (push) Successful in 24m31s
CI / Platform (Go) (push) Successful in 27m22s
CI / all-required (push) Successful in 26m22s
Handlers Postgres Integration / Handlers Postgres Integration (push) Successful in 7m43s
publish-workspace-server-image / Production auto-deploy (push) Successful in 29m59s
Sweep stale Cloudflare DNS records / Sweep CF orphans (push) Successful in 19s
ci-required-drift / drift (push) Successful in 1m13s
CI / Canvas Deploy Reminder (push) Successful in 0s
Sweep stale Cloudflare Tunnels / Sweep CF tunnels (push) Successful in 3s
Sweep stale e2e-* orgs (staging) / Sweep e2e orgs (push) Successful in 3s
status-reaper / reap (push) Successful in 56s
gitea-merge-queue / queue (push) Successful in 2s
Staging SaaS smoke (every 30 min) / Staging SaaS smoke (push) Successful in 4m15s
Continuous synthetic E2E (staging) / Synthetic E2E against staging (push) Has started running
main-red-watchdog / watchdog (push) Successful in 22s
gate-check-v3 / gate-check (push) Successful in 32s
2026-05-16 19:47:21 +00:00
infra-sre 16957b7c15 infra(ci): route publish/deploy ship jobs to dedicated publish lane (internal#462)
CI / Canvas Deploy Reminder (pull_request) Blocked by required conditions
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Blocked by required conditions
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 33s
cascade-list-drift-gate / check (pull_request) Successful in 26s
CI / Detect changes (pull_request) Successful in 33s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 43s
E2E API Smoke Test / detect-changes (pull_request) Successful in 40s
E2E Chat / detect-changes (pull_request) Successful in 43s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 24s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 37s
Lint curl status-code capture / Scan workflows for curl status-capture pollution (pull_request) Successful in 23s
Lint pre-flip continue-on-error / Verify continue-on-error flips have run-log proof (pull_request) Successful in 2m5s
lint-continue-on-error-tracking / lint-continue-on-error-tracking (pull_request) Successful in 2m24s
lint-required-no-paths / lint-required-no-paths (pull_request) Successful in 1m49s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 27s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 22s
lint-required-context-exists-in-bp / lint-required-context-exists-in-bp (pull_request) Successful in 2m34s
qa-review / approved (pull_request) Failing after 30s
gate-check-v3 / gate-check (pull_request) Successful in 43s
security-review / approved (pull_request) Failing after 25s
sop-checklist / all-items-acked (pull_request) Successful in 25s
sop-tier-check / tier-check (pull_request) Successful in 23s
Lint workflow YAML (Gitea-1.22.6-hostile shapes) / Lint workflow YAML for Gitea-1.22.6-hostile shapes (pull_request) Successful in 2m1s
CI / Python Lint & Test (pull_request) Successful in 8m50s
CI / Canvas (Next.js) (pull_request) Successful in 24m27s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 14s
CI / Platform (Go) (pull_request) Successful in 26m33s
CI / all-required (pull_request) Successful in 26m46s
E2E Chat / E2E Chat (pull_request) Successful in 22s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 18s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 18s
audit-force-merge / audit (pull_request) Successful in 21s
Urgent prod-deploy publish builds currently FIFO-compete with ordinary
PR required-CI on the shared 20-runner pool. PR#1350's (CTO-reported
canvas-message-loss fix) production image build sat ~25min behind the
PR-CI backlog after merge, directly delaying a user-facing fix.

internal#462 comment 32299 + the already-merged operator-config
publish-lane scaffolding (config.publish.yaml + publish-lane-ensure.sh,
internal#394/#399) define a reserved `publish`/`release` sub-pool
(molecule-runner-publish-*, OUTSIDE the managed 1..20 range so it is
never auto-drained / recycled / drift-flagged). This retargets the 7
post-merge ship jobs across 5 workflows from `runs-on: ubuntu-latest`
to `runs-on: publish` so a merged fix's image build/push/deploy gets
reserved capacity and starts immediately, while PR-CI keeps the
general pool:

  - publish-workspace-server-image.yml: build-and-push, deploy-production
  - publish-canvas-image.yml: build-and-push
  - publish-runtime.yml: publish, cascade
  - redeploy-tenants-on-main.yml: redeploy
  - redeploy-tenants-on-staging.yml: redeploy

publish-runtime-autobump.yml is intentionally NOT moved: it is
pull_request-triggered (PR-CI by nature, a required status), not a
post-merge ship job — the lane reserves capacity for the ship path,
not for PR checks.

HARD MERGE PRECONDITION: this MUST NOT merge until the publish-lane
runners are registered and advertising the `publish` label. Targeting
an unregistered label queues jobs indefinitely with zero eligible
runners — the exact #599/#576 `docker`-label failure mode. Lane
registration is a GO-gated live-fleet mutation (publish-lane-ensure.sh
ALLOW_FLEET_MUTATION=1, requires explicit Hongming in-chat GO).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 11:49:10 -07:00
core-be 1549a9a2fd ci: rerun — runner-host ENOSPC infra failure on af25019 (no code change)
CI / Canvas Deploy Reminder (pull_request) Blocked by required conditions
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 31s
CI / Detect changes (pull_request) Successful in 33s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 58s
E2E API Smoke Test / detect-changes (pull_request) Successful in 51s
E2E Chat / detect-changes (pull_request) Successful in 53s
E2E Staging SaaS (full lifecycle) / E2E Staging SaaS (pull_request) Has been skipped
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 37s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 28s
E2E Staging SaaS (full lifecycle) / pr-validate (pull_request) Successful in 1m28s
Harness Replays / detect-changes (pull_request) Successful in 26s
publish-runtime-autobump / bump-and-tag (pull_request) Has been skipped
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 38s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 25s
publish-runtime-autobump / pr-validate (pull_request) Successful in 1m14s
lint-required-no-paths / lint-required-no-paths (pull_request) Successful in 1m53s
gate-check-v3 / gate-check (pull_request) Successful in 39s
qa-review / approved (pull_request) Failing after 24s
security-review / approved (pull_request) Failing after 25s
sop-checklist / all-items-acked (pull_request) Successful in 18s
sop-tier-check / tier-check (pull_request) Successful in 17s
Ops Scripts Tests / Ops scripts (unittest) (pull_request) Successful in 1m47s
CI / Python Lint & Test (pull_request) Successful in 9m22s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 17s
Harness Replays / Harness Replays (pull_request) Successful in 10s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 3m18s
CI / Canvas (Next.js) (pull_request) Successful in 26m38s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 3m48s
CI / Platform (Go) (pull_request) Successful in 28m20s
CI / all-required (pull_request) Successful in 27m57s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 8m6s
E2E Chat / E2E Chat (pull_request) Failing after 11m43s
Run 57610 Canvas(Next.js)+Platform(Go) failed solely on runner-host
disk exhaustion (ENOSPC / 'no space left on device' in /tmp/go-build*
and node write). PR#1348 touches only Python (workspace/inbox.py +
.gitea sop-checklist); zero Go/TSX. main HEAD is green on both jobs.
Disk since reclaimed (74%/58G free). Empty commit = only Gitea 1.22.6
rerun mechanism. Tree unchanged from af25019.
2026-05-16 11:44:20 -07:00
devops-engineer 6cfe76b6dd Merge pull request 'fix(workspace-server): persist poll-mode canvas user message synchronously before queued 200 (internal#471, sibling of #1347)' (#1350) from fix/canvas-user-message-poll-mode-sync-persist into main
core-devops/token-probe (informational) Token scope verification test
Block internal-flavored paths / Block forbidden paths (push) Successful in 30s
CI / Detect changes (push) Successful in 19s
E2E API Smoke Test / detect-changes (push) Successful in 29s
E2E Staging Canvas (Playwright) / detect-changes (push) Successful in 27s
CI / Shellcheck (E2E scripts) (push) Successful in 49s
E2E Chat / detect-changes (push) Successful in 30s
Handlers Postgres Integration / detect-changes (push) Successful in 19s
Harness Replays / detect-changes (push) Successful in 21s
Secret scan / Scan diff for credential-shaped strings (push) Successful in 20s
Runtime PR-Built Compatibility / detect-changes (push) Successful in 29s
CI / Python Lint & Test (push) Successful in 9m9s
publish-workspace-server-image / build-and-push (push) Successful in 14m54s
CI / Canvas (Next.js) (push) Successful in 23m18s
CI / Platform (Go) (push) Successful in 26m19s
CI / all-required (push) Successful in 25m55s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (push) Successful in 13s
Harness Replays / Harness Replays (push) Successful in 11s
E2E API Smoke Test / E2E API Smoke Test (push) Successful in 2m51s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (push) Successful in 3m42s
publish-workspace-server-image / Production auto-deploy (push) Successful in 2m23s
Handlers Postgres Integration / Handlers Postgres Integration (push) Successful in 7m59s
E2E Chat / E2E Chat (push) Failing after 11m13s
CI / Canvas Deploy Reminder (push) Successful in 8s
Continuous synthetic E2E (staging) / Synthetic E2E against staging (push) Has started running
Sweep stale e2e-* orgs (staging) / Sweep e2e orgs (push) Successful in 25s
main-red-watchdog / watchdog (push) Successful in 1m2s
infra-sre/test-post infra-sre test status post
infra-sre/timing-test timing test
gate-check-v3 / gate-check (push) Successful in 2m25s
Sweep stale Cloudflare DNS records / Sweep CF orphans (push) Successful in 42s
Staging SaaS smoke (every 30 min) / Staging SaaS smoke (push) Successful in 5m27s
ci-required-drift / drift (push) Successful in 2m33s
Sweep stale Cloudflare Tunnels / Sweep CF tunnels (push) Successful in 18s
2026-05-16 15:49:25 +00:00
core-be 1d29e9ea24 fix(handlers): prevent poll-mode sync-persist test from hanging CI
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 23s
CI / Detect changes (pull_request) Successful in 26s
E2E API Smoke Test / detect-changes (pull_request) Successful in 32s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 34s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 51s
Harness Replays / detect-changes (pull_request) Successful in 35s
E2E Chat / detect-changes (pull_request) Successful in 44s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 43s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 23s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 35s
qa-review / approved (pull_request) Failing after 33s
gate-check-v3 / gate-check (pull_request) Failing after 39s
security-review / approved (pull_request) Failing after 36s
sop-checklist / all-items-acked (pull_request) Successful in 36s
sop-tier-check / tier-check (pull_request) Successful in 29s
lint-required-no-paths / lint-required-no-paths (pull_request) Successful in 2m11s
Harness Replays / Harness Replays (pull_request) Successful in 7s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 28s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Failing after 1m28s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Failing after 1m48s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 26s
CI / Python Lint & Test (pull_request) Successful in 8m21s
E2E Chat / E2E Chat (pull_request) Failing after 7m10s
CI / Platform (Go) (pull_request) Successful in 12m49s
CI / Canvas (Next.js) (pull_request) Successful in 12m54s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
CI / all-required (pull_request) Successful in 13m5s
audit-force-merge / audit (pull_request) Successful in 36s
sqlmock.ExpectationsWereMet() hangs indefinitely when the expected INSERT
mock never fires. If the production code ever regresses to goAsync
(pre-fix shape), the handler returns before the INSERT fires, the mock
never fires, and ExpectationsWereMet() blocks for the full test/-suite
timeout — wedging the CI run with no diagnostic.

Fix: check expectations in a goroutine with a 2s hard timeout. When
the mock has fired (synchronous production code), ExpectationsWereMet()
returns <1ms and the select fires the `case err := <-expectDone` arm.
When the mock has NOT fired (async regression), the 2s timeout fires and
the test fails with a clear message instead of hanging.

Also reduce insertDelay from 150ms → 50ms. 50ms is ~50× the normal INSERT
latency and sufficient to prove synchronous blocking; the larger value
was adding unnecessary suite-level wall-clock under -race detection,
where mock delays are amplified by the instrumenter's goroutine overhead.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 14:47:07 +00:00
core-be af25019900 fix(inbox): add delegate_result exclusion to _is_self_echo_row
Harness Replays / detect-changes (pull_request) Waiting to run
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 26s
CI / Detect changes (pull_request) Successful in 34s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 44s
E2E API Smoke Test / detect-changes (pull_request) Successful in 37s
E2E Staging SaaS (full lifecycle) / E2E Staging SaaS (pull_request) Has been skipped
E2E Chat / detect-changes (pull_request) Successful in 38s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 35s
publish-runtime-autobump / bump-and-tag (pull_request) Has been skipped
Handlers Postgres Integration / detect-changes (pull_request) Successful in 30s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 38s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 49s
E2E Staging SaaS (full lifecycle) / pr-validate (pull_request) Successful in 1m44s
publish-runtime-autobump / pr-validate (pull_request) Successful in 1m24s
lint-required-no-paths / lint-required-no-paths (pull_request) Successful in 1m58s
qa-review / approved (pull_request) Failing after 45s
gate-check-v3 / gate-check (pull_request) Successful in 1m4s
security-review / approved (pull_request) Failing after 52s
sop-tier-check / tier-check (pull_request) Successful in 34s
Ops Scripts Tests / Ops scripts (unittest) (pull_request) Successful in 2m7s
CI / Python Lint & Test (pull_request) Successful in 9m25s
CI / Canvas (Next.js) (pull_request) Failing after 11m37s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
CI / all-required (pull_request) Failing after 12m3s
CI / Platform (Go) (pull_request) Failing after 21m3s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 23s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 4m3s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 7m14s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 7m24s
E2E Chat / E2E Chat (pull_request) Failing after 11m37s
sop-checklist / all-items-acked (pull_request) acked: 0/7 — missing: comprehensive-testing, local-postgres-e2e, staging-smoke, +4 — body-unfilled: comprehensive-testing, local-postgres-e2
Harness Replays / Harness Replays (pull_request) Has been cancelled
RFC #2829 PR-2 regression fix: rows with method="delegate_result"
are now excluded from the self-echo guard even when source_id
matches our workspace_id. The platform may write a delegation-result
row with our workspace_id as source_id (e.g. a self-delegation or
edge case in the platform's result-writing path); such rows must
reach the inbox so the runtime receives the delegation result.

Fixes regression vs PR #1346 where this guard was present.

Added test_is_self_echo_row_false_for_delegate_result regression pin.
All 9 self-echo tests pass locally.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 13:31:30 +00:00
core-be a92beb5d49 fix(workspace-server): persist poll-mode canvas user message synchronously before queued 200
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 25s
CI / Detect changes (pull_request) Successful in 22s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 53s
E2E API Smoke Test / detect-changes (pull_request) Successful in 30s
E2E Chat / detect-changes (pull_request) Successful in 37s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 38s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 35s
Harness Replays / detect-changes (pull_request) Successful in 38s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 40s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 50s
gate-check-v3 / gate-check (pull_request) Successful in 57s
qa-review / approved (pull_request) Successful in 51s
security-review / approved (pull_request) Failing after 48s
lint-required-no-paths / lint-required-no-paths (pull_request) Successful in 2m15s
sop-tier-check / tier-check (pull_request) Successful in 33s
CI / Platform (Go) (pull_request) Failing after 10m44s
CI / Canvas (Next.js) (pull_request) Failing after 10m20s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
CI / Python Lint & Test (pull_request) Successful in 9m18s
CI / all-required (pull_request) Failing after 8m57s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 31s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 19s
Harness Replays / Harness Replays (pull_request) Successful in 21s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 3m15s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Failing after 5m0s
E2E Chat / E2E Chat (pull_request) Failing after 12m29s
sop-checklist / all-items-acked (pull_request) acked: 3/7 — missing: comprehensive-testing, local-postgres-e2e, staging-smoke, +1 — body-unfilled: comprehensive-testing, local-postgres-e2
Sibling of #1347/internal#470 — the POLL-mode arm of the canvas
user-message data-loss bug Hongming reported ("i sometimes lose my own
message when i exit chat", 2026-05-16).

Hongming's tenant is entirely poll-mode (4 external workspaces, no URL —
verified empirically: every workspace returns the {delivery_mode:poll,
status:queued} short-circuit envelope), so #1347 (push-mode only,
persists AFTER the poll short-circuit) structurally cannot cover his
reported case. #1347's "poll-mode was never affected" framing is
overstated: logA2AReceiveQueued's durable activity_logs INSERT ran
inside h.goAsync(...) — a detached goroutine with no happens-before
barrier against the synthetic {status:queued} 200. The canvas sees the
send acknowledged while the row may still be racing; a workspace-server
restart / deploy / OOM / EC2 hibernation between the 200 and the
goroutine's commit loses the message permanently (chat-history reads
activity_logs; missing row = message gone on reopen). No fallback
either, unlike push-mode's legacy-INSERT path.

Fix: make the poll-mode ingest persist SYNCHRONOUS — committed before
the queued 200 — on a context.WithoutCancel context (parity with
persistUserMessageAtIngest). Best-effort preserved (LogActivity
logs+swallows INSERT errors, never blocks the send). Post-commit
broadcast still fires inside LogActivity (a missed WS event is not data
loss; the durable row is the truth chat-history re-reads on reopen).

TDD: a2a_poll_ingest_persist_test.go — deterministic RED (queued 200
returned in ~0.5ms, before the 150ms INSERT → DATA LOSS) → GREEN after
fix. Full internal/handlers + internal/messagestore suites green; vet
clean.

Refs: molecule-ai/internal#471 (tracking), molecule-ai/internal#470 (push-mode sibling, PR #1347)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 06:04:14 -07:00
core-be deeff950be fix(inbox): drop self-delegation-echo rows from inbox poller
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 21s
CI / Detect changes (pull_request) Successful in 25s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 59s
E2E API Smoke Test / detect-changes (pull_request) Successful in 34s
E2E Chat / detect-changes (pull_request) Successful in 28s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 16s
E2E Chat / E2E Chat (pull_request) Successful in 27s
CI / Python Lint & Test (pull_request) Successful in 8m51s
CI / Canvas (Next.js) (pull_request) Successful in 23m42s
CI / Platform (Go) (pull_request) Successful in 26m52s
CI / all-required (pull_request) Successful in 26m56s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
Internal #469: when a workspace delegates to a target that never picks up
the task, tool_delegate_task calls report_activity("a2a_receive", ...) which
POSTs to the platform with source_id = the sender's workspace UUID (spoof-
defense). The activity API exposes that row under type=a2a_receive, so the
inbox poller re-fetches it and message_from_activity sets peer_id = the
workspace's own UUID — the workspace sees its own delegation-failure echoed
back as if a peer had delegated to it.

Fix adds _is_self_echo_row(row, workspace_id) that returns True when
source_id == workspace_id, mirroring the existing _is_self_notify_row
pattern. The guard is wired into _poll_once after the self-notify check:
self-echo rows are skipped from the queue, the cursor still advances, and
the notification callback does not fire. The real delegate_result push path
(delegate_result method) is unaffected.

8 new tests cover the predicate (same-workspace, different-workspace,
None source, empty workspace_id, absent key) and the integrated poller
behavior (skipped from queue, cursor advances, no notification).

Live-repro confirmed on hongming.moleculesai.app prior to this fix.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 12:38:55 +00:00
infra-sre 50de2f6155 chore: re-trigger CI (infra-sre 09:47Z)
Block internal-flavored paths / Block forbidden paths (pull_request) Failing after 0s
CI / Detect changes (pull_request) Failing after 0s
CI / Platform (Go) (pull_request) Failing after 0s
CI / Canvas (Next.js) (pull_request) Failing after 0s
CI / Shellcheck (E2E scripts) (pull_request) Failing after 1s
CI / Python Lint & Test (pull_request) Failing after 0s
CI / all-required (pull_request) Failing after 0s
E2E API Smoke Test / detect-changes (pull_request) Failing after 0s
E2E Chat / detect-changes (pull_request) Failing after 0s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Failing after 0s
Handlers Postgres Integration / detect-changes (pull_request) Failing after 0s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
lint-required-no-paths / lint-required-no-paths (pull_request) Failing after 0s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Has been skipped
E2E Chat / E2E Chat (pull_request) Has been skipped
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Has been skipped
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Has been skipped
Runtime PR-Built Compatibility / detect-changes (pull_request) Failing after 0s
Secret scan / Scan diff for credential-shaped strings (pull_request) Failing after 0s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Has been skipped
Ops Scripts Tests / Ops scripts (unittest) (pull_request) Failing after 0s
gate-check-v3 / gate-check (pull_request) Failing after 0s
qa-review / approved (pull_request) Failing after 0s
security-review / approved (pull_request) Failing after 0s
sop-checklist / all-items-acked (pull_request) Failing after 0s
sop-tier-check / tier-check (pull_request) Failing after 0s
audit-force-merge / audit (pull_request) Waiting to run
2026-05-16 09:47:45 +00:00
Molecule AI Core Platform Lead 3461b86cba fix(sop-checklist): post na-declarations status for review-check.sh
Block internal-flavored paths / Block forbidden paths (pull_request) Failing after 0s
CI / Detect changes (pull_request) Failing after 0s
CI / Platform (Go) (pull_request) Failing after 0s
CI / Canvas (Next.js) (pull_request) Failing after 0s
CI / Shellcheck (E2E scripts) (pull_request) Failing after 0s
CI / Python Lint & Test (pull_request) Failing after 0s
CI / all-required (pull_request) Failing after 0s
E2E API Smoke Test / detect-changes (pull_request) Failing after 0s
E2E Chat / detect-changes (pull_request) Failing after 0s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Failing after 0s
Handlers Postgres Integration / detect-changes (pull_request) Failing after 0s
lint-required-no-paths / lint-required-no-paths (pull_request) Failing after 0s
Runtime PR-Built Compatibility / detect-changes (pull_request) Failing after 0s
Secret scan / Scan diff for credential-shaped strings (pull_request) Failing after 0s
Ops Scripts Tests / Ops scripts (unittest) (pull_request) Failing after 0s
gate-check-v3 / gate-check (pull_request) Failing after 0s
qa-review / approved (pull_request) Failing after 0s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
security-review / approved (pull_request) Failing after 0s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Has been skipped
sop-checklist / all-items-acked (pull_request) Failing after 0s
E2E Chat / E2E Chat (pull_request) Has been skipped
sop-tier-check / tier-check (pull_request) Failing after 0s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Has been skipped
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Has been skipped
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Has been skipped
2026-05-16 09:39:27 +00:00
24 changed files with 927 additions and 295 deletions
+80 -17
View File
@@ -65,6 +65,11 @@ class ApiError(RuntimeError):
pass
class MergePermissionError(ApiError):
"""Merge failed with a permanent permission error (403/404/405).
The queue should skip this PR and move to the next one."""
@dataclasses.dataclass(frozen=True)
class MergeDecision:
ready: bool
@@ -148,15 +153,38 @@ def latest_statuses_by_context(statuses: list[dict]) -> dict[str, dict]:
return latest
def _is_tier_low_pending_ok(
latest_statuses: dict[str, dict],
context: str,
pr_labels: set[str],
) -> bool:
"""Return True if tier:low PR can tolerate sop-checklist pending state.
Per sop-checklist-config.yaml tier_failure_mode, tier:low uses soft-fail:
sop-checklist posts state=pending when acks are satisfied (missing
manager/ceo acks are informational only). The queue should accept
pending instead of waiting for success.
"""
if "tier:low" not in pr_labels:
return False
if "sop-checklist" not in context:
return False
status = latest_statuses.get(context) or {}
return status_state(status) == "pending"
def required_contexts_green(
latest_statuses: dict[str, dict],
contexts: list[str],
pr_labels: set[str] | None = None,
) -> tuple[bool, list[str]]:
missing_or_bad: list[str] = []
for context in contexts:
status = latest_statuses.get(context)
state = status_state(status or {})
if state != "success":
if pr_labels and _is_tier_low_pending_ok(latest_statuses, context, pr_labels):
continue # tier:low soft-fail: accept pending sop-checklist
missing_or_bad.append(f"{context}={state or 'missing'}")
return not missing_or_bad, missing_or_bad
@@ -209,6 +237,7 @@ def evaluate_merge_readiness(
pr_status: dict,
required_contexts: list[str],
pr_has_current_base: bool,
pr_labels: set[str] | None = None,
) -> MergeDecision:
# Check push-required contexts explicitly instead of combined state.
# Combined state can be "failure" due to non-blocking jobs
@@ -228,7 +257,7 @@ def evaluate_merge_readiness(
# The required_contexts list is the authoritative gate — it includes only
# the checks that actually block merges.
latest = latest_statuses_by_context(pr_status.get("statuses") or [])
ok, missing_or_bad = required_contexts_green(latest, required_contexts)
ok, missing_or_bad = required_contexts_green(latest, required_contexts, pr_labels)
if not ok:
return MergeDecision(False, "wait", "required contexts not green: " + ", ".join(missing_or_bad))
return MergeDecision(True, "merge", "ready")
@@ -253,27 +282,32 @@ def get_combined_status(sha: str) -> dict:
_, combined = api("GET", f"/repos/{OWNER}/{NAME}/commits/{sha}/status")
if not isinstance(combined, dict):
raise ApiError(f"status for {sha} response not object")
# Fetch full statuses list; 200 covers >99% of real-world runs.
# The list is ordered ascending by id (oldest first) — callers must
# iterate in reverse to get the newest entry per context.
# Best-effort: large repos (main with 550+ statuses) may time out.
# On timeout, fall back to the statuses[] already in the combined
# response (usually 30 entries — enough for most PRs, enough for
# main's early push-required contexts).
combined_statuses: list[dict] = combined.get("statuses") or []
try:
_, all_statuses = api(
_, all_statuses_raw = api(
"GET",
f"/repos/{OWNER}/{NAME}/commits/{sha}/statuses",
query={"limit": "50"},
)
if isinstance(all_statuses, list):
combined["statuses"] = all_statuses
if isinstance(all_statuses_raw, list):
all_statuses: list[dict] = list(all_statuses_raw)
else:
all_statuses = []
except (ApiError, urllib.error.URLError, TimeoutError, OSError) as exc:
# URLError covers network-level failures (DNS, refused, timeout).
# TimeoutError and OSError cover socket-level timeouts.
sys.stderr.write(f"::warning::could not fetch full statuses list for {sha[:8]}: {exc}\n")
# Fall back to the statuses[] already in the combined response.
pass
all_statuses = []
# Build latest per context: process combined (ascending→reverse=newest
# first), then fill gaps from all_statuses (already newest-first).
latest: dict[str, dict] = {}
for status in reversed(sorted(combined_statuses, key=lambda s: s.get("id") or 0)):
ctx = status.get("context")
if isinstance(ctx, str) and ctx not in latest:
latest[ctx] = status
for status in all_statuses:
ctx = status.get("context")
if isinstance(ctx, str) and ctx not in latest:
latest[ctx] = status
combined["statuses"] = list(latest.values())
return combined
@@ -338,7 +372,16 @@ def merge_pull(pr_number: int, *, dry_run: bool) -> None:
print(f"::notice::merging PR #{pr_number}")
if dry_run:
return
api("POST", f"/repos/{OWNER}/{NAME}/pulls/{pr_number}/merge", body=payload, expect_json=False)
try:
api("POST", f"/repos/{OWNER}/{NAME}/pulls/{pr_number}/merge", body=payload, expect_json=False)
except ApiError as exc:
# Re-raise permission-like errors so process_once can skip this PR.
# 403 = no push access, 404 = repo/pr not found, 405 = not allowed.
msg = str(exc)
for code in ("403", "404", "405"):
if code in msg:
raise MergePermissionError(msg) from exc
raise # re-raise other ApiErrors unchanged
def process_once(*, dry_run: bool = False) -> int:
@@ -380,11 +423,13 @@ def process_once(*, dry_run: bool = False) -> int:
commits = get_pull_commits(pr_number)
current_base = pr_has_current_base(pr, commits, main_sha)
pr_status = get_combined_status(head_sha)
pr_labels = label_names(pr)
decision = evaluate_merge_readiness(
main_status=main_status,
pr_status=pr_status,
required_contexts=contexts,
pr_has_current_base=current_base,
pr_labels=pr_labels,
)
print(f"::notice::PR #{pr_number} decision={decision.action}: {decision.reason}")
@@ -407,7 +452,25 @@ def process_once(*, dry_run: bool = False) -> int:
"deferring to next tick"
)
return 0
merge_pull(pr_number, dry_run=dry_run)
try:
merge_pull(pr_number, dry_run=dry_run)
except MergePermissionError as exc:
# Permanent merge failure (HTTP 403/404/405). Post a comment so
# maintainers know why, then return 0 so this tick is done.
# The PR stays in the queue; future ticks can retry after the
# permission issue is resolved.
sys.stderr.write(f"::error::merge permission error for PR #{pr_number}: {exc}\n")
post_comment(
pr_number,
(
"merge-queue: merge failed with HTTP 405 'User not allowed to merge PR'. "
"No available token has Can-merge permission on this repo. "
"Fix: grant Can-merge to a token, or add a maintain/admin collaborator. "
"Skipping to next queued PR on next tick."
),
dry_run=dry_run,
)
return 0
return 0
return 0
+168 -25
View File
@@ -68,7 +68,7 @@ import sys
import urllib.error
import urllib.parse
import urllib.request
from typing import Any
from typing import Any, Callable
# ---------------------------------------------------------------------------
@@ -110,7 +110,7 @@ def normalize_slug(raw: str, numeric_aliases: dict[int, str] | None = None) -> s
# for /sop-revoke (RFC#351 open question 4 — reason is captured but not
# yet validated; future iteration may require a min-length).
_DIRECTIVE_RE = re.compile(
r"^[ \t]*/(sop-ack|sop-revoke)[ \t]+([A-Za-z0-9_\- ]+?)(?:[ \t]+(.*))?[ \t]*$",
r"^[ \t]*/(sop-ack|sop-revoke|sop-n/a)[ \t]+([A-Za-z0-9_\- ]+?)(?:[ \t]+(.*))?[ \t]*$",
re.MULTILINE,
)
@@ -118,19 +118,21 @@ _DIRECTIVE_RE = re.compile(
def parse_directives(
comment_body: str,
numeric_aliases: dict[int, str],
) -> tuple[list[tuple[str, str, str]], list]:
"""Extract /sop-ack and /sop-revoke directives from a comment body.
) -> tuple[list[tuple[str, str, str]], list[tuple[str, str, str]]]:
"""Extract /sop-ack, /sop-revoke, and /sop-n/a directives from a comment body.
Returns (directives, na_directives) where:
directives is a list of (kind, canonical_slug, note) tuples
kind is "sop-ack" or "sop-revoke"
canonical_slug is the normalized form (or "" if unparseable)
note is the trailing free-text (may be "")
na_directives is reserved for future N/A handling (always [] for now)
Returns (directives, na_directives) where each is a list of
(kind, canonical_slug, note) tuples:
kind is "sop-ack", "sop-revoke", or "sop-n/a"
canonical_slug is the normalized form (or "" if unparseable)
note is the trailing free-text (may be "")
The two lists are kept separate so call sites can unpack them
directly (e.g. directives, na_directives = parse_directives(...)).
"""
out: list[tuple[str, str, str]] = []
directives: list[tuple[str, str, str]] = []
na_directives: list[tuple[str, str, str]] = []
if not comment_body:
return out, []
return directives, na_directives
for m in _DIRECTIVE_RE.finditer(comment_body):
kind = m.group(1)
raw_slug = (m.group(2) or "").strip()
@@ -160,8 +162,12 @@ def parse_directives(
note_from_group = (m.group(3) or "").strip()
# If we collapsed multi-word slug into kebab and there's a
# trailing-text group too, append it.
out.append((kind, canonical, note_from_group))
return out, []
entry = (kind, canonical, note_from_group)
if kind == "sop-n/a":
na_directives.append(entry)
else:
directives.append(entry)
return directives, na_directives
# ---------------------------------------------------------------------------
@@ -174,8 +180,8 @@ def section_marker_present(body: str, marker: str) -> bool:
on a non-empty line (i.e. the author actually filled it in).
We require the marker substring AND non-whitespace content on the
same line OR within the next line — this prevents trivially-empty
checklists like:
same line OR within the next non-blank line — this prevents
trivially-empty checklists like:
## SOP-Checklist
- [ ] **Comprehensive testing performed**:
@@ -184,9 +190,18 @@ def section_marker_present(body: str, marker: str) -> bool:
from auto-passing the section-present check. The peer-ack is still
required, but answering with empty content is captured as a soft
finding via the section-present test alone.
NOTE: we scan forward through blank lines (the markdown-header pattern
is ## Header\\n\\ncontent) so that a header + blank-line + content
structure still satisfies the check. The backward checkbox fallback
catches inline markers without a preceding checkbox (mc#1099).
"""
if not body or not marker:
return False
# Strip trailing whitespace so the blank-line scan below can find
# content that appears on the very last line of the body (without
# being misled by a trailing \n or spaces).
body = body.rstrip()
body_lower = body.lower()
marker_lower = marker.lower()
idx = body_lower.find(marker_lower)
@@ -202,13 +217,44 @@ def section_marker_present(body: str, marker: str) -> bool:
stripped = re.sub(r"[\s\*:\-\[\]]+", "", line)
if stripped:
return True
# Fall through: check the NEXT line (multi-line answers).
next_line_end = body.find("\n", line_end + 1)
if next_line_end < 0:
next_line_end = len(body)
next_line = body[line_end + 1:next_line_end]
stripped_next = re.sub(r"[\s\*:\-\[\]]+", "", next_line)
return bool(stripped_next)
# Fall through: scan forward, skipping blank-only lines, until we find
# non-empty content or run out of body. Handles:
# ## Header ← marker line (empty after marker)
# ← blank line (skipped)
# - actual content ← found
pos = line_end
while True:
# Skip the current newline and any additional newlines (blank lines).
while pos < len(body) and body[pos] == "\n":
pos += 1
if pos >= len(body):
break
line_end = body.find("\n", pos)
if line_end < 0:
line_end = len(body)
line = body[pos:line_end]
stripped = re.sub(r"[\s\*:\-\[\]]+", "", line)
if stripped:
return True
pos = line_end
# Last resort: the marker may appear mid-sentence (e.g.
# **Memory/saved-feedback consulted**: No applicable...).
# Search backward within the CURRENT LINE only (not preceding lines)
# to find a checkbox on the same line before the marker text.
# mc#1099 follow-up: memory-consulted detection was failing because
# the checkbox was on the same line before the inline marker.
_CHECKBOX_RE = re.compile(r"- \[[ x\]]|<input", re.IGNORECASE)
line_start = body.rfind("\n", 0, idx) + 1 # 0 if no newline before idx
before = body[line_start:idx]
m = _CHECKBOX_RE.search(before)
if not m:
return False
# Require meaningful content between the checkbox and the marker text
# (markdown formatting like ** or * must also be stripped).
# If only whitespace/markdown chars remain, the checkbox line is empty.
between = before[m.end() :]
stripped_between = re.sub(r"[\s\*:#\[\]_\-]+", "", between)
return bool(stripped_between)
# ---------------------------------------------------------------------------
@@ -251,8 +297,7 @@ def compute_ack_state(
user = (c.get("user") or {}).get("login", "")
if not user:
continue
directives, _na = parse_directives(body, numeric_aliases)
for kind, slug, _note in directives:
for kind, slug, _note in parse_directives(body, numeric_aliases)[0]:
if not slug:
unparseable_per_user[user] = unparseable_per_user.get(user, 0) + 1
continue
@@ -304,6 +349,63 @@ def compute_ack_state(
}
# ---------------------------------------------------------------------------
# N/A-gate evaluation
# ---------------------------------------------------------------------------
def compute_na_state(
comments: list[dict[str, Any]],
author: str,
na_gates: dict[str, Any],
probe: Callable[[str, list[str]], list[str]],
) -> dict[str, dict[str, Any]]:
"""Evaluate which N/A gates have a valid declaration from a team member.
Returns dict[gate_name, dict] where each dict has:
declared: bool — at least one valid non-author team-member declared N/A
decl_ackers: list[str] — usernames who declared this gate N/A
rejected: dict with keys:
not_in_team: list[str] — users who tried but aren't in required teams
"""
# Build per-user latest N/A directive (most-recent wins per RFC#324).
latest_na: dict[str, tuple[str, str]] = {} # user → (gate, note)
for c in comments:
body = c.get("body", "") or ""
user = (c.get("user") or {}).get("login", "")
if not user:
continue
for kind, gate, note in parse_directives(body, {})[1]:
# [1] = na_directives only
if gate in na_gates:
latest_na[user] = (gate, note)
result: dict[str, dict[str, Any]] = {}
for gate, gate_cfg in na_gates.items():
result[gate] = {
"declared": False,
"decl_ackers": [],
"rejected": {"not_in_team": []},
}
decl_ackers: list[str] = []
not_in_team: list[str] = []
for user, (g, _note) in latest_na.items():
if g != gate:
continue
if user == author:
continue # authors cannot self-declare N/A
approved = probe(gate, [user])
if approved:
decl_ackers.append(user)
else:
not_in_team.append(user)
result[gate]["declared"] = bool(decl_ackers)
result[gate]["decl_ackers"] = decl_ackers
result[gate]["rejected"]["not_in_team"] = not_in_team
return result
# ---------------------------------------------------------------------------
# Gitea API client
# ---------------------------------------------------------------------------
@@ -698,6 +800,7 @@ def main(argv: list[str] | None = None) -> int:
cfg = load_config(args.config)
items: list[dict[str, Any]] = cfg["items"]
items_by_slug = {it["slug"]: it for it in items}
na_gates: dict[str, Any] = cfg.get("n/a_gates", {})
numeric_aliases = {
int(it["numeric_alias"]): it["slug"] for it in items if it.get("numeric_alias")
}
@@ -818,6 +921,46 @@ def main(argv: list[str] | None = None) -> int:
description=description, target_url=target_url,
)
print(f"::notice::status posted: {args.status_context}{state}")
# --- N/A gate status (RFC#324 §N/A follow-up) ---
# Post a separate status so review-check.sh can discover N/A declarations
# and waive the Gitea-approve requirement for that gate.
na_state: dict[str, dict[str, Any]] = {}
if na_gates:
na_state = compute_na_state(comments, author, na_gates, probe)
na_descs: list[str] = []
for gate, s in na_state.items():
if s["declared"]:
na_descs.append(gate)
decl = s["decl_ackers"]
rej = s["rejected"]["not_in_team"]
if decl:
print(f"::notice:: [N/A OK] {gate} — declared by {','.join(decl)}")
if rej:
print(
f"::notice:: [N/A REJ] {gate} — not-in-team: {','.join(rej)}",
file=sys.stderr,
)
na_desc = ", ".join(sorted(na_descs)) if na_descs else "(none)"
na_status_state = "success" if na_descs else "pending"
# review-check.sh reads the description to discover which gates are N/A.
# Include the gate names so it can grep for them.
na_description = f"N/A: {na_desc}" if na_descs else "N/A: (none)"
if not args.dry_run:
client.post_status(
args.owner, args.repo, head_sha,
state=na_status_state,
context="sop-checklist / na-declarations (pull_request)",
description=na_description,
target_url=target_url,
)
print(
f"::notice::na-declarations status → {na_status_state}: {na_description}"
)
# By default exit 0 — the POSTed status IS the gate, NOT the job
# conclusion. If the job exits 1 BP will see TWO failure signals
# (one from the job's auto-status, one from our POST), making the
@@ -118,3 +118,13 @@ def test_merge_decision_updates_stale_pr_before_merge():
assert decision.ready is False
assert decision.action == "update"
def test_MergePermissionError_inherits_from_ApiError():
assert issubclass(mq.MergePermissionError, mq.ApiError)
def test_MergePermissionError_message_preserved():
exc = mq.MergePermissionError("POST /merge -> HTTP 405: User not allowed")
assert "405" in str(exc)
assert "User not allowed" in str(exc)
@@ -551,3 +551,55 @@ class TestEndToEndAckFlow(unittest.TestCase):
if __name__ == "__main__":
unittest.main(verbosity=2)
# ---------------------------------------------------------------------------
# compute_na_state
# ---------------------------------------------------------------------------
class TestComputeNaState(unittest.TestCase):
"""Tests for /sop-n/a directive evaluation."""
def test_no_na_declarations(self):
cfg = sop.load_config(CONFIG_PATH)
na_gates = cfg.get("n/a_gates", {})
comments = []
na_state = sop.compute_na_state(comments, "alice", na_gates, lambda *_: [])
self.assertFalse(na_state["qa-review"]["declared"])
self.assertFalse(na_state["security-review"]["declared"])
def test_na_declared_by_authorized_user(self):
cfg = sop.load_config(CONFIG_PATH)
na_gates = cfg.get("n/a_gates", {})
comments = [_comment("bob", "/sop-n/a qa-review N/A: pure tooling change")]
na_state = sop.compute_na_state(comments, "alice", na_gates, lambda g, u: u)
self.assertTrue(na_state["qa-review"]["declared"])
self.assertEqual(na_state["qa-review"]["decl_ackers"], ["bob"])
def test_na_declared_by_unauthorized_user_rejected(self):
cfg = sop.load_config(CONFIG_PATH)
na_gates = cfg.get("n/a_gates", {})
comments = [_comment("mallory", "/sop-n/a qa-review N/A: not real team")]
na_state = sop.compute_na_state(comments, "alice", na_gates, lambda g, u: [])
self.assertFalse(na_state["qa-review"]["declared"])
self.assertEqual(na_state["qa-review"]["rejected"]["not_in_team"], ["mallory"])
def test_author_cannot_self_declare_na(self):
cfg = sop.load_config(CONFIG_PATH)
na_gates = cfg.get("n/a_gates", {})
comments = [_comment("alice", "/sop-n/a qa-review N/A: I am the author")]
na_state = sop.compute_na_state(comments, "alice", na_gates, lambda g, u: u)
self.assertFalse(na_state["qa-review"]["declared"])
def test_parse_directives_separates_na_from_ack(self):
directives, na_directives = sop.parse_directives(
"/sop-ack comprehensive-testing\n/sop-n/a qa-review N/A: no surface",
{},
)
self.assertEqual(len(directives), 1)
self.assertEqual(directives[0][0], "sop-ack")
self.assertEqual(len(na_directives), 1)
self.assertEqual(na_directives[0][0], "sop-n/a")
self.assertEqual(na_directives[0][1], "qa-review")
self.assertIn("no surface", na_directives[0][2])
+11 -7
View File
@@ -49,13 +49,17 @@ jobs:
# bp-exempt: post-merge image publication side effect; CI / all-required gates source changes.
build-and-push:
name: Build & push canvas image
# REVERTED (infra/revert-docker-runner-label): `runs-on: ubuntu-latest` restored.
# The `docker` label is not registered on any act_runner. `runs-on: [ubuntu-latest, docker]`
# causes jobs to queue indefinitely with zero eligible runners — strictly worse than the
# pre-#599 coin-flip (50% success rate). Once the `docker` label is registered on
# ≥2 runners, re-apply the fix from #599 (infra/docker-runner-label).
# See issue #576 + infra-lead pulse ~00:30Z.
runs-on: ubuntu-latest
# Dedicated publish/release lane (internal#462 / #394 / #399). Ship
# path (on: push:main, canvas/**) — reserved capacity so a merged
# canvas fix's image build never FIFO-queues behind PR required-CI.
# The `publish` label resolves ONLY to the molecule-runner-publish-*
# sub-pool (config.publish.yaml). HARD DEPENDENCY: this MUST land
# AFTER the publish-lane runners are registered/advertising `publish`
# — the earlier #599 `docker` label attempt queued indefinitely with
# zero eligible runners precisely because the label was targeted
# before any runner advertised it (see #576). The lane is registered
# in this rollout (internal#462) so the precondition holds.
runs-on: publish
# Phase 3 (RFC #219 §1): surface broken workflows without blocking.
# mc#774: pre-existing continue-on-error mask; root-fix and remove, do not renew silently.
continue-on-error: true
+8 -2
View File
@@ -66,7 +66,10 @@ concurrency:
jobs:
publish:
runs-on: ubuntu-latest
# Dedicated publish/release lane (internal#462 / #394 / #399). Ship
# path (on: push tag runtime-v*) — reserved capacity, never FIFO
# behind PR-CI. `publish` resolves only to molecule-runner-publish-*.
runs-on: publish
outputs:
version: ${{ steps.version.outputs.version }}
wheel_sha256: ${{ steps.wheel_hash.outputs.wheel_sha256 }}
@@ -159,6 +162,7 @@ jobs:
exit 1
fi
python -m twine upload \
--verbose \
--repository pypi \
--username __token__ \
--password "$PYPI_TOKEN" \
@@ -166,7 +170,9 @@ jobs:
cascade:
needs: publish
runs-on: ubuntu-latest
# Publish/release lane (internal#462) — downstream of the runtime
# publish ship job; keep it on the reserved lane too.
runs-on: publish
steps:
- name: Wait for PyPI to propagate the new version
env:
@@ -54,7 +54,14 @@ env:
jobs:
build-and-push:
runs-on: ubuntu-latest
# Dedicated publish/release lane (internal#462 / #394 / #399). This
# is a post-merge ship job (on: push:main) — it must NOT FIFO-compete
# with PR required-CI on the shared pool (PR#1350's prod image build
# was delayed ~25min this way). The `publish` label resolves ONLY to
# the reserved molecule-runner-publish-* sub-pool (config.publish.yaml,
# OUTSIDE the managed 1..20 range) so a merged fix's image build
# starts immediately while PR-CI keeps the general pool.
runs-on: publish
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
@@ -181,7 +188,9 @@ jobs:
name: Production auto-deploy
needs: build-and-push
if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
runs-on: ubuntu-latest
# Publish/release lane (internal#462) — production deploy of a merged
# fix; reserved capacity, never queued behind PR-CI.
runs-on: publish
timeout-minutes: 75
env:
CP_URL: ${{ vars.PROD_CP_URL || 'https://api.moleculesai.app' }}
@@ -68,7 +68,10 @@ jobs:
# bp-exempt: production redeploy is a side-effect workflow, not a merge gate.
redeploy:
if: ${{ github.event_name == 'workflow_dispatch' }}
runs-on: ubuntu-latest
# Dedicated publish/release lane (internal#462 / #394 / #399).
# Production tenant redeploy — a deploy action, reserved capacity so
# it never queues behind PR-CI. `publish` -> molecule-runner-publish-*.
runs-on: publish
# Phase 3 (RFC #219 §1): surface broken workflows without blocking.
# mc#774: pre-existing continue-on-error mask; root-fix and remove, do not renew silently.
continue-on-error: true
@@ -75,7 +75,10 @@ env:
jobs:
# bp-exempt: post-merge staging redeploy side effect; CI / all-required gates source changes.
redeploy:
runs-on: ubuntu-latest
# Dedicated publish/release lane (internal#462 / #394 / #399).
# Post-merge staging redeploy — a deploy action, reserved capacity.
# `publish` -> molecule-runner-publish-* sub-pool.
runs-on: publish
# Phase 3 (RFC #219 §1): surface broken workflows without blocking.
# mc#774: pre-existing continue-on-error mask; root-fix and remove, do not renew silently.
continue-on-error: true
+1 -4
View File
@@ -30,10 +30,7 @@
{"name": "openclaw", "repo": "molecule-ai/molecule-ai-workspace-template-openclaw", "ref": "main"},
{"name": "codex", "repo": "molecule-ai/molecule-ai-workspace-template-codex", "ref": "main"},
{"name": "langgraph", "repo": "molecule-ai/molecule-ai-workspace-template-langgraph", "ref": "main"},
{"name": "crewai", "repo": "molecule-ai/molecule-ai-workspace-template-crewai", "ref": "main"},
{"name": "autogen", "repo": "molecule-ai/molecule-ai-workspace-template-autogen", "ref": "main"},
{"name": "deepagents", "repo": "molecule-ai/molecule-ai-workspace-template-deepagents", "ref": "main"},
{"name": "gemini-cli", "repo": "molecule-ai/molecule-ai-workspace-template-gemini-cli", "ref": "main"}
{"name": "autogen", "repo": "molecule-ai/molecule-ai-workspace-template-autogen", "ref": "main"}
],
"org_templates": [
{"name": "molecule-dev", "repo": "molecule-ai/molecule-ai-org-template-molecule-dev", "ref": "main"},
@@ -0,0 +1,160 @@
package handlers
// Regression coverage for the POLL-mode arm of the canvas user-message
// data-loss bug (internal#470 sibling — tracked on internal#471).
//
// Bug (reported 2026-05-16 by CTO Hongming): "in canvas i sometimes lose
// my own message when i exit chat". The push-mode arm was fixed by
// #1347 (persistUserMessageAtIngest — a SYNCHRONOUS, before-dispatch,
// context.WithoutCancel INSERT). #1347's framing asserted "poll-mode
// workspaces were never affected — logA2AReceiveQueued already persists
// at ingest". That assertion is OVERSTATED.
//
// Hongming's tenant (slug `hongming`, org 2c940477-...) has 4 workspaces,
// ALL runtime=external with empty URL → ALL delivery_mode=poll (proven
// empirically: a benign A2A probe returns the synthetic
// {"delivery_mode":"poll","status":"queued"} envelope for every one).
// So his reported loss is the POLL path, NOT the push path #1347 fixes.
//
// Root cause (poll arm): the poll-mode short-circuit (a2a_proxy.go ~402)
// calls logA2AReceiveQueued and then IMMEDIATELY returns the synthetic
// 200 {status:"queued"} to the canvas. But logA2AReceiveQueued's durable
// INSERT runs inside h.goAsync(...) — a DETACHED goroutine with NO
// happens-before barrier against the HTTP response. The canvas sees 200
// ("message accepted") while the activity_logs row may not yet be — and,
// on a workspace-server restart / deploy / OOM / EC2 hibernation between
// the 200 and the goroutine's commit, NEVER will be — durable. There is
// also no fallback (unlike push-mode's legacy-INSERT fallback): a
// swallowed LogActivity error loses the message with only a log line.
// Chat-history reads activity_logs (postgres_store.go:165-187); a missing
// row = message gone on reopen. That is exactly Hongming's symptom.
//
// Fix (parity with push-mode): the poll-mode ingest persist of the
// canvas user message must be SYNCHRONOUS — committed before the queued
// 200 is returned — on a context.WithoutCancel derived context, so a
// client disconnect on chat-exit and a post-response restart cannot lose
// it. Behavior is never worse than today (best-effort; a persist error
// still returns queued).
//
// TEST DESIGN NOTE: sqlmock.ExpectationsWereMet() hangs indefinitely if
// the expected query never fires. We use a select+default+time.After
// pattern so the test FAILS fast (not hangs) when the production code
// regresses to async (the INSERT never fires before handler returns),
// while still returning promptly when all expectations are met. The
// insertDelay is kept small (50ms) to minimise suite-level timing
// impact under -race detection, where mock delays are amplified by
// the instrumenter's goroutine overhead.
import (
"bytes"
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/DATA-DOG/go-sqlmock"
"github.com/gin-gonic/gin"
)
// TestProxyA2A_PollMode_PersistsUserMessageSynchronouslyBeforeQueuedResponse
// is the defining contract: for a poll-mode workspace, the canvas user
// message MUST be durably INSERTed into activity_logs BEFORE the synthetic
// queued 200 is returned to the client — with NO reliance on a detached
// async goroutine completing later.
//
// The test proves the ordering by making the INSERT block briefly and
// asserting the handler does NOT return until the INSERT has completed.
// Pre-fix (INSERT in h.goAsync, response returned immediately) the
// handler returns ~instantly while the INSERT is still pending in the
// goroutine → the elapsed time is far below the injected INSERT delay and
// ExpectationsWereMet() is racy/unmet at return. Post-fix (synchronous
// persist before the queued response) the handler return is gated on the
// INSERT, so elapsed >= the injected delay and the expectation is met
// deterministically at return WITHOUT any waitAsyncForTest()/sleep.
func TestProxyA2A_PollMode_PersistsUserMessageSynchronouslyBeforeQueuedResponse(t *testing.T) {
mock := setupTestDB(t)
setupTestRedis(t)
broadcaster := newTestBroadcaster()
handler := NewWorkspaceHandler(broadcaster, nil, "http://localhost:8080", t.TempDir())
const wsID = "ws-poll-sync-persist"
// Keep delay small: -race detection amplifies mock delays significantly.
// A 50ms delay is sufficient to prove synchronous blocking (~50× the
// normal INSERT latency) without bloating the full ./... suite runtime.
const insertDelay = 50 * time.Millisecond
expectBudgetCheck(mock, wsID)
// lookupDeliveryMode → poll, triggering the short-circuit.
mock.ExpectQuery("SELECT delivery_mode FROM workspaces WHERE id").
WithArgs(wsID).
WillReturnRows(sqlmock.NewRows([]string{"delivery_mode"}).AddRow("poll"))
// workspace-name lookup inside logA2AReceiveQueued.
mock.ExpectQuery(`SELECT name FROM workspaces WHERE id`).
WithArgs(wsID).
WillReturnRows(sqlmock.NewRows([]string{"name"}).AddRow("Poll WS"))
// The durable user-message write. We delay it so a synchronous
// persist visibly gates the handler return; a detached-goroutine
// persist (pre-fix) does not. The fix must keep using
// context.WithoutCancel so this write survives a chat-exit cancel.
mock.ExpectExec("INSERT INTO activity_logs").
WillDelayFor(insertDelay).
WillReturnResult(sqlmock.NewResult(0, 1))
w := httptest.NewRecorder()
c, _ := gin.CreateTestContext(w)
c.Params = gin.Params{{Key: "id", Value: wsID}}
// callerID == "" (no X-Workspace-ID) → this is a canvas_user message,
// exactly Hongming's case.
body := `{"jsonrpc":"2.0","id":"poll-canvas-1","method":"message/send","params":{"message":{"role":"user","parts":[{"text":"my own message"}]}}}`
c.Request = httptest.NewRequest("POST", "/workspaces/"+wsID+"/a2a", bytes.NewBufferString(body))
c.Request.Header.Set("Content-Type", "application/json")
start := time.Now()
handler.ProxyA2A(c)
elapsed := time.Since(start)
// Defining assertion #1: the handler must not have returned the
// queued response before the durable INSERT committed. Pre-fix this
// fails (elapsed ≈ 0, INSERT still racing in goAsync).
if elapsed < insertDelay {
t.Fatalf("poll-mode queued response returned in %v, before the %v user-message INSERT — "+
"the message is not durable when the client/process goes away (DATA LOSS). "+
"Persist must be synchronous before the queued 200.", elapsed, insertDelay)
}
// Defining assertion #2: the durable write actually happened by the
// time the handler returned. ExpectionsWereMet() hangs indefinitely if
// the mock never fires (e.g. production code regressed to async),
// so we check it in a goroutine with a hard 2s timeout — fails fast
// (no CI hang) on regression while returning promptly on success.
expectDone := make(chan error, 1)
go func() { expectDone <- mock.ExpectationsWereMet() }()
select {
case err := <-expectDone:
if err != nil {
t.Fatalf("user-message INSERT was not durable at handler return (unmet sqlmock expectations): %v", err)
}
case <-time.After(2 * time.Second):
t.Fatalf("ExpectationsWereMet() hung for >2s — INSERT mock never fired. " +
"Likely cause: production code regressed logA2AReceiveQueued to goAsync " +
"(INSERT fires after handler returns, not before).")
}
// Sanity: still the correct poll-mode envelope + status.
if w.Code != http.StatusOK {
t.Fatalf("expected 200 (queued), got %d: %s", w.Code, w.Body.String())
}
var resp map[string]interface{}
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
t.Fatalf("response is not valid JSON: %v", err)
}
if resp["status"] != "queued" || resp["delivery_mode"] != "poll" {
t.Errorf("poll envelope changed: got status=%v delivery_mode=%v, want queued/poll",
resp["status"], resp["delivery_mode"])
}
}
@@ -504,25 +504,49 @@ func lookupDeliveryMode(ctx context.Context, workspaceID string) string {
// reads in PR 3 — that's how a poll-mode workspace receives inbound A2A
// without a public URL.
func (h *WorkspaceHandler) logA2AReceiveQueued(ctx context.Context, workspaceID, callerID string, body []byte, a2aMethod string) {
// DATA-LOSS FIX (internal#471 — poll-mode sibling of #1347/internal#470):
// this is the ONLY durable write of a poll-mode inbound message,
// including a canvas_user message (callerID == "") typed in the canvas
// chat. It MUST be SYNCHRONOUS and complete BEFORE the caller returns
// the synthetic {status:"queued"} 200 — otherwise the canvas sees the
// send acknowledged while the activity_logs row is still racing in a
// detached goroutine, and a workspace-server restart / deploy / OOM /
// EC2 hibernation between the 200 and the goroutine's commit loses the
// user's message permanently (chat-history reads activity_logs, so a
// missing row = message gone on reopen). Hongming's tenant is entirely
// poll-mode (4 external workspaces, no URL — verified empirically), so
// his reported loss is THIS path; #1347 (push-mode, persists AFTER the
// poll short-circuit) structurally cannot cover it.
//
// Mirrors persistUserMessageAtIngest's discipline:
// - context.WithoutCancel: a client disconnect on chat-exit (which
// cancels the inbound request ctx) MUST NOT abort this write.
// - SYNCHRONOUS (no goAsync): the row must be durable before the
// queued 200 is returned to the caller.
// - Best-effort: LogActivity already logs+swallows INSERT errors, so
// a hiccup never blocks or fails the user's send (behavior for
// that one request is never worse than the pre-fix async path).
// The post-commit broadcast still fires inside LogActivity; a missed
// WebSocket event is not data loss (the durable row is the truth the
// canvas re-reads on reopen).
insCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 30*time.Second)
defer cancel()
var wsName string
db.DB.QueryRowContext(ctx, `SELECT name FROM workspaces WHERE id = $1`, workspaceID).Scan(&wsName)
db.DB.QueryRowContext(insCtx, `SELECT name FROM workspaces WHERE id = $1`, workspaceID).Scan(&wsName)
if wsName == "" {
wsName = workspaceID
}
summary := a2aMethod + " → " + wsName + " (queued for poll)"
h.goAsync(func() {
logCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 30*time.Second)
defer cancel()
LogActivity(logCtx, h.broadcaster, ActivityParams{
WorkspaceID: workspaceID,
ActivityType: "a2a_receive",
SourceID: nilIfEmpty(callerID),
TargetID: &workspaceID,
Method: &a2aMethod,
Summary: &summary,
RequestBody: json.RawMessage(body),
Status: "ok",
})
LogActivity(insCtx, h.broadcaster, ActivityParams{
WorkspaceID: workspaceID,
ActivityType: "a2a_receive",
SourceID: nilIfEmpty(callerID),
TargetID: &workspaceID,
Method: &a2aMethod,
Summary: &summary,
RequestBody: json.RawMessage(body),
Status: "ok",
})
}
@@ -44,8 +44,8 @@ func NewWorkspaceImageService(docker *dockerclient.Client) *WorkspaceImageServic
// AllRuntimes is the canonical list mirroring docs/workspace-runtime-package.md.
// Update both when a new template is added.
var AllRuntimes = []string{
"claude-code", "langgraph", "crewai", "autogen",
"deepagents", "hermes", "gemini-cli", "openclaw",
"claude-code", "langgraph", "autogen",
"hermes", "openclaw",
}
// RefreshResult is the per-call outcome surfaced to HTTP callers AND logged
@@ -35,8 +35,8 @@ func insertMCPDelegationRow(ctx context.Context, db *sql.DB, workspaceID, target
})
_, err := db.ExecContext(ctx, `
INSERT INTO activity_logs (workspace_id, activity_type, method, source_id, target_id, summary, request_body, status)
VALUES ($1, 'delegation', 'delegate', $2, $3, $4, $5::jsonb, 'pending')
`, workspaceID, workspaceID, targetID, "Delegating to "+targetID, string(taskJSON))
VALUES ($1, 'delegation', 'delegate', $2, $3, $4, $5::jsonb, $6)
`, workspaceID, workspaceID, targetID, "Delegating to "+targetID, string(taskJSON), "pending")
return err
}
@@ -1,8 +1,12 @@
package handlers
import (
"context"
"encoding/json"
"testing"
"github.com/DATA-DOG/go-sqlmock"
"github.com/Molecule-AI/molecule-monorepo/platform/internal/db"
)
// ─────────────────────────────────────────────────────────────────────────────
@@ -191,3 +195,115 @@ func TestExtractA2AText_PriorityArtifactsOverMessage(t *testing.T) {
t.Errorf("artifacts should take priority: got %q, want %q", got, want)
}
}
// ─────────────────────────────────────────────────────────────────────────────
// insertMCPDelegationRow tests
// ─────────────────────────────────────────────────────────────────────────────
func TestInsertMCPDelegationRow_Success(t *testing.T) {
mockDB, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("failed to create sqlmock: %v", err)
}
prevDB := db.DB
db.DB = mockDB
t.Cleanup(func() { db.DB = prevDB; mockDB.Close() })
mock.ExpectExec(`INSERT INTO activity_logs`).
WithArgs("ws-src", "ws-src", "ws-tgt", "Delegating to ws-tgt", sqlmock.AnyArg(), "pending").
WillReturnResult(sqlmock.NewResult(0, 1))
err = insertMCPDelegationRow(context.Background(), mockDB, "ws-src", "ws-tgt", "del-123", "summarise the report")
if err != nil {
t.Errorf("unexpected error: %v", err)
}
if err := mock.ExpectationsWereMet(); err != nil {
t.Errorf("sqlmock expectations: %v", err)
}
}
func TestInsertMCPDelegationRow_DBError(t *testing.T) {
mockDB, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("failed to create sqlmock: %v", err)
}
prevDB := db.DB
db.DB = mockDB
t.Cleanup(func() { db.DB = prevDB; mockDB.Close() })
mock.ExpectExec(`INSERT INTO activity_logs`).
WithArgs("ws-src", "ws-src", "ws-tgt", sqlmock.AnyArg(), sqlmock.AnyArg(), "pending").
WillReturnError(context.DeadlineExceeded)
err = insertMCPDelegationRow(context.Background(), mockDB, "ws-src", "ws-tgt", "del-456", "check the logs")
if err == nil {
t.Error("expected error, got nil")
}
if err := mock.ExpectationsWereMet(); err != nil {
t.Errorf("sqlmock expectations: %v", err)
}
}
// ─────────────────────────────────────────────────────────────────────────────
// updateMCPDelegationStatus tests
// ─────────────────────────────────────────────────────────────────────────────
func TestUpdateMCPDelegationStatus_Success(t *testing.T) {
mockDB, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("failed to create sqlmock: %v", err)
}
prevDB := db.DB
db.DB = mockDB
t.Cleanup(func() { db.DB = prevDB; mockDB.Close() })
mock.ExpectExec(`UPDATE activity_logs`).
WithArgs("completed", "", "ws-src", "del-789").
WillReturnResult(sqlmock.NewResult(0, 1))
// Should not panic, should not error
updateMCPDelegationStatus(context.Background(), mockDB, "ws-src", "del-789", "completed", "")
if err := mock.ExpectationsWereMet(); err != nil {
t.Errorf("sqlmock expectations: %v", err)
}
}
func TestUpdateMCPDelegationStatus_WithErrorDetail(t *testing.T) {
mockDB, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("failed to create sqlmock: %v", err)
}
prevDB := db.DB
db.DB = mockDB
t.Cleanup(func() { db.DB = prevDB; mockDB.Close() })
mock.ExpectExec(`UPDATE activity_logs`).
WithArgs("failed", "timeout", "ws-src", "del-000").
WillReturnResult(sqlmock.NewResult(0, 1))
updateMCPDelegationStatus(context.Background(), mockDB, "ws-src", "del-000", "failed", "timeout")
if err := mock.ExpectationsWereMet(); err != nil {
t.Errorf("sqlmock expectations: %v", err)
}
}
func TestUpdateMCPDelegationStatus_DBError_LoggedNotReturned(t *testing.T) {
mockDB, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("failed to create sqlmock: %v", err)
}
prevDB := db.DB
db.DB = mockDB
t.Cleanup(func() { db.DB = prevDB; mockDB.Close() })
mock.ExpectExec(`UPDATE activity_logs`).
WithArgs("failed", sqlmock.AnyArg(), "ws-src", "del-abc").
WillReturnError(context.DeadlineExceeded)
// Function returns no value — error is logged, not propagated.
// Verify it does not panic.
updateMCPDelegationStatus(context.Background(), mockDB, "ws-src", "del-abc", "failed", "connection refused")
if err := mock.ExpectationsWereMet(); err != nil {
t.Errorf("sqlmock expectations: %v", err)
}
}
@@ -23,8 +23,8 @@ package models
// - claude-code: "sonnet" — Anthropic's CLI accepts the short
// name and resolves it via the operator's anthropic-oauth or
// ANTHROPIC_API_KEY chain.
// - everything else (hermes, langgraph, crewai, autogen, deepagents,
// codex, openclaw, gemini-cli, external, ""): a fully-qualified
// - everything else (hermes, langgraph, autogen, codex, openclaw,
// external, ""): a fully-qualified
// vendor:model slug that the universal MODEL_PROVIDER chain in
// molecule-core PR #247 can route via per-vendor required_env.
//
@@ -21,12 +21,9 @@ func TestDefaultModel(t *testing.T) {
// as a generic "unknown" failure.
{"hermes", "anthropic:claude-opus-4-7"},
{"langgraph", "anthropic:claude-opus-4-7"},
{"crewai", "anthropic:claude-opus-4-7"},
{"autogen", "anthropic:claude-opus-4-7"},
{"deepagents", "anthropic:claude-opus-4-7"},
{"codex", "anthropic:claude-opus-4-7"},
{"openclaw", "anthropic:claude-opus-4-7"},
{"gemini-cli", "anthropic:claude-opus-4-7"},
{"external", "anthropic:claude-opus-4-7"},
// Unknown / empty — fall through to universal default rather
@@ -190,7 +190,7 @@ func TestEnsureLocalImage_RepoNotFound(t *testing.T) {
opts.HTTPClient = srv.Client()
opts.remoteHeadSha = nil // exercise real HTTP path
_, err := ensureLocalImageWithOpts(context.Background(), "crewai", opts)
_, err := ensureLocalImageWithOpts(context.Background(), "hermes", opts)
if err == nil {
t.Fatalf("expected error, got nil")
}
@@ -35,6 +35,19 @@ import (
// drift-risk #6.
var ErrNoBackend = errors.New("provisioner: no backend configured (zero-valued receiver)")
// ErrUnresolvableRuntime is returned by selectImage when a workspace
// names a runtime that has no resolvable image (not in RuntimeImages and
// no operator-pinned cfg.Image). RFC internal#483 + security review 4269:
// previously such a request silently fell through to DefaultImage
// (langgraph) — a user asking for crewai would get a langgraph container
// with no signal. The CTO standing directive
// (feedback_platform_must_hardgate_base_contract) is fail-closed: a
// named-but-unresolvable runtime must reject with a structured,
// runtime-naming error so the existing provision-failed notify/log path
// surfaces it, NOT silently degrade. The genuinely-unspecified (empty)
// runtime is still a distinct, legitimate path that keeps DefaultImage.
var ErrUnresolvableRuntime = errors.New("provisioner: requested runtime has no resolvable image")
// RuntimeImages maps runtime names to their Docker image refs.
// Each standalone template repo publishes its image via the reusable
// publish-template-image workflow in molecule-ci on every main merge.
@@ -104,20 +117,33 @@ type WorkspaceConfig struct {
// selectImage resolves the final Docker image ref for a workspace. The handler
// layer is the source of truth — if it set cfg.Image (the digest-pinned form
// from runtime_image_pins, #2272), honor that. Otherwise fall back to the
// runtime→tag lookup in RuntimeImages (legacy `:latest` behavior). When the
// runtime isn't recognized either, fall back to DefaultImage so Start() still
// has something to hand Docker — surfacing a "No such image" later is more
// actionable than a silent "" panic in ContainerCreate.
func selectImage(cfg WorkspaceConfig) string {
// runtime→tag lookup in RuntimeImages (legacy `:latest` behavior).
//
// Fail-closed contract (RFC internal#483 / security review 4269 /
// feedback_platform_must_hardgate_base_contract): if the workspace NAMES a
// runtime that resolves to no image (not in RuntimeImages, no pinned
// cfg.Image), reject with ErrUnresolvableRuntime instead of silently
// substituting DefaultImage. Pre-fix, removing crewai/deepagents/gemini-cli
// from the catalog left those create requests silently provisioning a
// langgraph container — the user asked for crewai and got langgraph with no
// signal. The error propagates through Start → markProvisionFailed, which
// already broadcasts WorkspaceProvisionFailed and records the message.
//
// The genuinely-unspecified runtime (empty cfg.Runtime, e.g. an org template
// that doesn't pin one) is an intended distinct path and still resolves to
// DefaultImage — only a NAMED-but-unresolvable runtime is rejected.
func selectImage(cfg WorkspaceConfig) (string, error) {
if cfg.Image != "" {
return cfg.Image
return cfg.Image, nil
}
if cfg.Runtime != "" {
if img, ok := RuntimeImages[cfg.Runtime]; ok {
return img
return img, nil
}
return "", fmt.Errorf("%w: runtime %q (known runtimes: %v)",
ErrUnresolvableRuntime, cfg.Runtime, knownRuntimes)
}
return DefaultImage
return DefaultImage, nil
}
// Workspace-access constants for #65. Matches the CHECK constraint on
@@ -336,7 +362,15 @@ func (p *Provisioner) Start(ctx context.Context, cfg WorkspaceConfig) (string, e
env := buildContainerEnv(cfg)
image := selectImage(cfg)
image, imgErr := selectImage(cfg)
if imgErr != nil {
// Fail-closed: a named-but-unresolvable runtime must not silently
// become DefaultImage (RFC internal#483 / review 4269). The caller's
// error path (markProvisionFailed) broadcasts the failure + records
// the message so the canvas surfaces it.
log.Printf("Provisioner: refusing to start %s: %v", cfg.WorkspaceID, imgErr)
return "", imgErr
}
// Local-build mode (issue #63 / Task #194): when MOLECULE_IMAGE_REGISTRY
// is unset, the OSS contributor path skips the registry pull entirely
@@ -513,7 +513,10 @@ func TestWorkspaceConfig_ResetClaudeSessionFieldPresent(t *testing.T) {
// we lose the "one bad publish doesn't break every workspace" guarantee.
func TestSelectImage_PrefersExplicitImage(t *testing.T) {
pinned := "ghcr.io/molecule-ai/workspace-template-claude-code@sha256:3d6761a97ed07d7d33cfc19a8fbab81175d9d9179618d493dbc00c5f7ef076a3"
got := selectImage(WorkspaceConfig{Runtime: "claude-code", Image: pinned})
got, err := selectImage(WorkspaceConfig{Runtime: "claude-code", Image: pinned})
if err != nil {
t.Fatalf("selectImage with cfg.Image=pinned: unexpected error %v", err)
}
if got != pinned {
t.Errorf("selectImage with cfg.Image=pinned: got %q, want %q", got, pinned)
}
@@ -523,28 +526,46 @@ func TestSelectImage_PrefersExplicitImage(t *testing.T) {
// pin lookup deliberately bypassed via WORKSPACE_IMAGE_LOCAL_OVERRIDE).
// selectImage must use the legacy runtime→:latest map.
func TestSelectImage_FallsBackToRuntimeMap(t *testing.T) {
got := selectImage(WorkspaceConfig{Runtime: "claude-code", Image: ""})
got, err := selectImage(WorkspaceConfig{Runtime: "claude-code", Image: ""})
if err != nil {
t.Fatalf("selectImage with empty Image: unexpected error %v", err)
}
want := RuntimeImages["claude-code"]
if got != want {
t.Errorf("selectImage with empty Image: got %q, want %q", got, want)
}
}
// TestSelectImage_UnknownRuntimeFallsBackToDefault preserves today's
// behavior — an unrecognized runtime resolves to DefaultImage rather than
// "" so ContainerCreate gets a usable arg and surfaces a meaningful
// "No such image" error if the default itself is missing.
func TestSelectImage_UnknownRuntimeFallsBackToDefault(t *testing.T) {
got := selectImage(WorkspaceConfig{Runtime: "no-such-runtime"})
if got != DefaultImage {
t.Errorf("selectImage with unknown runtime: got %q, want DefaultImage %q", got, DefaultImage)
// TestSelectImage_NamedUnresolvableRuntimeRejects pins the fail-closed
// contract (RFC internal#483 / security review 4269 /
// feedback_platform_must_hardgate_base_contract): a NAMED runtime with no
// resolvable image must reject with ErrUnresolvableRuntime, NOT silently
// substitute DefaultImage. Pre-fix this returned langgraph — a user asking
// for a removed runtime (crewai/deepagents/gemini-cli) silently got a
// langgraph container. "crewai" is the concrete regression from the
// security finding.
func TestSelectImage_NamedUnresolvableRuntimeRejects(t *testing.T) {
for _, rt := range []string{"no-such-runtime", "crewai", "deepagents", "gemini-cli"} {
got, err := selectImage(WorkspaceConfig{Runtime: rt})
if !errors.Is(err, ErrUnresolvableRuntime) {
t.Errorf("selectImage(%q): got err %v, want ErrUnresolvableRuntime", rt, err)
}
if got != "" {
t.Errorf("selectImage(%q): got image %q, want \"\" on reject", rt, got)
}
if err != nil && !strings.Contains(err.Error(), rt) {
t.Errorf("selectImage(%q): error must name the offending runtime, got %v", rt, err)
}
}
}
// TestSelectImage_EmptyRuntimeFallsBackToDefault: same invariant for the
// no-runtime-supplied path (legacy callers / older handler code).
func TestSelectImage_EmptyRuntimeFallsBackToDefault(t *testing.T) {
got := selectImage(WorkspaceConfig{})
got, err := selectImage(WorkspaceConfig{})
if err != nil {
t.Fatalf("selectImage with zero cfg: unexpected error %v (empty runtime is a legitimate DefaultImage path)", err)
}
if got != DefaultImage {
t.Errorf("selectImage with zero cfg: got %q, want DefaultImage %q", got, DefaultImage)
}
@@ -808,7 +829,7 @@ func TestIsImageNotFoundErr(t *testing.T) {
{"nil", nil, false},
{"moby no such image", fmtErr(`Error response from daemon: No such image: workspace-template:openclaw`), true},
{"no such image lowercase", fmtErr(`error: no such image: foo:bar`), true},
{"image not found", fmtErr(`Error: image "workspace-template:crewai" not found`), true},
{"image not found", fmtErr(`Error: image "workspace-template:hermes" not found`), true},
{"generic not found without image", fmtErr(`container not found`), false},
{"unrelated error", fmtErr(`connection refused`), false},
{"permission denied", fmtErr(`permission denied`), false},
@@ -21,9 +21,6 @@ var knownRuntimes = []string{
"autogen",
"claude-code",
"codex",
"crewai",
"deepagents",
"gemini-cli",
"hermes",
"langgraph",
"openclaw",
@@ -53,8 +53,8 @@ func TestRuntimeImage_AllKnownRuntimes(t *testing.T) {
}
}
// Pin the count so adding a runtime requires explicit test acknowledgement.
if len(knownRuntimes) != 9 {
t.Errorf("knownRuntimes length = %d, want 9 (autogen, claude-code, codex, crewai, deepagents, gemini-cli, hermes, langgraph, openclaw)", len(knownRuntimes))
if len(knownRuntimes) != 6 {
t.Errorf("knownRuntimes length = %d, want 6 (autogen, claude-code, codex, hermes, langgraph, openclaw)", len(knownRuntimes))
}
}
+34 -43
View File
@@ -432,49 +432,40 @@ def _is_self_notify_row(row: dict[str, Any]) -> bool:
def _is_self_echo_row(row: dict[str, Any], workspace_id: str) -> bool:
"""Return True if ``row`` is this workspace's OWN delegation-outcome
bookkeeping surfacing back through the activity API as a fake inbound
peer message (the self-echo bug, confirmed live 2026-05-16).
"""Return True if ``row`` is a self-originated a2a_receive row.
Shape: when ``tool_delegate_task`` reports the outcome of a delegation
it fired, ``a2a_tools.report_activity`` POSTs an
``activity_type='a2a_receive'`` row to ``/workspaces/<self>/activity``
with ``source_id = WORKSPACE_ID`` (its own id — required by the
workspace-server's source_id spoof-defense, handlers/activity.go:549)
and ``method='message/send'``. The inbox poller then re-fetches that
row from ``?type=a2a_receive``; ``message_from_activity`` sets
``peer_id = source_id`` (non-empty), so ``to_dict`` classifies it as
``kind=peer_agent`` and the registry resolves the workspace's own id
to its display name (e.g. "mac laptop"). The agent sees its own
delegation-failure echoed back as if a peer delegated to it.
Internal #469: when a workspace delegates to a target that never picks
up the task, ``tool_delegate_task`` calls ``report_activity`` which
POSTs to the platform with source_id set to the *sender's* workspace
UUID (mandated by spoof-defense in workspace-server's a2a_proxy). The
activity API exposes that row under type=a2a_receive, so the inbox
poller re-fetches it. Without this guard the row is surfaced as
kind='peer_agent' with the workspace's own identity as peer_id —
the workspace sees its own delegation-failure echoed back as if a
peer had delegated to it.
A genuine inbound peer message ALWAYS has ``source_id`` set to a
DIFFERENT workspace than the one being polled — self-delegation is
already rejected upstream by two independent guards (the platform's
DelegationHandler.Delegate and the agent-side tool_delegate_task).
So ``source_id == workspace_id`` on the a2a_receive feed only ever
means a self-originated bookkeeping row, never a real peer.
The guard mirrors the existing _is_self_notify_row pattern: both
skip rows that would otherwise create spurious inbound signal. The
long-term fix (making the platform write a distinct activity_type
for agent-outbound rows) is tracked separately; this guard stays
because it only excludes rows the agent never wants.
Exception: ``method='delegate_result'`` rows ARE legitimately written
to the caller's own inbox with ``source_id == workspace_id`` — that
is the RFC #2829 PR-2 delegation-result delivery path
(handlers/delegation.go:pushDelegationResultToInbox). Those must keep
flowing to the agent, so they are explicitly NOT treated as echo.
``workspace_id`` must be non-empty — an empty-string workspace_id
(single-workspace legacy path) can never match a UUID source_id, so
the predicate is always False there, which is safe.
Matched on (source_id == workspace_id) AND (method != 'delegate_result')
together so the RFC #2829 result-push is preserved while the
report_activity self-echo is filtered.
RFC #2829 PR-2 note: rows with method="delegate_result" are excluded
from the self-echo guard even when source_id matches our workspace_id.
The platform may write a delegation-result row with source_id set to
our workspace_id (e.g. a self-delegation or edge case in the platform's
result-writing path). Such rows must reach the inbox so that
message_from_activity can surface them as peer_agent inbound and the
runtime receives the delegation result. Silently filtering them as
self-echo would break delegation result delivery.
"""
if not workspace_id:
# Single-workspace pollers may pass "" as the cursor key but the
# poller always knows its real workspace_id; an empty value means
# we can't safely compare — fall through (treat as not-echo) so
# this guard never silently drops a real message.
return False
source_id = row.get("source_id")
if not source_id or source_id != workspace_id:
return False
return row.get("method") != "delegate_result"
return row.get("source_id") == workspace_id and row.get("method") != "delegate_result"
def message_from_activity(row: dict[str, Any]) -> InboxMessage:
@@ -670,13 +661,13 @@ def _poll_once(
last_id = str(row.get("id", "")) or last_id
continue
if _is_self_echo_row(row, workspace_id):
# This workspace's OWN delegation-outcome bookkeeping row
# (report_activity wrote source_id=WORKSPACE_ID). Re-delivering
# it would surface the agent's own delegation-failure as a
# bogus inbound peer_agent message from itself (self-echo,
# 2026-05-16). Skip + advance the cursor past it, exactly like
# the self-notify guard above, so the next poll doesn't keep
# re-seeing it.
# Internal #469: tool_delegate_task writes its own a2a_receive
# row with source_id = this workspace's UUID (spoof-defense).
# The poll fetches it back as kind='peer_agent', making the
# workspace echo its own delegation-failure as an inbound from
# a phantom peer. Skip it — the real delegation-result path
# (delegate_result push) is separate and unaffected. Cursor
# still advances so the next poll doesn't re-seen this row.
last_id = str(row.get("id", "")) or last_id
continue
message = message_from_activity(row)
+145 -143
View File
@@ -495,6 +495,151 @@ def test_poll_once_skips_self_notify_rows(state: inbox.InboxState):
assert [m.activity_id for m in queue] == ["act-real"]
# ---------------------------------------------------------------------------
# _is_self_echo_row — internal #469 fix
# ---------------------------------------------------------------------------
#
# When a workspace delegates to a target that never picks up the task,
# tool_delegate_task calls report_activity("a2a_receive", ...) which POSTs
# to the platform with source_id set to the *sender's* workspace UUID
# (spoof-defense). The activity API returns that row under type=a2a_receive
# on the next poll, so message_from_activity sets peer_id = workspace's own
# UUID — the workspace sees its own delegation-failure as an inbound from
# a phantom peer. _is_self_echo_row guards against this.
#
# Internal #469 was live-reproduced on hongming.moleculesai.app 2026-05-16.
def test_is_self_echo_row_true_when_source_id_matches_workspace():
row = {"source_id": "ws-abc123", "method": "a2a_receive"}
assert inbox._is_self_echo_row(row, "ws-abc123") is True
def test_is_self_echo_row_false_when_source_id_differs():
"""A real peer agent (different workspace_id) must NOT be filtered."""
row = {"source_id": "ws-peer", "method": "a2a_receive"}
assert inbox._is_self_echo_row(row, "ws-1") is False
def test_is_self_echo_row_false_when_source_id_is_none():
"""Canvas-user inbound has no source_id — never an echo."""
row = {"source_id": None, "method": "a2a_receive"}
assert inbox._is_self_echo_row(row, "ws-1") is False
def test_is_self_echo_row_false_when_workspace_id_is_empty():
"""Single-workspace legacy path with empty workspace_id cannot
match a UUID source_id — predicate is always False, which is safe."""
row = {"source_id": "ws-abc123", "method": "a2a_receive"}
assert inbox._is_self_echo_row(row, "") is False
def test_is_self_echo_row_false_when_source_id_key_absent():
row = {"method": "a2a_receive"}
assert inbox._is_self_echo_row(row, "ws-1") is False
def test_is_self_echo_row_false_for_delegate_result():
"""RFC #2829 PR-2 regression pin: a row with source_id matching our
workspace_id but method=delegate_result must NOT be filtered as a
self-echo. The platform may write a delegation-result row with our
workspace_id as source_id; such rows must reach the inbox so the
runtime receives the delegation result. Silently filtering them would
break delegate_result delivery."""
row = {"source_id": "ws-1", "method": "delegate_result"}
assert inbox._is_self_echo_row(row, "ws-1") is False
def test_poll_once_skips_self_echo_rows(state: inbox.InboxState):
"""Internal #469 regression pin: a row with source_id matching our
workspace_id must NOT land in the inbox queue — it is our own
delegation-report echoing back, not a real peer inbound."""
rows = [
{
"id": "act-real-peer",
"source_id": "ws-peer",
"method": "a2a_receive",
"summary": None,
"request_body": {"parts": [{"type": "text", "text": "real peer inbound"}]},
"created_at": "2026-04-30T22:00:00Z",
},
{
"id": "act-self-echo",
"source_id": "ws-1",
"method": "a2a_receive",
"summary": "task result: target timed out",
"request_body": None,
"created_at": "2026-04-30T22:00:01Z",
},
]
resp = _make_response(200, rows)
p, _ = _patch_httpx(resp)
with p:
n = inbox._poll_once(state, "http://platform", "ws-1", {})
# Only the real peer inbound counted; self-echo silently dropped.
assert n == 1
queue = state.peek(10)
assert [m.activity_id for m in queue] == ["act-real-peer"]
assert queue[0].peer_id == "ws-peer"
def test_poll_once_advances_cursor_past_self_echo(state: inbox.InboxState):
"""Cursor must advance past self-echo rows even though we don't
enqueue them. Otherwise the next poll re-fetches the same self-echo
on every iteration, wasting requests and blocking real inbound."""
state.save_cursor("act-old")
rows = [
{
"id": "act-self-echo",
"source_id": "ws-1",
"method": "a2a_receive",
"summary": "task result: timeout",
"request_body": None,
"created_at": "2026-04-30T22:00:00Z",
},
]
resp = _make_response(200, rows)
p, _ = _patch_httpx(resp)
with p:
n = inbox._poll_once(state, "http://platform", "ws-1", {})
assert n == 0
assert state.peek(10) == []
# Cursor must move past the skipped row so we don't re-poll it.
assert state.load_cursor() == "act-self-echo"
def test_poll_once_self_echo_does_not_fire_notification(state: inbox.InboxState):
"""The notification callback (channel push to Claude Code etc.)
must not fire for self-echo rows. Same rationale as self-notify:
push-capable hosts would see the echo loop on the push channel."""
rows = [
{
"id": "act-self-echo",
"source_id": "ws-1",
"method": "a2a_receive",
"summary": "task result: timeout",
"request_body": None,
"created_at": "2026-04-30T22:00:00Z",
},
]
received: list[dict] = []
inbox.set_notification_callback(received.append)
try:
resp = _make_response(200, rows)
p, _ = _patch_httpx(resp)
with p:
inbox._poll_once(state, "http://platform", "ws-1", {})
finally:
inbox.set_notification_callback(None)
assert received == [], (
"self-echo rows must not surface as MCP notifications — "
"doing so re-creates the echo loop on push-capable hosts"
)
def test_poll_once_advances_cursor_past_self_notify(state: inbox.InboxState):
"""Cursor must advance past self-notify rows even though we don't
enqueue them. Otherwise the next poll re-fetches the same self-
@@ -552,149 +697,6 @@ def test_poll_once_self_notify_does_not_fire_notification(state: inbox.InboxStat
)
# ---------------------------------------------------------------------------
# _is_self_echo_row + the self-delegation-echo guard in _poll_once
# ---------------------------------------------------------------------------
#
# Confirmed live on hongming.moleculesai.app 2026-05-16: a workspace
# delegating to another workspace that never picks up the task hits the
# 300s polling timeout; tool_delegate_task then calls
# a2a_tools.report_activity("a2a_receive", ...), which POSTs an
# activity_type='a2a_receive' row to /workspaces/<self>/activity with
# source_id=WORKSPACE_ID (its own id — mandated by the workspace-server's
# source_id spoof-defense) and method='message/send'. The poller then
# re-fetches that row from ?type=a2a_receive and message_from_activity
# sets peer_id=source_id (non-empty) → kind=peer_agent, registry-resolved
# to the workspace's own display name ("mac laptop"). The agent sees its
# own delegation-failure echoed back as an inbound peer message.
#
# A genuine inbound peer always has source_id != polled workspace_id
# (self-delegation is blocked upstream by two guards). The ONLY
# legitimate source_id==self a2a_receive rows are RFC #2829 PR-2
# delegation-result pushes (method='delegate_result'), which must keep
# flowing. These tests pin both halves so a refactor that drops either
# breaks loudly.
def test_is_self_echo_row_true_for_own_report_activity_row():
"""report_activity writes source_id=WORKSPACE_ID, method='message/send'.
Polled back on the same workspace, that is a self-echo."""
row = {"source_id": "ws-1", "method": "message/send"}
assert inbox._is_self_echo_row(row, "ws-1") is True
# tasks/send is the other A2A method report_activity-class rows use.
row2 = {"source_id": "ws-1", "method": "tasks/send"}
assert inbox._is_self_echo_row(row2, "ws-1") is True
def test_is_self_echo_row_false_for_real_peer_inbound():
"""Real peer-agent message: source_id is a DIFFERENT workspace."""
row = {"source_id": "ws-peer-uuid", "method": "message/send"}
assert inbox._is_self_echo_row(row, "ws-1") is False
def test_is_self_echo_row_false_for_canvas_user():
"""Canvas-user message: source_id is None/empty (no peer)."""
assert inbox._is_self_echo_row({"source_id": None, "method": "message/send"}, "ws-1") is False
assert inbox._is_self_echo_row({"source_id": "", "method": "message/send"}, "ws-1") is False
def test_is_self_echo_row_false_for_delegate_result_push():
"""RFC #2829 PR-2 delegation-result delivery legitimately writes the
caller's own inbox with source_id==self + method='delegate_result'.
That MUST NOT be filtered — it's how the agent gets its delegation
results back on the durable async path."""
row = {"source_id": "ws-1", "method": "delegate_result"}
assert inbox._is_self_echo_row(row, "ws-1") is False
def test_is_self_echo_row_false_when_workspace_id_unknown():
"""Empty workspace_id → can't safely compare → never drop a message."""
assert inbox._is_self_echo_row({"source_id": "ws-1", "method": "message/send"}, "") is False
def test_poll_once_skips_self_echo_delegation_bookkeeping(state: inbox.InboxState):
"""Integrated regression pin for the 2026-05-16 self-echo incident:
a report_activity-shaped row (source_id == polled workspace_id,
method='message/send', activity_type=a2a_receive) must NOT land in
the inbox queue as a fake inbound peer message."""
rows = [
{
"id": "act-real-peer",
"source_id": "ws-other-peer",
"method": "message/send",
"summary": None,
"request_body": {"parts": [{"type": "text", "text": "real peer task"}]},
"created_at": "2026-05-16T10:00:00Z",
},
{
"id": "act-self-echo",
"source_id": "ws-1", # == the polled workspace_id below
"method": "message/send",
"summary": "hongming-pc failed: polling timeout after 300.0s "
"(delegation_id=16a70d38, last_status=dispatched); the platform",
"request_body": None,
"created_at": "2026-05-16T10:00:01Z",
},
]
resp = _make_response(200, rows)
p, _ = _patch_httpx(resp)
with p:
n = inbox._poll_once(state, "http://platform", "ws-1", {})
# Only the genuine peer message counted; the self-echo dropped.
assert n == 1
queue = state.peek(10)
assert [m.activity_id for m in queue] == ["act-real-peer"]
def test_poll_once_advances_cursor_past_self_echo(state: inbox.InboxState):
"""Cursor must advance past the skipped self-echo row so the next
poll doesn't re-fetch it forever (the same invariant as self-notify)."""
state.save_cursor("act-old")
rows = [
{
"id": "act-self-echo",
"source_id": "ws-1",
"method": "message/send",
"summary": "ws-child failed: polling timeout after 300.0s",
"request_body": None,
"created_at": "2026-05-16T10:00:00Z",
},
]
resp = _make_response(200, rows)
p, _ = _patch_httpx(resp)
with p:
n = inbox._poll_once(state, "http://platform", "ws-1", {})
assert n == 0
assert state.peek(10) == []
assert state.load_cursor() == "act-self-echo"
def test_poll_once_delegate_result_push_still_delivered(state: inbox.InboxState):
"""RFC #2829 PR-2 delegation-result push (source_id==self,
method='delegate_result') must STILL reach the agent — the self-echo
guard must not over-filter the legitimate result-delivery path."""
rows = [
{
"id": "act-delegate-result",
"source_id": "ws-1",
"method": "delegate_result",
"summary": "Delegation result delivered",
"request_body": {"parts": [{"type": "text", "text": "peer's answer"}]},
"created_at": "2026-05-16T10:00:00Z",
},
]
resp = _make_response(200, rows)
p, _ = _patch_httpx(resp)
with p:
n = inbox._poll_once(state, "http://platform", "ws-1", {})
assert n == 1
queue = state.peek(10)
assert [m.activity_id for m in queue] == ["act-delegate-result"]
def test_start_poller_thread_is_daemon(state: inbox.InboxState):
"""Daemon flag is required so the poller dies with the parent
process; a non-daemon poller would leak across `claude` restarts