molecule-ai/molecule-core
52
0
Fork 2
Code Issues 184 Pull Requests 27 Actions 18 Packages Projects Releases Wiki Activity

[main-red] molecule-ai/molecule-core: b565ded984 #2786

New Issue
Closed
opened 2026-06-13 23:07:16 +00:00 by gitea-actions · 1 comment
gitea-actions bot commented 2026-06-13 23:07:16 +00:00
Copy Link
Copy Source

Main is RED on molecule-ai/molecule-core at b565ded984

Commit: https://git.moleculesai.app/molecule-ai/molecule-core/commit/b565ded9845c778e015ace2825385910c3ae05d1

Auto-filed by .gitea/workflows/main-red-watchdog.yml (Option C of the main-never-red directive). Per feedback_no_such_thing_as_flakes + feedback_fix_root_not_symptom: investigate the root cause; do NOT revert as a reflex. The watchdog itself never reverts.

Failed status contexts

  • E2E Chat / E2E Chat (push) — failure → logs
    • Failing after 4m57s

Resolution path

  1. Read the failed logs (links above).
  2. If reproducible locally, fix forward in a PR targeting main.
  3. If the failure is a real flake — STOP. Per feedback_no_such_thing_as_flakes, intermittent failures are real bugs. Investigate to root cause; do not mark as flake.
  4. If the failure is blocking unrelated work for >1 hour, file a follow-up issue and assign someone. Do NOT revert without a human GO per feedback_prod_apply_needs_hongming_chat_go (branch protection is a prod surface).

Debug

{
  "all_contexts": [
    {
      "context": "Harness Replays / Harness Replays (push)",
      "state": "success"
    },
    {
      "context": "CI / Detect changes (push)",
      "state": "success"
    },
    {
      "context": "E2E Chat / detect-changes (push)",
      "state": "success"
    },
    {
      "context": "E2E Staging Canvas (Playwright) / detect-changes (push)",
      "state": "success"
    },
    {
      "context": "CI / Canvas (Next.js) (push)",
      "state": "success"
    },
    {
      "context": "lint-no-coe-on-required / lint-no-coe-on-required (push)",
      "state": "success"
    },
    {
      "context": "CI / Canvas Deploy Status (push)",
      "state": "success"
    },
    {
      "context": "E2E Staging Canvas (Playwright) / Canvas tabs E2E (push)",
      "state": "success"
    },
    {
      "context": "E2E API Smoke Test / detect-changes (push)",
      "state": "success"
    },
    {
      "context": "E2E Peer Visibility (literal MCP list_peers) / E2E Peer Visibility (local) (push)",
      "state": "success"
    },
    {
      "context": "Local Provision Lifecycle E2E / Local Provision Lifecycle E2E (stub) (push)",
      "state": "success"
    },
    {
      "context": "Handlers Postgres Integration / Handlers Postgres Integration (push)",
      "state": "success"
    },
    {
      "context": "Local Provision Lifecycle E2E / Local Provision Lifecycle E2E (real image + MiniMax LLM, advisory) (push)",
      "state": "success"
    },
    {
      "context": "CI / Shellcheck (E2E scripts) (push)",
      "state": "success"
    },
    {
      "context": "E2E API Smoke Test / E2E API Smoke Test (push)",
      "state": "success"
    },
    {
      "context": "CI / Platform (Go) (push)",
      "state": "success"
    },
    {
      "context": "publish-workspace-server-image / build-and-push (push)",
      "state": "success"
    },
    {
      "context": "CI / all-required (push)",
      "state": "success"
    },
    {
      "context": "E2E Chat / E2E Chat (push)",
      "state": "failure"
    },
    {
      "context": "E2E Staging External Runtime / E2E Staging External Runtime (push)",
      "state": "success"
    },
    {
      "context": "E2E Peer Visibility (literal MCP list_peers) / E2E Peer Visibility (push)",
      "state": "success"
    },
    {
      "context": "publish-workspace-server-image / Production auto-deploy (push)",
      "state": "success"
    },
    {
      "context": "E2E Staging SaaS (full lifecycle) / pr-validate (push)",
      "state": "pending"
    },
    {
      "context": "E2E Staging SaaS (full lifecycle) / E2E Staging SaaS (push)",
      "state": "pending"
    },
    {
      "context": "E2E Staging SaaS (full lifecycle) / E2E Staging Platform Boot (push)",
      "state": "pending"
    },
    {
      "context": "E2E Staging SaaS (full lifecycle) / E2E Staging Concierge user_tasks (push)",
      "state": "pending"
    },
    {
      "context": "E2E Staging SaaS (full lifecycle) / E2E Staging Workspace Requests (core#2606) (push)",
      "state": "pending"
    },
    {
      "context": "E2E Staging SaaS (full lifecycle) / E2E Staging Concierge (compile+skip) (push)",
      "state": "pending"
    },
    {
      "context": "E2E Staging SaaS (full lifecycle) / E2E Staging Concierge Platform Agent (push)",
      "state": "pending"
    },
    {
      "context": "E2E Staging SaaS (full lifecycle) / E2E Staging Concierge Creates Workspace (push)",
      "state": "pending"
    }
  ],
  "branch": "main",
  "combined_state": "failure",
  "failed_contexts": [
    "E2E Chat / E2E Chat (push)"
  ],
  "recheck_combined_state": "failure",
  "recheck_failed_contexts": [
    "E2E Chat / E2E Chat (push)"
  ],
  "sha": "b565ded9845c778e015ace2825385910c3ae05d1"
}

This issue is idempotent: the watchdog runs hourly at :05 and edits this body in place. When main returns to green, the watchdog will close this issue automatically with a "main returned to green" comment.

# Main is RED on `molecule-ai/molecule-core` at `b565ded984` Commit: <https://git.moleculesai.app/molecule-ai/molecule-core/commit/b565ded9845c778e015ace2825385910c3ae05d1> Auto-filed by `.gitea/workflows/main-red-watchdog.yml` (Option C of the [main-never-red directive](https://git.moleculesai.app/molecule-ai/molecule-core/issues/420)). Per `feedback_no_such_thing_as_flakes` + `feedback_fix_root_not_symptom`: investigate the root cause; do NOT revert as a reflex. The watchdog itself never reverts. ## Failed status contexts - **E2E Chat / E2E Chat (push)** — `failure` → [logs](/molecule-ai/molecule-core/actions/runs/361017/jobs/491799) - Failing after 4m57s ## Resolution path 1. Read the failed logs (links above). 2. If reproducible locally, fix forward in a PR targeting `main`. 3. If the failure is a real flake — STOP. Per `feedback_no_such_thing_as_flakes`, intermittent failures are real bugs. Investigate to root cause; do not mark as flake. 4. If the failure is blocking unrelated work for >1 hour, file a follow-up issue and assign someone. Do NOT revert without a human GO per `feedback_prod_apply_needs_hongming_chat_go` (branch protection is a prod surface). ## Debug ```json { "all_contexts": [ { "context": "Harness Replays / Harness Replays (push)", "state": "success" }, { "context": "CI / Detect changes (push)", "state": "success" }, { "context": "E2E Chat / detect-changes (push)", "state": "success" }, { "context": "E2E Staging Canvas (Playwright) / detect-changes (push)", "state": "success" }, { "context": "CI / Canvas (Next.js) (push)", "state": "success" }, { "context": "lint-no-coe-on-required / lint-no-coe-on-required (push)", "state": "success" }, { "context": "CI / Canvas Deploy Status (push)", "state": "success" }, { "context": "E2E Staging Canvas (Playwright) / Canvas tabs E2E (push)", "state": "success" }, { "context": "E2E API Smoke Test / detect-changes (push)", "state": "success" }, { "context": "E2E Peer Visibility (literal MCP list_peers) / E2E Peer Visibility (local) (push)", "state": "success" }, { "context": "Local Provision Lifecycle E2E / Local Provision Lifecycle E2E (stub) (push)", "state": "success" }, { "context": "Handlers Postgres Integration / Handlers Postgres Integration (push)", "state": "success" }, { "context": "Local Provision Lifecycle E2E / Local Provision Lifecycle E2E (real image + MiniMax LLM, advisory) (push)", "state": "success" }, { "context": "CI / Shellcheck (E2E scripts) (push)", "state": "success" }, { "context": "E2E API Smoke Test / E2E API Smoke Test (push)", "state": "success" }, { "context": "CI / Platform (Go) (push)", "state": "success" }, { "context": "publish-workspace-server-image / build-and-push (push)", "state": "success" }, { "context": "CI / all-required (push)", "state": "success" }, { "context": "E2E Chat / E2E Chat (push)", "state": "failure" }, { "context": "E2E Staging External Runtime / E2E Staging External Runtime (push)", "state": "success" }, { "context": "E2E Peer Visibility (literal MCP list_peers) / E2E Peer Visibility (push)", "state": "success" }, { "context": "publish-workspace-server-image / Production auto-deploy (push)", "state": "success" }, { "context": "E2E Staging SaaS (full lifecycle) / pr-validate (push)", "state": "pending" }, { "context": "E2E Staging SaaS (full lifecycle) / E2E Staging SaaS (push)", "state": "pending" }, { "context": "E2E Staging SaaS (full lifecycle) / E2E Staging Platform Boot (push)", "state": "pending" }, { "context": "E2E Staging SaaS (full lifecycle) / E2E Staging Concierge user_tasks (push)", "state": "pending" }, { "context": "E2E Staging SaaS (full lifecycle) / E2E Staging Workspace Requests (core#2606) (push)", "state": "pending" }, { "context": "E2E Staging SaaS (full lifecycle) / E2E Staging Concierge (compile+skip) (push)", "state": "pending" }, { "context": "E2E Staging SaaS (full lifecycle) / E2E Staging Concierge Platform Agent (push)", "state": "pending" }, { "context": "E2E Staging SaaS (full lifecycle) / E2E Staging Concierge Creates Workspace (push)", "state": "pending" } ], "branch": "main", "combined_state": "failure", "failed_contexts": [ "E2E Chat / E2E Chat (push)" ], "recheck_combined_state": "failure", "recheck_failed_contexts": [ "E2E Chat / E2E Chat (push)" ], "sha": "b565ded9845c778e015ace2825385910c3ae05d1" } ``` _This issue is idempotent: the watchdog runs hourly at `:05` and edits this body in place. When `main` returns to green, the watchdog will close this issue automatically with a "main returned to green" comment._
agent-researcher commented 2026-06-13 23:27:19 +00:00
Member
Copy Link
Copy Source

MECHANISM: #2786 is a code/harness regression in E2E Chat, not staging infra. The shared fixture creates runtime: "external" workspaces (canvas/e2e/fixtures/chat-seed.ts:31-43) but never forces delivery_mode='push'; registry rules explicitly default external+empty delivery mode to poll (workspace-server/internal/handlers/registry_test.go:2155-2162), and the proxy short-circuits poll targets to {status:"queued"} without dispatching to the echo URL (workspace-server/internal/handlers/a2a_proxy.go:554-596). Desktop tests then wait for inline echo text (canvas/e2e/chat-desktop.spec.ts:70-77, :80-86, :101-117) that never arrives. The newly wired chat-separation spec also has two harness defects: it defaults API calls to http://localhost:8080 (canvas/e2e/chat-separation.spec.ts:11-12) even though the workflow exports only E2E_PLATFORM_URL/PLAYWRIGHT_BASE_URL with ephemeral ports (.gitea/workflows/e2e-chat.yml:801-803), and its history seeder inserts into nonexistent chat_messages (canvas/e2e/fixtures/chat-seed.ts:156-178) while chat history is backed by activity_logs/MessageStore (workspace-server/internal/handlers/chat_history.go:3-12).

EVIDENCE: Current main still reproduces after #2776: commit de7b35ed has E2E Chat / E2E Chat red in job 492079. The log shows 24 tests executed, then desktop echo waits fail (chat-desktop.spec.ts:77, excerpt: Expected: visible) after successful /workspaces/:id/a2a 200s; that is the poll queued path being treated as a push echo. The same job shows the separation API tests failing with connect ECONNREFUSED ::1:8080, matching the hard-coded default, and the data-flow test fails with relation "chat_messages" does not exist, matching the obsolete seed table. The culprit cluster is #2780 (aebdc6c1 wired the spec into E2E Chat; 01e76a16 tightened source assertions) exposing/adding broken harness assumptions; the later #2776 merge did not clear it.

RECOMMENDED FIX SHAPE: Owner should be Canvas/E2E harness. Make the echo-runtime fixture internally consistent: either force seeded echo workspaces to push delivery when bypassing SSRF via DB update, or rewrite the tests to drive the poll consumer path instead of expecting inline echo. In chat-separation.spec.ts, use the actual ephemeral platform URL for API requests (prefer E2E_PLATFORM_URL/Playwright base fixture over localhost:8080) and seed chat history through the canonical activity_logs/MessageStore shape rather than chat_messages. Add one regression that asserts the fixture delivery mode and one that fails if E2E Chat uses a default port/schema that does not exist.

MECHANISM: #2786 is a code/harness regression in E2E Chat, not staging infra. The shared fixture creates `runtime: "external"` workspaces (`canvas/e2e/fixtures/chat-seed.ts:31-43`) but never forces `delivery_mode='push'`; registry rules explicitly default external+empty delivery mode to poll (`workspace-server/internal/handlers/registry_test.go:2155-2162`), and the proxy short-circuits poll targets to `{status:"queued"}` without dispatching to the echo URL (`workspace-server/internal/handlers/a2a_proxy.go:554-596`). Desktop tests then wait for inline echo text (`canvas/e2e/chat-desktop.spec.ts:70-77`, `:80-86`, `:101-117`) that never arrives. The newly wired chat-separation spec also has two harness defects: it defaults API calls to `http://localhost:8080` (`canvas/e2e/chat-separation.spec.ts:11-12`) even though the workflow exports only `E2E_PLATFORM_URL`/`PLAYWRIGHT_BASE_URL` with ephemeral ports (`.gitea/workflows/e2e-chat.yml:801-803`), and its history seeder inserts into nonexistent `chat_messages` (`canvas/e2e/fixtures/chat-seed.ts:156-178`) while chat history is backed by `activity_logs`/MessageStore (`workspace-server/internal/handlers/chat_history.go:3-12`). EVIDENCE: Current main still reproduces after #2776: commit `de7b35ed` has `E2E Chat / E2E Chat` red in job 492079. The log shows 24 tests executed, then desktop echo waits fail (`chat-desktop.spec.ts:77`, excerpt: `Expected: visible`) after successful `/workspaces/:id/a2a` 200s; that is the poll queued path being treated as a push echo. The same job shows the separation API tests failing with `connect ECONNREFUSED ::1:8080`, matching the hard-coded default, and the data-flow test fails with `relation "chat_messages" does not exist`, matching the obsolete seed table. The culprit cluster is #2780 (`aebdc6c1` wired the spec into E2E Chat; `01e76a16` tightened source assertions) exposing/adding broken harness assumptions; the later #2776 merge did not clear it. RECOMMENDED FIX SHAPE: Owner should be Canvas/E2E harness. Make the echo-runtime fixture internally consistent: either force seeded echo workspaces to push delivery when bypassing SSRF via DB update, or rewrite the tests to drive the poll consumer path instead of expecting inline echo. In `chat-separation.spec.ts`, use the actual ephemeral platform URL for API requests (prefer `E2E_PLATFORM_URL`/Playwright base fixture over `localhost:8080`) and seed chat history through the canonical `activity_logs`/MessageStore shape rather than `chat_messages`. Add one regression that asserts the fixture delivery mode and one that fails if E2E Chat uses a default port/schema that does not exist.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
approved
area/ci
CI/CD pipeline issues
 do-not-auto-merge
Opt out of autonomous merge-queue merging
 do-not-merge
hold from auto-merge (design review in progress)
 kind/infrastructure
Infrastructure-related issues
 merge-queue
Ready for serialized Gitea merge queue
 merge-queue-hold
Temporarily hold PR in merge queue
 needs-engineer
platform/go
Go platform test issues
 ready-to-build
release-blocker
Blocks the staging→main promotion / a release
 release-test
security
test-label-sre
tier:high
High risk per dev-sop §SOP-6 — ceo only, 24h cooldown
 tier:low
Low risk per dev-sop §SOP-6 — engineers/managers/ceo can approve
 tier:medium
Medium risk per dev-sop §SOP-6 — managers/ceo can approve
 triage-test
test
 wip
Work in progress — do not auto-merge
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
agent-dev-a agent-dev-b agent-pm agent-researcher agent-reviewer agent-reviewer-1 agent-reviewer-cr2 app-fe (Molecule AI · app-fe) app-lead (Molecule AI · app-lead) app-qa (Molecule AI · app-qa) claude-ceo-assistant claude-ci-reader core-be (Molecule AI · core-be) core-devops (Molecule AI · core-devops) core-fe (Molecule AI · core-fe) core-lead (Molecule AI · core-lead) core-offsec (Molecule AI · core-offsec) core-qa (Molecule AI · core-qa) core-security (Molecule AI · core-security) core-uiux (Molecule AI · core-uiux) cp-be (Molecule AI · cp-be) cp-lead (Molecule AI · cp-lead) cp-qa (Molecule AI · cp-qa) cp-security (Molecule AI · cp-security) cui (Zhanlin Cui) dev-lead (Molecule AI · dev-lead) devops-engineer documentation-specialist (Molecule AI · documentation-specialist) fullstack-engineer (Molecule AI · fullstack-engineer) godwin hongming hongming-ceo-delegated hongming-codex-laptop hongming-kimi-laptop hongming-pc2 hongming-personal infra-lead (Molecule AI · infra-lead) infra-runtime-be (Molecule AI · infra-runtime-be) infra-sre (Molecule AI · infra-sre) integration-tester (Molecule AI · integration-tester) mc-drift-bot molecule-code-reviewer plugin-dev (Molecule AI · plugin-dev) pm publish-runtime-bot pypi-publisher (Molecule AI PyPI Publisher (RFC#596)) release-manager (Molecule AI · release-manager) sdk-dev (Molecule AI · sdk-dev) sdk-lead (Molecule AI · sdk-lead) sop-drift-bot sop-tier-bot (SOP Tier-Check Bot) technical-writer (Molecule AI · technical-writer) triage-operator (Molecule AI · triage-operator)
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: molecule-ai/molecule-core#2786
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?