molecule-ai/molecule-core
52
0
Fork 2
Code Issues 189 Pull Requests 61 Actions Packages Projects Releases Wiki Activity

[main-red] molecule-ai/molecule-core: 64c0736d06 #2782

New Issue
Closed
opened 2026-06-13 22:07:18 +00:00 by gitea-actions · 5 comments
gitea-actions bot commented 2026-06-13 22:07:18 +00:00
Copy Link
Copy Source

Main is RED on molecule-ai/molecule-core at 64c0736d06

Commit: https://git.moleculesai.app/molecule-ai/molecule-core/commit/64c0736d06e235ffd4704953367903807438b42c

Auto-filed by .gitea/workflows/main-red-watchdog.yml (Option C of the main-never-red directive). Per feedback_no_such_thing_as_flakes + feedback_fix_root_not_symptom: investigate the root cause; do NOT revert as a reflex. The watchdog itself never reverts.

Failed status contexts

  • E2E Staging SaaS (full lifecycle) / E2E Staging Platform Boot (push) — failure → logs
    • Failing after 19s
  • E2E Chat / E2E Chat (push) — failure → logs
    • Failing after 2m12s

Resolution path

  1. Read the failed logs (links above).
  2. If reproducible locally, fix forward in a PR targeting main.
  3. If the failure is a real flake — STOP. Per feedback_no_such_thing_as_flakes, intermittent failures are real bugs. Investigate to root cause; do not mark as flake.
  4. If the failure is blocking unrelated work for >1 hour, file a follow-up issue and assign someone. Do NOT revert without a human GO per feedback_prod_apply_needs_hongming_chat_go (branch protection is a prod surface).

Debug

{
  "all_contexts": [
    {
      "context": "Harness Replays / detect-changes (push)",
      "state": "success"
    },
    {
      "context": "Lint forbidden tenant-env keys / Scan workspace_secrets writers for forbidden env keys (push)",
      "state": "success"
    },
    {
      "context": "Lint forbidden tenant-env keys / Scan for repo-host token write into tenant workspace surface (push)",
      "state": "success"
    },
    {
      "context": "Secret scan / Scan diff for credential-shaped strings (push)",
      "state": "success"
    },
    {
      "context": "Harness Replays / Harness Replays (push)",
      "state": "success"
    },
    {
      "context": "E2E Staging Canvas (Playwright) / detect-changes (push)",
      "state": "success"
    },
    {
      "context": "CI / Detect changes (push)",
      "state": "success"
    },
    {
      "context": "E2E Chat / detect-changes (push)",
      "state": "success"
    },
    {
      "context": "E2E Staging SaaS (full lifecycle) / E2E Staging Platform Boot (push)",
      "state": "failure"
    },
    {
      "context": "E2E API Smoke Test / detect-changes (push)",
      "state": "success"
    },
    {
      "context": "E2E Peer Visibility (literal MCP list_peers) / E2E Peer Visibility (push)",
      "state": "success"
    },
    {
      "context": "CI / Canvas (Next.js) (push)",
      "state": "success"
    },
    {
      "context": "E2E Staging SaaS (full lifecycle) / E2E Staging Concierge (compile+skip) (push)",
      "state": "success"
    },
    {
      "context": "CI / Shellcheck (E2E scripts) (push)",
      "state": "success"
    },
    {
      "context": "lint-no-coe-on-required / lint-no-coe-on-required (push)",
      "state": "success"
    },
    {
      "context": "E2E Staging Canvas (Playwright) / Canvas tabs E2E (push)",
      "state": "success"
    },
    {
      "context": "CI / Canvas Deploy Status (push)",
      "state": "success"
    },
    {
      "context": "E2E Staging SaaS (full lifecycle) / pr-validate (push)",
      "state": "success"
    },
    {
      "context": "Local Provision Lifecycle E2E / Local Provision Lifecycle E2E (stub) (push)",
      "state": "success"
    },
    {
      "context": "Handlers Postgres Integration / Handlers Postgres Integration (push)",
      "state": "success"
    },
    {
      "context": "Local Provision Lifecycle E2E / Local Provision Lifecycle E2E (real image + MiniMax LLM, advisory) (push)",
      "state": "success"
    },
    {
      "context": "E2E Staging SaaS (full lifecycle) / E2E Staging Workspace Requests (core#2606) (push)",
      "state": "success"
    },
    {
      "context": "E2E Chat / E2E Chat (push)",
      "state": "failure"
    },
    {
      "context": "E2E API Smoke Test / E2E API Smoke Test (push)",
      "state": "success"
    },
    {
      "context": "E2E Staging SaaS (full lifecycle) / E2E Staging Concierge user_tasks (push)",
      "state": "success"
    },
    {
      "context": "E2E Staging SaaS (full lifecycle) / E2E Staging Concierge Platform Agent (push)",
      "state": "success"
    },
    {
      "context": "CI / Platform (Go) (push)",
      "state": "success"
    },
    {
      "context": "CI / all-required (push)",
      "state": "success"
    },
    {
      "context": "publish-workspace-server-image / build-and-push (push)",
      "state": "success"
    },
    {
      "context": "publish-workspace-server-image / Production auto-deploy (push)",
      "state": "success"
    }
  ],
  "branch": "main",
  "combined_state": "failure",
  "failed_contexts": [
    "E2E Staging SaaS (full lifecycle) / E2E Staging Platform Boot (push)",
    "E2E Chat / E2E Chat (push)"
  ],
  "recheck_combined_state": "failure",
  "recheck_failed_contexts": [
    "E2E Staging SaaS (full lifecycle) / E2E Staging Platform Boot (push)",
    "E2E Chat / E2E Chat (push)"
  ],
  "sha": "64c0736d06e235ffd4704953367903807438b42c"
}

This issue is idempotent: the watchdog runs hourly at :05 and edits this body in place. When main returns to green, the watchdog will close this issue automatically with a "main returned to green" comment.

# Main is RED on `molecule-ai/molecule-core` at `64c0736d06` Commit: <https://git.moleculesai.app/molecule-ai/molecule-core/commit/64c0736d06e235ffd4704953367903807438b42c> Auto-filed by `.gitea/workflows/main-red-watchdog.yml` (Option C of the [main-never-red directive](https://git.moleculesai.app/molecule-ai/molecule-core/issues/420)). Per `feedback_no_such_thing_as_flakes` + `feedback_fix_root_not_symptom`: investigate the root cause; do NOT revert as a reflex. The watchdog itself never reverts. ## Failed status contexts - **E2E Staging SaaS (full lifecycle) / E2E Staging Platform Boot (push)** — `failure` → [logs](/molecule-ai/molecule-core/actions/runs/360677/jobs/491207) - Failing after 19s - **E2E Chat / E2E Chat (push)** — `failure` → [logs](/molecule-ai/molecule-core/actions/runs/360674/jobs/491199) - Failing after 2m12s ## Resolution path 1. Read the failed logs (links above). 2. If reproducible locally, fix forward in a PR targeting `main`. 3. If the failure is a real flake — STOP. Per `feedback_no_such_thing_as_flakes`, intermittent failures are real bugs. Investigate to root cause; do not mark as flake. 4. If the failure is blocking unrelated work for >1 hour, file a follow-up issue and assign someone. Do NOT revert without a human GO per `feedback_prod_apply_needs_hongming_chat_go` (branch protection is a prod surface). ## Debug ```json { "all_contexts": [ { "context": "Harness Replays / detect-changes (push)", "state": "success" }, { "context": "Lint forbidden tenant-env keys / Scan workspace_secrets writers for forbidden env keys (push)", "state": "success" }, { "context": "Lint forbidden tenant-env keys / Scan for repo-host token write into tenant workspace surface (push)", "state": "success" }, { "context": "Secret scan / Scan diff for credential-shaped strings (push)", "state": "success" }, { "context": "Harness Replays / Harness Replays (push)", "state": "success" }, { "context": "E2E Staging Canvas (Playwright) / detect-changes (push)", "state": "success" }, { "context": "CI / Detect changes (push)", "state": "success" }, { "context": "E2E Chat / detect-changes (push)", "state": "success" }, { "context": "E2E Staging SaaS (full lifecycle) / E2E Staging Platform Boot (push)", "state": "failure" }, { "context": "E2E API Smoke Test / detect-changes (push)", "state": "success" }, { "context": "E2E Peer Visibility (literal MCP list_peers) / E2E Peer Visibility (push)", "state": "success" }, { "context": "CI / Canvas (Next.js) (push)", "state": "success" }, { "context": "E2E Staging SaaS (full lifecycle) / E2E Staging Concierge (compile+skip) (push)", "state": "success" }, { "context": "CI / Shellcheck (E2E scripts) (push)", "state": "success" }, { "context": "lint-no-coe-on-required / lint-no-coe-on-required (push)", "state": "success" }, { "context": "E2E Staging Canvas (Playwright) / Canvas tabs E2E (push)", "state": "success" }, { "context": "CI / Canvas Deploy Status (push)", "state": "success" }, { "context": "E2E Staging SaaS (full lifecycle) / pr-validate (push)", "state": "success" }, { "context": "Local Provision Lifecycle E2E / Local Provision Lifecycle E2E (stub) (push)", "state": "success" }, { "context": "Handlers Postgres Integration / Handlers Postgres Integration (push)", "state": "success" }, { "context": "Local Provision Lifecycle E2E / Local Provision Lifecycle E2E (real image + MiniMax LLM, advisory) (push)", "state": "success" }, { "context": "E2E Staging SaaS (full lifecycle) / E2E Staging Workspace Requests (core#2606) (push)", "state": "success" }, { "context": "E2E Chat / E2E Chat (push)", "state": "failure" }, { "context": "E2E API Smoke Test / E2E API Smoke Test (push)", "state": "success" }, { "context": "E2E Staging SaaS (full lifecycle) / E2E Staging Concierge user_tasks (push)", "state": "success" }, { "context": "E2E Staging SaaS (full lifecycle) / E2E Staging Concierge Platform Agent (push)", "state": "success" }, { "context": "CI / Platform (Go) (push)", "state": "success" }, { "context": "CI / all-required (push)", "state": "success" }, { "context": "publish-workspace-server-image / build-and-push (push)", "state": "success" }, { "context": "publish-workspace-server-image / Production auto-deploy (push)", "state": "success" } ], "branch": "main", "combined_state": "failure", "failed_contexts": [ "E2E Staging SaaS (full lifecycle) / E2E Staging Platform Boot (push)", "E2E Chat / E2E Chat (push)" ], "recheck_combined_state": "failure", "recheck_failed_contexts": [ "E2E Staging SaaS (full lifecycle) / E2E Staging Platform Boot (push)", "E2E Chat / E2E Chat (push)" ], "sha": "64c0736d06e235ffd4704953367903807438b42c" } ``` _This issue is idempotent: the watchdog runs hourly at `:05` and edits this body in place. When `main` returns to green, the watchdog will close this issue automatically with a "main returned to green" comment._
agent-researcher commented 2026-06-13 22:37:52 +00:00
Member
Copy Link
Copy Source

MECHANISM: This Platform Boot red is not a regression in the merged product code at 64c0736d06 (#2777 touched workspace-server/internal/handlers/a2a_proxy*.go and mock_runtime.go only). The Platform Boot job dies before any tenant/workspace boot path runs: .gitea/workflows/e2e-staging-saas.yml:432 sets E2E_RUN_ID, tests/e2e/test_staging_full_saas.sh:130,147-150 builds the staging org slug from it, and tests/e2e/test_staging_full_saas.sh:331-333 immediately POSTs /cp/admin/orgs. That POST returns 409, so the failure is a staging/CI harness slug-collision or stale-org state issue, not a workspace runtime/provisioning code regression.

EVIDENCE: main 64c0736d06, job 491207: CP health passed, then Creating org e2e-smoke-20260613-platform-3606, followed by curl: (22) ... 409. Unrelated #2783 head c28e4004, job 491414, shows the same mechanism: CP health passed, then Creating org e2e-smoke-20260613-platform-3607, followed by curl: (22) ... 409. Same failure on unrelated heads means shared staging harness/state. The separate concierge lane on 491210 later reached tenant provisioning but its concierge never became online/routable; that is a downstream staging image/provisioning signal, not the root of this Platform Boot red.

RECOMMENDED FIX SHAPE: Owner = CI/staging infra / core-be harness owner. First operationally purge the stale e2e-smoke-20260613-platform-3606* and e2e-smoke-20260613-platform-3607* staging orgs. Then fix molecule-core/.gitea/workflows/e2e-staging-saas.yml / tests/e2e/test_staging_full_saas.sh so Platform Boot slugs are collision-proof from runtime-visible values (full run id + attempt + job + random/uuid suffix, or retry-on-409 with a fresh suffix) and so the 409 body is logged. This is the same root as #2783's Platform Boot red.

MECHANISM: This Platform Boot red is not a regression in the merged product code at `64c0736d06` (#2777 touched `workspace-server/internal/handlers/a2a_proxy*.go` and `mock_runtime.go` only). The Platform Boot job dies before any tenant/workspace boot path runs: `.gitea/workflows/e2e-staging-saas.yml:432` sets `E2E_RUN_ID`, `tests/e2e/test_staging_full_saas.sh:130,147-150` builds the staging org slug from it, and `tests/e2e/test_staging_full_saas.sh:331-333` immediately POSTs `/cp/admin/orgs`. That POST returns 409, so the failure is a staging/CI harness slug-collision or stale-org state issue, not a workspace runtime/provisioning code regression. EVIDENCE: main `64c0736d06`, job 491207: CP health passed, then `Creating org e2e-smoke-20260613-platform-3606`, followed by `curl: (22) ... 409`. Unrelated #2783 head `c28e4004`, job 491414, shows the same mechanism: CP health passed, then `Creating org e2e-smoke-20260613-platform-3607`, followed by `curl: (22) ... 409`. Same failure on unrelated heads means shared staging harness/state. The separate concierge lane on 491210 later reached tenant provisioning but its concierge never became online/routable; that is a downstream staging image/provisioning signal, not the root of this Platform Boot red. RECOMMENDED FIX SHAPE: Owner = CI/staging infra / core-be harness owner. First operationally purge the stale `e2e-smoke-20260613-platform-3606*` and `e2e-smoke-20260613-platform-3607*` staging orgs. Then fix `molecule-core/.gitea/workflows/e2e-staging-saas.yml` / `tests/e2e/test_staging_full_saas.sh` so Platform Boot slugs are collision-proof from runtime-visible values (full run id + attempt + job + random/uuid suffix, or retry-on-409 with a fresh suffix) and so the 409 body is logged. This is the same root as #2783's Platform Boot red.
agent-researcher commented 2026-06-13 22:55:13 +00:00
Member
Copy Link
Copy Source

MECHANISM: Current main red on b565ded9845c778e015ace2825385910c3ae05d1 is a DIFFERENT failure from the earlier slug-collision RCA. The SaaS job reaches the new LLM dependency preflight before any org slug/create path runs. .gitea/workflows/e2e-staging-saas.yml:242-284 first proves the relevant staging LLM secret is present, then tests/e2e/lib/llm_proxy_preflight.sh:50-63 derives https://staging-api.moleculesai.app/api/v1/internal/llm/openai/v1/chat/completions; tests/e2e/lib/llm_proxy_preflight.sh:92-104 POSTs the cheap completion and exits 70 on non-200. The returned 401 means the staging LLM proxy/auth path is rejecting the request, not that workspace boot or slug creation regressed.

EVIDENCE: job 491807 on main logs LLM key present ✓ (runtime=claude-code, key=MOLECULE_STAGING_MINIMAX_API_KEY, len=125), CP /health passed, then the preflight failed with DEP-DOWN:staging-llm preflight failed and http_code=401. No Creating org ... line appears before the failure, so this run never reaches the earlier /cp/admin/orgs 409 slug-collision mechanism documented in #100639. This is exactly the classification behavior introduced by the preflight: fail fast and label staging LLM dependency/auth failures as dependency-down.

RECOMMENDED FIX SHAPE: Owner = staging LLM/CP operator or controlplane LLM-proxy owner, not workspace/runtime. Verify the credential accepted by /api/v1/internal/llm/openai/v1/chat/completions in staging: proxy auth header expectations, Railway/CP secret projection, and provider credential validity for the MiniMax-backed staging proxy. If the 401 is expected because the preflight omits a required internal bearer/header, fix tests/e2e/lib/llm_proxy_preflight.sh / .gitea/workflows/e2e-staging-saas.yml to pass the same auth the proxy requires. Keep #2785’s collision-proof slug work separate: it still closes the earlier 409 recurrence, but it will not fix this current DEP-DOWN 401.

MECHANISM: Current main red on `b565ded9845c778e015ace2825385910c3ae05d1` is a DIFFERENT failure from the earlier slug-collision RCA. The SaaS job reaches the new LLM dependency preflight before any org slug/create path runs. `.gitea/workflows/e2e-staging-saas.yml:242-284` first proves the relevant staging LLM secret is present, then `tests/e2e/lib/llm_proxy_preflight.sh:50-63` derives `https://staging-api.moleculesai.app/api/v1/internal/llm/openai/v1/chat/completions`; `tests/e2e/lib/llm_proxy_preflight.sh:92-104` POSTs the cheap completion and exits 70 on non-200. The returned 401 means the staging LLM proxy/auth path is rejecting the request, not that workspace boot or slug creation regressed. EVIDENCE: job `491807` on main logs `LLM key present ✓ (runtime=claude-code, key=MOLECULE_STAGING_MINIMAX_API_KEY, len=125)`, CP `/health` passed, then the preflight failed with `DEP-DOWN:staging-llm preflight failed` and `http_code=401`. No `Creating org ...` line appears before the failure, so this run never reaches the earlier `/cp/admin/orgs` 409 slug-collision mechanism documented in #100639. This is exactly the classification behavior introduced by the preflight: fail fast and label staging LLM dependency/auth failures as dependency-down. RECOMMENDED FIX SHAPE: Owner = staging LLM/CP operator or controlplane LLM-proxy owner, not workspace/runtime. Verify the credential accepted by `/api/v1/internal/llm/openai/v1/chat/completions` in staging: proxy auth header expectations, Railway/CP secret projection, and provider credential validity for the MiniMax-backed staging proxy. If the 401 is expected because the preflight omits a required internal bearer/header, fix `tests/e2e/lib/llm_proxy_preflight.sh` / `.gitea/workflows/e2e-staging-saas.yml` to pass the same auth the proxy requires. Keep #2785’s collision-proof slug work separate: it still closes the earlier 409 recurrence, but it will not fix this current DEP-DOWN 401.
agent-researcher commented 2026-06-13 23:03:12 +00:00
Member
Copy Link
Copy Source

MECHANISM: Current main-red shifted again: molecule-core main b565ded9845c778e015ace2825385910c3ae05d1 is failing local E2E Chat / E2E Chat, not the earlier staging slug collision or LLM preflight. There are two layers. First, the desktop echo failures predate #2780: post-#2759 main 78a922ce already failed chat-desktop.spec.ts waiting for rendered Echo: replies, so that is a chat send/WS completion regression in the canvas chat path (canvas/src/components/tabs/chat/hooks/useChatSend.ts:335-411, asserted by canvas/e2e/chat-desktop.spec.ts:70-77). Second, #2780 (62c528d1 / 01e76a16) made the orphaned chat-separation spec live in .gitea/workflows/e2e-chat.yml:422, exposing two harness bugs: chat-separation.spec.ts:11 defaults direct API probes to localhost:8080 while the workflow runs platform on an ephemeral port and exports only E2E_PLATFORM_URL at .gitea/workflows/e2e-chat.yml:410; chat-seed.ts:175 inserts into obsolete chat_messages, while current chat history is read from activity_logs through messagestore.PostgresMessageStore (workspace-server/internal/messagestore/postgres_store.go:68-73).

EVIDENCE: job 491799 on main b565ded... failed 16 tests. Log excerpts: connect ECONNREFUSED ::1:8080, ERROR: relation "chat_messages" does not exist, and desktop waits for Echo: What is the weather?. The same desktop echo failure existed on job 490711 at 78a922ce with 3 failures before chat-separation was added; after #2780, job 491511 added the localhost:8080 and chat_messages failures. Platform itself started healthy on the ephemeral port (Platform starting on 127.0.0.1:57145), so this is not shared staging infra.

RECOMMENDED FIX SHAPE: Owner: Canvas/frontend + E2E harness. Split into two routable fixes: (1) restore desktop ChatTab echo completion by tracing the #2759/#2777-era send/WS path from useChatSend.ts through A2A response/WS completion, with a regression asserting the echo runtime response reaches desktop. (2) fix #2780 chat-separation harness: export/use E2E_API_URL from the workflow or use E2E_PLATFORM_URL for direct probes, and seed initial chat history through the real activity-log/A2A path or the MessageStore-backed schema, not a nonexistent chat_messages table. Keep #2782 open until both lanes are green; the staging slug collision remains addressed separately by #2785, with infra purge still owner-action.

MECHANISM: Current main-red shifted again: molecule-core main `b565ded9845c778e015ace2825385910c3ae05d1` is failing local `E2E Chat / E2E Chat`, not the earlier staging slug collision or LLM preflight. There are two layers. First, the desktop echo failures predate #2780: post-#2759 main `78a922ce` already failed `chat-desktop.spec.ts` waiting for rendered `Echo:` replies, so that is a chat send/WS completion regression in the canvas chat path (`canvas/src/components/tabs/chat/hooks/useChatSend.ts:335-411`, asserted by `canvas/e2e/chat-desktop.spec.ts:70-77`). Second, #2780 (`62c528d1` / `01e76a16`) made the orphaned chat-separation spec live in `.gitea/workflows/e2e-chat.yml:422`, exposing two harness bugs: `chat-separation.spec.ts:11` defaults direct API probes to `localhost:8080` while the workflow runs platform on an ephemeral port and exports only `E2E_PLATFORM_URL` at `.gitea/workflows/e2e-chat.yml:410`; `chat-seed.ts:175` inserts into obsolete `chat_messages`, while current chat history is read from `activity_logs` through `messagestore.PostgresMessageStore` (`workspace-server/internal/messagestore/postgres_store.go:68-73`). EVIDENCE: job `491799` on main `b565ded...` failed 16 tests. Log excerpts: `connect ECONNREFUSED ::1:8080`, `ERROR: relation "chat_messages" does not exist`, and desktop waits for `Echo: What is the weather?`. The same desktop echo failure existed on job `490711` at `78a922ce` with 3 failures before chat-separation was added; after #2780, job `491511` added the `localhost:8080` and `chat_messages` failures. Platform itself started healthy on the ephemeral port (`Platform starting on 127.0.0.1:57145`), so this is not shared staging infra. RECOMMENDED FIX SHAPE: Owner: Canvas/frontend + E2E harness. Split into two routable fixes: (1) restore desktop ChatTab echo completion by tracing the #2759/#2777-era send/WS path from `useChatSend.ts` through A2A response/WS completion, with a regression asserting the echo runtime response reaches desktop. (2) fix #2780 chat-separation harness: export/use `E2E_API_URL` from the workflow or use `E2E_PLATFORM_URL` for direct probes, and seed initial chat history through the real activity-log/A2A path or the MessageStore-backed schema, not a nonexistent `chat_messages` table. Keep #2782 open until both lanes are green; the staging slug collision remains addressed separately by #2785, with infra purge still owner-action.
agent-researcher commented 2026-06-13 23:51:22 +00:00
Member
Copy Link
Copy Source

MECHANISM: Current main 0c4a4d1b is failing E2E Chat from harness code, not staging infra. The #2788 merge fixed the table choice (activity_logs) and push delivery in canvas/e2e/fixtures/chat-seed.ts:112-217, but seedChatHistory() still builds one SQL string containing JSON text with embedded quotes (escape() at chat-seed.ts:188, row construction at chat-seed.ts:211). When the seeded message includes quotes, Postgres receives invalid json and aborts before the seeded-history assertions. Separately, the same run shows desktop echo waits timing out while mobile echo passes against the same seedWorkspace() push fixture; keep the remaining desktop-only symptom routed to the E2E Chat/canvas harness owner after the seed quoting fix.

EVIDENCE: main head 0c4a4d1b6de11c7388a8129773042971735da52e (Merge PR #2788) has E2E Chat / E2E Chat red at job 492389. The hard seed failure is at canvas/e2e/fixtures/chat-seed.ts:40 via execFileSync(psql ...), with log excerpt: ERROR: invalid input syntax for type json and Token "quotes" is invalid. The same job also records desktop-only echo failures at canvas/e2e/chat-desktop.spec.ts:77, :86, and :117 waiting for Echo: ..., while mobile echo tests pass at chat-mobile.spec.ts:50, :59, and :99. Activity API source tests return 401 because they call /workspaces/:id/activity without auth headers (chat-separation.spec.ts:191-230), so those are a separate harness-auth gap exposed by the newly-live spec.

RECOMMENDED FIX SHAPE: Owner = Canvas/E2E Chat harness (Kimi). In canvas/e2e/fixtures/chat-seed.ts, stop hand-embedding JSON into SQL string literals; use a shell-safe psql invocation that preserves JSON exactly (for example stdin temp SQL with dollar-quoted JSON or parameterized helper semantics) and keep the quote-containing regression input. In canvas/e2e/chat-separation.spec.ts, add the workspace/admin auth headers to Activity API request.get() calls. Then rerun E2E Chat on main/PR and separately inspect the desktop-only echo waits if they persist after the seed/auth fixes; do not route this as staging infra.

MECHANISM: Current main `0c4a4d1b` is failing E2E Chat from harness code, not staging infra. The #2788 merge fixed the table choice (`activity_logs`) and push delivery in `canvas/e2e/fixtures/chat-seed.ts:112-217`, but `seedChatHistory()` still builds one SQL string containing JSON text with embedded quotes (`escape()` at `chat-seed.ts:188`, row construction at `chat-seed.ts:211`). When the seeded message includes quotes, Postgres receives invalid json and aborts before the seeded-history assertions. Separately, the same run shows desktop echo waits timing out while mobile echo passes against the same `seedWorkspace()` push fixture; keep the remaining desktop-only symptom routed to the E2E Chat/canvas harness owner after the seed quoting fix. EVIDENCE: main head `0c4a4d1b6de11c7388a8129773042971735da52e` (`Merge PR #2788`) has `E2E Chat / E2E Chat` red at job `492389`. The hard seed failure is at `canvas/e2e/fixtures/chat-seed.ts:40` via `execFileSync(psql ...)`, with log excerpt: `ERROR: invalid input syntax for type json` and `Token "quotes" is invalid`. The same job also records desktop-only echo failures at `canvas/e2e/chat-desktop.spec.ts:77`, `:86`, and `:117` waiting for `Echo: ...`, while mobile echo tests pass at `chat-mobile.spec.ts:50`, `:59`, and `:99`. Activity API source tests return 401 because they call `/workspaces/:id/activity` without auth headers (`chat-separation.spec.ts:191-230`), so those are a separate harness-auth gap exposed by the newly-live spec. RECOMMENDED FIX SHAPE: Owner = Canvas/E2E Chat harness (Kimi). In `canvas/e2e/fixtures/chat-seed.ts`, stop hand-embedding JSON into SQL string literals; use a shell-safe `psql` invocation that preserves JSON exactly (for example stdin temp SQL with dollar-quoted JSON or parameterized helper semantics) and keep the quote-containing regression input. In `canvas/e2e/chat-separation.spec.ts`, add the workspace/admin auth headers to Activity API `request.get()` calls. Then rerun `E2E Chat` on main/PR and separately inspect the desktop-only echo waits if they persist after the seed/auth fixes; do not route this as staging infra.
gitea-actions bot commented 2026-06-14 03:05:46 +00:00
Copy Link
Copy Source

main returned to green at SHA 9595757acf4ec5392d4bb79d0df7d9aeb1003af5 (https://git.moleculesai.app/molecule-ai/molecule-core/commit/9595757acf4ec5392d4bb79d0df7d9aeb1003af5). Closing automatically. If the underlying root cause is not yet understood, reopen this issue and file a postmortem — green-by-flake is still a bug per feedback_no_such_thing_as_flakes.

`main` returned to green at SHA `9595757acf4ec5392d4bb79d0df7d9aeb1003af5` (<https://git.moleculesai.app/molecule-ai/molecule-core/commit/9595757acf4ec5392d4bb79d0df7d9aeb1003af5>). Closing automatically. If the underlying root cause is not yet understood, reopen this issue and file a postmortem — green-by-flake is still a bug per `feedback_no_such_thing_as_flakes`.
gitea-actions bot closed this issue 2026-06-14 03:05:46 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
area/ci
CI/CD pipeline issues
 do-not-auto-merge
Opt out of autonomous merge-queue merging
 kind/infrastructure
Infrastructure-related issues
 merge-queue
Ready for serialized Gitea merge queue
 merge-queue-hold
Temporarily hold PR in merge queue
 platform/go
Go platform test issues
 release-blocker
Blocks the staging→main promotion / a release
 release-test
security
test-label-sre
tier:high
High risk per dev-sop §SOP-6 — ceo only, 24h cooldown
 tier:low
Low risk per dev-sop §SOP-6 — engineers/managers/ceo can approve
 tier:medium
Medium risk per dev-sop §SOP-6 — managers/ceo can approve
 triage-test
test
 wip
Work in progress — do not auto-merge
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
agent-dev-a agent-dev-b agent-pm agent-researcher agent-reviewer agent-reviewer-1 agent-reviewer-cr2 app-fe (Molecule AI · app-fe) app-lead (Molecule AI · app-lead) app-qa (Molecule AI · app-qa) claude-ceo-assistant claude-ci-reader claude-status-reaper core-be (Molecule AI · core-be) core-devops (Molecule AI · core-devops) core-fe (Molecule AI · core-fe) core-lead (Molecule AI · core-lead) core-offsec (Molecule AI · core-offsec) core-qa (Molecule AI · core-qa) core-security (Molecule AI · core-security) core-uiux (Molecule AI · core-uiux) cp-be (Molecule AI · cp-be) cp-lead (Molecule AI · cp-lead) cp-qa (Molecule AI · cp-qa) cp-security (Molecule AI · cp-security) cui (Zhanlin Cui) dev-lead (Molecule AI · dev-lead) devops-engineer documentation-specialist (Molecule AI · documentation-specialist) fullstack-engineer (Molecule AI · fullstack-engineer) godwin hongming hongming-ceo-delegated hongming-codex-laptop hongming-kimi-laptop hongming-pc2 infra-lead (Molecule AI · infra-lead) infra-runtime-be (Molecule AI · infra-runtime-be) infra-sre (Molecule AI · infra-sre) integration-tester (Molecule AI · integration-tester) mc-drift-bot molecule-code-reviewer plugin-dev (Molecule AI · plugin-dev) pm publish-runtime-bot pypi-publisher (Molecule AI PyPI Publisher (RFC#596)) release-manager (Molecule AI · release-manager) sdk-dev (Molecule AI · sdk-dev) sdk-lead (Molecule AI · sdk-lead) sop-drift-bot sop-tier-bot (SOP Tier-Check Bot) technical-writer (Molecule AI · technical-writer) triage-operator (Molecule AI · triage-operator)
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: molecule-ai/molecule-core#2782
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?