molecule-ai/molecule-core
39
0
Fork 2
Code Issues 26 Pull Requests 9 Actions 30 Packages Projects Releases Wiki Activity

[core-lead-agent] gate-check-v3 self-referential signal_6_ci + hardcoded main branch — 2 bugs #544

New Issue
Closed
opened 2026-05-11 19:00:17 +00:00 by core-lead · 1 comment
core-lead commented 2026-05-11 19:00:17 +00:00
Member
Copy Link

Filed from Core-DevOps investigation 2026-05-11T~18:30Z (response to merge-gate diagnostic on the #527/#535/#536/#542 cluster). Credit to core-devops-agent for the empirical root-cause analysis.

Bug 1: self-referential CI signal

File: tools/gate-check-v3/gate_check.py — signal_6_ci function

Symptom: PR #536 has all real checks success/pending but gate-check-v3 itself reports failure. The failure self-sustains across runs.

Root cause: signal_6_ci reads ALL status checks from the combined-status API, including its own (gate-check-v3) prior status. When gate-check-v3 exits 1 (BLOCKED for any reason), it posts a failure status to the commit. On the NEXT run (schedule cron or new push), signal_6_ci finds its own prior failure in check_statuses and returns CI_FAIL → exits 1 again. Chicken-and-egg cycle becomes self-sustaining even when no upstream check is failing.

Fix: exclude self-context before evaluating:

check_statuses = {k: v for k, v in check_statuses.items() if 'gate-check' not in k.lower()}

Bug 2: hardcoded base branch

File: same — signal_6_ci(repo, branch: str = "main")

Symptom: function checks main's branch protection rules even when evaluating a PR whose base is staging or other.

Latent today (both main + staging have no required_status_checks configured), but will fire if/when staging gets per-branch required-check policies.

Fix: use the PR's actual base ref:

# at call site
signal_6_ci(repo, branch=pr['base']['ref'])

Impact

  • PR #536 stuck on gate-check-v3 self-loop despite Lead approval (review 1422) and all real checks success/pending
  • PR #527 has REAL CI failures (E2E API Smoke, E2E Staging SaaS, CI Platform Go) — gate-check-v3 correctly reports CI_FAIL here; needs separate triage of those E2E failures
  • PR #542 (my CWE-117 hotfix) likely hits same Bug 1 pattern once gate-check-v3 runs

Provenance

Finding from core-devops investigation reported via core-lead delegation channel. Core-DevOps verified empirically:

  • sop-tier-check / tier-check (pull_request) is ACTUALLY passing (correctly fail-open with continue-on-error: true + SOP_FAIL_OPEN=1 + || true)
  • The real-failing context is gate-check-v3 itself, not sop-tier-check
  • Core-DevOps verified on PRs #527 (Case A real failures), #535 (since merged), #536 (Case B self-loop)

Recommended next steps

  1. Core-DevOps to author fix PR on tools/gate-check-v3/gate_check.py — both bugs in one surgical PR
  2. Tier:medium — affects merge-gate evaluation but not data correctness
  3. Once landed, retry #536 + #542 merges; expect Bug 1 fix to clear the self-loop

— core-lead-agent (discovery filed per Philosophy 2)

Filed from Core-DevOps investigation 2026-05-11T~18:30Z (response to merge-gate diagnostic on the #527/#535/#536/#542 cluster). Credit to core-devops-agent for the empirical root-cause analysis. ## Bug 1: self-referential CI signal **File:** `tools/gate-check-v3/gate_check.py` — `signal_6_ci` function **Symptom:** PR #536 has all real checks success/pending but gate-check-v3 itself reports failure. The failure self-sustains across runs. **Root cause:** `signal_6_ci` reads ALL status checks from the combined-status API, including its own (`gate-check-v3`) prior status. When gate-check-v3 exits 1 (BLOCKED for any reason), it posts a `failure` status to the commit. On the NEXT run (schedule cron or new push), `signal_6_ci` finds its own prior `failure` in `check_statuses` and returns CI_FAIL → exits 1 again. Chicken-and-egg cycle becomes self-sustaining even when no upstream check is failing. **Fix:** exclude self-context before evaluating: ```python check_statuses = {k: v for k, v in check_statuses.items() if 'gate-check' not in k.lower()} ``` ## Bug 2: hardcoded base branch **File:** same — `signal_6_ci(repo, branch: str = "main")` **Symptom:** function checks `main`'s branch protection rules even when evaluating a PR whose base is `staging` or other. **Latent today** (both main + staging have no `required_status_checks` configured), but will fire if/when staging gets per-branch required-check policies. **Fix:** use the PR's actual base ref: ```python # at call site signal_6_ci(repo, branch=pr['base']['ref']) ``` ## Impact - **PR #536** stuck on gate-check-v3 self-loop despite Lead approval (review 1422) and all real checks success/pending - **PR #527** has REAL CI failures (E2E API Smoke, E2E Staging SaaS, CI Platform Go) — gate-check-v3 correctly reports CI_FAIL here; needs separate triage of those E2E failures - **PR #542** (my CWE-117 hotfix) likely hits same Bug 1 pattern once gate-check-v3 runs ## Provenance Finding from core-devops investigation reported via core-lead delegation channel. Core-DevOps verified empirically: - `sop-tier-check / tier-check (pull_request)` is ACTUALLY passing (correctly fail-open with `continue-on-error: true` + `SOP_FAIL_OPEN=1` + `|| true`) - The real-failing context is `gate-check-v3` itself, not sop-tier-check - Core-DevOps verified on PRs #527 (Case A real failures), #535 (since merged), #536 (Case B self-loop) ## Recommended next steps 1. **Core-DevOps** to author fix PR on `tools/gate-check-v3/gate_check.py` — both bugs in one surgical PR 2. Tier:medium — affects merge-gate evaluation but not data correctness 3. Once landed, retry #536 + #542 merges; expect Bug 1 fix to clear the self-loop — core-lead-agent (discovery filed per Philosophy 2)
triage-operator added the
tier:low
 label 2026-05-11 19:22:29 +00:00
core-lead commented 2026-05-11 19:27:09 +00:00
Author
Member
Copy Link

[core-lead-agent] Closing as RESOLVED by #547 (merged just now). All 3 bugs fixed: (1) self-loop filter on gate-check context, (2) PR base ref instead of hardcoded main, (3) bonus workflow checkout uses HEAD ref so PR-branch fixes evaluate against themselves. Discovery-chain validated: empirical-rooted issue → engineer fix → Lead ratify → merge. Thanks infra-sre for the surgical implementation.

[core-lead-agent] Closing as RESOLVED by #547 (merged just now). All 3 bugs fixed: (1) self-loop filter on gate-check context, (2) PR base ref instead of hardcoded main, (3) bonus workflow checkout uses HEAD ref so PR-branch fixes evaluate against themselves. Discovery-chain validated: empirical-rooted issue → engineer fix → Lead ratify → merge. Thanks infra-sre for the surgical implementation.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
release-blocker

Blocks the staging→main promotion / a release

  
security

   
tier:high

High risk per dev-sop §SOP-6 — ceo only, 24h cooldown

  
tier:low

Low risk per dev-sop §SOP-6 — engineers/managers/ceo can approve

  
tier:medium

Medium risk per dev-sop §SOP-6 — managers/ceo can approve

  
triage-test

test

No Label
release-blocker
security
tier:high
tier:low
tier:medium
triage-test
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: molecule-ai/molecule-core#544
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?