Compare commits

..

4 Commits

Author SHA1 Message Date
core-devops 65831c839e fix(queue): re-fetch PR head before merge to detect stale SHA
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 2s
cascade-list-drift-gate / check (pull_request) Failing after 3s
CI / Detect changes (pull_request) Successful in 4s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 9s
E2E API Smoke Test / detect-changes (pull_request) Successful in 7s
E2E Chat / detect-changes (pull_request) Successful in 5s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 6s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 4s
lint-continue-on-error-tracking / lint-continue-on-error-tracking (pull_request) Successful in 1m10s
Lint curl status-code capture / Scan workflows for curl status-capture pollution (pull_request) Successful in 3s
Lint pre-flip continue-on-error / Verify continue-on-error flips have run-log proof (pull_request) Successful in 54s
CI / Platform (Go) (pull_request) Successful in 4m48s
lint-required-context-exists-in-bp / lint-required-context-exists-in-bp (pull_request) Successful in 1m10s
lint-required-no-paths / lint-required-no-paths (pull_request) Successful in 54s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 4s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 3s
CI / Canvas (Next.js) (pull_request) Successful in 6m12s
gate-check-v3 / gate-check (pull_request) Successful in 3s
Lint workflow YAML (Gitea-1.22.6-hostile shapes) / Lint workflow YAML for Gitea-1.22.6-hostile shapes (pull_request) Successful in 1m0s
qa-review / approved (pull_request) Failing after 3s
security-review / approved (pull_request) Failing after 2s
sop-tier-check / tier-check (pull_request) Successful in 4s
Ops Scripts Tests / Ops scripts (unittest) (pull_request) Successful in 1m1s
CI / Python Lint & Test (pull_request) Successful in 6m37s
CI / all-required (pull_request) Successful in 6m38s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 2s
E2E Chat / E2E Chat (pull_request) Successful in 1s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 1s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 1s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 2s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
sop-checklist / review-refire (pull_request_target) Has been skipped
sop-checklist / all-items-acked (pull_request) [info tier:low] acked: 2/7 — missing: comprehensive-testing, local-postgres-e2e, staging-smoke, +2 — body-unfilled: comprehensive-testing, l
sop-checklist / na-declarations (pull_request) N/A: (none)
sop-checklist / all-items-acked (pull_request_target) Successful in 10s
sop-tier-check / tier-check (pull_request_target) Failing after 7s
If CI updates the PR head between the initial status check and the
merge call, the queue might try to merge an outdated head.  Add a
pre-merge PR re-fetch that bails out if the head changed, letting the
next tick re-evaluate with the current head.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 04:06:22 +00:00
core-devops d342149646 fix(queue): handle Gitea empty-body 200 on merge endpoint
Block internal-flavored paths / Block forbidden paths (pull_request) Waiting to run
cascade-list-drift-gate / check (pull_request) Waiting to run
CI / all-required (pull_request) Waiting to run
CI / Detect changes (pull_request) Waiting to run
CI / Platform (Go) (pull_request) Waiting to run
CI / Canvas (Next.js) (pull_request) Waiting to run
CI / Shellcheck (E2E scripts) (pull_request) Waiting to run
CI / Canvas Deploy Reminder (pull_request) Blocked by required conditions
CI / Python Lint & Test (pull_request) Waiting to run
E2E API Smoke Test / detect-changes (pull_request) Waiting to run
E2E API Smoke Test / E2E API Smoke Test (pull_request) Blocked by required conditions
E2E Chat / detect-changes (pull_request) Waiting to run
E2E Chat / E2E Chat (pull_request) Blocked by required conditions
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Waiting to run
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Blocked by required conditions
Handlers Postgres Integration / detect-changes (pull_request) Waiting to run
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Blocked by required conditions
lint-continue-on-error-tracking / lint-continue-on-error-tracking (pull_request) Waiting to run
Lint curl status-code capture / Scan workflows for curl status-capture pollution (pull_request) Waiting to run
Lint pre-flip continue-on-error / Verify continue-on-error flips have run-log proof (pull_request) Waiting to run
lint-required-context-exists-in-bp / lint-required-context-exists-in-bp (pull_request) Waiting to run
lint-required-no-paths / lint-required-no-paths (pull_request) Waiting to run
Lint workflow YAML (Gitea-1.22.6-hostile shapes) / Lint workflow YAML for Gitea-1.22.6-hostile shapes (pull_request) Waiting to run
Runtime PR-Built Compatibility / detect-changes (pull_request) Waiting to run
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Blocked by required conditions
Secret scan / Scan diff for credential-shaped strings (pull_request) Waiting to run
Ops Scripts Tests / Ops scripts (unittest) (pull_request) Waiting to run
gate-check-v3 / gate-check (pull_request) Waiting to run
qa-review / approved (pull_request) Waiting to run
security-review / approved (pull_request) Waiting to run
sop-checklist / all-items-acked (pull_request) Waiting to run
sop-tier-check / tier-check (pull_request) Waiting to run
Gitea's /pulls/{n}/merge returns HTTP 200 with an empty body on success.
The api() wrapper tries to json.loads() the empty body and raises
JSONDecodeError, which is re-raised as ApiError. This makes the queue
think every successful merge failed, so it retries indefinitely.

Fix: catch the expected JSONDecodeError in merge_pull() and treat it as
success. Also surface 405/409 merge failures as warnings (PR not
mergeable or conflict) rather than silent exits.

Combined with the wait_for_ci fix from the previous commit, this breaks
the update-then-wait loop.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 04:05:37 +00:00
core-devops 54907ee852 fix(queue): wait for CI after update instead of immediate re-check
Block internal-flavored paths / Block forbidden paths (pull_request) Waiting to run
cascade-list-drift-gate / check (pull_request) Waiting to run
CI / all-required (pull_request) Waiting to run
CI / Detect changes (pull_request) Waiting to run
CI / Platform (Go) (pull_request) Waiting to run
CI / Canvas (Next.js) (pull_request) Waiting to run
CI / Shellcheck (E2E scripts) (pull_request) Waiting to run
CI / Canvas Deploy Reminder (pull_request) Blocked by required conditions
CI / Python Lint & Test (pull_request) Waiting to run
E2E API Smoke Test / detect-changes (pull_request) Waiting to run
E2E API Smoke Test / E2E API Smoke Test (pull_request) Blocked by required conditions
E2E Chat / detect-changes (pull_request) Waiting to run
E2E Chat / E2E Chat (pull_request) Blocked by required conditions
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Waiting to run
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Blocked by required conditions
Handlers Postgres Integration / detect-changes (pull_request) Waiting to run
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Blocked by required conditions
lint-continue-on-error-tracking / lint-continue-on-error-tracking (pull_request) Waiting to run
Lint curl status-code capture / Scan workflows for curl status-capture pollution (pull_request) Waiting to run
Lint pre-flip continue-on-error / Verify continue-on-error flips have run-log proof (pull_request) Waiting to run
lint-required-context-exists-in-bp / lint-required-context-exists-in-bp (pull_request) Waiting to run
lint-required-no-paths / lint-required-no-paths (pull_request) Waiting to run
Lint workflow YAML (Gitea-1.22.6-hostile shapes) / Lint workflow YAML for Gitea-1.22.6-hostile shapes (pull_request) Waiting to run
Runtime PR-Built Compatibility / detect-changes (pull_request) Waiting to run
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Blocked by required conditions
Secret scan / Scan diff for credential-shaped strings (pull_request) Waiting to run
Ops Scripts Tests / Ops scripts (unittest) (pull_request) Waiting to run
gate-check-v3 / gate-check (pull_request) Waiting to run
qa-review / approved (pull_request) Waiting to run
security-review / approved (pull_request) Waiting to run
sop-checklist / all-items-acked (pull_request) Waiting to run
sop-tier-check / tier-check (pull_request) Waiting to run
The queue was in an update-then-wait loop:
1. Queue updates PR → new CI run triggered on new head
2. Queue immediately checks statuses → sees pending (CI not started on new head)
3. Queue exits "wait"
4. Next tick: same cycle, CI never completes on any single head

Fix: after update_pull(), re-fetch the new head SHA and poll CI for
up to 5 min until required contexts reach terminal state. If CI
finishes within the window, merge on the same tick. If not, exit and
retry next tick.

Also adds `import time` required for the wait loop.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 04:03:11 +00:00
core-devops 99453c6a71 infra(ci): add concurrency blocks to 3 scheduled workflows
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 3s
CI / Detect changes (pull_request) Successful in 4s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 8s
CI / Platform (Go) (pull_request) Successful in 4m24s
E2E API Smoke Test / detect-changes (pull_request) Successful in 6s
E2E Chat / detect-changes (pull_request) Successful in 5s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 5s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 2s
Lint curl status-code capture / Scan workflows for curl status-capture pollution (pull_request) Successful in 5s
lint-continue-on-error-tracking / lint-continue-on-error-tracking (pull_request) Successful in 1m7s
CI / Canvas (Next.js) (pull_request) Successful in 6m4s
Lint pre-flip continue-on-error / Verify continue-on-error flips have run-log proof (pull_request) Successful in 1m1s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 4s
lint-required-context-exists-in-bp / lint-required-context-exists-in-bp (pull_request) Successful in 1m8s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 7s
gate-check-v3 / gate-check (pull_request) Successful in 2s
sop-tier-check / tier-check (pull_request) Successful in 4s
lint-required-no-paths / lint-required-no-paths (pull_request) Successful in 50s
CI / Python Lint & Test (pull_request) Successful in 6m28s
CI / all-required (pull_request) Successful in 6m22s
Lint workflow YAML (Gitea-1.22.6-hostile shapes) / Lint workflow YAML for Gitea-1.22.6-hostile shapes (pull_request) Successful in 1m9s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 3s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 2s
E2E Chat / E2E Chat (pull_request) Successful in 3s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 3s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 1s
qa-review / approved (pull_request) N/A declared by core-devops; qa-review waived per sop-checklist config
security-review / approved (pull_request) N/A declared by core-devops; security-review waived per sop-checklist config
sop-checklist / review-refire (pull_request_target) Has been skipped
sop-tier-check / tier-check (pull_request_target) Failing after 8s
sop-checklist / all-items-acked (pull_request) [info tier:low] acked: 5/7 — missing: root-cause, no-backwards-compat
sop-checklist / na-declarations (pull_request) N/A: (none)
sop-checklist / all-items-acked (pull_request_target) Successful in 12s
Add per-SHA concurrency groups with cancel-in-progress: true to
scheduled workflows missing concurrency blocks:

- gate-check-v3.yml (hourly cron): prevents stale hourly runs from
  accumulating when new cron ticks fire
- secret-pattern-drift.yml (daily 05:00 UTC): same
- weekly-platform-go.yml (Mondays 04:17 UTC): same

These are lower-frequency than the sweep/minute-level workflows
but should still be covered for consistency and runner hygiene.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 02:47:52 +00:00
11 changed files with 123 additions and 322 deletions
+103 -135
View File
@@ -23,6 +23,7 @@ import dataclasses
import json
import os
import sys
import time
import urllib.error
import urllib.parse
import urllib.request
@@ -44,10 +45,7 @@ REQUIRED_CONTEXTS_RAW = _env(
"REQUIRED_CONTEXTS",
default=(
"CI / all-required (pull_request),"
"sop-checklist / all-items-acked (pull_request),"
"E2E Chat / E2E Chat (pull_request),"
"qa-review / approved (pull_request),"
"security-review / approved (pull_request)"
"sop-checklist / all-items-acked (pull_request)"
),
)
# Required contexts for push (main/staging) runs. The push CI uses the same
@@ -68,11 +66,6 @@ class ApiError(RuntimeError):
pass
class MergePermissionError(ApiError):
"""Merge failed with a permanent permission error (403/404/405).
The queue should skip this PR and move to the next one."""
@dataclasses.dataclass(frozen=True)
class MergeDecision:
ready: bool
@@ -156,38 +149,15 @@ def latest_statuses_by_context(statuses: list[dict]) -> dict[str, dict]:
return latest
def _is_tier_low_pending_ok(
latest_statuses: dict[str, dict],
context: str,
pr_labels: set[str],
) -> bool:
"""Return True if tier:low PR can tolerate sop-checklist pending state.
Per sop-checklist-config.yaml tier_failure_mode, tier:low uses soft-fail:
sop-checklist posts state=pending when acks are satisfied (missing
manager/ceo acks are informational only). The queue should accept
pending instead of waiting for success.
"""
if "tier:low" not in pr_labels:
return False
if "sop-checklist" not in context:
return False
status = latest_statuses.get(context) or {}
return status_state(status) == "pending"
def required_contexts_green(
latest_statuses: dict[str, dict],
contexts: list[str],
pr_labels: set[str] | None = None,
) -> tuple[bool, list[str]]:
missing_or_bad: list[str] = []
for context in contexts:
status = latest_statuses.get(context)
state = status_state(status or {})
if state != "success":
if pr_labels and _is_tier_low_pending_ok(latest_statuses, context, pr_labels):
continue # tier:low soft-fail: accept pending sop-checklist
missing_or_bad.append(f"{context}={state or 'missing'}")
return not missing_or_bad, missing_or_bad
@@ -240,7 +210,6 @@ def evaluate_merge_readiness(
pr_status: dict,
required_contexts: list[str],
pr_has_current_base: bool,
pr_labels: set[str] | None = None,
) -> MergeDecision:
# Check push-required contexts explicitly instead of combined state.
# Combined state can be "failure" due to non-blocking jobs
@@ -260,7 +229,7 @@ def evaluate_merge_readiness(
# The required_contexts list is the authoritative gate — it includes only
# the checks that actually block merges.
latest = latest_statuses_by_context(pr_status.get("statuses") or [])
ok, missing_or_bad = required_contexts_green(latest, required_contexts, pr_labels)
ok, missing_or_bad = required_contexts_green(latest, required_contexts)
if not ok:
return MergeDecision(False, "wait", "required contexts not green: " + ", ".join(missing_or_bad))
return MergeDecision(True, "merge", "ready")
@@ -285,32 +254,27 @@ def get_combined_status(sha: str) -> dict:
_, combined = api("GET", f"/repos/{OWNER}/{NAME}/commits/{sha}/status")
if not isinstance(combined, dict):
raise ApiError(f"status for {sha} response not object")
combined_statuses: list[dict] = combined.get("statuses") or []
# Fetch full statuses list; 200 covers >99% of real-world runs.
# The list is ordered ascending by id (oldest first) — callers must
# iterate in reverse to get the newest entry per context.
# Best-effort: large repos (main with 550+ statuses) may time out.
# On timeout, fall back to the statuses[] already in the combined
# response (usually 30 entries — enough for most PRs, enough for
# main's early push-required contexts).
try:
_, all_statuses_raw = api(
_, all_statuses = api(
"GET",
f"/repos/{OWNER}/{NAME}/commits/{sha}/statuses",
query={"limit": "50"},
)
if isinstance(all_statuses_raw, list):
all_statuses: list[dict] = list(all_statuses_raw)
else:
all_statuses = []
if isinstance(all_statuses, list):
combined["statuses"] = all_statuses
except (ApiError, urllib.error.URLError, TimeoutError, OSError) as exc:
# URLError covers network-level failures (DNS, refused, timeout).
# TimeoutError and OSError cover socket-level timeouts.
sys.stderr.write(f"::warning::could not fetch full statuses list for {sha[:8]}: {exc}\n")
all_statuses = []
# Build latest per context: process combined (ascending→reverse=newest
# first), then fill gaps from all_statuses (already newest-first).
latest: dict[str, dict] = {}
for status in reversed(sorted(combined_statuses, key=lambda s: s.get("id") or 0)):
ctx = status.get("context")
if isinstance(ctx, str) and ctx not in latest:
latest[ctx] = status
for status in all_statuses:
ctx = status.get("context")
if isinstance(ctx, str) and ctx not in latest:
latest[ctx] = status
combined["statuses"] = list(latest.values())
# Fall back to the statuses[] already in the combined response.
pass
return combined
@@ -351,25 +315,6 @@ def post_comment(pr_number: int, body: str, *, dry_run: bool) -> None:
api("POST", f"/repos/{OWNER}/{NAME}/issues/{pr_number}/comments", body={"body": body})
def add_hold_label(pr_number: int, *, dry_run: bool) -> None:
"""Apply the hold label so the queue skips this PR and processes the next."""
print(f"::notice::adding `{HOLD_LABEL}` to PR #{pr_number}")
if dry_run:
return
try:
api(
"POST",
f"/repos/{OWNER}/{NAME}/issues/{pr_number}/labels",
body={"labels": [HOLD_LABEL]},
)
except ApiError as exc:
# 404 = PR already closed/deleted; 422 = label already present (Gitea
# returns 422 for duplicate label assignment — not a real error).
if "404" in str(exc) or "422" in str(exc):
return
sys.stderr.write(f"::warning::could not add hold label to PR #{pr_number}: {exc}\n")
def update_pull(pr_number: int, *, dry_run: bool) -> None:
print(f"::notice::updating PR #{pr_number} with base branch via style={UPDATE_STYLE}")
if dry_run:
@@ -382,6 +327,43 @@ def update_pull(pr_number: int, *, dry_run: bool) -> None:
)
def wait_for_ci(
head_sha: str,
contexts: list[str],
*,
max_wait_seconds: int = 300,
poll_interval: int = 15,
) -> bool:
"""Poll CI statuses for head_sha until all required contexts are terminal.
Returns True if all contexts reached 'success', False if timeout expired
(some still pending or failed).
Background: after a queue-triggered PR update, CI re-runs on the new head.
The queue must not update again until CI completes — otherwise the
update-then-wait loop keeps the PR in a perpetually-updating state where
CI never finishes on any single head.
"""
deadline = time.time() + max_wait_seconds
while time.time() < deadline:
time.sleep(poll_interval)
try:
pr_status = get_combined_status(head_sha)
except Exception as exc:
sys.stderr.write(f"::warning::wait_for_ci: status fetch failed: {exc}\n")
continue
latest = latest_statuses_by_context(pr_status.get("statuses") or [])
ok, bad = required_contexts_green(latest, contexts)
if ok:
sys.stderr.write(f"::notice::wait_for_ci: all contexts green after {int(time.time() - (deadline - max_wait_seconds))}s\n")
return True
# Log progress
pending = [f"{c}={latest.get(c, {}).get('status', 'missing')}" for c in contexts if latest.get(c, {}).get('status') != 'success']
sys.stderr.write(f"::notice::wait_for_ci: still waiting ({int(deadline - time.time())}s left): {', '.join(pending[:3])}\n")
sys.stderr.write(f"::warning::wait_for_ci: timeout after {max_wait_seconds}s; proceeding with merge check\n")
return False
def merge_pull(pr_number: int, *, dry_run: bool) -> None:
payload = {
"Do": "merge",
@@ -394,16 +376,24 @@ def merge_pull(pr_number: int, *, dry_run: bool) -> None:
print(f"::notice::merging PR #{pr_number}")
if dry_run:
return
# Gitea's merge endpoint returns HTTP 200 with an empty body on success.
# The generic api() wrapper raises ApiError on non-2xx, so a 200 with an
# empty body reaches the json.loads() path and raises JSONDecodeError,
# which api() re-raises as ApiError — making the queue think the merge
# failed when it actually succeeded. Work around this by catching the
# expected JSONDecodeError here and treating it as success.
try:
api("POST", f"/repos/{OWNER}/{NAME}/pulls/{pr_number}/merge", body=payload, expect_json=False)
except ApiError as exc:
# Re-raise permission-like errors so process_once can skip this PR.
# 403 = no push access, 404 = repo/pr not found, 405 = not allowed.
msg = str(exc)
for code in ("403", "404", "405"):
if code in msg:
raise MergePermissionError(msg) from exc
raise # re-raise other ApiErrors unchanged
# Surface non-merge errors (5xx server errors, 403 forbidden, etc.)
if "merge" in str(exc).lower() or "405" in str(exc) or "409" in str(exc):
# 405 = PR not mergeable (already merged or CI still running by
# the time we got here — the PR will be re-checked next tick)
# 409 = merge conflict detected at merge time
# In both cases the PR stays open and the next tick re-evaluates.
sys.stderr.write(f"::warning::merge call returned: {exc}\n")
else:
raise
def process_once(*, dry_run: bool = False) -> int:
@@ -445,18 +435,42 @@ def process_once(*, dry_run: bool = False) -> int:
commits = get_pull_commits(pr_number)
current_base = pr_has_current_base(pr, commits, main_sha)
pr_status = get_combined_status(head_sha)
pr_labels = label_names(pr)
decision = evaluate_merge_readiness(
main_status=main_status,
pr_status=pr_status,
required_contexts=contexts,
pr_has_current_base=current_base,
pr_labels=pr_labels,
)
print(f"::notice::PR #{pr_number} decision={decision.action}: {decision.reason}")
if decision.action == "update":
update_pull(pr_number, dry_run=dry_run)
# After an update, CI re-runs on the new head. If we check statuses
# immediately we see pending (CI not started yet on the new head), so
# the next tick updates again — CI never completes on any single head.
# Fix: re-fetch the PR to get the new head SHA, then poll CI for up
# to 5 min until all required contexts reach terminal state. If CI
# finishes in time, proceed to merge on the same tick.
if not dry_run:
updated_pr = get_pull(pr_number)
new_head = updated_pr.get("head", {}).get("sha", "")
if new_head and new_head != head_sha:
sys.stderr.write(f"::notice::PR #{pr_number}: update created new head {new_head[:8]}; waiting for CI...\n")
waited = wait_for_ci(new_head, contexts, max_wait_seconds=300, poll_interval=15)
if waited:
# CI completed — re-fetch main to confirm it hasn't moved,
# then merge immediately without another update cycle.
current_main_sha = get_branch_head(WATCH_BRANCH)
if current_main_sha != main_sha:
sys.stderr.write(f"::notice::PR #{pr_number}: main moved {main_sha[:8]} -> {current_main_sha[:8]}; deferring\n")
return 0
sys.stderr.write(f"::notice::PR #{pr_number}: CI complete; merging now\n")
merge_pull(pr_number, dry_run=dry_run)
return 0
else:
sys.stderr.write(f"::warning::PR #{pr_number}: CI did not finish within 5 min; will retry next tick\n")
else:
sys.stderr.write(f"::notice::PR #{pr_number}: update did not change head SHA; will retry\n")
post_comment(
pr_number,
(
@@ -466,23 +480,14 @@ def process_once(*, dry_run: bool = False) -> int:
dry_run=dry_run,
)
return 0
if decision.action == "wait":
# Required contexts are not green. Auto-hold so the queue stops cycling
# on this PR and processes the next. Holds are removed manually once the
# blocker (e.g. qa/sec gate, missing SOP_TIER_CHECK_TOKEN) is resolved.
add_hold_label(pr_number, dry_run=dry_run)
post_comment(
pr_number,
(
f"merge-queue: auto-held — required contexts not green: "
f"{decision.reason}. "
"Remove the `merge-queue-hold` label and re-label `merge-queue` "
"to restart queue processing once the blocker is resolved."
),
dry_run=dry_run,
)
return 0
if decision.ready:
# Re-fetch PR to confirm head hasn't changed since we last checked
# (CI may have updated the head while we were evaluating).
current_pr = get_pull(pr_number)
current_head = current_pr.get("head", {}).get("sha", "")
if current_head != head_sha:
print(f"::notice::PR #{pr_number} head changed {head_sha[:8]} -> {current_head[:8]}; re-evaluating")
return 0
latest_main_sha = get_branch_head(WATCH_BRANCH)
if latest_main_sha != main_sha:
print(
@@ -490,44 +495,7 @@ def process_once(*, dry_run: bool = False) -> int:
"deferring to next tick"
)
return 0
try:
merge_pull(pr_number, dry_run=dry_run)
except MergePermissionError as exc:
# HTTP 403/404/405. Distinguish status-check gate (405 with
# "Not all required status checks") from a genuine permission
# error. Case-insensitive match — Gitea uses "Not all required..."
# (capital N) while other paths may return lowercase.
msg_lower = str(exc).lower()
is_status_check_failure = "not all required status checks successful" in msg_lower
if is_status_check_failure:
# Gitea's merge gate blocked us — a required context (e.g.
# E2E Chat, qa-review, security-review) is failing. Auto-add
# hold so the queue skips this PR and processes the next.
add_hold_label(pr_number, dry_run=dry_run)
post_comment(
pr_number,
(
"merge-queue: merge blocked by Gitea's status-check gate "
"(E2E Chat, qa-review, security-review, or other required "
"context failing). Auto-held via `merge-queue-hold`. "
"Remove the hold label to requeue once CI is green."
),
dry_run=dry_run,
)
return 0
# Genuine permission error — token lacks Can-merge.
sys.stderr.write(f"::error::merge permission error for PR #{pr_number}: {exc}\n")
post_comment(
pr_number,
(
"merge-queue: merge failed with HTTP 405 'User not allowed to merge PR'. "
"No available token has Can-merge permission on this repo. "
"Fix: grant Can-merge to a token, or add a maintain/admin collaborator. "
"Skipping to next queued PR on next tick."
),
dry_run=dry_run,
)
return 0
merge_pull(pr_number, dry_run=dry_run)
return 0
return 0
@@ -118,64 +118,3 @@ def test_merge_decision_updates_stale_pr_before_merge():
assert decision.ready is False
assert decision.action == "update"
def test_MergePermissionError_inherits_from_ApiError():
assert issubclass(mq.MergePermissionError, mq.ApiError)
def test_MergePermissionError_message_preserved():
exc = mq.MergePermissionError("POST /merge -> HTTP 405: User not allowed")
assert "405" in str(exc)
assert "User not allowed" in str(exc)
def test_merge_decision_waits_when_required_contexts_not_green():
"""When a required context (e.g. qa-review, E2E Chat) is not success, the
decision is 'wait' — the queue can then auto-hold on this."""
required = [
"CI / all-required (pull_request)",
"sop-checklist / all-items-acked (pull_request)",
"qa-review / approved (pull_request)",
]
decision = mq.evaluate_merge_readiness(
main_status={
"state": "success",
"statuses": [{"context": "CI / all-required (push)", "status": "success"}],
},
pr_status={
"state": "failure",
"statuses": [
{"context": "CI / all-required (pull_request)", "status": "success"},
{"context": "sop-checklist / all-items-acked (pull_request)", "status": "success"},
{"context": "qa-review / approved (pull_request)", "status": "failure"},
],
},
required_contexts=required,
pr_has_current_base=True,
pr_labels=None,
)
assert decision.ready is False
assert decision.action == "wait"
assert "qa-review" in decision.reason
def test_tier_low_sop_checklist_pending_soft_fail():
"""tier:low PRs get soft-fail on sop-checklist: pending is accepted."""
required = ["sop-checklist / all-items-acked (pull_request)"]
statuses = {
"sop-checklist / all-items-acked (pull_request)": {"status": "pending"}
}
ok, missing = mq.required_contexts_green(statuses, required, pr_labels={"tier:low"})
assert ok is True
assert missing == []
def test_tier_low_sop_checklist_failure_not_soft_fail():
"""tier:low soft-fail only covers pending, not actual failure."""
required = ["sop-checklist / all-items-acked (pull_request)"]
statuses = {
"sop-checklist / all-items-acked (pull_request)": {"status": "failure"}
}
ok, missing = mq.required_contexts_green(statuses, required, pr_labels={"tier:low"})
assert ok is False
+6
View File
@@ -32,6 +32,12 @@ on:
# iterating all open PRs when PR_NUMBER is empty.
workflow_dispatch:
# Cancel stale runs so the 8-runner pool stays available for PR jobs.
# Per-SHA group ensures push and cron runs at different SHAs don't cancel each other.
concurrency:
group: gate-check-v3-${{ github.event.pull_request.head.sha || github.sha }}
cancel-in-progress: true
permissions:
# read: contents — for checkout (base ref, not PR head for security)
# read: pull-requests — for reading PR info via API
+1 -2
View File
@@ -162,7 +162,6 @@ jobs:
exit 1
fi
python -m twine upload \
--verbose \
--repository pypi \
--username __token__ \
--password "$PYPI_TOKEN" \
@@ -267,7 +266,7 @@ jobs:
fi
GITEA_URL="${GITEA_URL:-https://git.moleculesai.app}"
TEMPLATES="claude-code hermes openclaw codex langgraph autogen"
TEMPLATES="claude-code hermes openclaw codex langgraph crewai autogen deepagents gemini-cli"
FAILED=""
SKIPPED=""
-1
View File
@@ -89,7 +89,6 @@ on:
permissions:
contents: read
pull-requests: read
secrets: read # required for SOP_TIER_CHECK_TOKEN team-membership probe
jobs:
# bp-exempt: PR review bot signal; required merge state is enforced by CI / all-required.
@@ -44,6 +44,12 @@ on:
- ".github/scripts/lint_secret_pattern_drift.py"
- ".githooks/pre-commit"
# Cancel stale runs to keep the 8-runner pool available for PR jobs.
# Per-SHA group ensures push and scheduled runs at different SHAs don't cancel each other.
concurrency:
group: secret-pattern-drift-${{ github.event.pull_request.head.sha || github.sha }}
cancel-in-progress: true
env:
GITHUB_SERVER_URL: https://git.moleculesai.app
-1
View File
@@ -16,7 +16,6 @@ on:
permissions:
contents: read
pull-requests: read
secrets: read # required for SOP_TIER_CHECK_TOKEN team-membership probe
jobs:
# bp-exempt: PR security review bot signal; required merge state is enforced by CI / all-required.
+5
View File
@@ -22,6 +22,11 @@ on:
- cron: '17 4 * * 1' # Mondays at 04:17 UTC
workflow_dispatch:
# Cancel stale runs to keep the 8-runner pool available for PR jobs.
concurrency:
group: weekly-platform-go-${{ github.event.pull_request.head.sha || github.sha }}
cancel-in-progress: true
permissions:
contents: read
statuses: write
+1
View File
@@ -41,3 +41,4 @@
{"name": "mock-bigorg", "repo": "molecule-ai/molecule-ai-org-template-mock-bigorg", "ref": "main"}
]
}
// Triggered by Integration Tester at 2026-05-10T08:52Z
@@ -96,70 +96,13 @@ var fallbackRuntimes = map[string]struct{}{
// Caller logs + falls back to fallbackRuntimes on any error. Not
// returning the fallback here ourselves so the caller can decide
// how loud to be about the miss (prod = WARN, tests = silent).
// stripJSON5Comments removes a JSON5-style // trailing comment from manifest.json.
// The Integration Tester appends "// Triggered by ..." at the very end of the file.
// This comment is always after the final closing brace, so we scan only that
// suffix rather than trying to track string-context across the whole file.
// This avoids false-positives on legitimate // in URL values (e.g. http://foo.com/bar).
func stripJSON5Comments(data []byte) []byte {
// Find the last '}' — everything before it is guaranteed standard JSON.
lastBrace := -1
for i := len(data) - 1; i >= 0; i-- {
if data[i] == '}' {
lastBrace = i
break
}
}
if lastBrace == -1 {
return data // no JSON structure found — return as-is, json.Unmarshal will error
}
// Everything after lastBrace is the trailing suffix to clean.
suffixStart := lastBrace + 1
if suffixStart >= len(data) {
return data // no suffix
}
suffix := data[suffixStart:]
// Strip leading whitespace at the start of the suffix.
cleanSuffix := trimLeadingWhitespace(suffix)
if len(cleanSuffix) == 0 || cleanSuffix[0] != '/' {
return data // suffix is empty or starts with non-comment — nothing to strip
}
// Remove the trailing comment (everything from the first // to end of file).
// Rebuild: prefix + suffix with comment stripped.
before := data[:suffixStart]
// Trim trailing whitespace from before so we don't leave a dangling newline.
trimmedBefore := trimTrailingWhitespace(before)
// Append a single newline so the JSON file ends cleanly.
result := append(trimmedBefore, '\n')
return result
}
func trimLeadingWhitespace(b []byte) []byte {
i := 0
for i < len(b) && (b[i] == ' ' || b[i] == '\t' || b[i] == '\n' || b[i] == '\r') {
i++
}
return b[i:]
}
func trimTrailingWhitespace(b []byte) []byte {
i := len(b)
for i > 0 && (b[i-1] == ' ' || b[i-1] == '\t' || b[i-1] == '\n' || b[i-1] == '\r') {
i--
}
return b[:i]
}
func loadRuntimesFromManifest(path string) (map[string]struct{}, error) {
data, err := os.ReadFile(path)
if err != nil {
return nil, err
}
// The Integration Tester appends "// Triggered by ..." to manifest.json.
// json.Unmarshal rejects it; strip // comments first (same as clone-manifest.sh).
clean := stripJSON5Comments(data)
var m manifestFile
if err := json.Unmarshal(clean, &m); err != nil {
if err := json.Unmarshal(data, &m); err != nil {
return nil, err
}
out := map[string]struct{}{
@@ -83,70 +83,6 @@ func TestLoadRuntimesFromManifest_MalformedJSON(t *testing.T) {
}
}
func TestLoadRuntimesFromManifest_TrailingJSON5Comment(t *testing.T) {
// The Integration Tester appends "// Triggered by Integration Tester at ..."
// to manifest.json after cloning. json.Unmarshal rejects it; stripJSON5Comments
// must remove the trailing comment so load succeeds.
dir := t.TempDir()
path := filepath.Join(dir, "manifest.json")
_ = os.WriteFile(path, []byte(`{
"workspace_templates": [
{"name": "langgraph", "repo": "org/t"}
]
}
// Triggered by Integration Tester at 2026-05-10T08:52Z`), 0600)
got, err := loadRuntimesFromManifest(path)
if err != nil {
t.Fatalf("load failed despite trailing comment: %v", err)
}
if _, ok := got["langgraph"]; !ok {
t.Errorf("langgraph missing from result: %v", keys(got))
}
}
func TestStripJSON5Comments(t *testing.T) {
cases := []struct {
name string
in string
want string
}{
{
name: "trailing comment after closing brace removed",
in: "{}\n// Triggered by Integration Tester\n",
want: "{}\n",
},
{
name: "embedded_in_url_preserved",
in: `{"url":"http://foo.com/bar"}`,
want: `{"url":"http://foo.com/bar"}`,
},
{
name: "no_closing_brace_returns_input_unchanged",
in: "no json here // comment",
want: "no json here // comment",
},
{
name: "comment_only_after_closing_brace_stripped",
in: `{"a":1}` + "\n// Triggered by Integration Tester at 2026-05-10T08:52Z",
want: `{"a":1}` + "\n",
},
{
name: "clean_json_unchanged",
in: `{"workspace_templates":[]}` + "\n",
want: `{"workspace_templates":[]}` + "\n",
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got := string(stripJSON5Comments([]byte(tc.in)))
if got != tc.want {
t.Errorf("stripJSON5Comments(%q): got %q, want %q", tc.in, got, tc.want)
}
})
}
}
// TestRealManifestParses — sanity check against the actual
// monorepo manifest.json so a future schema change to that file
// (e.g. workspace_templates → workspace_runtime_templates) surfaces