Merge pull request 'fix(ci)(interim): disable status-reaper + main-red-watchdog crons (machinery-down)' (#645) from infra/interim-disable-reaper-watchdog-crons into main

RFC#420 Option-C machinery has been down ~2.5h:
- status-reaper rev2 (PR#633, merged 01:48Z): 0 'Compensated by status-reaper'
  status on the last 14 main commits. Schedule reds stranded on stale
  commits despite the rev2 sweep-last-10 design.
- main-red-watchdog: 'Failing after 10m56s' with timeout-minutes:5 — runner
  saturation queue-lag pushed it past its own timeout. No [main-red] issues
  filed during the outage despite 5 reds on HEAD e7965a0f at the high
  watermark.

Both workflows were themselves contributing to the red pileup on main +
queuing the ubuntu-latest pool. Cheap-and-safe interim: comment out the
schedule: blocks. workflow_dispatch: stays so they can be triggered
manually for debugging.

Re-enable after:
1. rev3 lands (likely scan_workflows() should LOG-and-skip rather than
   sys.exit on a malformed workflow; list_recent_commit_shas() should
   degrade gracefully)
2. Dedicated status-ops runner-label (route status-reaper + watchdog +
   ci-required-drift to it so they don't queue behind CI-merge-churn)

Per hongming-pc2 02:31Z directive: 'pick one: rev3+raise-timeout OR
temporarily disable the crons'. Choosing disable for safety while rev3
investigation proceeds.

Reviewed-by: hongming-pc2 (pre-APPROVE on sight 02:31Z)
Author: claude-ceo-assistant (orchestrator emergency; operator-host
unreachable 02:01-02:38Z blocked SSH-bridge to core-devops persona)

Cross-links: task #90 (rev2), task #75 (main-red sweep), RFC#420 Option-C

.gitea/scripts/tests/test_review_check.sh

@@ -317,7 +317,8 @@ JQ_FILTER='.[]
 
 T12_INPUT='[{"state":"APPROVED","dismissed":false,"user":{"login":"core-devops"}},{"state":"CHANGES_REQUESTED","dismissed":false,"user":{"login":"bob"}},{"state":"APPROVED","dismissed":false,"user":{"login":"alice"}},{"state":"APPROVED","dismissed":true,"user":{"login":"carol"}}]'
 
-T12_CANDIDATES=$(echo "$T12_INPUT" | /tmp/jq -r "$JQ_FILTER" 2>/dev/null | sort -u)
+JQ_CMD=$(command -v jq 2>/dev/null || echo /tmp/jq)
+T12_CANDIDATES=$(echo "$T12_INPUT" | "$JQ_CMD" -r "$JQ_FILTER" 2>/dev/null | sort -u)
 assert_contains "T12 jq: core-devops (non-author APPROVED) in candidates" "core-devops" "$T12_CANDIDATES"
 assert_eq "T12 jq: alice (author) NOT in candidates" "" "$(echo "$T12_CANDIDATES" | grep '^alice$' || true)"
 assert_eq "T12 jq: carol (dismissed) NOT in candidates" "" "$(echo "$T12_CANDIDATES" | grep '^carol$' || true)"

.gitea/workflows/main-red-watchdog.yml

@@ -37,10 +37,13 @@ name: main-red-watchdog
 # "unknown on type" when `workflow_dispatch.inputs.X` is present. Revisit
 # when Gitea ≥ 1.23 is fleet-wide.
 on:
-  schedule:
-    # Hourly at :05 — task spec calls for "off-zero" (`5 * * * *`),
-    # offset from :17 (ci-required-drift) and :00 (peak cron load).
-    - cron: '5 * * * *'
+  # SCHEDULE DISABLED 2026-05-12 — interim per RFC#420 Option-C machinery-down emergency
+  # Watchdog timing out behind runner saturation; rev3+dedicated-runner-label in flight
+  # Re-enable after rev3 lands + runner saturation root resolved
+  #   schedule:
+  #     # Hourly at :05 — task spec calls for "off-zero" (`5 * * * *`),
+  #     # offset from :17 (ci-required-drift) and :00 (peak cron load).
+  #     - cron: '5 * * * *'
   workflow_dispatch:
 
 # Read commit status + branch ref + issues; write issues (open/PATCH/close).

.gitea/workflows/review-check-tests.yml

@@ -0,0 +1,70 @@
+name: review-check-tests
+
+# Runs review-check.sh regression tests on every PR + push that touches
+# the evaluator script or its test fixtures.
+#
+# Follows RFC#324 follow-up (issue #540):
+#   .gitea/scripts/review-check.sh is load-bearing for PR merge gates.
+#   It has ZERO production CI coverage. This workflow closes that gap.
+#
+# Design choices:
+#   - Bash test harness (not bats). The existing test_review_check.sh
+#     uses a custom assert_eq/assert_contains framework that is already
+#     working and covers all 13 acceptance criteria (issue #540 §Acceptance).
+#     Converting to bats would be refactoring, not closing the gap.
+#   - No bats dependency: the runner-base image needs no extra tooling.
+#   - continue-on-error: false — these tests must pass; a failure means
+#     the review-gate evaluator is broken and must not be merged.
+
+on:
+  push:
+    branches: [main, staging]
+    paths:
+      - '.gitea/scripts/review-check.sh'
+      - '.gitea/scripts/tests/test_review_check.sh'
+      - '.gitea/scripts/tests/_review_check_fixture.py'
+      - '.gitea/workflows/review-check-tests.yml'
+  pull_request:
+    branches: [main, staging]
+    paths:
+      - '.gitea/scripts/review-check.sh'
+      - '.gitea/scripts/tests/test_review_check.sh'
+      - '.gitea/scripts/tests/_review_check_fixture.py'
+      - '.gitea/workflows/review-check-tests.yml'
+  workflow_dispatch:
+
+env:
+  GITHUB_SERVER_URL: https://git.moleculesai.app
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  test:
+    name: review-check.sh regression tests
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+
+      - name: Install jq
+        # Required for T12 jq-filter test case. Gitea Actions runners (ubuntu-latest
+        # label) do not bundle jq. Install via apt-get first (reliable for Ubuntu
+        # runners with internet access to package mirrors). Falls back to GitHub
+        # binary download. GitHub releases may be blocked on some runner networks
+        # (infra#241 follow-up).
+        continue-on-error: true
+        run: |
+          if apt-get update -qq && apt-get install -y -qq jq; then
+            echo "::notice::jq installed via apt-get: $(jq --version)"
+          elif timeout 120 curl -sSL \
+            "https://github.com/jqlang/jq/releases/download/jq-1.7.1/jq-linux-amd64" \
+            -o /usr/local/bin/jq && chmod +x /usr/local/bin/jq; then
+            echo "::notice::jq binary downloaded: $(/usr/local/bin/jq --version)"
+          else
+            echo "::warning::jq install failed — apt-get and GitHub download both failed."
+          fi
+          jq --version 2>/dev/null || echo "::notice::jq not yet available — continuing"
+
+      - name: Run review-check.sh regression suite
+        run: bash .gitea/scripts/tests/test_review_check.sh

.gitea/workflows/status-reaper.yml

@@ -53,13 +53,16 @@ name: status-reaper
 # `inputs:` block here. Gitea 1.22.6 rejects the whole workflow as
 # "unknown on type" when `workflow_dispatch.inputs.X` is present.
 on:
-  schedule:
-    # Every 5 minutes. Off-zero alignment with sibling cron workflows:
-    # ci-required-drift (`:17`), main-red-watchdog (`:05`),
-    # railway-pin-audit (`:23`). 5-min cadence gives a tight enough
-    # close on schedule-triggered false-reds that main-red-watchdog
-    # (hourly :05) almost never files an issue on the false case.
-    - cron: '*/5 * * * *'
+  # SCHEDULE DISABLED 2026-05-12 — interim per RFC#420 Option-C machinery-down emergency
+  # Reaper rev2 not compensating + watchdog timeout-cascade; rev3 in flight
+  # Re-enable after rev3 lands + runner saturation root resolved
+  #   schedule:
+  #     # Every 5 minutes. Off-zero alignment with sibling cron workflows:
+  #     # ci-required-drift (`:17`), main-red-watchdog (`:05`),
+  #     # railway-pin-audit (`:23`). 5-min cadence gives a tight enough
+  #     # close on schedule-triggered false-reds that main-red-watchdog
+  #     # (hourly :05) almost never files an issue on the false case.
+  #     - cron: '*/5 * * * *'
   workflow_dispatch:
 
 # Compensating-status POST needs write on repo statuses; no other

CONTRIBUTING.md

@@ -156,6 +156,16 @@ and run CI manually.
 | python-lint | pytest with coverage |
 | e2e-api | Full API test suite (62 tests) |
 | shellcheck | Shell script linting |
+| review-check-tests | `review-check.sh` evaluator regression suite (13 scenarios) |
+| ops-scripts | Python unittest suite for `scripts/*.py` |
+
+## Local Testing
+
+### review-check.sh
+```bash
+bash .gitea/scripts/tests/test_review_check.sh
+```
+Runs the full regression suite against a fixture HTTP server. No network access required.
 
 ## Code Style