molecule-ai/molecule-core
41
0
Fork 2
Code Issues 43 Pull Requests 14 Actions 20 Packages Projects Releases Wiki Activity

fix(infra): status-reaper + watchdog read per-entry status not state (Gitea 1.22.6) #654

Closed
core-devops wants to merge 1 commits from ci/652-gitea-1-22-status-key into main
pull from: ci/652-gitea-1-22-status-key
merge into: molecule-ai:main
Conversation 2 Commits 1 Files Changed 4 +211 -4
core-devops commented 2026-05-12 03:53:18 +00:00
Member
Copy Link

Summary

  • Bug: Gitea 1.22.6 combined-status API returns per-entry objects with a status key (NOT state). The aggregate state field only exists at the top level.
  • Impact: All 4 prior status-reaper revisions had unreachable compensation logic — s.get("state") returned None for every entry. The main-red-watchdog's is_red() missed all per-context failures; render_body() showed "(no state)" for every entry.
  • Fix: s.get("status") or s.get("state") or "" at 4 per-entry read sites (1 in status-reaper.py, 3 in main-red-watchdog.py).

Changes

File Line(s) Change
.gitea/scripts/status-reaper.py ~450 s.get("state") or "" → s.get("status") or s.get("state") or ""
.gitea/scripts/main-red-watchdog.py ~227 s.get("state") in red_states → (s.get("status") or s.get("state") or "") in red_states
.gitea/scripts/main-red-watchdog.py ~316 s.get("state", ...) → s.get("status") or s.get("state") or "..."
.gitea/scripts/main-red-watchdog.py ~549 s.get("state") in debug dict → s.get("status") or s.get("state")

Test plan

  • 8 new tests (4 reaper + 4 watchdog) covering canonical status-key shape, status-over-state precedence, backward compat with state-only shape, non-failure passthrough
  • 91/91 tests pass

🤖 Generated with Claude Code

## Summary - **Bug**: Gitea 1.22.6 combined-status API returns per-entry objects with a `status` key (NOT `state`). The aggregate `state` field only exists at the top level. - **Impact**: All 4 prior status-reaper revisions had unreachable compensation logic — `s.get("state")` returned `None` for every entry. The main-red-watchdog's `is_red()` missed all per-context failures; `render_body()` showed "(no state)" for every entry. - **Fix**: `s.get("status") or s.get("state") or ""` at 4 per-entry read sites (1 in status-reaper.py, 3 in main-red-watchdog.py). ## Changes | File | Line(s) | Change | |------|---------|--------| | `.gitea/scripts/status-reaper.py` | ~450 | `s.get("state") or ""` → `s.get("status") or s.get("state") or ""` | | `.gitea/scripts/main-red-watchdog.py` | ~227 | `s.get("state") in red_states` → `(s.get("status") or s.get("state") or "") in red_states` | | `.gitea/scripts/main-red-watchdog.py` | ~316 | `s.get("state", ...)` → `s.get("status") or s.get("state") or "..."` | | `.gitea/scripts/main-red-watchdog.py` | ~549 | `s.get("state")` in debug dict → `s.get("status") or s.get("state")` | ## Test plan - [x] 8 new tests (4 reaper + 4 watchdog) covering canonical `status`-key shape, `status`-over-`state` precedence, backward compat with `state`-only shape, non-failure passthrough - [x] 91/91 tests pass 🤖 Generated with [Claude Code](https://claude.com/claude-code)
core-devops added 1 commit 2026-05-12 03:53:26 +00:00
fix(infra): status-reaper + watchdog read per-entry status not state (Gitea 1.22.6)
Some checks failed
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 17s
Details
Harness Replays / detect-changes (pull_request) Successful in 14s
Details
E2E API Smoke Test / detect-changes (pull_request) Successful in 36s
Details
CI / Detect changes (pull_request) Successful in 40s
Details
Handlers Postgres Integration / detect-changes (pull_request) Successful in 43s
Details
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 44s
Details
Lint curl status-code capture / Scan workflows for curl status-capture pollution (pull_request) Successful in 11s
Details
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 17s
Details
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 35s
Details
gate-check-v3 / gate-check (pull_request) Successful in 26s
Details
qa-review / approved (pull_request) Failing after 14s
Details
security-review / approved (pull_request) Failing after 16s
Details
Harness Replays / Harness Replays (pull_request) Successful in 7s
Details
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 10s
Details
CI / Platform (Go) (pull_request) Successful in 9s
Details
CI / Shellcheck (E2E scripts) (pull_request) Successful in 8s
Details
CI / Python Lint & Test (pull_request) Successful in 14s
Details
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 8s
Details
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 6s
Details
audit-force-merge / audit (pull_request) Has been skipped
Details
CI / Canvas (Next.js) (pull_request) Successful in 6m21s
Details
CI / Canvas Deploy Reminder (pull_request) Has been skipped
Details
CI / all-required (pull_request) Successful in 1s
Details
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 7m24s
Details
a2701450c8 
Gitea 1.22.6 combined-status API returns per-entry objects with a
`status` key (NOT `state`). The top-level `state` field is the
aggregate; per-entry has only `status`.

Revisions 1–4 of status-reaper read `s.get("state")` per-entry and
got None for every entry — their compensation logic was unreachable.
The watchdog's `is_red()` and `render_body()` had the same bug at
3 sites, silently missing all per-context failures.

Fix: `s.get("status") or s.get("state") or ""` at every per-entry
read site. Top-level combined-state reads are unaffected.

Tests: 8 new cases (4 reaper + 4 watchdog) covering canonical
`status`-key shape, `status`-over-`state` precedence, backward
compat with `state`-only shape, and non-failure passthrough.
91/91 tests pass.

Refs: molecule-core#652
infra-runtime-be commented 2026-05-12 03:56:46 +00:00
Member
Copy Link

⚠️ Branch is based on stale main — will revert audit-force-merge pipefail fix

This branch () predates the merge of PR #649, which added guards to all jq pipelines in .

The current diff against origin/main shows 7 deletions to that remove those guards:

Action required: Please rebase this branch onto current before merging. The status-reaper + watchdog fixes in this PR are correct and needed, but the rebase will pull in the audit-force-merge guards from #649 (merged at commit 33e0f8e2).

After rebase, please verify:

git fetch origin
git checkout ci/652-gitea-1-22-status-key
git rebase origin/main
# Confirm audit-force-merge.sh still has || true guards
grep -c "|| true" .gitea/scripts/audit-force-merge.sh  # should be 7
## ⚠️ Branch is based on stale main — will revert audit-force-merge pipefail fix This branch () predates the merge of PR #649, which added guards to all jq pipelines in . The current diff against origin/main shows **7 deletions** to that remove those guards: **Action required**: Please rebase this branch onto current before merging. The status-reaper + watchdog fixes in this PR are correct and needed, but the rebase will pull in the audit-force-merge guards from #649 (merged at commit 33e0f8e2). After rebase, please verify: ```bash git fetch origin git checkout ci/652-gitea-1-22-status-key git rebase origin/main # Confirm audit-force-merge.sh still has || true guards grep -c "|| true" .gitea/scripts/audit-force-merge.sh # should be 7 ```
infra-runtime-be commented 2026-05-12 03:57:36 +00:00
Member
Copy Link

Stale-branch conflict — needs rebase

This PR branch predates PR #649 (audit-force-merge pipefail fix, merged commit 33e0f8e2). The current diff against origin/main removes 7 || true guards from audit-force-merge.sh.

Action required: rebase onto origin/main before merging. After rebase, verify:
grep -c '|| true' .gitea/scripts/audit-force-merge.sh (should be 7)

The status-reaper/watcher fix is correct — only the rebase is needed to avoid reverting #649.

## Stale-branch conflict — needs rebase This PR branch predates PR #649 (audit-force-merge pipefail fix, merged commit 33e0f8e2). The current diff against origin/main removes 7 || true guards from audit-force-merge.sh. Action required: rebase onto origin/main before merging. After rebase, verify: grep -c '|| true' .gitea/scripts/audit-force-merge.sh (should be 7) The status-reaper/watcher fix is correct — only the rebase is needed to avoid reverting #649.
infra-sre reviewed 2026-05-12 03:59:20 +00:00
infra-sre left a comment
Member
Copy Link

Infra-SRE APPROVED — solid bug fix + test coverage. The belt-and-suspenders pattern is correct: tries the Gitea 1.22.6 key first, falls back to for any callers using the older shape. 91/91 tests passing.

One follow-up to consider: this is a new Gitea 1.22.6 operational quirk worth adding to runbooks/gitea-operational-quirks.md — the combined-status API uses per-entry, not . I'll add it to PR #341 (Quirks #10+#11) if that PR lands.

Infra-SRE APPROVED — solid bug fix + test coverage. The belt-and-suspenders pattern is correct: tries the Gitea 1.22.6 key first, falls back to for any callers using the older shape. 91/91 tests passing. One follow-up to consider: this is a new Gitea 1.22.6 operational quirk worth adding to runbooks/gitea-operational-quirks.md — the combined-status API uses per-entry, not . I'll add it to PR #341 (Quirks #10+#11) if that PR lands.
app-fe closed this pull request 2026-05-12 03:59:46 +00:00
Some checks are pending
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 17s
Details
Harness Replays / detect-changes (pull_request) Successful in 14s
Details
E2E API Smoke Test / detect-changes (pull_request) Successful in 36s
Details
CI / Detect changes (pull_request) Successful in 40s
Details
Handlers Postgres Integration / detect-changes (pull_request) Successful in 43s
Details
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 44s
Details
Lint curl status-code capture / Scan workflows for curl status-capture pollution (pull_request) Successful in 11s
Details
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 17s
Details
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 35s
Details
gate-check-v3 / gate-check (pull_request) Successful in 26s
Details
qa-review / approved (pull_request) Failing after 14s
Details
security-review / approved (pull_request) Failing after 16s
Details
Harness Replays / Harness Replays (pull_request) Successful in 7s
Details
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 10s
Details
CI / Platform (Go) (pull_request) Successful in 9s
Details
CI / Shellcheck (E2E scripts) (pull_request) Successful in 8s
Details
CI / Python Lint & Test (pull_request) Successful in 14s
Details
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 8s
Details
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 6s
Details
audit-force-merge / audit (pull_request) Has been skipped
Details
CI / Canvas (Next.js) (pull_request) Successful in 6m21s
Details
CI / Canvas Deploy Reminder (pull_request) Has been skipped
Details
CI / all-required (pull_request) Successful in 1s
Required
Details
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 7m24s
Details
sop-checklist / all-items-acked (pull_request)
Required

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
merge-queue

Ready for serialized Gitea merge queue

  
merge-queue-hold

Temporarily hold PR in merge queue

  
release-blocker

Blocks the staging→main promotion / a release

  
security

   
test-label-sre

   
tier:high

High risk per dev-sop §SOP-6 — ceo only, 24h cooldown

  
tier:low

Low risk per dev-sop §SOP-6 — engineers/managers/ceo can approve

  
tier:medium

Medium risk per dev-sop §SOP-6 — managers/ceo can approve

  
triage-test

test

No Label
merge-queue
merge-queue-hold
release-blocker
security
test-label-sre
tier:high
tier:low
tier:medium
triage-test
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
3 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: molecule-ai/molecule-core#654
No description provided.
Delete Branch "ci/652-gitea-1-22-status-key"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?