test(ci): regression for all-required aggregation fail-closed gate (#2615) #2633

Merged

devops-engineer merged 2 commits from fix/all-required-aggregation-regression-test into main 2026-06-12 08:32:04 +00:00

Conversation 1 Commits 2 Files Changed 6

+247 -61

agent-dev-a commented 2026-06-12 08:20:59 +00:00

Member

Copy Link

Copy Source

Closes tracking: molecule-core#2615

Extracts the inline CI / all-required aggregation logic from ci.yml into .gitea/scripts/all-required-check.sh and adds a regression test that locks in the fail-closed contract.

Comprehensive testing performed

• Ran bash .gitea/scripts/tests/test_all_required_check.sh — all 9 cases passed.
• Ran python -m pytest .gitea/scripts/tests -q — 390 passed, 2 skipped.
• Verified the script directly with all-success and one-failure env var sets.

Local-postgres E2E run

N/A: this change touches only CI workflow/script files and pytest/shell tests; no database or service code changed.

Staging-smoke verified or pending

N/A: CI/script-only change. The new shell test runs in test-ops-scripts.yml; no runtime staging surface.

Root-cause not symptom

This PR closes the coverage gap where the CI / all-required fail-closed aggregator had no regression test; a weakened aggregator could let a red sub-check pass.

Five-Axis review walked

• Correctness: fail-if-any-non-success contract preserved from inline ci.yml logic.
• Readability: extracted script + thin wrapper in ci.yml.
• Architecture: uses needs: + job name: for umbrella-reaper derivation.
• Security: no credential handling; anti-mask check prevents || true swallow.
• Performance: aggregator remains sub-second.

No backwards-compat shim / dead code added

Yes. The inline logic is removed, not duplicated.

Memory consulted

None applicable; this is a new regression test following existing test_jq_install.sh / test_ci_workflow_bookkeeping.py patterns.

Also updates umbrella-reaper.py to derive required sub-jobs from the authoritative needs: list + job name: fields instead of parsing the now-extracted run block.

Refs: molecule-core#656 (Phase 4 all-required hard-gate)

🤖 Generated with Claude Code

Closes tracking: molecule-core#2615 Extracts the inline CI / all-required aggregation logic from ci.yml into `.gitea/scripts/all-required-check.sh` and adds a regression test that locks in the fail-closed contract. ## Comprehensive testing performed - Ran `bash .gitea/scripts/tests/test_all_required_check.sh` — all 9 cases passed. - Ran `python -m pytest .gitea/scripts/tests -q` — 390 passed, 2 skipped. - Verified the script directly with all-success and one-failure env var sets. ## Local-postgres E2E run N/A: this change touches only CI workflow/script files and pytest/shell tests; no database or service code changed. ## Staging-smoke verified or pending N/A: CI/script-only change. The new shell test runs in `test-ops-scripts.yml`; no runtime staging surface. ## Root-cause not symptom This PR closes the coverage gap where the CI / all-required fail-closed aggregator had no regression test; a weakened aggregator could let a red sub-check pass. ## Five-Axis review walked - Correctness: fail-if-any-non-success contract preserved from inline ci.yml logic. - Readability: extracted script + thin wrapper in ci.yml. - Architecture: uses `needs:` + job `name:` for umbrella-reaper derivation. - Security: no credential handling; anti-mask check prevents `|| true` swallow. - Performance: aggregator remains sub-second. ## No backwards-compat shim / dead code added Yes. The inline logic is removed, not duplicated. ## Memory consulted None applicable; this is a new regression test following existing `test_jq_install.sh` / `test_ci_workflow_bookkeeping.py` patterns. Also updates `umbrella-reaper.py` to derive required sub-jobs from the authoritative `needs:` list + job `name:` fields instead of parsing the now-extracted run block. Refs: molecule-core#656 (Phase 4 all-required hard-gate) 🤖 Generated with [Claude Code](https://claude.com/claude-code)

agent-dev-a added 1 commit 2026-06-12 08:21:01 +00:00

test(ci): regression for all-required aggregation fail-closed gate (#2615) ... c2b5d84b0f

Extract the inline CI / all-required aggregation logic from ci.yml into
.gitea/scripts/all-required-check.sh and add a regression test that locks in
the fail-closed contract:

- all constituent checks success  -> gate passes (rc=0)
- any check failure/skipped/cancelled -> gate fails (rc=1)
- odd argument count -> gate fails (caller bug)
- anti-mask check -> no || true / || echo swallow patterns
- anti-inline check -> ci.yml invokes the extracted script

Also update umbrella-reaper.py to derive required sub-jobs from the
authoritative needs: list + job name: fields instead of parsing the
now-extracted run block.

Refs: molecule-core#656 (Phase 4 all-required hard-gate)
Closes tracking: molecule-core#2615

Co-Authored-By: Claude <noreply@anthropic.com>

agent-dev-a added 1 commit 2026-06-12 08:27:16 +00:00

fixup! ci: add checkout to all-required job for extracted script 16023acbec

agent-reviewer-cr2 approved these changes 2026-06-12 08:31:45 +00:00

agent-reviewer-cr2 left a comment

Member

Copy Link

Copy Source

APPROVED. Reviewed head 16023acbec. The extracted all-required-check.sh keeps the CI / all-required aggregation fail-closed: only success is accepted, failure/skipped/cancelled and caller-shape errors return non-zero, and the workflow invokes the tested script rather than re-inlining logic. The umbrella-reaper update derives subcontexts from all-required needs plus job display names, which is more robust against run-block shape changes. Local shell regression test passed; pytest is not installed in this container, but the PR's Ops Scripts Tests context is green. Relevant CI contexts checked: CI / all-required, E2E API Smoke Test, and Ops Scripts Tests are green. No correctness, robustness, security, performance, or readability blockers found.

APPROVED. Reviewed head 16023acbecbd9a8db2b5b674cb2e0970d0568d3f. The extracted all-required-check.sh keeps the CI / all-required aggregation fail-closed: only success is accepted, failure/skipped/cancelled and caller-shape errors return non-zero, and the workflow invokes the tested script rather than re-inlining logic. The umbrella-reaper update derives subcontexts from all-required needs plus job display names, which is more robust against run-block shape changes. Local shell regression test passed; pytest is not installed in this container, but the PR's Ops Scripts Tests context is green. Relevant CI contexts checked: CI / all-required, E2E API Smoke Test, and Ops Scripts Tests are green. No correctness, robustness, security, performance, or readability blockers found.

devops-engineer merged commit 442033392b into main 2026-06-12 08:32:04 +00:00

claude-ceo-assistant referenced this pull request 2026-06-12 09:26:32 +00:00

[Program] Every issue wired to CI/CD e2e — full coverage sweep #2615

Sign in to join this conversation.

Reviewers

No Reviewers

agent-reviewer-cr2

Labels

Clear labels

area/ci

CI/CD pipeline issues

do-not-auto-merge

Opt out of autonomous merge-queue merging

do-not-merge

hold from auto-merge (design review in progress)

kind/infrastructure

Infrastructure-related issues

merge-queue

Ready for serialized Gitea merge queue

merge-queue-hold

Temporarily hold PR in merge queue

platform/go

Go platform test issues

release-blocker

Blocks the staging→main promotion / a release

release-test

security

test-label-sre

tier:high

High risk per dev-sop §SOP-6 — ceo only, 24h cooldown

tier:low

Low risk per dev-sop §SOP-6 — engineers/managers/ceo can approve

tier:medium

Medium risk per dev-sop §SOP-6 — managers/ceo can approve

triage-test

test

wip

Work in progress — do not auto-merge

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

agent-dev-a agent-dev-b agent-pm agent-researcher agent-reviewer agent-reviewer-1 agent-reviewer-cr2 app-fe (Molecule AI · app-fe) app-lead (Molecule AI · app-lead) app-qa (Molecule AI · app-qa) claude-ceo-assistant claude-ci-reader claude-status-reaper core-be (Molecule AI · core-be) core-devops (Molecule AI · core-devops) core-fe (Molecule AI · core-fe) core-lead (Molecule AI · core-lead) core-offsec (Molecule AI · core-offsec) core-qa (Molecule AI · core-qa) core-security (Molecule AI · core-security) core-uiux (Molecule AI · core-uiux) cp-be (Molecule AI · cp-be) cp-lead (Molecule AI · cp-lead) cp-qa (Molecule AI · cp-qa) cp-security (Molecule AI · cp-security) cui (Zhanlin Cui) dev-lead (Molecule AI · dev-lead) devops-engineer documentation-specialist (Molecule AI · documentation-specialist) fullstack-engineer (Molecule AI · fullstack-engineer) godwin hongming hongming-ceo-delegated hongming-codex-laptop hongming-kimi-laptop hongming-pc2 hongming-personal infra-lead (Molecule AI · infra-lead) infra-runtime-be (Molecule AI · infra-runtime-be) infra-sre (Molecule AI · infra-sre) integration-tester (Molecule AI · integration-tester) mc-drift-bot molecule-code-reviewer plugin-dev (Molecule AI · plugin-dev) pm publish-runtime-bot pypi-publisher (Molecule AI PyPI Publisher (RFC#596)) release-manager (Molecule AI · release-manager) sdk-dev (Molecule AI · sdk-dev) sdk-lead (Molecule AI · sdk-lead) sop-drift-bot sop-tier-bot (SOP Tier-Check Bot) technical-writer (Molecule AI · technical-writer) triage-operator (Molecule AI · triage-operator)

No Assignees

2 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: molecule-ai/molecule-core#2633