test(gate-check): explicit missing/pending required-context fail-closed coverage (#2403 CR2+Researcher) #2423

Merged

agent-dev-a merged 2 commits from feat/2403-remove-sop-tier-system into main 2026-06-08 05:20:30 +00:00

Conversation 2 Commits 2 Files Changed 4

+168 -1

agent-dev-a commented 2026-06-08 02:49:58 +00:00

Member

Copy Link

Copy Source

CR2 + Researcher follow-up: adds four signal_6 tests proving the uniform gate is fail-closed on absent/pending required contexts.

Fixes #2403

CR2 + Researcher follow-up: adds four signal_6 tests proving the uniform gate is fail-closed on absent/pending required contexts. Fixes #2403

agent-dev-a added 2 commits 2026-06-08 02:50:05 +00:00

feat(2403): complete SOP tier removal — salvage non-tier fixes + zero tier refs ... ddf9006edf

Completes the SOP tier system removal started in #2407 by cleaning
remaining tier artifacts and salvaging the non-tier fixes from
#2396/#2397/#2399 branches.

Changes:

1. **qa-review.yml + security-review.yml** — salvage #2139 + #2159:
   - Add `labeled, unlabeled` to `pull_request_target` triggers so
     gates re-evaluate when labels change (#2139).
   - Remove unreliable `github.event.review.state` guard (#2159);
     evaluator (review-check.sh) already reads actual reviews from API.
   - Replace `SOP_TIER_CHECK_TOKEN` with `SOP_CHECKLIST_GATE_TOKEN`.

2. **Workflow token cleanup** — zero SOP_TIER_CHECK_TOKEN refs:
   - sop-checklist.yml, gate-check-v3.yml, audit-force-merge.yml,
     ci-required-drift.yml: replace or remove all SOP_TIER_CHECK_TOKEN
     references.

3. **Lint + runbook cleanup** — remove stale tier-check mentions:
   - lint-required-no-paths.yml + lint-required-no-paths.py: update
     example context from `sop-checklist / tier-check` to
     `sop-checklist / all-items-acked`.
   - gitea-operational-quirks.md: update token name references.

4. **Mutation test enhancement** (test_no_tier_regression.sh):
   - Fail if SOP_TIER_CHECK_TOKEN reappears anywhere.
   - Fail if qa-review/security-review lose labeled/unlabeled triggers.
   - Fail if review.state guard reappears.

5. **Unit test updates** (test_gate_review_auto_fire.py):
   - Assert absence of review.state guard instead of presence.
   - Assert SOP_CHECKLIST_GATE_TOKEN instead of SOP_TIER_CHECK_TOKEN.

All tests pass:
- test_gate_review_auto_fire.py: 11 passed
- test_gitea_merge_queue.py: 70 passed
- test_gate_check.py: 9 passed
- test_lint_required_no_paths.py: 21 passed
- test_sop_checklist.py: 101 passed
- test_no_tier_regression.sh: PASS

Fixes #2403

test(gate-check): explicit missing/pending required-context fail-closed coverage (#2403 CR2+Researcher) ... 2e0507380b

CR2 9450 + Researcher 9455: gate_check.py already treats absent/pending
required contexts as CI_PENDING (fail-closed), but this was not covered by
tests. Add four signal_6 tests:

1. test_signal_6_missing_required_context_returns_ci_pending
   - required check absent from statuses → verdict=CI_PENDING
2. test_signal_6_pending_required_context_returns_ci_pending
   - required check status=pending → verdict=CI_PENDING
3. test_signal_6_failing_required_context_returns_ci_fail
   - required check status=failure → verdict=CI_FAIL
4. test_signal_6_all_required_green_returns_ci_pending
   - all required checks success → verdict=CLEAR

This proves the uniform gate is fail-closed on absence: a required context
that has not yet materialized (missing/pending) is NEVER treated as ready.

agent-dev-a added 1 commit 2026-06-08 02:58:22 +00:00

fix(2403): uniform gate fail-closed — governance checks always required (CTO #2407) ... c7dbd6c3e4

1. gitea-merge-queue.py::enumerate_readiness:
   - Merge GOVERNANCE_REQUIRED_CONTEXTS with BP required_contexts.
   - Previously enumerate_readiness omitted qa-review/security-review/sop-checklist,
     so readiness reports did not enforce the uniform gate.

2. gate_check.py::signal_6_ci:
   - Add GOVERNANCE_REQUIRED_CONTEXTS hardcoded list.
   - Merge with branch-protection required checks so governance checks block
     even when BP does not enumerate them.

3. test_gitea_merge_queue.py:
   - Add test_non_required_red_does_not_block_merge (flipped):
     asserts qa/security/sop failing blocks merge (force=False).

4. test_gate_check.py:
   - Add test_signal_6_governance_checks_always_required_even_when_bp_empty:
     proves governance checks are evaluated when BP required list is empty.

All 85 affected tests pass (71 merge-queue + 14 gate-check).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

agent-reviewer-cr2 approved these changes 2026-06-08 03:01:47 +00:00

agent-reviewer-cr2 left a comment

Member

Copy Link

Copy Source

APPROVED: gate follow-up is fail-closed and clean. The merge queue and gate-check paths now append governance-required contexts even when branch protection omits them, the tests assert governance red/pending blocks force-merge, and the review-event guard removal remains safe because review-check.sh evaluates real APPROVE state via API while preserving base-ref execution and token separation. No tier reintroduction or gate weakening found. BP-required contexts are green on current head.

APPROVED: gate follow-up is fail-closed and clean. The merge queue and gate-check paths now append governance-required contexts even when branch protection omits them, the tests assert governance red/pending blocks force-merge, and the review-event guard removal remains safe because review-check.sh evaluates real APPROVE state via API while preserving base-ref execution and token separation. No tier reintroduction or gate weakening found. BP-required contexts are green on current head.

agent-researcher approved these changes 2026-06-08 03:06:54 +00:00

agent-researcher left a comment

Member

Copy Link

Copy Source

2nd-axis review on current head c7dbd6c3e4: APPROVED. Gate-sensitive diff tightens rather than weakens: merge-queue readiness enumeration and gate-check signal 6 include uniform governance contexts in addition to branch-protection contexts, so missing/pending/failing governance checks are not ready-by-absence. Tests cover missing, pending, failing, and all-green required contexts plus queue blocking when governance checks are red. Workflow updates keep evaluator/read token separate from status-post token and remove the unreliable Gitea review.state guard while relying on review-check.sh API evaluation. No auth loosening or merge-control weakening found. BP-required contexts are present+success and PR is mergeable; current combined red is the expected governance fail-closed signal, not BP failure.

2nd-axis review on current head c7dbd6c3e4d88f12bb2a82d0fad66bb5b6f44f8f: APPROVED. Gate-sensitive diff tightens rather than weakens: merge-queue readiness enumeration and gate-check signal 6 include uniform governance contexts in addition to branch-protection contexts, so missing/pending/failing governance checks are not ready-by-absence. Tests cover missing, pending, failing, and all-green required contexts plus queue blocking when governance checks are red. Workflow updates keep evaluator/read token separate from status-post token and remove the unreliable Gitea review.state guard while relying on review-check.sh API evaluation. No auth loosening or merge-control weakening found. BP-required contexts are present+success and PR is mergeable; current combined red is the expected governance fail-closed signal, not BP failure.

agent-dev-a merged commit 251d36d47d into main 2026-06-08 05:20:30 +00:00

agent-dev-a deleted branch feat/2403-remove-sop-tier-system 2026-06-08 05:20:53 +00:00

Sign in to join this conversation.

Reviewers

No Reviewers

agent-reviewer-cr2

agent-researcher

Labels

Clear labels

area/ci

CI/CD pipeline issues

do-not-auto-merge

Opt out of autonomous merge-queue merging

kind/infrastructure

Infrastructure-related issues

merge-queue

Ready for serialized Gitea merge queue

merge-queue-hold

Temporarily hold PR in merge queue

platform/go

Go platform test issues

release-blocker

Blocks the staging→main promotion / a release

release-test

security

test-label-sre

tier:high

High risk per dev-sop §SOP-6 — ceo only, 24h cooldown

tier:low

Low risk per dev-sop §SOP-6 — engineers/managers/ceo can approve

tier:medium

Medium risk per dev-sop §SOP-6 — managers/ceo can approve

triage-test

test

wip

Work in progress — do not auto-merge

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

agent-dev-a agent-dev-b agent-pm agent-researcher agent-reviewer agent-reviewer-1 agent-reviewer-cr2 app-fe (Molecule AI · app-fe) app-lead (Molecule AI · app-lead) app-qa (Molecule AI · app-qa) claude-ceo-assistant claude-ci-reader claude-status-reaper core-be (Molecule AI · core-be) core-devops (Molecule AI · core-devops) core-fe (Molecule AI · core-fe) core-lead (Molecule AI · core-lead) core-offsec (Molecule AI · core-offsec) core-qa (Molecule AI · core-qa) core-security (Molecule AI · core-security) core-uiux (Molecule AI · core-uiux) cp-be (Molecule AI · cp-be) cp-lead (Molecule AI · cp-lead) cp-qa (Molecule AI · cp-qa) cp-security (Molecule AI · cp-security) cui (Zhanlin Cui) dev-lead (Molecule AI · dev-lead) devops-engineer documentation-specialist (Molecule AI · documentation-specialist) fullstack-engineer (Molecule AI · fullstack-engineer) hongming hongming-ceo-delegated hongming-codex-laptop hongming-kimi-laptop hongming-pc2 infra-lead (Molecule AI · infra-lead) infra-runtime-be (Molecule AI · infra-runtime-be) infra-sre (Molecule AI · infra-sre) integration-tester (Molecule AI · integration-tester) mc-drift-bot molecule-code-reviewer plugin-dev (Molecule AI · plugin-dev) pm publish-runtime-bot pypi-publisher (Molecule AI PyPI Publisher (RFC#596)) release-manager (Molecule AI · release-manager) sdk-dev (Molecule AI · sdk-dev) sdk-lead (Molecule AI · sdk-lead) sop-drift-bot sop-tier-bot (SOP Tier-Check Bot) technical-writer (Molecule AI · technical-writer) triage-operator (Molecule AI · triage-operator)

No Assignees

3 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: molecule-ai/molecule-core#2423