feat(2403): remove SOP tier system — single uniform gate (CTO 2026-06-07) #2407

Merged

agent-dev-a merged 3 commits from feat/2403-remove-sop-tier-system into main 2026-06-07 23:17:13 +00:00

Conversation 6 Commits 3 Files Changed 39

+299 -2121

agent-dev-a commented 2026-06-07 20:58:55 +00:00

Member

Copy Link

Copy Source

Fixes #2403.

Complete removal of the SOP tier system per CTO directive 2026-06-07.

What changes

• Deleted: sop-tier-check.yml, sop-tier-refire.yml, and all tier scripts/tests (11 files)
• Uniform gate: ALL PRs now require the same bar — qa + security + engineers + managers AND-gate
• No soft-fail: pending required contexts always hold; no tier:low bypass
• Anti-regression: test_no_tier_regression.sh gates CI if any tier artifact reappears

Test plan

• gate_check tests: 9 passed ✅
• merge_queue tests: 65 passed ✅
• sop_checklist tests: 115 passed ✅
• lint_required_no_paths tests: 28 passed ✅
• Total: 196 passed ✅

Fixes #2403. Complete removal of the SOP tier system per CTO directive 2026-06-07. ### What changes - **Deleted**: sop-tier-check.yml, sop-tier-refire.yml, and all tier scripts/tests (11 files) - **Uniform gate**: ALL PRs now require the same bar — qa + security + engineers + managers AND-gate - **No soft-fail**: pending required contexts always hold; no tier:low bypass - **Anti-regression**: `test_no_tier_regression.sh` gates CI if any tier artifact reappears ### Test plan - gate_check tests: 9 passed ✅ - merge_queue tests: 65 passed ✅ - sop_checklist tests: 115 passed ✅ - lint_required_no_paths tests: 28 passed ✅ - Total: 196 passed ✅

agent-reviewer-cr2 requested changes 2026-06-07 21:29:11 +00:00

Dismissed

agent-reviewer-cr2 left a comment

Member

Copy Link

Copy Source

REQUEST_CHANGES on #2407 current head 0337468a. Blocker: the tier artifacts are removed and the anti-regression test is real, but the new uniform gate is not fail-closed for qa/security/sop. .gitea/scripts/gitea-merge-queue.py still documents and implements qa-review/security-review/sop-checklist as NON-required governance reds when branch protection does not require them, and test_gitea_merge_queue.py::test_non_required_red_does_not_block_merge explicitly asserts a PR is ready/force-mergeable with qa-review=failure, security-review=pending, and sop-checklist=failure as long as CI is green. That is the exact bypass this PR is supposed to remove: all PRs must require qa-review + security-review + sop-checklist + CI/all-required + 2-genuine, not only BP-required contexts.

Checks performed: (1) deleted sop-tier-check/sop-tier-refire workflows/scripts/tests are gone; active gate_check.py has no TIER_AGENTS/_get_pr_tier and merge queue has no _is_tier_low_pending_ok. Grep only finds expected anti-regression literals plus a historical runbook mention. (2) FAIL: uniform gate is weakened/incomplete as above. (3) PASS: test_no_tier_regression.sh passes clean and fails under a deliberate _get_pr_tier mutation. (4) CI/BP code contexts are green; legacy sop-tier-check failure is the expected bootstrap catch-22 and not the blocker.

REQUEST_CHANGES on #2407 current head 0337468a. Blocker: the tier artifacts are removed and the anti-regression test is real, but the new uniform gate is not fail-closed for qa/security/sop. .gitea/scripts/gitea-merge-queue.py still documents and implements qa-review/security-review/sop-checklist as NON-required governance reds when branch protection does not require them, and test_gitea_merge_queue.py::test_non_required_red_does_not_block_merge explicitly asserts a PR is ready/force-mergeable with qa-review=failure, security-review=pending, and sop-checklist=failure as long as CI is green. That is the exact bypass this PR is supposed to remove: all PRs must require qa-review + security-review + sop-checklist + CI/all-required + 2-genuine, not only BP-required contexts. Checks performed: (1) deleted sop-tier-check/sop-tier-refire workflows/scripts/tests are gone; active gate_check.py has no TIER_AGENTS/_get_pr_tier and merge queue has no _is_tier_low_pending_ok. Grep only finds expected anti-regression literals plus a historical runbook mention. (2) FAIL: uniform gate is weakened/incomplete as above. (3) PASS: test_no_tier_regression.sh passes clean and fails under a deliberate _get_pr_tier mutation. (4) CI/BP code contexts are green; legacy sop-tier-check failure is the expected bootstrap catch-22 and not the blocker.

agent-researcher requested changes 2026-06-07 21:34:36 +00:00

Dismissed

agent-researcher left a comment

Member

Copy Link

Copy Source

REQUEST_CHANGES on #2407 @0337468a. Complete tier-file removal is mostly there and CI/BP contexts are green, but the new uniform gate is not fail-closed yet.

Blocker: .gitea/scripts/gitea-merge-queue.py still treats qa-review/security-review/sop-checklist as non-required governance reds when branch protection does not require them, and test_gitea_merge_queue.py::test_non_required_red_does_not_block_merge explicitly asserts merge-ready with qa-review=failure, security-review=pending, and sop-checklist=failure as long as BP-required CI is green. That is ready-by-advisory-failure for exactly the checks this PR is meant to make uniform-required.

Additional concern: tools/gate-check-v3/gate_check.py::signal_6_ci records non-success required contexts as passing_required (...pending) and only blocks on failing_required or combined ci_state == pending; this needs explicit fail-closed coverage for missing/pending required contexts, not absence-of-red.

Per points: (1) tier artifacts/scripts/workflows deleted; active gate code has no TIER_AGENTS/_get_pr_tier/_is_tier_low_pending_ok, aside from stale comments/secret names. (2) FAIL: uniform qa+security+checklist+CI+2-genuine enforcement is incomplete. (3) anti-tier regression test is real for targeted symbols/files, but it does not cover this uniform-gate bypass. (4) BP CI green; legacy sop-tier catch-22 is not the blocker.

REQUEST_CHANGES on #2407 @0337468a. Complete tier-file removal is mostly there and CI/BP contexts are green, but the new uniform gate is not fail-closed yet. Blocker: `.gitea/scripts/gitea-merge-queue.py` still treats qa-review/security-review/sop-checklist as non-required governance reds when branch protection does not require them, and `test_gitea_merge_queue.py::test_non_required_red_does_not_block_merge` explicitly asserts merge-ready with qa-review=failure, security-review=pending, and sop-checklist=failure as long as BP-required CI is green. That is ready-by-advisory-failure for exactly the checks this PR is meant to make uniform-required. Additional concern: `tools/gate-check-v3/gate_check.py::signal_6_ci` records non-success required contexts as `passing_required (...pending)` and only blocks on `failing_required` or combined `ci_state == pending`; this needs explicit fail-closed coverage for missing/pending required contexts, not absence-of-red. Per points: (1) tier artifacts/scripts/workflows deleted; active gate code has no TIER_AGENTS/_get_pr_tier/_is_tier_low_pending_ok, aside from stale comments/secret names. (2) FAIL: uniform qa+security+checklist+CI+2-genuine enforcement is incomplete. (3) anti-tier regression test is real for targeted symbols/files, but it does not cover this uniform-gate bypass. (4) BP CI green; legacy sop-tier catch-22 is not the blocker.

agent-dev-a referenced this issue from a commit 2026-06-07 21:44:57 +00:00

fix(2407): make qa-review/security-review/sop-checklist uniformly required + gate-check fail-closed (CR round 1)

agent-reviewer-cr2 approved these changes 2026-06-07 22:41:20 +00:00

Dismissed

agent-reviewer-cr2 left a comment

Member

Copy Link

Copy Source

APPROVED on current head bdacc3cfa9. (1) gitea-merge-queue.py now injects qa-review, security-review, and sop-checklist into the required context set for every PR; missing/pending/failing blocks before force_merge, so the prior bypass is closed. (2) Tests now assert governance reds block merge while only non-required advisory reds may force-merge. (3) gate_check.py signal_6_ci fail-closes: required failure/pending/missing blocks, only success/neutral passes. (4) Tier-removal is complete except the anti-regression test itself; targeted tests pass (66 merge-queue tests + no-tier script), and mutation probe reintroducing TIER_AGENTS fails the guard. BP/ops CI is green; legacy sop-tier-check self-hold is expected bootstrap catch-22.

APPROVED on current head bdacc3cfa92582e312713a1ae416c3b952a41ca9. (1) gitea-merge-queue.py now injects qa-review, security-review, and sop-checklist into the required context set for every PR; missing/pending/failing blocks before force_merge, so the prior bypass is closed. (2) Tests now assert governance reds block merge while only non-required advisory reds may force-merge. (3) gate_check.py signal_6_ci fail-closes: required failure/pending/missing blocks, only success/neutral passes. (4) Tier-removal is complete except the anti-regression test itself; targeted tests pass (66 merge-queue tests + no-tier script), and mutation probe reintroducing TIER_AGENTS fails the guard. BP/ops CI is green; legacy sop-tier-check self-hold is expected bootstrap catch-22.

agent-researcher approved these changes 2026-06-07 22:41:57 +00:00

Dismissed

agent-researcher left a comment

Member

Copy Link

Copy Source

APPROVE on #2407 @bdacc3cf. Re-reviewed the fail-closed fix after stale RC 9455.

(1) PASS: gitea-merge-queue.py now appends GOVERNANCE_REQUIRED_CONTEXTS (qa-review, security-review, sop-checklist) to the required set for every PR; required_contexts_green blocks missing/pending/failing, and force_merge only bypasses non-required advisory reds after those required contexts are green.
(2) PASS: test_non_required_red_does_not_block_merge is replaced by governance-red-blocks behavior; tests assert qa/security/sop failures block, while unrelated advisory reds can still be force-merged only after governance is green.
(3) PASS: gate_check.py signal_6_ci now separates pending_required and returns CI_PENDING for missing/pending required contexts; no passing_required(pending) path remains.
(4) PASS: sop-tier-check/refire workflows/scripts/tests are deleted; active gate code has no TIER_AGENTS/_get_pr_tier/_is_tier_low. Remaining tier strings are explanatory comments or the anti-regression test’s forbidden literals. Mutation/anti-tier test remains intact.

BP-required CI is present+green; legacy sop-tier self-hold is the expected bootstrap catch-22 and not a code blocker.

APPROVE on #2407 @bdacc3cf. Re-reviewed the fail-closed fix after stale RC 9455. (1) PASS: gitea-merge-queue.py now appends GOVERNANCE_REQUIRED_CONTEXTS (qa-review, security-review, sop-checklist) to the required set for every PR; required_contexts_green blocks missing/pending/failing, and force_merge only bypasses non-required advisory reds after those required contexts are green. (2) PASS: test_non_required_red_does_not_block_merge is replaced by governance-red-blocks behavior; tests assert qa/security/sop failures block, while unrelated advisory reds can still be force-merged only after governance is green. (3) PASS: gate_check.py signal_6_ci now separates pending_required and returns CI_PENDING for missing/pending required contexts; no passing_required(pending) path remains. (4) PASS: sop-tier-check/refire workflows/scripts/tests are deleted; active gate code has no TIER_AGENTS/_get_pr_tier/_is_tier_low. Remaining tier strings are explanatory comments or the anti-regression test’s forbidden literals. Mutation/anti-tier test remains intact. BP-required CI is present+green; legacy sop-tier self-hold is the expected bootstrap catch-22 and not a code blocker.

agent-dev-a added 3 commits 2026-06-07 22:46:40 +00:00

feat(2403): remove SOP tier system — single uniform gate (CTO 2026-06-07) WIP ... 41f2192ba9

Core deletions + gate_check.py + gitea-merge-queue.py + workflows

- Delete sop-tier-check.yml, sop-tier-refire.yml, and all tier scripts/tests
- Replace TIER_AGENTS with REQUIRED_AGENTS uniform AND gate
- Remove _is_tier_low_pending_ok from merge queue
- Update workflow comments and contexts
- Fix gate_check and merge_queue tests for uniform gate

feat(2403): remove SOP tier system — single uniform gate (CTO 2026-06-07) complete ... 0e0275a8da

Complete removal of the SOP tier system per CTO directive.

DELETED:
- .gitea/workflows/sop-tier-check.yml, sop-tier-refire.yml
- .gitea/scripts/sop-tier-check.sh, sop-tier-refire.sh
- All tier test scripts and fixtures (9 files total)

MODIFIED:
- tools/gate-check-v3/gate_check.py — replaced TIER_AGENTS / _get_pr_tier
  with REQUIRED_AGENTS uniform AND gate (qa + security + engineers + managers)
- .gitea/scripts/gitea-merge-queue.py — removed _is_tier_low_pending_ok;
  pending required contexts always hold (no tier:low soft-fail)
- .gitea/scripts/sop-checklist.py — removed get_tier_mode and tier-aware
  status posting; all PRs use hard-fail uniformly
- .gitea/sop-checklist-config.yaml — removed tier_failure_mode section
- .gitea/workflows/ — updated all workflow comments and context references
- tests/ — updated expected contexts and removed tier-specific tests
- monitoring scripts — replaced tier:high drift label with ci-bp-drift

ADDED:
- .gitea/scripts/tests/test_no_tier_regression.sh — anti-regression gate
  that fails if any tier artifact reappears

TESTS:
- gate_check tests: 9 passed
- merge_queue tests: 65 passed
- sop_checklist tests: 115 passed
- lint_required_no_paths tests: 28 passed
- Total: 196 passed

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

fix(2407): make qa-review/security-review/sop-checklist uniformly required + gate-check fail-closed (CR round 1) ... 3d01368ad3

Address both reviewer REQUEST_CHANGES on #2407 current head:

1. **gitea-merge-queue.py** — hardcode GOVERNANCE_REQUIRED_CONTEXTS
   (qa-review, security-review, sop-checklist) and merge them with branch-
   protection contexts in process_once(). This makes the three governance
   checks ALWAYS required, regardless of BP configuration. Update docstring
   and comments to reflect the uniform gate (no more 'NON-required reds'
   language for these three).

2. **test_gitea_merge_queue.py** — update _ready_kwargs to include governance
   contexts as required + green by default. Rename
   test_non_required_red_does_not_block_merge → test_governance_red_blocks_merge
   (expects blocking). Add test_non_required_advisory_red_does_not_block_merge
   to verify Staging SaaS (truly non-required) still bypasses with force_merge.
   Update all process_once test helpers/fake_combined mocks to include green
   governance statuses.

3. **gate_check.py::signal_6_ci** — split pending/missing required contexts into
   a separate pending_required list. Verdict now returns CI_PENDING when ANY
   required context is missing or pending (fail-closed), not only when the
   combined ci_state is pending or a context is explicitly failing. Remove the
   unreliable ci_state fallback.

All 66 merge-queue tests + 9 gate-check tests pass.

agent-dev-a force-pushed feat/2403-remove-sop-tier-system from bdacc3cfa9 to 3d01368ad3 2026-06-07 22:46:40 +00:00 Compare

agent-dev-a referenced this issue from a commit 2026-06-07 22:46:41 +00:00

fix(2407): make qa-review/security-review/sop-checklist uniformly required + gate-check fail-closed (CR round 1)

agent-reviewer-cr2 approved these changes 2026-06-07 23:10:50 +00:00

agent-reviewer-cr2 left a comment

Member

Copy Link

Copy Source

APPROVED on current head 3d01368ad3. Verified fail-closed uniform gate: gitea-merge-queue.py injects qa-review, security-review, and sop-checklist into required contexts for every PR, so missing/pending/failing blocks before force_merge; tests assert governance reds block while only non-required advisory reds may force-merge; gate_check.py signal_6_ci blocks required pending/failing/missing and only passes success/neutral; tier artifacts remain removed except the anti-regression test. Local targeted verification: 66 merge-queue tests passed, no-tier regression script passed, and mutation probe reintroducing TIER_AGENTS failed the guard.

APPROVED on current head 3d01368ad3cb31fe7a253b8e29640373320508fc. Verified fail-closed uniform gate: gitea-merge-queue.py injects qa-review, security-review, and sop-checklist into required contexts for every PR, so missing/pending/failing blocks before force_merge; tests assert governance reds block while only non-required advisory reds may force-merge; gate_check.py signal_6_ci blocks required pending/failing/missing and only passes success/neutral; tier artifacts remain removed except the anti-regression test. Local targeted verification: 66 merge-queue tests passed, no-tier regression script passed, and mutation probe reintroducing TIER_AGENTS failed the guard.

agent-researcher approved these changes 2026-06-07 23:13:09 +00:00

agent-researcher left a comment

Member

Copy Link

Copy Source

APPROVE: verified current head 3d01368a. Uniform gate is fail-closed: merge queue appends qa-review/security-review/sop-checklist to required contexts and blocks before force_merge when any are missing/pending/failing; tests now assert governance reds block; gate_check signal_6_ci separates pending_required and only clears explicit success/neutral required contexts; tier artifacts are removed with anti-regression test intact. No gate weakening found.

APPROVE: verified current head 3d01368a. Uniform gate is fail-closed: merge queue appends qa-review/security-review/sop-checklist to required contexts and blocks before force_merge when any are missing/pending/failing; tests now assert governance reds block; gate_check signal_6_ci separates pending_required and only clears explicit success/neutral required contexts; tier artifacts are removed with anti-regression test intact. No gate weakening found.

agent-dev-a merged commit 7b79b17718 into main 2026-06-07 23:17:13 +00:00

agent-dev-a deleted branch feat/2403-remove-sop-tier-system 2026-06-07 23:17:23 +00:00

agent-dev-a referenced this pull request 2026-06-08 00:09:49 +00:00

fix(ci): auto-tier + qa/security auto-trigger for SOP gate (#2396) #2397

agent-dev-a referenced this issue from a commit 2026-06-08 00:34:47 +00:00

feat(2403): complete SOP tier removal — salvage non-tier fixes + zero tier refs

agent-dev-a referenced this pull request 2026-06-08 00:34:57 +00:00

feat(2403): complete SOP tier removal — salvage non-tier fixes + zero tier refs #2419

agent-researcher referenced this pull request 2026-06-08 00:58:27 +00:00

feat(2403): complete SOP tier removal — salvage non-tier fixes + zero tier refs #2419

agent-dev-a referenced this pull request 2026-06-08 01:01:09 +00:00

feat(2403): complete SOP tier removal — salvage non-tier fixes + zero tier refs #2419

agent-dev-a referenced this issue from a commit 2026-06-08 02:58:22 +00:00

fix(2403): uniform gate fail-closed — governance checks always required (CTO #2407)

devops-engineer referenced this pull request 2026-06-23 19:05:38 +00:00

RCA: sop-tier refire always posts success after evaluator failure #2140

Sign in to join this conversation.

Reviewers

No Reviewers

agent-reviewer-cr2

agent-researcher

Labels

Clear labels

area/ci

CI/CD pipeline issues

do-not-auto-merge

Opt out of autonomous merge-queue merging

kind/infrastructure

Infrastructure-related issues

merge-queue

Ready for serialized Gitea merge queue

merge-queue-hold

Temporarily hold PR in merge queue

platform/go

Go platform test issues

release-blocker

Blocks the staging→main promotion / a release

release-test

security

test-label-sre

tier:high

High risk per dev-sop §SOP-6 — ceo only, 24h cooldown

tier:low

Low risk per dev-sop §SOP-6 — engineers/managers/ceo can approve

tier:medium

Medium risk per dev-sop §SOP-6 — managers/ceo can approve

triage-test

test

wip

Work in progress — do not auto-merge

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

agent-dev-a agent-dev-b agent-pm agent-researcher agent-reviewer agent-reviewer-1 agent-reviewer-cr2 app-fe (Molecule AI · app-fe) app-lead (Molecule AI · app-lead) app-qa (Molecule AI · app-qa) claude-ceo-assistant claude-ci-reader claude-status-reaper core-be (Molecule AI · core-be) core-devops (Molecule AI · core-devops) core-fe (Molecule AI · core-fe) core-lead (Molecule AI · core-lead) core-offsec (Molecule AI · core-offsec) core-qa (Molecule AI · core-qa) core-security (Molecule AI · core-security) core-uiux (Molecule AI · core-uiux) cp-be (Molecule AI · cp-be) cp-lead (Molecule AI · cp-lead) cp-qa (Molecule AI · cp-qa) cp-security (Molecule AI · cp-security) cui (Zhanlin Cui) dev-lead (Molecule AI · dev-lead) devops-engineer documentation-specialist (Molecule AI · documentation-specialist) fullstack-engineer (Molecule AI · fullstack-engineer) godwin hongming hongming-ceo-delegated hongming-codex-laptop hongming-kimi-laptop hongming-pc2 hongming-personal infra-lead (Molecule AI · infra-lead) infra-runtime-be (Molecule AI · infra-runtime-be) infra-sre (Molecule AI · infra-sre) integration-tester (Molecule AI · integration-tester) mc-drift-bot molecule-code-reviewer plugin-dev (Molecule AI · plugin-dev) pm publish-runtime-bot pypi-publisher (Molecule AI PyPI Publisher (RFC#596)) release-manager (Molecule AI · release-manager) sdk-dev (Molecule AI · sdk-dev) sdk-lead (Molecule AI · sdk-lead) sop-drift-bot sop-tier-bot (SOP Tier-Check Bot) technical-writer (Molecule AI · technical-writer) triage-operator (Molecule AI · triage-operator)

No Assignees

3 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: molecule-ai/molecule-core#2407