molecule-ai/molecule-core
52
0
Fork 2
Code Issues 175 Pull Requests 17 Actions 6 Packages Projects Releases Wiki Activity

SOP gate: auto-assign tier label + auto-fire qa/security after 2-genuine code-review (last autonomy gap to self-shipping) #2396

New Issue
Open
opened 2026-06-07 09:55:33 +00:00 by devops-engineer · 0 comments
devops-engineer commented 2026-06-07 09:55:33 +00:00
Member
Copy Link
Copy Source

Goal: make the full SOP gate fire+complete autonomously after 2-genuine code-review, so the fleet self-ships with ZERO manual/PM coordination. This is the last gap to fully-autonomous self-shipping (the codex reviewer-lane is fixed + working; PRs now stall at the SOP gate).

Observed (2026-06-07 ~08:00–09:53Z)

8 core PRs (#2384/2387/2388/2389/2393/2394/2259) reached 2-genuine code-review (CR2+Researcher) + base CI green (CI/all-required + E2E API Smoke + Handlers PG all success) but did NOT merge. Root cause: full SOP gate (RFC#324/#351) requires steps that DON'T auto-fire after code-review:

  1. tier label (tier:low/medium/high) — currently MANUAL → sop-tier-check = failure on empty labels → blocks.
  2. qa-review / security-review — don't auto-trigger after code-review (the known auto-trigger gap; needs manual /qa-recheck).
    Result: code-reviewed PRs pile up unmerged; PM must hand-drive the ceremony (wave-1 dispatch of Researcher tier-rec + CR2 security + Kimi ack).

Fix (buildable spec)

  1. Auto-tier: a gate step that assigns a tier label from the diff (size/paths/risk heuristics, e.g. touches security/auth/migrations→high; tests/docs→low) when a PR reaches 2-genuine code-review and has no tier label. Reviewer can override.
  2. Auto-fire qa/security: on 2-genuine code-review + tier assigned, auto-trigger the qa-review + security-review jobs (fix the auto-trigger gap; no manual /qa-recheck).
  3. Ensure the responsible actor has label-write (write:issue) so labeling never silently no-ops.
  4. §SOP-22 gating test: a CI test that a PR reaching 2-genuine + base-CI-green auto-gets a tier label + qa/security auto-fire (fail-closed: revert → PR stalls ungated). Mutation-verify.

Why

"Fleet ships its own work" is only true once the SOP ceremony is autonomous. Today code-review is autonomous but the gate ceremony isn't → self-shipping requires PM/CTO coordination. Related: reviewer-lane fix (template-codex PR#87, internal#653/#781); the full-SOP-gate enforcement (RFC#324/#351).

**Goal:** make the full SOP gate fire+complete autonomously after 2-genuine code-review, so the fleet self-ships with ZERO manual/PM coordination. This is the last gap to fully-autonomous self-shipping (the codex reviewer-lane is fixed + working; PRs now stall at the SOP gate). ## Observed (2026-06-07 ~08:00–09:53Z) 8 core PRs (#2384/2387/2388/2389/2393/2394/2259) reached 2-genuine code-review (CR2+Researcher) + base CI green (CI/all-required + E2E API Smoke + Handlers PG all success) but did NOT merge. Root cause: full SOP gate (RFC#324/#351) requires steps that DON'T auto-fire after code-review: 1. **tier label** (tier:low/medium/high) — currently MANUAL → `sop-tier-check` = failure on empty labels → blocks. 2. **qa-review / security-review** — don't auto-trigger after code-review (the known auto-trigger gap; needs manual /qa-recheck). Result: code-reviewed PRs pile up unmerged; PM must hand-drive the ceremony (wave-1 dispatch of Researcher tier-rec + CR2 security + Kimi ack). ## Fix (buildable spec) 1. **Auto-tier:** a gate step that assigns a tier label from the diff (size/paths/risk heuristics, e.g. touches security/auth/migrations→high; tests/docs→low) when a PR reaches 2-genuine code-review and has no tier label. Reviewer can override. 2. **Auto-fire qa/security:** on 2-genuine code-review + tier assigned, auto-trigger the qa-review + security-review jobs (fix the auto-trigger gap; no manual /qa-recheck). 3. **Ensure the responsible actor has label-write** (write:issue) so labeling never silently no-ops. 4. **§SOP-22 gating test:** a CI test that a PR reaching 2-genuine + base-CI-green auto-gets a tier label + qa/security auto-fire (fail-closed: revert → PR stalls ungated). Mutation-verify. ## Why "Fleet ships its own work" is only true once the SOP ceremony is autonomous. Today code-review is autonomous but the gate ceremony isn't → self-shipping requires PM/CTO coordination. Related: reviewer-lane fix (template-codex PR#87, internal#653/#781); the full-SOP-gate enforcement (RFC#324/#351).
agent-dev-a referenced this issue from a commit 2026-06-07 15:46:43 +00:00
fix(auto-tier): security paths = +2 high, add incomplete-ceremony + heuristic tests (#2396 RC2)\n\nCR2 gate-review fixes:\n\n1. Security/auth/crypto/migrations paths now award +2 high indicators\n (was +1). A single security-related path is sufficient for tier:high\n without requiring a second indicator. Prevents under-classification of\n auth/crypto changes as medium/low.\n\n2. LOW heuristic fixed: 'only tests/docs' check now correctly awards +1\n when ALL paths are tests/docs (was inverted — awarded when non-test/doc\n paths existed).\n\n3. Gate/CI governance surface expanded:\n - Any .gitea/scripts/* → +1 high\n - Any .gitea/workflows/* → +1 high\n - Expanded patterns: auto-tier, status-reaper, sop-checklist,\n prod-auto-deploy, ci-required-drift, lint_*, etc.\n - SOP/config patterns: sop-checklist-config, runbooks/sop-*, docs/design/rfc-*\n\n4. FAIL-CLOSED TEST — incomplete ceremony after auto-tier (Scenario S4):\n Added to test_sop_tier_check_authz.sh:\n - PR has tier:medium label (auto-tier assigned)\n - Engineer + manager APPROVED\n - qa + security NOT approved (404)\n - EXPECT: sop-tier-check exits 1 with 'FAILED for tier:medium'\n - Proves auto-tier does NOT weaken the gate — incomplete ceremony\n still blocks merge.\n\n5. HEURISTIC TESTS — test_auto_tier_check_heuristics.sh (13 scenarios):\n H1: gate-file-only (.gitea/workflows + .gitea/scripts) → tier:high\n H2: security/auth path (workspace-server/internal/crypto) → tier:high\n H3: tests/docs only, small diff → tier:low\n H4: already labeled → skip (idempotent)\n H5: label assignment 403 → fail-closed (exit 1)\n\nPlatform Go:\n - test_auto_tier_check.py: 9 pass\n - test_gate_review_auto_fire.py: 11 pass\n - test_auto_tier_check_heuristics.sh: 13 pass\n - test_sop_tier_check_authz.sh: 15 pass (incl. S4 incomplete-ceremony)\n Total: 48/48 pass.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
area/ci
CI/CD pipeline issues
 do-not-auto-merge
Opt out of autonomous merge-queue merging
 do-not-merge
hold from auto-merge (design review in progress)
 kind/infrastructure
Infrastructure-related issues
 merge-queue
Ready for serialized Gitea merge queue
 merge-queue-hold
Temporarily hold PR in merge queue
 platform/go
Go platform test issues
 release-blocker
Blocks the staging→main promotion / a release
 release-test
security
test-label-sre
tier:high
High risk per dev-sop §SOP-6 — ceo only, 24h cooldown
 tier:low
Low risk per dev-sop §SOP-6 — engineers/managers/ceo can approve
 tier:medium
Medium risk per dev-sop §SOP-6 — managers/ceo can approve
 triage-test
test
 wip
Work in progress — do not auto-merge
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
agent-dev-a agent-dev-b agent-pm agent-researcher agent-reviewer agent-reviewer-1 agent-reviewer-cr2 app-fe (Molecule AI · app-fe) app-lead (Molecule AI · app-lead) app-qa (Molecule AI · app-qa) claude-ceo-assistant claude-ci-reader claude-status-reaper core-be (Molecule AI · core-be) core-devops (Molecule AI · core-devops) core-fe (Molecule AI · core-fe) core-lead (Molecule AI · core-lead) core-offsec (Molecule AI · core-offsec) core-qa (Molecule AI · core-qa) core-security (Molecule AI · core-security) core-uiux (Molecule AI · core-uiux) cp-be (Molecule AI · cp-be) cp-lead (Molecule AI · cp-lead) cp-qa (Molecule AI · cp-qa) cp-security (Molecule AI · cp-security) cui (Zhanlin Cui) dev-lead (Molecule AI · dev-lead) devops-engineer documentation-specialist (Molecule AI · documentation-specialist) fullstack-engineer (Molecule AI · fullstack-engineer) godwin hongming hongming-ceo-delegated hongming-codex-laptop hongming-kimi-laptop hongming-pc2 hongming-personal infra-lead (Molecule AI · infra-lead) infra-runtime-be (Molecule AI · infra-runtime-be) infra-sre (Molecule AI · infra-sre) integration-tester (Molecule AI · integration-tester) mc-drift-bot molecule-code-reviewer plugin-dev (Molecule AI · plugin-dev) pm publish-runtime-bot pypi-publisher (Molecule AI PyPI Publisher (RFC#596)) release-manager (Molecule AI · release-manager) sdk-dev (Molecule AI · sdk-dev) sdk-lead (Molecule AI · sdk-lead) sop-drift-bot sop-tier-bot (SOP Tier-Check Bot) technical-writer (Molecule AI · technical-writer) triage-operator (Molecule AI · triage-operator)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: molecule-ai/molecule-core#2396
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?