Commit Graph

6 Commits

Author SHA1 Message Date
0970feef70 feat(ci)(hard-gate): lint-pre-flip catches continue-on-error true→false without run-log proof
Some checks failed
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 23s
CI / Detect changes (pull_request) Successful in 56s
Lint curl status-code capture / Scan workflows for curl status-capture pollution (pull_request) Successful in 15s
E2E API Smoke Test / detect-changes (pull_request) Successful in 42s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 44s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 46s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 15s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 35s
gate-check-v3 / gate-check (pull_request) Failing after 22s
lint-continue-on-error-tracking / lint-continue-on-error-tracking (pull_request) Failing after 1m17s
qa-review / approved (pull_request) Failing after 19s
sop-checklist / all-items-acked (pull_request) acked: 0/7 — missing: comprehensive-testing, local-postgres-e2e, staging-smoke, +4 — body-unfilled: 7
security-review / approved (pull_request) Failing after 21s
sop-checklist-gate / gate (pull_request) Successful in 19s
lint-required-no-paths / lint-required-no-paths (pull_request) Successful in 1m16s
sop-tier-check / tier-check (pull_request) Successful in 25s
Lint workflow YAML (Gitea-1.22.6-hostile shapes) / Lint workflow YAML for Gitea-1.22.6-hostile shapes (pull_request) Successful in 1m32s
Lint pre-flip continue-on-error / Verify continue-on-error flips have run-log proof (pull_request) Successful in 1m52s
CI / Platform (Go) (pull_request) Successful in 11s
CI / Canvas (Next.js) (pull_request) Successful in 8s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 5s
CI / Python Lint & Test (pull_request) Successful in 9s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 10s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 9s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 11s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 9s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
CI / all-required (pull_request) Successful in 9s
Empirical class — PR #656 / mc#664:
PR #656 (RFC internal#219 Phase 4) flipped 5 platform-build-class jobs
`continue-on-error: true → false` on the basis of a "verified green
on main via combined-status check". But that "green" was the LIE
the prior `continue-on-error: true` produced: Gitea Quirk #10
(internal#342 + dup #287) — a failed step inside a CoE:true job rolls
up to a success job-level status. The precondition the PR claimed to
verify was structurally fooled by the bug being flipped.

mc#664 captured the surfaced defects (2 mutually-masked regressions):
- Class 1: sqlmock helper drift since 2f36bb9a (24 days old)
- Class 2: OFFSEC-001 contract collision since 7d1a189f (1 day old)

Codified 04:35Z as hongming-pc2 charter §SOP-N rule (e)
"run-log-grep-before-flip": pull the actual run log + grep for
--- FAIL / FAIL\s BEFORE flipping; don't trust the masked
combined-status. This commit structurally enforces that rule.

What this PR adds:

.gitea/workflows/lint-pre-flip-continue-on-error.yml — pre-merge
  pull_request gate, path-scoped to .gitea/workflows/**. Lands at
  continue-on-error:true (Phase 3 dogfood — flip to false in a
  follow-up only after this workflow has clean recent runs on main).

.gitea/scripts/lint_pre_flip_continue_on_error.py — the lint:
  1. Reads every .gitea/workflows/*.yml at the PR base SHA AND head
     SHA via git show <sha>:<path>. No checkout needed.
  2. Parses both sides via PyYAML AST (per
     feedback_behavior_based_ast_gates — NOT grep, so comment churn
     and key-order changes don't false-positive).
  3. For each flipped job (base=true, head=false), renders the
     commit-status context as "{workflow.name} / {job.name or job.key}
     (push)" and pulls combined commit-status for the last 5
     commits on the PR base branch.
  4. Fetches each matching run's log via the web-UI route
     {server_url}/{repo}/actions/runs/{run_id}/jobs/{job_idx}/logs
     (per reference_gitea_actions_log_fetch — Gitea 1.22.6 lacks
     REST /actions/runs/*; web-UI is the only working path, see
     reference_gitea_1_22_6_lacks_rest_rerun_endpoints).
  5. Greps for --- FAIL / FAIL\s / ::error::. If status==success
     AND log shows fail markers, the job was masked. Emit
     ::error::file=... naming the failing test + offending run URL.

.gitea/scripts/tests/test_lint_pre_flip_continue_on_error.py —
  35 unittest cases covering the 5 acceptance tests from the spec
  + CoE coercion (truthy/falsy/quoted/absent) + context-name
  rendering + multi-flip aggregation + dry-run semantics + 3
  graceful-degrade halt conditions (log-unavailable, zero-runs-
  history, zero-commits-on-branch).

Live empirical confirmation:
Ran the script against the PR#656 base→merge diff with
RECENT_COMMITS_N=3 on main. Result:
- platform-build flip BLOCKED — masked --- FAIL on
  TestExecuteDelegation_DeliveryConfirmedProxyError_TreatsAsSuccess
  + 4 more on action_run 13353.
- canvas-build / shellcheck / python-lint flips PASS — no FAIL
  markers in their recent logs.
Exactly the diagnosis hongming-pc2 charter §SOP-N rule (e) requires.

Halt-condition graceful-degrade contract:
- Log fetch 404 (act_runner pruned, transient outage): warn-not-block.
- Zero recent runs of the flipped context (newly-added workflow):
  chicken-and-egg exemption — warn and allow.
- YAML parse error in one workflow file: warn-not-block (the YAML
  lint workflows catch this separately).

Cross-links: PR#656, mc#664, PR#665 (interim re-mask), Quirk #10
(internal#342 + dup #287), hongming-pc2 charter §SOP-N rule (e),
feedback_strict_root_only_after_class_a,
feedback_no_shared_persona_token_use.

Refs: internal#342, internal#287, molecule-core#664, molecule-core#665
2026-05-12 07:27:19 +00:00
72df12ecef feat(ci): sop-checklist-gate — peer-ack merge gate (RFC#351 Phase 2)
All checks were successful
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 16s
Lint curl status-code capture / Scan workflows for curl status-capture pollution (pull_request) Successful in 15s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 33s
CI / Detect changes (pull_request) Successful in 43s
E2E API Smoke Test / detect-changes (pull_request) Successful in 44s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 40s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 13s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 11s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 7s
CI / Platform (Go) (pull_request) Successful in 8s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 7s
CI / Canvas (Next.js) (pull_request) Successful in 9s
CI / Python Lint & Test (pull_request) Successful in 15s
lint-required-no-paths / lint-required-no-paths (pull_request) Successful in 1m25s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
CI / all-required (pull_request) Successful in 6s
RFC#351 Step 2 of 6: implementation MVP of the SOP-checklist peer-ack
merge gate. NOT yet wired to branch protection (Phase 4 needs separate
authorization).

What:
- .gitea/sop-checklist-config.yaml — 7-item checklist with slug,
  numeric_alias (1..7), pr_section_marker, required_teams. Includes
  tier-aware failure-mode map: tier:high/medium=hard, tier:low=soft,
  default=hard (never silently lower the bar).
- .gitea/scripts/sop-checklist-gate.py — parses PR body + comments,
  computes per-item ack state, posts commit-status
  "sop-checklist / all-items-acked (pull_request)".
- .gitea/scripts/tests/test_sop_checklist_gate.py — 51 unit tests
  covering slug normalization, directive parsing, section-marker
  detection, ack-state computation (self-ack reject, revoke
  semantics, multi-user/multi-item, numeric aliases), tier-mode
  selection, and end-to-end happy path.
- .gitea/workflows/sop-checklist-gate.yml — pull_request_target
  [opened/edited/synchronize/reopened] + issue_comment
  [created/edited/deleted]. Checks out BASE ref only (trust boundary
  per RFC#324 §A4). Mirrors qa-review/security-review patterns.

Why:
Hongming 2026-05-12T05:42Z asked for SOP-enforcing CI/CD that requires
peer-ack on each checklist item before merge. Composes the existing
patterns (scripts-lint PR-body parser + RFC#324 persona-whitelist
commit-status + sop-tier-check tier-awareness) into one gate.

Slash-command contract:
  /sop-ack <slug> [note]      — register peer-ack (most-recent wins)
  /sop-revoke <slug> [reason] — invalidate own prior ack

Slug normalization accepts kebab-case, snake_case, natural-spaces,
or numeric 1..7 shorthand (all canonicalize to kebab-case via the
config-driven alias table).

Tests: 51/51 pass locally. Dry-run probe against PR#685 verified the
full pipeline (PR fetch, comment fetch, ack computation, status
description rendering inside the 140-char budget).

Not yet:
- Phase 3 (24h soak)
- Phase 4 (BP PATCH to require this context — needs Hongming GO)
- Phase 5 (cross-repo)
- Phase 6 (dev-sop.md codification)
- SOP_CHECKLIST_GATE_TOKEN secret provisioning (separate follow-up;
  fail-closed until provisioned, same as RFC_324_TEAM_READ_TOKEN
  pattern in qa-review.yml).

Cross-links:
- internal#351 (RFC body)
- RFC#324 (qa-review/security-review — reused mechanism)
- internal#346 (dev-sop.md SOP-14..SOP-20 — sibling rules)
- feedback_pull_request_review_no_refire (why issue_comment trigger)
- feedback_checkpointed_workflow_over_good_practice_doc (motivation)
- feedback_fix_root_not_symptom (default-mode=hard rationale)

## What
Add a SOP-checklist peer-ack merge gate: workflow + script + config + 51 unit tests.

## Why
Hongming-requested mechanism to enforce SOP via CI/CD: each PR checklist
item must be peer-acked before merge, with team-membership-verified
ackers and tier-aware failure mode.

## Verification
- 51/51 unit tests pass (slug normalization, parse_directives, section
  marker detection, ack-state including self-ack rejection + revoke
  semantics, tier-mode mapping, end-to-end happy path).
- YAML lint clean (yaml.safe_load + lint-workflow-yaml.py on the new
  workflow — pre-existing fatals on unrelated files only).
- Python syntax clean (py_compile).
- Dry-run against live PR#685: PR fetch, comment enumeration, status
  description render all within 140-char budget — works end-to-end.

## Tier
tier:medium — net-new CI workflow; no production impact; no BP change
yet (Phase 4 separate auth).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 06:08:36 +00:00
c74c0a0283 fix(ci): add jq install to review-check-tests workflow + fix /tmp/jq hardcode
Some checks failed
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 14s
Lint curl status-code capture / Scan workflows for curl status-capture pollution (pull_request) Successful in 14s
CI / Detect changes (pull_request) Successful in 25s
review-check-tests / review-check.sh regression tests (pull_request) Successful in 15s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 32s
E2E API Smoke Test / detect-changes (pull_request) Successful in 34s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 34s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 17s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 30s
qa-review / approved (pull_request) Failing after 17s
security-review / approved (pull_request) Failing after 16s
sop-tier-check / tier-check (pull_request) Successful in 18s
gate-check-v3 / gate-check (pull_request) Successful in 27s
CI / Platform (Go) (pull_request) Successful in 7s
CI / Canvas (Next.js) (pull_request) Successful in 7s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 5s
CI / Python Lint & Test (pull_request) Successful in 6s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 6s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 8s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 9s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 6s
CI / all-required (pull_request) Successful in 3s
Two fixes found during first CI run:

1. Workflow missing jq installation step — T12 jq-filter test needs jq
   which is not in the Gitea Actions ubuntu-latest runner image.
   Add the same install dance as sop-tier-check.yml (apt-get first,
   GitHub binary download fallback, infra#241 belt-and-suspenders).

2. test_review_check.sh hardcodes /tmp/jq in T12. In CI jq gets
   installed to /usr/bin/jq via apt-get. Fix: use `command -v jq` to
   resolve from PATH first, fall back to /tmp/jq for local dev.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 01:24:24 +00:00
43cc27ade5 test(ci): add bats-style integration tests for review-check.sh (#540)
Some checks failed
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 23s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 13s
CI / Detect changes (pull_request) Successful in 1m6s
gate-check-v3 / gate-check (pull_request) Successful in 26s
E2E API Smoke Test / detect-changes (pull_request) Successful in 1m6s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 1m2s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 1m3s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 1m0s
qa-review / approved (pull_request) Failing after 20s
security-review / approved (pull_request) Failing after 17s
sop-tier-check / tier-check (pull_request) Successful in 23s
CI / Canvas (Next.js) (pull_request) Successful in 9s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 6s
CI / Python Lint & Test (pull_request) Successful in 6s
CI / Platform (Go) (pull_request) Successful in 7s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 15s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 11s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 10s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 8s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
CI / all-required (pull_request) Successful in 5s
Add 13 test cases (22 assertions) covering all key paths:
- open/closed PR handling
- non-author APPROVED review detection
- dismissed review exclusion
- team membership probe (204 member, 404 not-member, 403 fail-closed)
- missing GITEA_TOKEN exits 1
- CURL_AUTH_FILE mode 600 and header format
- jq filter correctness

Uses a Python HTTP fixture server that reads scenario from a temp
state dir, with a curl shim rewriting https://fixture.local/* to
http://127.0.0.1:{port}/*.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-11 20:33:14 +00:00
claude-ceo-assistant
2d096aa7ae feat(ci): sop-tier-check refire workflow via issue_comment (internal#292)
Some checks failed
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 28s
Harness Replays / detect-changes (pull_request) Failing after 15s
Lint curl status-code capture / Scan workflows for curl status-capture pollution (pull_request) Successful in 14s
Harness Replays / Harness Replays (pull_request) Has been skipped
CI / Detect changes (pull_request) Successful in 59s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 18s
E2E API Smoke Test / detect-changes (pull_request) Successful in 1m5s
sop-tier-check / tier-check (pull_request) Successful in 19s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 59s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 1m10s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 54s
CI / Platform (Go) (pull_request) Successful in 11s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 10s
CI / Python Lint & Test (pull_request) Successful in 8s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 10s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 7s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 9s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 9m10s
CI / Canvas (Next.js) (pull_request) Failing after 10m31s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
## Why

Gitea 1.22.6's `pull_request_review` event doesn't refire workflows
(go-gitea/gitea#33700). The existing sop-tier-check workflow subscribes
to the review event, but the subscription is silently dead. When an
approving review lands AFTER tier-check ran on PR-open/synchronize, the
PR's `sop-tier-check / tier-check (pull_request)` status stays at
failure forever, forcing the orchestrator down the admin force-merge
path (audited via audit-force-merge.yml, but the audit trail keeps
growing — see feedback_never_admin_merge_bypass).

## What

New `.gitea/workflows/sop-tier-refire.yml` listening on `issue_comment`
events. When a repo MEMBER/OWNER/COLLABORATOR comments
`/refire-tier-check` on a PR, the workflow re-invokes the canonical
sop-tier-check.sh and POSTs the resulting status directly to the PR
head SHA (no empty commit, no git history bloat, no cascade re-fire of
every other workflow).

## Security model

Three gates in the workflow `if:` expression — all required:

1. `github.event.issue.pull_request != null` — comment is on a PR, not
   a plain issue.
2. `author_association` ∈ {MEMBER, OWNER, COLLABORATOR} — only repo
   collaborators+ can flip the status (per the internal#292 core-security
   review#1066 ask).
3. Comment body contains `/refire-tier-check` — slash-command-shaped,
   not just any word in normal review prose.

Workflow does NOT check out PR HEAD; only HTTP-calls the Gitea API.
Same trust boundary as sop-tier-check.yml's `pull_request_target`.

## DRY: re-uses sop-tier-check.sh

Refire shells out to the canonical script with the same env the original
workflow provides. We get the EXACT AND-composition gate, not a
watered-down approving-count check.

## Rate-limit

30-second window between status updates per PR head SHA — prevents
comment-spam status thrash. Override via SOP_REFIRE_RATE_LIMIT_SEC or
disable for tests via SOP_REFIRE_DISABLE_RATE_LIMIT=1.

## Tests

`.gitea/scripts/tests/test_sop_tier_refire.sh` — 23 assertions across
T1-T7 covering: success POST, failure POST, no-op on closed, rate-limit
skip, plus YAML-level checks of all three security gates. Real script
runs against a local-fixture HTTP server (`_refire_fixture.py`) with a
mock tier-check (`_mock_tier_check.sh`) — the latter sidesteps the
known bash 3.2 (macOS dev) parser bug on `declare -A`; Linux Gitea
runners (bash 4/5) use the real sop-tier-check.sh in production.

Hostile self-review verified:
- Tests FAIL on absent code (exit 1, FAIL=2 PASS=0 in existence-block).
- Tests FAIL on swapped success/failure label (exit 1).
- Tests PASS on correct code (exit 0, 23/23).

## Brief-falsification log

(a) Keep using force_merge — no, this is the issue being closed.
(b) Empty-commit re-trigger — no, status-POST is cleaner + faster +
    doesn't bloat git history.
(c) author_association check in the script not the workflow — both work
    but workflow-level short-circuits faster (saves runner spin).
(d) Re-implement a watered-down tier-check inside refire — no, that's a
    security regression (skips team-membership AND-composition).
    Refire shells out to the canonical script.

Tier: tier:high (unblocks approved-PR-backlog drain class).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 02:44:31 -07:00
dev-lead
b75187d11c fix(sop-tier-check): clause splitter strips newlines, OR-set collapses to one token (#229)
Some checks failed
sop-tier-check / tier-check (pull_request) Failing after 5s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 5s
audit-force-merge / audit (pull_request) Successful in 4s
PR #225 introduced the AND-composition clause evaluator. PR #231
patched the per-team case-pattern matching but did NOT fix the
underlying clause-splitter bug. This PR fixes the actual root cause
behind issue #229.

Root cause (.gitea/scripts/sop-tier-check.sh ~line 289):

    _clause=$(echo "$_raw_clause" \
      | tr -d '()' \
      | tr ',' '\n' \
      | tr -d '[:space:]' \
      | grep -v '^$')

`tr -d '[:space:]'` strips the newlines that `tr ',' '\n'` just
inserted. For tier:low (expression "engineers,managers,ceo") the
intermediate value is:

    engineers\nmanagers\nceo

then `tr -d '[:space:]'` flattens it to:

    engineersmanagersceo

The for-loop iterates ONCE over this single bogus token. The case
pattern `*engineersmanagersceo*` never matches APPROVER_TEAMS values
like " managers ", so EVERY tier:low PR fails:

    ::error::clause [engineers/managers/ceo]: FAIL — no approving
    reviewer belongs to any of these teamsengineersmanagersceo
    ::error::sop-tier-check FAILED for tier:low

(Note: the missing separators in the error string `teamsengineersmanagersceo`
were a SECOND, masked bug — `_clause_names="${_clause_names:+, }${_t}"`
overwrites the variable on every iteration instead of appending. With
the splitter bug, the inner loop only ran once so the overwrite was
invisible. Fixing the splitter unmasks the accumulator bug, so we fix
both atomically.)

Fix:

  _no_parens=${_raw_clause//[()]/}
  _clause=${_no_parens//,/ }   # comma -> space, bash word-split iterates

  # Append, don't overwrite:
  _clause_names="${_clause_names}${_clause_names:+, }${_t}"
  _passed_clauses="${_passed_clauses}${_passed_clauses:+, }$_label"
  _failed_clauses="${_failed_clauses}${_failed_clauses:+, }$_label"

Per-tier policy is UNCHANGED — this is a parser fix, not a policy
relaxation:

  tier:low    — engineers,managers,ceo   (OR-set, ANY ONE suffices)
  tier:medium — managers AND engineers AND qa???,security???
  tier:high   — ceo

Test: .gitea/scripts/tests/test_sop_tier_check_clause_split.sh
asserts the splitter, accumulators, and end-to-end OR-gate matching
against APPROVER_TEAMS=" managers " (the exact shape PRs #233-238 hit).
7/7 pass on the new logic.

Refs: #229, supersedes attempted fix in #231 for the same root cause.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 22:03:12 -07:00