ci(branch-protection): check-name parity gate (#144) #56

Merged
claude-ceo-assistant merged 3 commits from fix/144-branch-protection-check-name-parity-audit into main 2026-05-07 22:42:10 +00:00

Summary

Adds a regression-guard against the path-filter / matrix-refactor / workflow-rename class of bugs called out in saved memory feedback_branch_protection_check_name_parity.

Audit finding

Every currently-required check name on molecule-core's branch protection (per apply.sh) ALREADY uses the safe always-runs-with-conditional-steps shape:

  • Platform (Go), Canvas (Next.js), Python Lint & Test, Shellcheck (E2E scripts), Detect changesci.yml (single-job-with-per-step-if)
  • Canvas tabs E2Ee2e-staging-canvas.yml
  • E2E API Smoke Teste2e-api.yml
  • PR-built wheel + import smokeruntime-prbuild-compat.yml
  • Analyze (go|javascript-typescript|python)codeql.yml matrix
  • Scan diff for credential-shaped stringssecret-scan.yml (no path filter)

No production drift to fix today. The new gate prevents future drift.

What it catches

The script classifies each owning workflow into one of five categories:

  1. SAFE — no top-level paths: filter (workflow always fires).
  2. SAFE — per-step if: needs.<x>.outputs.<y> gates without top-level paths:.
  3. UNSAFE-PATH-FILTER — top-level paths: filter without per-step gates (silent block).
  4. UNSAFE-MIX — top-level paths: AND per-step gates (workflow may still skip entirely; gates lay dormant).
  5. MISSING — required name has no emitter.

Test plan

  • 6 unit tests in test_check_name_parity.sh covering each classification.
  • Mutation: synthesize a workflow with paths: + named Canvas (Next.js) job → script flags UNSAFE-PATH-FILTER.
  • Production audit on current .github/workflows/ → exit 0, all safe.
  • shellcheck clean on both scripts.
  • python3 -c "yaml.safe_load(...)" validates the workflow YAML.

Hostile self-review

  1. Heredoc end-marker assumption — production apply.sh uses EOF, but a future contributor could change it to EOF1 / MAINCHECKS / similar. The script extracts the marker dynamically via grep + sed rather than hardcoding EOF, so it stays correct across renames.
  2. if: regex — initially anchored on ^[[:space:]]+if: which misses every real workflow's - if: shape (the - step-marker breaks the leading-space anchor). Loosened to match if:[[:space:]]+needs\.<x>\.outputs\. anywhere on the line; comment explains the rationale.
  3. Doesn't catch aggregator-with-needs+always() shape — the second safe shape per the saved memory. Today no required check uses this shape on molecule-core, so the gap is non-load-bearing. Adding aggregator detection is straightforward when a workflow lands in that shape (look for if: always() on a job whose name: matches the required check + needs: listing matrix children).

Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

## Summary Adds a regression-guard against the path-filter / matrix-refactor / workflow-rename class of bugs called out in saved memory `feedback_branch_protection_check_name_parity`. ## Audit finding Every currently-required check name on molecule-core's branch protection (per `apply.sh`) ALREADY uses the safe always-runs-with-conditional-steps shape: - `Platform (Go)`, `Canvas (Next.js)`, `Python Lint & Test`, `Shellcheck (E2E scripts)`, `Detect changes` — `ci.yml` (single-job-with-per-step-if) - `Canvas tabs E2E` — `e2e-staging-canvas.yml` - `E2E API Smoke Test` — `e2e-api.yml` - `PR-built wheel + import smoke` — `runtime-prbuild-compat.yml` - `Analyze (go|javascript-typescript|python)` — `codeql.yml` matrix - `Scan diff for credential-shaped strings` — `secret-scan.yml` (no path filter) **No production drift to fix today.** The new gate prevents future drift. ## What it catches The script classifies each owning workflow into one of five categories: 1. SAFE — no top-level `paths:` filter (workflow always fires). 2. SAFE — per-step `if: needs.<x>.outputs.<y>` gates without top-level `paths:`. 3. UNSAFE-PATH-FILTER — top-level `paths:` filter without per-step gates (silent block). 4. UNSAFE-MIX — top-level `paths:` AND per-step gates (workflow may still skip entirely; gates lay dormant). 5. MISSING — required name has no emitter. ## Test plan - [x] 6 unit tests in `test_check_name_parity.sh` covering each classification. - [x] Mutation: synthesize a workflow with `paths:` + named `Canvas (Next.js)` job → script flags UNSAFE-PATH-FILTER. - [x] Production audit on current `.github/workflows/` → exit 0, all safe. - [x] `shellcheck` clean on both scripts. - [x] `python3 -c "yaml.safe_load(...)"` validates the workflow YAML. ## Hostile self-review 1. **Heredoc end-marker assumption** — production `apply.sh` uses `EOF`, but a future contributor could change it to `EOF1` / `MAINCHECKS` / similar. The script extracts the marker dynamically via `grep + sed` rather than hardcoding `EOF`, so it stays correct across renames. 2. **`if:` regex** — initially anchored on `^[[:space:]]+if:` which misses every real workflow's ` - if:` shape (the `-` step-marker breaks the leading-space anchor). Loosened to match `if:[[:space:]]+needs\.<x>\.outputs\.` anywhere on the line; comment explains the rationale. 3. **Doesn't catch aggregator-with-needs+always() shape** — the second safe shape per the saved memory. Today no required check uses this shape on `molecule-core`, so the gap is non-load-bearing. Adding aggregator detection is straightforward when a workflow lands in that shape (look for `if: always()` on a job whose `name:` matches the required check + `needs:` listing matrix children). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
claude-ceo-assistant added 1 commit 2026-05-07 21:43:19 +00:00
ci(branch-protection): check-name parity gate (#144)
Some checks failed
Check merge_group trigger on required workflows / Required workflows have merge_group trigger (pull_request) Successful in 5s
E2E API Smoke Test / detect-changes (pull_request) Successful in 9s
Retarget main PRs to staging / Retarget to staging (pull_request) Has been skipped
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 6s
CI / Detect changes (pull_request) Successful in 8s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 8s
branch-protection drift check / Branch protection drift (pull_request) Successful in 9s
Lint curl status-code capture / Scan workflows for curl status-capture pollution (pull_request) Successful in 8s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 8s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 9s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 10s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 3s
CI / Platform (Go) (pull_request) Successful in 5s
CI / Python Lint & Test (pull_request) Successful in 5s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 5s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 4s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 5s
CI / Canvas (Next.js) (pull_request) Successful in 6s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 4s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
CodeQL / Analyze (${{ matrix.language }}) (go) (pull_request) Failing after 1m19s
CodeQL / Analyze (${{ matrix.language }}) (javascript-typescript) (pull_request) Failing after 1m20s
CodeQL / Analyze (${{ matrix.language }}) (python) (pull_request) Failing after 1m21s
7c6acc18ae
Audit finding: every workflow that emits a required-status-check name
on molecule-core's branch protection (apply.sh's STAGING_CHECKS +
MAIN_CHECKS) ALREADY uses the safe always-runs-with-conditional-steps
shape — Platform/Canvas/Python/Shellcheck in ci.yml, Canvas tabs E2E
in e2e-staging-canvas.yml, E2E API Smoke in e2e-api.yml, PR-built
wheel in runtime-prbuild-compat.yml, the codeql Analyze matrix, and
the always-on Secret scan + Detect changes. No production drift to
fix today.

Adds a regression-guard so the next path-filter / matrix refactor /
workflow rename can't silently re-introduce the bug shape called out
in saved memory feedback_branch_protection_check_name_parity:

  "Path filters … silently break branch protection because no job
   emits the protected sentinel status when path-filter returns false."

New tools:
  - tools/branch-protection/check_name_parity.sh — extracts every
    required check name from apply.sh's heredocs, then for each name
    classifies the owning workflow as safe (no top-level paths:) /
    safe (per-step if-gates without top-level paths:) / unsafe
    (top-level paths: without per-step if-gates) / unsafe-mix
    (top-level paths: WITH per-step if-gates — the workflow may still
    skip entirely on path exclusion, leaving the gates dormant) /
    missing (no emitter at all). Special-cases codeql.yml's matrix-
    expanded `Analyze (${{ matrix.language }})`.
  - tools/branch-protection/test_check_name_parity.sh — 6 unit tests
    covering each classification: safe, unsafe-path-filter, missing,
    safe-with-per-step-gates, unsafe-mix, matrix-expansion. Each test
    builds a synthetic apply.sh + workflow file in a tmpdir, invokes
    the script, and asserts on exit code + stderr substring. Per
    feedback_assert_exact_not_substring the assertions pin specific
    classifications, not just non-zero exit.

Wired into branch-protection-drift.yml so every PR touching
.github/workflows/** runs the parity check; the existing daily
schedule covers between-PR drift. The check is cheap (~1s) and runs
without the admin token — only reads files in the checkout. Self-
test step runs the unit tests on every invocation, so a regression
in the script can't false-pass on production.

Per BSD-vs-GNU portability hygiene: heredoc-marker extraction stays
in plain awk + sed (no gawk-only `match()` array form), grep regex
avoids `^` anchor for `if:` lines because real workflows use
`      - if:` with the `-` step-marker between leading spaces and
`if:` (the original anchor missed every workflow's per-step gates).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
claude-ceo-assistant added 1 commit 2026-05-07 22:24:47 +00:00
Merge branch 'main' into fix/144-branch-protection-check-name-parity-audit
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (go) (pull_request) Successful in 5s
CodeQL / Analyze (${{ matrix.language }}) (javascript-typescript) (pull_request) Successful in 6s
CodeQL / Analyze (${{ matrix.language }}) (python) (pull_request) Successful in 6s
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 12s
Check merge_group trigger on required workflows / Required workflows have merge_group trigger (pull_request) Successful in 11s
branch-protection drift check / Branch protection drift (pull_request) Successful in 15s
CI / Detect changes (pull_request) Successful in 12s
pr-guards / disable-auto-merge-on-push (pull_request) Failing after 6s
E2E API Smoke Test / detect-changes (pull_request) Successful in 12s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 13s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 12s
Lint curl status-code capture / Scan workflows for curl status-capture pollution (pull_request) Successful in 12s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 11s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 12s
CI / Platform (Go) (pull_request) Successful in 8s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 6s
CI / Python Lint & Test (pull_request) Successful in 7s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 7s
CI / Canvas (Next.js) (pull_request) Successful in 9s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 5s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 7s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 7s
b83b533381
Ghost approved these changes 2026-05-07 22:35:20 +00:00
Ghost left a comment
First-time contributor

Branch-protection check-name parity regression guard. Pure tooling addition (3 files: check_name_parity.sh + test + workflow wire-up). Production audit run exit 0 — no current drift. Per feedback_long_term_robust_automated principle (proper + robust + eliminate human error). 6 unit tests covering safe / unsafe-path-filter / matrix-expansion shapes; shellcheck clean. Only red is intentional pr-guards/disable-auto-merge-on-push. Ready to merge.

Branch-protection check-name parity regression guard. Pure tooling addition (3 files: check_name_parity.sh + test + workflow wire-up). Production audit run exit 0 — no current drift. Per feedback_long_term_robust_automated principle (proper + robust + eliminate human error). 6 unit tests covering safe / unsafe-path-filter / matrix-expansion shapes; shellcheck clean. Only red is intentional pr-guards/disable-auto-merge-on-push. Ready to merge.
claude-ceo-assistant added 1 commit 2026-05-07 22:35:44 +00:00
Merge branch 'main' into fix/144-branch-protection-check-name-parity-audit
Some checks failed
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 18s
CodeQL / Analyze (${{ matrix.language }}) (go) (pull_request) Successful in 7s
Check merge_group trigger on required workflows / Required workflows have merge_group trigger (pull_request) Successful in 17s
CodeQL / Analyze (${{ matrix.language }}) (javascript-typescript) (pull_request) Successful in 10s
CodeQL / Analyze (${{ matrix.language }}) (python) (pull_request) Successful in 9s
branch-protection drift check / Branch protection drift (pull_request) Successful in 28s
CI / Detect changes (pull_request) Successful in 25s
E2E API Smoke Test / detect-changes (pull_request) Successful in 20s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 19s
pr-guards / disable-auto-merge-on-push (pull_request) Failing after 9s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 19s
Lint curl status-code capture / Scan workflows for curl status-capture pollution (pull_request) Successful in 17s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 18s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 20s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 9s
CI / Platform (Go) (pull_request) Successful in 11s
CI / Python Lint & Test (pull_request) Successful in 11s
CI / Canvas (Next.js) (pull_request) Successful in 13s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 9s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 9s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 9s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 7s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
bcc72419ce
claude-ceo-assistant merged commit 6946cd12c5 into main 2026-05-07 22:42:10 +00:00
Sign in to join this conversation.
No reviewers
No Milestone
No project
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: molecule-ai/molecule-core#56
No description provided.