[ci-hardening] Refresh tracker for remaining continue-on-error masks after mc#664 closed #774

New Issue

hongming-codex-laptop · 2026-05-13T00:51:37Z

2026-05-13 00:51:37 +00:00

Summary

Fresh tracker for remaining .gitea/workflows/** continue-on-error: true masks after the previous tracker mc#664 was closed. The lint gate now correctly rejects masks that reference closed issues.

Why this exists

Main/PR CI still contains a broad set of Phase-3 masks. Some are known root-fix work, some may be removable after the latest CI repairs. Until each mask is removed, every continue-on-error: true must point at an open tracker so the debt cannot disappear silently.

Audit Findings (2026-05-13, core-devops)

`.gitea/workflows/sop-tier-check.yml`

Line	Scope	Rationale	Can Remove?
68	Job-level	BURN-IN (internal#189): prevents AND-composition from blocking PRs during 7-day window	Yes — after 2026-05-17. See mc#774-followup issue.
92	Step-level	jq install belt-and-suspenders — script has fallback, this is third line of defense	Conditional: remove once jq install is reliable in all runner networks (see internal#241).
112	Step-level	Tier check verification — triple-layer: step-level mask + SOP_FAIL_OPEN=1 + `\|\| true`	Conditional: remove when SOP tier check is fully stable and `SOP_FAIL_OPEN=1` is removed from script.

Other workflows

.github/workflows/codeql.yml: Comment-only — no active continue-on-error: true mask (references historical PR #35 in comment). No action needed.
.gitea/workflows/audit-force-merge.yml: No masks — clean.
.gitea/workflows/gate-check-v3.yml: Job-level mask documented with mc#774 comment (see .gitea/workflows/gate-check-v3.yml line 50). Acceptable: detector failure should not block PRs.

Exit Criteria

Audit every remaining continue-on-error: true in .gitea/workflows/ and .github/workflows/
Remove job-level mask in sop-tier-check.yml after 2026-05-17 burn-in window closes (track in dedicated issue)
Evaluate step-level jq-install mask removal after internal#241 resolved
Evaluate step-level tier-check mask removal when SOP tier check is stable
Close this issue only after no workflow comment references it.

## Summary Fresh tracker for remaining `.gitea/workflows/**` `continue-on-error: true` masks after the previous tracker `mc#664` was closed. The lint gate now correctly rejects masks that reference closed issues. ## Why this exists Main/PR CI still contains a broad set of Phase-3 masks. Some are known root-fix work, some may be removable after the latest CI repairs. Until each mask is removed, every `continue-on-error: true` must point at an open tracker so the debt cannot disappear silently. ## Audit Findings (2026-05-13, core-devops) ### `.gitea/workflows/sop-tier-check.yml` | Line | Scope | Rationale | Can Remove? | |------|-------|----------|-------------| | 68 | Job-level | BURN-IN (internal#189): prevents AND-composition from blocking PRs during 7-day window | **Yes — after 2026-05-17**. See mc#774-followup issue. | | 92 | Step-level | jq install belt-and-suspenders — script has fallback, this is third line of defense | **Conditional**: remove once jq install is reliable in all runner networks (see internal#241). | | 112 | Step-level | Tier check verification — triple-layer: step-level mask + SOP_FAIL_OPEN=1 + `\|\| true` | **Conditional**: remove when SOP tier check is fully stable and `SOP_FAIL_OPEN=1` is removed from script. | ### Other workflows - `.github/workflows/codeql.yml`: **Comment-only** — no active `continue-on-error: true` mask (references historical PR #35 in comment). No action needed. - `.gitea/workflows/audit-force-merge.yml`: **No masks** — clean. - `.gitea/workflows/gate-check-v3.yml`: **Job-level mask documented** with mc#774 comment (see `.gitea/workflows/gate-check-v3.yml` line 50). Acceptable: detector failure should not block PRs. ## Exit Criteria - [x] Audit every remaining `continue-on-error: true` in `.gitea/workflows/` and `.github/workflows/` - [ ] Remove job-level mask in `sop-tier-check.yml` after 2026-05-17 burn-in window closes (track in dedicated issue) - [ ] Evaluate step-level jq-install mask removal after internal#241 resolved - [ ] Evaluate step-level tier-check mask removal when SOP tier check is stable - [ ] Close this issue only after no workflow comment references it.

hongming-codex-laptop added the

tier:high

label 2026-05-13 00:51:37 +00:00

hongming-codex-laptop commented

2026-05-13 04:14:27 +00:00

Hourly CI/CD triage evidence, 2026-05-13T04:12Z:

Runner fleet was alive, not dead: operator host showed molecule-runner-1..12 up, plus active GITEA-ACTIONS task containers.
Current open-PR status pattern across the triage repos is backlog/required-condition fan-out: most non-success statuses are Waiting to run or Blocked by required conditions.
Verified one concrete mc#774 subcase in live runner logs: Platform Go ran golangci-lint run --timeout 3m ./... || true, logged golangci-lint: command not found, then reported the step successful because the command was masked.

Action taken: opened PR #781 to remove the step-level || true masks from Platform Go go vet and golangci-lint, while leaving the broader job-level burn-in context unchanged.

Hourly CI/CD triage evidence, 2026-05-13T04:12Z: - Runner fleet was alive, not dead: operator host showed molecule-runner-1..12 up, plus active GITEA-ACTIONS task containers. - Current open-PR status pattern across the triage repos is backlog/required-condition fan-out: most non-success statuses are `Waiting to run` or `Blocked by required conditions`. - Verified one concrete mc#774 subcase in live runner logs: Platform Go ran `golangci-lint run --timeout 3m ./... || true`, logged `golangci-lint: command not found`, then reported the step successful because the command was masked. Action taken: opened PR #781 to remove the step-level `|| true` masks from Platform Go `go vet` and `golangci-lint`, while leaving the broader job-level burn-in context unchanged.

Sign in to join this conversation.