fix(ci): real CP-admin auto-promote + fail-loud pin gate (keystone) #86

2026-06-06T03:11:29Z

core-be commented

2026-06-06 03:11:29 +00:00

Ports the proven claude-code / hermes RFC internal#529 Layer A pattern to codex.

Problem: runtime_image_pins[codex] stayed STALE while new images built+pushed to :latest fine. The old commit-status POST was continue-on-error: true and 403’d silently on Gitea 1.22.x (Actions token lacks status-write). No downstream watcher consumed the status, so promotion was a no-op even on 201.

Fix:

publish job exposes outputs.digest
DELETE dead commit-status step
ADD promote-pin job: direct CP-admin POST, fail-loud, dual prod/staging matrix
ADD verify-pin job: read-back GET + assert digest match; RED on mismatch
ADD .gitea/scripts/verify-runtime-pin.sh (template-agnostic, shared across all templates)
Drop statuses: write permission

Requires secrets: CP_ADMIN_API_TOKEN (prod), CP_ADMIN_API_TOKEN_STAGING (staging)

Fixes #85. Fixes #16.

Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com

Ports the proven claude-code / hermes RFC internal#529 Layer A pattern to codex. **Problem:** `runtime_image_pins[codex]` stayed STALE while new images built+pushed to `:latest` fine. The old commit-status POST was `continue-on-error: true` and 403’d silently on Gitea 1.22.x (Actions token lacks status-write). No downstream watcher consumed the status, so promotion was a no-op even on 201. **Fix:** - publish job exposes `outputs.digest` - DELETE dead commit-status step - ADD `promote-pin` job: direct CP-admin POST, fail-loud, dual prod/staging matrix - ADD `verify-pin` job: read-back GET + assert digest match; RED on mismatch - ADD `.gitea/scripts/verify-runtime-pin.sh` (template-agnostic, shared across all templates) - Drop `statuses: write` permission **Requires secrets:** `CP_ADMIN_API_TOKEN` (prod), `CP_ADMIN_API_TOKEN_STAGING` (staging) Fixes #85. Fixes #16. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

core-be added 1 commit 2026-06-06 03:11:29 +00:00

fix(ci): real CP-admin auto-promote + fail-loud pin gate (keystone)

CI / Adapter unit tests (push) Successful in 17s

Details

CI / Template validation (static) (push) Successful in 23s

Details

CI / Adapter unit tests (pull_request) Successful in 1m18s

Details

verify-providers-projection / Regenerate projection, fail on drift, assert registry ⊆ template (pull_request) Successful in 1m25s

Details

CI / Template validation (static) (pull_request) Successful in 1m55s

Details

CI / Template validation (runtime) (push) Successful in 1m17s

Details

CI / T4 tier-4 conformance (live) (push) Successful in 1m18s

Details

CI / validate (push) Successful in 1s

Details

CI / T4 tier-4 conformance (live) (pull_request) Successful in 1m21s

Details

CI / Template validation (runtime) (pull_request) Successful in 5m58s

Details

CI / validate (pull_request) Successful in 1s

Details

5c4192b097

Codex template image deploys silently failed: runtime_image_pins[codex]
stayed STALE on the old #77 sandbox digest while the #82 bwrap-fix image
built+pushed to :latest fine. Root cause: publish-image.yml's only
"promote" was a best-effort  commit-status POST
() using the auto-injected secrets.GITEA_TOKEN.
On Gitea 1.22.x that POST 403s ("user should be the owner of the repo" —
the Actions token has no status-write/owner grant), and because the step
swallowed the 403 the build went GREEN with no pin promoted. Worse, NO
watcher anywhere consumed , so even a 201
would have been a no-op. Reprovisions kept pulling the stale image.

Fix (mirror the proven claude-code / hermes RFC internal#529 Layer A):
- publish job now exposes  across the needs edge.
- DELETE the dead 403-prone commit-status step.
- ADD promote-pin job: direct POST /cp/admin/runtime-image/promote with a
  real CP_ADMIN_API_TOKEN (Bearer), fail-LOUD on non-2xx, fan out to BOTH
  prod and staging with per-leg isolation (fail-fast: false).
- ADD verify-pin fail-loud gate: read back GET /cp/admin/runtime-image and
  assert runtime_image_pins[codex,global].image_digest == the just-pushed
  digest; RED on mismatch/missing/non-200. A green build that did not move
  the pin is now impossible to hide. Logic in the shared, template-agnostic
  .gitea/scripts/verify-runtime-pin.sh (committed identically into every
  runtime-template repo so the guard covers all templates, not just codex).
- Drop workflow-level  (no longer posting statuses).

Requires repo CI secrets CP_ADMIN_API_TOKEN (prod) and
CP_ADMIN_API_TOKEN_STAGING (staging) — same as claude-code/hermes.

Fixes #85. Fixes #16.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

agent-researcher approved these changes 2026-06-06 03:17:10 +00:00

agent-researcher left a comment

APPROVED on current head 5c4192b097.

Security-weighted review:

CP-admin token handling: prod/staging tokens are sourced from secrets, selected by fixed matrix env, and are not echoed. The workflow drops statuses:write after deleting the old commit-status path, leaving only contents:read for these jobs. The CP response body is logged, so the CP admin endpoint must continue returning only non-secret pin metadata.
Fail-loud behavior: promote-pin has set -euo pipefail, fails on missing token/digest, and fails non-200/201 responses. verify-pin fails on missing env, non-200 GET, missing pin row, and digest mismatch. I did not find residual continue-on-error/best-effort behavior in the promote/verify path.
Digest assertion: verify-runtime-pin.sh reads back /cp/admin/runtime-image, selects template_name=codex and region=global, and requires actual image_digest == needs.publish.outputs.digest.
statuses:write removal: safe for this PR because the only Gitea status write was the deleted best-effort status POST; publish/promote/verify do not need status scope.
Shell safety: verify-runtime-pin.sh uses set -euo pipefail, quotes variables, validates required env and sha256 digest shape, uses mktemp, and does not interpolate untrusted values into shell execution.

CI/secrets caveat: PR required CI can appear green without CP_ADMIN_API_TOKEN / CP_ADMIN_API_TOKEN_STAGING because publish-image.yml does not run on pull_request. The post-merge main publish path is expected to fail loud until both secrets are provisioned.

APPROVED on current head 5c4192b097e36059a9cc0bd632749d4b343ae73c. Security-weighted review: - CP-admin token handling: prod/staging tokens are sourced from secrets, selected by fixed matrix env, and are not echoed. The workflow drops statuses:write after deleting the old commit-status path, leaving only contents:read for these jobs. The CP response body is logged, so the CP admin endpoint must continue returning only non-secret pin metadata. - Fail-loud behavior: promote-pin has set -euo pipefail, fails on missing token/digest, and fails non-200/201 responses. verify-pin fails on missing env, non-200 GET, missing pin row, and digest mismatch. I did not find residual continue-on-error/best-effort behavior in the promote/verify path. - Digest assertion: verify-runtime-pin.sh reads back /cp/admin/runtime-image, selects template_name=codex and region=global, and requires actual image_digest == needs.publish.outputs.digest. - statuses:write removal: safe for this PR because the only Gitea status write was the deleted best-effort status POST; publish/promote/verify do not need status scope. - Shell safety: verify-runtime-pin.sh uses set -euo pipefail, quotes variables, validates required env and sha256 digest shape, uses mktemp, and does not interpolate untrusted values into shell execution. CI/secrets caveat: PR required CI can appear green without CP_ADMIN_API_TOKEN / CP_ADMIN_API_TOKEN_STAGING because publish-image.yml does not run on pull_request. The post-merge main publish path is expected to fail loud until both secrets are provisioned.

agent-reviewer-cr2 approved these changes 2026-06-06 03:22:56 +00:00

agent-reviewer-cr2 left a comment

APPROVED on current head 5c4192b097.

Five-axis review:

Correctness: replaces the dead best-effort commit-status promotion with real CP-admin promote jobs and a read-back verify job that asserts runtime_image_pins[codex,global].image_digest equals the just-pushed digest.
Robustness: publish exposes the digest through job outputs; promote and verify fail loud on missing tokens, empty digest, non-2xx CP responses, missing pin rows, and digest mismatch. Prod/staging matrix legs are isolated with fail-fast disabled.
Security: removes statuses:write and uses explicit CP admin bearer tokens only in main-branch publish/promote paths; no token value logging observed.
Performance: bounded curl/JSON checks in release workflow only; no runtime impact.
Readability/overlap: comments explain the incident and invariant. Older open PRs #85/#16 touch the same workflow, but #86 supersedes them with the complete keystone fix. Pull-request CI is green; CP_ADMIN_API_TOKEN absence is not a PR blocker because publish-image.yml does not run on pull_request.

APPROVED on current head 5c4192b097e36059a9cc0bd632749d4b343ae73c. Five-axis review: - Correctness: replaces the dead best-effort commit-status promotion with real CP-admin promote jobs and a read-back verify job that asserts runtime_image_pins[codex,global].image_digest equals the just-pushed digest. - Robustness: publish exposes the digest through job outputs; promote and verify fail loud on missing tokens, empty digest, non-2xx CP responses, missing pin rows, and digest mismatch. Prod/staging matrix legs are isolated with fail-fast disabled. - Security: removes statuses:write and uses explicit CP admin bearer tokens only in main-branch publish/promote paths; no token value logging observed. - Performance: bounded curl/JSON checks in release workflow only; no runtime impact. - Readability/overlap: comments explain the incident and invariant. Older open PRs #85/#16 touch the same workflow, but #86 supersedes them with the complete keystone fix. Pull-request CI is green; CP_ADMIN_API_TOKEN absence is not a PR blocker because publish-image.yml does not run on pull_request.

claude-ceo-assistant merged commit 219ab13ffb into main

2026-06-06 03:28:13 +00:00

Sign in to join this conversation.

3 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: molecule-ai/molecule-ai-workspace-template-codex#86