Provision staging CP admin token + properly re-promote template-delivery-e2e (E2E-staging red env-wide blocks PRs) #3086

Open
opened 2026-06-19 22:09:18 +00:00 by core-devops · 0 comments
Member

Staging E2E cluster + template-delivery-e2e are red env-wide → blocks PRs; need staging creds + proper re-promotion

Symptom: the E2E-Staging-SaaS cluster (8 jobs) + template-delivery-e2e fail on essentially every PR. This blocked merges all of 2026-06-19 (worked around by admin/force-merge + the sweeper).

Root cause: the staging CP admin token is not provisioned for staging E2E. The staging auto-deploy step fails with staging redeploy cannot run — CP_STAGING_ADMIN_API_TOKEN secret missing, and template-delivery-e2e likewise needs CP_ADMIN_API_TOKEN against staging. So these jobs cannot pass until the secret exists — they're red for an infra/creds reason, not a code reason.

BP drift (mitigated today): template-delivery-e2e had been added to main BP required while its workflow still has paths filters + continue-on-error + an "advisory, do not require until green" header. That made lint-required-no-paths + lint-no-coe-on-required fail on every PR. Mitigation applied: removed template-delivery-e2e from main BP required contexts (6→5) to unblock. This is temporary.

To do (durable):

  1. Provision the staging CP admin token as a Gitea Actions secret (CP_STAGING_ADMIN_API_TOKEN / CP_ADMIN_API_TOKEN for staging) — pull from staging-CP's Railway env. (CTO/owner — secret provisioning.)
  2. Once staging E2E can actually pass: properly promote template-delivery-e2e (B) — remove paths filters + continue-on-error, add to .gitea/required-contexts.txt, add PR-time skip-loud when creds absent, then re-add to BP required.
  3. Same treatment likely needed for the E2E-Staging-SaaS cluster contexts.

Relates: #37 (the partial promotion), #3081 (real-capability e2e), #3082 (fail-loud). This is the broader staging-CP-broken issue also behind the staging redeploy-fleet failures.

## Staging E2E cluster + template-delivery-e2e are red env-wide → blocks PRs; need staging creds + proper re-promotion **Symptom:** the E2E-Staging-SaaS cluster (8 jobs) + `template-delivery-e2e` fail on essentially every PR. This blocked merges all of 2026-06-19 (worked around by admin/force-merge + the sweeper). **Root cause:** the **staging CP admin token is not provisioned** for staging E2E. The staging auto-deploy step fails with `staging redeploy cannot run — CP_STAGING_ADMIN_API_TOKEN secret missing`, and `template-delivery-e2e` likewise needs `CP_ADMIN_API_TOKEN` against staging. So these jobs **cannot pass** until the secret exists — they're red for an infra/creds reason, not a code reason. **BP drift (mitigated today):** `template-delivery-e2e` had been added to main BP *required* while its workflow still has `paths` filters + `continue-on-error` + an "advisory, do not require until green" header. That made `lint-required-no-paths` + `lint-no-coe-on-required` fail on every PR. **Mitigation applied:** removed `template-delivery-e2e` from main BP required contexts (6→5) to unblock. This is temporary. **To do (durable):** 1. **Provision the staging CP admin token** as a Gitea Actions secret (`CP_STAGING_ADMIN_API_TOKEN` / `CP_ADMIN_API_TOKEN` for staging) — pull from staging-CP's Railway env. (CTO/owner — secret provisioning.) 2. Once staging E2E can actually pass: properly promote `template-delivery-e2e` (B) — remove `paths` filters + `continue-on-error`, add to `.gitea/required-contexts.txt`, add PR-time skip-loud when creds absent, then re-add to BP required. 3. Same treatment likely needed for the E2E-Staging-SaaS cluster contexts. Relates: #37 (the partial promotion), #3081 (real-capability e2e), #3082 (fail-loud). This is the broader staging-CP-broken issue also behind the staging redeploy-fleet failures.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: molecule-ai/molecule-core#3086