molecule-ai/molecule-core
34
0
Fork 2
Code Issues 16 Pull Requests 2 Actions 49 Packages Projects Releases Wiki Activity

Auto-sync main → staging fails every push: gh CLI calls /api/graphql (Gitea returns 405) #65

New Issue
Closed
opened 2026-05-07 22:05:25 +00:00 by Ghost · 0 comments
Ghost commented 2026-05-07 22:05:25 +00:00
Copy Link

Phase 1 — Investigation findings

Symptom

Required check Auto-sync main → staging / sync-staging (push) has been failing on every push to main since the GitHub→Gitea migration on 2026-05-06. Last verified failing run: actions/runs/1117/jobs/0 (target SHA 1e1f4d63).

Root cause

The pre-suspension auto-sync-main-to-staging.yml was written for GitHub.com and assumed:

  1. A merge_queue ruleset on staging blocking direct push (GitHub-only feature).
  2. gh pr create and gh pr merge --auto working against the SCM API (GitHub GraphQL).
  3. The runner-issued secrets.GITHUB_TOKEN representing an integration that can land PRs through the queue.

On Gitea Actions / act_runner:

  1. Gitea has no merge_queue concept; staging branch protection on this repo is enable_push: true with push_whitelist_usernames: [devops-engineer]. Direct push from the persona is allowed.
  2. Gitea exposes no GraphQL endpoint. Every gh pr create call against https://git.moleculesai.app/api/graphql returns HTTP 405 Method Not Allowed.
  3. secrets.GITHUB_TOKEN is a per-job runner-issued Gitea-scoped token, not a github.com identity.

Concrete failure trace

From runs/1117/jobs/0 (logs at /api/v1/repos/molecule-ai/molecule-core/runs/1117/jobs/0/logs):

  • Step Checkout staging ✓ (line 11–95: clones, sets http extraheader, checks out staging)
  • Step Configure git author ✓
  • Step Check if staging already contains main ✓ → needs_sync=true
  • Step Create auto-sync branch + merge main ✓ (auto-sync/main-1e1f4d63, merge commit on top of staging)
  • Step Push auto-sync branch ✓ (line 277: * [new branch] auto-sync/main-1e1f4d63 -> auto-sync/main-1e1f4d63)
  • Step Open auto-sync PR + enable auto-merge ✗ — line 343: HTTP 405: 405 Method Not Allowed (https://git.moleculesai.app/api/graphql). gh pr create calls Gitea GraphQL → 405 → exit 1.

Result on every push to main: an orphan auto-sync/main-<sha> branch is created and the workflow fails red. No PR is ever opened, so nothing lands on staging via this path.

Why an existing fix-branch (fix/auto-sync-use-devops-token) is insufficient

The earlier sister-agent attempt swapped secrets.GITHUB_TOKEN → secrets.AUTO_SYNC_TOKEN in two places. Verified by diff against origin/main: those are the only changes. This does not address the root cause — the gh pr create command still hits /api/graphql regardless of the token, and Gitea still returns 405. Token-scope is not the bug.

Affected surfaces (audit per feedback_gitea_actions_migration_audit_pattern)

  1. Workflow YAML — auto-sync-main-to-staging.yml (this issue). Also auto-promote-staging.yml, retarget-main-to-staging.yml, and others use gh pr create / gh api against Gitea — out of scope for this issue, follow-ups parked below.
  2. Token + scope — AUTO_SYNC_TOKEN repo secret already exists (created 2026-05-07 14:00). Issued to devops-engineer persona. Whitelisted on staging branch protection. No rotation needed.
  3. Branch protection — staging already configured correctly (push_whitelist_usernames: [devops-engineer]). No change needed.
  4. Runner config — GITHUB_SERVER_URL env (per feedback_act_runner_github_server_url) is relevant for setup-go etc., NOT for this workflow. Out of scope here.
  5. Docs — only one stale reference in auto-promote-staging.yml comment block; updated as part of follow-up (NOT this PR).

Cleanup

Orphaned branch auto-sync/main-1e1f4d63 (created by the failed run) still exists on origin. Will be deleted once the new workflow lands a successful sync (the new workflow does not create per-SHA branches).

Parked follow-ups

  • HIGH: auto-promote-staging.yml calls gh pr create / gh workflow run against Gitea. Same class of failure. Separate issue.
  • MEDIUM: retarget-main-to-staging.yml uses gh api -X PATCH /pulls. Will fail when triggered. Separate issue.
  • LOW: ~30 other workflows have gh CLI calls; comprehensive audit per feedback_gitea_actions_migration_audit_pattern. Separate epic.

Fix

PR https://git.moleculesai.app/molecule-ai/molecule-core/pulls/: rewrite to use direct push from devops-engineer persona via AUTO_SYNC_TOKEN. No gh CLI dependency, no PR-through-queue dance. Header comment block fully documents the new architecture, identity model, and 4 failure modes (A–D) with operator runbooks.

Verification plan: this fix-PR's merge to main is itself the trigger; expect ≥2 consecutive green runs.

# Phase 1 — Investigation findings ## Symptom Required check `Auto-sync main → staging / sync-staging (push)` has been failing on every push to `main` since the GitHub→Gitea migration on 2026-05-06. Last verified failing run: `actions/runs/1117/jobs/0` (target SHA `1e1f4d63`). ## Root cause The pre-suspension `auto-sync-main-to-staging.yml` was written for GitHub.com and assumed: 1. A `merge_queue` ruleset on `staging` blocking direct push (GitHub-only feature). 2. `gh pr create` and `gh pr merge --auto` working against the SCM API (GitHub GraphQL). 3. The runner-issued `secrets.GITHUB_TOKEN` representing an integration that can land PRs through the queue. On Gitea Actions / `act_runner`: 1. Gitea has no `merge_queue` concept; staging branch protection on this repo is `enable_push: true` with `push_whitelist_usernames: [devops-engineer]`. Direct push from the persona is allowed. 2. Gitea exposes no GraphQL endpoint. Every `gh pr create` call against `https://git.moleculesai.app/api/graphql` returns `HTTP 405 Method Not Allowed`. 3. `secrets.GITHUB_TOKEN` is a per-job runner-issued Gitea-scoped token, not a github.com identity. ### Concrete failure trace From `runs/1117/jobs/0` (logs at `/api/v1/repos/molecule-ai/molecule-core/runs/1117/jobs/0/logs`): - Step `Checkout staging` ✓ (line 11–95: clones, sets http extraheader, checks out staging) - Step `Configure git author` ✓ - Step `Check if staging already contains main` ✓ → `needs_sync=true` - Step `Create auto-sync branch + merge main` ✓ (`auto-sync/main-1e1f4d63`, merge commit on top of staging) - Step `Push auto-sync branch` ✓ (line 277: `* [new branch] auto-sync/main-1e1f4d63 -> auto-sync/main-1e1f4d63`) - Step `Open auto-sync PR + enable auto-merge` ✗ — line 343: `HTTP 405: 405 Method Not Allowed (https://git.moleculesai.app/api/graphql)`. `gh pr create` calls Gitea GraphQL → 405 → exit 1. Result on every push to main: an orphan `auto-sync/main-<sha>` branch is created and the workflow fails red. No PR is ever opened, so nothing lands on staging via this path. ## Why an existing fix-branch (`fix/auto-sync-use-devops-token`) is insufficient The earlier sister-agent attempt swapped `secrets.GITHUB_TOKEN` → `secrets.AUTO_SYNC_TOKEN` in two places. Verified by diff against `origin/main`: those are the only changes. This does not address the root cause — the `gh pr create` command still hits `/api/graphql` regardless of the token, and Gitea still returns 405. Token-scope is not the bug. ## Affected surfaces (audit per `feedback_gitea_actions_migration_audit_pattern`) 1. Workflow YAML — `auto-sync-main-to-staging.yml` (this issue). Also `auto-promote-staging.yml`, `retarget-main-to-staging.yml`, and others use `gh pr create` / `gh api` against Gitea — out of scope for this issue, follow-ups parked below. 2. Token + scope — `AUTO_SYNC_TOKEN` repo secret already exists (created 2026-05-07 14:00). Issued to `devops-engineer` persona. Whitelisted on staging branch protection. No rotation needed. 3. Branch protection — staging already configured correctly (push_whitelist_usernames: [devops-engineer]). No change needed. 4. Runner config — `GITHUB_SERVER_URL` env (per `feedback_act_runner_github_server_url`) is relevant for setup-go etc., NOT for this workflow. Out of scope here. 5. Docs — only one stale reference in `auto-promote-staging.yml` comment block; updated as part of follow-up (NOT this PR). ## Cleanup Orphaned branch `auto-sync/main-1e1f4d63` (created by the failed run) still exists on origin. Will be deleted once the new workflow lands a successful sync (the new workflow does not create per-SHA branches). ## Parked follow-ups - **HIGH**: `auto-promote-staging.yml` calls `gh pr create` / `gh workflow run` against Gitea. Same class of failure. Separate issue. - **MEDIUM**: `retarget-main-to-staging.yml` uses `gh api -X PATCH /pulls`. Will fail when triggered. Separate issue. - **LOW**: ~30 other workflows have `gh` CLI calls; comprehensive audit per `feedback_gitea_actions_migration_audit_pattern`. Separate epic. ## Fix PR https://git.moleculesai.app/molecule-ai/molecule-core/pulls/<NEW>: rewrite to use direct push from `devops-engineer` persona via `AUTO_SYNC_TOKEN`. No `gh` CLI dependency, no PR-through-queue dance. Header comment block fully documents the new architecture, identity model, and 4 failure modes (A–D) with operator runbooks. Verification plan: this fix-PR's merge to main is itself the trigger; expect ≥2 consecutive green runs.
Ghost closed this issue 2026-05-07 22:07:01 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
tier:high

High risk per dev-sop §SOP-6 — ceo only, 24h cooldown

  
tier:low

Low risk per dev-sop §SOP-6 — engineers/managers/ceo can approve

  
tier:medium

Medium risk per dev-sop §SOP-6 — managers/ceo can approve

No Label
tier:high
tier:low
tier:medium
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: molecule-ai/molecule-core#65
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?