fix(ci): pre-clone manifest deps in workflow, drop in-image clone (closes #173)
Check merge_group trigger on required workflows / Required workflows have merge_group trigger (pull_request) Successful in 7s
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 7s
Retarget main PRs to staging / Retarget to staging (pull_request) Has been skipped
CI / Detect changes (pull_request) Successful in 9s
Lint curl status-code capture / Scan workflows for curl status-capture pollution (pull_request) Successful in 9s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 10s
E2E API Smoke Test / detect-changes (pull_request) Successful in 10s
Harness Replays / detect-changes (pull_request) Successful in 10s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 10s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 10s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 10s
CI / Python Lint & Test (pull_request) Successful in 6s
CI / Canvas (Next.js) (pull_request) Successful in 6s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 6s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 8s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 8s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 13s
Ops Scripts Tests / Ops scripts (unittest) (pull_request) Successful in 34s
Harness Replays / Harness Replays (pull_request) Failing after 33s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 53s
CodeQL / Analyze (${{ matrix.language }}) (go) (pull_request) Failing after 1m28s
CodeQL / Analyze (${{ matrix.language }}) (python) (pull_request) Failing after 1m29s
CodeQL / Analyze (${{ matrix.language }}) (javascript-typescript) (pull_request) Failing after 1m31s
CI / Platform (Go) (pull_request) Failing after 4m4s

publish-workspace-server-image.yml could not run on Gitea Actions because
Dockerfile.tenant's stage 3 ran `git clone` against private Gitea repos
from inside the Docker build context, where no auth path exists. Every
workspace-server rebuild required a manual operator-host push.

Move cloning to the trusted CI context (where AUTO_SYNC_TOKEN — the
devops-engineer persona PAT — is naturally available). Dockerfile.tenant
now COPYs from .tenant-bundle-deps/, populated by the workflow's new
"Pre-clone manifest deps" step. The Gitea token never enters the image.

- scripts/clone-manifest.sh: optional MOLECULE_GITEA_TOKEN env embeds
  basic-auth in the clone URL; redacted in log output. Anonymous fallback
  preserved for future public-repo path.
- .github/workflows/publish-workspace-server-image.yml: new pre-clone
  step before docker build; injects AUTO_SYNC_TOKEN. Fail-fast if the
  secret is empty.
- workspace-server/Dockerfile.tenant: drop stage 3 (templates), COPY
  from .tenant-bundle-deps/ instead. Header documents the prereq.
- .gitignore: ignore /.tenant-bundle-deps/ so a local build can't
  accidentally commit cloned repos.

Verified locally: clone-manifest.sh with the devops-engineer persona
token cloned all 37 repos (9 ws + 7 org + 21 plugins, 4.9MB after
.git strip).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
devops-engineer
2026-05-07 12:59:46 -07:00
parent d2da0c8d34
commit a6d67b4c68
4 changed files with 127 additions and 20 deletions
+39 -4
View File
@@ -6,6 +6,29 @@
# ./scripts/clone-manifest.sh <manifest.json> <ws-templates-dir> <org-templates-dir> <plugins-dir>
#
# Requires: git, jq (lighter than python3 — ~2MB vs ~50MB in Alpine)
#
# Auth (optional):
# When MOLECULE_GITEA_TOKEN is set, embed it as the basic-auth password so
# private Gitea repos clone successfully. When unset, clone anonymously
# (works only for repos that are public on git.moleculesai.app).
#
# This is the path the publish-workspace-server-image.yml workflow uses:
# it injects AUTO_SYNC_TOKEN (devops-engineer persona PAT, repo:read on
# the molecule-ai org) so the in-CI pre-clone step succeeds for ALL
# manifest entries — including the 5 private workspace-template-* repos
# (codex, crewai, deepagents, gemini-cli, langgraph) and all 7
# org-template-* repos.
#
# The token never enters the Docker image: this script runs in the
# trusted CI context BEFORE `docker buildx build`, populates
# .tenant-bundle-deps/, then `Dockerfile.tenant` COPYs from there with
# the .git directories already stripped (see line ~67 below).
#
# For backward compatibility — and so a fresh clone works without
# secrets when (eventually) the workspace-template-* repos flip public —
# the unset path remains a plain anonymous HTTPS clone. That path will
# FAIL with "could not read Username" on private repos today; CI MUST
# set MOLECULE_GITEA_TOKEN.
set -euo pipefail
@@ -52,11 +75,23 @@ clone_category() {
# every manifest entry.
repo_gitea="$(echo "$repo" | awk -F/ '{ printf "%s", tolower($1); for (i=2; i<=NF; i++) printf "/%s", $i; print "" }')"
echo " cloning $repo_gitea -> $target_dir/$name (ref=$ref)"
if [ "$ref" = "main" ]; then
git clone --depth=1 -q "https://git.moleculesai.app/${repo_gitea}.git" "$target_dir/$name"
# Build the clone URL. When MOLECULE_GITEA_TOKEN is set (CI path)
# embed it as basic-auth so private repos succeed. The username
# part ("oauth2") is conventional and ignored by Gitea — only the
# token-as-password is verified.
if [ -n "${MOLECULE_GITEA_TOKEN:-}" ]; then
clone_url="https://oauth2:${MOLECULE_GITEA_TOKEN}@git.moleculesai.app/${repo_gitea}.git"
display_url="https://oauth2:***@git.moleculesai.app/${repo_gitea}.git"
else
git clone --depth=1 -q --branch "$ref" "https://git.moleculesai.app/${repo_gitea}.git" "$target_dir/$name"
clone_url="https://git.moleculesai.app/${repo_gitea}.git"
display_url="$clone_url"
fi
echo " cloning $display_url -> $target_dir/$name (ref=$ref)"
if [ "$ref" = "main" ]; then
git clone --depth=1 -q "$clone_url" "$target_dir/$name"
else
git clone --depth=1 -q --branch "$ref" "$clone_url" "$target_dir/$name"
fi
CLONED=$((CLONED + 1))
i=$((i + 1))