molecule-core/scripts/clone-manifest.sh
devops-engineer 8313b2a7a7
Some checks failed
Retarget main PRs to staging / Retarget to staging (pull_request) Has been skipped
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 5s
CI / Detect changes (pull_request) Successful in 7s
E2E API Smoke Test / detect-changes (pull_request) Successful in 9s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 9s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 11s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 11s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 14s
CI / Platform (Go) (pull_request) Successful in 11s
CI / Canvas (Next.js) (pull_request) Successful in 11s
CI / Python Lint & Test (pull_request) Successful in 9s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 10s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 12s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 8s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 17s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 12s
Ops Scripts Tests / Ops scripts (unittest) (pull_request) Successful in 40s
CodeQL / Analyze (${{ matrix.language }}) (go) (pull_request) Failing after 1m30s
CodeQL / Analyze (${{ matrix.language }}) (javascript-typescript) (pull_request) Failing after 1m32s
CodeQL / Analyze (${{ matrix.language }}) (python) (pull_request) Failing after 1m35s
fix(scripts): clone-manifest.sh — use Gitea + lowercase org slug
Post-2026-05-06 GitHub-org suspension: scripts/clone-manifest.sh
was still pointing at https://github.com/${repo}.git, so the
Docker build for workspace-server'\''s platform image fails at:

  fatal: could not read Username for 'https://github.com':
         No such device or address

with no credentials available in the build container.

Fix: clone from https://git.moleculesai.app/${repo}.git instead.
manifest.json'\''s repo paths still read 'Molecule-AI/...' (the
historic GitHub slug, mixed-case); Gitea lowercases the org
component to 'molecule-ai/...'. Lowercase the org segment on
the fly with awk so we don'\''t need to rewrite every manifest
entry.

Local verify: bash -n passes, lowercase transform produces correct
Gitea paths, anonymous git clone of one of the manifest plugins
over HTTPS to git.moleculesai.app succeeds.

Class G in the prod-ship CI sweep — same shape as the github.com
ref Harness Replays hits, this is the second instance found.
2026-05-07 08:17:58 -07:00

85 lines
3.0 KiB
Bash
Executable File

#!/bin/sh
# clone-manifest.sh — clone all repos listed in manifest.json into their
# target directories. Replaces hardcoded git-clone lines in Dockerfiles.
#
# Usage:
# ./scripts/clone-manifest.sh <manifest.json> <ws-templates-dir> <org-templates-dir> <plugins-dir>
#
# Requires: git, jq (lighter than python3 — ~2MB vs ~50MB in Alpine)
set -euo pipefail
MANIFEST="${1:?Usage: clone-manifest.sh <manifest.json> <ws-dir> <org-dir> <plugins-dir>}"
WS_DIR="${2:?Missing workspace-templates dir}"
ORG_DIR="${3:?Missing org-templates dir}"
PLUGINS_DIR="${4:?Missing plugins dir}"
EXPECTED=0
CLONED=0
clone_category() {
local category="$1"
local target_dir="$2"
mkdir -p "$target_dir"
local count
count=$(jq -r ".${category} | length" "$MANIFEST")
EXPECTED=$((EXPECTED + count))
local i=0
while [ "$i" -lt "$count" ]; do
local name repo ref
name=$(jq -r ".${category}[$i].name" "$MANIFEST")
repo=$(jq -r ".${category}[$i].repo" "$MANIFEST")
ref=$(jq -r ".${category}[$i].ref // \"main\"" "$MANIFEST")
# Idempotent: skip if the target already looks populated. Lets the
# README quickstart rerun setup.sh safely without having to delete
# already-cloned repos. A directory with any entries counts as
# populated; empty dirs reclone (may exist from a prior failed run).
if [ -d "$target_dir/$name" ] && [ -n "$(ls -A "$target_dir/$name" 2>/dev/null || true)" ]; then
echo " skipping $target_dir/$name (already populated)"
CLONED=$((CLONED + 1))
i=$((i + 1))
continue
fi
# Post-2026-05-06 GitHub-org-suspension: clone from Gitea instead.
# manifest.json paths still read "Molecule-AI/..." (the historic
# github.com slug); Gitea lowercases the org part to "molecule-ai/".
# Lowercase the org segment on the fly so we don't need to rewrite
# every manifest entry.
repo_gitea="$(echo "$repo" | awk -F/ '{ printf "%s", tolower($1); for (i=2; i<=NF; i++) printf "/%s", $i; print "" }')"
echo " cloning $repo_gitea -> $target_dir/$name (ref=$ref)"
if [ "$ref" = "main" ]; then
git clone --depth=1 -q "https://git.moleculesai.app/${repo_gitea}.git" "$target_dir/$name"
else
git clone --depth=1 -q --branch "$ref" "https://git.moleculesai.app/${repo_gitea}.git" "$target_dir/$name"
fi
CLONED=$((CLONED + 1))
i=$((i + 1))
done
# Strip .git dirs to save space
find "$target_dir" -name '.git' -type d -exec rm -rf {} + 2>/dev/null || true
}
echo "==> Cloning workspace templates..."
clone_category "workspace_templates" "$WS_DIR"
echo "==> Cloning org templates..."
clone_category "org_templates" "$ORG_DIR"
echo "==> Cloning plugins..."
clone_category "plugins" "$PLUGINS_DIR"
# Verify all repos were cloned
if [ "$CLONED" -ne "$EXPECTED" ]; then
echo "::error::Expected $EXPECTED repos but only cloned $CLONED — some clones failed"
exit 1
fi
echo "==> Done. $CLONED/$EXPECTED repos cloned successfully."