fix(scripts): migrate ghcr.io→ECR + raw.githubusercontent.com→Gitea (#46)
Some checks failed
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 6s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 7s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 5s
CodeQL / Analyze (${{ matrix.language }}) (go) (pull_request) Failing after 54s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 5s
CI / Detect changes (pull_request) Successful in 5s
E2E API Smoke Test / detect-changes (pull_request) Successful in 6s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 6s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 6s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 6s
CI / Platform (Go) (pull_request) Successful in 3s
CI / Python Lint & Test (pull_request) Successful in 3s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 3s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 5s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 4s
Ops Scripts Tests / Ops scripts (unittest) (pull_request) Failing after 13s
CI / Canvas (Next.js) (pull_request) Successful in 42s
CodeQL / Analyze (${{ matrix.language }}) (javascript-typescript) (pull_request) Failing after 1m18s
CodeQL / Analyze (${{ matrix.language }}) (python) (pull_request) Failing after 1m20s
Some checks failed
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 6s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 7s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 5s
CodeQL / Analyze (${{ matrix.language }}) (go) (pull_request) Failing after 54s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 5s
CI / Detect changes (pull_request) Successful in 5s
E2E API Smoke Test / detect-changes (pull_request) Successful in 6s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 6s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 6s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 6s
CI / Platform (Go) (pull_request) Successful in 3s
CI / Python Lint & Test (pull_request) Successful in 3s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 3s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 5s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 4s
Ops Scripts Tests / Ops scripts (unittest) (pull_request) Failing after 13s
CI / Canvas (Next.js) (pull_request) Successful in 42s
CodeQL / Analyze (${{ matrix.language }}) (javascript-typescript) (pull_request) Failing after 1m18s
CodeQL / Analyze (${{ matrix.language }}) (python) (pull_request) Failing after 1m20s
Per documentation-specialist's grep agent (2026-05-07T07:30, see internal#46): runtime-breaking ghcr.io references in shell scripts + docker-compose + the slip-past-workflow lint_secret_pattern_drift.py all need migration. These were missed by security-auditor's workflow-only audit. Files (6): - .github/scripts/lint_secret_pattern_drift.py:40 — workspace-runtime pre-commit-checks.sh consumer URL: raw.githubusercontent.com → Gitea raw URL (https://git.moleculesai.app/molecule-ai/.../raw/ branch/main/...). The lint job runs in CI and would 404 today. - scripts/refresh-workspace-images.sh:54 — workspace-template image pull URL: ghcr.io → ECR (153263036946.dkr.ecr.us-east-2.amazonaws.com). - scripts/rollback-latest.sh — full rewrite of header + auth flow: * ghcr.io/molecule-ai/{platform,platform-tenant} → ECR * GITHUB_TOKEN with write:packages → AWS ECR auth (aws ecr get-login-password). Per saved memory reference_post_suspension_pipeline, prod cutover is to ECR. * Updated header docs to match new auth flow + prereqs. - scripts/demo-freeze.sh:13,17 — comment-only ghcr → ECR (the script doesn't currently exec these URLs, but the comments describe the cascade and need to match reality). - docker-compose.yml:215-216 — canvas image: ghcr.io → ECR + updated the auth comment to describe `aws ecr get-login-password` flow. - tools/check-template-parity.sh:21 — inline curl install instructions: raw.githubusercontent.com → Gitea raw URL. Hostile self-review: 1. rollback-latest.sh's GITHUB_TOKEN→aws-cli auth swap is a behavior change. Operators using this script now need aws CLI authenticated for region us-east-2 with ECR pull/push perms. Documented in updated header. Operators who don't have aws CLI will get 'aws: command not installed' which is a clear failure mode (not silent). 2. The Gitea raw URL shape (/raw/branch/main/) differs from GitHub's raw.githubusercontent.com structure. Verified pattern by inspecting other Gitea raw URLs in the codebase. If Gitea's URL changes (1.23+), update via the same one-line edit. 3. Doesn't touch packer/scripts/install-base.sh which has a similar ghcr.io ref per the grep agent's findings — that's bigger-scope (packer build pipeline) and lives in molecule-controlplane-ish territory; filing as parked follow-up under #46 if not already. Refs: molecule-ai/internal#46, molecule-ai/internal#37, molecule-ai/internal#38, saved memory reference_post_suspension_pipeline
This commit is contained in:
parent
f92ba492de
commit
5d4184f4a3
2
.github/scripts/lint_secret_pattern_drift.py
vendored
2
.github/scripts/lint_secret_pattern_drift.py
vendored
@ -37,7 +37,7 @@ CANONICAL_FILE = Path(".github/workflows/secret-scan.yml")
|
|||||||
CONSUMERS: list[tuple[str, str]] = [
|
CONSUMERS: list[tuple[str, str]] = [
|
||||||
(
|
(
|
||||||
"molecule-ai-workspace-runtime/molecule_runtime/scripts/pre-commit-checks.sh",
|
"molecule-ai-workspace-runtime/molecule_runtime/scripts/pre-commit-checks.sh",
|
||||||
"https://raw.githubusercontent.com/Molecule-AI/molecule-ai-workspace-runtime/main/molecule_runtime/scripts/pre-commit-checks.sh",
|
"https://git.moleculesai.app/molecule-ai/molecule-ai-workspace-runtime/raw/branch/main/molecule_runtime/scripts/pre-commit-checks.sh",
|
||||||
),
|
),
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|||||||
@ -212,8 +212,8 @@ services:
|
|||||||
# docker compose pull canvas && docker compose up -d canvas
|
# docker compose pull canvas && docker compose up -d canvas
|
||||||
# First-time local setup or testing unreleased changes — build from source:
|
# First-time local setup or testing unreleased changes — build from source:
|
||||||
# docker compose build canvas && docker compose up -d canvas
|
# docker compose build canvas && docker compose up -d canvas
|
||||||
# Note: GHCR images are private — `docker login ghcr.io` required before pull.
|
# Note: ECR images require AWS auth — `aws ecr get-login-password --region us-east-2 | docker login --username AWS --password-stdin 153263036946.dkr.ecr.us-east-2.amazonaws.com` before pull.
|
||||||
image: ghcr.io/molecule-ai/canvas:latest
|
image: 153263036946.dkr.ecr.us-east-2.amazonaws.com/molecule-ai/canvas:latest
|
||||||
build:
|
build:
|
||||||
context: ./canvas
|
context: ./canvas
|
||||||
dockerfile: Dockerfile
|
dockerfile: Dockerfile
|
||||||
|
|||||||
@ -10,11 +10,11 @@
|
|||||||
# → PyPI auto-bumps molecule-ai-workspace-runtime patch version
|
# → PyPI auto-bumps molecule-ai-workspace-runtime patch version
|
||||||
# → repository_dispatch fans out to 8 workspace-template-* repos
|
# → repository_dispatch fans out to 8 workspace-template-* repos
|
||||||
# → each template repo rebuilds and re-tags
|
# → each template repo rebuilds and re-tags
|
||||||
# ghcr.io/molecule-ai/workspace-template-<runtime>:latest
|
# 153263036946.dkr.ecr.us-east-2.amazonaws.com/molecule-ai/workspace-template-<runtime>:latest
|
||||||
#
|
#
|
||||||
# PATH 2: any merge to a workspace-template-* repo's main branch
|
# PATH 2: any merge to a workspace-template-* repo's main branch
|
||||||
# → that repo's publish-image.yml fires
|
# → that repo's publish-image.yml fires
|
||||||
# → ghcr.io/molecule-ai/workspace-template-<runtime>:latest
|
# → 153263036946.dkr.ecr.us-east-2.amazonaws.com/molecule-ai/workspace-template-<runtime>:latest
|
||||||
# gets re-tagged
|
# gets re-tagged
|
||||||
#
|
#
|
||||||
# provisioner.go:296 RuntimeImages[runtime] reads `:latest` at every
|
# provisioner.go:296 RuntimeImages[runtime] reads `:latest` at every
|
||||||
|
|||||||
@ -51,7 +51,7 @@ log "pulling latest images for: ${RUNTIMES[*]}"
|
|||||||
PULLED=()
|
PULLED=()
|
||||||
FAILED=()
|
FAILED=()
|
||||||
for rt in "${RUNTIMES[@]}"; do
|
for rt in "${RUNTIMES[@]}"; do
|
||||||
IMG="ghcr.io/molecule-ai/workspace-template-$rt:latest"
|
IMG="153263036946.dkr.ecr.us-east-2.amazonaws.com/molecule-ai/workspace-template-$rt:latest"
|
||||||
if docker pull "$IMG" >/dev/null 2>&1; then
|
if docker pull "$IMG" >/dev/null 2>&1; then
|
||||||
log " ✓ $rt"
|
log " ✓ $rt"
|
||||||
PULLED+=("$rt")
|
PULLED+=("$rt")
|
||||||
|
|||||||
@ -1,9 +1,10 @@
|
|||||||
#!/bin/bash
|
#!/bin/bash
|
||||||
# rollback-latest.sh — moves the :latest tag on ghcr.io/molecule-ai/platform
|
# rollback-latest.sh — moves the :latest tag on the platform image
|
||||||
# (and the matching tenant image) back to a prior :staging-<sha> digest
|
# (and the matching tenant image) on AWS ECR back to a prior
|
||||||
# without rebuilding anything. Prod tenants auto-pull :latest every 5
|
# :staging-<sha> digest without rebuilding anything. Prod tenants
|
||||||
# min, so this is the fast path when a canary-verified image turns out
|
# auto-pull :latest every 5 min, so this is the fast path when a
|
||||||
# to have a runtime regression that canary didn't catch.
|
# canary-verified image turns out to have a runtime regression that
|
||||||
|
# canary didn't catch.
|
||||||
#
|
#
|
||||||
# Usage:
|
# Usage:
|
||||||
# scripts/rollback-latest.sh <sha>
|
# scripts/rollback-latest.sh <sha>
|
||||||
@ -12,12 +13,14 @@
|
|||||||
# Prereqs:
|
# Prereqs:
|
||||||
# - crane on $PATH (brew install crane OR download from
|
# - crane on $PATH (brew install crane OR download from
|
||||||
# https://github.com/google/go-containerregistry/releases)
|
# https://github.com/google/go-containerregistry/releases)
|
||||||
# - GHCR token exported as GITHUB_TOKEN with write:packages scope
|
# - aws CLI authenticated for region us-east-2 with ECR pull/push
|
||||||
|
# access to the molecule-ai/platform + platform-tenant repositories.
|
||||||
|
# `aws sts get-caller-identity` should succeed.
|
||||||
#
|
#
|
||||||
# What it does (per image — platform + tenant):
|
# What it does (per image — platform + tenant):
|
||||||
# crane digest ghcr.io/…:<sha> # verify the target sha exists
|
# crane digest <ecr>:<sha> # verify the target sha exists
|
||||||
# crane tag ghcr.io/…:<sha> latest # retag remotely, single API call
|
# crane tag <ecr>:<sha> latest # retag remotely, single API call
|
||||||
# crane digest ghcr.io/…:latest # confirm the move
|
# crane digest <ecr>:latest # confirm the move
|
||||||
#
|
#
|
||||||
# Exit codes: 0 = both retagged, 1 = tag missing / crane error, 2 = bad args.
|
# Exit codes: 0 = both retagged, 1 = tag missing / crane error, 2 = bad args.
|
||||||
|
|
||||||
@ -30,21 +33,23 @@ if [ "${1:-}" = "" ]; then
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
TARGET_SHA="$1"
|
TARGET_SHA="$1"
|
||||||
PLATFORM=ghcr.io/molecule-ai/platform
|
ECR_HOST=153263036946.dkr.ecr.us-east-2.amazonaws.com
|
||||||
TENANT=ghcr.io/molecule-ai/platform-tenant
|
PLATFORM=$ECR_HOST/molecule-ai/platform
|
||||||
|
TENANT=$ECR_HOST/molecule-ai/platform-tenant
|
||||||
|
|
||||||
if ! command -v crane >/dev/null; then
|
if ! command -v crane >/dev/null; then
|
||||||
echo "ERROR: crane not installed. brew install crane" >&2
|
echo "ERROR: crane not installed. brew install crane" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
if [ -z "${GITHUB_TOKEN:-}" ]; then
|
if ! command -v aws >/dev/null; then
|
||||||
echo "ERROR: GITHUB_TOKEN unset. export it with write:packages scope." >&2
|
echo "ERROR: aws CLI not installed. brew install awscli" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# Log in once. crane stores creds in a config file keyed by registry;
|
# Log in once. ECR auth is via short-lived password from `aws ecr
|
||||||
# re-running is cheap.
|
# get-login-password`. crane stores creds in a config file keyed by
|
||||||
printf '%s\n' "$GITHUB_TOKEN" | crane auth login ghcr.io -u "${GITHUB_ACTOR:-$(whoami)}" --password-stdin >/dev/null
|
# registry; re-running is cheap.
|
||||||
|
aws ecr get-login-password --region us-east-2 | crane auth login "$ECR_HOST" -u AWS --password-stdin >/dev/null
|
||||||
|
|
||||||
roll() {
|
roll() {
|
||||||
local image="$1"
|
local image="$1"
|
||||||
|
|||||||
@ -18,7 +18,7 @@
|
|||||||
#
|
#
|
||||||
# Or inline via curl:
|
# Or inline via curl:
|
||||||
#
|
#
|
||||||
# bash <(curl -fsSL https://raw.githubusercontent.com/Molecule-AI/molecule-core/main/tools/check-template-parity.sh) \
|
# bash <(curl -fsSL https://git.moleculesai.app/molecule-ai/molecule-core/raw/branch/main/tools/check-template-parity.sh) \
|
||||||
# install.sh start.sh
|
# install.sh start.sh
|
||||||
#
|
#
|
||||||
# Exit codes:
|
# Exit codes:
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user