feat(workspace-server): local-dev provisioner builds from Gitea source (#63, Task #194) #70

Merged
claude-ceo-assistant merged 3 commits from feat/issue-63-local-build-from-gitea-v2 into main 2026-05-07 23:37:57 +00:00

Closes #63 (Task #194).

Summary

When MOLECULE_IMAGE_REGISTRY is unset, the workspace-server provisioner now switches to local-build mode: clone the workspace-template repo from Gitea, docker build it locally, and use the SHA-pinned tag for ContainerCreate. OSS contributors no longer need authenticated GHCR/ECR access to provision a workspace.

Production tenants are unaffected — every prod tenant sets MOLECULE_IMAGE_REGISTRY to its private ECR mirror, so the SaaS pull path is byte-for-byte identical.

Why now

Post-2026-05-06 the Molecule-AI GitHub org was suspended; GHCR returns 403 for every molecule-ai/workspace-template-* manifest. OSS-default RegistryPrefix() = ghcr.io/molecule-ai is now broken. Reproduced 2026-05-07:

$ curl -H 'Authorization: Bearer <token>' -I \
    https://ghcr.io/v2/molecule-ai/workspace-template-claude-code/manifests/latest
HTTP/2 403

Design (full rationale in ADR-002)

  • SSOT for mode detection — new Resolve() returning RegistrySource{Mode, Prefix}. Discriminated value, not a bare string, so call sites that branch on mode get a compile-time push instead of a string-equality footgun.
  • Mode signal — presence/absence of MOLECULE_IMAGE_REGISTRY (Q2 design lock from Hongming, 2026-05-07). Set ⇒ SaaS pull (existing behavior). Unset ⇒ local-build path.
  • Cache key — Gitea HEAD sha → ~/.cache/molecule/workspace-template-build/<runtime>/<sha12>/. SHA-pinned image tag short-circuits clone+build on subsequent provisions.
  • Build platformlinux/amd64 (matches the provisioner's existing defaultImagePlatform()); honors feedback_local_must_mimic_production. 5–10 min first build on Apple Silicon via QEMU; cached afterwards.
  • Fail-closed — Gitea unreachable / runtime not mirrored → clear error mentioning the repo URL. Never silent-falls-back to GHCR/ECR.

Files

  • workspace-server/internal/provisioner/registry_mode.go — Resolve(), RegistryMode, IsKnownRuntime (new SSOT)
  • workspace-server/internal/provisioner/registry_mode_test.go — 8 tests pinning mode-decision contract
  • workspace-server/internal/provisioner/localbuild.go — clone+build pipeline (570 LOC, 22 unit tests)
  • workspace-server/internal/provisioner/localbuild_test.go — happy/sad paths, fail-closed contracts
  • workspace-server/internal/provisioner/provisioner.go — Start() inserts ensureLocalImageHook in local mode
  • docs/adr/ADR-002-local-build-mode-via-registry-presence.md — design rationale + alternatives + security review
  • docs/development/local-development.md — local-build flow + env overrides

Coverage

  • Resolve() mode detection — set / unset / empty / garbage URL → all four contracts pinned
  • EnsureLocalImage — happy path, cache hit, unknown runtime, Gitea unreachable, repo not found, auth failure, JSON-shaped HEAD response, build failure, missing Dockerfile, concurrent same-runtime, context cancellation, retag-after-cache-hit, HTTP body overflow, short-sha rejection, stale-cache cleanup
  • maskTokenInURL / maskTokenInString — token never echoed in logs (3 cases each)
  • giteaBranchAPIURL / parseGiteaBranchHeadSha — URL composer + JSON parser pinned against the Gitea API shape
  • All existing provisioner_test.go / registry_test.go cases preserved unchanged.

Security

  • Allowlist-only runtime namesIsKnownRuntime() gates the clone path. Defence-in-depth against future code paths that might let cfg.Runtime carry untrusted input.
  • Hardcoded repo prefixhttps://git.moleculesai.app/molecule-ai/molecule-ai-workspace-template- by default. Forks via opt-in MOLECULE_LOCAL_TEMPLATE_REPO_PREFIX.
  • Token maskingMOLECULE_GITEA_TOKEN, if set, is masked in every log line via maskTokenInURL / maskTokenInString.
  • No silent fallback — Gitea unreachable / runtime not mirrored → clear error, never falls back to GHCR/ECR.
  • No build-arg injectiondocker build invoked with no --build-arg from external input.
  • HTTP body cap — 64KB on Gitea API responses (defence vs malicious upstream).

Hostile self-review (3 weakest spots)

  1. dockerBuildProd shells out to docker rather than using the SDK. If the host has the daemon but no docker CLI on PATH (rare; mostly podman-only setups), the build fails with exec: docker not found. Mitigation: error message is clear; runbook lists docker CLI as prereq.
  2. HEAD-sha API endpoint hardcodes branch main. A fork with a master default branch would 404. Mitigation: error names the repo URL; future improvement could probe /branches for the default. Out of scope.
  3. Per-runtime build lock is package-global. Hot-reload mid-build (rare in dev) could split the lock state. Acceptable for dev; production never hits the local-build path.

Test plan

  • go test -race ./internal/provisioner/ — all 60+ tests pass (existing + new)
  • go test -race ./... — full workspace-server suite green
  • go vet ./... — clean
  • gofmt -l — clean
  • CI green (waiting on Gitea Actions)
  • E2E with real docker daemon (manual): MOLECULE_IMAGE_REGISTRY= + workspace-create → confirm clone + build + boot end-to-end

Followups

  • Template-repo doc updates — issue molecule-ai-workspace-template-claude-code#5 filed for the runbook + known-issues §5 on the Gitea repo (local sandbox copy points at the suspended GitHub origin per feedback_workspace_template_sandbox_is_stale).
  • Mirror crewai, deepagents, codex, gemini-cli, openclaw workspace-template repos to Gitea — currently 4 of 9 are mirrored; the others fail with an actionable error in local-build mode.

Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

Closes #63 (Task #194). ## Summary When `MOLECULE_IMAGE_REGISTRY` is unset, the workspace-server provisioner now switches to **local-build mode**: clone the workspace-template repo from Gitea, `docker build` it locally, and use the SHA-pinned tag for ContainerCreate. OSS contributors no longer need authenticated GHCR/ECR access to provision a workspace. **Production tenants are unaffected** — every prod tenant sets `MOLECULE_IMAGE_REGISTRY` to its private ECR mirror, so the SaaS pull path is byte-for-byte identical. ## Why now Post-2026-05-06 the `Molecule-AI` GitHub org was suspended; GHCR returns 403 for every `molecule-ai/workspace-template-*` manifest. OSS-default `RegistryPrefix() = ghcr.io/molecule-ai` is now broken. Reproduced 2026-05-07: ``` $ curl -H 'Authorization: Bearer <token>' -I \ https://ghcr.io/v2/molecule-ai/workspace-template-claude-code/manifests/latest HTTP/2 403 ``` ## Design (full rationale in ADR-002) * **SSOT for mode detection** — new `Resolve()` returning `RegistrySource{Mode, Prefix}`. Discriminated value, not a bare string, so call sites that branch on mode get a compile-time push instead of a string-equality footgun. * **Mode signal** — presence/absence of `MOLECULE_IMAGE_REGISTRY` (Q2 design lock from Hongming, 2026-05-07). Set ⇒ SaaS pull (existing behavior). Unset ⇒ local-build path. * **Cache key** — Gitea HEAD sha → `~/.cache/molecule/workspace-template-build/<runtime>/<sha12>/`. SHA-pinned image tag short-circuits clone+build on subsequent provisions. * **Build platform** — `linux/amd64` (matches the provisioner's existing `defaultImagePlatform()`); honors `feedback_local_must_mimic_production`. 5–10 min first build on Apple Silicon via QEMU; cached afterwards. * **Fail-closed** — Gitea unreachable / runtime not mirrored → clear error mentioning the repo URL. **Never** silent-falls-back to GHCR/ECR. ## Files * `workspace-server/internal/provisioner/registry_mode.go` — Resolve(), RegistryMode, IsKnownRuntime (new SSOT) * `workspace-server/internal/provisioner/registry_mode_test.go` — 8 tests pinning mode-decision contract * `workspace-server/internal/provisioner/localbuild.go` — clone+build pipeline (570 LOC, 22 unit tests) * `workspace-server/internal/provisioner/localbuild_test.go` — happy/sad paths, fail-closed contracts * `workspace-server/internal/provisioner/provisioner.go` — Start() inserts `ensureLocalImageHook` in local mode * `docs/adr/ADR-002-local-build-mode-via-registry-presence.md` — design rationale + alternatives + security review * `docs/development/local-development.md` — local-build flow + env overrides ## Coverage * `Resolve()` mode detection — set / unset / empty / garbage URL → all four contracts pinned * `EnsureLocalImage` — happy path, cache hit, unknown runtime, Gitea unreachable, repo not found, auth failure, JSON-shaped HEAD response, build failure, missing Dockerfile, concurrent same-runtime, context cancellation, retag-after-cache-hit, HTTP body overflow, short-sha rejection, stale-cache cleanup * `maskTokenInURL` / `maskTokenInString` — token never echoed in logs (3 cases each) * `giteaBranchAPIURL` / `parseGiteaBranchHeadSha` — URL composer + JSON parser pinned against the Gitea API shape * All existing `provisioner_test.go` / `registry_test.go` cases preserved unchanged. ## Security * **Allowlist-only runtime names** — `IsKnownRuntime()` gates the clone path. Defence-in-depth against future code paths that might let cfg.Runtime carry untrusted input. * **Hardcoded repo prefix** — `https://git.moleculesai.app/molecule-ai/molecule-ai-workspace-template-` by default. Forks via opt-in `MOLECULE_LOCAL_TEMPLATE_REPO_PREFIX`. * **Token masking** — `MOLECULE_GITEA_TOKEN`, if set, is masked in every log line via `maskTokenInURL` / `maskTokenInString`. * **No silent fallback** — Gitea unreachable / runtime not mirrored → clear error, never falls back to GHCR/ECR. * **No build-arg injection** — `docker build` invoked with no `--build-arg` from external input. * **HTTP body cap** — 64KB on Gitea API responses (defence vs malicious upstream). ## Hostile self-review (3 weakest spots) 1. `dockerBuildProd` shells out to `docker` rather than using the SDK. If the host has the daemon but no `docker` CLI on PATH (rare; mostly podman-only setups), the build fails with `exec: docker not found`. Mitigation: error message is clear; runbook lists docker CLI as prereq. 2. HEAD-sha API endpoint hardcodes branch `main`. A fork with a `master` default branch would 404. Mitigation: error names the repo URL; future improvement could probe `/branches` for the default. Out of scope. 3. Per-runtime build lock is package-global. Hot-reload mid-build (rare in dev) could split the lock state. Acceptable for dev; production never hits the local-build path. ## Test plan - [x] `go test -race ./internal/provisioner/` — all 60+ tests pass (existing + new) - [x] `go test -race ./...` — full workspace-server suite green - [x] `go vet ./...` — clean - [x] `gofmt -l` — clean - [ ] CI green (waiting on Gitea Actions) - [ ] E2E with real docker daemon (manual): `MOLECULE_IMAGE_REGISTRY=` + workspace-create → confirm clone + build + boot end-to-end ## Followups * Template-repo doc updates — issue molecule-ai-workspace-template-claude-code#5 filed for the runbook + known-issues §5 on the Gitea repo (local sandbox copy points at the suspended GitHub origin per `feedback_workspace_template_sandbox_is_stale`). * Mirror `crewai`, `deepagents`, `codex`, `gemini-cli`, `openclaw` workspace-template repos to Gitea — currently 4 of 9 are mirrored; the others fail with an actionable error in local-build mode. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
claude-ceo-assistant added 1 commit 2026-05-07 22:17:50 +00:00
feat(workspace-server): local-dev provisioner builds from Gitea source when MOLECULE_IMAGE_REGISTRY is unset (#63, Task #194)
Some checks failed
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 7s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 7s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 5s
CI / Detect changes (pull_request) Successful in 7s
CI / Canvas (Next.js) (pull_request) Successful in 7s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 1m38s
E2E API Smoke Test / detect-changes (pull_request) Successful in 7s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 7s
Harness Replays / detect-changes (pull_request) Successful in 7s
Harness Replays / Harness Replays (pull_request) Failing after 42s
CI / Platform (Go) (pull_request) Successful in 3m32s
CodeQL / Analyze (${{ matrix.language }}) (javascript-typescript) (pull_request) Successful in 1s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
CodeQL / Analyze (${{ matrix.language }}) (go) (pull_request) Successful in 1s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 6s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 7s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 7s
CI / Python Lint & Test (pull_request) Successful in 6s
CodeQL / Analyze (${{ matrix.language }}) (python) (pull_request) Successful in 1s
Retarget main PRs to staging / Retarget to staging (pull_request) Has been skipped
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 4s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 5s
d9e380c5bc
OSS contributors who clone molecule-core and `go run ./workspace-server/cmd/server`
now get a working end-to-end provision without authenticating to GHCR or AWS ECR.

Pre-fix: with MOLECULE_IMAGE_REGISTRY unset, the provisioner attempted to pull
ghcr.io/molecule-ai/workspace-template-<runtime>:latest, which has been
returning 403 since the 2026-05-06 GitHub-org suspension.

Post-fix: when MOLECULE_IMAGE_REGISTRY is unset, the provisioner switches to
local-build mode — looks up the workspace-template-<runtime> repo's HEAD sha
on Gitea via a single API call, shallow-clones into ~/.cache/molecule/, and
runs `docker build --platform=linux/amd64`. SHA-pinned cache key skips the
clone+build entirely on subsequent provisions.

Production tenants are unaffected: every prod tenant sets the var to its
private ECR mirror, so the SaaS pull path is byte-for-byte identical.

SSOT for mode detection lives in Resolve() (registry_mode.go) returning a
discriminated RegistrySource{Mode, Prefix} so call sites that branch on
mode get a compile-time push instead of a string-equality footgun.

Coverage:
* registry_mode.go            — new SSOT (Resolve, RegistryMode, IsKnownRuntime)
* registry_mode_test.go       — 8 tests pinning mode-decision contract
* localbuild.go               — clone+build pipeline (570 LOC, fully unit-tested)
* localbuild_test.go          — 22 tests covering happy/sad paths, fail-closed
* provisioner.go              — Start() inserts ensureLocalImageHook in local mode
* docs/adr/ADR-002            — design rationale + alternatives + security review
* docs/development/local-development.md — local-build flow + env overrides

Security:
* Allowlist-only runtime names (knownRuntimes) gate the clone path.
* Repo prefix hardcoded to git.moleculesai.app/molecule-ai/molecule-ai-workspace-template-;
  forks via opt-in MOLECULE_LOCAL_TEMPLATE_REPO_PREFIX.
* MOLECULE_GITEA_TOKEN masked in every log line via maskTokenInURL/maskTokenInString.
* Fail-closed: Gitea unreachable / runtime not mirrored → clear error, never
  silently fall back to GHCR/ECR.
* docker build invocation passes no --build-arg from external input.
* HTTP body cap 64KB on Gitea API responses (defence vs malicious upstream).

Closes #63 / Task #194.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ghost approved these changes 2026-05-07 22:52:13 +00:00
Ghost left a comment
First-time contributor

Local-dev provisioner SaaS/OSS split (#194). 1585 lines additive, 30 new tests + 64 existing preserved. Hongming-locked Option C: MOLECULE_IMAGE_REGISTRY presence as mode marker. ADR-002 captures rationale. Hostile-review weakest 3 filed as #204/#205/#206 follow-ups. Ready.

Local-dev provisioner SaaS/OSS split (#194). 1585 lines additive, 30 new tests + 64 existing preserved. Hongming-locked Option C: MOLECULE_IMAGE_REGISTRY presence as mode marker. ADR-002 captures rationale. Hostile-review weakest 3 filed as #204/#205/#206 follow-ups. Ready.
claude-ceo-assistant added 1 commit 2026-05-07 22:53:11 +00:00
Merge branch 'main' into feat/issue-63-local-build-from-gitea-v2
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (go) (pull_request) Successful in 5s
CodeQL / Analyze (${{ matrix.language }}) (javascript-typescript) (pull_request) Successful in 5s
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 12s
CodeQL / Analyze (${{ matrix.language }}) (python) (pull_request) Successful in 6s
pr-guards / disable-auto-merge-on-push (pull_request) Failing after 8s
CI / Detect changes (pull_request) Successful in 14s
E2E API Smoke Test / detect-changes (pull_request) Successful in 16s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 17s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 17s
Harness Replays / detect-changes (pull_request) Successful in 18s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 18s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 18s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 7s
CI / Canvas (Next.js) (pull_request) Successful in 8s
CI / Python Lint & Test (pull_request) Successful in 9s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 12s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 10s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 16s
Harness Replays / Harness Replays (pull_request) Failing after 1m16s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Failing after 1m23s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
CI / Platform (Go) (pull_request) Successful in 11m3s
141dfdae52
claude-ceo-assistant added 1 commit 2026-05-07 23:33:11 +00:00
Merge branch 'main' into feat/issue-63-local-build-from-gitea-v2
Some checks failed
CI / Canvas (Next.js) (pull_request) Successful in 10s
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 17s
CodeQL / Analyze (${{ matrix.language }}) (go) (pull_request) Successful in 11s
CI / Detect changes (pull_request) Successful in 19s
CodeQL / Analyze (${{ matrix.language }}) (python) (pull_request) Successful in 5s
CodeQL / Analyze (${{ matrix.language }}) (javascript-typescript) (pull_request) Successful in 6s
pr-guards / disable-auto-merge-on-push (pull_request) Failing after 8s
E2E API Smoke Test / detect-changes (pull_request) Successful in 16s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 16s
Harness Replays / detect-changes (pull_request) Successful in 16s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 18s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 19s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 19s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 10s
CI / Python Lint & Test (pull_request) Successful in 9s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 8s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 10s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 12s
Harness Replays / Harness Replays (pull_request) Failing after 1m6s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 1m25s
CI / Platform (Go) (pull_request) Successful in 3m19s
6bb272360d
claude-ceo-assistant merged commit d84d88ad70 into main 2026-05-07 23:37:57 +00:00
Sign in to join this conversation.
No reviewers
No Milestone
No project
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: molecule-ai/molecule-core#70
No description provided.