The Canvas template palette was empty on a fresh clone because
`workspace-configs-templates/`, `org-templates/`, and `plugins/` are
gitignored and nothing populated them. The registry already exists —
`manifest.json` at repo root lists every curated
`workspace-template-*`, `org-template-*`, and `plugin-*` repo, and
`scripts/clone-manifest.sh` clones them — but the step was absent
from the README and setup.sh, so new users never ran it.
### What this commit does
**1. `setup.sh` runs `clone-manifest.sh` automatically** (once).
After starting the Docker network but before booting infra, iterate
`manifest.json` and clone any workspace_templates / org_templates /
plugins that aren't already populated. Idempotent — subsequent
runs skip dirs that have content. Requires `jq`; when jq is missing
the step prints a clear install hint and skips (doesn't fail).
**2. `clone-manifest.sh` is idempotent.** Before running `git clone`,
check whether the target directory already exists and is non-empty —
skip if so. Lets `setup.sh` rerun safely without forcing the operator
to delete already-cloned template repos.
**3. `ListTemplates` logs the reason it skips a template.** The
handler previously swallowed `resolveYAMLIncludes` errors with
`continue`, so a broken template showed up as an empty palette with
no log trail. Now the include-expansion and yaml.Unmarshal failure
paths both emit a descriptive `log.Printf` — the exact message that
made the stale `org-templates/molecule-dev/` snapshot debuggable:
ListTemplates: skipping molecule-dev — !include expansion failed:
!include "core-platform.yaml" at line 25: open .../teams/
core-platform.yaml: no such file or directory
**4. Remove the in-tree `org-templates/molecule-dev/` snapshot** (170
files). Matches the explicit intent of prior commit
`bfec9e53` — "remove org-templates/molecule-dev/ — standalone repo
is source of truth". A later "full staging snapshot" re-added a
partial copy that had `!include` references to 7 role files that
never existed in the snapshot (`core-platform.yaml`,
`controlplane.yaml`, `app-docs.yaml`, `infra.yaml`, `sdk.yaml`,
`release-manager/workspace.yaml`, `integration-tester/workspace.yaml`).
`clone-manifest.sh` repopulates it fresh from
`Molecule-AI/molecule-ai-org-template-molecule-dev`.
.gitignore exception for `molecule-dev/` is dropped accordingly
— the whole `/org-templates/*` tree is now gitignored, symmetric
with `/plugins/` and `/workspace-configs-templates/`.
**5. Doc updates** (README, README.zh-CN, CONTRIBUTING) mention `jq`
as a prerequisite and describe what setup.sh now does.
### Verification
On a fresh-nuked DB with the updated branch:
1. `bash infra/scripts/setup.sh` — cleanly clones 33/33 manifest
repos (20 plugins, 8 workspace_templates, 5 org_templates), then
boots infra. Second run skips all 33 (idempotent).
2. `go run ./cmd/server` — "Applied 41 migrations", :8080 healthy.
3. `curl http://localhost:8080/org/templates` returns 4 templates
(was `[]`):
- Free Beats All
- MeDo Smoke Test
- Molecule AI Worker Team (Gemini)
- Reno Stars Agent Team
4. `bash tests/e2e/test_api.sh` — 61/61 pass.
5. `npx vitest run` in canvas — 902/902 pass.
6. `shellcheck infra/scripts/setup.sh` — clean.
### SaaS parity
All changes are local-dev surface. `setup.sh`, `clone-manifest.sh`,
and the local `org-templates/` directory aren't part of the CP
provisioner path — SaaS tenant machines get their templates via
Dockerfile layers or CP-side provisioning, not `clone-manifest.sh`.
The `ListTemplates` log addition is harmless either way (replaces a
silent `continue` with a `log.Printf + continue`).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reproducing the README's quickstart on a clean clone surfaced seven
independent bugs between `git clone` and seeing the Canvas in a browser.
Each fix is minimal and local-dev-only — the SaaS/EC2 provisioner path
(issue #1822) is untouched.
Bugs fixed:
1. `infra/scripts/setup.sh` applied migrations via raw psql, bypassing
the platform's `schema_migrations` tracker. The platform then re-ran
every migration on first boot and crashed on non-idempotent ALTER
TABLE statements (e.g. `036_org_api_tokens_org_id.up.sql`). Dropped
the migration block — `workspace-server/internal/db/postgres.go:53`
already tracks and skips applied files.
2. `.env.example` shipped `DATABASE_URL=postgres://USER:PASS@postgres:...`
with literal `USER:PASS` placeholders and the Docker-internal hostname
`postgres`. A `cp .env.example .env` followed by `go run ./cmd/server`
on the host failed with `dial tcp: lookup postgres: no such host`.
Replaced with working `dev:dev@localhost:5432` defaults that match
`docker-compose.infra.yml`.
3. `docker-compose.infra.yml` and `docker-compose.yml` set
`CLICKHOUSE_URL: clickhouse://...:9000/...`. Langfuse v2 rejects
anything other than `http://` or `https://`, so the container
crash-looped and returned HTTP 500. Switched to
`http://...:8123` (HTTP interface) and added `CLICKHOUSE_MIGRATION_URL`
for the migration-time native-protocol connection. Also removed
`LANGFUSE_AUTO_CLICKHOUSE_MIGRATION_DISABLED` so migrations actually
run.
4. `canvas/package.json` dev script crashed with `EADDRINUSE :::8080`
when `.env` was sourced before `npm run dev` — Next.js reads `PORT`
from env and the platform owns 8080. Pinned `dev` to
`-p 3000` so sourced env can't hijack it. `start` left as-is because
production `node server.js` (Dockerfile CMD) must respect `PORT`
from the orchestrator.
5. README/CONTRIBUTING told users to clone `Molecule-AI/molecule-monorepo`
— that repo 404s; the actual name is `molecule-core`. The Railway
and Render deploy buttons had the same broken URL. Replaced in both
English and Chinese READMEs and in CONTRIBUTING. Internal identifiers
(Go module path, Docker network `molecule-monorepo-net`, Python helper
`molecule-monorepo-status`) deliberately left alone — renaming those
is an invasive refactor orthogonal to this fix.
6. README quickstart was missing `cp .env.example .env`. Users who went
straight from `git clone` to `./infra/scripts/setup.sh` got a script
that warned about an unset `ADMIN_TOKEN` (harmless) but then couldn't
run the platform without figuring out the env setup on their own.
Added the step in both READMEs and CONTRIBUTING. Deliberately NOT
generating `ADMIN_TOKEN`/`SECRETS_ENCRYPTION_KEY` here — the e2e-api
suite (`tests/e2e/test_api.sh`) assumes AdminAuth fallback mode
(no server-side `ADMIN_TOKEN`), which is how CI runs it.
7. CI shellcheck only covered `tests/e2e/*.sh` — `infra/scripts/setup.sh`
is in the critical path of every new-user onboarding but was never
linted. Extended the `shellcheck` job and the `changes` filter to
cover `infra/scripts/`. `scripts/` deliberately excluded until its
pre-existing SC3040/SC3043 warnings are cleaned up separately.
Verification (fresh nuke-and-rebuild following the updated README):
- `docker compose -f docker-compose.infra.yml down -v` + `rm .env`
- `cp .env.example .env` → defaults work as-is
- `bash infra/scripts/setup.sh` — clean, no migration errors, all 6
infra containers healthy
- `cd workspace-server && go run ./cmd/server` — "Applied 41 migrations
(0 already applied)", platform on :8080/health 200
- `cd canvas && npm install && npm run dev` — Canvas on :3000/ 200
even with `.env` sourced (PORT=8080 in env)
- `bash tests/e2e/test_api.sh` — **61 passed, 0 failed**
- `cd canvas && npx vitest run` — **900 tests passed**
- `cd canvas && npm run build` — production build clean
- `shellcheck --severity=warning infra/scripts/*.sh` — clean
- Langfuse `/api/public/health` 200 (was 500)
Scope notes:
- SaaS/EC2 parity (issue #1822): all files touched here are local-dev
surface. Canvas container uses `node server.js` with `ENV PORT=3000`
in `canvas/Dockerfile` — the `-p 3000` pin in `package.json` dev
script only affects `npm run dev`, not the production CMD.
- Test coverage (issue #1821): project policy is tiered coverage floors,
not a blanket 100% target. Files touched here are shell scripts,
YAML, Markdown, and one package.json script — not classes covered
by the coverage matrix.
- No overlap with open PRs — searched `setup.sh`, `quickstart`,
`langfuse`, `clickhouse`, `migration`, `README`; nothing conflicts.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: molecule-ai[bot] <276602405+molecule-ai[bot]@users.noreply.github.com>
