fix(quickstart): make README cp-paste flow bugless end-to-end (#1871)
Reproducing the README's quickstart on a clean clone surfaced seven independent bugs between `git clone` and seeing the Canvas in a browser. Each fix is minimal and local-dev-only — the SaaS/EC2 provisioner path (issue #1822) is untouched. Bugs fixed: 1. `infra/scripts/setup.sh` applied migrations via raw psql, bypassing the platform's `schema_migrations` tracker. The platform then re-ran every migration on first boot and crashed on non-idempotent ALTER TABLE statements (e.g. `036_org_api_tokens_org_id.up.sql`). Dropped the migration block — `workspace-server/internal/db/postgres.go:53` already tracks and skips applied files. 2. `.env.example` shipped `DATABASE_URL=postgres://USER:PASS@postgres:...` with literal `USER:PASS` placeholders and the Docker-internal hostname `postgres`. A `cp .env.example .env` followed by `go run ./cmd/server` on the host failed with `dial tcp: lookup postgres: no such host`. Replaced with working `dev:dev@localhost:5432` defaults that match `docker-compose.infra.yml`. 3. `docker-compose.infra.yml` and `docker-compose.yml` set `CLICKHOUSE_URL: clickhouse://...:9000/...`. Langfuse v2 rejects anything other than `http://` or `https://`, so the container crash-looped and returned HTTP 500. Switched to `http://...:8123` (HTTP interface) and added `CLICKHOUSE_MIGRATION_URL` for the migration-time native-protocol connection. Also removed `LANGFUSE_AUTO_CLICKHOUSE_MIGRATION_DISABLED` so migrations actually run. 4. `canvas/package.json` dev script crashed with `EADDRINUSE :::8080` when `.env` was sourced before `npm run dev` — Next.js reads `PORT` from env and the platform owns 8080. Pinned `dev` to `-p 3000` so sourced env can't hijack it. `start` left as-is because production `node server.js` (Dockerfile CMD) must respect `PORT` from the orchestrator. 5. README/CONTRIBUTING told users to clone `Molecule-AI/molecule-monorepo` — that repo 404s; the actual name is `molecule-core`. The Railway and Render deploy buttons had the same broken URL. Replaced in both English and Chinese READMEs and in CONTRIBUTING. Internal identifiers (Go module path, Docker network `molecule-monorepo-net`, Python helper `molecule-monorepo-status`) deliberately left alone — renaming those is an invasive refactor orthogonal to this fix. 6. README quickstart was missing `cp .env.example .env`. Users who went straight from `git clone` to `./infra/scripts/setup.sh` got a script that warned about an unset `ADMIN_TOKEN` (harmless) but then couldn't run the platform without figuring out the env setup on their own. Added the step in both READMEs and CONTRIBUTING. Deliberately NOT generating `ADMIN_TOKEN`/`SECRETS_ENCRYPTION_KEY` here — the e2e-api suite (`tests/e2e/test_api.sh`) assumes AdminAuth fallback mode (no server-side `ADMIN_TOKEN`), which is how CI runs it. 7. CI shellcheck only covered `tests/e2e/*.sh` — `infra/scripts/setup.sh` is in the critical path of every new-user onboarding but was never linted. Extended the `shellcheck` job and the `changes` filter to cover `infra/scripts/`. `scripts/` deliberately excluded until its pre-existing SC3040/SC3043 warnings are cleaned up separately. Verification (fresh nuke-and-rebuild following the updated README): - `docker compose -f docker-compose.infra.yml down -v` + `rm .env` - `cp .env.example .env` → defaults work as-is - `bash infra/scripts/setup.sh` — clean, no migration errors, all 6 infra containers healthy - `cd workspace-server && go run ./cmd/server` — "Applied 41 migrations (0 already applied)", platform on :8080/health 200 - `cd canvas && npm install && npm run dev` — Canvas on :3000/ 200 even with `.env` sourced (PORT=8080 in env) - `bash tests/e2e/test_api.sh` — **61 passed, 0 failed** - `cd canvas && npx vitest run` — **900 tests passed** - `cd canvas && npm run build` — production build clean - `shellcheck --severity=warning infra/scripts/*.sh` — clean - Langfuse `/api/public/health` 200 (was 500) Scope notes: - SaaS/EC2 parity (issue #1822): all files touched here are local-dev surface. Canvas container uses `node server.js` with `ENV PORT=3000` in `canvas/Dockerfile` — the `-p 3000` pin in `package.json` dev script only affects `npm run dev`, not the production CMD. - Test coverage (issue #1821): project policy is tiered coverage floors, not a blanket 100% target. Files touched here are shell scripts, YAML, Markdown, and one package.json script — not classes covered by the coverage matrix. - No overlap with open PRs — searched `setup.sh`, `quickstart`, `langfuse`, `clickhouse`, `migration`, `README`; nothing conflicts. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: molecule-ai[bot] <276602405+molecule-ai[bot]@users.noreply.github.com>
This commit is contained in:
Hongming Wang
GitHub
parent
6342449b68
commit
9ad803a802
20
.env.example
20
.env.example
@ -1,13 +1,23 @@
|
||||
# Postgres
|
||||
POSTGRES_USER=
|
||||
POSTGRES_PASSWORD=
|
||||
# These defaults match docker-compose.infra.yml, which is the stack
|
||||
# launched by `./infra/scripts/setup.sh`. Override for production.
|
||||
POSTGRES_USER=dev
|
||||
POSTGRES_PASSWORD=dev
|
||||
POSTGRES_DB=molecule
|
||||
DATABASE_URL=postgres://USER:PASS@postgres:5432/molecule?sslmode=disable
|
||||
# DATABASE_URL points at the host-published Postgres port so that
|
||||
# `go run ./cmd/server` on the host (the README quickstart path) can
|
||||
# connect. When running the platform *inside* docker-compose.yml, the
|
||||
# compose file builds a DATABASE_URL with host `postgres` automatically
|
||||
# from POSTGRES_USER/PASSWORD/DB above — that path ignores this value.
|
||||
DATABASE_URL=postgres://dev:dev@localhost:5432/molecule?sslmode=disable
|
||||
|
||||
# Redis
|
||||
REDIS_URL=redis://redis:6379
|
||||
# Redis — same host-vs-container story as DATABASE_URL above.
|
||||
REDIS_URL=redis://localhost:6379
|
||||
|
||||
# Platform
|
||||
# PORT only applies to the Go platform (workspace-server). The Canvas pins
|
||||
# itself to 3000 in canvas/package.json, so sourcing this file before
|
||||
# `npm run dev` won't accidentally make Next.js try to bind 8080.
|
||||
PORT=8080
|
||||
# ---- Admin credential — REQUIRED to close issue #684 (AdminAuth bearer bypass) ----
|
||||
# When ADMIN_TOKEN is set, only this value is accepted on /admin/* and /approvals/* routes.
|
||||
|
||||
10
.github/workflows/ci.yml
vendored
10
.github/workflows/ci.yml
vendored
@ -53,7 +53,7 @@ jobs:
|
||||
echo "platform=$(echo "$DIFF" | grep -qE '^workspace-server/|^\.github/workflows/ci\.yml$' && echo true || echo false)" >> "$GITHUB_OUTPUT"
|
||||
echo "canvas=$(echo "$DIFF" | grep -qE '^canvas/|^\.github/workflows/ci\.yml$' && echo true || echo false)" >> "$GITHUB_OUTPUT"
|
||||
echo "python=$(echo "$DIFF" | grep -qE '^workspace/|^\.github/workflows/ci\.yml$' && echo true || echo false)" >> "$GITHUB_OUTPUT"
|
||||
echo "scripts=$(echo "$DIFF" | grep -qE '^tests/e2e/|^scripts/|^\.github/workflows/ci\.yml$' && echo true || echo false)" >> "$GITHUB_OUTPUT"
|
||||
echo "scripts=$(echo "$DIFF" | grep -qE '^tests/e2e/|^scripts/|^infra/scripts/|^\.github/workflows/ci\.yml$' && echo true || echo false)" >> "$GITHUB_OUTPUT"
|
||||
|
||||
platform-build:
|
||||
name: Platform (Go)
|
||||
@ -207,10 +207,14 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Run shellcheck on tests/e2e/*.sh
|
||||
- name: Run shellcheck on tests/e2e/*.sh and infra/scripts/*.sh
|
||||
# shellcheck is pre-installed on ubuntu-latest runners (via apt).
|
||||
# infra/scripts/ is included because setup.sh + nuke.sh gate the
|
||||
# README quickstart — a shellcheck regression there silently breaks
|
||||
# new-user onboarding. scripts/ is intentionally excluded until its
|
||||
# pre-existing SC3040/SC3043 warnings are cleaned up.
|
||||
run: |
|
||||
find tests/e2e -type f -name '*.sh' -print0 \
|
||||
find tests/e2e infra/scripts -type f -name '*.sh' -print0 \
|
||||
| xargs -0 shellcheck --severity=warning
|
||||
|
||||
canvas-deploy-reminder:
|
||||
|
||||
@ -17,16 +17,19 @@ development workflow, conventions, and how to get your changes merged.
|
||||
|
||||
```bash
|
||||
# Clone the repo
|
||||
git clone https://github.com/Molecule-AI/molecule-monorepo.git
|
||||
cd molecule-monorepo
|
||||
git clone https://github.com/Molecule-AI/molecule-core.git
|
||||
cd molecule-core
|
||||
|
||||
# Install git hooks
|
||||
git config core.hooksPath .githooks
|
||||
|
||||
# Copy and edit .env (generate ADMIN_TOKEN + SECRETS_ENCRYPTION_KEY)
|
||||
cp .env.example .env
|
||||
|
||||
# Start infrastructure (Postgres, Redis, Langfuse, Temporal)
|
||||
./infra/scripts/setup.sh
|
||||
|
||||
# Build and run the platform
|
||||
# Build and run the platform — applies pending migrations on first boot
|
||||
cd workspace-server
|
||||
go run ./cmd/server
|
||||
|
||||
|
||||
14
README.md
14
README.md
@ -39,8 +39,8 @@
|
||||
<a href="./docs/agent-runtime/workspace-runtime.md"><strong>Workspace Runtime</strong></a>
|
||||
</p>
|
||||
|
||||
[](https://railway.app/new/template?template=https://github.com/Molecule-AI/molecule-monorepo)
|
||||
[](https://render.com/deploy?repo=https://github.com/Molecule-AI/molecule-monorepo)
|
||||
[](https://railway.app/new/template?template=https://github.com/Molecule-AI/molecule-core)
|
||||
[](https://render.com/deploy?repo=https://github.com/Molecule-AI/molecule-core)
|
||||
|
||||
</div>
|
||||
|
||||
@ -249,8 +249,12 @@ Workspace Runtime (Python image with adapters)
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
git clone https://github.com/Molecule-AI/molecule-monorepo.git
|
||||
cd molecule-monorepo
|
||||
git clone https://github.com/Molecule-AI/molecule-core.git
|
||||
cd molecule-core
|
||||
|
||||
cp .env.example .env
|
||||
# Defaults boot the stack locally out of the box. See .env.example for
|
||||
# production hardening knobs (ADMIN_TOKEN, SECRETS_ENCRYPTION_KEY, etc.).
|
||||
|
||||
./infra/scripts/setup.sh
|
||||
# Boots Postgres (:5432), Redis (:6379), Langfuse (:3001),
|
||||
@ -259,7 +263,7 @@ cd molecule-monorepo
|
||||
# no auth on localhost — dev-only; production must gate it.
|
||||
|
||||
cd workspace-server
|
||||
go run ./cmd/server
|
||||
go run ./cmd/server # applies pending migrations on first boot
|
||||
|
||||
cd ../canvas
|
||||
npm install
|
||||
|
||||
@ -38,8 +38,8 @@
|
||||
<a href="./docs/agent-runtime/workspace-runtime.md"><strong>Workspace Runtime</strong></a>
|
||||
</p>
|
||||
|
||||
[](https://railway.app/new/template?template=https://github.com/Molecule-AI/molecule-monorepo)
|
||||
[](https://render.com/deploy?repo=https://github.com/Molecule-AI/molecule-monorepo)
|
||||
[](https://railway.app/new/template?template=https://github.com/Molecule-AI/molecule-core)
|
||||
[](https://render.com/deploy?repo=https://github.com/Molecule-AI/molecule-core)
|
||||
|
||||
</div>
|
||||
|
||||
@ -248,8 +248,12 @@ Workspace Runtime (Python image with adapters)
|
||||
## 快速开始
|
||||
|
||||
```bash
|
||||
git clone https://github.com/Molecule-AI/molecule-monorepo.git
|
||||
cd molecule-monorepo
|
||||
git clone https://github.com/Molecule-AI/molecule-core.git
|
||||
cd molecule-core
|
||||
|
||||
cp .env.example .env
|
||||
# 默认值即可在本地启动整套服务。.env.example 里有针对生产部署的
|
||||
# 安全配置说明（ADMIN_TOKEN、SECRETS_ENCRYPTION_KEY 等）。
|
||||
|
||||
./infra/scripts/setup.sh
|
||||
# 启动 Postgres (:5432)、Redis (:6379)、Langfuse (:3001)
|
||||
@ -258,7 +262,7 @@ cd molecule-monorepo
|
||||
# 仅用于本地开发；生产环境必须加 mTLS / API Key。
|
||||
|
||||
cd workspace-server
|
||||
go run ./cmd/server
|
||||
go run ./cmd/server # 首次启动会自动跑 schema_migrations 里未应用的迁移
|
||||
|
||||
cd ../canvas
|
||||
npm install
|
||||
|
||||
@ -3,7 +3,7 @@
|
||||
"version": "0.1.0",
|
||||
"private": true,
|
||||
"scripts": {
|
||||
"dev": "next dev --turbopack",
|
||||
"dev": "next dev --turbopack -p 3000",
|
||||
"build": "next build",
|
||||
"start": "next start",
|
||||
"lint": "next lint",
|
||||
|
||||
@ -1,5 +1,3 @@
|
||||
version: "3.9"
|
||||
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16-alpine
|
||||
@ -106,10 +104,13 @@ services:
|
||||
condition: service_completed_successfully
|
||||
environment:
|
||||
DATABASE_URL: postgres://${POSTGRES_USER:-dev}:${POSTGRES_PASSWORD:-dev}@postgres:5432/langfuse
|
||||
CLICKHOUSE_URL: clickhouse://langfuse:${CLICKHOUSE_PASSWORD:-langfuse-dev}@clickhouse:9000/langfuse
|
||||
# Langfuse v2 expects the HTTP interface (port 8123). The previous
|
||||
# clickhouse://...:9000 native-protocol URL is rejected with
|
||||
# "ClickHouse URL protocol must be either http or https".
|
||||
CLICKHOUSE_URL: http://clickhouse:8123
|
||||
CLICKHOUSE_MIGRATION_URL: clickhouse://clickhouse:9000
|
||||
CLICKHOUSE_USER: langfuse
|
||||
CLICKHOUSE_PASSWORD: ${CLICKHOUSE_PASSWORD:-langfuse-dev}
|
||||
LANGFUSE_AUTO_CLICKHOUSE_MIGRATION_DISABLED: "true"
|
||||
NEXTAUTH_SECRET: ${LANGFUSE_SECRET:-changeme-langfuse-secret}
|
||||
NEXTAUTH_URL: http://localhost:3001
|
||||
SALT: ${LANGFUSE_SALT:-changeme-langfuse-salt}
|
||||
|
||||
@ -82,10 +82,13 @@ services:
|
||||
condition: service_completed_successfully
|
||||
environment:
|
||||
DATABASE_URL: postgres://${POSTGRES_USER:-dev}:${POSTGRES_PASSWORD:-dev}@postgres:5432/langfuse
|
||||
CLICKHOUSE_URL: clickhouse://langfuse:langfuse@langfuse-clickhouse:9000/langfuse
|
||||
# Langfuse v2 expects the HTTP interface (port 8123). The previous
|
||||
# clickhouse://...:9000 native-protocol URL is rejected with
|
||||
# "ClickHouse URL protocol must be either http or https".
|
||||
CLICKHOUSE_URL: http://langfuse-clickhouse:8123
|
||||
CLICKHOUSE_MIGRATION_URL: clickhouse://langfuse-clickhouse:9000
|
||||
CLICKHOUSE_USER: langfuse
|
||||
CLICKHOUSE_PASSWORD: langfuse
|
||||
LANGFUSE_AUTO_CLICKHOUSE_MIGRATION_DISABLED: "true"
|
||||
NEXTAUTH_SECRET: ${LANGFUSE_SECRET:-changeme-langfuse-secret}
|
||||
NEXTAUTH_URL: http://localhost:3001
|
||||
SALT: ${LANGFUSE_SALT:-changeme-langfuse-salt}
|
||||
|
||||
@ -26,23 +26,30 @@ echo "==> Verifying Redis KEA config..."
|
||||
KEA=$(docker compose -f "$ROOT_DIR/docker-compose.infra.yml" exec -T redis redis-cli config get notify-keyspace-events | tail -1)
|
||||
echo " notify-keyspace-events = $KEA"
|
||||
|
||||
echo "==> Running migrations..."
|
||||
MIGRATIONS_DIR="$ROOT_DIR/workspace-server/migrations"
|
||||
if [ -d "$MIGRATIONS_DIR" ]; then
|
||||
for f in "$MIGRATIONS_DIR"/*.sql; do
|
||||
echo " Applying $(basename "$f")..."
|
||||
docker compose -f "$ROOT_DIR/docker-compose.infra.yml" exec -T postgres \
|
||||
psql -U "${POSTGRES_USER:-dev}" -d "${POSTGRES_DB:-molecule}" -f - < "$f"
|
||||
done
|
||||
echo " Migrations complete."
|
||||
else
|
||||
echo " No migrations directory found, skipping."
|
||||
fi
|
||||
# Migrations are intentionally not applied here. The platform's own runner
|
||||
# (workspace-server/internal/db/postgres.go::RunMigrations) tracks applied
|
||||
# files in `schema_migrations` on every boot. Applying them out-of-band via
|
||||
# psql leaves that table empty, so the platform re-applies everything and
|
||||
# fails on non-idempotent ALTER TABLE statements. Let `go run ./cmd/server`
|
||||
# handle it.
|
||||
|
||||
echo "==> Infrastructure ready!"
|
||||
echo " Postgres: localhost:5432"
|
||||
echo " Redis: localhost:6379"
|
||||
echo " Langfuse: localhost:3001"
|
||||
echo " Temporal: localhost:7233 (gRPC) / localhost:8233 (UI)"
|
||||
echo ""
|
||||
echo " Next: cd workspace-server && go run ./cmd/server"
|
||||
echo " (the platform applies pending migrations on first boot)"
|
||||
|
||||
# Source .env if it exists so the ADMIN_TOKEN check below reflects what the
|
||||
# platform will actually see at startup, not just the current shell env.
|
||||
if [ -f "$ROOT_DIR/.env" ]; then
|
||||
set -a
|
||||
# shellcheck disable=SC1091
|
||||
. "$ROOT_DIR/.env"
|
||||
set +a
|
||||
fi
|
||||
|
||||
# Security check — issue #684 (AdminAuth bearer bypass, PR #729).
|
||||
# Without ADMIN_TOKEN, any valid workspace bearer token can call /admin/* routes.
|
||||
|
||||
Loading…
Reference in New Issue
Block a user