From 638427e01be384e80f3358877963f19f47cbe2ac Mon Sep 17 00:00:00 2001 From: Molecule AI DevOps Engineer Date: Sat, 18 Apr 2026 03:03:37 +0000 Subject: [PATCH] docs(infra): document ANTHROPIC_API_KEY as required global secret (closes #894) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add comment to .env.example explaining ANTHROPIC_API_KEY must be set as a *global* secret (not just workspace-level) so SDK-direct workspaces (e.g. molecule-hitl, hermes) receive it without 401 errors - Add ANTHROPIC_API_KEY to saas-secrets.md secret map with context on why global propagation matters - Add full rotation procedure section (generate → PUT /settings/secrets → verify restart → revoke old key) with blast-radius note Closes #894 Co-Authored-By: Claude Sonnet 4.6 --- .env.example | 2 +- docs/runbooks/saas-secrets.md | 42 +++++++++++++++++++++++++++++++++++ 2 files changed, 43 insertions(+), 1 deletion(-) diff --git a/.env.example b/.env.example index 0eb60228..f342be41 100644 --- a/.env.example +++ b/.env.example @@ -70,7 +70,7 @@ NEXT_PUBLIC_PLATFORM_URL=http://localhost:8080 NEXT_PUBLIC_WS_URL=ws://localhost:8080/ws # Workspace Runtime -ANTHROPIC_API_KEY= +ANTHROPIC_API_KEY= # Anthropic API key (console.anthropic.com). Required for MODEL_PROVIDER=anthropic (default). Also used by workspaces that call the Anthropic SDK directly (e.g. molecule-hitl, hermes runtime). Register as a global secret via POST /settings/secrets so it is auto-propagated to all workspace containers — do NOT set only as a workspace-level secret or SDK-direct workspaces will silently fail with 401. See docs/runbooks/saas-secrets.md#anthropic_api_key. OPENROUTER_API_KEY= # OpenRouter API key (openrouter.ai). Use with model: openrouter:anthropic/claude-3.5-haiku. Also acts as the fallback key for the hermes runtime when HERMES_API_KEY is unset. HERMES_API_KEY= # Nous Research Portal API key (inference-prod.nousresearch.com). Used by the hermes runtime; falls back to OPENROUTER_API_KEY if unset. GROQ_API_KEY= # Groq API key (console.groq.com). Use with model: groq:llama-3.3-70b-versatile diff --git a/docs/runbooks/saas-secrets.md b/docs/runbooks/saas-secrets.md index 249b1120..ce3f7cd0 100644 --- a/docs/runbooks/saas-secrets.md +++ b/docs/runbooks/saas-secrets.md @@ -17,6 +17,7 @@ update doesn't silently break production. | `STRIPE_API_KEY` | `fly secrets` on `molecule-cp` | `sk_live_…` secret key used by `internal/billing.StripeProvider` for customer/subscription/checkout mutations + GDPR Art. 17 cascade | | `STRIPE_WEBHOOK_SECRET` | `fly secrets` on `molecule-cp` | `whsec_…` used by `internal/billing.verifySignature` to reject forged webhook calls. Rotated independently from the API key — Stripe treats them as separate secrets | | `GITHUB_TOKEN` | Built-in GitHub Actions token | GHCR push; rotated automatically | +| `ANTHROPIC_API_KEY` | **Global secret** via `PUT /settings/secrets` on each tenant platform instance | Default LLM provider (`MODEL_PROVIDER=anthropic`). Must be set as a **global** secret so it propagates to all workspace containers — workspace-level-only is not sufficient for SDK-direct workspaces (e.g. molecule-hitl). See [rotation procedure below](#anthropic_api_key). | ## Coupled secrets — MUST rotate together @@ -175,6 +176,47 @@ For `STRIPE_WEBHOOK_SECRET`: 5. Wait for the overlap window to expire or click "Delete old secret" in Stripe dashboard. +## Rotation procedure — ANTHROPIC_API_KEY + +This key is set as a **platform global secret** (not a Fly secret). It propagates +automatically to every non-paused workspace container via the Phase 15 global-secrets +fan-out (`PUT /settings/secrets` triggers auto-restart of all affected workspaces). + +Per-workspace overrides (e.g. a workspace with its own `ANTHROPIC_API_KEY` secret) +shadow the global value — the per-workspace value takes precedence. + +1. Generate a new key at [console.anthropic.com](https://console.anthropic.com) → + API Keys → Create key. Name it `molecule--rotation-$(date +%Y%m%d)`. + +2. Set the new key as a global secret on each platform instance: + ```bash + # Self-hosted (local/staging) + curl -X PUT http://localhost:8080/settings/secrets \ + -H "Authorization: Bearer $ADMIN_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{"key":"ANTHROPIC_API_KEY","value":"sk-ant-api03-..."}' + + # SaaS control plane — set on the tenant platform via control-plane API + # (details TBD when molecule-cp exposes a /cp/orgs/:id/secrets endpoint) + ``` + The platform auto-restarts every non-paused workspace on set. + +3. Verify: restart one workspace and confirm it starts up without 401 errors: + ```bash + curl -X POST http://localhost:8080/workspaces/$WORKSPACE_ID/restart \ + -H "Authorization: Bearer $ADMIN_TOKEN" + # Watch logs — no "401 unauthorized" from Anthropic SDK should appear + ``` + +4. Revoke the old key in the Anthropic console once all workspaces have restarted. + +### Blast-radius note + +Rotating `ANTHROPIC_API_KEY` restarts **every non-paused workspace** on the +instance. Schedule rotation during low-traffic windows. Paused workspaces pick +up the new key when they are next resumed (secrets are injected at container +start, not from the running container env). + ## Emergency contacts - **Fly**: billing dashboard at fly.io → Support