docs: reframe secret encryption as KMS envelopes (with static-key fallback)
The platform's actual crypto model is two-mode envelope encryption (workspace-server/internal/crypto/envelope.go): - KMS mode (production): KMS_KEY_ARN selects an AWS KMS CMK; each Encrypt() calls GenerateDataKey for a fresh per-secret DEK, seals the payload with AES-256-GCM, stores the KMS-encrypted DEK + ciphertext together. CMK rotation is a no-op for existing blobs. - Static mode (dev / self-host): SECRETS_ENCRYPTION_KEY is a single long-lived 32-byte AES-256 key. Cannot rotate without a data migration. Both modes coexist during cutover (v2 prefix byte tags KMS blobs). The platform refuses to start with neither configured rather than silently storing plaintext. Previous docs framed this as "AES-256-GCM at the application layer" and named only SECRETS_ENCRYPTION_KEY, which under-described the production path and made the KMS migration invisible to readers. Files updated: - content/docs/architecture.mdx — env table adds KMS_KEY_ARN, clarifies SECRETS_ENCRYPTION_KEY as static-mode/self-host - content/docs/self-hosting.mdx — env table + Secrets Encryption section rewritten to cover both modes; cites envelope.go - content/docs/security/owasp-agentic-top-10.mdx — A02 control description now lists envelope encryption with KMS as production path - content/docs/development/constraints-and-rules.md — Rule 11 reframes storage model as envelope encryption (KMS prod, static dev) - content/docs/architecture/database-schema.md — workspace_secrets description updated to mention envelope encryption + v2 prefix + source file pointer - content/docs/architecture/molecule-technical-doc.md — five touchpoints (capability bullet, schema table, codebase tree, env table now includes KMS_KEY_ARN, recent-features global API keys row) No infra/runtime/Nemotron claims touched. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Hongming Wang
parent
aa46faeb78
commit
b625445357
@ -77,7 +77,8 @@ The Platform is the central control plane responsible for:
|
||||
| `REDIS_URL` | (required) | Redis connection string |
|
||||
| `PORT` | `8080` | Server listen port |
|
||||
| `PLATFORM_URL` | `http://host.docker.internal:PORT` | URL passed to agent containers |
|
||||
| `SECRETS_ENCRYPTION_KEY` | (optional) | AES-256 key, 32 bytes |
|
||||
| `SECRETS_ENCRYPTION_KEY` | (optional) | AES-256 key, 32 bytes — static-mode envelope encryption (dev/self-host). Set `KMS_KEY_ARN` instead for production AWS KMS envelopes. |
|
||||
| `KMS_KEY_ARN` | (optional) | AWS KMS CMK ARN — production envelope encryption with per-secret data keys |
|
||||
| `CORS_ORIGINS` | `http://localhost:3000,http://localhost:3001` | Allowed CORS origins |
|
||||
| `RATE_LIMIT` | `600` | Requests per minute |
|
||||
| `MOLECULE_ENV` | (optional) | Set `production` to hide test endpoints |
|
||||
|
||||
@ -80,7 +80,7 @@ CREATE TABLE workspace_secrets (
|
||||
);
|
||||
```
|
||||
|
||||
Stores API keys, credentials, and other secrets needed by workspace agents. Values are encrypted with AES-256 at the application layer. The encryption key comes from the `SECRETS_ENCRYPTION_KEY` environment variable on the platform — never stored in the database.
|
||||
Stores API keys, credentials, and other secrets needed by workspace agents. Values are sealed with envelope encryption — AWS KMS (per-secret data keys via `GenerateDataKey`, AES-256-GCM payload) when `KMS_KEY_ARN` is configured (production), or static-key AES-256-GCM under `SECRETS_ENCRYPTION_KEY` (dev / self-host). The static key, when used, is read from the platform environment and is never stored in the database. Both modes coexist during a KMS cutover, distinguished by a v2 prefix byte on KMS blobs. Implementation: `workspace-server/internal/crypto/envelope.go`.
|
||||
|
||||
The provisioner reads secrets from this table, decrypts them, and injects them as environment variables when spinning up workspace containers. Secrets are never included in bundles (see [Constraints — Rule 5](../development/constraints-and-rules.md)).
|
||||
|
||||
|
||||
@ -149,7 +149,7 @@ Six runtime adapters ship production-ready on `main`: LangGraph, DeepAgents, Cla
|
||||
- Event broadcasting (Redis pub/sub → WebSocket fanout)
|
||||
- Docker provisioner with T1–T4 tier enforcement
|
||||
- Activity logging with configurable retention (default 7 days)
|
||||
- Secrets management (AES-256-GCM encryption)
|
||||
- Secrets management (KMS-envelope encryption in prod; AES-256-GCM static-key mode for dev/self-host)
|
||||
- File, terminal, bundle, template, traces APIs
|
||||
- Langfuse integration
|
||||
- Prometheus metrics endpoint
|
||||
@ -186,7 +186,7 @@ Six runtime adapters ship production-ready on `main`: LangGraph, DeepAgents, Cla
|
||||
|-------|---------|-------------|
|
||||
| `workspaces` | Current state registry | `id`, `name`, `role`, `tier` (1-4), `status`, `parent_id`, `agent_card` (JSONB), `url`, `forwarded_to`, `last_heartbeat_at`, `last_error_rate`, `active_tasks`, `uptime_seconds`, `current_task`, `runtime` |
|
||||
| `agents` | Agent assignment history | `workspace_id`, `model`, `status`, `removed_at`, `removal_reason` |
|
||||
| `workspace_secrets` | Encrypted credentials | `workspace_id`, `key`, `encrypted_value` (BYTEA, AES-256-GCM) |
|
||||
| `workspace_secrets` | Encrypted credentials | `workspace_id`, `key`, `encrypted_value` (BYTEA — KMS envelope in prod, AES-256-GCM static-key blob in dev/self-host) |
|
||||
| `agent_memories` | HMA-scoped memory | `workspace_id`, `content`, `scope` (LOCAL/TEAM/GLOBAL) |
|
||||
| `structure_events` | **Immutable** event log (APPEND-ONLY, never UPDATE/DELETE) | `event_type`, `workspace_id`, `agent_id`, `target_id`, `payload` (JSONB) |
|
||||
| `activity_logs` | Operational activity with retention | `workspace_id`, `activity_type`, `source_id`, `target_id`, `method`, `request_body`, `response_body`, `duration_ms`, `status`, `error_detail` |
|
||||
@ -822,7 +822,7 @@ workspace-server/
|
||||
│ ├── events/ # 3 files — event broadcasting + Postgres persistence
|
||||
│ ├── router/ # 2 files — route definitions + middleware
|
||||
│ ├── db/ # 6 files — Postgres + Redis drivers, migrations
|
||||
│ └── crypto/ # 2 files — AES-256-GCM secrets encryption
|
||||
│ └── crypto/ # 2 files — envelope encryption (KMS or AES-256-GCM static key)
|
||||
└── migrations/ # 11 SQL migration files
|
||||
```
|
||||
|
||||
@ -905,7 +905,8 @@ Postgres + Redis + Langfuse only (for local development without containerized wo
|
||||
| `REDIS_URL` | `redis://localhost:6379` | Redis connection |
|
||||
| `PORT` | `8080` | Platform listen port |
|
||||
| `PLATFORM_URL` | `http://host.docker.internal:8080` | Injected to workspace containers |
|
||||
| `SECRETS_ENCRYPTION_KEY` | Optional | AES-256 key (32 bytes) for secret encryption |
|
||||
| `SECRETS_ENCRYPTION_KEY` | Optional | AES-256 key (32 bytes) for static-mode secret encryption — used when `KMS_KEY_ARN` is unset (dev/self-host) or to decrypt legacy blobs during a KMS cutover |
|
||||
| `KMS_KEY_ARN` | Optional | AWS KMS CMK ARN — when set, secrets use KMS envelope encryption (per-secret data keys via `GenerateDataKey`); production deployments use this path |
|
||||
| `CONFIGS_DIR` | `/configs` | Workspace config template directory |
|
||||
| `PLUGINS_DIR` | `/plugins` | Shared plugin directory |
|
||||
| `ACTIVITY_RETENTION_DAYS` | `7` | Activity log retention |
|
||||
@ -949,7 +950,7 @@ Postgres + Redis + Langfuse only (for local development without containerized wo
|
||||
|---------|-------------|
|
||||
| **A2A streaming response** | Real-time task result delivery via SSE (`message/sendSubscribe`) |
|
||||
| **Onboarding wizard** | 4-step guided first-run experience in Canvas |
|
||||
| **Global API keys** | Platform-wide secrets with per-workspace override + AES-256 encryption |
|
||||
| **Global API keys** | Platform-wide secrets with per-workspace override; KMS envelope encryption in prod (AES-256-GCM static-key mode in dev/self-host) |
|
||||
| **Coordinator enforcement** | Team leads cannot do work, only route and aggregate |
|
||||
| **Cascade pause/resume** | Pausing a parent cascades to all children; paused children can't be individually resumed |
|
||||
| **Graceful A2A errors** | `[A2A_ERROR]` sentinel + retry with exponential backoff + fallback |
|
||||
|
||||
@ -59,7 +59,7 @@ Direct A2A calls between workspaces are unauthenticated in MVP. Access control i
|
||||
|
||||
## 11. Secrets in Postgres, Encrypted
|
||||
|
||||
Workspace secrets (API keys, credentials) are stored in Postgres with AES-256 encryption at the application layer. The encryption key comes from the `SECRETS_ENCRYPTION_KEY` environment variable. Secrets are never included in bundles, never logged, never exposed via API responses.
|
||||
Workspace secrets (API keys, credentials) are stored in Postgres under envelope encryption. Production deployments use AWS KMS (`KMS_KEY_ARN`): each secret gets a fresh data key via `GenerateDataKey`, the payload is sealed with AES-256-GCM, and the KMS-encrypted DEK is stored alongside the ciphertext — rotating the CMK is a no-op for existing blobs. Dev and self-host deployments fall back to static-key AES-256-GCM under `SECRETS_ENCRYPTION_KEY`. Secrets are never included in bundles, never logged, never exposed via API responses.
|
||||
|
||||
## 12. Last-Write-Wins for MVP
|
||||
|
||||
|
||||
@ -59,9 +59,11 @@ documents — through tool calls, logs, or responses.
|
||||
|
||||
**Molecule AI controls:**
|
||||
|
||||
- **Encrypted secrets at rest:** Workspace secrets are encrypted with
|
||||
`SECRETS_ENCRYPTION_KEY` (AES-256) before storage. Plaintext never hits the
|
||||
database.
|
||||
- **Encrypted secrets at rest:** Workspace secrets are sealed with envelope
|
||||
encryption before storage — AWS KMS (per-secret data keys via `GenerateDataKey`,
|
||||
AES-256-GCM payload) when `KMS_KEY_ARN` is set, or static-key AES-256-GCM
|
||||
under `SECRETS_ENCRYPTION_KEY` for dev / self-host. Plaintext never hits the
|
||||
database, and the platform refuses to start with neither configured.
|
||||
- **Secrets scoped per-workspace:** A token scoped to workspace A cannot access
|
||||
workspace B's secrets.
|
||||
- **Memory access controls:** The MCP server's memory tools respect workspace
|
||||
|
||||
@ -98,7 +98,8 @@ docker compose up
|
||||
| `PORT` | `8080` | Platform HTTP port |
|
||||
| `PLATFORM_URL` | `http://host.docker.internal:PORT` | URL passed to agent containers to reach the platform |
|
||||
| `CORS_ORIGINS` | `http://localhost:3000,http://localhost:3001` | Comma-separated allowed origins |
|
||||
| `SECRETS_ENCRYPTION_KEY` | -- | AES-256 key (32 bytes) for encrypting workspace secrets |
|
||||
| `SECRETS_ENCRYPTION_KEY` | -- | AES-256 key (32 bytes) for static-mode envelope encryption of workspace secrets (dev/self-host path) |
|
||||
| `KMS_KEY_ARN` | -- | AWS KMS CMK ARN — when set, secrets use KMS envelope encryption (per-secret data keys); production SaaS deployments use this path |
|
||||
| `WORKSPACE_DIR` | -- | Global fallback host path for `/workspace` bind-mount |
|
||||
| `MOLECULE_ENV` | -- | Set to `production` to hide E2E helper endpoints |
|
||||
| `ACTIVITY_RETENTION_DAYS` | `7` | How long activity logs are retained |
|
||||
@ -154,7 +155,9 @@ This image serves both the API and the canvas frontend from a single container.
|
||||
|
||||
### Secrets Encryption
|
||||
|
||||
Set `SECRETS_ENCRYPTION_KEY` to a 32-byte AES-256 key to encrypt workspace secrets at rest. Without this variable, secrets are stored in plaintext.
|
||||
The platform supports two envelope-encryption modes for workspace secrets, picked at boot:
|
||||
|
||||
**Static mode (self-host / dev).** Set `SECRETS_ENCRYPTION_KEY` to a 32-byte AES-256 key. Each secret is sealed with AES-256-GCM under that single long-lived key. Without this variable (and without `KMS_KEY_ARN`), the platform refuses to start rather than silently storing plaintext.
|
||||
|
||||
```bash
|
||||
# Generate a key
|
||||
@ -163,6 +166,12 @@ openssl rand -hex 32
|
||||
|
||||
**Warning:** `SECRETS_ENCRYPTION_KEY` cannot be rotated without a data migration. Choose carefully before deploying to production.
|
||||
|
||||
**KMS mode (production SaaS).** Set `KMS_KEY_ARN` to an AWS KMS Customer Master Key ARN. Each `Encrypt()` call asks KMS for a fresh per-secret data encryption key (`GenerateDataKey`), seals the secret payload with AES-256-GCM under that DEK, and stores the KMS-encrypted DEK alongside the ciphertext (envelope encryption). Rotating the CMK is a no-op for existing blobs — KMS tracks key versions internally.
|
||||
|
||||
The two modes coexist during cutover: a v2 prefix byte tags KMS blobs; older static-mode blobs decrypt with `SECRETS_ENCRYPTION_KEY` until they're next written. Operators migrating to KMS can leave both env vars set during the transition.
|
||||
|
||||
Implementation: `workspace-server/internal/crypto/envelope.go`.
|
||||
|
||||
### Rate Limiting
|
||||
|
||||
The `RATE_LIMIT` variable (default 600 requests/min) applies per client. Adjust based on your expected traffic.
|
||||
|
||||
Loading…
Reference in New Issue
Block a user