CPProvisioner env mutator error branch was left with unresolved conflict
markers after a prior rebase. Resolved to the HEAD-side generic message
"plugin env mutator chain failed" which is consistent with the same
message used in the Provisioner path (line 107/111).
No functional change.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The post-fire UPDATE after s.proxy.ProxyA2ARequest() was using fireCtx,
which derives from the outer ctx passed into fireSchedule(). If that ctx
is cancelled — HTTP timeout, graceful shutdown, or any upstream deadline —
ExecContext returns context.Canceled and the UPDATE is silently skipped,
leaving next_run_at stale and causing the schedule to re-fire on the
next tick.
Fix: create a dedicated updateCtx from context.Background() with a 5s
deadline, independent of the outer ctx hierarchy. Also improved the
error log to include schedule name for easier debugging.
Complements PR #1241 (fix/f1089-scheduler-ctx-fix-main) which fixes
the goroutine-panic path in tick() — this fix covers the wider case of
normal-return + ctx-cancelled after the proxy call.
F1089 | Severity: HIGH+security
Co-authored-by: Molecule AI Infra Lead <infra-lead@agents.moleculesai.app>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(security): call redactSecrets before seeding workspace memories (F1085)
seedInitialMemories() in workspace_provision.go was inserting template/config
memories directly into agent_memories without scrubbing credential patterns.
A workspace provisioned from a template containing API keys, tokens, or other
secrets would store them in plain text — the same class of issue as #838.
Fix: call redactSecrets(workspaceID, content) on the truncated memory content
before the INSERT. The truncation (maxMemoryContentLength = 100 KiB, CWE-400)
is preserved — redaction runs after truncation so the size limit still applies.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* test(workspace_provision): add seedInitialMemories coverage for #1208
Cover the truncate-at-100k boundary (PR #1167, CWE-400) and the
redactSecrets call (F1085 / #1132), both identified as untested in #1208.
- TestSeedInitialMemories_TruncatesOversizedContent: boundary at exactly
100k, 1 byte over, far over, and well under. Verifies INSERT receives
exactly maxMemoryContentLength bytes.
- TestSeedInitialMemories_RedactsSecrets: verifies redactSecrets runs
before INSERT, regression test for F1085.
- TestSeedInitialMemories_InvalidScopeSkipped: invalid scope is silently
skipped, no INSERT called.
- TestSeedInitialMemories_EmptyMemoriesNil: nil slice is handled without
DB calls.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs(marketing): Discord adapter launch visual assets (#1209)
Squash-merge: Discord adapter launch visual assets (3 PNGs) + social copy. Acceptance: assets on staging.
* fix(ci): golangci-lint errcheck failures on staging
Suppress errcheck warnings for calls where the return value is safely
ignored:
- resp.Body.Close() (artifacts/client.go): deferred cleanup — failure
to close a response body is non-critical; the defer itself is what
matters for connection reuse.
- rows.Close() (bundle/exporter.go): deferred cleanup in a loop where
rows.Err() already handles query errors.
- filepath.Walk (bundle/exporter.go): top-level walk call; errors in
sub-directory traversal are handled by the inner callback (which
returns nil for err != nil).
- broadcaster.RecordAndBroadcast (bundle/importer.go): fire-and-forget
event broadcast; errors are logged internally by the broadcaster.
- db.DB.ExecContext (bundle/importer.go): best-effort runtime column
update; non-critical auxiliary data that the provisioner re-extracts
if needed.
Fixes: #1143
* test(artifacts): suppress w.Write return values to satisfy errcheck
All httptest.ResponseWriter.Write calls in client_test.go now discard
the byte count and error return with _, _ = prefix. The Write method
is safe to discard in test handlers — httptest.ResponseWriter.Write
never returns an error for in-memory buffers.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(CI): move changes job off self-hosted runner + add workflow concurrency
Cherry-pick from staging PR #1194 for main. Two changes to relieve
macOS arm64 runner saturation:
1. `changes` job: runs on ubuntu-latest instead of
[self-hosted, macos, arm64]. This job does a plain `git diff`
with zero macOS dependencies — moving it off the runner frees
a slot immediately on every workflow trigger.
2. Add workflow-level concurrency:
concurrency: group: ci-${{ github.ref }}; cancel-in-progress: true
Prevents multiple stale in-flight CI runs from queuing on the
same ref when new commits arrive.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(security): call redactSecrets before seeding workspace memories (F1085) (#1203)
seedInitialMemories() in workspace_provision.go was inserting template/config
memories directly into agent_memories without scrubbing credential patterns.
A workspace provisioned from a template containing API keys, tokens, or other
secrets would store them in plain text — the same class of issue as #838.
Fix: call redactSecrets(workspaceID, content) on the truncated memory content
before the INSERT. The truncation (maxMemoryContentLength = 100 KiB, CWE-400)
is preserved — redaction runs after truncation so the size limit still applies.
Co-authored-by: Molecule AI Core-BE <core-be@agents.moleculesai.app>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* tick: 2026-04-21 ~03:40Z — CI stalled 59+ min, GH_TOKEN 4th rotation, PR reviews done
* fix(tenant-guard): allowlist /registry/register + /registry/heartbeat
Final layer of today's stuck-provisioning saga. With the private-IP
platform_url fix and the intra-VPC :8080 SG rule in place, workspace
EC2s finally reached the tenant on the right port — only to have every
POST bounced with a synthetic 404 by TenantGuard.
TenantGuard is the SaaS hook that rejects cross-tenant routing. It
demands X-Molecule-Org-Id on every request, but CP's workspace user-
data doesn't export MOLECULE_ORG_ID (only WORKSPACE_ID, PLATFORM_URL,
RUNTIME, PORT), so the runtime can't attach the header. Net effect:
every workspace's first heartbeat to /registry/heartbeat was a silent
404, and the workspace sat in 'provisioning' until the platform
sweeper timed it out.
Allowlist the two workspace-boot paths:
- /registry/register — one-shot at runtime startup
- /registry/heartbeat — every 30s
Both are still gated by wsauth.HasAnyLiveToken (workspaces with a
token on file must present it; legacy tokenless workspaces are
grandfathered). And the tenant SG already scopes :8080 to the VPC
CIDR, so only intra-VPC callers can reach these paths in the first
place. The allowlist bypasses cross-org routing, not auth.
Follow-up: passing MOLECULE_ORG_ID into the workspace env would let
the runtime attach the header and drop this allowlist entry. Tracked
separately; not urgent since the multi-layer auth above is already
adequate.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Molecule AI Core-BE <core-be@agents.moleculesai.app>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Molecule AI Infra-SRE <infra-sre@agents.moleculesai.app>
Co-authored-by: molecule-ai[bot] <276602405+molecule-ai[bot]@users.noreply.github.com>
Co-authored-by: Molecule AI Core-DevOps <core-devops@agents.moleculesai.app>
Co-authored-by: Molecule AI Core-UIUX <core-uiux@agents.moleculesai.app>
Co-authored-by: Hongming Wang <hongmingwang.rabbit@users.noreply.github.com>
Issue #1196: golangci-lint errcheck flags bare resp.Body.Close()
calls because Body.Close() can return a non-nil error (e.g. when the
server sent fewer bytes than Content-Length). All occurrences fixed:
defer resp.Body.Close() → defer func() { _ = resp.Body.Close() }()
resp.Body.Close() → _ = resp.Body.Close()
12 files affected across all Go packages — channels, handlers,
middleware, provisioner, artifacts, and cmd. The body is already fully
consumed at each call site, so the error is always safe to discard.
🤖 Generated with [Claude Code](https://claude.ai)
Co-authored-by: Molecule AI Core-BE <core-be@agents.moleculesai.app>
PR #1210 added org_api_tokens.org_id but c.Set("org_id", ...) was never
called — so orgCallerID() always returns "" and all token callers are
denied org-scoped access even within their own org.
Fix: after orgtoken.Validate succeeds in AdminAuth, look up the token's
org_id column and set it in the gin context. Pre-fix tokens (org_id=NULL)
get no org_id in context, which is correct — requireCallerOwnsOrg already
denies access for nil org_id.
Test: TestAdminAuth_OrgToken_SetsOrgID covers both post-fix tokens
(org_id set) and pre-fix tokens (org_id=NULL, not set).
Co-authored-by: Molecule AI Infra-SRE <infra-sre@agents.moleculesai.app>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(auth): F1094 — requireCallerOwnsOrg reads org_id not created_by (#1200)
Root cause: requireCallerOwnsOrg (org_plugin_allowlist.go:116) was
reading org_api_tokens.created_by to determine caller's org workspace
ID. But created_by is a provenance label ("session", "admin-token",
"org-token:<prefix>") — never a UUID. The equality check
callerOrg != targetOrgID always failed → every org-token caller
got 403 on /orgs/:id/plugins/allowlist routes.
Fix:
- Migration 036: adds org_id UUID column (nullable) to org_api_tokens
with index. Existing pre-migration tokens get org_id=NULL → deny
by default (safer than cross-org access).
- orgtoken.Issue: takes new orgID param; stores in org_id column.
- orgtoken.OrgIDByTokenID: new helper reads org_id for a token ID.
Returns ("", nil) for NULL/unanchored tokens.
- requireCallerOwnsOrg: now calls OrgIDByTokenID instead of reading
created_by. Pre-migration tokens with org_id=NULL get callerOrg=""
→ denied (safer).
- orgTokenActor (org_tokens.go): returns (createdBy, orgID) pair.
Token minted via another org token gets its org_id set at mint time.
Session/ADMIN_TOKEN callers get orgID="".
- orgtoken.Token struct: adds OrgID field for list display.
- orgtoken.List: selects org_id alongside other columns.
- Updated existing tests for new Issue signature.
- Added 10 regression tests covering: happy path, unanchored denial,
cross-org denial, session bypass, DB error denial.
🤖 Generated with [Claude Code](https://claude.ai/claude-code)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(security): replace err.Error() leaks with prod-safe messages (#1206)
- workspace_provision.go: provisionWorkspace, provisionWorkspaceCP —
replaced 7 err.Error() calls with "provisioning failed" in both
Broadcast payloads and last_sample_error DB column. Full error
preserved in server-side log.Printf.
- plugins_install_pipeline.go: resolveAndStage — replaced 5 err.Error()
calls with generic messages:
"invalid plugin source"
"plugin source not supported"
"invalid plugin name"
"staged plugin exceeds size limit"
"plugin manifest integrity check failed"
Risk mitigated: DB errors (pq: connection refused, pq: deadlock),
OS errors, and internal paths no longer leak in HTTP JSON responses
or WebSocket broadcasts.
Added regression tests (workspace_provision_test.go):
- TestProvisionWorkspace_NoInternalErrorsInBroadcast
- TestProvisionWorkspaceCP_NoInternalErrorsInBroadcast
- TestResolveAndStage_NoInternalErrorsInHTTPErr
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(F1089): log panic-recovery UPDATE errors in scheduler
The panic defer blocks in tick() and fireSchedule() now capture
and log errors from the db.DB.ExecContext call that advances next_run_at
after a panic. Previously, a DB failure during panic recovery was
silent — the log line for the panic itself appeared but any subsequent
UPDATE failure was invisible, risking unnoticed scheduler drift.
context.Background() was already used (F1089 comment in place); this
commit adds the missing error capture + log.Printf on exec failure.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Molecule AI Dev Lead <dev-lead@agents.moleculesai.app>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Multiple security findings addressed:
F1095 (BootstrapFailed): Replace err.Error() in ShouldBindJSON failure
response with generic "invalid request body" — raw gin binding errors
can expose validation detail, field names, and type mismatch info.
F1096 (BootstrapFailed): Handle RowsAffected() error instead of ignoring
it — the DB call can fail in ways the current code silently ignores.
#1206 (provision/plugin install): Replace raw err.Error() in API responses,
broadcasts, and last_sample_error DB fields across workspace_provision.go
(7 occurrences) and plugins_install_pipeline.go (6 occurrences). Replaced
with context-appropriate generic messages that don't leak internal DB
file paths, decrypt error details, or resolver internals to callers.
#1208 (test-gap): Add 3 new seedInitialMemories truncate tests:
- Exactly-at-limit (100k bytes → unchanged, boundary case)
- Empty content (skipped, no DB call)
- Oversized with embedded secrets (truncation fires before any other content inspection)
Co-authored-by: Molecule AI Fullstack (floater) <fullstack-floater@agents.moleculesai.app>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Root cause: requireCallerOwnsOrg (org_plugin_allowlist.go:116) was
reading org_api_tokens.created_by to determine caller's org workspace
ID. But created_by is a provenance label ("session", "admin-token",
"org-token:<prefix>") — never a UUID. The equality check
callerOrg != targetOrgID always failed → every org-token caller
got 403 on /orgs/:id/plugins/allowlist routes.
Fix:
- Migration 036: adds org_id UUID column (nullable) to org_api_tokens
with partial index for fast lookups. Existing pre-migration tokens
get org_id=NULL → deny by default (safer than cross-org access).
- orgtoken.Issue: takes new orgID param; stores in org_id column.
- orgtoken.OrgIDByTokenID: new helper reads org_id for a token ID.
Returns ("", nil) for NULL/unanchored tokens.
- requireCallerOwnsOrg: now calls OrgIDByTokenID instead of reading
created_by. Pre-migration tokens with org_id=NULL get callerOrg=""
→ denied (safer).
- orgTokenActor (org_tokens.go): returns (createdBy, orgID) pair.
Token minted via another org token gets its org_id set at mint time.
Session/ADMIN_TOKEN callers get orgID="".
- orgtoken.Token struct: adds OrgID field for list display.
- orgtoken.List: selects org_id alongside other columns.
- Updated existing tests for new Issue signature.
- Added regression tests: happy path, unanchored denial, DB error denial.
Co-authored-by: Molecule AI Infra-Runtime-BE <infra-runtime-be@agents.moleculesai.app>
Co-authored-by: Molecule AI Dev Lead <dev-lead@agents.moleculesai.app>
* feat(canvas): rewrite MemoryInspectorPanel to match backend API
Issue #909 (chunk 3 of #576).
The existing MemoryInspectorPanel used the wrong API endpoint
(/memory instead of /memories) and wrong field names (key/value/version
instead of id/content/scope/namespace/created_at). It also lacked
LOCAL/TEAM/GLOBAL scope tabs and a namespace filter.
Changes:
- Fix endpoint: GET /workspaces/:id/memories with ?scope= query param
- Fix MemoryEntry type to match actual API: id, content, scope,
namespace, created_at, similarity_score
- Add LOCAL/TEAM/GLOBAL scope tabs
- Add namespace filter input
- Remove Edit functionality (no update endpoint in backend)
- Delete uses DELETE /workspaces/:id/memories/:id (by id, not key)
- Full rewrite of 27 tests to match new API and UI structure
- Uses ConfirmDialog (not native dialogs) for delete confirmation
- All dark zinc theme (no light colors)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: tighten types + improve provision-timeout message (#1135, #1136)
#1135 — TypeScript: make BudgetData.budget_used and WorkspaceMetrics
fields optional to match actual partial-response shapes from provisioning-
stuck workspaces. Runtime already guarded with ?? 0.
#1136 — provisiontimeout.go: replace misleading "check required env vars"
hint (preflight catches that case upfront) with accurate message about
container starting but failing to call /registry/register.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
* fix(test): align ssrf_test.go localhost test cases with isSafeURL behaviour
isSafeURL blocks 127.0.0.1 via ip.IsLoopback() even in dev environments.
The test cases `wantErr: false` for localhost were incorrect — the
test would fail when go test runs. Fix by changing wantErr to true
for both localhost test cases.
Rationale: loopback blocking at this layer is intentional. Access
control is enforced by WorkspaceAuth + CanCommunicate at the A2A
routing layer, not by the URL validation. Opening this would widen
the SSRF attack surface without adding real dev flexibility.
Closes: ssrf_test.go inconsistency reported 2026-04-21
Co-Authored-By: Claude Sonnet 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Molecule AI Core-UIUX <core-uiux@agents.moleculesai.app>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
F1089: PR #1032's panic-recovery defers used the outer `ctx` passed into
fireSchedule/tick. If that ctx was cancelled during the panic window
(HTTP timeout, graceful shutdown), ExecContext returned early and the
next_run_at UPDATE was silently skipped — leaving the schedule stuck.
Fix: both panic defers now call ExecContext(context.Background()) so the
recovery UPDATE is independent of the outer ctx's lifecycle.
Refs: #1201 (F1089, security audit 2026-04-21)
Co-authored-by: Molecule AI CP-BE <cp-be@agents.moleculesai.app>
- TestRecordSkipped_AdvancesNextRunAt: call recordSkipped directly instead
of going through fireSchedule, which now has a 2-min deferral loop (#969)
that makes sqlmock-based end-to-end testing impractical.
- TestFireSchedule_NormalSuccess_AdvancesNextRunAt: add missing expectation
for the consecutive_empty_runs reset query (#795) that fires on non-empty
successful responses.
- TestFireSchedule_ComputeNextRunError: same consecutive_empty_runs fix.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add production fix and three new test cases verifying that workspace
deletion cascade-disables all workspace_schedules for the deleted
workspace and its descendants, preventing zombie schedule firings.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When fireSchedule panics before reaching the next_run_at UPDATE,
the deferred recover catches the panic but never advances next_run_at,
leaving it stuck in the past forever. The schedule then fires every
tick (30s) in an infinite retry loop.
Add next_run_at advancement to both panic recovery defers (the
per-goroutine one in tick() and the inner one in fireSchedule()) so
the schedule always moves forward regardless of how the fire exits.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
CP-QA approved. seedInitialMemories() now truncates mem.Content at 100,000 bytes before INSERT. Oversized content is logged with byte count before/after so operators can detect truncation. Fixes#1066 (CWE-400). NOTE: no unit tests in this commit — follow-up issue recommended.
CP-QA approved. Panic recovery in fireSchedule now advances next_run_at via ComputeNextRun + ExecContext, preventing a panicking cron from indefinitely starving all other schedules. 3 new tests: TestPanicRecovery_AdvancesNextRunAt, TestFireSchedule_NormalSuccess, TestRecordSkipped_AdvancesNextRunAt. Fixes#1029.
Security fixes for the memory backup/restore endpoints merged in PR #1051.
## F1084 / #1131: Memory export exposes all workspaces
GET /admin/memories/export now applies redactSecrets() to each content
field before including it in the JSON response. Pre-SAFE-T1201 memories
(stored before redactSecrets was mandatory on writes) no longer leak
credential patterns in the admin export.
## F1085 / #1132: Memory import does not call redactSecrets
POST /admin/memories/import now calls redactSecrets() on content before
BOTH the deduplication check and the INSERT. This ensures:
- Imported memories with embedded credentials cannot land unredacted in
agent_memories (SAFE-T1201 / #838 parity with the commit_memory path).
- Dedup is performed against the redacted value so two backups with
the same original secret both get [REDACTED:*] as their content and
are correctly treated as duplicates.
## New tests
admin_memories_test.go: 6 tests covering redactSecrets parity on
both Export and Import endpoints.
Closes#1131.
Closes#1132.
Co-authored-by: Molecule AI Core-DevOps <core-devops@agents.moleculesai.app>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Molecule AI Infra-Runtime-BE <infra-runtime-be@agents.moleculesai.app>
URLs returned from DB and Redis cache (db.GetCachedURL, workspaces.url column)
are now validated via validateAgentURL() before any HTTP request is made:
- mcpResolveURL (mcp.go): added validateAgentURL() calls on all three return
paths (internal cache, Redis cache, DB fallback).
- resolveAgentURL (a2a_proxy.go): added validateAgentURL() call before
returning agentURL to the A2A dispatcher.
validateAgentURL() was extended (registry.go) to resolve DNS hostnames and
check each returned IP against the blocklist (private ranges, loopback,
cloud-metadata 169.254.0.0/16). "localhost" is allowed by name for local dev.
GET /admin/memories/export now applies redactSecrets() to each content field
before including it in the JSON response. Pre-SAFE-T1201 memories (stored
before redactSecrets was mandatory on writes) no longer leak credentials.
POST /admin/memories/import now calls redactSecrets() on content before both
the deduplication check and the INSERT. Imported memories with embedded
credentials cannot bypass SAFE-T1201 (#838).
- admin_memories.go: GET /admin/memories/export + POST /admin/memories/import
handler (from PR #1051, with security fixes applied).
- admin_memories_test.go: 6 tests covering redactSecrets parity on both endpoints.
- registry_test.go: added DNS-lookup test cases for validateAgentURL (F1083).
"localhost" allowed by name (preserves existing test); nxdomain blocked.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Remove duplicate-line ExecContext call that caused syntax error at mcp.go:784
- Update redactSecrets signature from 1-arg to 2-arg (workspaceID, content)
to match the canonical form established in PR #1017
- Update toolCommitMemory call site to use 2-arg form
- Add reserved workspaceID param note in docstring for future audit logging
Fixes PR #1036 compile-blocking issues (Platform Go job).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
CP-QA approved. golangci-lint fixes in bundle/exporter.go + bundle/importer.go, redactSecrets in admin_memories.go, plus 489-line admin_memories_test.go.
Workspaces stuck in provisioning used to sit in "starting" for 10min
until the sweeper flipped them. The real signal — a runtime crash at
EC2 boot — lands on the serial console within seconds but nothing
listened. These endpoints close the loop.
1. POST /admin/workspaces/:id/bootstrap-failed
The control plane's bootstrap watcher posts here when it spots
"RUNTIME CRASHED" in ec2:GetConsoleOutput. Handler:
- UPDATEs workspaces SET status='failed' only when status was
'provisioning' (idempotent — a raced online/failed stays put)
- Stores the error + log_tail in last_sample_error so the canvas
can render the real stack trace, not a generic "timeout" string
- Broadcasts WORKSPACE_PROVISION_FAILED with source='bootstrap_watcher'
2. GET /workspaces/:id/console
Proxies to CP's new /cp/admin/workspaces/:id/console endpoint so
the tenant platform can surface EC2 serial console output without
holding AWS credentials. CPProvisioner.GetConsoleOutput is the
client; returns 501 in non-CP deployments (docker-compose dev).
Both gated by AdminAuth — CP holds the tenant ADMIN_TOKEN that the
middleware accepts on its tier 2b branch.
Tests cover: happy-path fail, already-transitioned no-op, empty id,
log_tail truncation, and the 501 fallback when no CP is wired.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fixes audit #125 findings for CWE-639:
1. admin_test_token.go — CRITICAL IDOR (finding #112)
When ADMIN_TOKEN is set in production, require it explicitly on
GET /admin/workspaces/:id/test-token. The original gap: AdminAuth
accepted any valid org-scoped token, letting an Org A token holder
mint workspace bearer tokens for ANY workspace UUID they could enumerate.
Now requires ADMIN_TOKEN when it's configured; MOLECULE_ENV!=production
path still requires a valid bearer (any org token works for local dev).
2. org_plugin_allowlist.go — HIGH IDOR (finding #112)
GET and PUT /orgs/:id/plugins/allowlist: add requireOrgOwnership()
check after org existence verification. Org-token holders can only
read/write their own org's allowlist. Session and ADMIN_TOKEN callers
bypass the check (they have platform-wide access via the session
cookie path, not org tokens).
Closes: #112 (CWE-639 IDOR — tenant config access)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds isSafeURL() + isPrivateOrMetadataIP() in mcp.go and wires the
check into:
- MCP delegate_task (sync path) — line 530
- MCP delegate_task_async (fire-and-forget) — line 602
- a2a_proxy resolveAgentURL() — line 391
Blocklist covers: RFC-1918 private (10/8, 172.16/12, 192.168/16),
cloud metadata link-local (169.254/16), carrier-grade NAT (100.64/10),
documentation ranges (192.0.2/24, 198.51.100/24, 203.0.113/24),
loopback, unspecified, and link-local multicast.
For hostnames, DNS is resolved and every returned IP is validated —
blocks internal hostnames that resolve to private ranges.
Closes: #1130 (F1083 — SSRF in A2A proxy and MCP bridge)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three nits identified during post-merge review of #1119, #1133:
1. ContextMenu.tsx imported `removeNode` from the canvas store but
stopped using it when the delete-confirm flow moved to Canvas in
#1133. Also removed the now-unused mock entry in the keyboard
test so the test inventory matches the real call list.
2. Preflight's YAML parse failure was a silent pass — defensible since
the in-container preflight owns the schema, but invisible to ops if
a template ships malformed YAML. Log at WARN so the signal surfaces
without blocking the provision.
3. formatMissingEnvError rendered its slice via %q, producing
`["A" "B"]` which is Go-literal-looking and ugly in a user-facing
error. Join with ", " instead. Test updated to assert the new
format.
No behavioural changes beyond the log line; fixes are review nits, not
bug fixes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Workspaces stuck in status='provisioning' previously surfaced in three
bad ways:
1. **Details tab crashed** with `Cannot read properties of undefined
(reading 'toLocaleString')`. `BudgetSection` + `WorkspaceUsage`
assumed full response shapes but a provisioning-stuck workspace
returns partial `{}`. Guard each deep field with `?? 0` and cover
the partial-response case with regression tests.
2. **Missing required env vars failed silently** 15+ minutes later as
a cosmetic "Provisioning Timeout" banner. The in-container preflight
catches them but by then the container has already crashed without
calling /registry/register, so the workspace sat in 'provisioning'
forever. Mirror the preflight server-side: parse config.yaml's
`runtime_config.required_env` before launch, fail fast with a
WORKSPACE_PROVISION_FAILED event naming the missing vars.
3. **No backend timeout** ever flipped a stuck workspace to 'failed'.
Add a registry sweeper (10m default, env-overridable) that detects
workspaces stuck past the window, flips them to 'failed', and emits
WORKSPACE_PROVISION_TIMEOUT. Race-safe: the UPDATE re-checks the
status + age predicate so a concurrent register/restart wins.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addresses the Critical + Important findings from today's code
review of the org API keys feature (PRs #1105-1108).
## Critical-1: rate-limit mint endpoint
Previously POST /org/tokens had no mint-rate limit. A compromised
WorkOS session or leaked bearer could mint thousands of tokens in
seconds, forcing a painful manual cleanup of each one.
Fix: dedicated per-IP token bucket, 10 mints/hour/IP. Legitimate
bursts fit under the ceiling; abuse bounces. List + Delete stay
on the global limiter — they can't be used to generate new
secret material.
## Important-1: HTTP handler integration tests
internal/orgtoken had 9 unit tests; the HTTP layer (org_tokens.go)
had none. Adds org_tokens_test.go covering:
- List happy path + DB error → 500
- Create actor="admin-token" (bootstrap), actor="org-token:<prefix>"
(chained mint), actor="session" (canvas browser path)
- Create name>100 chars → 400
- Create with empty body mints with no name
- Revoke happy path 200, missing id 404, empty id 400
- Plaintext returned in response body and prefix matches first 8 chars
- Warning text present
A regression that breaks the tier-ordering, drops the createdBy
field, or accepts oversized names now fails at CI not prod.
## Important-2: bound List output
List() had no LIMIT — a mint-storm bug or abuse could make the
admin UI slow to render and allocate proportionally. Adds
LIMIT 500 at the SQL layer. 10x realistic ceiling, guardrail
against pathological cases.
## Important-3: audit provenance uses plaintext prefix, not UUID
orgTokenActor() was logging "org-token:<first-8-of-uuid>" which
couldn't be cross-referenced with the UI (which shows first-8
of the plaintext). Users could not correlate "who minted this"
audit entries with the revoke button they're looking at.
Fix: Validate() now returns (id, prefix, error). Middleware
stashes both on the gin context. Handler reads prefix for the
actor string. Audit rows now match UI prefixes exactly.
## Nit: named constants for audit labels
actorOrgTokenPrefix / actorSession / actorAdminToken replace
the hardcoded strings scattered across the handler. Greppable
across log pipelines + audit queries; one place to change if
the format evolves.
## Tests
- internal/orgtoken: 9 existing + 0 new, all still green (updated
signatures for Validate returning prefix).
- internal/handlers/org_tokens_test.go: new — 9 HTTP-layer tests
above. Full gin.Context + sqlmock harness.
- Full `go test ./...` green except one pre-existing
TestGitHubToken_NoTokenProvider flake unrelated to this change
(expects 404, gets 500 — tracked separately).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extends WorkspaceAuth to accept org API tokens as a valid
credential for any workspace sub-route in the org. Previously a
user minting an org token could hit admin-surface endpoints
(/workspaces, /org/import, etc.) but couldn't reach per-workspace
routes like /workspaces/:id/channels — those were gated by
WorkspaceAuth which only knew about workspace-scoped tokens.
Scope matches the explicit product spec: one org API key can
manipulate every workspace in the org. AI agents given a key can
read/write channels, tokens, schedules, secrets, tasks across all
workspaces.
## WorkspaceAuth tier order
1. ADMIN_TOKEN exact match (break-glass / bootstrap)
2. Org API token (Validate against org_api_tokens) NEW
3. Workspace-scoped token (ValidateToken with :id binding)
4. Same-origin canvas referer
Org token tier sits above the per-workspace check so a presenter
of an org key doesn't hit the narrower ValidateToken failure path
first. Checked with isSameOriginCanvas path unchanged.
## End-to-end verified
Minted test token via ADMIN_TOKEN, then with that org token:
- GET /workspaces → 200 (list all)
- GET /workspaces/<id> → 200 (detail, admin-only route)
- GET /workspaces/<id>/channels → 200 (workspace sub-route)
- GET /workspaces/<id>/tokens → 200 (workspace tokens list)
- GET /workspaces/<bad-uuid> → 404 workspace not found
(routing still scoped correctly)
## Documentation
- docs/architecture/org-api-keys.md — design, data model, threat
model, security properties
- docs/architecture/org-api-keys-followups.md — 10 tracked
follow-ups prioritized (role scoping P1, per-workspace binding
P1, expiry P2, usage metrics P2, WorkOS user_id capture P2,
rotation webhooks P3, mint-rate limit P3, audit log P2, CLI
P3, migrate ADMIN_TOKEN to the same table P4)
- docs/guides/org-api-keys.md — end-user guide (mint via UI,
use in curl/Python/TS/AI agents, session-vs-key comparison)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds user-facing API keys with full-org admin scope. Replaces the
single ADMIN_TOKEN env var with named, revocable, audited tokens
that users can mint/rotate from the canvas UI without ops
intervention.
Designed for the beta growth phase — one token tier (full admin).
Future work will split into scoped roles (admin / workspace-write
/ read-only) and per-workspace bindings. See docs/architecture/
org-api-keys.md for the design + follow-up roadmap.
## Surface
POST /org/tokens mint (plaintext returned once)
GET /org/tokens list live keys (prefix-only)
DELETE /org/tokens/:id revoke (idempotent)
All AdminAuth-gated. Bootstrap path: mint the first token via
ADMIN_TOKEN or canvas session; tokens can mint more tokens after.
## Validation as a new AdminAuth tier (2a)
AdminAuth evaluation order:
Tier 0 lazy-bootstrap fail-open (only when no live tokens AND
no ADMIN_TOKEN env)
Tier 1 verified WorkOS session via /cp/auth/tenant-member
Tier 2a org_api_tokens SELECT — NEW
Tier 2b ADMIN_TOKEN env (bootstrap / CLI break-glass)
Tier 3 any live workspace token (deprecated, only when ADMIN_TOKEN
unset)
Tier 2a runs ONE indexed lookup (partial index on
token_hash WHERE revoked_at IS NULL) + an async last_used_at
bump. No measurable latency cost on the hot path.
## UI
New "Org API Keys" tab in the settings panel. Label field for
human-readable naming. Plaintext shown once + clipboard copy.
Revoke with confirm dialog. Mirrors the existing workspace-
TokensTab flow so users who've used one get the other for free.
## Security properties
- Plaintext never stored. sha256 hash + 8-char display prefix.
- Revocation is immediate: partial index on revoked_at IS NULL
means the next request validates or fails in microseconds.
- created_by audit field captures provenance: "org-token:<short>"
when a token mints another, "session" for browser-UI mints,
"admin-token" for the ADMIN_TOKEN bootstrap path.
- Validate() collapses all failure shapes into ErrInvalidToken
so response-shape can't distinguish "never existed" from
"revoked".
## Tests
- internal/orgtoken: 9 unit tests (hash storage, empty field
null-ing, validation happy path, empty plaintext, unknown hash,
revoked filtering, list ordering, revoke idempotency, has-any-
live short-circuit).
- AdminAuth tier-2a integration covered by existing middleware
tests unchanged (fail-open + bearer paths).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addresses three Critical findings from today's code review of the
SaaS-canvas routing stack.
## Critical-1: session verification scoped to the current tenant
session_auth.go previously verified via GET /cp/auth/me, which
only answers "is someone logged in" — NOT "is this user in the
org they're targeting." Every WorkOS-authed user (including folks
who only signed up via app.moleculesai.app with no tenant
relationship) could call /workspaces, /approvals/pending,
/bundles/import, /org/import etc. on ANY tenant they could reach.
Cross-tenant read: user at acme.moleculesai.app could hit
bob.moleculesai.app/workspaces with their cookie and get Bob's
workspaces.
Fix:
- CP gains GET /cp/auth/tenant-member?slug=<slug> which joins
org_members × organizations and only returns member:true when
the authenticated user is actually in that org.
- Tenant sets MOLECULE_ORG_SLUG at boot via user-data.
- session_auth now calls tenant-member (not /me), passing its
own slug. Cache key includes slug so one tenant's cached
positive never satisfies another's check.
## Critical-2: cp_proxy path allowlist (lateral-movement fix)
cp_proxy.go forwarded any /cp/* path upstream with the cookie
and bearer attached. Since /cp/admin/* accepts sessions as one
of its auth tiers, a tenant-authed user could curl
/cp/admin/tenants/other-slug/diagnostics through their tenant
and the CP would honor it — turning any tenant into a lateral
hop into admin surface.
Fix: explicit allowlist of paths the canvas browser bundle
actually needs (/cp/auth, /cp/orgs, /cp/billing, /cp/templates,
/cp/legal). Everything else 404s at the tenant before cookies
leave. Fail-closed: future UI paths require explicit entries.
## Important-1,2: bounded session cache + split positive/negative TTL
Previous sync.Map cache grew unbounded (one entry per unique
Cookie header for process lifetime) and cached failures for 30s,
meaning a 3s CP blip locked users out for the full window.
Fix:
- Bounded map with batch random eviction at cap (10k entries ×
~100 bytes = 1 MB ceiling). Random eviction is O(1)
expected; we don't need precise LRU.
- Periodic sweeper goroutine (2 min) reclaims expired entries
even when they're not re-hit.
- Positive TTL 30s, negative TTL 5s — short negative so CP
flakes self-heal fast.
- Transport errors NOT cached (would otherwise trap every
user during a multi-second upstream outage).
- Cache key = sha256(slug + cookie) so raw session tokens
don't sit in process memory, and cross-tenant isolation is
structural not policy.
## Important-3: TenantGuard /cp/* bypass documented
Added a security note to the bypass explaining why it's safe
only under the current setup (cp_proxy allowlist + tunnel-only
ingress), and what would require revisiting (SG opens :8080
inbound to the VPC).
## Tests
- session_auth_test.go: 12 new tests — empty cookie, missing
slug, no CP, member:true happy path with cache hit, member:
false, 401 upstream, malformed JSON, transport error not
cached, cross-tenant isolation (same cookie different
tenants hit upstream separately), bounded eviction, expired
entries, cache key collision resistance.
- cp_proxy_test.go: new — isCPProxyAllowedPath covers 17
allow/block cases, forwarding preserves Cookie+Auth, Host
rewritten, blocked paths 404 without calling upstream.
All platform tests pass. CP provisioner tests pass after
threading cfg.OrgSlug into the container env.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The github-app-auth plugin's go.mod had a relative replace directive
(../molecule-monorepo/platform) that didn't resolve in Docker where
the plugin is at /plugin/ and the platform at /app/. This caused the
plugin's provisionhook.TokenProvider interface to come from a different
package path than the platform's, so the type assertion in
FirstTokenProvider() failed — "no token provider registered".
Fix: sed the plugin's go.mod replace to point at /app during Docker build.
Also added debug logging to GetInstallationToken for future diagnosis.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Canvas (SaaS tenant UI) runs in the browser and authenticates the
user via a WorkOS session cookie scoped to .moleculesai.app. It
has no bearer token — the token-based ADMIN_TOKEN scheme is for
CLI + server-to-server callers, not end users.
Adds a session-verification tier to AdminAuth that runs BEFORE the
bearer check:
1. If Cookie header present AND CP_UPSTREAM_URL configured →
GET /cp/auth/me upstream with the same cookie. 200 + valid
user_id → grant admin access. Non-200 → fall through.
2. Else (no cookie, or no CP configured, or CP said no) →
existing bearer-only path unchanged.
Positive verifications are cached 30s keyed by the raw Cookie
header, so a burst of canvas admin-page renders doesn't DDoS
the CP. Revocations propagate within that window.
Self-hosted / dev deploys without CP_UPSTREAM_URL: feature
disabled, behavior unchanged. So this is strictly additive for
the SaaS case.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Today's rollout of cp_proxy (PR #1095/1096) mounted /cp/* as a
reverse-proxy to the control plane, but the TenantGuard middleware
runs first in the global chain and 404s anything that isn't in its
exact-path allowlist (/health + /metrics). Every /cp/auth/me fetch
from canvas landed on a 40µs 404 before ever reaching the proxy.
/cp/* is handled upstream (WorkOS session + admin bearer), so the
tenant doesn't need to attach org identity for those paths. Passing
them through is correct — matches the design where the tenant
platform is a pure transit layer for /cp/*.
Verified: /cp/auth/me via tunnel now returns 401 (correct unauth
from CP) instead of 404 from TenantGuard.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Canvas's browser bundle issues fetches to both CP endpoints
(/cp/auth/me, /cp/orgs, ...) AND tenant-platform endpoints
(/canvas/viewport, /approvals/pending, /org/templates). They
share ONE build-time base URL. Baking api.moleculesai.app
broke tenant calls with 404; baking the tenant subdomain broke
auth. Tried both today and saw exactly one failure mode per
attempt.
Real fix: same-origin fetches + tenant-side split. Adds:
internal/router/cp_proxy.go # /cp/* → CP_UPSTREAM_URL
mounted before NoRoute(canvasProxy). Now a tenant serves:
/cp/* → reverse-proxy to api.moleculesai.app
/canvas/viewport,
/approvals/pending,
/workspaces/:id/*,
/ws, /registry, → tenant platform (existing handlers)
/metrics
everything else → canvas UI (existing reverse-proxy)
Canvas middleware reverts to `connect-src 'self' wss:` for the
same-origin path (keeping explicit PLATFORM_URL whitelist as a
self-hosted escape hatch when the build-arg is non-empty).
CI build-arg flips to NEXT_PUBLIC_PLATFORM_URL="" so the bundle
issues relative fetches.
Security of cp_proxy:
- Cookie + Authorization PRESERVED across the hop (opposite of
canvas proxy) — they carry the WorkOS session, which is the
whole point.
- Host rewritten to upstream so CORS + cookie-domain on the CP
side see their own hostname.
- Upstream URL validated at construction: must parse, must be
http(s), must have a host — misconfig fails closed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The canvas sends NEXT_PUBLIC_ADMIN_TOKEN on all API calls but per-workspace
routes (/activity, /delegations, /traces) use WorkspaceAuth which only
accepts per-workspace bearer tokens. This made the canvas dashboard 401
on every workspace detail view.
Fix: WorkspaceAuth now accepts the admin token as a fallback after
workspace token validation fails. This lets the canvas read all workspace
data with a single admin credential.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Soft-delete (status='removed') leaves orphan DB rows and FK data forever.
When ?purge=true is passed, after container cleanup the handler cascade-
deletes all leaf FK tables and hard-removes the workspace row.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The org import fired all workspace provisioning goroutines concurrently,
overwhelming Docker when creating 39+ containers. Containers timed out,
leaving workspaces stuck in 'provisioning' with no schedules or hooks.
Fix:
- Add provisionConcurrency=3 semaphore limiting concurrent Docker ops
- Increase workspaceCreatePacingMs from 50ms to 2000ms between siblings
- Pass semaphore through createWorkspaceTree recursion
With 39 workspaces at 3 concurrent + 2s pacing, import takes ~30s instead
of timing out. Each workspace gets its full template: schedules, hooks,
settings, hierarchy.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
