molecule-ai/molecule-core
52
0
Fork 2
Code Issues 183 Pull Requests 19 Actions 1 Packages Projects Releases Wiki Activity

molecule-dev import returns 400 "org template expansion failed" — !external resolver lacks /org-templates write perms #226

New Issue
Closed
opened 2026-05-10 02:48:15 +00:00 by claude-ceo-assistant · 1 comment
claude-ceo-assistant commented 2026-05-10 02:48:15 +00:00
Owner
Copy Link
Copy Source

Symptom

POST /org/import {"dir":"molecule-dev"}    → 400 {"error":"org template expansion failed"}
POST /org/import {"dir":"free-beats-all"}  → 201 (works — no !external)

The handler returns a generic 400 (org.go:592); only the server log carries the real error:

Org import: refusing import: !include expansion failed:
!external at line 156: fetch git.moleculesai.app/molecule-ai/molecule-dev-department@v1.0.0:
mkdir cache root: mkdir /org-templates/molecule-dev/.external-cache: permission denied

Root cause

Dockerfile.tenant chowns /canvas /platform /memory-plugin /migrations to canvas:canvas but NOT /org-templates. The image runs as the canvas user (uid 1000). When the !external resolver (workspace-server/internal/handlers/org_external.go:314) calls os.MkdirAll(cacheRoot, 0o755) to create /org-templates/<tmpl>/.external-cache/<repo>/<sha>/ on first import, mkdir(2) returns EACCES.

Any org template that uses !external (today: molecule-dev → dev-lead from molecule-ai/molecule-dev-department@v1.0.0) trips this on first import.

Reproduction

Tenant staging-cplead-2 (canary AWS 004947743811, image SHA a93c4ce17725...).

Live stop-gap (2026-05-10 ~02:42 UTC)

docker exec --user 0 molecule-tenant chown -R canvas:canvas /org-templates/molecule-dev

Verified: POST /org/import {"dir":"molecule-dev"} → 201 with count=39 workspaces; cp-lead team (Controlplane Lead, CP-BE, CP-QA, CP-Security) all status=online within 2 min.

Durable fix

PR #223 — adds /org-templates to the existing chown -R canvas:canvas argv in Dockerfile.tenant. 9-line change, image-resident, no runtime perf impact.

Discovery context

Phase 5 of RFC internal#168 (cross-account staging tenant split). cp-lead team bring-up on canary tenant. Refs: internal#77 (!external RFC), task #222 (resolver PR that introduced the unflagged perms dependency).

## Symptom ``` POST /org/import {"dir":"molecule-dev"} → 400 {"error":"org template expansion failed"} POST /org/import {"dir":"free-beats-all"} → 201 (works — no !external) ``` The handler returns a generic 400 (`org.go:592`); only the server log carries the real error: ``` Org import: refusing import: !include expansion failed: !external at line 156: fetch git.moleculesai.app/molecule-ai/molecule-dev-department@v1.0.0: mkdir cache root: mkdir /org-templates/molecule-dev/.external-cache: permission denied ``` ## Root cause `Dockerfile.tenant` chowns `/canvas /platform /memory-plugin /migrations` to `canvas:canvas` but NOT `/org-templates`. The image runs as the `canvas` user (uid 1000). When the `!external` resolver (`workspace-server/internal/handlers/org_external.go:314`) calls `os.MkdirAll(cacheRoot, 0o755)` to create `/org-templates/<tmpl>/.external-cache/<repo>/<sha>/` on first import, mkdir(2) returns EACCES. Any org template that uses `!external` (today: `molecule-dev` → `dev-lead` from `molecule-ai/molecule-dev-department@v1.0.0`) trips this on first import. ## Reproduction Tenant `staging-cplead-2` (canary AWS 004947743811, image SHA `a93c4ce17725...`). ## Live stop-gap (2026-05-10 ~02:42 UTC) ``` docker exec --user 0 molecule-tenant chown -R canvas:canvas /org-templates/molecule-dev ``` Verified: `POST /org/import {"dir":"molecule-dev"}` → 201 with `count=39` workspaces; cp-lead team (Controlplane Lead, CP-BE, CP-QA, CP-Security) all `status=online` within 2 min. ## Durable fix PR #223 — adds `/org-templates` to the existing `chown -R canvas:canvas` argv in `Dockerfile.tenant`. 9-line change, image-resident, no runtime perf impact. ## Discovery context Phase 5 of RFC internal#168 (cross-account staging tenant split). cp-lead team bring-up on canary tenant. Refs: internal#77 (!external RFC), task #222 (resolver PR that introduced the unflagged perms dependency).
claude-ceo-assistant commented 2026-05-10 02:49:24 +00:00
Author
Owner
Copy Link
Copy Source

Fixed by molecule-core#223, merged at 34cdd8cc by core-lead 2026-05-10 02:48:07 UTC. Verified end-to-end on staging-cplead-2 (canary 004947743811): POST /org/import {"dir":"molecule-dev"} returns 201 with 39 workspaces; cp-lead team (Controlplane Lead 941a929e, CP-BE 99de7cab, CP-QA a8ba9dc8, CP-Security a00e74df) all status=online.

Fixed by molecule-core#223, merged at `34cdd8cc` by core-lead 2026-05-10 02:48:07 UTC. Verified end-to-end on staging-cplead-2 (canary 004947743811): `POST /org/import {"dir":"molecule-dev"}` returns 201 with 39 workspaces; cp-lead team (Controlplane Lead 941a929e, CP-BE 99de7cab, CP-QA a8ba9dc8, CP-Security a00e74df) all status=online.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
approved
area/ci
CI/CD pipeline issues
 do-not-auto-merge
Opt out of autonomous merge-queue merging
 do-not-merge
hold from auto-merge (design review in progress)
 kind/infrastructure
Infrastructure-related issues
 merge-queue
Ready for serialized Gitea merge queue
 merge-queue-hold
Temporarily hold PR in merge queue
 needs-engineer
platform/go
Go platform test issues
 ready-to-build
release-blocker
Blocks the staging→main promotion / a release
 release-test
security
test-label-sre
tier:high
High risk per dev-sop §SOP-6 — ceo only, 24h cooldown
 tier:low
Low risk per dev-sop §SOP-6 — engineers/managers/ceo can approve
 tier:medium
Medium risk per dev-sop §SOP-6 — managers/ceo can approve
 triage-test
test
 wip
Work in progress — do not auto-merge
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
agent-dev-a agent-dev-b agent-pm agent-researcher agent-reviewer agent-reviewer-1 agent-reviewer-cr2 app-fe (Molecule AI · app-fe) app-lead (Molecule AI · app-lead) app-qa (Molecule AI · app-qa) claude-ceo-assistant claude-ci-reader claude-status-reaper core-be (Molecule AI · core-be) core-devops (Molecule AI · core-devops) core-fe (Molecule AI · core-fe) core-lead (Molecule AI · core-lead) core-offsec (Molecule AI · core-offsec) core-qa (Molecule AI · core-qa) core-security (Molecule AI · core-security) core-uiux (Molecule AI · core-uiux) cp-be (Molecule AI · cp-be) cp-lead (Molecule AI · cp-lead) cp-qa (Molecule AI · cp-qa) cp-security (Molecule AI · cp-security) cui (Zhanlin Cui) dev-lead (Molecule AI · dev-lead) devops-engineer documentation-specialist (Molecule AI · documentation-specialist) fullstack-engineer (Molecule AI · fullstack-engineer) godwin hongming hongming-ceo-delegated hongming-codex-laptop hongming-kimi-laptop hongming-pc2 hongming-personal infra-lead (Molecule AI · infra-lead) infra-runtime-be (Molecule AI · infra-runtime-be) infra-sre (Molecule AI · infra-sre) integration-tester (Molecule AI · integration-tester) mc-drift-bot molecule-code-reviewer plugin-dev (Molecule AI · plugin-dev) pm publish-runtime-bot pypi-publisher (Molecule AI PyPI Publisher (RFC#596)) release-manager (Molecule AI · release-manager) sdk-dev (Molecule AI · sdk-dev) sdk-lead (Molecule AI · sdk-lead) sop-drift-bot sop-tier-bot (SOP Tier-Check Bot) technical-writer (Molecule AI · technical-writer) triage-operator (Molecule AI · triage-operator)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: molecule-ai/molecule-core#226
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?