Some checks failed
CI / Canvas Deploy Reminder (pull_request) Blocked by required conditions
CodeQL / Analyze (${{ matrix.language }}) (go) (pull_request) Successful in 11s
CodeQL / Analyze (${{ matrix.language }}) (javascript-typescript) (pull_request) Successful in 11s
CodeQL / Analyze (${{ matrix.language }}) (python) (pull_request) Successful in 11s
Check merge_group trigger on required workflows / Required workflows have merge_group trigger (pull_request) Successful in 24s
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 36s
cascade-list-drift-gate / check (pull_request) Successful in 35s
E2E API Smoke Test / detect-changes (pull_request) Successful in 36s
CI / Detect changes (pull_request) Successful in 39s
Lint curl status-code capture / Scan workflows for curl status-capture pollution (pull_request) Successful in 27s
branch-protection drift check / Branch protection drift (pull_request) Successful in 45s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 47s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 37s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 58s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 57s
Harness Replays / detect-changes (pull_request) Successful in 50s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 29s
CI / Python Lint & Test (pull_request) Successful in 33s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 56s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 30s
Ops Scripts Tests / Ops scripts (unittest) (pull_request) Successful in 2m5s
Harness Replays / Harness Replays (pull_request) Failing after 1m37s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 4m54s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 6m49s
CI / Platform (Go) (pull_request) Successful in 9m13s
CI / Canvas (Next.js) (pull_request) Failing after 11m30s
Lets a workspace declare it (and its entire subtree) should be skipped during /org/import. Pointer-typed `*bool` so we distinguish "explicitly false" from "unset" (default = spawn). ## Use case The dev-tree org template ships the full role taxonomy (Dev Lead with Core Platform / Controlplane / App & Docs / Infra / SDK Leads, each with their own engineering / QA / security / UI-UX children — 27 personas total in a single import). Some setups need a smaller set: - Local dev on a memory-constrained machine - Demo / smoke runs that don't need the full org breathing - Customer trials starting with leadership-only before fan-out Pre-fix the only options were: - Edit the canonical template (mutates shared state) - Author a parallel slimmer template (duplicates structure) - Manual workspace deprovision after full import (wasteful — already paid the docker pull / build cost) `spawning: false` is the per-workspace knob that solves this without touching the canonical template structure. ## Semantics - Unset: workspace spawns (current behaviour, no migration) - `spawning: true`: explicitly spawns (same as unset) - `spawning: false`: workspace is skipped AND every descendant is skipped. The guard sits BEFORE any side effect in createWorkspaceTree — no DB row, no docker provision, no children recursion. A false-spawning subtree is genuinely a no-op except for the log line. countWorkspaces still counts the subtree (so /org/templates numbers reflect the full structure). ## Stage A — verified Local dev-only template that wraps teams/dev.yaml (Dev Lead) with children:[] cleared on the 5 sub-team yaml files, plus 3 floater personas (Release Manager / Integration Tester / Fullstack Engineer). /org/import returned 9 workspaces. Drop-in: same result via `spawning: false` on each sub-tree root in the future. ## Stage B — N/A Pure additive feature on the org-template handler. No SaaS deploy chain implications. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
736 lines
30 KiB
Go
736 lines
30 KiB
Go
package handlers
|
|
|
|
// org.go — core org handler: types, struct, ListTemplates, Import.
|
|
// Tree creation logic is in org_import.go; utility helpers in org_helpers.go.
|
|
|
|
import (
|
|
"context"
|
|
"encoding/json"
|
|
"fmt"
|
|
"log"
|
|
"net/http"
|
|
"os"
|
|
"path/filepath"
|
|
"strconv"
|
|
"strings"
|
|
|
|
"github.com/Molecule-AI/molecule-monorepo/platform/internal/channels"
|
|
"github.com/Molecule-AI/molecule-monorepo/platform/internal/events"
|
|
"github.com/Molecule-AI/molecule-monorepo/platform/internal/models"
|
|
"github.com/Molecule-AI/molecule-monorepo/platform/internal/provisioner"
|
|
"github.com/gin-gonic/gin"
|
|
"gopkg.in/yaml.v3"
|
|
)
|
|
|
|
// OrgHandler manages org template import/export.
|
|
// workspaceCreatePacingMs is the brief delay between sibling workspace creations
|
|
// during org import. Prevents overwhelming Docker when creating many containers.
|
|
const workspaceCreatePacingMs = 2000
|
|
|
|
// defaultProvisionConcurrency is the fallback cap for parallel
|
|
// workspace-provision goroutines when MOLECULE_PROVISION_CONCURRENCY
|
|
// is unset. Originally a hard constant of 3 (PR #1084) calibrated for
|
|
// Docker-mode workspaces. The constant is now a default — operators
|
|
// running on EC2 (where each provision is a RunInstances call AWS
|
|
// happily parallelises) typically want a much higher cap, while
|
|
// Docker-mode dev environments still prefer the conservative 3.
|
|
//
|
|
// 3 keeps the existing Docker-mode behavior. SaaS deployments override
|
|
// via env (see resolveProvisionConcurrency below).
|
|
const defaultProvisionConcurrency = 3
|
|
|
|
// resolveProvisionConcurrency returns the effective semaphore size for
|
|
// org-import workspace provisioning, honoring MOLECULE_PROVISION_CONCURRENCY:
|
|
//
|
|
// - unset / empty / non-numeric → defaultProvisionConcurrency (3)
|
|
// - "0" → unlimited (a very large cap;
|
|
// practically no semaphore — used on
|
|
// SaaS where AWS RunInstances is the
|
|
// rate-limiter, not us)
|
|
// - any positive integer N → N
|
|
// - negative integer → defaultProvisionConcurrency (3),
|
|
// log warning so operator notices
|
|
// the misconfiguration
|
|
//
|
|
// The "0 = unlimited" mapping was a deliberate choice: an env var of "0"
|
|
// is the natural shorthand for "no cap" without forcing operators to
|
|
// type a magic large number. The implementation hands off a large but
|
|
// finite value (1<<20) so the channel still works as a regular
|
|
// buffered chan; goroutines will never block on the semaphore in
|
|
// practice.
|
|
func resolveProvisionConcurrency() int {
|
|
raw := strings.TrimSpace(os.Getenv("MOLECULE_PROVISION_CONCURRENCY"))
|
|
if raw == "" {
|
|
return defaultProvisionConcurrency
|
|
}
|
|
n, err := strconv.Atoi(raw)
|
|
if err != nil {
|
|
log.Printf("org_import: MOLECULE_PROVISION_CONCURRENCY=%q is not an integer; falling back to default %d",
|
|
raw, defaultProvisionConcurrency)
|
|
return defaultProvisionConcurrency
|
|
}
|
|
if n < 0 {
|
|
log.Printf("org_import: MOLECULE_PROVISION_CONCURRENCY=%d is negative; falling back to default %d",
|
|
n, defaultProvisionConcurrency)
|
|
return defaultProvisionConcurrency
|
|
}
|
|
if n == 0 {
|
|
// Unlimited semantics — use a large but finite cap so the
|
|
// chan-based semaphore stays a no-op. 1M is well past any
|
|
// realistic org-import size; AWS RunInstances rate-limit and
|
|
// account vCPU quota are the real backpressure here.
|
|
return 1 << 20
|
|
}
|
|
return n
|
|
}
|
|
|
|
// Child grid layout constants — kept in sync with canvas-topology.ts on
|
|
// the client. Children laid on import use the same 2-column grid so the
|
|
// nested view is clean out of the box. Before this, YAML-declared
|
|
// canvas coords (absolute, horizontally fanned at y=180) produced an
|
|
// overlapping mess under the nested render (see screenshot in PR
|
|
// #1981 thread).
|
|
const (
|
|
childDefaultWidth = 240.0
|
|
childDefaultHeight = 130.0
|
|
childGutter = 14.0
|
|
parentHeaderPadding = 130.0
|
|
parentSidePadding = 16.0
|
|
childGridColumnCount = 2
|
|
)
|
|
|
|
// childSlot computes the child-relative position for the N-th sibling in
|
|
// a parent's 2-column grid. Matches defaultChildSlot in
|
|
// canvas-topology.ts exactly — change them together. Leaf-sized slots
|
|
// only; for variable-size siblings use childSlotInGrid below.
|
|
func childSlot(index int) (x, y float64) {
|
|
col := index % childGridColumnCount
|
|
row := index / childGridColumnCount
|
|
x = parentSidePadding + float64(col)*(childDefaultWidth+childGutter)
|
|
y = parentHeaderPadding + float64(row)*(childDefaultHeight+childGutter)
|
|
return
|
|
}
|
|
|
|
type nodeSize struct {
|
|
width, height float64
|
|
}
|
|
|
|
// sizeOfSubtree computes the bounding-box size for a workspace and its
|
|
// entire descendant tree as rendered by the canvas grid layout.
|
|
// Post-order: leaves return the CHILD_DEFAULT footprint; parents return
|
|
// the size that fits all direct children (which may themselves be
|
|
// parents with grandchildren). Matches the client's
|
|
// `subtreeSize` pass in canvas-topology.ts so the server can lay out
|
|
// org imports the same way the canvas will render them.
|
|
func sizeOfSubtree(ws OrgWorkspace) nodeSize {
|
|
if len(ws.Children) == 0 {
|
|
return nodeSize{childDefaultWidth, childDefaultHeight}
|
|
}
|
|
cols := childGridColumnCount
|
|
if len(ws.Children) < cols {
|
|
cols = len(ws.Children)
|
|
}
|
|
rows := (len(ws.Children) + cols - 1) / cols
|
|
childSizes := make([]nodeSize, len(ws.Children))
|
|
maxColW := 0.0
|
|
for i, c := range ws.Children {
|
|
childSizes[i] = sizeOfSubtree(c)
|
|
if childSizes[i].width > maxColW {
|
|
maxColW = childSizes[i].width
|
|
}
|
|
}
|
|
rowHeights := make([]float64, rows)
|
|
for i, cs := range childSizes {
|
|
row := i / cols
|
|
if cs.height > rowHeights[row] {
|
|
rowHeights[row] = cs.height
|
|
}
|
|
}
|
|
totalRowH := 0.0
|
|
for _, h := range rowHeights {
|
|
totalRowH += h
|
|
}
|
|
return nodeSize{
|
|
width: parentSidePadding*2 + maxColW*float64(cols) + childGutter*float64(cols-1),
|
|
height: parentHeaderPadding + totalRowH + childGutter*float64(rows-1) + parentSidePadding,
|
|
}
|
|
}
|
|
|
|
// childSlotInGrid computes the relative position of sibling `index`
|
|
// given all siblings' subtree sizes. Uniform column width (= max width
|
|
// across siblings), per-row max height, so a nested parent sibling
|
|
// pushes its row down without displacing the column grid. Matches the
|
|
// TS mirror in canvas-topology.ts.
|
|
func childSlotInGrid(index int, siblingSizes []nodeSize) (x, y float64) {
|
|
if len(siblingSizes) == 0 {
|
|
return parentSidePadding, parentHeaderPadding
|
|
}
|
|
cols := childGridColumnCount
|
|
if len(siblingSizes) < cols {
|
|
cols = len(siblingSizes)
|
|
}
|
|
rows := (len(siblingSizes) + cols - 1) / cols
|
|
maxColW := 0.0
|
|
for _, s := range siblingSizes {
|
|
if s.width > maxColW {
|
|
maxColW = s.width
|
|
}
|
|
}
|
|
rowHeights := make([]float64, rows)
|
|
for i, s := range siblingSizes {
|
|
row := i / cols
|
|
if s.height > rowHeights[row] {
|
|
rowHeights[row] = s.height
|
|
}
|
|
}
|
|
col := index % cols
|
|
row := index / cols
|
|
x = parentSidePadding + float64(col)*(maxColW+childGutter)
|
|
y = parentHeaderPadding
|
|
for r := 0; r < row; r++ {
|
|
y += rowHeights[r] + childGutter
|
|
}
|
|
return
|
|
}
|
|
|
|
// orgImportScheduleSQL is the upsert executed for every schedule during
|
|
// org/import. Extracted to a const so TestImport_OrgScheduleSQLShape can
|
|
// assert its shape without regex-scanning org.go (issue #24 follow-up).
|
|
//
|
|
// Guarantees, in one statement:
|
|
// - INSERT new rows with source='template'
|
|
// - On (workspace_id, name) collision, only refresh template-source rows
|
|
// (runtime-added schedules are preserved across re-imports)
|
|
// - No DELETE — removal is out of scope (additive semantics)
|
|
const orgImportScheduleSQL = `
|
|
INSERT INTO workspace_schedules (workspace_id, name, cron_expr, timezone, prompt, enabled, next_run_at, source)
|
|
VALUES ($1, $2, $3, $4, $5, $6, $7, 'template')
|
|
ON CONFLICT (workspace_id, name) DO UPDATE
|
|
SET cron_expr = EXCLUDED.cron_expr,
|
|
timezone = EXCLUDED.timezone,
|
|
prompt = EXCLUDED.prompt,
|
|
enabled = EXCLUDED.enabled,
|
|
next_run_at = EXCLUDED.next_run_at,
|
|
updated_at = now()
|
|
WHERE workspace_schedules.source = 'template'
|
|
`
|
|
|
|
type OrgHandler struct {
|
|
workspace *WorkspaceHandler
|
|
broadcaster *events.Broadcaster
|
|
provisioner *provisioner.Provisioner
|
|
channelMgr *channels.Manager
|
|
configsDir string
|
|
orgDir string // path to org-templates/
|
|
}
|
|
|
|
func NewOrgHandler(wh *WorkspaceHandler, b *events.Broadcaster, p *provisioner.Provisioner, channelMgr *channels.Manager, configsDir, orgDir string) *OrgHandler {
|
|
return &OrgHandler{
|
|
workspace: wh,
|
|
broadcaster: b,
|
|
provisioner: p,
|
|
channelMgr: channelMgr,
|
|
configsDir: configsDir,
|
|
orgDir: orgDir,
|
|
}
|
|
}
|
|
|
|
// EnvRequirement is either a single env var name (strict: that exact
|
|
// var must be configured) or an any-of group (any one of the listed
|
|
// names satisfies the requirement).
|
|
//
|
|
// YAML shapes accepted:
|
|
//
|
|
// required_env:
|
|
// - GITHUB_TOKEN # single
|
|
// - any_of: [ANTHROPIC_API_KEY, CLAUDE_CODE_OAUTH_TOKEN] # OR group
|
|
//
|
|
// The any-of form exists because some runtimes accept either of two
|
|
// credential shapes — Claude Code takes ANTHROPIC_API_KEY or an OAuth
|
|
// token interchangeably, and forcing an org template to pick one
|
|
// would falsely block the other. For JSON (GET /org/templates),
|
|
// the same shapes round-trip: strings stay strings, groups stay
|
|
// {any_of: [...]}.
|
|
type EnvRequirement struct {
|
|
// Name is non-empty for a single required env var.
|
|
Name string
|
|
// AnyOf is non-empty for an OR group; any one member satisfies.
|
|
AnyOf []string
|
|
}
|
|
|
|
// Members returns every env name this requirement considers —
|
|
// [Name] for single, AnyOf for groups. Used by preflight, collect,
|
|
// and the name-validation regex gate.
|
|
func (e EnvRequirement) Members() []string {
|
|
if e.Name != "" {
|
|
return []string{e.Name}
|
|
}
|
|
return e.AnyOf
|
|
}
|
|
|
|
// IsSatisfied reports whether any member of the requirement is
|
|
// present in `configured`. Single: exact-match. AnyOf: at least
|
|
// one hit.
|
|
func (e EnvRequirement) IsSatisfied(configured map[string]struct{}) bool {
|
|
for _, m := range e.Members() {
|
|
if _, ok := configured[m]; ok {
|
|
return true
|
|
}
|
|
}
|
|
return false
|
|
}
|
|
|
|
// UnmarshalYAML accepts either a scalar (string → single) or a map
|
|
// with an `any_of` list (→ group).
|
|
func (e *EnvRequirement) UnmarshalYAML(value *yaml.Node) error {
|
|
if value.Kind == yaml.ScalarNode {
|
|
var s string
|
|
if err := value.Decode(&s); err != nil {
|
|
return err
|
|
}
|
|
e.Name = s
|
|
return nil
|
|
}
|
|
var alt struct {
|
|
AnyOf []string `yaml:"any_of"`
|
|
}
|
|
if err := value.Decode(&alt); err != nil {
|
|
return fmt.Errorf("env requirement must be a string or {any_of: [...]}: %w", err)
|
|
}
|
|
if len(alt.AnyOf) == 0 {
|
|
return fmt.Errorf("env requirement any_of must contain at least one env var")
|
|
}
|
|
e.AnyOf = alt.AnyOf
|
|
return nil
|
|
}
|
|
|
|
// MarshalJSON emits the dual shape so GET /org/templates callers get
|
|
// {"required_env": ["GITHUB_TOKEN", {"any_of": [...]}]}, matching
|
|
// the YAML syntax.
|
|
func (e EnvRequirement) MarshalJSON() ([]byte, error) {
|
|
if e.Name != "" {
|
|
return json.Marshal(e.Name)
|
|
}
|
|
return json.Marshal(struct {
|
|
AnyOf []string `json:"any_of"`
|
|
}{AnyOf: e.AnyOf})
|
|
}
|
|
|
|
// UnmarshalJSON is the inverse — accepts the same dual shape so
|
|
// POST /org/import with an inline `template` body works too.
|
|
func (e *EnvRequirement) UnmarshalJSON(data []byte) error {
|
|
var s string
|
|
if err := json.Unmarshal(data, &s); err == nil {
|
|
e.Name = s
|
|
return nil
|
|
}
|
|
var alt struct {
|
|
AnyOf []string `json:"any_of"`
|
|
}
|
|
if err := json.Unmarshal(data, &alt); err != nil {
|
|
return fmt.Errorf("env requirement must be a string or {any_of: [...]}: %w", err)
|
|
}
|
|
if len(alt.AnyOf) == 0 {
|
|
return fmt.Errorf("env requirement any_of must contain at least one env var")
|
|
}
|
|
e.AnyOf = alt.AnyOf
|
|
return nil
|
|
}
|
|
|
|
// OrgTemplate is the YAML structure for an org hierarchy.
|
|
type OrgTemplate struct {
|
|
Name string `yaml:"name" json:"name"`
|
|
Description string `yaml:"description" json:"description"`
|
|
Defaults OrgDefaults `yaml:"defaults" json:"defaults"`
|
|
Workspaces []OrgWorkspace `yaml:"workspaces" json:"workspaces"`
|
|
// GlobalMemories is a list of org-wide memories seeded as GLOBAL scope
|
|
// on the first root workspace (PM) during org import. Issue #1050.
|
|
GlobalMemories []models.MemorySeed `yaml:"global_memories" json:"global_memories"`
|
|
// RequiredEnv lists env vars that MUST be configured globally (or
|
|
// on every workspace in the subtree that needs them) before import
|
|
// succeeds. Each entry is either a plain string (strict) or an
|
|
// {any_of: [...]} group (at least one member must be set). Declared
|
|
// at the org level for shared creds; also extensible per-workspace
|
|
// via OrgWorkspace.RequiredEnv for team-scoped credentials.
|
|
RequiredEnv []EnvRequirement `yaml:"required_env" json:"required_env"`
|
|
// RecommendedEnv is the "nice-to-have" tier — import still succeeds
|
|
// without them, but features degrade. Same single|any_of shape as
|
|
// RequiredEnv so a recommended OR group reads "set any one of these
|
|
// to unlock the feature; all missing = warning".
|
|
RecommendedEnv []EnvRequirement `yaml:"recommended_env" json:"recommended_env"`
|
|
}
|
|
|
|
type OrgDefaults struct {
|
|
Runtime string `yaml:"runtime" json:"runtime"`
|
|
Tier int `yaml:"tier" json:"tier"`
|
|
Model string `yaml:"model" json:"model"`
|
|
Plugins []string `yaml:"plugins" json:"plugins"`
|
|
InitialPrompt string `yaml:"initial_prompt" json:"initial_prompt"`
|
|
// InitialPromptFile is a file ref alternative to InitialPrompt. Path is
|
|
// resolved relative to the workspace's files_dir (or the org base dir
|
|
// when used at defaults level — defaults don't have their own files_dir,
|
|
// so the file must live at the org root). Inline InitialPrompt wins
|
|
// when both are set.
|
|
InitialPromptFile string `yaml:"initial_prompt_file" json:"initial_prompt_file"`
|
|
// IdlePrompt / IdleIntervalSeconds are the workspace-default idle-loop
|
|
// body and cadence (see workspace/heartbeat.py). They were
|
|
// previously dropped by the org importer because the struct didn't
|
|
// declare them — causing live configs to boot without idle_prompts
|
|
// even when org.yaml had them. Phase 1 scalability work adds both
|
|
// inline + file-ref forms.
|
|
IdlePrompt string `yaml:"idle_prompt" json:"idle_prompt"`
|
|
IdlePromptFile string `yaml:"idle_prompt_file" json:"idle_prompt_file"`
|
|
IdleIntervalSeconds int `yaml:"idle_interval_seconds" json:"idle_interval_seconds"`
|
|
// CategoryRouting maps issue/audit category → list of target roles.
|
|
// Per-workspace blocks UNION + override per-key with these defaults.
|
|
// Rendered into each workspace's config.yaml so agent prompts can read it
|
|
// generically (no hardcoded role names in prompts). See issue #51.
|
|
CategoryRouting map[string][]string `yaml:"category_routing" json:"category_routing"`
|
|
// InitialMemories are default memories seeded into every workspace at
|
|
// creation time unless the workspace overrides them. Issue #1050.
|
|
InitialMemories []models.MemorySeed `yaml:"initial_memories" json:"initial_memories"`
|
|
}
|
|
|
|
type OrgSchedule struct {
|
|
Name string `yaml:"name" json:"name"`
|
|
CronExpr string `yaml:"cron_expr" json:"cron_expr"`
|
|
Timezone string `yaml:"timezone" json:"timezone"`
|
|
Prompt string `yaml:"prompt" json:"prompt"`
|
|
// PromptFile is a file ref alternative to inline Prompt. Path is
|
|
// resolved relative to the workspace's files_dir. Inline Prompt wins
|
|
// when both are set. Scalability: hourly/weekly cron prompts are the
|
|
// largest text bodies in org.yaml (~1-5 KB each); externalizing them
|
|
// cuts the file by ~62%.
|
|
PromptFile string `yaml:"prompt_file" json:"prompt_file"`
|
|
Enabled *bool `yaml:"enabled" json:"enabled"`
|
|
}
|
|
|
|
// OrgChannel defines a social channel (Telegram, Slack, etc.) to auto-link
|
|
// when the workspace is created. Config values may reference env vars
|
|
// using ${VAR_NAME} syntax — useful for keeping bot tokens out of YAML.
|
|
type OrgChannel struct {
|
|
Type string `yaml:"type" json:"type"`
|
|
Config map[string]string `yaml:"config" json:"config"`
|
|
AllowedUsers []string `yaml:"allowed_users" json:"allowed_users"`
|
|
Enabled *bool `yaml:"enabled" json:"enabled"`
|
|
}
|
|
|
|
type OrgWorkspace struct {
|
|
Name string `yaml:"name" json:"name"`
|
|
Role string `yaml:"role" json:"role"`
|
|
Runtime string `yaml:"runtime" json:"runtime"`
|
|
Tier int `yaml:"tier" json:"tier"`
|
|
Template string `yaml:"template" json:"template"`
|
|
FilesDir string `yaml:"files_dir" json:"files_dir"`
|
|
// Spawning gates whether this workspace (AND its descendants) gets
|
|
// provisioned during /org/import. Pointer so we can distinguish
|
|
// "explicitly set to false" from "unset" (default = spawn). Use case:
|
|
// the dev-tree org template declares the full team structure but a
|
|
// developer's local machine only has RAM for a subset; setting
|
|
// spawning: false on a leaf or a sub-tree root skips that branch
|
|
// entirely without editing the canonical template structure.
|
|
// Counted in countWorkspaces same as actual; subtree-skip happens
|
|
// at provision time in createWorkspaceTree.
|
|
Spawning *bool `yaml:"spawning,omitempty" json:"spawning,omitempty"`
|
|
// SystemPrompt is an inline override. Normally each role's system-prompt.md
|
|
// lives at `<files_dir>/system-prompt.md` and is copied via the files_dir
|
|
// template-copy step; inline overrides that path for ad-hoc workspaces.
|
|
SystemPrompt string `yaml:"system_prompt" json:"system_prompt"`
|
|
Model string `yaml:"model" json:"model"`
|
|
WorkspaceDir string `yaml:"workspace_dir" json:"workspace_dir"`
|
|
WorkspaceAccess string `yaml:"workspace_access" json:"workspace_access"` // #65: "none" (default), "read_only", "read_write"
|
|
Plugins []string `yaml:"plugins" json:"plugins"`
|
|
// InitialPrompt is the one-shot boot prompt. Agents run this once on first
|
|
// start; the body often clones the repo, reads CLAUDE.md + system-prompt,
|
|
// and commits conventions to memory. InitialPromptFile is the file-ref
|
|
// alternative — read at import time from `<files_dir>/<InitialPromptFile>`.
|
|
// Inline wins when both are set.
|
|
InitialPrompt string `yaml:"initial_prompt" json:"initial_prompt"`
|
|
InitialPromptFile string `yaml:"initial_prompt_file" json:"initial_prompt_file"`
|
|
// IdlePrompt / IdleIntervalSeconds drive the idle-loop reflection
|
|
// pattern (#205). When IdlePrompt is non-empty, the workspace self-sends
|
|
// this prompt every IdleIntervalSeconds while heartbeat.active_tasks == 0.
|
|
// Both fields were previously dropped by the org importer (struct didn't
|
|
// declare them); Phase 1 scalability PR adds them so engineer + researcher
|
|
// idle loops propagate correctly from org.yaml → /configs/config.yaml.
|
|
// IdlePromptFile is the file-ref alternative — same semantics as
|
|
// InitialPromptFile. Inline wins when both are set.
|
|
IdlePrompt string `yaml:"idle_prompt" json:"idle_prompt"`
|
|
IdlePromptFile string `yaml:"idle_prompt_file" json:"idle_prompt_file"`
|
|
IdleIntervalSeconds int `yaml:"idle_interval_seconds" json:"idle_interval_seconds"`
|
|
// CategoryRouting extends/overrides defaults.category_routing per-workspace.
|
|
// Merge semantics: workspace keys replace defaults' value for the same key
|
|
// (empty list drops the category entirely); new keys are added. See
|
|
// mergeCategoryRouting.
|
|
CategoryRouting map[string][]string `yaml:"category_routing" json:"category_routing"`
|
|
// InitialMemories are memories seeded into this workspace at creation
|
|
// time. If empty, defaults.initial_memories are used. Issue #1050.
|
|
InitialMemories []models.MemorySeed `yaml:"initial_memories" json:"initial_memories"`
|
|
// MaxConcurrentTasks: see models.CreateWorkspacePayload.
|
|
MaxConcurrentTasks int `yaml:"max_concurrent_tasks" json:"max_concurrent_tasks"`
|
|
Schedules []OrgSchedule `yaml:"schedules" json:"schedules"`
|
|
Channels []OrgChannel `yaml:"channels" json:"channels"`
|
|
External bool `yaml:"external" json:"external"`
|
|
URL string `yaml:"url" json:"url"`
|
|
Canvas struct {
|
|
X float64 `yaml:"x" json:"x"`
|
|
Y float64 `yaml:"y" json:"y"`
|
|
} `yaml:"canvas" json:"canvas"`
|
|
// RequiredEnv / RecommendedEnv declared at the workspace level
|
|
// narrow down what a specific team needs beyond the org-wide union.
|
|
// When GET /org/templates walks the tree, these flow up into
|
|
// OrgTemplate.RequiredEnv / RecommendedEnv. A workspace's subtree
|
|
// inherits: a parent declaring ANTHROPIC_API_KEY as required
|
|
// means every descendant considers it required too (no override
|
|
// needed at each leaf). Same single|any_of shape as the org-level
|
|
// lists.
|
|
RequiredEnv []EnvRequirement `yaml:"required_env" json:"required_env"`
|
|
RecommendedEnv []EnvRequirement `yaml:"recommended_env" json:"recommended_env"`
|
|
Children []OrgWorkspace `yaml:"children" json:"children"`
|
|
}
|
|
|
|
// ListTemplates handles GET /org/templates — lists available org templates.
|
|
func (h *OrgHandler) ListTemplates(c *gin.Context) {
|
|
templates := []map[string]interface{}{}
|
|
|
|
entries, err := os.ReadDir(h.orgDir)
|
|
if err != nil {
|
|
c.JSON(http.StatusOK, templates)
|
|
return
|
|
}
|
|
|
|
for _, e := range entries {
|
|
if !e.IsDir() {
|
|
continue
|
|
}
|
|
// Look for org.yaml inside the directory
|
|
templateDir := filepath.Join(h.orgDir, e.Name())
|
|
orgFile := filepath.Join(templateDir, "org.yaml")
|
|
data, err := os.ReadFile(orgFile)
|
|
if err != nil {
|
|
// Try org.yml
|
|
orgFile = filepath.Join(templateDir, "org.yml")
|
|
data, err = os.ReadFile(orgFile)
|
|
if err != nil {
|
|
// Half-clone detection: a directory that contains a `.git/`
|
|
// but no `org.yaml`/`org.yml` is almost always a manifest
|
|
// clone that got truncated mid-checkout. Surfacing this as
|
|
// a warning instead of a silent skip prevents the
|
|
// "template missing from registry" failure mode (audit
|
|
// 2026-04-24: org-templates/molecule-dev/ had only `.git/`
|
|
// and silently dropped from the Canvas palette for hours
|
|
// before anyone noticed).
|
|
gitDir := filepath.Join(templateDir, ".git")
|
|
if _, gitErr := os.Stat(gitDir); gitErr == nil {
|
|
log.Printf("ListTemplates: WARNING %q has .git but no org.yaml/.yml — likely a half-checkout. Try 'cd %s && git checkout main -- .' to restore the working tree.", e.Name(), templateDir)
|
|
}
|
|
continue
|
|
}
|
|
}
|
|
// Expand !include directives before unmarshal so templates that
|
|
// split across team/role files still report an accurate workspace
|
|
// count on the /org/templates listing. Fail loudly on expansion
|
|
// errors — the previous silent-continue made a broken template
|
|
// show up as "no templates" in the Canvas palette with no log
|
|
// trail, which is how a fresh-clone user first discovers the gap.
|
|
if expanded, err := resolveYAMLIncludes(data, templateDir); err == nil {
|
|
data = expanded
|
|
} else {
|
|
log.Printf("ListTemplates: skipping %s — !include expansion failed: %v", e.Name(), err)
|
|
continue
|
|
}
|
|
var tmpl OrgTemplate
|
|
if err := yaml.Unmarshal(data, &tmpl); err != nil {
|
|
log.Printf("ListTemplates: skipping %s — yaml unmarshal failed: %v", e.Name(), err)
|
|
continue
|
|
}
|
|
count := countWorkspaces(tmpl.Workspaces)
|
|
// Walk the tree to collect required + recommended env union.
|
|
// Canvas uses these to render a preflight modal BEFORE firing
|
|
// the import — saves the user from a 15-workspace import that
|
|
// dies one container at a time on missing creds.
|
|
required, recommended := collectOrgEnv(&tmpl)
|
|
templates = append(templates, map[string]interface{}{
|
|
"dir": e.Name(),
|
|
"name": tmpl.Name,
|
|
"description": tmpl.Description,
|
|
"workspaces": count,
|
|
"required_env": required,
|
|
"recommended_env": recommended,
|
|
})
|
|
}
|
|
|
|
c.JSON(http.StatusOK, templates)
|
|
}
|
|
|
|
// Import handles POST /org/import — creates an entire org from a template.
|
|
func (h *OrgHandler) Import(c *gin.Context) {
|
|
var body struct {
|
|
Dir string `json:"dir"` // org template directory name
|
|
Template OrgTemplate `json:"template"` // or inline template
|
|
}
|
|
if err := c.ShouldBindJSON(&body); err != nil {
|
|
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request body"})
|
|
return
|
|
}
|
|
|
|
var tmpl OrgTemplate
|
|
var orgBaseDir string // base directory for files_dir resolution
|
|
|
|
if body.Dir != "" {
|
|
// Reject traversal attempts — `dir` must resolve inside h.orgDir.
|
|
// Without this, `dir: "../../../etc"` gets joined into h.orgDir and
|
|
// filepath.Join's lexical cleanup resolves it outside the root,
|
|
// letting an unauthenticated caller probe arbitrary filesystem paths.
|
|
resolved, err := resolveInsideRoot(h.orgDir, body.Dir)
|
|
if err != nil {
|
|
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid org directory"})
|
|
return
|
|
}
|
|
orgBaseDir = resolved
|
|
orgFile := filepath.Join(orgBaseDir, "org.yaml")
|
|
data, err := os.ReadFile(orgFile)
|
|
if err != nil {
|
|
c.JSON(http.StatusNotFound, gin.H{"error": fmt.Sprintf("org template not found: %s", body.Dir)})
|
|
return
|
|
}
|
|
// Expand !include directives before unmarshal. Splits org.yaml
|
|
// into per-team or per-role files; Phase 3 of the scalability
|
|
// refactor. Fails loudly on missing / cyclic / escaping includes.
|
|
expanded, err := resolveYAMLIncludes(data, orgBaseDir)
|
|
if err != nil {
|
|
c.JSON(http.StatusBadRequest, gin.H{"error": "org template expansion failed"})
|
|
return
|
|
}
|
|
if err := yaml.Unmarshal(expanded, &tmpl); err != nil {
|
|
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid org template"})
|
|
return
|
|
}
|
|
} else if body.Template.Name != "" {
|
|
tmpl = body.Template
|
|
} else {
|
|
c.JSON(http.StatusBadRequest, gin.H{"error": "provide 'dir' or 'template'"})
|
|
return
|
|
}
|
|
|
|
// Required-env preflight — refuses import when any required_env is
|
|
// missing from global_secrets. No bypass: the prior `force: true`
|
|
// escape hatch was removed (issue #2290) because it was the silent
|
|
// failure mode that let an org import without ANTHROPIC_API_KEY and
|
|
// ship workspaces that 401'd on every LLM call. The canvas runs the
|
|
// same check client-side against GET /org/templates output and shows
|
|
// a modal so users set keys before clicking Import; this server-side
|
|
// check is the authoritative guard in case a caller bypasses the UI
|
|
// (CLI, API clients, etc.). 412 Precondition Failed carries the
|
|
// missing-key list so tooling can render the same add-key flow.
|
|
required, _ := collectOrgEnv(&tmpl)
|
|
if len(required) > 0 {
|
|
ctx := c.Request.Context()
|
|
configured, err := loadConfiguredGlobalSecretKeys(ctx)
|
|
if err != nil {
|
|
// Fail closed. Previously this fell through and imported
|
|
// anyway, defeating the preflight for exactly the case
|
|
// it's meant to cover. A DB hiccup should look like a
|
|
// retryable 500, not a silent green light for an import
|
|
// that will fail at container-start time on every node.
|
|
log.Printf("Org import preflight: global secrets lookup failed: %v", err)
|
|
c.JSON(http.StatusInternalServerError, gin.H{
|
|
"error": "could not verify required environment variables; try again",
|
|
})
|
|
return
|
|
}
|
|
var missing []EnvRequirement
|
|
for _, req := range required {
|
|
// For a single requirement this is exact-match; for an
|
|
// any-of group, any one member satisfies. Groups whose
|
|
// alternative is already configured drop out here — the
|
|
// user doesn't need to re-configure them.
|
|
if !req.IsSatisfied(configured) {
|
|
missing = append(missing, req)
|
|
}
|
|
}
|
|
if len(missing) > 0 {
|
|
c.JSON(http.StatusPreconditionFailed, gin.H{
|
|
"error": "missing required environment variables",
|
|
"missing_env": missing,
|
|
"required_env": required,
|
|
"template": tmpl.Name,
|
|
"suggestion": "set these as global secrets (POST /settings/secrets) before importing",
|
|
})
|
|
return
|
|
}
|
|
}
|
|
|
|
results := []map[string]interface{}{}
|
|
var createErr error
|
|
|
|
// Semaphore limits concurrent provision goroutines (#1084).
|
|
// Cap is configurable via MOLECULE_PROVISION_CONCURRENCY:
|
|
// unset → 3 (Docker-mode default)
|
|
// "0" → effectively unlimited (SaaS / EC2 backend)
|
|
// N>0 → exactly N
|
|
// See resolveProvisionConcurrency for the full env-parse contract.
|
|
concurrency := resolveProvisionConcurrency()
|
|
provisionSem := make(chan struct{}, concurrency)
|
|
log.Printf("org_import: provision concurrency cap=%d (env MOLECULE_PROVISION_CONCURRENCY=%q)",
|
|
concurrency, os.Getenv("MOLECULE_PROVISION_CONCURRENCY"))
|
|
|
|
// Recursively create workspaces. Root workspaces keep their YAML
|
|
// canvas coords; children are positioned by createWorkspaceTree
|
|
// using subtree-aware grid slots (children that are themselves
|
|
// parents get a bigger slot so they don't overflow into siblings).
|
|
for _, ws := range tmpl.Workspaces {
|
|
// Root: relX/relY == absX/absY (no parent to be relative to).
|
|
if err := h.createWorkspaceTree(ws, nil, ws.Canvas.X, ws.Canvas.Y, ws.Canvas.X, ws.Canvas.Y, tmpl.Defaults, orgBaseDir, &results, provisionSem); err != nil {
|
|
createErr = err
|
|
break
|
|
}
|
|
}
|
|
|
|
// Seed org-wide global_memories on the first root workspace (issue #1050).
|
|
// These are GLOBAL scope memories visible to all workspaces in the org.
|
|
if len(tmpl.GlobalMemories) > 0 && len(results) > 0 {
|
|
rootID, _ := results[0]["id"].(string)
|
|
if rootID != "" {
|
|
rootNS := workspaceAwarenessNamespace(rootID)
|
|
// Force scope to GLOBAL regardless of what the YAML says.
|
|
globalSeeds := make([]models.MemorySeed, len(tmpl.GlobalMemories))
|
|
for i, gm := range tmpl.GlobalMemories {
|
|
globalSeeds[i] = models.MemorySeed{Content: gm.Content, Scope: "GLOBAL"}
|
|
}
|
|
seedInitialMemories(context.Background(), rootID, globalSeeds, rootNS)
|
|
log.Printf("Org import: seeded %d global memories on root workspace %s", len(globalSeeds), rootID)
|
|
}
|
|
}
|
|
|
|
// Hot-reload channel manager once after all channels are inserted
|
|
// (instead of per-workspace, avoiding N redundant DB queries + diffs).
|
|
if h.channelMgr != nil {
|
|
hasAnyChannels := false
|
|
for _, r := range results {
|
|
if _, ok := r["channels"]; ok {
|
|
hasAnyChannels = true
|
|
break
|
|
}
|
|
}
|
|
if hasAnyChannels {
|
|
h.channelMgr.Reload(context.Background())
|
|
}
|
|
}
|
|
|
|
status := http.StatusCreated
|
|
resp := gin.H{
|
|
"org": tmpl.Name,
|
|
"workspaces": results,
|
|
"count": len(results),
|
|
}
|
|
if createErr != nil {
|
|
status = http.StatusMultiStatus
|
|
resp["error"] = createErr.Error()
|
|
}
|
|
|
|
log.Printf("Org import: %s — %d workspaces created", tmpl.Name, len(results))
|
|
c.JSON(status, resp)
|
|
}
|
|
|