core-lead/molecule-core
1
0
Fork 0
forked from molecule-ai/molecule-core
Code Pull Requests Activity

feat(org-templates): Phase 1 — externalize prompt bodies to sibling files (#389)

Browse Source 
Part 1 of 4 in the scalability refactor. Each role can now keep its
initial_prompt / idle_prompt / schedule prompts as sibling .md files
under files_dir/; inline YAML literals still work for backwards-compat.

## What changes

**Platform (org.go importer):**
- `OrgWorkspace` gains `InitialPromptFile`, `IdlePrompt`, `IdlePromptFile`,
  `IdleIntervalSeconds`. The idle_* fields were previously dropped by the
  org importer entirely — struct didn't declare them — which is why
  engineer idle_prompts never propagated from org.yaml to live /configs
  (I've been manually docker-cp'ing them in every maintenance cron).
- `OrgSchedule` gains `PromptFile`. Hourly/weekly cron prompts are the
  largest bodies in org.yaml (1-5 KB each) and get resolved at import
  time just like initial_prompt.
- `OrgDefaults` gains the same idle_* + *_file fields for org-wide fallback.
- New `resolvePromptRef(inline, fileRef, orgBaseDir, filesDir)` helper —
  the single chokepoint for inline-vs-file resolution. Inline wins when
  both are set. File refs route through `resolveInsideRoot` so a crafted
  ref can't escape the org template directory (same traversal defense as
  files_dir).
- `createWorkspaceTree` now injects idle_prompt + idle_interval_seconds
  into the workspace's config.yaml (previously missing — that's the
  second half of the idle-prompt propagation bug).

**Tests:**
- `org_prompt_ref_test.go` — 10 cases: inline-wins, file-read-when-empty,
  both-empty, defaults-level resolution, inline-template mode errors,
  traversal rejection (via file ref AND via files_dir), missing-file
  errors, and YAML-unmarshal parsing for each new field.

**Proof migration:**
- Documentation Specialist (biggest role at 6.9 KB of prompts) moves from
  inline YAML to `documentation-specialist/{initial-prompt.md,
  schedules/daily-docs-sync.md, schedules/weekly-terminology-audit.md}`.
- org.yaml drops 1801 → 1687 lines (-6.3%) from just this one role.

## Why this matters

org.yaml is 108 KB of which 67 KB (62%) is prompt text. At the current
12-role template size that's already unreadable; the marketing + triage-
operator additions pushed it to 1801 lines. The 4-phase refactor aims:

- **Phase 1 (this PR):** platform support + 1 role proof.
- **Phase 2:** migrate remaining ~20 roles to file refs. Target: org.yaml
  at ~600 lines of pure structural scaffolding.
- **Phase 3:** YAML `!include` preprocessor — split org.yaml into
  teams/{research,dev,marketing,ops}.yaml shards.
- **Phase 4:** per-workspace atomization — each role gets its own
  workspace.yaml manifest; org.yaml composes them.

## Backwards compatibility

- Inline `initial_prompt: |` / `prompt: |` / `idle_prompt: |` all still work.
- Missing `prompt_file` refs log + skip the schedule (not fatal) — fail
  loud so bugs surface during deployment rather than silent-drop.
- Inline-template mode (POST /org/import with raw JSON body, no `dir`)
  errors cleanly when a file ref is used — can't resolve files without a
  base dir, surface that rather than guessing.

## Test plan

- [x] `go build ./...` clean
- [x] `go test -run 'TestResolvePromptRef|TestOrgYAML' ./internal/handlers/`
      — 10 tests pass
- [x] `python -c "yaml.safe_load(...)"` on the edited org.yaml — parses
- [ ] Post-merge: deploy platform rebuild, run `POST /org/import` against
      a fresh workspace, verify Documentation Specialist's /configs/config.yaml
      contains the initial_prompt body and workspace_schedules rows contain
      the cron prompts (phantom-success check: grep the actual content, not
      just the row count).

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Hongming Wang 2026-04-16 00:32:09 -07:00 committed by GitHub
parent bab145b520
commit ce0e793673
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
6 changed files with 549 additions and 171 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

36
org-templates/molecule-dev/documentation-specialist/initial-prompt.md Normal file
View File

@ -0,0 +1,36 @@
You just started as Documentation Specialist. Set up silently — do NOT contact other agents.
⚠️ PRIVACY RULE (read first, never violate):
molecule-controlplane is a PRIVATE repo. Its source code, file paths,
internal endpoints, schema details, infra config, billing/auth
implementation — none of that goes into the public docs site
(Molecule-AI/docs) or the public README in molecule-monorepo. Public
docs may describe the SaaS PRODUCT (signup, billing, tenant isolation
guarantees) but never the provisioner's internals. When in doubt:
don't publish.
1. Clone all three repos:
git clone https://github.com/${GITHUB_REPO}.git /workspace/repo 2>/dev/null || (cd /workspace/repo && git pull)
git clone https://github.com/Molecule-AI/docs.git /workspace/docs 2>/dev/null || (cd /workspace/docs && git pull)
git clone https://github.com/Molecule-AI/molecule-controlplane.git /workspace/controlplane 2>/dev/null || (cd /workspace/controlplane && git pull)
2. Read /workspace/repo/CLAUDE.md — full architecture, what's public-facing
3. Read /configs/system-prompt.md
4. Read /workspace/docs/README.md and /workspace/docs/content/docs/index.mdx
5. Read /workspace/controlplane/README.md and /workspace/controlplane/PLAN.md
— understand what the SaaS provisioner does (private) vs what users see (public)
6. Run: cd /workspace/docs && ls content/docs/*.mdx
— note which pages are stubs ("Coming soon" marker) vs hand-written
7. Run: cd /workspace/repo && git log --oneline -20 -- platform/internal/handlers/ org-templates/ plugins/
— note recent public-surface changes in the platform repo
8. Run: cd /workspace/controlplane && git log --oneline -20
— note recent controlplane changes (these need internal docs only)
9. Use commit_memory to save:
- Stubs that need backfilling (docs site)
- Recent platform PRs that have NO docs PR yet
- Recent controlplane PRs whose internal README needs an update
- Public concepts that lack a canonical naming entry
10. Wait for tasks from PM. Your owned surfaces are:
- https://github.com/Molecule-AI/docs (customer site, Fumadocs) — PUBLIC
- /workspace/repo/docs/ (internal architecture / edit-history) — PUBLIC
- /workspace/repo/README.md and per-package READMEs — PUBLIC
- /workspace/controlplane/README.md, PLAN.md, internal docs — PRIVATE

74
org-templates/molecule-dev/documentation-specialist/schedules/daily-docs-sync.md Normal file
View File

@ -0,0 +1,74 @@
Daily documentation maintenance. Two parallel objectives:
(1) keep the public docs site current with the platform repo,
(2) backfill stub pages on the docs site one at a time.
SETUP:
cd /workspace/repo && git pull 2>/dev/null || true
cd /workspace/docs && git pull 2>/dev/null || true
cd /workspace/controlplane && git pull 2>/dev/null || true
1a. PAIR RECENT PLATFORM PRS (last 24h):
cd /workspace/repo
gh pr list --repo Molecule-AI/molecule-monorepo --state merged \
--search "merged:>$(date -u -d '24 hours ago' +%Y-%m-%dT%H:%M:%SZ)" \
--json number,title,files
For each merged PR that touches a public surface
(platform/internal/handlers/, plugins/*, org-templates/*,
docs/architecture.md, README.md, workspace-template/adapters/*):
- Identify which docs page(s) on the public site cover that surface.
- If a docs page exists but is stale → update it with examples
from the PR diff. Open a PR to Molecule-AI/docs with the change.
- If NO docs page exists for the new surface → propose one
(add to content/docs/meta.json + new .mdx file). Open a PR.
- Always close PRs with `Closes platform PR #N` so the link is durable.
1b. PAIR RECENT CONTROLPLANE PRS (last 24h):
cd /workspace/controlplane
gh pr list --repo Molecule-AI/molecule-controlplane --state merged \
--search "merged:>$(date -u -d '24 hours ago' +%Y-%m-%dT%H:%M:%SZ)" \
--json number,title,files
⚠️ PRIVATE REPO. Two cases:
(i) Internal-only change (handler, schema, infra, fly.toml,
billing logic): update README.md + PLAN.md + any
docs/internal/*.md inside molecule-controlplane itself.
Open the PR against Molecule-AI/molecule-controlplane.
NEVER mention these changes in /workspace/docs.
(ii) Customer-facing change (new tier, new region, new SLA,
pricing change, signup flow change): write a sanitized
description for the PUBLIC docs site (e.g. "We now offer
EU-region tenants" — NOT "controlplane reads FLY_REGION
from env and passes it to provisioner.go:142"). Open a
PR against Molecule-AI/docs.
When unsure which category a change falls into: default to
INTERNAL-only and ask PM for explicit approval before publishing.
2. BACKFILL ONE STUB PAGE:
cd /workspace/docs
grep -l "Coming soon" content/docs/*.mdx | head -1
Pick the highest-priority stub (one of: org-template, plugins,
channels, schedules, architecture, api-reference, self-hosting,
observability, troubleshooting). Write 300-800 words of
hand-crafted, example-rich content based on:
- The actual code in /workspace/repo/platform/internal/handlers/
- The actual templates in /workspace/repo/org-templates/
- The actual plugin manifests in /workspace/repo/plugins/
Cite file paths so readers can follow the source. Open a PR.
3. LINK + ANCHOR CHECK:
Use the browser-automation plugin to crawl
https://doc.moleculesai.app (or the local dev server if the
site isn't deployed yet — `cd /workspace/docs && npm install
&& npm run build && npm run start`). Report broken links and
missing anchors back to PM.
4. ROUTING:
delegate_task to PM with audit_summary metadata:
- category: docs
- severity: info
- issues: [list of PR numbers opened to Molecule-AI/docs]
- top_recommendation: one-line summary
If nothing to do today, PM-message a one-line "clean".
5. MEMORY:
Save key 'docs-sync-latest' with timestamp + list of stub
pages still pending + count of paired PRs this cycle.

28
org-templates/molecule-dev/documentation-specialist/schedules/weekly-terminology-audit.md Normal file
View File

@ -0,0 +1,28 @@
Weekly audit of documentation freshness and terminology consistency.
1. STALE PAGE DETECTION:
cd /workspace/docs && for f in content/docs/*.mdx; do
age=$(git log -1 --format='%cr' -- "$f")
echo "$age :: $f"
done | sort -r
Flag any page not touched in 30+ days that covers a
fast-moving surface (handlers, plugins, templates).
2. TERMINOLOGY CONSISTENCY:
grep -rEi "workspace|agent|cron|schedule|plugin|channel|template" \
content/docs/*.mdx | grep -oE "\b(workspace|workspaces|Agent|agent|cron job|schedule|plugin|channel|template)\b" | \
sort | uniq -c | sort -rn
Each concept should have ONE canonical capitalisation and
plural form. Open a PR fixing inconsistencies.
3. LINK ROT:
grep -rE "\[.*\]\(http[^)]+\)" content/docs/*.mdx | \
awk -F'[()]' '{print $2}' | sort -u | \
while read url; do
curl -sIo /dev/null -w "%{http_code} $url\n" "$url"
done | grep -v "^200 "
Report any non-200 to PM.
4. ROUTING + MEMORY:
Same audit_summary contract as the daily cron.
Save findings to memory key 'docs-weekly-audit'.

150
org-templates/molecule-dev/org.yaml
View File

@ -1150,153 +1150,21 @@ workspaces:
# docs site (visual regressions, broken links, dead anchors) plus
# update-docs skill (already in defaults) for cross-repo docs sync.
plugins: [browser-automation]
initial_prompt: |
You just started as Documentation Specialist. Set up silently — do NOT contact other agents.
⚠️ PRIVACY RULE (read first, never violate):
molecule-controlplane is a PRIVATE repo. Its source code, file paths,
internal endpoints, schema details, infra config, billing/auth
implementation — none of that goes into the public docs site
(Molecule-AI/docs) or the public README in molecule-monorepo. Public
docs may describe the SaaS PRODUCT (signup, billing, tenant isolation
guarantees) but never the provisioner's internals. When in doubt:
don't publish.
1. Clone all three repos:
git clone https://github.com/${GITHUB_REPO}.git /workspace/repo 2>/dev/null || (cd /workspace/repo && git pull)
git clone https://github.com/Molecule-AI/docs.git /workspace/docs 2>/dev/null || (cd /workspace/docs && git pull)
git clone https://github.com/Molecule-AI/molecule-controlplane.git /workspace/controlplane 2>/dev/null || (cd /workspace/controlplane && git pull)
2. Read /workspace/repo/CLAUDE.md — full architecture, what's public-facing
3. Read /configs/system-prompt.md
4. Read /workspace/docs/README.md and /workspace/docs/content/docs/index.mdx
5. Read /workspace/controlplane/README.md and /workspace/controlplane/PLAN.md
— understand what the SaaS provisioner does (private) vs what users see (public)
6. Run: cd /workspace/docs && ls content/docs/*.mdx
— note which pages are stubs ("Coming soon" marker) vs hand-written
7. Run: cd /workspace/repo && git log --oneline -20 -- platform/internal/handlers/ org-templates/ plugins/
— note recent public-surface changes in the platform repo
8. Run: cd /workspace/controlplane && git log --oneline -20
— note recent controlplane changes (these need internal docs only)
9. Use commit_memory to save:
- Stubs that need backfilling (docs site)
- Recent platform PRs that have NO docs PR yet
- Recent controlplane PRs whose internal README needs an update
- Public concepts that lack a canonical naming entry
10. Wait for tasks from PM. Your owned surfaces are:
- https://github.com/Molecule-AI/docs (customer site, Fumadocs) — PUBLIC
- /workspace/repo/docs/ (internal architecture / edit-history) — PUBLIC
- /workspace/repo/README.md and per-package READMEs — PUBLIC
- /workspace/controlplane/README.md, PLAN.md, internal docs — PRIVATE
# Phase 1 scalability: prompts externalized to sibling .md files.
# See documentation-specialist/{initial-prompt.md, schedules/*.md}.
# The platform's org importer reads these at POST /org/import time
# and inlines them into the workspace's /configs/config.yaml and
# workspace_schedules rows. Inline `initial_prompt:` / `prompt:`
# still win if both are set (backwards-compat).
initial_prompt_file: initial-prompt.md
schedules:
- name: Daily docs sync — backfill stubs and pair recent platform PRs
cron_expr: "0 9 * * *"
prompt: |
Daily documentation maintenance. Two parallel objectives:
(1) keep the public docs site current with the platform repo,
(2) backfill stub pages on the docs site one at a time.
SETUP:
cd /workspace/repo && git pull 2>/dev/null || true
cd /workspace/docs && git pull 2>/dev/null || true
cd /workspace/controlplane && git pull 2>/dev/null || true
1a. PAIR RECENT PLATFORM PRS (last 24h):
cd /workspace/repo
gh pr list --repo Molecule-AI/molecule-monorepo --state merged \
--search "merged:>$(date -u -d '24 hours ago' +%Y-%m-%dT%H:%M:%SZ)" \
--json number,title,files
For each merged PR that touches a public surface
(platform/internal/handlers/, plugins/*, org-templates/*,
docs/architecture.md, README.md, workspace-template/adapters/*):
- Identify which docs page(s) on the public site cover that surface.
- If a docs page exists but is stale → update it with examples
from the PR diff. Open a PR to Molecule-AI/docs with the change.
- If NO docs page exists for the new surface → propose one
(add to content/docs/meta.json + new .mdx file). Open a PR.
- Always close PRs with `Closes platform PR #N` so the link is durable.
1b. PAIR RECENT CONTROLPLANE PRS (last 24h):
cd /workspace/controlplane
gh pr list --repo Molecule-AI/molecule-controlplane --state merged \
--search "merged:>$(date -u -d '24 hours ago' +%Y-%m-%dT%H:%M:%SZ)" \
--json number,title,files
⚠️ PRIVATE REPO. Two cases:
(i) Internal-only change (handler, schema, infra, fly.toml,
billing logic): update README.md + PLAN.md + any
docs/internal/*.md inside molecule-controlplane itself.
Open the PR against Molecule-AI/molecule-controlplane.
NEVER mention these changes in /workspace/docs.
(ii) Customer-facing change (new tier, new region, new SLA,
pricing change, signup flow change): write a sanitized
description for the PUBLIC docs site (e.g. "We now offer
EU-region tenants" — NOT "controlplane reads FLY_REGION
from env and passes it to provisioner.go:142"). Open a
PR against Molecule-AI/docs.
When unsure which category a change falls into: default to
INTERNAL-only and ask PM for explicit approval before publishing.
2. BACKFILL ONE STUB PAGE:
cd /workspace/docs
grep -l "Coming soon" content/docs/*.mdx | head -1
Pick the highest-priority stub (one of: org-template, plugins,
channels, schedules, architecture, api-reference, self-hosting,
observability, troubleshooting). Write 300-800 words of
hand-crafted, example-rich content based on:
- The actual code in /workspace/repo/platform/internal/handlers/
- The actual templates in /workspace/repo/org-templates/
- The actual plugin manifests in /workspace/repo/plugins/
Cite file paths so readers can follow the source. Open a PR.
3. LINK + ANCHOR CHECK:
Use the browser-automation plugin to crawl
https://doc.moleculesai.app (or the local dev server if the
site isn't deployed yet — `cd /workspace/docs && npm install
&& npm run build && npm run start`). Report broken links and
missing anchors back to PM.
4. ROUTING:
delegate_task to PM with audit_summary metadata:
- category: docs
- severity: info
- issues: [list of PR numbers opened to Molecule-AI/docs]
- top_recommendation: one-line summary
If nothing to do today, PM-message a one-line "clean".
5. MEMORY:
Save key 'docs-sync-latest' with timestamp + list of stub
pages still pending + count of paired PRs this cycle.
prompt_file: schedules/daily-docs-sync.md
enabled: true
- name: Weekly terminology + freshness audit
cron_expr: "0 11 * * 1"
prompt: |
Weekly audit of documentation freshness and terminology consistency.
1. STALE PAGE DETECTION:
cd /workspace/docs && for f in content/docs/*.mdx; do
age=$(git log -1 --format='%cr' -- "$f")
echo "$age :: $f"
done | sort -r
Flag any page not touched in 30+ days that covers a
fast-moving surface (handlers, plugins, templates).
2. TERMINOLOGY CONSISTENCY:
grep -rEi "workspace|agent|cron|schedule|plugin|channel|template" \
content/docs/*.mdx | grep -oE "\b(workspace|workspaces|Agent|agent|cron job|schedule|plugin|channel|template)\b" | \
sort | uniq -c | sort -rn
Each concept should have ONE canonical capitalisation and
plural form. Open a PR fixing inconsistencies.
3. LINK ROT:
grep -rE "\\[.*\\]\\(http[^)]+\\)" content/docs/*.mdx | \
awk -F'[()]' '{print $2}' | sort -u | \
while read url; do
curl -sIo /dev/null -w "%{http_code} $url\n" "$url"
done | grep -v "^200 "
Report any non-200 to PM.
4. ROUTING + MEMORY:
Same audit_summary contract as the daily cron.
Save findings to memory key 'docs-weekly-audit'.
prompt_file: schedules/weekly-terminology-audit.md
enabled: true
- name: Triage Operator

211
platform/internal/handlers/org.go
View File

@ -81,11 +81,26 @@ type OrgTemplate struct {
}
type OrgDefaults struct {
Runtime string `yaml:"runtime" json:"runtime"`
Tier int `yaml:"tier" json:"tier"`
Model string `yaml:"model" json:"model"`
Plugins []string `yaml:"plugins" json:"plugins"`
InitialPrompt string `yaml:"initial_prompt" json:"initial_prompt"`
Runtime string `yaml:"runtime" json:"runtime"`
Tier int `yaml:"tier" json:"tier"`
Model string `yaml:"model" json:"model"`
Plugins []string `yaml:"plugins" json:"plugins"`
InitialPrompt string `yaml:"initial_prompt" json:"initial_prompt"`
// InitialPromptFile is a file ref alternative to InitialPrompt. Path is
// resolved relative to the workspace's files_dir (or the org base dir
// when used at defaults level — defaults don't have their own files_dir,
// so the file must live at the org root). Inline InitialPrompt wins
// when both are set.
InitialPromptFile string `yaml:"initial_prompt_file" json:"initial_prompt_file"`
// IdlePrompt / IdleIntervalSeconds are the workspace-default idle-loop
// body and cadence (see workspace-template/heartbeat.py). They were
// previously dropped by the org importer because the struct didn't
// declare them — causing live configs to boot without idle_prompts
// even when org.yaml had them. Phase 1 scalability work adds both
// inline + file-ref forms.
IdlePrompt string `yaml:"idle_prompt" json:"idle_prompt"`
IdlePromptFile string `yaml:"idle_prompt_file" json:"idle_prompt_file"`
IdleIntervalSeconds int `yaml:"idle_interval_seconds" json:"idle_interval_seconds"`
// CategoryRouting maps issue/audit category → list of target roles.
// Per-workspace blocks UNION + override per-key with these defaults.
// Rendered into each workspace's config.yaml so agent prompts can read it
@ -98,7 +113,13 @@ type OrgSchedule struct {
CronExpr string `yaml:"cron_expr" json:"cron_expr"`
Timezone string `yaml:"timezone" json:"timezone"`
Prompt string `yaml:"prompt" json:"prompt"`
Enabled *bool `yaml:"enabled" json:"enabled"`
// PromptFile is a file ref alternative to inline Prompt. Path is
// resolved relative to the workspace's files_dir. Inline Prompt wins
// when both are set. Scalability: hourly/weekly cron prompts are the
// largest text bodies in org.yaml (~1-5 KB each); externalizing them
// cuts the file by ~62%.
PromptFile string `yaml:"prompt_file" json:"prompt_file"`
Enabled *bool `yaml:"enabled" json:"enabled"`
}
// OrgChannel defines a social channel (Telegram, Slack, etc.) to auto-link
@ -112,32 +133,52 @@ type OrgChannel struct {
}
type OrgWorkspace struct {
Name string `yaml:"name" json:"name"`
Role string `yaml:"role" json:"role"`
Runtime string `yaml:"runtime" json:"runtime"`
Tier int `yaml:"tier" json:"tier"`
Template string `yaml:"template" json:"template"`
FilesDir string `yaml:"files_dir" json:"files_dir"`
SystemPrompt string `yaml:"system_prompt" json:"system_prompt"`
Model string `yaml:"model" json:"model"`
WorkspaceDir string `yaml:"workspace_dir" json:"workspace_dir"`
WorkspaceAccess string `yaml:"workspace_access" json:"workspace_access"` // #65: "none" (default), "read_only", "read_write"
Plugins []string `yaml:"plugins" json:"plugins"`
InitialPrompt string `yaml:"initial_prompt" json:"initial_prompt"`
Name string `yaml:"name" json:"name"`
Role string `yaml:"role" json:"role"`
Runtime string `yaml:"runtime" json:"runtime"`
Tier int `yaml:"tier" json:"tier"`
Template string `yaml:"template" json:"template"`
FilesDir string `yaml:"files_dir" json:"files_dir"`
// SystemPrompt is an inline override. Normally each role's system-prompt.md
// lives at `<files_dir>/system-prompt.md` and is copied via the files_dir
// template-copy step; inline overrides that path for ad-hoc workspaces.
SystemPrompt string `yaml:"system_prompt" json:"system_prompt"`
Model string `yaml:"model" json:"model"`
WorkspaceDir string `yaml:"workspace_dir" json:"workspace_dir"`
WorkspaceAccess string `yaml:"workspace_access" json:"workspace_access"` // #65: "none" (default), "read_only", "read_write"
Plugins []string `yaml:"plugins" json:"plugins"`
// InitialPrompt is the one-shot boot prompt. Agents run this once on first
// start; the body often clones the repo, reads CLAUDE.md + system-prompt,
// and commits conventions to memory. InitialPromptFile is the file-ref
// alternative — read at import time from `<files_dir>/<InitialPromptFile>`.
// Inline wins when both are set.
InitialPrompt string `yaml:"initial_prompt" json:"initial_prompt"`
InitialPromptFile string `yaml:"initial_prompt_file" json:"initial_prompt_file"`
// IdlePrompt / IdleIntervalSeconds drive the idle-loop reflection
// pattern (#205). When IdlePrompt is non-empty, the workspace self-sends
// this prompt every IdleIntervalSeconds while heartbeat.active_tasks == 0.
// Both fields were previously dropped by the org importer (struct didn't
// declare them); Phase 1 scalability PR adds them so engineer + researcher
// idle loops propagate correctly from org.yaml → /configs/config.yaml.
// IdlePromptFile is the file-ref alternative — same semantics as
// InitialPromptFile. Inline wins when both are set.
IdlePrompt string `yaml:"idle_prompt" json:"idle_prompt"`
IdlePromptFile string `yaml:"idle_prompt_file" json:"idle_prompt_file"`
IdleIntervalSeconds int `yaml:"idle_interval_seconds" json:"idle_interval_seconds"`
// CategoryRouting extends/overrides defaults.category_routing per-workspace.
// Merge semantics: workspace keys replace defaults' value for the same key
// (empty list drops the category entirely); new keys are added. See
// mergeCategoryRouting.
CategoryRouting map[string][]string `yaml:"category_routing" json:"category_routing"`
Schedules []OrgSchedule `yaml:"schedules" json:"schedules"`
Channels []OrgChannel `yaml:"channels" json:"channels"`
External bool `yaml:"external" json:"external"`
URL string `yaml:"url" json:"url"`
Canvas struct {
Schedules []OrgSchedule `yaml:"schedules" json:"schedules"`
Channels []OrgChannel `yaml:"channels" json:"channels"`
External bool `yaml:"external" json:"external"`
URL string `yaml:"url" json:"url"`
Canvas struct {
X float64 `yaml:"x" json:"x"`
Y float64 `yaml:"y" json:"y"`
} `yaml:"canvas" json:"canvas"`
Children []OrgWorkspace `yaml:"children" json:"children"`
Children []OrgWorkspace `yaml:"children" json:"children"`
}
// ListTemplates handles GET /org/templates — lists available org templates.
@ -437,10 +478,21 @@ func (h *OrgHandler) createWorkspaceTree(ws OrgWorkspace, parentID *string, defa
}
}
// Inject initial_prompt into config.yaml (workspace-level overrides default)
initialPrompt := ws.InitialPrompt
// Resolve initial_prompt — inline wins, then file-ref, then defaults
// (inline → file → defaults.inline → defaults.file). File refs are
// rooted at <orgBaseDir>/<files_dir>/ per resolvePromptRef semantics.
initialPrompt, err := resolvePromptRef(ws.InitialPrompt, ws.InitialPromptFile, orgBaseDir, ws.FilesDir)
if err != nil {
log.Printf("Org import: failed to resolve initial_prompt for %s: %v", ws.Name, err)
}
if initialPrompt == "" {
initialPrompt = defaults.InitialPrompt
// Fall back to defaults. Defaults live at the org root, so they
// resolve with empty filesDir (relative to orgBaseDir).
var defaultErr error
initialPrompt, defaultErr = resolvePromptRef(defaults.InitialPrompt, defaults.InitialPromptFile, orgBaseDir, "")
if defaultErr != nil {
log.Printf("Org import: failed to resolve defaults.initial_prompt for %s: %v", ws.Name, defaultErr)
}
}
if initialPrompt != "" {
if configFiles == nil {
@ -458,6 +510,46 @@ func (h *OrgHandler) createWorkspaceTree(ws OrgWorkspace, parentID *string, defa
log.Printf("Org import: injected initial_prompt (%d chars) into config.yaml for %s", len(trimmed), ws.Name)
}
// Resolve idle_prompt — same precedence (ws inline → ws file → defaults).
// Inject into config.yaml alongside idle_interval_seconds so the
// workspace's heartbeat loop picks up the idle-reflection cadence on
// boot (see workspace-template/heartbeat.py + config.py).
idlePrompt, err := resolvePromptRef(ws.IdlePrompt, ws.IdlePromptFile, orgBaseDir, ws.FilesDir)
if err != nil {
log.Printf("Org import: failed to resolve idle_prompt for %s: %v", ws.Name, err)
}
if idlePrompt == "" {
var defaultErr error
idlePrompt, defaultErr = resolvePromptRef(defaults.IdlePrompt, defaults.IdlePromptFile, orgBaseDir, "")
if defaultErr != nil {
log.Printf("Org import: failed to resolve defaults.idle_prompt for %s: %v", ws.Name, defaultErr)
}
}
idleInterval := ws.IdleIntervalSeconds
if idleInterval == 0 {
idleInterval = defaults.IdleIntervalSeconds
}
if idlePrompt != "" {
if configFiles == nil {
configFiles = map[string][]byte{}
}
trimmed := strings.TrimSpace(idlePrompt)
lines := strings.Split(trimmed, "\n")
for i, line := range lines {
lines[i] = strings.TrimRight(line, " \t")
}
indented := strings.Join(lines, "\n ")
// idle_interval_seconds belongs with idle_prompt — empty idle_prompt
// means the idle loop never fires regardless of interval, so we
// only emit interval when there's a body to go with it.
if idleInterval <= 0 {
idleInterval = 600 // same default as workspace-template/config.py
}
block := fmt.Sprintf("idle_interval_seconds: %d\nidle_prompt: |\n %s\n", idleInterval, indented)
configFiles["config.yaml"] = appendYAMLBlock(configFiles["config.yaml"], block)
log.Printf("Org import: injected idle_prompt (%d chars, interval=%ds) into config.yaml for %s", len(trimmed), idleInterval, ws.Name)
}
// Inline system_prompt (only if no files_dir provides one)
if ws.SystemPrompt != "" {
if configFiles == nil {
@ -503,7 +595,10 @@ func (h *OrgHandler) createWorkspaceTree(ws OrgWorkspace, parentID *string, defa
go h.workspace.provisionWorkspace(id, templatePath, configFiles, payload)
}
// Insert schedules if defined
// Insert schedules if defined. Resolve each schedule's prompt body from
// either inline `prompt:` or `prompt_file:` (file ref relative to the
// workspace's files_dir). Inline wins; empty prompt after resolution is
// a configuration error (cron with no body would never do anything).
for _, sched := range ws.Schedules {
tz := sched.Timezone
if tz == "" {
@ -513,12 +608,21 @@ func (h *OrgHandler) createWorkspaceTree(ws OrgWorkspace, parentID *string, defa
if sched.Enabled != nil {
enabled = *sched.Enabled
}
prompt, promptErr := resolvePromptRef(sched.Prompt, sched.PromptFile, orgBaseDir, ws.FilesDir)
if promptErr != nil {
log.Printf("Org import: failed to resolve prompt for schedule '%s' on %s: %v — skipping insert", sched.Name, ws.Name, promptErr)
continue
}
if prompt == "" {
log.Printf("Org import: schedule '%s' on %s has empty prompt (neither prompt nor prompt_file set) — skipping insert", sched.Name, ws.Name)
continue
}
nextRun, _ := scheduler.ComputeNextRun(sched.CronExpr, tz, time.Now())
if _, err := db.DB.ExecContext(context.Background(), orgImportScheduleSQL,
id, sched.Name, sched.CronExpr, tz, sched.Prompt, enabled, nextRun); err != nil {
id, sched.Name, sched.CronExpr, tz, prompt, enabled, nextRun); err != nil {
log.Printf("Org import: failed to upsert schedule '%s' for %s: %v", sched.Name, ws.Name, err)
} else {
log.Printf("Org import: schedule '%s' (%s) upserted for %s (source=template)", sched.Name, sched.CronExpr, ws.Name)
log.Printf("Org import: schedule '%s' (%s, %d chars) upserted for %s (source=template)", sched.Name, sched.CronExpr, len(prompt), ws.Name)
}
}
@ -638,6 +742,53 @@ func countWorkspaces(workspaces []OrgWorkspace) int {
return count
}
// resolvePromptRef reads a prompt body from either an inline string or a
// file ref relative to the workspace's files_dir. Inline always wins when
// both are non-empty (caller-provided inline is more authoritative than a
// file path that may not exist yet during dev loops).
//
// File resolution:
// - `<orgBaseDir>/<filesDir>/<fileRef>` when filesDir is non-empty
// - `<orgBaseDir>/<fileRef>` when filesDir is empty (defaults-level refs)
//
// Both paths go through resolveInsideRoot so a crafted fileRef can't escape
// the org template directory via traversal (same defense the files_dir
// copy-step uses).
//
// Returns (resolved body, error). If both inline and fileRef are empty,
// returns ("", nil) — caller decides whether that's a problem.
func resolvePromptRef(inline, fileRef, orgBaseDir, filesDir string) (string, error) {
if inline != "" {
return inline, nil
}
if fileRef == "" {
return "", nil
}
if orgBaseDir == "" {
// Inline-only template (POST /org/import with a raw Template in the
// JSON body, not a dir). File refs can't be resolved — surface the
// problem rather than silently returning empty.
return "", fmt.Errorf("prompt_file %q requires a dir-based org template (no orgBaseDir in inline-template mode)", fileRef)
}
searchRoot := orgBaseDir
if filesDir != "" {
p, err := resolveInsideRoot(orgBaseDir, filesDir)
if err != nil {
return "", fmt.Errorf("invalid files_dir %q: %w", filesDir, err)
}
searchRoot = p
}
abs, err := resolveInsideRoot(searchRoot, fileRef)
if err != nil {
return "", fmt.Errorf("invalid prompt_file %q: %w", fileRef, err)
}
data, err := os.ReadFile(abs)
if err != nil {
return "", fmt.Errorf("read prompt_file %q: %w", fileRef, err)
}
return string(data), nil
}
// envVarRefPattern matches actual ${VAR} or $VAR references (not literal $).
// Used to detect unresolved placeholders without false positives like "$5".
var envVarRefPattern = regexp.MustCompile(`\$\{?[A-Za-z_][A-Za-z0-9_]*\}?`)

221
platform/internal/handlers/org_prompt_ref_test.go Normal file
View File

@ -0,0 +1,221 @@
package handlers
import (
"os"
"path/filepath"
"strings"
"testing"
"gopkg.in/yaml.v3"
)
// YAML parsing coverage for the new phase-1 scalability fields.
// Catches regressions where someone renames a struct tag and the YAML
// loader silently drops the value (what the prior idle_prompt bug was —
// the struct simply didn't declare the field, so org.yaml entries were
// dropped on import).
func TestOrgYAML_IdlePromptFieldsParse(t *testing.T) {
src := `
name: Test Org
workspaces:
- name: Role A
files_dir: role-a
idle_prompt: "inline idle body"
idle_interval_seconds: 300
- name: Role B
files_dir: role-b
idle_prompt_file: idle-prompt.md
idle_interval_seconds: 600
`
var tmpl OrgTemplate
if err := yaml.Unmarshal([]byte(src), &tmpl); err != nil {
t.Fatalf("unmarshal failed: %v", err)
}
if len(tmpl.Workspaces) != 2 {
t.Fatalf("expected 2 workspaces, got %d", len(tmpl.Workspaces))
}
a := tmpl.Workspaces[0]
if a.IdlePrompt != "inline idle body" {
t.Errorf("idle_prompt inline not parsed: %q", a.IdlePrompt)
}
if a.IdleIntervalSeconds != 300 {
t.Errorf("idle_interval_seconds not parsed: %d", a.IdleIntervalSeconds)
}
b := tmpl.Workspaces[1]
if b.IdlePromptFile != "idle-prompt.md" {
t.Errorf("idle_prompt_file not parsed: %q", b.IdlePromptFile)
}
if b.IdleIntervalSeconds != 600 {
t.Errorf("idle_interval_seconds not parsed: %d", b.IdleIntervalSeconds)
}
}
func TestOrgYAML_PromptFileFieldsParse(t *testing.T) {
src := `
name: Test Org
defaults:
initial_prompt_file: shared-boot.md
workspaces:
- name: Role A
files_dir: role-a
initial_prompt_file: initial-prompt.md
schedules:
- name: Hourly audit
cron_expr: "17 * * * *"
prompt_file: schedules/hourly-audit.md
`
var tmpl OrgTemplate
if err := yaml.Unmarshal([]byte(src), &tmpl); err != nil {
t.Fatalf("unmarshal failed: %v", err)
}
if tmpl.Defaults.InitialPromptFile != "shared-boot.md" {
t.Errorf("defaults.initial_prompt_file not parsed: %q", tmpl.Defaults.InitialPromptFile)
}
if len(tmpl.Workspaces) != 1 {
t.Fatalf("expected 1 workspace, got %d", len(tmpl.Workspaces))
}
w := tmpl.Workspaces[0]
if w.InitialPromptFile != "initial-prompt.md" {
t.Errorf("initial_prompt_file not parsed: %q", w.InitialPromptFile)
}
if len(w.Schedules) != 1 {
t.Fatalf("expected 1 schedule, got %d", len(w.Schedules))
}
if w.Schedules[0].PromptFile != "schedules/hourly-audit.md" {
t.Errorf("schedule.prompt_file not parsed: %q", w.Schedules[0].PromptFile)
}
}
// resolvePromptRef is the single entry point for reading workspace prompt
// bodies from either inline YAML or sibling .md files. Phase 1 of the
// org.yaml externalization work (1801-line file → ~600-line structural
// manifest). Tests cover the 6 cases callers exercise + the traversal
// defense (same class as resolveInsideRoot).
func TestResolvePromptRef_InlineWinsOverFile(t *testing.T) {
tmp := t.TempDir()
rolesDir := filepath.Join(tmp, "my-role")
if err := os.MkdirAll(rolesDir, 0o755); err != nil {
t.Fatal(err)
}
// Write a file that would be read if inline were empty — prove inline wins.
if err := os.WriteFile(filepath.Join(rolesDir, "idle.md"), []byte("FROM FILE"), 0o644); err != nil {
t.Fatal(err)
}
got, err := resolvePromptRef("FROM INLINE", "idle.md", tmp, "my-role")
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if got != "FROM INLINE" {
t.Errorf("inline should win: got %q", got)
}
}
func TestResolvePromptRef_FileReadWhenInlineEmpty(t *testing.T) {
tmp := t.TempDir()
rolesDir := filepath.Join(tmp, "my-role")
if err := os.MkdirAll(rolesDir, 0o755); err != nil {
t.Fatal(err)
}
want := "You have no active task.\nBacklog-pull + reflect."
if err := os.WriteFile(filepath.Join(rolesDir, "idle.md"), []byte(want), 0o644); err != nil {
t.Fatal(err)
}
got, err := resolvePromptRef("", "idle.md", tmp, "my-role")
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if got != want {
t.Errorf("got %q, want %q", got, want)
}
}
func TestResolvePromptRef_BothEmptyReturnsEmpty(t *testing.T) {
tmp := t.TempDir()
got, err := resolvePromptRef("", "", tmp, "any")
if err != nil {
t.Errorf("empty inputs should not error, got: %v", err)
}
if got != "" {
t.Errorf("empty inputs should return empty body, got %q", got)
}
}
func TestResolvePromptRef_DefaultsLevelResolvesRelativeToOrgBase(t *testing.T) {
// Defaults don't have a files_dir — ref is resolved relative to
// orgBaseDir itself (shared prompts at the org root).
tmp := t.TempDir()
if err := os.WriteFile(filepath.Join(tmp, "shared.md"), []byte("SHARED"), 0o644); err != nil {
t.Fatal(err)
}
got, err := resolvePromptRef("", "shared.md", tmp, "")
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if got != "SHARED" {
t.Errorf("got %q, want SHARED", got)
}
}
func TestResolvePromptRef_InlineTemplateErrors(t *testing.T) {
// When the caller used inline-template mode (POST /org/import with a
// raw Template in the JSON body, no dir), orgBaseDir is empty and file
// refs are unresolvable. Surface that loudly.
_, err := resolvePromptRef("", "idle.md", "", "my-role")
if err == nil {
t.Error("expected error for file ref without orgBaseDir")
}
if !strings.Contains(err.Error(), "inline-template") {
t.Errorf("error should mention inline-template mode; got: %v", err)
}
}
func TestResolvePromptRef_RejectsTraversalViaFileRef(t *testing.T) {
tmp := t.TempDir()
rolesDir := filepath.Join(tmp, "my-role")
if err := os.MkdirAll(rolesDir, 0o755); err != nil {
t.Fatal(err)
}
// Create a sensitive file OUTSIDE the orgBaseDir we can try to exfiltrate.
outside := filepath.Join(filepath.Dir(tmp), "secret.md")
if err := os.WriteFile(outside, []byte("SECRET"), 0o644); err != nil {
t.Fatal(err)
}
defer os.Remove(outside)
cases := []string{
"../../secret.md",
"../secret.md",
"../../../../etc/passwd",
}
for _, tc := range cases {
t.Run(tc, func(t *testing.T) {
_, err := resolvePromptRef("", tc, tmp, "my-role")
if err == nil {
t.Errorf("expected error for traversal %q", tc)
}
})
}
}
func TestResolvePromptRef_RejectsTraversalViaFilesDir(t *testing.T) {
tmp := t.TempDir()
// The files_dir argument also comes from YAML — ensure it can't escape.
_, err := resolvePromptRef("", "idle.md", tmp, "../escape")
if err == nil {
t.Error("expected error for traversal via files_dir")
}
}
func TestResolvePromptRef_MissingFileErrors(t *testing.T) {
tmp := t.TempDir()
rolesDir := filepath.Join(tmp, "my-role")
if err := os.MkdirAll(rolesDir, 0o755); err != nil {
t.Fatal(err)
}
_, err := resolvePromptRef("", "nonexistent.md", tmp, "my-role")
if err == nil {
t.Error("expected error for missing file")
}
}