diff --git a/content/docs/api-reference.mdx b/content/docs/api-reference.mdx deleted file mode 100644 index 0cd8680..0000000 --- a/content/docs/api-reference.mdx +++ /dev/null @@ -1,710 +0,0 @@ ---- -title: API Reference -description: Complete reference for all Molecule AI Platform HTTP and WebSocket endpoints. ---- - -# API Reference - -The Molecule AI Platform exposes a REST API (default port 8080) for workspace management, agent registry, communication, and administration. All endpoints return JSON unless otherwise noted. - - - **Breaking changes — PR #701 (2026-04-17)** - - - **`PATCH /workspaces/:id` now requires authentication.** Previously, requests without a bearer token could update cosmetic fields (name, x/y position). All `PATCH` calls now require `Authorization: Bearer ` or receive **401 Unauthorized**. - - **`GET /templates` and `GET /org/templates` now require AdminAuth.** Unauthenticated callers receive **401 Unauthorized**. - - **All `/workspaces/:id` endpoints validate the `:id` path parameter** as a UUID. Non-UUID values return **400 Bad Request** before any database interaction. - - **Migration:** add `Authorization: Bearer ` to all `PATCH /workspaces/:id` calls. Use an admin bearer token for `GET /templates` and `GET /org/templates`. Ensure `:id` values in automation scripts are valid UUIDs. - - -**Base URL:** `http://localhost:8080` (self-hosted) or `https://api.moleculesai.app` (SaaS) - ---- - -## Authentication Model - -The platform uses three authentication middleware variants depending on the sensitivity of the route. - -### AdminAuth - -Strict bearer-token authentication. Required for any route where a forged request could leak prompts/memory, create/mutate workspaces, or leak operational data. - -``` -Authorization: Bearer -``` - -**Fail-open behavior:** When no live tokens exist globally (fresh install), AdminAuth passes all requests through. Once the first token is created, all AdminAuth routes require a valid bearer. - -### WorkspaceAuth - -Per-workspace bearer token binding. Workspace A's token cannot access workspace B's sub-routes. Used for the entire `/workspaces/:id/*` group (except the A2A proxy, which uses `CanCommunicate`). - -``` -Authorization: Bearer -``` - -### CanvasOrBearer - -Accepts either a valid bearer token OR a request whose `Origin` header matches `CORS_ORIGINS`. Used only for cosmetic-only routes where a forged request has zero data/security impact. - -Currently applies only to `PUT /canvas/viewport`. Do not extend to data-sensitive routes. - ---- - -## Health and Monitoring - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| GET | `/health` | None | Returns `200 OK` if the platform is running. Use for load balancer health checks. | -| GET | `/metrics` | None | Prometheus text format (v0.0.4) metrics. Scrape-safe, no auth required. | -| GET | `/admin/liveness` | AdminAuth | Per-subsystem `supervised.Snapshot()` ages. Check before debugging stuck scheduler/heartbeat goroutines. | - ---- - -## Workspaces - -Core workspace CRUD and lifecycle operations. - -### CRUD - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| POST | `/workspaces` | AdminAuth | Create a new workspace. Accepts `name`, `runtime`, `template`, `parent_id`, `tier`, `workspace_dir`, and other fields. Runtime is auto-detected from template config if omitted (defaults to `langgraph`). | -| GET | `/workspaces` | AdminAuth | List all workspaces with status, runtime, agent card, position, and hierarchy info. | -| GET | `/workspaces/:id` | WorkspaceAuth | Get a single workspace by ID. | -| PATCH | `/workspaces/:id` | WorkspaceAuth | Update workspace fields. A workspace bearer token is always required — unauthenticated calls return 401. Validates field constraints: `name` ≤ 255 chars, `role` ≤ 1,000 chars, `model` and `runtime` ≤ 100 chars each; `name` and `role` must not contain newlines (`\\n`, `\\r`) or YAML-special characters (`{}[]|>*&!`). Oversized or invalid field values return 400. `:id` must be a valid UUID. Financial fields (`budget_limit`) are not accepted here — use `PATCH /workspaces/:id/budget` (AdminAuth). | -| DELETE | `/workspaces/:id` | AdminAuth | Delete a workspace. Stops the container, revokes all auth tokens, and removes all associated data. | - -### Lifecycle - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| POST | `/workspaces/:id/restart` | WorkspaceAuth | Restart the workspace container. Sends a `restart_context` A2A message after successful re-registration. | -| POST | `/workspaces/:id/pause` | WorkspaceAuth | Stop the container and set status to `paused`. Paused workspaces skip health sweep, liveness monitor, and auto-restart. Resume manually via `/resume`. | -| POST | `/workspaces/:id/resume` | WorkspaceAuth | Re-provision a paused workspace. Status transitions to `provisioning`. | -| POST | `/workspaces/:id/hibernate` | WorkspaceAuth | Immediately hibernate a workspace (stop container, set status to `hibernated`). Useful for manual cost control. See hibernation note below. | - - - **Workspace hibernation** - - A workspace with `hibernation_idle_minutes` set in its config will be **automatically hibernated** by the platform after that many idle minutes (no active tasks, no recent heartbeat). The monitor checks every 2 minutes. - - `hibernated` differs from `paused`: - - **`paused`** — manual, resumes only via `POST /resume`. - - **`hibernated`** — automatic (or via `POST /hibernate`), resumes **automatically** when an A2A message arrives. - - When a message is sent to a hibernated workspace, the platform returns: - ``` - HTTP 503 Retry-After: 15 - {"waking": true} - ``` - Callers should retry after ~15 seconds. The workspace typically returns to `online` within that window. - - To opt a workspace into auto-hibernation, add to its `config.yaml`: - ```yaml - hibernation_idle_minutes: 30 # hibernate after 30 min idle; null (default) = disabled - ``` - - **Atomic hibernation guarantee:** The platform uses a single atomic SQL claim (`UPDATE … WHERE active_tasks = 0`) before stopping the container. If a task arrives between the idle check and the container stop, the claim fails and hibernation is aborted — no in-flight tasks are silently lost. - - -### Budget - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| GET | `/workspaces/:id/budget` | AdminAuth | Read a workspace's current spend and ceiling. Returns `budget_limit`, `monthly_spend`, and `budget_remaining` (all in USD cents). | -| PATCH | `/workspaces/:id/budget` | AdminAuth | Set or clear a workspace's monthly spend ceiling. Body: `{ "budget_limit": N }` (positive integer, USD cents) or `{ "budget_limit": null }` to remove the cap. Negative values → 400. Returns same shape as GET. | - -**Request / response shape:** - -```json -// PATCH request body -{ "budget_limit": 500 } // $5.00/month ceiling -{ "budget_limit": null } // no ceiling - -// GET and PATCH success response (200) -{ - "budget_limit": 500, // null when no ceiling - "monthly_spend": 312, // accumulated spend this period, USD cents - "budget_remaining": 188 // null when no ceiling; max(0, limit-spend) — can be negative -} -``` - - - **`budget_limit` and `monthly_spend` are absent from `GET /workspaces/:id`** - - Financial fields are stripped unconditionally from the workspace detail - response — they do not appear for any caller, authenticated or not. Always - use `GET /workspaces/:id/budget` (AdminAuth) to read spend data. - - `budget_limit` is also **not** accepted on the general `PATCH /workspaces/:id` - endpoint. Use the dedicated `/budget` route. - - - - **Enforcement and fail-open behaviour** - - When `monthly_spend >= budget_limit`, `POST /workspaces/:id/a2a` returns: - ``` - HTTP 402 Payment Required - {"error": "workspace budget limit exceeded"} - ``` - Channel sends (Slack, Telegram, Discord, Lark) are also budget-gated with - the same 402 response. The workspace itself is **not paused** — it keeps - running; only inbound A2A and channel traffic is blocked. - - **Fail-open:** if the budget check encounters a DB error, traffic is allowed - through rather than blocked. The spend ceiling is a soft guardrail, not a - hard guarantee. - - ---- - -## Registry - -Workspace registration and heartbeat endpoints. Called by workspace runtimes, not by end users. - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| POST | `/registry/register` | None | Register a workspace with the platform. Sets status to `online`. Body includes agent URL, agent card, capabilities. | -| POST | `/registry/heartbeat` | Bearer (if token exists) | Send a heartbeat. Updates Redis TTL key (60s expiry). Body can include `active_tasks`, `current_task`, `error_rate`. Triggers `degraded` status if `error_rate > 0.5`. | -| POST | `/registry/update-card` | Bearer (if token exists) | Update the workspace's agent card (name, description, skills, etc.). | - ---- - -## Discovery - -Peer discovery and access control verification. - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| GET | `/registry/discover/:id` | Bearer + `X-Workspace-ID` | Discover a workspace's agent card and URL. Requires caller identification. Fails open on DB hiccup since hierarchy check is primary. | -| GET | `/registry/:id/peers` | Bearer + `X-Workspace-ID` | List all peers (siblings, parent, children) that the caller can communicate with. | -| POST | `/registry/check-access` | None | Check whether two workspaces can communicate. Body: `{ "caller_id": "...", "target_id": "..." }`. Returns `{ "allowed": true/false }`. | - ---- - -## Communication - -### A2A Proxy - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| POST | `/workspaces/:id/a2a` | CanCommunicate | Proxy an A2A JSON-RPC message to the target workspace. Caller identified via `X-Workspace-ID` header. Canvas requests (no header) bypass access check. On connection error, checks if container is dead and triggers auto-restart. | - -### Delegation - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| POST | `/workspaces/:id/delegate` | WorkspaceAuth | Async fire-and-forget delegation. Supports idempotency keys. Body includes target workspace, prompt, and metadata. | -| GET | `/workspaces/:id/delegations` | WorkspaceAuth | List delegation status for a workspace. Returns delegation rows with status, result, timestamps. | - ---- - -## Configuration - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| GET | `/workspaces/:id/config` | WorkspaceAuth | Get the workspace's `config.yaml` contents. | -| PATCH | `/workspaces/:id/config` | WorkspaceAuth | Update the workspace config. "Save & Restart" writes config and auto-restarts; "Save" writes only and shows a restart banner in the Canvas. | -| GET | `/workspaces/:id/model` | WorkspaceAuth | Get the workspace's current model selection. | -| PUT | `/workspaces/:id/model` | WorkspaceAuth | Set the workspace model (e.g. `anthropic:claude-sonnet-4-6`). | -| GET | `/workspaces/:id/provider` | WorkspaceAuth | Get the resolved LLM provider for the workspace's runtime + model. | -| PUT | `/workspaces/:id/provider` | WorkspaceAuth | Override the LLM provider for the workspace. | - ---- - -## Secrets - -### Per-Workspace Secrets - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| GET | `/workspaces/:id/secrets` | WorkspaceAuth | List secret keys for a workspace (keys only, values masked). | -| POST | `/workspaces/:id/secrets` | WorkspaceAuth | Set a secret `{ "key": "...", "value": "..." }`. Auto-restarts the workspace. | -| PUT | `/workspaces/:id/secrets` | WorkspaceAuth | Alias for POST (upsert semantics). Auto-restarts the workspace. | -| DELETE | `/workspaces/:id/secrets/:key` | WorkspaceAuth | Delete a secret by key. Auto-restarts the workspace. | -| GET | `/workspaces/:id/model` | WorkspaceAuth | Return the model configuration derived from available API keys (which provider keys are set). | - -### Global Secrets - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| GET | `/settings/secrets` | AdminAuth | List global secrets (keys only, values masked). | -| PUT | `/settings/secrets` | AdminAuth | Set a global secret `{ "key": "...", "value": "..." }`. Auto-restarts every non-paused/non-removed workspace that does not shadow the key with a workspace-level override. | -| POST | `/settings/secrets` | AdminAuth | Alias for PUT. | -| DELETE | `/settings/secrets/:key` | AdminAuth | Delete a global secret. Same auto-restart fan-out as PUT. | - -Legacy aliases `GET/POST/DELETE /admin/secrets[/:key]` also exist and behave identically. - ---- - -## Memory - -### Key-Value Memory - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| GET | `/workspaces/:id/memory` | WorkspaceAuth | List all key-value memory entries for a workspace. | -| POST | `/workspaces/:id/memory` | WorkspaceAuth | Set a memory entry `{ "key": "...", "value": "..." }`. | -| DELETE | `/workspaces/:id/memory/:key` | WorkspaceAuth | Delete a memory entry by key. | - -### Agent Memories (HMA-scoped) - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| GET | `/workspaces/:id/memories` | WorkspaceAuth | List or search agent memories. Supports `?q=` for semantic search (see below). | -| POST | `/workspaces/:id/memories` | WorkspaceAuth | Create an agent memory entry. | -| GET | `/workspaces/:id/v2/namespaces` | WorkspaceAuth | List the HMA memory namespaces visible to this workspace (LOCAL / TEAM / GLOBAL scopes resolved along the org hierarchy). | -| GET | `/workspaces/:id/v2/memories` | WorkspaceAuth | List agent memories via the v2 namespace-scoped API. | -| DELETE | `/workspaces/:id/v2/memories/:memoryId` | WorkspaceAuth | Delete an agent memory by its ID (v2 API). | - -#### Semantic search (`?q=`) - -When a platform-level embedding function is configured, passing `?q=` -on `GET /workspaces/:id/memories` triggers vector similarity search instead of -the default full-text / ILIKE path: - -``` -GET /workspaces/{id}/memories?q=authentication+flow&limit=10 -Authorization: Bearer {token} -``` - -Matching entries are returned **ordered by cosine similarity** (most similar -first). Each row includes an additional `similarity_score` field (0–1, higher -is closer): - -```json -[ - { - "id": "mem_abc123", - "key": "auth-design", - "value": "We use short-lived JWTs issued by the platform and refreshed via /auth/token.", - "similarity_score": 0.91, - "created_at": "2026-04-10T14:22:00Z" - } -] -``` - -**Graceful fallback**: if no embedding function is configured, or if the -embedding call fails for a given query, the platform falls back transparently -to the text-search path. The `similarity_score` field is absent in fallback -responses. You do not need to change client code to handle both modes. - ---- - -## Files - -Workspace file management. Files are stored in the workspace's config directory. - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| GET | `/workspaces/:id/files` | WorkspaceAuth | List files in the workspace config directory. | -| GET | `/workspaces/:id/files/*path` | WorkspaceAuth | Read a specific file. | -| PUT | `/workspaces/:id/files/*path` | WorkspaceAuth | Write a file. Creates parent directories as needed. On SaaS workspaces (EC2, no Docker), routes via EC2 Instance Connect endpoint using an ephemeral SSH key pair — the key is scoped to the file-write operation and deleted within 30 seconds. Max payload ~10 MiB. Self-hosted Docker workspaces write via `docker cp` as before. | -| DELETE | `/workspaces/:id/files/*path` | WorkspaceAuth | Delete a file. | - ---- - -## Activity - -Activity logging and search for A2A communications, task updates, and agent logs. - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| GET | `/workspaces/:id/activity` | WorkspaceAuth | List activity logs for a workspace. Supports `?source=canvas` or `?source=agent` filter, and `?type=delegation` for A2A topology overlay polling. | -| POST | `/workspaces/:id/activity` | WorkspaceAuth | Log an activity entry (used by workspace runtimes to self-report). | -| POST | `/workspaces/:id/notify` | WorkspaceAuth | Agent-to-user push message via WebSocket. Delivers a notification to connected Canvas clients. | - ---- - -## Session Search - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| GET | `/workspaces/:id/session-search` | WorkspaceAuth | Search activity logs with filters for type, date range, and text content. Returns paginated results. | - ---- - -## Workflow Checkpoints - -Step-level progress persistence for long-running Temporal workflows. Workspaces with `runtime: langgraph` (Temporal) automatically save a checkpoint after each of the three workflow stages (`task_receive`, `llm_call`, `task_complete`) and resume from the last completed stage on restart. - - - **Automatic resume behavior (runtime: langgraph only)** - - When a Temporal workspace restarts mid-workflow, the runtime reads the highest-index checkpoint and sets `resume_from_step` accordingly. Already-completed stages are skipped — the agent picks up exactly where it left off without re-running earlier steps. - - Checkpoint I/O is non-fatal: network errors are silently swallowed. A crashed or unreachable platform never prevents the agent from running. - - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| POST | `/workspaces/:id/checkpoints` | WorkspaceAuth | Upsert a step checkpoint. Body: `{ "workflow_id": "...", "step_name": "task_receive\|llm_call\|task_complete", "step_index": 0, "payload": {...} }`. Uses `ON CONFLICT DO UPDATE` — safe to call multiple times. | -| GET | `/workspaces/:id/checkpoints/:wfid` | WorkspaceAuth | Return all checkpoints for a workflow, ordered by `step_index DESC`. Returns 404 if no checkpoints exist for the workflow. | -| DELETE | `/workspaces/:id/checkpoints/:wfid` | WorkspaceAuth | Clear all checkpoints for a workflow. Called by the runtime on clean task completion. Returns 404 if none exist. | - -**Step names and indices:** - -| Step | `step_index` | Meaning | -|------|-------------|---------| -| `task_receive` | 0 | Task received from A2A message | -| `llm_call` | 1 | LLM inference completed | -| `task_complete` | 2 | Task result sent back to caller | - ---- - -## Schedules - -Cron-based scheduled tasks per workspace. - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| GET | `/workspaces/:id/schedules` | WorkspaceAuth | List all schedules for a workspace. | -| POST | `/workspaces/:id/schedules` | WorkspaceAuth | Create a schedule. Body: `{ "expression": "0 */6 * * *", "timezone": "UTC", "prompt": "...", "enabled": true }`. | -| PATCH | `/workspaces/:id/schedules/:scheduleId` | WorkspaceAuth | Update a schedule (expression, timezone, prompt, enabled). | -| DELETE | `/workspaces/:id/schedules/:scheduleId` | WorkspaceAuth | Delete a schedule. | -| POST | `/workspaces/:id/schedules/:scheduleId/run` | WorkspaceAuth | Manually trigger a schedule immediately. | -| GET | `/workspaces/:id/schedules/:scheduleId/history` | WorkspaceAuth | List past runs for a schedule. Includes status (`success`, `error`, `skipped`) and `error_detail`. | - -Schedule `source` field: `template` for org/import-seeded schedules, `runtime` for Canvas/API-created. The `last_status` includes `skipped` when the scheduler concurrency-aware-skips a busy workspace. - ---- - -## Channels - -Social channel integrations (Telegram, Slack, etc.) for workspace agents. - -### Per-Workspace Channels - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| GET | `/workspaces/:id/channels` | WorkspaceAuth | List channels for a workspace. | -| POST | `/workspaces/:id/channels` | WorkspaceAuth | Create a channel. Body includes platform type, JSONB config, and allowlist. | -| PATCH | `/workspaces/:id/channels/:channelId` | WorkspaceAuth | Update a channel's config or allowlist. | -| DELETE | `/workspaces/:id/channels/:channelId` | WorkspaceAuth | Delete a channel. | -| POST | `/workspaces/:id/channels/:channelId/send` | WorkspaceAuth | Send an outbound message through the channel. | -| POST | `/workspaces/:id/channels/:channelId/test` | WorkspaceAuth | Test the channel connection (send a test message). | - -### Global Channel Endpoints - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| GET | `/channels/adapters` | None | List available social platform adapters (Telegram, Slack, etc.). | -| POST | `/channels/discover` | AdminAuth | Auto-detect available chats/groups for a bot token. | -| POST | `/webhooks/:type` | None | Incoming webhook endpoint for social platforms. The `:type` parameter identifies the platform (e.g., `telegram`, `slack`). | - ---- - -## Plugins - -Plugin registry and per-workspace plugin management. - -### Global Plugin Registry - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| GET | `/plugins` | None | List all plugins in the registry. Supports `?runtime=` filter to show only compatible plugins. | -| GET | `/plugins/sources` | None | List registered install-source schemes (e.g., `github://`, `local://`). | - -### Per-Workspace Plugins - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| GET | `/workspaces/:id/plugins` | WorkspaceAuth | List installed plugins for a workspace. | -| POST | `/workspaces/:id/plugins` | WorkspaceAuth | Install a plugin. Body: `{ "source": "github://org/repo" }`. Safeguards: 64 KiB body limit, 5 min fetch timeout, 100 MiB max staged-tree. | -| DELETE | `/workspaces/:id/plugins/:name` | WorkspaceAuth | Uninstall a plugin by name. | -| GET | `/workspaces/:id/plugins/available` | WorkspaceAuth | List plugins available for this workspace (filtered by workspace runtime). | -| GET | `/workspaces/:id/plugins/compatibility` | WorkspaceAuth | Preflight runtime-change check. Query: `?runtime=X`. Returns which currently-installed plugins would be incompatible with the target runtime. | - ---- - -## Auth Tokens - -Bearer token management for workspaces. - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| GET | `/workspaces/:id/tokens` | WorkspaceAuth | List active tokens for a workspace (token values are masked). | -| POST | `/workspaces/:id/tokens` | WorkspaceAuth | Create a new bearer token for the workspace. | -| DELETE | `/workspaces/:id/tokens/:tokenId` | WorkspaceAuth | Revoke a specific token. | - -### Admin token minting - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| POST | `/admin/workspaces/:id/tokens` | AdminAuth | Mint a fresh bearer token for a workspace (admin / bootstrap / E2E use). The earlier unauthenticated `GET /admin/workspaces/:id/test-token` route was removed; minting now requires AdminAuth. | - ---- - -## Templates and Bundles - -### Templates - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| GET | `/templates` | AdminAuth | List available workspace templates with their runtime, description, and config schema. | -| POST | `/templates/import` | AdminAuth | Import a workspace template from a `github://` source URL. | - -### Org Templates - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| GET | `/org/templates` | AdminAuth | List available organization templates. | -| POST | `/org/import` | AdminAuth | Import an org template. Applies `resolveInsideRoot` path sanitization. Creates the full workspace hierarchy defined in `org.yaml`. | - -### Bundles - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| GET | `/bundles/export/:id` | AdminAuth | Export a workspace (or workspace tree) as a portable bundle. Includes config, secrets (keys only), memory, schedules, and hierarchy. | -| POST | `/bundles/import` | AdminAuth | Import a previously-exported bundle. Recreates the workspace tree with all associated data. | - ---- - -## Approvals - -Human-in-the-loop approval system for agent actions. - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| POST | `/workspaces/:id/approvals` | WorkspaceAuth | Create an approval request. Body includes the action description, metadata, and options. | -| GET | `/workspaces/:id/approvals` | WorkspaceAuth | List approval requests for a workspace. | -| POST | `/workspaces/:id/approvals/:id/decide` | WorkspaceAuth | Approve or reject an approval request. Body: `{ "decision": "approve" }` or `{ "decision": "reject" }`. | -| GET | `/approvals/pending` | AdminAuth | List all pending approval requests across all workspaces. | - ---- - -## Canvas - -Canvas viewport persistence (cosmetic only). - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| GET | `/canvas/viewport` | None | Get the saved canvas viewport (zoom, pan position). Open endpoint for bootstrap-friendliness. | -| PUT | `/canvas/viewport` | CanvasOrBearer | Save the canvas viewport. Accepts bearer OR matching `Origin` header. Worst case on forgery: viewport corruption, recovered by page refresh. | - ---- - -## Traces - -LLM trace retrieval from Langfuse. - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| GET | `/workspaces/:id/traces` | WorkspaceAuth | List LLM traces for a workspace from Langfuse. | - ---- - -## Audit Ledger - -HMAC-SHA256-chained immutable agent event log for compliance record-keeping (EU AI Act Art. 12 / Art. 13). Each event is cryptographically chained to the previous one — tampering with any record breaks all subsequent HMACs. - - - **`AUDIT_LEDGER_SALT` required.** The platform and workspace containers must share the same `AUDIT_LEDGER_SALT` environment variable to compute and verify event HMACs. Set it in both your platform env and workspace container env. If the variable is absent, `chain_valid` returns `null` (not `false`) — no records are lost, verification is simply unavailable. - - -### Query - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| GET | `/workspaces/:id/audit` | WorkspaceAuth | Query the audit ledger for a workspace. Returns events in descending chronological order with inline chain verification. | - -**Query parameters:** - -| Parameter | Type | Description | -|-----------|------|-------------| -| `agent_id` | string | Filter to a specific agent. | -| `session_id` | string | Filter to a specific session. | -| `from` | RFC 3339 | Start of time range (e.g. `2026-04-01T00:00:00Z`). | -| `to` | RFC 3339 | End of time range. | -| `limit` | int | Max records to return. Capped at **500**. | -| `offset` | int | Pagination offset. | - -**Response shape:** - -```json -{ - "events": [ - { - "id": "uuid", - "workspace_id": "uuid", - "agent_id": "my-researcher", - "session_id": "sess_abc123", - "event_type": "tool_call", - "payload": { "tool": "bash", "input": "ls /workspace" }, - "hmac": "sha256hex...", - "prev_hmac": "sha256hex...", - "created_at": "2026-04-17T12:00:00Z" - } - ], - "chain_valid": true -} -``` - -`chain_valid` values: -- `true` — all HMACs verified; ledger is intact. -- `false` — at least one HMAC mismatch; possible tampering. -- `null` — `AUDIT_LEDGER_SALT` is absent from the platform env; verification skipped. - -### Workspace-side: recording events - -In your workspace template, wire `LedgerHooks` into the agent pipeline: - -```python -from molecule_audit.hooks import LedgerHooks - -hooks = LedgerHooks(agent_id="my-researcher", session_id=session_id) - -async with hooks: - # hooks.on_task_start / on_llm_call / on_tool_call / on_task_end - # fire automatically at each pipeline stage - result = await agent.run(task) -``` - -`LedgerHooks` is exception-safe — a failed ledger write never aborts the agent task. - -### CLI chain verification - -```bash -# Verify the full chain for an agent; exit 0 = intact -python -m molecule_audit.verify --agent-id my-researcher - -# Custom DB URL -python -m molecule_audit.verify --agent-id my-researcher --db postgresql://user:pass@host/db -``` - -Exit codes: `0` = chain valid · `1` = broken chain · `2` = `AUDIT_LEDGER_SALT` missing · `3` = DB error. - ---- - -## Events - -Append-only event log for structure changes. - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| GET | `/events` | AdminAuth | List all structure events across all workspaces. | -| GET | `/events/:workspaceId` | AdminAuth | List structure events for a specific workspace. | - ---- - -## Terminal - -WebSocket-based terminal access to workspace containers. - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| WS | `/workspaces/:id/terminal` | WorkspaceAuth | Open a WebSocket terminal session to the workspace container. Provides interactive shell access. | - ---- - -## WebSocket - -Real-time event streaming for Canvas clients. - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| WS | `/ws` | None | Connect to the WebSocket hub. Receives all structure events (`WORKSPACE_ONLINE`, `WORKSPACE_OFFLINE`, `HEARTBEAT`, `CONFIG_UPDATED`, `A2A_RESPONSE`, `AGENT_MESSAGE`, etc.). Canvas clients connect here for real-time updates. | - ---- - -## Server-Sent Events (AG-UI) - -Per-workspace SSE stream compatible with the [AG-UI protocol](https://github.com/ag-ui-protocol/ag-ui). Use this endpoint to consume structured agent events from a web client or external tool without a WebSocket library. - -| Method | Path | Auth | Description | -|--------|------|------|-------------| -| GET | `/workspaces/:id/events/stream` | WorkspaceAuth | Open an SSE stream for the workspace. Returns `Content-Type: text/event-stream`. Sends an initial `: ping` comment on connect, then delivers every event emitted by the workspace in AG-UI envelope format. Events from other workspaces are filtered out. Returns `404` if the workspace does not exist. | - -### Event envelope format - -Each event is delivered as an SSE `data:` line containing a JSON object: - -```json -{ - "type": "AGENT_MESSAGE", - "timestamp": 1713398400000, - "data": { ... } -} -``` - -- **`type`** — event type string (e.g. `AGENT_MESSAGE`, `A2A_RESPONSE`, `TASK_UPDATED`) -- **`timestamp`** — Unix milliseconds at time of broadcast -- **`data`** — event-specific payload (same payload as the WebSocket hub delivers) - -### Event types streamed - -All event types emitted by `RecordAndBroadcast` **and** `BroadcastOnly` reach the SSE stream. The `BroadcastOnly` path is important: events like `AGENT_MESSAGE`, `A2A_RESPONSE`, and `TASK_UPDATED` skip Redis and would be invisible to a Redis-only subscriber — the in-process SSE layer catches them. - -### Example: connect with `curl` - -```bash -curl -N \ - -H "Authorization: Bearer " \ - http://localhost:8080/workspaces//events/stream -``` - -``` -: ping - -data: {"type":"AGENT_MESSAGE","timestamp":1713398401234,"data":{"text":"Starting task..."}} - -data: {"type":"TASK_UPDATED","timestamp":1713398405678,"data":{"status":"running"}} -``` - -### Example: connect from JavaScript - -```js -const es = new EventSource( - `/workspaces/${workspaceId}/events/stream`, - { headers: { Authorization: `Bearer ${token}` } } -); - -es.onmessage = (e) => { - const event = JSON.parse(e.data); - console.log(event.type, event.data); -}; -``` - - - The SSE endpoint uses WorkspaceAuth — the bearer token must be bound to the `:id` in the path. A token for workspace A cannot open a stream for workspace B. - - ---- - -## Error Responses - -All endpoints return standard HTTP status codes: - -| Status | Meaning | -|--------|---------| -| 200 | Success | -| 201 | Created | -| 400 | Bad request (malformed body, missing required fields) | -| 401 | Unauthorized (missing or invalid bearer token) | -| 403 | Forbidden (valid token but insufficient access) | -| 404 | Not found (workspace, schedule, channel, etc. does not exist) | -| 409 | Conflict (idempotency key collision on delegation) | -| 429 | Rate limited (exceeds `RATE_LIMIT` requests/min) | -| 500 | Internal server error | - -Error response body format: - -```json -{ - "error": "human-readable error message" -} -``` - ---- - -## Rate Limiting - -All endpoints are subject to a global rate limit of `RATE_LIMIT` requests per minute (default: 600). When exceeded, the platform returns `429 Too Many Requests` with a `Retry-After` header. - ---- - -## CORS - -The platform sets CORS headers based on the `CORS_ORIGINS` environment variable (comma-separated list, default: `http://localhost:3000,http://localhost:3001`). Preflight (`OPTIONS`) requests are handled automatically by the Gin CORS middleware. diff --git a/content/docs/api-reference/communication.mdx b/content/docs/api-reference/communication.mdx new file mode 100644 index 0000000..d80e4e2 --- /dev/null +++ b/content/docs/api-reference/communication.mdx @@ -0,0 +1,45 @@ +--- +title: "Communication API" +description: "Agent registry, discovery, A2A proxy, and delegation endpoints." +--- + +## Registry + +Workspace registration and heartbeat endpoints. Called by workspace runtimes, not by end users. + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| POST | `/registry/register` | None | Register a workspace with the platform. Sets status to `online`. Body includes agent URL, agent card, capabilities. | +| POST | `/registry/heartbeat` | Bearer (if token exists) | Send a heartbeat. Updates Redis TTL key (60s expiry). Body can include `active_tasks`, `current_task`, `error_rate`. Triggers `degraded` status if `error_rate > 0.5`. | +| POST | `/registry/update-card` | Bearer (if token exists) | Update the workspace's agent card (name, description, skills, etc.). | + +--- + +## Discovery + +Peer discovery and access control verification. + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/registry/discover/:id` | Bearer + `X-Workspace-ID` | Discover a workspace's agent card and URL. Requires caller identification. Fails open on DB hiccup since hierarchy check is primary. | +| GET | `/registry/:id/peers` | Bearer + `X-Workspace-ID` | List all peers (siblings, parent, children) that the caller can communicate with. | +| POST | `/registry/check-access` | None | Check whether two workspaces can communicate. Body: `{ "caller_id": "...", "target_id": "..." }`. Returns `{ "allowed": true/false }`. | + +--- + +## Communication + +### A2A Proxy + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| POST | `/workspaces/:id/a2a` | CanCommunicate | Proxy an A2A JSON-RPC message to the target workspace. Caller identified via `X-Workspace-ID` header. Canvas requests (no header) bypass access check. On connection error, checks if container is dead and triggers auto-restart. | + +### Delegation + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| POST | `/workspaces/:id/delegate` | WorkspaceAuth | Async fire-and-forget delegation. Supports idempotency keys. Body includes target workspace, prompt, and metadata. | +| GET | `/workspaces/:id/delegations` | WorkspaceAuth | List delegation status for a workspace. Returns delegation rows with status, result, timestamps. | + +--- diff --git a/content/docs/api-reference/index.mdx b/content/docs/api-reference/index.mdx new file mode 100644 index 0000000..3c8545e --- /dev/null +++ b/content/docs/api-reference/index.mdx @@ -0,0 +1,113 @@ +--- +title: "API Reference" +description: "Complete reference for all Molecule AI Platform HTTP and WebSocket endpoints." +--- + + +# API Reference + +The Molecule AI Platform exposes a REST API (default port 8080) for workspace management, agent registry, communication, and administration. All endpoints return JSON unless otherwise noted. + + + **Breaking changes — PR #701 (2026-04-17)** + + - **`PATCH /workspaces/:id` now requires authentication.** Previously, requests without a bearer token could update cosmetic fields (name, x/y position). All `PATCH` calls now require `Authorization: Bearer ` or receive **401 Unauthorized**. + - **`GET /templates` and `GET /org/templates` now require AdminAuth.** Unauthenticated callers receive **401 Unauthorized**. + - **All `/workspaces/:id` endpoints validate the `:id` path parameter** as a UUID. Non-UUID values return **400 Bad Request** before any database interaction. + + **Migration:** add `Authorization: Bearer ` to all `PATCH /workspaces/:id` calls. Use an admin bearer token for `GET /templates` and `GET /org/templates`. Ensure `:id` values in automation scripts are valid UUIDs. + + +**Base URL:** `http://localhost:8080` (self-hosted) or `https://api.moleculesai.app` (SaaS) + +--- + +## Endpoint reference + +The full endpoint reference is grouped into these pages: + +- **[Workspaces](/docs/api-reference/workspaces)** — workspace CRUD & lifecycle, budget, configuration, files, activity, session search. +- **[Communication](/docs/api-reference/communication)** — registry, discovery, A2A proxy & delegation. +- **[Memory & Secrets](/docs/api-reference/memory)** — key-value & HMA-scoped memory, secrets, workflow checkpoints. +- **[Integrations](/docs/api-reference/integrations)** — schedules, channels, plugins, auth tokens, templates & bundles, approvals. +- **[Real-time & Observability](/docs/api-reference/realtime)** — canvas, traces, audit ledger, events, terminal, WebSocket, SSE. + +--- + +## Authentication Model + +The platform uses three authentication middleware variants depending on the sensitivity of the route. + +### AdminAuth + +Strict bearer-token authentication. Required for any route where a forged request could leak prompts/memory, create/mutate workspaces, or leak operational data. + +``` +Authorization: Bearer +``` + +**Fail-open behavior:** When no live tokens exist globally (fresh install), AdminAuth passes all requests through. Once the first token is created, all AdminAuth routes require a valid bearer. + +### WorkspaceAuth + +Per-workspace bearer token binding. Workspace A's token cannot access workspace B's sub-routes. Used for the entire `/workspaces/:id/*` group (except the A2A proxy, which uses `CanCommunicate`). + +``` +Authorization: Bearer +``` + +### CanvasOrBearer + +Accepts either a valid bearer token OR a request whose `Origin` header matches `CORS_ORIGINS`. Used only for cosmetic-only routes where a forged request has zero data/security impact. + +Currently applies only to `PUT /canvas/viewport`. Do not extend to data-sensitive routes. + +--- + +## Health and Monitoring + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/health` | None | Returns `200 OK` if the platform is running. Use for load balancer health checks. | +| GET | `/metrics` | None | Prometheus text format (v0.0.4) metrics. Scrape-safe, no auth required. | +| GET | `/admin/liveness` | AdminAuth | Per-subsystem `supervised.Snapshot()` ages. Check before debugging stuck scheduler/heartbeat goroutines. | + +--- + +--- + +## Error Responses + +All endpoints return standard HTTP status codes: + +| Status | Meaning | +|--------|---------| +| 200 | Success | +| 201 | Created | +| 400 | Bad request (malformed body, missing required fields) | +| 401 | Unauthorized (missing or invalid bearer token) | +| 403 | Forbidden (valid token but insufficient access) | +| 404 | Not found (workspace, schedule, channel, etc. does not exist) | +| 409 | Conflict (idempotency key collision on delegation) | +| 429 | Rate limited (exceeds `RATE_LIMIT` requests/min) | +| 500 | Internal server error | + +Error response body format: + +```json +{ + "error": "human-readable error message" +} +``` + +--- + +## Rate Limiting + +All endpoints are subject to a global rate limit of `RATE_LIMIT` requests per minute (default: 600). When exceeded, the platform returns `429 Too Many Requests` with a `Retry-After` header. + +--- + +## CORS + +The platform sets CORS headers based on the `CORS_ORIGINS` environment variable (comma-separated list, default: `http://localhost:3000,http://localhost:3001`). Preflight (`OPTIONS`) requests are handled automatically by the Gin CORS middleware. diff --git a/content/docs/api-reference/integrations.mdx b/content/docs/api-reference/integrations.mdx new file mode 100644 index 0000000..ab3afdc --- /dev/null +++ b/content/docs/api-reference/integrations.mdx @@ -0,0 +1,125 @@ +--- +title: "Integrations API" +description: "Schedules, channels, plugins, auth tokens, templates & bundles, and approvals endpoints." +--- + +## Schedules + +Cron-based scheduled tasks per workspace. + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/workspaces/:id/schedules` | WorkspaceAuth | List all schedules for a workspace. | +| POST | `/workspaces/:id/schedules` | WorkspaceAuth | Create a schedule. Body: `{ "expression": "0 */6 * * *", "timezone": "UTC", "prompt": "...", "enabled": true }`. | +| PATCH | `/workspaces/:id/schedules/:scheduleId` | WorkspaceAuth | Update a schedule (expression, timezone, prompt, enabled). | +| DELETE | `/workspaces/:id/schedules/:scheduleId` | WorkspaceAuth | Delete a schedule. | +| POST | `/workspaces/:id/schedules/:scheduleId/run` | WorkspaceAuth | Manually trigger a schedule immediately. | +| GET | `/workspaces/:id/schedules/:scheduleId/history` | WorkspaceAuth | List past runs for a schedule. Includes status (`success`, `error`, `skipped`) and `error_detail`. | + +Schedule `source` field: `template` for org/import-seeded schedules, `runtime` for Canvas/API-created. The `last_status` includes `skipped` when the scheduler concurrency-aware-skips a busy workspace. + +--- + +## Channels + +Social channel integrations (Telegram, Slack, etc.) for workspace agents. + +### Per-Workspace Channels + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/workspaces/:id/channels` | WorkspaceAuth | List channels for a workspace. | +| POST | `/workspaces/:id/channels` | WorkspaceAuth | Create a channel. Body includes platform type, JSONB config, and allowlist. | +| PATCH | `/workspaces/:id/channels/:channelId` | WorkspaceAuth | Update a channel's config or allowlist. | +| DELETE | `/workspaces/:id/channels/:channelId` | WorkspaceAuth | Delete a channel. | +| POST | `/workspaces/:id/channels/:channelId/send` | WorkspaceAuth | Send an outbound message through the channel. | +| POST | `/workspaces/:id/channels/:channelId/test` | WorkspaceAuth | Test the channel connection (send a test message). | + +### Global Channel Endpoints + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/channels/adapters` | None | List available social platform adapters (Telegram, Slack, etc.). | +| POST | `/channels/discover` | AdminAuth | Auto-detect available chats/groups for a bot token. | +| POST | `/webhooks/:type` | None | Incoming webhook endpoint for social platforms. The `:type` parameter identifies the platform (e.g., `telegram`, `slack`). | + +--- + +## Plugins + +Plugin registry and per-workspace plugin management. + +### Global Plugin Registry + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/plugins` | None | List all plugins in the registry. Supports `?runtime=` filter to show only compatible plugins. | +| GET | `/plugins/sources` | None | List registered install-source schemes (e.g., `github://`, `local://`). | + +### Per-Workspace Plugins + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/workspaces/:id/plugins` | WorkspaceAuth | List installed plugins for a workspace. | +| POST | `/workspaces/:id/plugins` | WorkspaceAuth | Install a plugin. Body: `{ "source": "github://org/repo" }`. Safeguards: 64 KiB body limit, 5 min fetch timeout, 100 MiB max staged-tree. | +| DELETE | `/workspaces/:id/plugins/:name` | WorkspaceAuth | Uninstall a plugin by name. | +| GET | `/workspaces/:id/plugins/available` | WorkspaceAuth | List plugins available for this workspace (filtered by workspace runtime). | +| GET | `/workspaces/:id/plugins/compatibility` | WorkspaceAuth | Preflight runtime-change check. Query: `?runtime=X`. Returns which currently-installed plugins would be incompatible with the target runtime. | + +--- + +## Auth Tokens + +Bearer token management for workspaces. + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/workspaces/:id/tokens` | WorkspaceAuth | List active tokens for a workspace (token values are masked). | +| POST | `/workspaces/:id/tokens` | WorkspaceAuth | Create a new bearer token for the workspace. | +| DELETE | `/workspaces/:id/tokens/:tokenId` | WorkspaceAuth | Revoke a specific token. | + +### Admin token minting + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| POST | `/admin/workspaces/:id/tokens` | AdminAuth | Mint a fresh bearer token for a workspace (admin / bootstrap / E2E use). The earlier unauthenticated `GET /admin/workspaces/:id/test-token` route was removed; minting now requires AdminAuth. | + +--- + +## Templates and Bundles + +### Templates + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/templates` | AdminAuth | List available workspace templates with their runtime, description, and config schema. | +| POST | `/templates/import` | AdminAuth | Import a workspace template from a `github://` source URL. | + +### Org Templates + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/org/templates` | AdminAuth | List available organization templates. | +| POST | `/org/import` | AdminAuth | Import an org template. Applies `resolveInsideRoot` path sanitization. Creates the full workspace hierarchy defined in `org.yaml`. | + +### Bundles + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/bundles/export/:id` | AdminAuth | Export a workspace (or workspace tree) as a portable bundle. Includes config, secrets (keys only), memory, schedules, and hierarchy. | +| POST | `/bundles/import` | AdminAuth | Import a previously-exported bundle. Recreates the workspace tree with all associated data. | + +--- + +## Approvals + +Human-in-the-loop approval system for agent actions. + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| POST | `/workspaces/:id/approvals` | WorkspaceAuth | Create an approval request. Body includes the action description, metadata, and options. | +| GET | `/workspaces/:id/approvals` | WorkspaceAuth | List approval requests for a workspace. | +| POST | `/workspaces/:id/approvals/:id/decide` | WorkspaceAuth | Approve or reject an approval request. Body: `{ "decision": "approve" }` or `{ "decision": "reject" }`. | +| GET | `/approvals/pending` | AdminAuth | List all pending approval requests across all workspaces. | + +--- diff --git a/content/docs/api-reference/memory.mdx b/content/docs/api-reference/memory.mdx new file mode 100644 index 0000000..0ee3e56 --- /dev/null +++ b/content/docs/api-reference/memory.mdx @@ -0,0 +1,111 @@ +--- +title: "Memory & Secrets API" +description: "Key-value and HMA-scoped agent memory, secrets, and workflow checkpoint endpoints." +--- + +## Secrets + +### Per-Workspace Secrets + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/workspaces/:id/secrets` | WorkspaceAuth | List secret keys for a workspace (keys only, values masked). | +| POST | `/workspaces/:id/secrets` | WorkspaceAuth | Set a secret `{ "key": "...", "value": "..." }`. Auto-restarts the workspace. | +| PUT | `/workspaces/:id/secrets` | WorkspaceAuth | Alias for POST (upsert semantics). Auto-restarts the workspace. | +| DELETE | `/workspaces/:id/secrets/:key` | WorkspaceAuth | Delete a secret by key. Auto-restarts the workspace. | +| GET | `/workspaces/:id/model` | WorkspaceAuth | Return the model configuration derived from available API keys (which provider keys are set). | + +### Global Secrets + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/settings/secrets` | AdminAuth | List global secrets (keys only, values masked). | +| PUT | `/settings/secrets` | AdminAuth | Set a global secret `{ "key": "...", "value": "..." }`. Auto-restarts every non-paused/non-removed workspace that does not shadow the key with a workspace-level override. | +| POST | `/settings/secrets` | AdminAuth | Alias for PUT. | +| DELETE | `/settings/secrets/:key` | AdminAuth | Delete a global secret. Same auto-restart fan-out as PUT. | + +Legacy aliases `GET/POST/DELETE /admin/secrets[/:key]` also exist and behave identically. + +--- + +## Memory + +### Key-Value Memory + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/workspaces/:id/memory` | WorkspaceAuth | List all key-value memory entries for a workspace. | +| POST | `/workspaces/:id/memory` | WorkspaceAuth | Set a memory entry `{ "key": "...", "value": "..." }`. | +| DELETE | `/workspaces/:id/memory/:key` | WorkspaceAuth | Delete a memory entry by key. | + +### Agent Memories (HMA-scoped) + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/workspaces/:id/memories` | WorkspaceAuth | List or search agent memories. Supports `?q=` for semantic search (see below). | +| POST | `/workspaces/:id/memories` | WorkspaceAuth | Create an agent memory entry. | +| GET | `/workspaces/:id/v2/namespaces` | WorkspaceAuth | List the HMA memory namespaces visible to this workspace (LOCAL / TEAM / GLOBAL scopes resolved along the org hierarchy). | +| GET | `/workspaces/:id/v2/memories` | WorkspaceAuth | List agent memories via the v2 namespace-scoped API. | +| DELETE | `/workspaces/:id/v2/memories/:memoryId` | WorkspaceAuth | Delete an agent memory by its ID (v2 API). | + +#### Semantic search (`?q=`) + +When a platform-level embedding function is configured, passing `?q=` +on `GET /workspaces/:id/memories` triggers vector similarity search instead of +the default full-text / ILIKE path: + +``` +GET /workspaces/{id}/memories?q=authentication+flow&limit=10 +Authorization: Bearer {token} +``` + +Matching entries are returned **ordered by cosine similarity** (most similar +first). Each row includes an additional `similarity_score` field (0–1, higher +is closer): + +```json +[ + { + "id": "mem_abc123", + "key": "auth-design", + "value": "We use short-lived JWTs issued by the platform and refreshed via /auth/token.", + "similarity_score": 0.91, + "created_at": "2026-04-10T14:22:00Z" + } +] +``` + +**Graceful fallback**: if no embedding function is configured, or if the +embedding call fails for a given query, the platform falls back transparently +to the text-search path. The `similarity_score` field is absent in fallback +responses. You do not need to change client code to handle both modes. + +--- + +## Workflow Checkpoints + +Step-level progress persistence for long-running Temporal workflows. Workspaces with `runtime: langgraph` (Temporal) automatically save a checkpoint after each of the three workflow stages (`task_receive`, `llm_call`, `task_complete`) and resume from the last completed stage on restart. + + + **Automatic resume behavior (runtime: langgraph only)** + + When a Temporal workspace restarts mid-workflow, the runtime reads the highest-index checkpoint and sets `resume_from_step` accordingly. Already-completed stages are skipped — the agent picks up exactly where it left off without re-running earlier steps. + + Checkpoint I/O is non-fatal: network errors are silently swallowed. A crashed or unreachable platform never prevents the agent from running. + + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| POST | `/workspaces/:id/checkpoints` | WorkspaceAuth | Upsert a step checkpoint. Body: `{ "workflow_id": "...", "step_name": "task_receive\|llm_call\|task_complete", "step_index": 0, "payload": {...} }`. Uses `ON CONFLICT DO UPDATE` — safe to call multiple times. | +| GET | `/workspaces/:id/checkpoints/:wfid` | WorkspaceAuth | Return all checkpoints for a workflow, ordered by `step_index DESC`. Returns 404 if no checkpoints exist for the workflow. | +| DELETE | `/workspaces/:id/checkpoints/:wfid` | WorkspaceAuth | Clear all checkpoints for a workflow. Called by the runtime on clean task completion. Returns 404 if none exist. | + +**Step names and indices:** + +| Step | `step_index` | Meaning | +|------|-------------|---------| +| `task_receive` | 0 | Task received from A2A message | +| `llm_call` | 1 | LLM inference completed | +| `task_complete` | 2 | Task result sent back to caller | + +--- diff --git a/content/docs/api-reference/meta.json b/content/docs/api-reference/meta.json new file mode 100644 index 0000000..e25feb1 --- /dev/null +++ b/content/docs/api-reference/meta.json @@ -0,0 +1,4 @@ +{ + "title": "API Reference", + "pages": ["index", "workspaces", "communication", "memory", "integrations", "realtime"] +} diff --git a/content/docs/api-reference/realtime.mdx b/content/docs/api-reference/realtime.mdx new file mode 100644 index 0000000..c19a812 --- /dev/null +++ b/content/docs/api-reference/realtime.mdx @@ -0,0 +1,202 @@ +--- +title: "Real-time & Observability API" +description: "Canvas, traces, audit ledger, events, terminal, WebSocket, and Server-Sent Events endpoints." +--- + +## Canvas + +Canvas viewport persistence (cosmetic only). + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/canvas/viewport` | None | Get the saved canvas viewport (zoom, pan position). Open endpoint for bootstrap-friendliness. | +| PUT | `/canvas/viewport` | CanvasOrBearer | Save the canvas viewport. Accepts bearer OR matching `Origin` header. Worst case on forgery: viewport corruption, recovered by page refresh. | + +--- + +## Traces + +LLM trace retrieval from Langfuse. + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/workspaces/:id/traces` | WorkspaceAuth | List LLM traces for a workspace from Langfuse. | + +--- + +## Audit Ledger + +HMAC-SHA256-chained immutable agent event log for compliance record-keeping (EU AI Act Art. 12 / Art. 13). Each event is cryptographically chained to the previous one — tampering with any record breaks all subsequent HMACs. + + + **`AUDIT_LEDGER_SALT` required.** The platform and workspace containers must share the same `AUDIT_LEDGER_SALT` environment variable to compute and verify event HMACs. Set it in both your platform env and workspace container env. If the variable is absent, `chain_valid` returns `null` (not `false`) — no records are lost, verification is simply unavailable. + + +### Query + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/workspaces/:id/audit` | WorkspaceAuth | Query the audit ledger for a workspace. Returns events in descending chronological order with inline chain verification. | + +**Query parameters:** + +| Parameter | Type | Description | +|-----------|------|-------------| +| `agent_id` | string | Filter to a specific agent. | +| `session_id` | string | Filter to a specific session. | +| `from` | RFC 3339 | Start of time range (e.g. `2026-04-01T00:00:00Z`). | +| `to` | RFC 3339 | End of time range. | +| `limit` | int | Max records to return. Capped at **500**. | +| `offset` | int | Pagination offset. | + +**Response shape:** + +```json +{ + "events": [ + { + "id": "uuid", + "workspace_id": "uuid", + "agent_id": "my-researcher", + "session_id": "sess_abc123", + "event_type": "tool_call", + "payload": { "tool": "bash", "input": "ls /workspace" }, + "hmac": "sha256hex...", + "prev_hmac": "sha256hex...", + "created_at": "2026-04-17T12:00:00Z" + } + ], + "chain_valid": true +} +``` + +`chain_valid` values: +- `true` — all HMACs verified; ledger is intact. +- `false` — at least one HMAC mismatch; possible tampering. +- `null` — `AUDIT_LEDGER_SALT` is absent from the platform env; verification skipped. + +### Workspace-side: recording events + +In your workspace template, wire `LedgerHooks` into the agent pipeline: + +```python +from molecule_audit.hooks import LedgerHooks + +hooks = LedgerHooks(agent_id="my-researcher", session_id=session_id) + +async with hooks: + # hooks.on_task_start / on_llm_call / on_tool_call / on_task_end + # fire automatically at each pipeline stage + result = await agent.run(task) +``` + +`LedgerHooks` is exception-safe — a failed ledger write never aborts the agent task. + +### CLI chain verification + +```bash +# Verify the full chain for an agent; exit 0 = intact +python -m molecule_audit.verify --agent-id my-researcher + +# Custom DB URL +python -m molecule_audit.verify --agent-id my-researcher --db postgresql://user:pass@host/db +``` + +Exit codes: `0` = chain valid · `1` = broken chain · `2` = `AUDIT_LEDGER_SALT` missing · `3` = DB error. + +--- + +## Events + +Append-only event log for structure changes. + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/events` | AdminAuth | List all structure events across all workspaces. | +| GET | `/events/:workspaceId` | AdminAuth | List structure events for a specific workspace. | + +--- + +## Terminal + +WebSocket-based terminal access to workspace containers. + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| WS | `/workspaces/:id/terminal` | WorkspaceAuth | Open a WebSocket terminal session to the workspace container. Provides interactive shell access. | + +--- + +## WebSocket + +Real-time event streaming for Canvas clients. + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| WS | `/ws` | None | Connect to the WebSocket hub. Receives all structure events (`WORKSPACE_ONLINE`, `WORKSPACE_OFFLINE`, `HEARTBEAT`, `CONFIG_UPDATED`, `A2A_RESPONSE`, `AGENT_MESSAGE`, etc.). Canvas clients connect here for real-time updates. | + +--- + +## Server-Sent Events (AG-UI) + +Per-workspace SSE stream compatible with the [AG-UI protocol](https://github.com/ag-ui-protocol/ag-ui). Use this endpoint to consume structured agent events from a web client or external tool without a WebSocket library. + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/workspaces/:id/events/stream` | WorkspaceAuth | Open an SSE stream for the workspace. Returns `Content-Type: text/event-stream`. Sends an initial `: ping` comment on connect, then delivers every event emitted by the workspace in AG-UI envelope format. Events from other workspaces are filtered out. Returns `404` if the workspace does not exist. | + +### Event envelope format + +Each event is delivered as an SSE `data:` line containing a JSON object: + +```json +{ + "type": "AGENT_MESSAGE", + "timestamp": 1713398400000, + "data": { ... } +} +``` + +- **`type`** — event type string (e.g. `AGENT_MESSAGE`, `A2A_RESPONSE`, `TASK_UPDATED`) +- **`timestamp`** — Unix milliseconds at time of broadcast +- **`data`** — event-specific payload (same payload as the WebSocket hub delivers) + +### Event types streamed + +All event types emitted by `RecordAndBroadcast` **and** `BroadcastOnly` reach the SSE stream. The `BroadcastOnly` path is important: events like `AGENT_MESSAGE`, `A2A_RESPONSE`, and `TASK_UPDATED` skip Redis and would be invisible to a Redis-only subscriber — the in-process SSE layer catches them. + +### Example: connect with `curl` + +```bash +curl -N \ + -H "Authorization: Bearer " \ + http://localhost:8080/workspaces//events/stream +``` + +``` +: ping + +data: {"type":"AGENT_MESSAGE","timestamp":1713398401234,"data":{"text":"Starting task..."}} + +data: {"type":"TASK_UPDATED","timestamp":1713398405678,"data":{"status":"running"}} +``` + +### Example: connect from JavaScript + +```js +const es = new EventSource( + `/workspaces/${workspaceId}/events/stream`, + { headers: { Authorization: `Bearer ${token}` } } +); + +es.onmessage = (e) => { + const event = JSON.parse(e.data); + console.log(event.type, event.data); +}; +``` + + + The SSE endpoint uses WorkspaceAuth — the bearer token must be bound to the `:id` in the path. A token for workspace A cannot open a stream for workspace B. + + +--- diff --git a/content/docs/api-reference/workspaces.mdx b/content/docs/api-reference/workspaces.mdx new file mode 100644 index 0000000..384ed94 --- /dev/null +++ b/content/docs/api-reference/workspaces.mdx @@ -0,0 +1,149 @@ +--- +title: "Workspaces API" +description: "Workspace CRUD, lifecycle, budget, configuration, files, activity, and session search endpoints." +--- + +## Workspaces + +Core workspace CRUD and lifecycle operations. + +### CRUD + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| POST | `/workspaces` | AdminAuth | Create a new workspace. Accepts `name`, `runtime`, `template`, `parent_id`, `tier`, `workspace_dir`, and other fields. Runtime is auto-detected from template config if omitted (defaults to `langgraph`). | +| GET | `/workspaces` | AdminAuth | List all workspaces with status, runtime, agent card, position, and hierarchy info. | +| GET | `/workspaces/:id` | WorkspaceAuth | Get a single workspace by ID. | +| PATCH | `/workspaces/:id` | WorkspaceAuth | Update workspace fields. A workspace bearer token is always required — unauthenticated calls return 401. Validates field constraints: `name` ≤ 255 chars, `role` ≤ 1,000 chars, `model` and `runtime` ≤ 100 chars each; `name` and `role` must not contain newlines (`\\n`, `\\r`) or YAML-special characters (`{}[]|>*&!`). Oversized or invalid field values return 400. `:id` must be a valid UUID. Financial fields (`budget_limit`) are not accepted here — use `PATCH /workspaces/:id/budget` (AdminAuth). | +| DELETE | `/workspaces/:id` | AdminAuth | Delete a workspace. Stops the container, revokes all auth tokens, and removes all associated data. | + +### Lifecycle + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| POST | `/workspaces/:id/restart` | WorkspaceAuth | Restart the workspace container. Sends a `restart_context` A2A message after successful re-registration. | +| POST | `/workspaces/:id/pause` | WorkspaceAuth | Stop the container and set status to `paused`. Paused workspaces skip health sweep, liveness monitor, and auto-restart. Resume manually via `/resume`. | +| POST | `/workspaces/:id/resume` | WorkspaceAuth | Re-provision a paused workspace. Status transitions to `provisioning`. | +| POST | `/workspaces/:id/hibernate` | WorkspaceAuth | Immediately hibernate a workspace (stop container, set status to `hibernated`). Useful for manual cost control. See hibernation note below. | + + + **Workspace hibernation** + + A workspace with `hibernation_idle_minutes` set in its config will be **automatically hibernated** by the platform after that many idle minutes (no active tasks, no recent heartbeat). The monitor checks every 2 minutes. + + `hibernated` differs from `paused`: + - **`paused`** — manual, resumes only via `POST /resume`. + - **`hibernated`** — automatic (or via `POST /hibernate`), resumes **automatically** when an A2A message arrives. + + When a message is sent to a hibernated workspace, the platform returns: + ``` + HTTP 503 Retry-After: 15 + {"waking": true} + ``` + Callers should retry after ~15 seconds. The workspace typically returns to `online` within that window. + + To opt a workspace into auto-hibernation, add to its `config.yaml`: + ```yaml + hibernation_idle_minutes: 30 # hibernate after 30 min idle; null (default) = disabled + ``` + + **Atomic hibernation guarantee:** The platform uses a single atomic SQL claim (`UPDATE … WHERE active_tasks = 0`) before stopping the container. If a task arrives between the idle check and the container stop, the claim fails and hibernation is aborted — no in-flight tasks are silently lost. + + +### Budget + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/workspaces/:id/budget` | AdminAuth | Read a workspace's current spend and ceiling. Returns `budget_limit`, `monthly_spend`, and `budget_remaining` (all in USD cents). | +| PATCH | `/workspaces/:id/budget` | AdminAuth | Set or clear a workspace's monthly spend ceiling. Body: `{ "budget_limit": N }` (positive integer, USD cents) or `{ "budget_limit": null }` to remove the cap. Negative values → 400. Returns same shape as GET. | + +**Request / response shape:** + +```json +// PATCH request body +{ "budget_limit": 500 } // $5.00/month ceiling +{ "budget_limit": null } // no ceiling + +// GET and PATCH success response (200) +{ + "budget_limit": 500, // null when no ceiling + "monthly_spend": 312, // accumulated spend this period, USD cents + "budget_remaining": 188 // null when no ceiling; max(0, limit-spend) — can be negative +} +``` + + + **`budget_limit` and `monthly_spend` are absent from `GET /workspaces/:id`** + + Financial fields are stripped unconditionally from the workspace detail + response — they do not appear for any caller, authenticated or not. Always + use `GET /workspaces/:id/budget` (AdminAuth) to read spend data. + + `budget_limit` is also **not** accepted on the general `PATCH /workspaces/:id` + endpoint. Use the dedicated `/budget` route. + + + + **Enforcement and fail-open behaviour** + + When `monthly_spend >= budget_limit`, `POST /workspaces/:id/a2a` returns: + ``` + HTTP 402 Payment Required + {"error": "workspace budget limit exceeded"} + ``` + Channel sends (Slack, Telegram, Discord, Lark) are also budget-gated with + the same 402 response. The workspace itself is **not paused** — it keeps + running; only inbound A2A and channel traffic is blocked. + + **Fail-open:** if the budget check encounters a DB error, traffic is allowed + through rather than blocked. The spend ceiling is a soft guardrail, not a + hard guarantee. + + +--- + +## Configuration + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/workspaces/:id/config` | WorkspaceAuth | Get the workspace's `config.yaml` contents. | +| PATCH | `/workspaces/:id/config` | WorkspaceAuth | Update the workspace config. "Save & Restart" writes config and auto-restarts; "Save" writes only and shows a restart banner in the Canvas. | +| GET | `/workspaces/:id/model` | WorkspaceAuth | Get the workspace's current model selection. | +| PUT | `/workspaces/:id/model` | WorkspaceAuth | Set the workspace model (e.g. `anthropic:claude-sonnet-4-6`). | +| GET | `/workspaces/:id/provider` | WorkspaceAuth | Get the resolved LLM provider for the workspace's runtime + model. | +| PUT | `/workspaces/:id/provider` | WorkspaceAuth | Override the LLM provider for the workspace. | + +--- + +## Files + +Workspace file management. Files are stored in the workspace's config directory. + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/workspaces/:id/files` | WorkspaceAuth | List files in the workspace config directory. | +| GET | `/workspaces/:id/files/*path` | WorkspaceAuth | Read a specific file. | +| PUT | `/workspaces/:id/files/*path` | WorkspaceAuth | Write a file. Creates parent directories as needed. On SaaS workspaces (EC2, no Docker), routes via EC2 Instance Connect endpoint using an ephemeral SSH key pair — the key is scoped to the file-write operation and deleted within 30 seconds. Max payload ~10 MiB. Self-hosted Docker workspaces write via `docker cp` as before. | +| DELETE | `/workspaces/:id/files/*path` | WorkspaceAuth | Delete a file. | + +--- + +## Activity + +Activity logging and search for A2A communications, task updates, and agent logs. + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/workspaces/:id/activity` | WorkspaceAuth | List activity logs for a workspace. Supports `?source=canvas` or `?source=agent` filter, and `?type=delegation` for A2A topology overlay polling. | +| POST | `/workspaces/:id/activity` | WorkspaceAuth | Log an activity entry (used by workspace runtimes to self-report). | +| POST | `/workspaces/:id/notify` | WorkspaceAuth | Agent-to-user push message via WebSocket. Delivers a notification to connected Canvas clients. | + +--- + +## Session Search + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/workspaces/:id/session-search` | WorkspaceAuth | Search activity logs with filters for type, date range, and text content. Returns paginated results. | + +--- diff --git a/content/docs/architecture.mdx b/content/docs/architecture.mdx index 74d64cc..79a4c42 100644 --- a/content/docs/architecture.mdx +++ b/content/docs/architecture.mdx @@ -144,7 +144,7 @@ Set `SMOLAGENTS_ENV_DENYLIST=VAR1,VAR2` in the workspace's secrets to extend the 2. Truncated at 2 000 characters to prevent oversized payloads 3. HTML-entity-escaped to block social-engineering injections embedded in agent output -These controls complement the platform-level secret redaction described in the [API Reference](/docs/api-reference#agent-memories-hma-scoped). +These controls complement the platform-level secret redaction described in the [API Reference](/docs/api-reference/memory#agent-memories-hma-scoped). ### molecli diff --git a/content/docs/concepts.mdx b/content/docs/concepts.mdx index 608ae50..65110b0 100644 --- a/content/docs/concepts.mdx +++ b/content/docs/concepts.mdx @@ -38,7 +38,7 @@ Workspaces talk to each other via **A2A** (agent-to-agent) messages, routed by the platform. Communication rules: same workspace, siblings, and parent/child are allowed; everything else is denied. -See the [API Reference](/docs/api-reference#budget) for the full endpoint specification. +See the [API Reference](/docs/api-reference/workspaces#budget) for the full endpoint specification. ### Workspace status lifecycle @@ -52,7 +52,7 @@ See the [API Reference](/docs/api-reference#budget) for the full endpoint specif | `hibernated` | Auto-paused after idle timeout (or via `/hibernate`) | automatic on next A2A message | | `removed` | Deleted | — | -**Hibernation** is an opt-in automatic cost-saving mode. Set `hibernation_idle_minutes` in the workspace's `config.yaml` to enable it. When a hibernated workspace receives an A2A message, the platform wakes it automatically (returning `503 Retry-After: 15` while it comes online). See [API Reference — Lifecycle](/docs/api-reference#lifecycle) for the `/hibernate` endpoint and configuration details. +**Hibernation** is an opt-in automatic cost-saving mode. Set `hibernation_idle_minutes` in the workspace's `config.yaml` to enable it. When a hibernated workspace receives an A2A message, the platform wakes it automatically (returning `503 Retry-After: 15` while it comes online). See [API Reference — Lifecycle](/docs/api-reference/workspaces#lifecycle) for the `/hibernate` endpoint and configuration details. ## External agents diff --git a/content/docs/hermes.mdx b/content/docs/hermes.mdx index e805f49..928f502 100644 --- a/content/docs/hermes.mdx +++ b/content/docs/hermes.mdx @@ -343,7 +343,7 @@ If you are routing a Gemini model through a key that triggers the compat shim (e ## See also - [Concepts — Workspaces](/docs/concepts#workspaces) -- [API Reference — POST /workspaces](/docs/api-reference#workspaces) +- [API Reference — POST /workspaces](/docs/api-reference/workspaces#workspaces) - [Google ADK Runtime](/docs/google-adk) — Gemini-native alternative to Hermes for ADK-first workflows - PR #240: [Phase 2a — native Anthropic dispatch](https://git.moleculesai.app/molecule-ai/molecule-core/pull/240) - PR #255: [Phase 2b — native Gemini dispatch](https://git.moleculesai.app/molecule-ai/molecule-core/pull/255) diff --git a/content/docs/workspace-config.mdx b/content/docs/workspace-config.mdx index 3027e0c..c226bd0 100644 --- a/content/docs/workspace-config.mdx +++ b/content/docs/workspace-config.mdx @@ -168,5 +168,5 @@ This header is added automatically by the workspace executor when `task_budget > - [Concepts — Workspaces](/docs/concepts#workspaces) — workspace primitives overview - [Org Template](/docs/org-template) — deploy effort/task_budget settings across an entire team via `org.yaml` - [Observability](/docs/observability) — monitor token usage per workspace to tune your budget settings -- [API Reference — POST /workspaces](/docs/api-reference#workspaces) +- [API Reference — POST /workspaces](/docs/api-reference/workspaces#workspaces) - [Claude Opus 4.7 — Anthropic docs](https://platform.claude.com/docs/) — upstream reference for `output_config` \ No newline at end of file