molecule-core/workspace-server/migrations
Hongming Wang 86fdaad111 feat(rfc): poll-mode chat upload — phase 1 platform staging layer
External-runtime workspaces (registered via molecule connect, behind
NAT, no public callback URL) currently see HTTP 422 "workspace has no
callback URL" on every chat file upload. The only escape is to wrap the
laptop in ngrok / Cloudflare tunnel + re-register push-mode — a tax
that shouldn't exist for a one-line use case.

This phase introduces the platform-side staging layer that lets
canvas → external workspace uploads ride the same poll loop the inbox
already uses for text messages.

Architecture (mirrors inbox poll, SSOT principle):
  Canvas POST /chat/uploads (multipart)
      ↓ delivery_mode=poll
  Platform: chat_files.uploadPollMode
      ↓ pendinguploads.Storage.Put + LogActivity(chat_upload_receive)
  Workspace's existing inbox poller picks up the activity row (Phase 2)
  Workspace fetches: GET /workspaces/:id/pending-uploads/:fid/content
  Workspace acks:    POST /workspaces/:id/pending-uploads/:fid/ack

Pieces in this PR:
  * Migration 20260505100000 — pending_uploads table; partial indexes
    on unacked + expires_at for the workspace fetch + Phase 3 sweep
    hot paths. No FK to workspaces (audit retention), 24h hard TTL.
  * internal/pendinguploads — Storage interface + Postgres impl. Bytes
    inline (bytea) today; the interface lets a future PR replace with
    S3 (RFC #2789) by swapping one constructor. 100% test coverage on
    the Postgres impl via sqlmock-pinned SQL.
  * handlers.PendingUploadsHandler — GET /content + POST /ack endpoints.
    wsAuth-gated; cross-workspace bleed protection via per-row
    workspace_id check (token leak from A can't read B's pending bytes).
    Handler tests pin happy path + every 4xx/5xx mapping including
    cross-workspace + race-with-sweep.
  * chat_files.go — Upload poll-mode branch behind WithPendingUploads
    builder. Push-mode unchanged (regression-tested). Multipart parse
    + per-file sanitize + storage.Put + activity_logs row per file.
  * SanitizeFilename — Go mirror of workspace/internal_chat_uploads.py
    sanitize_filename. Tests pin parity case-by-case so canvas-emitted
    URIs stay identical regardless of which path handles the upload.
  * Comprehensive logging — every state transition (staged, fetch,
    ack, error) emits a structured log line with workspace_id +
    file_id + size + sanitized name. Phase 3 metrics will hook these.

The pendinguploads.Storage wiring is opt-in (WithPendingUploads on
ChatFilesHandler) so a binary deployed without the migration keeps the
pre-existing 422 behavior — no boot-order coupling between code roll
and schema roll.

Phase 2 (separate PR): workspace inbox extension — inbox_uploads.py
fetches via the GET endpoint, writes to /workspace/.molecule/chat-
uploads/, acks, and rewrites the URI from platform-pending: → workspace:
so the agent's existing send-attachments path needs no changes.
Phase 3: GC sweep + dashboards. Phase 4: poll-mode E2E on staging.

Tests:
  * 100% coverage on pendinguploads (sqlmock-pinned SQL drift gate).
  * Functional 100% on new handler code (uncovered branches are
    documented defensive duplicates: uuid re-parse, multipart Open
    error, Writer.Write fail — none reproducible in unit tests).
  * Push-mode + NULL delivery_mode regression tests pin no behavior
    change for existing workspaces.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 04:22:24 -07:00
..
001_workspaces.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
002_agents.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
003_events.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
004_secrets.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
005_canvas_layouts.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
006_workspace_config_memory.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
007_approvals.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
008_agent_memories.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
009_activity_logs.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
010_workspace_awareness.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
011_workspace_runtime.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
012_global_secrets.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
013_workspace_dir.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
014_indexes.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
015_workspace_schedules.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
016_workspace_channels.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
017_memories_fts_namespace.down.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
017_memories_fts_namespace.up.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
018_secrets_encryption_version.down.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
018_secrets_encryption_version.up.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
019_workspace_access.down.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
019_workspace_access.up.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
020_workspace_auth_tokens.down.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
020_workspace_auth_tokens.up.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
021_delegation_idempotency.down.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
021_delegation_idempotency.up.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
022_workspace_schedules_source.down.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
022_workspace_schedules_source.up.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
023_workspace_memory_version.down.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
023_workspace_memory_version.up.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
024_channel_budget.down.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
024_channel_budget.up.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
025_workspace_token_usage.down.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
025_workspace_token_usage.up.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
026_org_plugin_allowlist.down.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
026_org_plugin_allowlist.up.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
027_workspace_budget.down.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
027_workspace_budget.up.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
028_workspace_artifacts.down.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
028_workspace_artifacts.up.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
029_workspace_hibernation.down.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
029_workspace_hibernation.up.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
030_audit_events.down.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
030_audit_events.up.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
031_memories_pgvector.down.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
031_memories_pgvector.up.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
032_schedule_consecutive_empty.down.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
032_schedule_consecutive_empty.up.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
033_strip_crlf_cron_prompts.up.sql fix(scheduler): strip CRLF from cron prompts on insert/update (closes #958) 2026-04-18 07:45:14 -07:00
034_workspaces_last_outbound_at.up.sql feat(platform): track last_outbound_at for silent-workspace detection (closes #817) 2026-04-18 13:04:54 -07:00
035_org_api_tokens.down.sql feat(auth): organization-scoped API keys for admin access 2026-04-20 14:01:41 -07:00
035_org_api_tokens.up.sql feat(auth): organization-scoped API keys for admin access 2026-04-20 14:01:41 -07:00
036_org_api_tokens_org_id.down.sql fix(auth): F1094 — requireCallerOwnsOrg reads org_id not created_by (#1234) 2026-04-21 02:47:12 +00:00
036_org_api_tokens_org_id.up.sql fix(auth): F1094 — requireCallerOwnsOrg reads org_id not created_by (#1234) 2026-04-21 02:47:12 +00:00
037_max_concurrent_tasks.down.sql fix: CWE-78 rm scope, go vet failures, delegation idempotency 2026-04-21 18:22:30 +00:00
037_max_concurrent_tasks.up.sql fix: CWE-78 rm scope, go vet failures, delegation idempotency 2026-04-21 18:22:30 +00:00
038_workspace_instance_id.down.sql feat(workspace): persist CP-returned EC2 instance_id on provision 2026-04-21 17:56:15 -07:00
038_workspace_instance_id.up.sql feat(workspace): persist CP-returned EC2 instance_id on provision 2026-04-21 17:56:15 -07:00
039_activity_tool_trace.down.sql feat: add tool_trace to activity_logs for platform-level agent observability 2026-04-22 15:17:14 -07:00
039_activity_tool_trace.up.sql feat: add tool_trace to activity_logs for platform-level agent observability 2026-04-22 15:17:14 -07:00
040_platform_instructions.down.sql feat: platform instructions system with global/team/workspace scope 2026-04-22 15:17:14 -07:00
040_platform_instructions.up.sql fix(review): address code review blockers on tool-trace + instructions 2026-04-22 16:18:06 -07:00
042_a2a_queue.down.sql feat(a2a): queue-on-busy — Phase 1 of priority queue (#1870) 2026-04-23 14:09:29 -07:00
042_a2a_queue.up.sql feat(a2a): queue-on-busy — Phase 1 of priority queue (#1870) 2026-04-23 14:09:29 -07:00
043_workspace_status_enum.down.sql chore: second-pass review polish — symmetry + clearer test fixtures 2026-04-25 08:48:30 -07:00
043_workspace_status_enum.up.sql fix: review-driven hardening of wedge detector + idle timeout + progress feed 2026-04-25 08:43:10 -07:00
044_platform_inbound_secret.down.sql feat(wsauth): platform→workspace inbound secret (RFC #2312, PR-A) 2026-04-29 14:09:33 -07:00
044_platform_inbound_secret.up.sql feat(wsauth): platform→workspace inbound secret (RFC #2312, PR-A) 2026-04-29 14:09:33 -07:00
045_workspaces_delivery_mode.down.sql feat(workspaces): delivery_mode column + poll-mode register flow (#2339 PR 1) 2026-04-29 21:47:14 -07:00
045_workspaces_delivery_mode.up.sql feat(workspaces): delivery_mode column + poll-mode register flow (#2339 PR 1) 2026-04-29 21:47:14 -07:00
046_workspace_status_awaiting_agent.down.sql fix(workspaces): add missing 'awaiting_agent' + 'hibernating' to workspace_status enum 2026-04-30 08:52:05 -07:00
046_workspace_status_awaiting_agent.up.sql fix(workspaces): add missing 'awaiting_agent' + 'hibernating' to workspace_status enum 2026-04-30 08:52:05 -07:00
047_runtime_image_pins.down.sql feat(provisioner): digest-pin workspace images via runtime_image_pins (#2272 layer 1) 2026-05-03 02:30:00 -07:00
047_runtime_image_pins.up.sql feat(provisioner): digest-pin workspace images via runtime_image_pins (#2272 layer 1) 2026-05-03 02:30:00 -07:00
048_activity_logs_peer_indexes.down.sql feat(db): add per-peer btree indexes on activity_logs for chat_history scale (#2478) 2026-05-03 11:34:35 -07:00
048_activity_logs_peer_indexes.up.sql feat(db): add per-peer btree indexes on activity_logs for chat_history scale (#2478) 2026-05-03 11:34:35 -07:00
049_delegations.down.sql feat(delegations): durable per-task ledger + audit-write helper (RFC #2829 PR-1) 2026-05-04 20:43:06 -07:00
049_delegations.up.sql feat(delegations): durable per-task ledger + audit-write helper (RFC #2829 PR-1) 2026-05-04 20:43:06 -07:00
20260417000000_workflow_checkpoints.down.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
20260417000000_workflow_checkpoints.up.sql chore: open-source restructure — rename dirs, remove internal files, scrub secrets 2026-04-18 00:24:44 -07:00
20260505100000_pending_uploads.down.sql feat(rfc): poll-mode chat upload — phase 1 platform staging layer 2026-05-05 04:22:24 -07:00
20260505100000_pending_uploads.up.sql feat(rfc): poll-mode chat upload — phase 1 platform staging layer 2026-05-05 04:22:24 -07:00