molecule-ai/molecule-core
35
0
Fork 2
Code Issues 16 Pull Requests 2 Actions 49 Packages Projects Releases Wiki Activity
66142c1eab
molecule-core/scripts
History
Hongming Wang e955597a98 feat(chat_files): rewrite Download as HTTP-forward (RFC #2312, PR-D) 
Mirrors PR-C's Upload migration: replaces the docker-cp tar-stream
extraction with a streaming HTTP GET to the workspace's own
/internal/file/read endpoint. Closes the SaaS gap for downloads —
without this PR, GET /workspaces/:id/chat/download still returns 503
on Railway-hosted SaaS even after A+B+C+F land.

Stacks: PR-A #2313 → PR-B #2314 → PR-C #2315 → PR-F #2319 → this PR.

Why a single broad /internal/file/read instead of /internal/chat/download:

  Today's chat_files.go::Download already accepts paths under any of the
  four allowed roots {/configs, /workspace, /home, /plugins} — it's not
  strictly chat. Future PRs (template export, etc.) will reuse this
  endpoint via the same forward pattern; reusing avoids three near-
  identical handlers (one per domain) with duplicated path-safety logic.

Path safety is duplicated on platform + workspace sides — defence in
depth via two parallel checks, not "trust the workspace."

Changes:
  * workspace/internal_file_read.py — Starlette handler. Validates path
    (must be absolute, under allowed roots, no traversal, canonicalises
    cleanly). lstat (not stat) so a symlink at the path doesn't redirect
    the read. Streams via FileResponse (no buffering). Mirrors Go's
    contentDispositionAttachment for Content-Disposition header.
  * workspace/main.py — registers GET /internal/file/read alongside the
    POST /internal/chat/uploads/ingest from PR-B.
  * scripts/build_runtime_package.py — adds internal_file_read to
    TOP_LEVEL_MODULES so the publish-runtime cascade rewrites its
    imports correctly. Also includes the PR-B additions
    (internal_chat_uploads, platform_inbound_auth) since this branch
    was rooted before PR-B's drift-gate fix; merge-clean alphabetic
    additions.
  * workspace-server/internal/handlers/chat_files.go — Download
    rewritten as streaming HTTP GET forward. Resolves workspace URL +
    platform_inbound_secret (same shape as Upload), builds GET request
    with path query param, propagates response headers (Content-Type /
    Content-Length / Content-Disposition) + body. Drops archive/tar
    + mime imports (no longer needed). Drops Docker-exec branch entirely
    — Download is now uniform across self-hosted Docker and SaaS EC2.
  * workspace-server/internal/handlers/chat_files_test.go — replaces
    TestChatDownload_DockerUnavailable (stale post-rewrite) with 4
    new tests:
      - TestChatDownload_WorkspaceNotInDB → 404 on missing row
      - TestChatDownload_NoInboundSecret → 503 on NULL column
        (with RFC #2312 detail in body)
      - TestChatDownload_ForwardsToWorkspace_HappyPath → forward shape
        (auth header, GET method, /internal/file/read path) + headers
        propagated + body byte-for-byte
      - TestChatDownload_404FromWorkspacePropagated → 404 from
        workspace propagates (NOT remapped to 500)
    Existing TestChatDownload_InvalidPath path-safety tests preserved.
  * workspace/tests/test_internal_file_read.py — 21 tests covering
    _validate_path matrix (absolute, allowed roots, traversal, double-
    slash, exact-match-on-root), 401 on missing/wrong/no-secret-file
    bearer, 400 on missing path/outside-root/traversal, 404 on missing
    file, happy-path streaming with correct Content-Type +
    Content-Disposition, special-char escaping in Content-Disposition,
    symlink-redirect-rejection (lstat-not-stat protection).

Test results:
  * go test ./internal/handlers/ ./internal/wsauth/ — green
  * pytest workspace/tests/ — 1292 passed (was 1272 before PR-D)

Refs #2312 (parent RFC), #2308 (chat upload+download 503 incident).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 15:19:02 -07:00
..
ops ops: add Railway SHA-pin drift audit script + regression test (#2001) 2026-04-27 05:01:23 -07:00
build_runtime_package.py feat(chat_files): rewrite Download as HTTP-forward (RFC #2312, PR-D) 2026-04-29 15:19:02 -07:00
build-images.sh initial commit — Molecule AI platform 2026-04-13 11:55:37 -07:00
bundle-compile.sh initial commit — Molecule AI platform 2026-04-13 11:55:37 -07:00
canary-smoke.sh feat(canary): smoke harness + GHA verification workflow (Phase 2) 2026-04-19 03:30:19 -07:00
cleanup-rogue-workspaces.sh fix(provisioner): stop rogue config-missing restart loop (#17) 2026-04-14 07:32:58 -07:00
clone-manifest.sh fix(quickstart): wire up template/plugin registry via manifest.json 2026-04-23 14:55:34 -07:00
dev-start.sh feat(dev-start): true single-command spinup — infra + templates + auth posture 2026-04-27 16:29:37 -07:00
import-agent.sh initial commit — Molecule AI platform 2026-04-13 11:55:37 -07:00
lockdown-tenant-sg.sh feat(security): Phase 35.1 — SG lockdown script for tenant EC2 instances 2026-04-18 12:01:41 -07:00
measure-coordinator-task-bounds-runner.sh fix(harness-runner): switch from non-existent /heartbeat-history to /activity 2026-04-28 23:12:51 -07:00
measure-coordinator-task-bounds.sh docs: registry pattern + harness scripts READMEs 2026-04-28 22:19:40 -07:00
nuke-and-rebuild.sh fix(scripts): nuke-and-rebuild self-bootstraps templates; add E2E test 2026-04-26 14:37:04 -07:00
post-rebuild-setup.sh security: remove hardcoded API keys from post-rebuild-setup.sh 2026-04-20 13:02:52 -07:00
README.md docs(scripts): rename /heartbeat-history → /activity in README 2026-04-29 02:23:00 -07:00
refresh-workspace-images.sh feat(platform/admin): /admin/workspace-images/refresh + Docker SDK + GHCR auth 2026-04-26 10:17:21 -07:00
rollback-latest.sh fix(scripts): correct platform dir path + add ROOT isolation (shellcheck clean) 2026-04-22 15:42:24 +00:00
test-a2a-cross-runtime.sh initial commit — Molecule AI platform 2026-04-13 11:55:37 -07:00
test-all-adapters.sh initial commit — Molecule AI platform 2026-04-13 11:55:37 -07:00
test-all.sh initial commit — Molecule AI platform 2026-04-13 11:55:37 -07:00
test-cross-agent-chat.sh initial commit — Molecule AI platform 2026-04-13 11:55:37 -07:00
test-nuke-and-rebuild.sh fix(scripts): nuke-and-rebuild self-bootstraps templates; add E2E test 2026-04-26 14:37:04 -07:00
test-team-e2e.sh initial commit — Molecule AI platform 2026-04-13 11:55:37 -07:00

README.md

scripts/

Operational and one-off scripts for molecule-core. Most are self-documenting — see the header comments in each file.

RFC #2251 coordinator task-bound harnesses

There are three related scripts; pick the right one:

Script Purpose Targets
measure-coordinator-task-bounds.sh Canonical v1 harness for the RFC #2251 / Issue 4 reproduction. Provisions a PM coordinator + Researcher child via claude-code-default + langgraph templates, sends a synthesis-heavy A2A kickoff, observes elapsed time + activity trace. OSS-shape platform — localhost or any /workspaces-shaped endpoint. Has tenant/admin-token guards for non-localhost runs.
measure-coordinator-task-bounds-runner.sh Generalised runner for the same measurement contract but with arbitrary template + secret + model combinations (Hermes/MiniMax, etc.). Useful for cross-runtime variants without modifying the canonical harness. Same as above (local or SaaS via MODE=saas).
measure-coordinator-task-bounds.sh (in molecule-controlplane) Production-shape variant that bootstraps a real staging tenant via POST /cp/admin/orgs, then runs the same measurement against <slug>.staging.moleculesai.app. Staging controlplane only — refuses to run against production.

See reference_harness_pair_pattern (auto-memory) for when to use which and the cross-repo design rationale.

Common safety pattern across all three

  • Cleanup trap on EXIT/INT/TERM auto-deletes provisioned resources.
  • DRY_RUN=1 prints plan + auth fingerprint, exits before any state mutation. Run this before pointing at staging or any shared infrastructure.
  • Non-target guard refuses arbitrary endpoints (the controlplane variant is locked to staging-api.moleculesai.app; the OSS variant requires explicit auth + tenant scoping for non-localhost PLATFORM).
  • Cleanup failures emit cleanup_*_failed events with remediation hints; no silenced curl. ADMIN_TOKEN expiring mid-run surfaces as a structured event rather than a silent leak.

Activity trace caveat

If activity_trace.raw == "<endpoint_unavailable>", the per-workspace /activity endpoint isn't wired on the target build — the bound measurement is INCONCLUSIVE on the platform-ceiling question. Either wire the endpoint or replace with the equivalent Datadog query. Note that /activity accepts a since_secs query parameter; see the endpoint handler for the supported range.

Other scripts

  • cleanup-rogue-workspaces.sh — emergency teardown for leaked workspaces. Prompts for confirmation. Pair with the harnesses if a cleanup trap fails (see cleanup_*_failed events).
  • canary-smoke.sh — quick smoke test for canary releases.
  • dev-start.sh — local-dev platform bring-up.

The rest are self-documenting in their header comments.