molecule-core/tests/e2e/test_chat_upload_e2e.sh
Hongming Wang e632a31347 feat(chat_files): rewrite Upload as HTTP-forward to workspace (RFC #2312, PR-C)
Closes the SaaS upload gap (#2308) with the unified architecture from
RFC #2312: same code path on local Docker and SaaS, no Docker socket
dependency, no `dockerCli == nil` cliff. Stacked on PR-A (#2313) +
PR-B (#2314).

Before:
  Upload → findContainer (nil in SaaS) → 503

After:
  Upload → resolve workspaces.url + platform_inbound_secret
        → stream multipart to <url>/internal/chat/uploads/ingest
        → forward response back unchanged

Same call site whether the workspace runs on local docker-compose
("http://ws-<id>:8000") or SaaS EC2 ("https://<id>.<tenant>...").
The bug behind #2308 cannot exist by construction.

Why streaming, not parse-then-re-encode:
  * No 50 MB intermediate buffer on the platform
  * Per-file size + path-safety enforcement is the workspace's job
    (see workspace/internal_chat_uploads.py, PR-B)
  * Workspace's error responses (413 with offending filename, 400 on
    missing files field, etc.) propagate through unchanged

Changes:
  * workspace-server/internal/handlers/chat_files.go — Upload rewritten
    as a streaming HTTP proxy. Drops sanitizeFilename, copyFlatToContainer,
    and the entire docker-exec path. ChatFilesHandler gains an httpClient
    (broken out for test injection). Download stays docker-exec for now;
    follow-up PR will migrate it to the same shape.
  * workspace-server/internal/handlers/chat_files_external_test.go —
    deleted. Pinned the wrong-headed runtime=external 422 gate from
    #2309 (already reverted in #2311). Superseded by the proxy tests.
  * workspace-server/internal/handlers/chat_files_test.go — replaced
    sanitize-filename tests (now in workspace/tests/test_internal_chat_uploads.py)
    with sqlmock + httptest proxy tests:
      - 400 invalid workspace id
      - 404 workspace row missing
      - 503 platform_inbound_secret NULL (with RFC #2312 detail)
      - 503 workspaces.url empty
      - happy-path forward (asserts auth header, content-type forwarded,
        body streamed, response propagated back)
      - 413 from workspace propagated unchanged (NOT remapped to 500)
      - 502 on workspace unreachable (connect refused)
    Existing Download + ContentDisposition tests preserved.
  * tests/e2e/test_chat_upload_e2e.sh — single-script-everywhere E2E.
    Takes BASE as env (default http://localhost:8080). Creates a
    workspace, waits for online, mints a test token, uploads a fixture,
    reads it back via /chat/download, asserts content matches +
    bearer-required. Same script runs against staging tenants (set
    BASE=https://<id>.<tenant>.staging.moleculesai.app).

Test plan:
  * go build ./... — green
  * go test ./internal/handlers/ ./internal/wsauth/ — green (full suite)
  * tests/e2e/test_chat_upload_e2e.sh against local docker-compose
    after PR-A + PR-B + this PR all merge — TODO before merge

Refs #2312 (parent RFC), #2308 (chat upload 503 incident).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 14:26:37 -07:00

136 lines
5.7 KiB
Bash
Executable File

#!/usr/bin/env bash
# E2E for the v2 chat upload path (RFC #2312):
#
# POST /workspaces/:id/chat/uploads
# └─▶ platform Go workspace-server (proxies)
# └─▶ workspace's own /internal/chat/uploads/ingest
# └─▶ writes to /workspace/.molecule/chat-uploads
#
# The same script runs against ANY environment because the architecture
# is now uniform — local docker-compose, staging tenant, production
# health-probe — all hit the same call site with the same expected
# behavior. This is the design goal RFC #2312 set: "test local will
# pretty much match production."
#
# Required env:
# BASE default http://localhost:8080
# override to https://<id>.<tenant>.staging...
# WORKSPACE_RUNTIME default langgraph (any internal runtime)
#
# Exit codes:
# 0 upload + read-back round-trip succeeded
# 1 setup failed (couldn't create workspace, never came online, etc.)
# 2 upload returned non-2xx
# 3 upload succeeded but the file isn't readable via download
set -uo pipefail
BASE="${BASE:-http://localhost:8080}"
RUNTIME="${WORKSPACE_RUNTIME:-langgraph}"
PARENT=""
PARENT_TOK=""
# shellcheck disable=SC1091
source "$(dirname "$0")/_lib.sh"
cleanup() {
local rc=$?
set +e
if [ -n "$PARENT" ]; then
curl -sS -X DELETE "$BASE/workspaces/$PARENT?confirm=true&purge=true" \
${PARENT_TOK:+-H "Authorization: Bearer $PARENT_TOK"} >/dev/null 2>&1
fi
exit $rc
}
trap cleanup EXIT INT TERM
# ─── 1. Create workspace ───────────────────────────────────────────────
echo "[1/5] POST /workspaces (runtime=$RUNTIME)..."
P_RESP=$(curl -sS -X POST "$BASE/workspaces" \
-H "Content-Type: application/json" \
-d "{\"name\":\"e2e-chat-upload\",\"runtime\":\"$RUNTIME\",\"tier\":2}")
PARENT=$(echo "$P_RESP" | python3 -c "import sys,json; print(json.load(sys.stdin).get('id',''))" 2>/dev/null)
[ -n "$PARENT" ] || { echo " ✗ workspace create failed: $P_RESP"; exit 1; }
echo " ✓ workspace=$PARENT"
# ─── 2. Wait for online ────────────────────────────────────────────────
echo "[2/5] waiting for workspace online (up to 5min)..."
for i in $(seq 1 60); do
S=$(curl -sS "$BASE/workspaces/$PARENT" 2>/dev/null \
| python3 -c "import sys,json; d=json.load(sys.stdin); w=d.get('workspace') if isinstance(d.get('workspace'),dict) else d; print(w.get('status') or '')" 2>/dev/null)
[ $((i % 6)) -eq 1 ] && echo " attempt $i: status=$S"
[ "$S" = "online" ] && break
sleep 5
done
[ "$S" = "online" ] || { echo " ✗ workspace never online (last=$S)"; exit 1; }
echo " ✓ online"
# Mint a workspace bearer for the test (the auth needed to call
# /workspaces/:id/chat/uploads, which is wsAuth-gated).
PARENT_TOK=$(e2e_mint_test_token "$PARENT") || {
echo " ✗ couldn't mint test token (MOLECULE_ENV=production?)"
exit 1
}
# ─── 3. Upload a fixture ───────────────────────────────────────────────
echo "[3/5] POST /workspaces/$PARENT/chat/uploads ..."
FIXTURE=$(mktemp)
echo "e2e fixture content $(date +%s)" > "$FIXTURE"
EXPECTED=$(cat "$FIXTURE")
UPLOAD=$(curl -sS -X POST "$BASE/workspaces/$PARENT/chat/uploads" \
-H "Authorization: Bearer $PARENT_TOK" \
-F "files=@$FIXTURE;filename=greeting.txt;type=text/plain" \
-w "\nHTTP_CODE=%{http_code}\n")
CODE=$(echo "$UPLOAD" | grep -oE 'HTTP_CODE=[0-9]+' | cut -d= -f2)
BODY=$(echo "$UPLOAD" | sed '/^HTTP_CODE=/,$d')
echo " status=$CODE"
echo " body=$(echo "$BODY" | head -c 300)"
if [ "$CODE" != "200" ]; then
echo " ✗ upload returned $CODE"
rm -f "$FIXTURE"
exit 2
fi
URI=$(echo "$BODY" | python3 -c "import sys,json; d=json.load(sys.stdin); print(d['files'][0]['uri'])" 2>/dev/null)
NAME=$(echo "$BODY" | python3 -c "import sys,json; d=json.load(sys.stdin); print(d['files'][0]['name'])" 2>/dev/null)
SIZE=$(echo "$BODY" | python3 -c "import sys,json; d=json.load(sys.stdin); print(d['files'][0]['size'])" 2>/dev/null)
[ -n "$URI" ] || { echo " ✗ no URI in response"; rm -f "$FIXTURE"; exit 2; }
[ "$NAME" = "greeting.txt" ] || { echo " ✗ name mismatch: $NAME"; rm -f "$FIXTURE"; exit 2; }
[ "$SIZE" = "$(wc -c <"$FIXTURE" | tr -d ' ')" ] || { echo " ✗ size mismatch: $SIZE"; rm -f "$FIXTURE"; exit 2; }
echo " ✓ uri=$URI"
echo " ✓ name=$NAME size=$SIZE"
# Extract the absolute path inside the workspace (strip workspace: scheme).
PATH_IN_WS="${URI#workspace:}"
# ─── 4. Read it back via /chat/download ────────────────────────────────
echo "[4/5] GET /workspaces/$PARENT/chat/download?path=$PATH_IN_WS"
DOWNLOADED=$(curl -sS "$BASE/workspaces/$PARENT/chat/download?path=$PATH_IN_WS" \
-H "Authorization: Bearer $PARENT_TOK")
if [ "$DOWNLOADED" != "$EXPECTED" ]; then
echo " ✗ content mismatch"
echo " expected: $EXPECTED"
echo " got: $DOWNLOADED"
rm -f "$FIXTURE"
exit 3
fi
echo " ✓ round-trip content matches"
# ─── 5. Auth: bare upload without bearer is rejected ───────────────────
echo "[5/5] POST without bearer must be 401..."
NA_CODE=$(curl -sS -o /dev/null -w "%{http_code}" -X POST "$BASE/workspaces/$PARENT/chat/uploads" \
-F "files=@$FIXTURE")
if [ "$NA_CODE" != "401" ]; then
echo " ✗ expected 401 without bearer, got $NA_CODE"
rm -f "$FIXTURE"
exit 2
fi
echo " ✓ 401 without bearer"
rm -f "$FIXTURE"
echo ""
echo "✓ chat upload v2 (RFC #2312) end-to-end passed against $BASE"