molecule-core/canvas
Hongming Wang 95fdf86187 feat(canvas/chat): inline video + audio HTML5 native players (RFC #2991 PR-2)
Second specialized renderer pair landing under RFC #2991. Stacks on
PR-1 (#2997) — extends the AttachmentPreview dispatcher with video/
audio cases.

Why HTML5-native (not custom JS player)
---------------------------------------

- Browser vendors ship hardware-accelerated decoders, captions,
  pinch + scrub UX, and fullscreen UI. We get all of it for free.
- Native fullscreen via the <video> control bar — no
  AttachmentLightbox needed for video (the browser's built-in
  fullscreen handles it).
- Mobile-friendly without us writing the touch handlers.

Auth model
----------

Identical to AttachmentImage (PR-1): platform-auth URIs need our
cookie/token, so we fetch the bytes, wrap in a Blob, hand the
browser an ObjectURL via <video src=> / <audio src=>. External
http(s) URIs skip the fetch.

Memory caveat: a Blob holds the entire media in JS memory until the
bubble unmounts. The server's 25MB single-file cap (chat_files.go)
bounds this; v2 can switch to MediaSource + streaming if larger
files become a real shape.

Failure modes
-------------

- Fetch failure (404, 403, network) → AttachmentChip fallback.
- Bytes that aren't valid media (corrupt, wrong Content-Type) →
  <video onError> / <audio onError> swap to chip.

Tests
-----

5 new component tests in AttachmentPreview.test.tsx (now 14 total):
  - kind=video → <video controls> with blob URL src
  - kind=video fetch fails → falls back to chip
  - kind=video extension fallback (no mime) → routes to video path
  - kind=audio → <audio controls> + filename label visible
  - kind=audio fetch fails → falls back to chip

The preview-kind unit tests from PR-1 (49 cases) already cover the
MIME → video / audio dispatch logic; this PR's component tests pin
the rendered DOM shape (controls attribute, blob URL src, fallback
behavior).

Hostile self-review
-------------------

1. Memory bound: 25MB cap protects us today; documented future
   migration path (MediaSource).
2. iOS Safari autoplay: playsInline pinned on <video> so mobile
   doesn't auto-fullscreen on play.
3. Captions accessibility: <track kind="captions" /> placeholder so
   the element is tagged correctly even though we don't have caption
   files yet (forward-compatible).

Verified
- tsc --noEmit clean
- 173 chat tests green (49 unit + 14 component + 110 pre-existing)

Stacks on PR-1 (#2997). PR-3 (PDF + text/code) is the final piece.

Refs RFC #2991, PR #2997 (PR-1).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 20:10:19 -07:00
..
e2e canvas/e2e: surface admin-orgs row + workspace body on failure 2026-05-03 14:01:50 -07:00
public chore: replace brand icon and add HANDOFF.md 2026-04-13 13:03:40 -07:00
src feat(canvas/chat): inline video + audio HTML5 native players (RFC #2991 PR-2) 2026-05-05 20:10:19 -07:00
.env.example fix(canvas): close 4 gaps in WS status indicator (env, toast, tests) 2026-04-14 08:26:38 +00:00
.gitignore feat(canvas): SaaS cross-origin — slug header + cookie credentials (Phase F) 2026-04-14 20:08:39 -07:00
components.json fix(canvas): restore text-white on saturated buttons + close zinc gaps 2026-05-03 02:04:20 -07:00
Dockerfile chore(canvas): upgrade node:20-alpine → node:22-alpine 2026-04-24 18:54:30 +00:00
next.config.ts fix(canvas,dotenv): review-driven hardening of fit gate + parser parity 2026-04-24 22:23:51 -07:00
package-lock.json fix(canvas): regenerate lockfile with cross-platform optional deps 2026-05-03 01:52:42 -07:00
package.json feat(canvas): warm-paper theme + Tailwind v4 migration 2026-05-03 01:43:55 -07:00
playwright.config.ts initial commit — Molecule AI platform 2026-04-13 11:55:37 -07:00
playwright.staging.config.ts feat(e2e): canary + canvas Playwright workflows; delegation mechanics 2026-04-21 04:15:10 -07:00
postcss.config.js feat(canvas): warm-paper theme + Tailwind v4 migration 2026-05-03 01:43:55 -07:00
tsconfig.json initial commit — Molecule AI platform 2026-04-13 11:55:37 -07:00
vitest.config.ts feat(canvas): vitest coverage instrumentation (#1815, no CI gate yet) 2026-04-26 23:44:07 -07:00