molecule-ai/molecule-core
53
0
Fork 2
Code Issues 187 Pull Requests 43 Actions 3 Packages Projects Releases Wiki Activity

RFC: Per-workspace EC2 configurability — instance type, volume size, OS, GPU #1686

New Issue
Open
opened 2026-05-22 21:19:21 +00:00 by hongming · 19 comments
hongming commented 2026-05-22 21:19:21 +00:00
Owner
Copy Link
Copy Source

Current state (2026-05-22)

Workspace EC2 provisioning is hardcoded in the control-plane provisioner:

  • OS / AMI: Ubuntu 24.04 (per docker inspect of any prod workspace EC2 — kernel 6.17.0-1012-aws)
  • Instance type: single hardcoded type — observed (need to confirm exact ID via aws ec2 describe-instances but appears t3-class or similar across all ws-tenant-* instances
  • Root volume: single hardcoded size (35% used at 48G observed → likely 64GB gp3)
  • No GPU SKU support: no g5.*/p4d.* path, no nvidia-container-runtime in the workspace template
  • No per-workspace customization: every workspace under every tenant gets the same shape, regardless of runtime needs (codex CLI is light, but a future model-serving workspace might need an A100)

This blocks several real use-cases:

  1. GPU-backed agents — running local inference (vLLM, ollama-with-CUDA, LLaVA, etc.) inside the workspace. Today the only path is API-out to third-party providers; can't run anything locally.
  2. High-RAM workloads — large codebase analysis, large doc embedding, training-data prep that wants >4GB.
  3. Larger persistent volumes — workspaces accumulating session state, build caches, sample data. We've seen Reno Stars SEO agent fill its /workspace over weeks.
  4. Different base OS — some users may want Alpine/Debian-slim, some may want a corporate base image (compliance/baseline).
  5. Pre-warming spot/savings-plan-priced fleets — pin a flavor to a specific savings plan for cost.

Proposed shape

Extend the workspace creation contract (the POST /cp/admin/workspaces / canvas-side Create flow) with a compute block. Validate at create time; provisioner reads it to drive RunInstances.

compute:
  instance_type: t3.medium       # default if unset: platform-managed (current behavior)
  os:                            # default ubuntu-24.04-arm64
    family: ubuntu               # {ubuntu, debian-slim, alpine, custom-ami}
    version: "24.04"
    arch: arm64                  # {arm64, x86_64}
  volume:
    root_gb: 64                  # default 64; min 32; max enforced per tenant tier
    iops: 3000                   # optional; gp3 only
  gpu:                           # optional block
    type: T4                     # {T4, L4, A10G, A100, H100, none}
    count: 1
    runtime: nvidia              # opts in nvidia-container-runtime in the image
  network:
    egress_open: true            # default true; future: restricted egress whitelist

Design considerations

  • Quotas: per-tenant tier caps (T1 sandboxed can't request A100; T4 unlimited)
  • Pricing surface: canvas must show estimated $/h per shape so users see cost before clicking Create
  • Image variants: GPU SKUs need a different workspace template image (nvidia-docker baked in). Either ship per-variant images OR detect GPU at boot and apt install nvidia bits (slower boot)
  • AMI rotation: today provisioner pins one AMI; need a map (os.family, os.version, arch, gpu) → AMI ID that's kept fresh by a maintenance job
  • Spot vs on-demand: cost-sensitive workloads (research, sandboxes) could opt into spot with checkpoint-on-interrupt semantics. Couples to the workspace-lifecycle MCP work (#642)
  • Defaults: existing workspaces get the current shape implicitly when compute: is absent — no breaking change
  • Re-provision flow: changing compute requires terminate + recreate (can't hot-swap instance type). Tie to RFC #642 workspace lifecycle MCP tools.

Phasing

  • Phase 1 (~2 weeks): schema in workspace create + provisioner reads instance_type, volume.root_gb. No GPU. No OS variants. Backwards-compatible default. Canvas Create-tab gets two new fields.
  • Phase 2 (~2 weeks): GPU block. Per-tenant quota enforcement. Image variant registry. Pricing surface.
  • Phase 3 (~3 weeks): OS variants (debian-slim, alpine). Custom-AMI escape hatch (advanced tier). Spot-friendly checkpoint semantics tied to #642.

Related

  • RFC #642 — workspace lifecycle MCP tools (recreate / restart / configure) — natural counterpart
  • RFC #640 — workspace EC2 logs to Loki — already in-flight
  • This RFC is part of the broader workspace-customization track

Open questions

  • Pricing/billing integration: do we surface AWS list price to users, or apply a markup? Margin policy needed before Phase 2.
  • Image variant maintenance burden: how often do we rebuild per-variant images? Reuse the cascade-rebuild pipeline RFC #596 already established?
  • GPU access in dev/staging — only allow in prod tier 4 at first?

Owner: TBD
Target: scope + design doc by Phase 1 kickoff
Filed by: Hongming (CTO) — request via canvas 2026-05-22 21:18 UTC

## Current state (2026-05-22) Workspace EC2 provisioning is hardcoded in the control-plane provisioner: - **OS / AMI**: Ubuntu 24.04 (per `docker inspect` of any prod workspace EC2 — kernel `6.17.0-1012-aws`) - **Instance type**: single hardcoded type — observed (need to confirm exact ID via `aws ec2 describe-instances` but appears t3-class or similar across all `ws-tenant-*` instances - **Root volume**: single hardcoded size (35% used at 48G observed → likely 64GB gp3) - **No GPU SKU support**: no `g5.*`/`p4d.*` path, no `nvidia-container-runtime` in the workspace template - **No per-workspace customization**: every workspace under every tenant gets the same shape, regardless of runtime needs (codex CLI is light, but a future model-serving workspace might need an A100) This blocks several real use-cases: 1. **GPU-backed agents** — running local inference (vLLM, ollama-with-CUDA, LLaVA, etc.) inside the workspace. Today the only path is API-out to third-party providers; can't run anything locally. 2. **High-RAM workloads** — large codebase analysis, large doc embedding, training-data prep that wants >4GB. 3. **Larger persistent volumes** — workspaces accumulating session state, build caches, sample data. We've seen Reno Stars SEO agent fill its `/workspace` over weeks. 4. **Different base OS** — some users may want Alpine/Debian-slim, some may want a corporate base image (compliance/baseline). 5. **Pre-warming spot/savings-plan-priced fleets** — pin a flavor to a specific savings plan for cost. ## Proposed shape Extend the workspace creation contract (the `POST /cp/admin/workspaces` / canvas-side Create flow) with a `compute` block. Validate at create time; provisioner reads it to drive `RunInstances`. ```yaml compute: instance_type: t3.medium # default if unset: platform-managed (current behavior) os: # default ubuntu-24.04-arm64 family: ubuntu # {ubuntu, debian-slim, alpine, custom-ami} version: "24.04" arch: arm64 # {arm64, x86_64} volume: root_gb: 64 # default 64; min 32; max enforced per tenant tier iops: 3000 # optional; gp3 only gpu: # optional block type: T4 # {T4, L4, A10G, A100, H100, none} count: 1 runtime: nvidia # opts in nvidia-container-runtime in the image network: egress_open: true # default true; future: restricted egress whitelist ``` ## Design considerations - **Quotas**: per-tenant tier caps (T1 sandboxed can't request A100; T4 unlimited) - **Pricing surface**: canvas must show estimated $/h per shape so users see cost before clicking Create - **Image variants**: GPU SKUs need a different workspace template image (nvidia-docker baked in). Either ship per-variant images OR detect GPU at boot and `apt install` nvidia bits (slower boot) - **AMI rotation**: today provisioner pins one AMI; need a map `(os.family, os.version, arch, gpu) → AMI ID` that's kept fresh by a maintenance job - **Spot vs on-demand**: cost-sensitive workloads (research, sandboxes) could opt into spot with checkpoint-on-interrupt semantics. Couples to the workspace-lifecycle MCP work (#642) - **Defaults**: existing workspaces get the current shape implicitly when `compute:` is absent — no breaking change - **Re-provision flow**: changing `compute` requires terminate + recreate (can't hot-swap instance type). Tie to RFC #642 workspace lifecycle MCP tools. ## Phasing - **Phase 1** (~2 weeks): schema in workspace create + provisioner reads `instance_type`, `volume.root_gb`. No GPU. No OS variants. Backwards-compatible default. Canvas Create-tab gets two new fields. - **Phase 2** (~2 weeks): GPU block. Per-tenant quota enforcement. Image variant registry. Pricing surface. - **Phase 3** (~3 weeks): OS variants (debian-slim, alpine). Custom-AMI escape hatch (advanced tier). Spot-friendly checkpoint semantics tied to #642. ## Related - RFC #642 — workspace lifecycle MCP tools (recreate / restart / configure) — natural counterpart - RFC #640 — workspace EC2 logs to Loki — already in-flight - This RFC is part of the broader workspace-customization track ## Open questions - Pricing/billing integration: do we surface AWS list price to users, or apply a markup? Margin policy needed before Phase 2. - Image variant maintenance burden: how often do we rebuild per-variant images? Reuse the cascade-rebuild pipeline RFC #596 already established? - GPU access in dev/staging — only allow in prod tier 4 at first? --- **Owner**: TBD **Target**: scope + design doc by Phase 1 kickoff **Filed by**: Hongming (CTO) — request via canvas 2026-05-22 21:18 UTC
hongming commented 2026-05-22 22:03:52 +00:00
Author
Owner
Copy Link
Copy Source

RFC addendum for review: desktop-control display workspaces

Follow-up from product discussion: the "display" requirement is not just Playwright/Puppeteer/headed-browser support. The desired workflow is native computer-level control:

  1. agent captures the full desktop screenshot
  2. model reasons over the visible screen
  3. agent sends OS-level mouse/keyboard actions
  4. desktop changes
  5. repeat

No DOM access, no browser protocol, no browser extension, no injected JS, no direct cookie/session manipulation. Browser use should be mechanically equivalent to normal desktop use: Chrome/Firefox running as regular apps on a visible desktop.

Proposed contract extension

Keep compute sizing separate from display capability:

compute:
  instance_type: m6i.xlarge
  volume:
    root_gb: 100
  display:
    mode: none | desktop-control | gpu-desktop-control
    width: 1920
    height: 1080
    protocol: dcv

Phase 1 should support only:

display:
  mode: desktop-control
  width: 1920
  height: 1080
  protocol: dcv

GPU should be a later explicit mode, not part of the first cut.

Recommended backend shape

A desktop-control workspace EC2 should run:

  • Ubuntu Desktop or lightweight XFCE/Xorg session
  • Chrome/Firefox as normal applications
  • Amazon DCV server for human viewing/takeover from Canvas
  • Molecule runtime container
  • a local desktop-control-sidecar for screenshot + input tools

The runtime/agent talks to the sidecar, not the browser.

Sidecar tool surface:

desktop.screenshot
desktop.get_screen_size
desktop.click
desktop.double_click
desktop.move_mouse
desktop.drag
desktop.type_text
desktop.press_key
desktop.wait

Optional later, with stronger gating because they can leak data or bypass visible-state reasoning:

desktop.set_clipboard
desktop.read_clipboard
desktop.focus_window
desktop.list_windows

Security/ops requirements:

  • sidecar is localhost/private only; no public exposure
  • require per-workspace auth token between runtime and sidecar
  • log every tool call with workspace ID, timestamp, action, coordinates/key metadata
  • do not log typed secret values
  • require a recent screenshot before sensitive/destructive actions
  • expose DCV only through platform-authenticated/proxied access; do not open raw DCV/VNC/RDP ports to the internet

Canvas changes requested

Add two tabs to workspace detail UI:

Container Config tab

Purpose: inspect/configure runtime/container-level settings, separate from EC2 sizing.

Initial contents:

  • runtime image/version, read-only at first
  • workspace access mode: none / read-only / read-write
  • max concurrent tasks
  • restart/reset-session controls
  • mounted workspace path/access status
  • container privilege/status flags, inspect-only initially
  • changes that require restart should be clearly marked

This should not be the EC2 shape editor. EC2 shape lives under create/recreate compute settings.

Display tab

Purpose: view and take over the desktop for display-enabled workspaces.

Behavior:

  • if compute.display.mode is absent or none, show unavailable state:
    • Display is not enabled for this workspace.
    • do not call/provision display session
    • future CTA can be Recreate with display
  • if desktop-control or gpu-desktop-control, show:
    • display status: provisioning / starting / ready / offline
    • resolution and protocol
    • Open Display / embedded DCV viewer
    • observe/takeover state
    • current controller: agent / user / none

Suggested display endpoint:

GET /workspaces/:id/display

Non-display response:

{
  "available": false,
  "reason": "display_not_enabled"
}

Display-enabled response:

{
  "available": true,
  "mode": "desktop-control",
  "status": "ready",
  "protocol": "dcv",
  "url": "short-lived-proxied-url",
  "expires_at": "2026-05-22T22:00:00Z",
  "controller": "agent"
}

Do not expose raw DCV credentials in the Canvas API response.

Control lock / takeover

Add a lightweight control-lock model so human and agent do not fight over mouse/keyboard:

{
  "controller": "agent" | "user" | "none",
  "controlled_by": "<user-or-workspace-id>",
  "expires_at": "..."
}

Canvas Display tab can support:

  • observe only
  • request takeover
  • release control
  • pause/resume agent desktop actions

Implementation phasing

  1. RFC/schema update: add compute.display and persist workspaces.compute JSONB.
  2. Expose compute in workspace read responses.
  3. Canvas: add Display tab with unavailable state first.
  4. Backend: add GET /workspaces/:id/display, returning available:false for non-display workspaces.
  5. Core→CP: forward sizing + display fields on /cp/workspaces/provision.
  6. CP: add desktop-control boot path or desktop AMI selection.
  7. Build desktop-control sidecar MVP.
  8. Runtime: expose desktop tools and require screenshot-grounded action loop.
  9. Canvas: wire DCV session URL/open viewer and takeover lock.
  10. Staging E2E: provision display workspace, screenshot non-empty, click/type changes screen, Canvas can open display.
  11. Follow-up: gpu-desktop-control with GPU instance allowlist, NVIDIA/DCV GPU image, tenant quotas, and cost warning.

Non-goals for Phase 1

  • no GPU support
  • no OS variants
  • no browser automation protocol
  • no Playwright/Puppeteer/DOM control path
  • no public VNC/DCV port exposure
  • no hot-swapping display mode on an existing EC2; changing display mode should require recreate/reprovision

Review questions

  • Should desktop-control default to m6i.xlarge/100GB, or is t3.xlarge/100GB acceptable for the first CPU desktop profile?
  • Should display-enabled workspaces require T4 access, or should display remain orthogonal to access tier? Recommendation: keep orthogonal.
  • Should Canvas embed DCV inline or open a separate authenticated display window? Recommendation: start with separate window, then embed once proxy/session handling is stable.
  • Should publish/reply/DM actions in social workflows require human approval by default? Recommendation: yes, with screenshots before/after and audit trail.
## RFC addendum for review: desktop-control display workspaces Follow-up from product discussion: the "display" requirement is not just Playwright/Puppeteer/headed-browser support. The desired workflow is **native computer-level control**: 1. agent captures the full desktop screenshot 2. model reasons over the visible screen 3. agent sends OS-level mouse/keyboard actions 4. desktop changes 5. repeat No DOM access, no browser protocol, no browser extension, no injected JS, no direct cookie/session manipulation. Browser use should be mechanically equivalent to normal desktop use: Chrome/Firefox running as regular apps on a visible desktop. ### Proposed contract extension Keep compute sizing separate from display capability: ```yaml compute: instance_type: m6i.xlarge volume: root_gb: 100 display: mode: none | desktop-control | gpu-desktop-control width: 1920 height: 1080 protocol: dcv ``` Phase 1 should support only: ```yaml display: mode: desktop-control width: 1920 height: 1080 protocol: dcv ``` GPU should be a later explicit mode, not part of the first cut. ### Recommended backend shape A `desktop-control` workspace EC2 should run: - Ubuntu Desktop or lightweight XFCE/Xorg session - Chrome/Firefox as normal applications - Amazon DCV server for human viewing/takeover from Canvas - Molecule runtime container - a local `desktop-control-sidecar` for screenshot + input tools The runtime/agent talks to the sidecar, not the browser. Sidecar tool surface: ```text desktop.screenshot desktop.get_screen_size desktop.click desktop.double_click desktop.move_mouse desktop.drag desktop.type_text desktop.press_key desktop.wait ``` Optional later, with stronger gating because they can leak data or bypass visible-state reasoning: ```text desktop.set_clipboard desktop.read_clipboard desktop.focus_window desktop.list_windows ``` Security/ops requirements: - sidecar is localhost/private only; no public exposure - require per-workspace auth token between runtime and sidecar - log every tool call with workspace ID, timestamp, action, coordinates/key metadata - do not log typed secret values - require a recent screenshot before sensitive/destructive actions - expose DCV only through platform-authenticated/proxied access; do not open raw DCV/VNC/RDP ports to the internet ### Canvas changes requested Add two tabs to workspace detail UI: #### Container Config tab Purpose: inspect/configure runtime/container-level settings, separate from EC2 sizing. Initial contents: - runtime image/version, read-only at first - workspace access mode: none / read-only / read-write - max concurrent tasks - restart/reset-session controls - mounted workspace path/access status - container privilege/status flags, inspect-only initially - changes that require restart should be clearly marked This should not be the EC2 shape editor. EC2 shape lives under create/recreate compute settings. #### Display tab Purpose: view and take over the desktop for display-enabled workspaces. Behavior: - if `compute.display.mode` is absent or `none`, show unavailable state: - `Display is not enabled for this workspace.` - do not call/provision display session - future CTA can be `Recreate with display` - if `desktop-control` or `gpu-desktop-control`, show: - display status: provisioning / starting / ready / offline - resolution and protocol - Open Display / embedded DCV viewer - observe/takeover state - current controller: agent / user / none Suggested display endpoint: ```http GET /workspaces/:id/display ``` Non-display response: ```json { "available": false, "reason": "display_not_enabled" } ``` Display-enabled response: ```json { "available": true, "mode": "desktop-control", "status": "ready", "protocol": "dcv", "url": "short-lived-proxied-url", "expires_at": "2026-05-22T22:00:00Z", "controller": "agent" } ``` Do not expose raw DCV credentials in the Canvas API response. ### Control lock / takeover Add a lightweight control-lock model so human and agent do not fight over mouse/keyboard: ```json { "controller": "agent" | "user" | "none", "controlled_by": "<user-or-workspace-id>", "expires_at": "..." } ``` Canvas Display tab can support: - observe only - request takeover - release control - pause/resume agent desktop actions ### Implementation phasing 1. RFC/schema update: add `compute.display` and persist `workspaces.compute` JSONB. 2. Expose `compute` in workspace read responses. 3. Canvas: add Display tab with unavailable state first. 4. Backend: add `GET /workspaces/:id/display`, returning `available:false` for non-display workspaces. 5. Core→CP: forward sizing + display fields on `/cp/workspaces/provision`. 6. CP: add desktop-control boot path or desktop AMI selection. 7. Build desktop-control sidecar MVP. 8. Runtime: expose desktop tools and require screenshot-grounded action loop. 9. Canvas: wire DCV session URL/open viewer and takeover lock. 10. Staging E2E: provision display workspace, screenshot non-empty, click/type changes screen, Canvas can open display. 11. Follow-up: `gpu-desktop-control` with GPU instance allowlist, NVIDIA/DCV GPU image, tenant quotas, and cost warning. ### Non-goals for Phase 1 - no GPU support - no OS variants - no browser automation protocol - no Playwright/Puppeteer/DOM control path - no public VNC/DCV port exposure - no hot-swapping display mode on an existing EC2; changing display mode should require recreate/reprovision ### Review questions - Should `desktop-control` default to `m6i.xlarge/100GB`, or is `t3.xlarge/100GB` acceptable for the first CPU desktop profile? - Should display-enabled workspaces require T4 access, or should display remain orthogonal to access tier? Recommendation: keep orthogonal. - Should Canvas embed DCV inline or open a separate authenticated display window? Recommendation: start with separate window, then embed once proxy/session handling is stable. - Should publish/reply/DM actions in social workflows require human approval by default? Recommendation: yes, with screenshots before/after and audit trail.
hongming commented 2026-05-22 22:57:12 +00:00
Author
Owner
Copy Link
Copy Source

Review of base RFC + desktop-control addendum

Strong design overall. Solid scoping, explicit non-goals, security genuinely thought through. Below are concerns + suggestions to fold in before someone starts implementation.

Strengths

  • Clean separation of compute sizing vs compute.display capability — orthogonal, both can evolve independently
  • Phase 1 scoped tightly (CPU only, DCV only, no GPU) — realistic ~2 week scope
  • Security genuinely thought through: localhost-only sidecar, per-workspace auth token, audit logging, screenshot-before-destructive, no public DCV/VNC ports, no secret values in logs
  • Control-lock model (controller=agent|user|none) addresses real human-agent contention
  • Explicit non-goals section — good RFC hygiene
  • Commitment to "screen reasoning" over DOM-bypass is the correct positioning

Gaps to close before implementation starts

1. State persistence is missing

The RFC doesn't say where browser cookies / Chrome profile / downloads live. Without a persistent volume mount, every container restart kills login sessions — the workflow is dead on the first restart. Tie this RFC to issue #35 (/home/agent/ volume mount) — every desktop-control workspace MUST have /home/agent/.config/google-chrome/ and downloads dir on a persistent mount. Add explicit acceptance: "restart preserves browser cookies + downloads dir."

2. Sidecar resilience model unspecified

What happens when the sidecar crashes mid-task? Spec lists tool surface but no supervision model. Add:

  • Sidecar runs under systemd (or docker restart: always)
  • Sidecar health endpoint that the runtime checks before sending tools
  • Agent gets a clear sidecar_unavailable error (not a hang) when sidecar is down
  • Audit log retention if sidecar restarts (don't lose history)

3. Screenshot cost / cadence not discussed

Every desktop.screenshot is a base64 PNG flowing to the model. A 1920x1080 PNG is ~500KB-2MB. At one screenshot per tool call, a 30-step workflow consumes ~30-60MB of model input. Cost-control surface:

  • Quality knob: screenshot.quality: low|medium|high to trade off model latency vs reasoning fidelity
  • Delta screenshots: track previous + send only changed regions (cheaper on second+ turns)
  • Document the per-turn token + $/run envelope in the RFC

4. DCV licensing + AMI prebake

Amazon DCV is free on EC2 but the Ubuntu DCV server install is non-trivial. The RFC should call out:

  • Pre-bake DCV into the AMI (don't apt install dcv-server at boot — slow + flaky)
  • Confirm DCV license terms support multi-tenant use case (workspace-per-customer)
  • TODO: validate with AWS legal/sales

5. Captcha / bot-detection wall (set expectations)

Real browsers operated by agents WILL trigger captcha / bot-detection on many sites (Cloudflare Bot Management, Google reCAPTCHA, etc.). Not a blocker for Phase 1, but the RFC should set expectations:

  • Document: "agents will hit captcha walls; expect human-in-the-loop for some workflows"
  • Add as non-goal: "bypassing bot detection is out of scope"

6. Control-lock semantics on conflict

"Request takeover" by user while agent is mid-action — what happens? Specify:

  • Agent yields at next decision point (between tool calls), gracefully releases lock
  • Force-takeover only after a 5-second grace timeout (configurable)
  • Active tool call is allowed to complete; queued tool calls are cancelled

7. Tool namespace alignment

desktop.screenshot / desktop.click — make the namespace explicit:

  • MCP tool group desktop_control_* exposed by the sidecar via the existing MCP transport
  • Adapters subscribe to this group same way they subscribe to a2a_* / molecule tools
  • This way claude-code / codex / hermes all get the same tool surface for free

8. Resource leak on user-close-of-browser

If user takes over and closes Chrome, agent's next screenshot returns "no window." Spec needs:

  • Sidecar supervises Chrome (auto-respawn if killed) OR
  • Explicit desktop.launch_browser tool the agent calls if it sees no browser open

9. Compute defaults — m6i.xlarge over t3.xlarge

The addendum asks: m6i.xlarge vs t3.xlarge for Phase 1 desktop profile. Recommendation: m6i.xlarge ($0.192/hr) over t3.xlarge ($0.166/hr). t3's burstable CPU credits will exhaust under sustained agent activity (browsing + screenshot encoding is CPU-heavy). The ~15% cost premium prevents support tickets.

10. Failure-mode states

RFC has provisioning | starting | ready | offline. Add one: ready-but-display-server-down — the EC2 is healthy and the agent is processing but DCV server crashed. Different operator action than offline.

11. Browser-update cadence

Phase 1 ships Chrome from the AMI bake. Chrome auto-updates by default → behavior may drift. Spec should commit to one of:

  • Pin Chrome version in AMI; deliberate bump via PR (recommended)
  • Allow Chrome auto-update; accept the drift risk

Answers to the four review questions

  1. m6i.xlarge over t3.xlarge for Phase 1 (see #9 above)
  2. Keep display orthogonal to tier (agree with addendum) BUT require T2+ minimum (T1 sandboxed shouldn't get a desktop)
  3. Separate window for Phase 1, embed in Phase 2 — agree
  4. Yes, require human approval for publish/reply/DM in social workflows, with before/after screenshots in the approval prompt — agree, right default

Suggested acceptance additions for Phase 1 E2E

The current Phase 1 E2E test in the addendum: "provision display workspace, screenshot non-empty, click/type changes screen, Canvas can open display."

Add:

  • Takeover lock works (agent + user don't fight)
  • Restart preserves browser session (cookie persistence via /home/agent/ volume mount per #1)
  • Sidecar auto-restarts on crash (per #2)
  • DCV access is platform-authenticated; raw DCV port not reachable from internet (per current security section)

Reviewer: hongming (CEO Assistant, post-codex-investigation). Happy to discuss any of these on a call before implementation kicks off.

## Review of base RFC + desktop-control addendum Strong design overall. Solid scoping, explicit non-goals, security genuinely thought through. Below are concerns + suggestions to fold in before someone starts implementation. ### Strengths - Clean separation of `compute` sizing vs `compute.display` capability — orthogonal, both can evolve independently - Phase 1 scoped tightly (CPU only, DCV only, no GPU) — realistic ~2 week scope - Security genuinely thought through: localhost-only sidecar, per-workspace auth token, audit logging, screenshot-before-destructive, no public DCV/VNC ports, no secret values in logs - Control-lock model (controller=agent|user|none) addresses real human-agent contention - Explicit non-goals section — good RFC hygiene - Commitment to "screen reasoning" over DOM-bypass is the correct positioning ### Gaps to close before implementation starts #### 1. State persistence is missing The RFC doesn't say where browser cookies / Chrome profile / downloads live. **Without a persistent volume mount, every container restart kills login sessions** — the workflow is dead on the first restart. Tie this RFC to issue #35 (`/home/agent/` volume mount) — every desktop-control workspace MUST have `/home/agent/.config/google-chrome/` and downloads dir on a persistent mount. Add explicit acceptance: "restart preserves browser cookies + downloads dir." #### 2. Sidecar resilience model unspecified What happens when the sidecar crashes mid-task? Spec lists tool surface but no supervision model. Add: - Sidecar runs under systemd (or docker `restart: always`) - Sidecar health endpoint that the runtime checks before sending tools - Agent gets a clear `sidecar_unavailable` error (not a hang) when sidecar is down - Audit log retention if sidecar restarts (don't lose history) #### 3. Screenshot cost / cadence not discussed Every `desktop.screenshot` is a base64 PNG flowing to the model. A 1920x1080 PNG is ~500KB-2MB. At one screenshot per tool call, a 30-step workflow consumes ~30-60MB of model input. Cost-control surface: - Quality knob: `screenshot.quality: low|medium|high` to trade off model latency vs reasoning fidelity - Delta screenshots: track previous + send only changed regions (cheaper on second+ turns) - Document the per-turn token + $/run envelope in the RFC #### 4. DCV licensing + AMI prebake Amazon DCV is free on EC2 but the Ubuntu DCV server install is non-trivial. The RFC should call out: - Pre-bake DCV into the AMI (don't `apt install dcv-server` at boot — slow + flaky) - Confirm DCV license terms support multi-tenant use case (workspace-per-customer) - TODO: validate with AWS legal/sales #### 5. Captcha / bot-detection wall (set expectations) Real browsers operated by agents WILL trigger captcha / bot-detection on many sites (Cloudflare Bot Management, Google reCAPTCHA, etc.). Not a blocker for Phase 1, but the RFC should set expectations: - Document: "agents will hit captcha walls; expect human-in-the-loop for some workflows" - Add as non-goal: "bypassing bot detection is out of scope" #### 6. Control-lock semantics on conflict "Request takeover" by user while agent is mid-action — what happens? Specify: - Agent yields at next decision point (between tool calls), gracefully releases lock - Force-takeover only after a 5-second grace timeout (configurable) - Active tool call is allowed to complete; queued tool calls are cancelled #### 7. Tool namespace alignment `desktop.screenshot` / `desktop.click` — make the namespace explicit: - MCP tool group `desktop_control_*` exposed by the sidecar via the existing MCP transport - Adapters subscribe to this group same way they subscribe to `a2a_*` / molecule tools - This way claude-code / codex / hermes all get the same tool surface for free #### 8. Resource leak on user-close-of-browser If user takes over and closes Chrome, agent's next screenshot returns "no window." Spec needs: - Sidecar supervises Chrome (auto-respawn if killed) OR - Explicit `desktop.launch_browser` tool the agent calls if it sees no browser open #### 9. Compute defaults — m6i.xlarge over t3.xlarge The addendum asks: m6i.xlarge vs t3.xlarge for Phase 1 desktop profile. Recommendation: **m6i.xlarge ($0.192/hr) over t3.xlarge ($0.166/hr)**. t3's burstable CPU credits will exhaust under sustained agent activity (browsing + screenshot encoding is CPU-heavy). The ~15% cost premium prevents support tickets. #### 10. Failure-mode states RFC has `provisioning | starting | ready | offline`. Add one: `ready-but-display-server-down` — the EC2 is healthy and the agent is processing but DCV server crashed. Different operator action than `offline`. #### 11. Browser-update cadence Phase 1 ships Chrome from the AMI bake. Chrome auto-updates by default → behavior may drift. Spec should commit to one of: - **Pin Chrome version in AMI; deliberate bump via PR** (recommended) - Allow Chrome auto-update; accept the drift risk ### Answers to the four review questions 1. **m6i.xlarge** over t3.xlarge for Phase 1 (see #9 above) 2. **Keep display orthogonal to tier** (agree with addendum) BUT require T2+ minimum (T1 sandboxed shouldn't get a desktop) 3. **Separate window for Phase 1**, embed in Phase 2 — agree 4. **Yes, require human approval for publish/reply/DM** in social workflows, with before/after screenshots in the approval prompt — agree, right default ### Suggested acceptance additions for Phase 1 E2E The current Phase 1 E2E test in the addendum: "provision display workspace, screenshot non-empty, click/type changes screen, Canvas can open display." Add: - Takeover lock works (agent + user don't fight) - **Restart preserves browser session** (cookie persistence via /home/agent/ volume mount per #1) - Sidecar auto-restarts on crash (per #2) - DCV access is platform-authenticated; raw DCV port not reachable from internet (per current security section) --- Reviewer: hongming (CEO Assistant, post-codex-investigation). Happy to discuss any of these on a call before implementation kicks off.
hongming commented 2026-05-23 00:26:38 +00:00
Author
Owner
Copy Link
Copy Source

Review notes:

Required: The RFC's Phase 1 contract is stale/incomplete. molecule-controlplane already accepts flat instance_type and disk_gb on /cp/workspaces/provision, validates instance type, and maps that to ProvisionWorkspace; but molecule-core's CreateWorkspacePayload and cpProvisionRequest do not expose or forward either field. A new nested compute: block would currently be dropped unless both sides are updated. Relevant code: workspace-server/internal/models/workspace.go, workspace-server/internal/provisioner/cp_provisioner.go, and control-plane internal/handlers/workspace_provision.go.

Required: Update the current-state/defaults section. The live control-plane code no longer looks like “single hardcoded t3-class / likely 64GB”; it has a documented default of t3.large / 50GB, with user override support already implemented in the control-plane resolver. Also, tier is intentionally access level only, not sizing.

Required: GPU/custom AMI needs a stricter security and billing gate than the RFC currently states. Existing CPU sizing is allowlisted and disk-clamped before RunInstances; GPU families, custom AMIs, and alternate OS images need the same explicit allowlist, tenant entitlement, AWS quota check, and cost authorization before launch. Otherwise the create path becomes a spend-escalation and supply-chain boundary.

Optional: Reframe Phase 1 as “finish end-to-end plumbing for existing CPU sizing”: add core API fields, validation, persistence if needed for restart/recreate, Canvas controls, and tests proving POST /workspaces forwards instance_type/disk_gb to CP. Save GPU, OS variants, AMI registry, and spot semantics for later RFC slices.

Review notes: **Required:** The RFC's Phase 1 contract is stale/incomplete. `molecule-controlplane` already accepts flat `instance_type` and `disk_gb` on `/cp/workspaces/provision`, validates instance type, and maps that to `ProvisionWorkspace`; but `molecule-core`'s `CreateWorkspacePayload` and `cpProvisionRequest` do not expose or forward either field. A new nested `compute:` block would currently be dropped unless both sides are updated. Relevant code: `workspace-server/internal/models/workspace.go`, `workspace-server/internal/provisioner/cp_provisioner.go`, and control-plane `internal/handlers/workspace_provision.go`. **Required:** Update the current-state/defaults section. The live control-plane code no longer looks like “single hardcoded t3-class / likely 64GB”; it has a documented default of `t3.large / 50GB`, with user override support already implemented in the control-plane resolver. Also, `tier` is intentionally access level only, not sizing. **Required:** GPU/custom AMI needs a stricter security and billing gate than the RFC currently states. Existing CPU sizing is allowlisted and disk-clamped before `RunInstances`; GPU families, custom AMIs, and alternate OS images need the same explicit allowlist, tenant entitlement, AWS quota check, and cost authorization before launch. Otherwise the create path becomes a spend-escalation and supply-chain boundary. **Optional:** Reframe Phase 1 as “finish end-to-end plumbing for existing CPU sizing”: add core API fields, validation, persistence if needed for restart/recreate, Canvas controls, and tests proving `POST /workspaces` forwards `instance_type/disk_gb` to CP. Save GPU, OS variants, AMI registry, and spot semantics for later RFC slices.
hongming commented 2026-05-23 00:30:39 +00:00
Author
Owner
Copy Link
Copy Source

RFC reconciliation: proposed implementation split after review

Based on the review comments above, I think we should reframe #1686 into a narrower implementable Phase 1 plus explicit follow-up slices.

Decision 1: Phase 1 is CPU sizing plumbing, not new EC2 sizing invention

molecule-controlplane already has partial CPU sizing support:

  • /cp/workspaces/provision accepts flat instance_type and disk_gb
  • CP validates instance_type against an allowlist
  • CP clamps disk in the resolver
  • CP defaults are currently t3.large / 50GB
  • tier is intentionally access level only, not sizing

The gap is that molecule-core and Canvas do not expose, persist, or forward these fields. So Phase 1 should be:

Finish end-to-end CPU sizing plumbing from Canvas/core to existing CP support.

Decision 2: Keep product API nested, translate to CP flat fields

Canvas/core should use a product-facing nested shape:

compute:
  instance_type: m6i.xlarge
  volume:
    root_gb: 100
  display:
    mode: none | desktop-control
    width: 1920
    height: 1080
    protocol: dcv

Core translates the Phase 1 sizing subset to CP's current flat request:

compute.instance_type     -> instance_type
compute.volume.root_gb    -> disk_gb

Display fields should be persisted now, but only the unavailable/product-surface pieces should ship in Phase 1 unless we explicitly choose to include DCV infra.

Decision 3: Split Phase 1 into two tracks

Track A — CPU sizing plumbing

Scope:

  1. Add workspaces.compute JSONB NOT NULL DEFAULT '{}' in molecule-core.
  2. Add CreateWorkspacePayload.Compute with:
    • instance_type
    • volume.root_gb
    • display.mode/width/height/protocol for future display persistence
  3. Validate CPU sizing in core before provisioning:
    • instance type allowlist should match CP initially
    • disk bounds should match CP; recommend rejecting out-of-range in core for user clarity rather than silently relying on CP clamp
  4. Persist compute at create time.
  5. Include compute in workspace read/list responses.
  6. Load stored compute on restart/resume/recreate so sizing does not silently revert to defaults.
  7. Extend provisioner.WorkspaceConfig and cpProvisionRequest to carry sizing.
  8. Forward instance_type and disk_gb to CP.
  9. Canvas Create flow adds compute controls:
    • instance type dropdown, default Platform default
    • root volume GB input, default empty/platform default
  10. Tests prove old behavior is unchanged when compute is omitted.

Acceptance:

  • POST /workspaces without compute behaves exactly as today.
  • Valid compute.instance_type and compute.volume.root_gb are persisted.
  • Valid sizing is forwarded to CP as instance_type/disk_gb.
  • Invalid instance type is rejected before CP call.
  • Invalid disk size is rejected before CP call, or explicitly normalized with visible response metadata if we choose clamp semantics.
  • Restart/resume preserve stored compute.
  • Tier remains orthogonal to sizing.

Track B — Canvas display/product surface groundwork

Scope:

  1. Add Display tab to workspace detail.
  2. If compute.display.mode is absent or none, show unavailable state:
    • Display is not enabled for this workspace.
    • do not try to open/provision display session
    • future CTA can be Recreate with display
  3. Add backend endpoint shape:
GET /workspaces/:id/display

Non-display response:

{
  "available": false,
  "reason": "display_not_enabled"
}
  1. Add Container Config tab skeleton for runtime/container settings, separate from EC2 shape:
    • runtime image/version, read-only at first
    • workspace access mode
    • max concurrent tasks
    • restart/reset-session controls
    • mounted path/access status
    • container privilege/status flags, inspect-only initially

Acceptance:

  • Normal non-display workspace shows Display tab unavailable state.
  • Canvas only calls display session endpoint when Display tab is opened.
  • GET /workspaces/:id/display returns available:false for non-display workspaces.
  • Container Config tab does not become the EC2 shape editor; compute shape belongs to create/recreate flow.

Phase 2: Desktop-control infra

Move actual desktop-control infrastructure to a follow-up slice unless we explicitly decide to expand Phase 1.

Scope:

  1. Desktop AMI or boot profile:
    • Ubuntu Desktop or XFCE/Xorg
    • Chrome/Firefox as normal apps
    • Amazon DCV server prebaked, not installed during workspace boot
    • Molecule runtime container
    • desktop-control sidecar
  2. Sidecar tool surface:
desktop.screenshot
desktop.get_screen_size
desktop.click
desktop.double_click
desktop.move_mouse
desktop.drag
desktop.type_text
desktop.press_key
desktop.wait
  1. Sidecar resilience:
    • systemd or equivalent supervision
    • health endpoint
    • clear sidecar_unavailable tool error
    • audit log survives sidecar restart
  2. Persistent browser/user state:
    • browser profile and downloads must live on persistent storage
    • acceptance: restart preserves browser cookies and downloads
  3. Canvas DCV session:
    • platform-authenticated/proxied access only
    • no raw DCV/VNC/RDP public port
    • start with separate authenticated display window; embed later
  4. Control lock:
{
  "controller": "agent" | "user" | "none",
  "controlled_by": "<user-or-workspace-id>",
  "expires_at": "..."
}
  1. Conflict semantics:
    • user takeover asks agent to yield at next decision point
    • force takeover after short grace timeout
    • active tool call may complete; queued calls are cancelled

Recommended default for CPU desktop profile:

  • m6i.xlarge / 100GB
  • 1920x1080
  • display orthogonal to tier, but consider T2+ minimum if product wants a low-trust sandbox boundary

Explicit non-goal:

  • bypassing captcha/bot-detection or platform anti-abuse systems is out of scope; expect human-in-the-loop on captcha walls and publishing-sensitive flows

Phase 3: GPU / OS / custom AMI / spot

Keep these out of Phase 1 and Phase 2 unless separately approved:

  • gpu-desktop-control
  • GPU instance allowlist and AWS quota checks
  • NVIDIA/DCV GPU AMI
  • tenant entitlement and cost authorization before launch
  • OS variants
  • custom AMI registry and supply-chain validation
  • spot/checkpoint semantics

These need stricter gates than CPU sizing because they change cost, quota, and trust boundaries.

Suggested child issues

  1. Core: add workspaces.compute, Create payload, validation, persistence, read responses.
  2. Core: forward CPU sizing to CP and preserve compute on restart/resume.
  3. Canvas: create-flow CPU sizing controls.
  4. Canvas: Display tab unavailable state + lazy display endpoint call.
  5. Core: GET /workspaces/:id/display returns available:false for non-display workspaces.
  6. Canvas: Container Config tab skeleton.
  7. CP/design: desktop-control AMI/DCV plan.
  8. Runtime/infra: desktop-control sidecar MVP.
  9. Runtime: desktop tools + screenshot-grounded action loop.
  10. Canvas/Core/CP: DCV session proxy + control lock.
  11. Infra: browser profile/download persistence for desktop-control workspaces.
  12. Follow-up RFC: GPU desktop-control entitlement/quota/cost model.

My recommended immediate next PR

Start with Track A only:

  • add workspaces.compute
  • add core payload/validation/persistence/readback
  • forward instance_type/disk_gb to CP
  • tests around omitted/valid/invalid compute and restart preservation

This unlocks real value immediately because CP already knows how to apply CPU sizing. Then Canvas Display/Container tabs can land in parallel without blocking the sizing path.

## RFC reconciliation: proposed implementation split after review Based on the review comments above, I think we should reframe #1686 into a narrower implementable Phase 1 plus explicit follow-up slices. ### Decision 1: Phase 1 is CPU sizing plumbing, not new EC2 sizing invention `molecule-controlplane` already has partial CPU sizing support: - `/cp/workspaces/provision` accepts flat `instance_type` and `disk_gb` - CP validates `instance_type` against an allowlist - CP clamps disk in the resolver - CP defaults are currently `t3.large / 50GB` - `tier` is intentionally access level only, not sizing The gap is that `molecule-core` and Canvas do not expose, persist, or forward these fields. So Phase 1 should be: > Finish end-to-end CPU sizing plumbing from Canvas/core to existing CP support. ### Decision 2: Keep product API nested, translate to CP flat fields Canvas/core should use a product-facing nested shape: ```yaml compute: instance_type: m6i.xlarge volume: root_gb: 100 display: mode: none | desktop-control width: 1920 height: 1080 protocol: dcv ``` Core translates the Phase 1 sizing subset to CP's current flat request: ```text compute.instance_type -> instance_type compute.volume.root_gb -> disk_gb ``` Display fields should be persisted now, but only the unavailable/product-surface pieces should ship in Phase 1 unless we explicitly choose to include DCV infra. ### Decision 3: Split Phase 1 into two tracks #### Track A — CPU sizing plumbing Scope: 1. Add `workspaces.compute JSONB NOT NULL DEFAULT '{}'` in `molecule-core`. 2. Add `CreateWorkspacePayload.Compute` with: - `instance_type` - `volume.root_gb` - `display.mode/width/height/protocol` for future display persistence 3. Validate CPU sizing in core before provisioning: - instance type allowlist should match CP initially - disk bounds should match CP; recommend rejecting out-of-range in core for user clarity rather than silently relying on CP clamp 4. Persist compute at create time. 5. Include compute in workspace read/list responses. 6. Load stored compute on restart/resume/recreate so sizing does not silently revert to defaults. 7. Extend `provisioner.WorkspaceConfig` and `cpProvisionRequest` to carry sizing. 8. Forward `instance_type` and `disk_gb` to CP. 9. Canvas Create flow adds compute controls: - instance type dropdown, default `Platform default` - root volume GB input, default empty/platform default 10. Tests prove old behavior is unchanged when `compute` is omitted. Acceptance: - `POST /workspaces` without `compute` behaves exactly as today. - Valid `compute.instance_type` and `compute.volume.root_gb` are persisted. - Valid sizing is forwarded to CP as `instance_type`/`disk_gb`. - Invalid instance type is rejected before CP call. - Invalid disk size is rejected before CP call, or explicitly normalized with visible response metadata if we choose clamp semantics. - Restart/resume preserve stored compute. - Tier remains orthogonal to sizing. #### Track B — Canvas display/product surface groundwork Scope: 1. Add Display tab to workspace detail. 2. If `compute.display.mode` is absent or `none`, show unavailable state: - `Display is not enabled for this workspace.` - do not try to open/provision display session - future CTA can be `Recreate with display` 3. Add backend endpoint shape: ```http GET /workspaces/:id/display ``` Non-display response: ```json { "available": false, "reason": "display_not_enabled" } ``` 4. Add Container Config tab skeleton for runtime/container settings, separate from EC2 shape: - runtime image/version, read-only at first - workspace access mode - max concurrent tasks - restart/reset-session controls - mounted path/access status - container privilege/status flags, inspect-only initially Acceptance: - Normal non-display workspace shows Display tab unavailable state. - Canvas only calls display session endpoint when Display tab is opened. - `GET /workspaces/:id/display` returns `available:false` for non-display workspaces. - Container Config tab does not become the EC2 shape editor; compute shape belongs to create/recreate flow. ### Phase 2: Desktop-control infra Move actual desktop-control infrastructure to a follow-up slice unless we explicitly decide to expand Phase 1. Scope: 1. Desktop AMI or boot profile: - Ubuntu Desktop or XFCE/Xorg - Chrome/Firefox as normal apps - Amazon DCV server prebaked, not installed during workspace boot - Molecule runtime container - desktop-control sidecar 2. Sidecar tool surface: ```text desktop.screenshot desktop.get_screen_size desktop.click desktop.double_click desktop.move_mouse desktop.drag desktop.type_text desktop.press_key desktop.wait ``` 3. Sidecar resilience: - systemd or equivalent supervision - health endpoint - clear `sidecar_unavailable` tool error - audit log survives sidecar restart 4. Persistent browser/user state: - browser profile and downloads must live on persistent storage - acceptance: restart preserves browser cookies and downloads 5. Canvas DCV session: - platform-authenticated/proxied access only - no raw DCV/VNC/RDP public port - start with separate authenticated display window; embed later 6. Control lock: ```json { "controller": "agent" | "user" | "none", "controlled_by": "<user-or-workspace-id>", "expires_at": "..." } ``` 7. Conflict semantics: - user takeover asks agent to yield at next decision point - force takeover after short grace timeout - active tool call may complete; queued calls are cancelled Recommended default for CPU desktop profile: - `m6i.xlarge / 100GB` - `1920x1080` - display orthogonal to tier, but consider T2+ minimum if product wants a low-trust sandbox boundary Explicit non-goal: - bypassing captcha/bot-detection or platform anti-abuse systems is out of scope; expect human-in-the-loop on captcha walls and publishing-sensitive flows ### Phase 3: GPU / OS / custom AMI / spot Keep these out of Phase 1 and Phase 2 unless separately approved: - `gpu-desktop-control` - GPU instance allowlist and AWS quota checks - NVIDIA/DCV GPU AMI - tenant entitlement and cost authorization before launch - OS variants - custom AMI registry and supply-chain validation - spot/checkpoint semantics These need stricter gates than CPU sizing because they change cost, quota, and trust boundaries. ### Suggested child issues 1. Core: add `workspaces.compute`, Create payload, validation, persistence, read responses. 2. Core: forward CPU sizing to CP and preserve compute on restart/resume. 3. Canvas: create-flow CPU sizing controls. 4. Canvas: Display tab unavailable state + lazy display endpoint call. 5. Core: `GET /workspaces/:id/display` returns `available:false` for non-display workspaces. 6. Canvas: Container Config tab skeleton. 7. CP/design: desktop-control AMI/DCV plan. 8. Runtime/infra: desktop-control sidecar MVP. 9. Runtime: desktop tools + screenshot-grounded action loop. 10. Canvas/Core/CP: DCV session proxy + control lock. 11. Infra: browser profile/download persistence for desktop-control workspaces. 12. Follow-up RFC: GPU desktop-control entitlement/quota/cost model. ### My recommended immediate next PR Start with Track A only: - add `workspaces.compute` - add core payload/validation/persistence/readback - forward `instance_type/disk_gb` to CP - tests around omitted/valid/invalid compute and restart preservation This unlocks real value immediately because CP already knows how to apply CPU sizing. Then Canvas Display/Container tabs can land in parallel without blocking the sizing path.
core-devops self-assigned this 2026-05-23 00:32:46 +00:00
core-devops commented 2026-05-23 00:32:46 +00:00
Member
Copy Link
Copy Source

claiming as core-devops — starting Phase 1 Track A implementation: core compute JSONB/payload validation/persistence/readback and CP sizing forward path, with tests first per dev SOP. core-be persona token appears invalid (401), so using core-devops for issue ownership.

claiming as core-devops — starting Phase 1 Track A implementation: core compute JSONB/payload validation/persistence/readback and CP sizing forward path, with tests first per dev SOP. core-be persona token appears invalid (401), so using core-devops for issue ownership.
hongming commented 2026-05-23 05:42:50 +00:00
Author
Owner
Copy Link
Copy Source

Opened follow-up PR #1701 for #1686 Track B groundwork: admin-gated GET /workspaces/:id/display and Canvas Display tab unavailable state for non-display workspaces. Verification is in the PR body; Stage A live Docker/Postgres remains pending on an isolated host.

Opened follow-up PR #1701 for #1686 Track B groundwork: admin-gated GET /workspaces/:id/display and Canvas Display tab unavailable state for non-display workspaces. Verification is in the PR body; Stage A live Docker/Postgres remains pending on an isolated host.
hongming commented 2026-05-23 06:16:11 +00:00
Author
Owner
Copy Link
Copy Source

Track B update: PR #1701 merged to main at a44f98e177. It adds the Display tab unavailable state plus admin-gated GET /workspaces/:id/display, and includes the review-requested route-level auth regression test.

Track B update: PR #1701 merged to main at a44f98e177fa8420715801f35dbd799cd77840f6. It adds the Display tab unavailable state plus admin-gated GET /workspaces/:id/display, and includes the review-requested route-level auth regression test.
hongming commented 2026-05-23 06:33:29 +00:00
Author
Owner
Copy Link
Copy Source

Opened PR #1705 for #1686 Track B Container Config tab skeleton: read-only Canvas surface for runtime/container settings, separate from EC2 compute/display shape editing. Verification is in the PR body.

Opened PR #1705 for #1686 Track B Container Config tab skeleton: read-only Canvas surface for runtime/container settings, separate from EC2 compute/display shape editing. Verification is in the PR body.
hongming commented 2026-05-23 06:50:27 +00:00
Author
Owner
Copy Link
Copy Source

Track B update: PR #1705 merged to main at 5cc570a18f. It adds the Canvas Container Config tab skeleton as a read-only runtime/container surface separate from EC2 compute/display shape editing. With #1701, Track B groundwork is now complete.

Track B update: PR #1705 merged to main at 5cc570a18fe3045291ab9ed45774b3dc9710e74d. It adds the Canvas Container Config tab skeleton as a read-only runtime/container surface separate from EC2 compute/display shape editing. With #1701, Track B groundwork is now complete.
hongming commented 2026-05-23 06:58:32 +00:00
Author
Owner
Copy Link
Copy Source

Opened PR #1711 for #1686 next slice: hardens the display-configured status contract for GET /workspaces/:id/display before live DCV/session infra. Backend and Canvas subagents confirmed this is the narrow backend-only bridge for Phase 2.

Opened PR #1711 for #1686 next slice: hardens the display-configured status contract for GET /workspaces/:id/display before live DCV/session infra. Backend and Canvas subagents confirmed this is the narrow backend-only bridge for Phase 2.
hongming commented 2026-05-23 07:25:58 +00:00
Author
Owner
Copy Link
Copy Source

Phase 2 bridge update: PR #1711 merged to main at 221b93faec. It hardens GET /workspaces/:id/display for display-configured workspaces: explicit unavailable status contract, DTO-based response, no URL/session exposure, and validation of stored display config without failing on unrelated CPU sizing drift.

Phase 2 bridge update: PR #1711 merged to main at 221b93faec15679e6698b37ffb21b68204b96f27. It hardens GET /workspaces/:id/display for display-configured workspaces: explicit unavailable status contract, DTO-based response, no URL/session exposure, and validation of stored display config without failing on unrelated CPU sizing drift.
hongming commented 2026-05-23 08:08:24 +00:00
Author
Owner
Copy Link
Copy Source

Merged backend display control-lock MVP in molecule-core PR #1718. Merge commit: 665f0a2405. Scope: admin-gated observe/acquire/release endpoints, volatile TTL lock table, display-enabled validation, admin/org-token mutation only, force release admin-token only, and structure_events audit rows. Local proof: go test ./internal/handlers ./internal/router ./internal/db; go test ./...; go build ./cmd/server; Stage A temp Postgres/Redis endpoint probe verified acquire/release DB state and audit rows.

Merged backend display control-lock MVP in molecule-core PR #1718. Merge commit: 665f0a2405967288b9d1ca18ea772ea43513137c. Scope: admin-gated observe/acquire/release endpoints, volatile TTL lock table, display-enabled validation, admin/org-token mutation only, force release admin-token only, and structure_events audit rows. Local proof: go test ./internal/handlers ./internal/router ./internal/db; go test ./...; go build ./cmd/server; Stage A temp Postgres/Redis endpoint probe verified acquire/release DB state and audit rows.
hongming commented 2026-05-23 09:42:22 +00:00
Author
Owner
Copy Link
Copy Source

#1686 update: Canvas Display tab control-state slice merged.

PR #1726 merged normally at 3161d43cec098ff06539dba890bc5b824fd0fb78 and is present in current main history. The live origin/main DisplayTab includes:

  • display-configured workspaces fetch /display/control;
  • non-display workspaces skip control fetch;
  • controller: none renders Take control;
  • active locks are observe-only until backend exposes ownership/can-release;
  • failed acquire refetches current lock state;
  • stale acquire responses are ignored across workspace switches;
  • raw backend error/actor strings are not displayed.

Verification completed:

  • Focused DisplayTab tests: 8 passed.
  • Full Canvas Vitest: 221 files passed, 3386 passed, 1 skipped.
  • Canvas production build passed.
  • PR CI passed: Canvas (Next.js), CI all-required, E2E path-filtered checks, handlers integration path-filtered checks, secret scan, gate-check, QA, security, SOP checklist, tier check.
  • Canvas image publish passed; ECR has molecule-ai/canvas:sha-3161d43 and latest at digest sha256:2b1306253ab69fcc58ec6fda6888ae22bf7907cb8dd789f957fc255f84ad1df2.

Post-merge note: rapid subsequent merges cancelled some push runs for intermediate main SHAs, but #1726’s PR CI was green before merge and the Canvas image publish for the merge SHA succeeded. Live Canvas buildinfo route verification is currently blocked because canvas.moleculesai.app and canvas-staging.moleculesai.app do not resolve from this environment; no app-level response is available to smoke until DNS/deploy routing exists.

#1686 update: Canvas Display tab control-state slice merged. PR #1726 merged normally at `3161d43cec098ff06539dba890bc5b824fd0fb78` and is present in current `main` history. The live `origin/main` DisplayTab includes: - display-configured workspaces fetch `/display/control`; - non-display workspaces skip control fetch; - `controller: none` renders `Take control`; - active locks are observe-only until backend exposes ownership/can-release; - failed acquire refetches current lock state; - stale acquire responses are ignored across workspace switches; - raw backend error/actor strings are not displayed. Verification completed: - Focused DisplayTab tests: 8 passed. - Full Canvas Vitest: 221 files passed, 3386 passed, 1 skipped. - Canvas production build passed. - PR CI passed: Canvas (Next.js), CI all-required, E2E path-filtered checks, handlers integration path-filtered checks, secret scan, gate-check, QA, security, SOP checklist, tier check. - Canvas image publish passed; ECR has `molecule-ai/canvas:sha-3161d43` and `latest` at digest `sha256:2b1306253ab69fcc58ec6fda6888ae22bf7907cb8dd789f957fc255f84ad1df2`. Post-merge note: rapid subsequent merges cancelled some push runs for intermediate main SHAs, but #1726’s PR CI was green before merge and the Canvas image publish for the merge SHA succeeded. Live Canvas buildinfo route verification is currently blocked because `canvas.moleculesai.app` and `canvas-staging.moleculesai.app` do not resolve from this environment; no app-level response is available to smoke until DNS/deploy routing exists.
hongming commented 2026-05-23 19:15:17 +00:00
Author
Owner
Copy Link
Copy Source

Follow-up live deploy check for #1726 / #1686:

  • canvas.moleculesai.app and canvas-staging.moleculesai.app still have no DNS records from this environment; those are not the live Canvas surfaces today.
  • The live production tenant surface is the combined tenant image: Go platform on /buildinfo plus proxied Canvas UI.
  • https://reno-stars.moleculesai.app/buildinfo now returns e05fc4daaedc92a9cd86c367113431504e0f1d1c.
  • 3161d43c (#1726 merge commit) is an ancestor of current origin/main at e05fc4d, so the deployed tenant build contains the DisplayTab changes.
  • The served production UI bundle at reno-stars.moleculesai.app includes the #1726 DisplayTab code markers: display/control, Take control, Failed to take control, No active controller, Controlled by, and Automation.

Remaining for a true click-through smoke: use an authenticated tenant browser session and a workspace with display.enabled=true to open the Display tab and exercise the visible states. No code/deploy blocker found for the #1726 control-state slice.

Follow-up live deploy check for #1726 / #1686: - `canvas.moleculesai.app` and `canvas-staging.moleculesai.app` still have no DNS records from this environment; those are not the live Canvas surfaces today. - The live production tenant surface is the combined tenant image: Go platform on `/buildinfo` plus proxied Canvas UI. - `https://reno-stars.moleculesai.app/buildinfo` now returns `e05fc4daaedc92a9cd86c367113431504e0f1d1c`. - `3161d43c` (#1726 merge commit) is an ancestor of current `origin/main` at `e05fc4d`, so the deployed tenant build contains the DisplayTab changes. - The served production UI bundle at `reno-stars.moleculesai.app` includes the #1726 DisplayTab code markers: `display/control`, `Take control`, `Failed to take control`, `No active controller`, `Controlled by`, and `Automation`. Remaining for a true click-through smoke: use an authenticated tenant browser session and a workspace with `display.enabled=true` to open the Display tab and exercise the visible states. No code/deploy blocker found for the #1726 control-state slice.
hongming commented 2026-05-23 19:49:34 +00:00
Author
Owner
Copy Link
Copy Source

Update for #1686 target display flow:\n\nOpened coordinated PRs:\n- molecule-core #1732: #1732\n- molecule-controlplane #258: molecule-ai/molecule-controlplane#258\n\nImplemented flow coverage:\n1. New Workspace now has Container Config -> Display enablement.\n2. Defaults are t3.xlarge, 80GB, 1920x1080.\n3. Create payload persists compute.display and sends it through to CP.\n4. Workspace EC2 bootstrap installs a native display stack using Xvfb/XFCE/x11vnc/noVNC.\n5. Display tab can embed the desktop viewer when the display endpoint returns an available session.\n6. Take control / Release uses the existing display control lock endpoints.\n\nVerification run locally:\n- Canvas Display/Create tests: 39 passed.\n- Canvas production build: passed.\n- molecule-core focused Go display/provision tests: passed.\n- molecule-controlplane internal/handlers: passed.\n- molecule-controlplane internal/provisioner: passed.\n\nNote: this first implementation uses noVNC for the browser stream rather than DCV, because it is apt-installable in user-data and avoids fragile DCV package-version bootstrapping. The control remains screen-level mouse/keyboard, not Playwright/Puppeteer browser control.

Update for #1686 target display flow:\n\nOpened coordinated PRs:\n- molecule-core #1732: https://git.moleculesai.app/molecule-ai/molecule-core/pulls/1732\n- molecule-controlplane #258: https://git.moleculesai.app/molecule-ai/molecule-controlplane/pulls/258\n\nImplemented flow coverage:\n1. New Workspace now has Container Config -> Display enablement.\n2. Defaults are t3.xlarge, 80GB, 1920x1080.\n3. Create payload persists compute.display and sends it through to CP.\n4. Workspace EC2 bootstrap installs a native display stack using Xvfb/XFCE/x11vnc/noVNC.\n5. Display tab can embed the desktop viewer when the display endpoint returns an available session.\n6. Take control / Release uses the existing display control lock endpoints.\n\nVerification run locally:\n- Canvas Display/Create tests: 39 passed.\n- Canvas production build: passed.\n- molecule-core focused Go display/provision tests: passed.\n- molecule-controlplane internal/handlers: passed.\n- molecule-controlplane internal/provisioner: passed.\n\nNote: this first implementation uses noVNC for the browser stream rather than DCV, because it is apt-installable in user-data and avoids fragile DCV package-version bootstrapping. The control remains screen-level mouse/keyboard, not Playwright/Puppeteer browser control.
hongming commented 2026-05-23 21:00:23 +00:00
Author
Owner
Copy Link
Copy Source

Implemented and merged the Target User Flow across core + controlplane.

Merged PRs:

  • molecule-core #1732 -> merge commit 2d1a853bf9
  • molecule-controlplane #258 -> merge commit 3e84aa43a4a2bf9aeaded308a3f4addf796566a9

What landed:

  • Canvas New Workspace now has Container Config with Display enable/profile controls.
  • Display profile sends desktop-control / noVNC / 1920x1080 / t3.xlarge / 80GB compute config.
  • Workspace Display tab renders authenticated viewer URL when configured and supports Take control / Release control.
  • workspace-server persists and forwards display config to CP, caps display dimensions, and only accepts HTTPS viewer base URLs.
  • controlplane provisions the display host stack with Xvfb + XFCE + x11vnc + noVNC/websockify.
  • noVNC port 6080 is private VPC-only, not public 0.0.0.0/0; VNC password is generated instead of derived from workspace id.

Verification:

  • Core focused Canvas tests passed.
  • Core focused Go tests passed.
  • Controlplane handlers/provisioner tests passed.
  • Live Gitea CI for controlplane PR: 3/3 success.
  • Live Gitea CI for core PR: required checks green; only expected advisory ARM cancellations and auxiliary N/A status remained.
  • QA/security persona approvals submitted before merge.
Implemented and merged the Target User Flow across core + controlplane. Merged PRs: - molecule-core #1732 -> merge commit 2d1a853bf9155a8b279158b8c82233c5e905b163 - molecule-controlplane #258 -> merge commit 3e84aa43a4a2bf9aeaded308a3f4addf796566a9 What landed: - Canvas New Workspace now has Container Config with Display enable/profile controls. - Display profile sends desktop-control / noVNC / 1920x1080 / t3.xlarge / 80GB compute config. - Workspace Display tab renders authenticated viewer URL when configured and supports Take control / Release control. - workspace-server persists and forwards display config to CP, caps display dimensions, and only accepts HTTPS viewer base URLs. - controlplane provisions the display host stack with Xvfb + XFCE + x11vnc + noVNC/websockify. - noVNC port 6080 is private VPC-only, not public 0.0.0.0/0; VNC password is generated instead of derived from workspace id. Verification: - Core focused Canvas tests passed. - Core focused Go tests passed. - Controlplane handlers/provisioner tests passed. - Live Gitea CI for controlplane PR: 3/3 success. - Live Gitea CI for core PR: required checks green; only expected advisory ARM cancellations and auxiliary N/A status remained. - QA/security persona approvals submitted before merge.
hongming commented 2026-05-23 23:50:14 +00:00
Author
Owner
Copy Link
Copy Source

Implementation update for #1686 Display takeover flow:

PRs ready for review/merge:

What changed:

  • Canvas now opens a trusted @novnc/novnc client in the Display tab after Take control.
  • workspace-server mints short-lived display session tokens and proxies only /websockify after validating the active display-control lock.
  • The display token is not sent in the request query string; it is carried from a URL fragment into a WebSocket subprotocol, then stripped before forwarding upstream.
  • Controlplane provisions display services localhost-only and does not expose 6080 in workspace security groups.
  • New tenants get DISPLAY_SESSION_SIGNING_SECRET in the bootstrap bundle.
  • Existing tenants are covered by lazy /cp/tenants/config backfill on restart/redeploy; if the secret cannot be produced, tenant config now returns 503 instead of silently booting without display session support.

Verification run locally:

  • molecule-core/workspace-server: go test ./cmd/server ./internal/handlers ./internal/router
  • molecule-core/canvas: npm test -- --run src/components/tabs/tests/DisplayTab.test.tsx src/components/tests/CreateWorkspaceDialog.test.tsx
  • molecule-core/canvas: npm run build -- --no-lint (passes with existing @novnc/novnc top-level-await warning)
  • molecule-controlplane: go test ./internal/provisioner ./internal/handlers ./internal/router ./cmd/server

Review/CI:

  • Core approved by core-qa and core-security.
  • Controlplane approved by infra-sre and devops-engineer (CP-specific persona tokens on the operator are stale/401).
  • Gitea required status contexts are currently still Waiting on runners, so I have not merged around branch protection.
Implementation update for #1686 Display takeover flow: PRs ready for review/merge: - molecule-core: https://git.moleculesai.app/molecule-ai/molecule-core/pulls/1752 - molecule-controlplane: https://git.moleculesai.app/molecule-ai/molecule-controlplane/pulls/263 What changed: - Canvas now opens a trusted @novnc/novnc client in the Display tab after Take control. - workspace-server mints short-lived display session tokens and proxies only /websockify after validating the active display-control lock. - The display token is not sent in the request query string; it is carried from a URL fragment into a WebSocket subprotocol, then stripped before forwarding upstream. - Controlplane provisions display services localhost-only and does not expose 6080 in workspace security groups. - New tenants get DISPLAY_SESSION_SIGNING_SECRET in the bootstrap bundle. - Existing tenants are covered by lazy /cp/tenants/config backfill on restart/redeploy; if the secret cannot be produced, tenant config now returns 503 instead of silently booting without display session support. Verification run locally: - molecule-core/workspace-server: go test ./cmd/server ./internal/handlers ./internal/router - molecule-core/canvas: npm test -- --run src/components/tabs/__tests__/DisplayTab.test.tsx src/components/__tests__/CreateWorkspaceDialog.test.tsx - molecule-core/canvas: npm run build -- --no-lint (passes with existing @novnc/novnc top-level-await warning) - molecule-controlplane: go test ./internal/provisioner ./internal/handlers ./internal/router ./cmd/server Review/CI: - Core approved by core-qa and core-security. - Controlplane approved by infra-sre and devops-engineer (CP-specific persona tokens on the operator are stale/401). - Gitea required status contexts are currently still Waiting on runners, so I have not merged around branch protection.
hongming commented 2026-05-24 02:46:21 +00:00
Author
Owner
Copy Link
Copy Source

Issue #1686 implementation is merged and live.

Merged PRs:

  • molecule-controlplane #263 -> merge commit cdf9733bb97c42ef51658eadbc1269e16a5dbe0b
  • molecule-core #1752 -> merge commit 43422e0ba9

Deployment verification:

  • Controlplane production Railway deployment is online; /health returns 200.
  • Core canvas image publish succeeded for 43422e0.
  • Core workspace-server/tenant images were built and pushed for 43422e0.
  • Production tenant fleet was redeployed with target_tag staging-43422e0; redeploy-fleet returned ok=true with SSM Success and healthz_ok=true for hongming, agents-team, chloe-dong, go-to-market, and reno-stars.
  • Live tenant /buildinfo now reports git_sha 43422e0ba9137a854dfadd1404e85bcf1939c062 on those tenants.

Post-merge CI note: the PR checks were green before merge. Some main-push post-merge workflows later show Failure/Skipped because runner queue saturation caused jobs to remain pending until deploy waiters timed out or queued jobs were cancelled before task assignment. I verified the red rows are cancelled/no-task rows rather than failing test commands. Gitea 1.22 has no rerun/dispatch API endpoint available here, so I did not create a no-op commit just to churn the queue further.

Issue #1686 implementation is merged and live. Merged PRs: - molecule-controlplane #263 -> merge commit cdf9733bb97c42ef51658eadbc1269e16a5dbe0b - molecule-core #1752 -> merge commit 43422e0ba9137a854dfadd1404e85bcf1939c062 Deployment verification: - Controlplane production Railway deployment is online; `/health` returns 200. - Core canvas image publish succeeded for 43422e0. - Core workspace-server/tenant images were built and pushed for 43422e0. - Production tenant fleet was redeployed with target_tag `staging-43422e0`; redeploy-fleet returned ok=true with SSM Success and healthz_ok=true for hongming, agents-team, chloe-dong, go-to-market, and reno-stars. - Live tenant `/buildinfo` now reports git_sha `43422e0ba9137a854dfadd1404e85bcf1939c062` on those tenants. Post-merge CI note: the PR checks were green before merge. Some main-push post-merge workflows later show Failure/Skipped because runner queue saturation caused jobs to remain pending until deploy waiters timed out or queued jobs were cancelled before task assignment. I verified the red rows are cancelled/no-task rows rather than failing test commands. Gitea 1.22 has no rerun/dispatch API endpoint available here, so I did not create a no-op commit just to churn the queue further.
hongming commented 2026-05-25 22:14:39 +00:00
Author
Owner
Copy Link
Copy Source

Filed molecule-controlplane#300 as the dedicated follow-up for preserving display browser profile/cookies/downloads across destructive EC2 recreate. This is related to the display/EC2 configurability work here, but separate from the compute settings themselves.

Filed molecule-controlplane#300 as the dedicated follow-up for preserving display browser profile/cookies/downloads across destructive EC2 recreate. This is related to the display/EC2 configurability work here, but separate from the compute settings themselves.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
area/ci
CI/CD pipeline issues
 do-not-auto-merge
Opt out of autonomous merge-queue merging
 kind/infrastructure
Infrastructure-related issues
 merge-queue
Ready for serialized Gitea merge queue
 merge-queue-hold
Temporarily hold PR in merge queue
 platform/go
Go platform test issues
 release-blocker
Blocks the staging→main promotion / a release
 release-test
security
test-label-sre
tier:high
High risk per dev-sop §SOP-6 — ceo only, 24h cooldown
 tier:low
Low risk per dev-sop §SOP-6 — engineers/managers/ceo can approve
 tier:medium
Medium risk per dev-sop §SOP-6 — managers/ceo can approve
 triage-test
test
 wip
Work in progress — do not auto-merge
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
agent-dev-a agent-dev-b agent-pm agent-researcher agent-reviewer agent-reviewer-1 agent-reviewer-cr2 app-fe (Molecule AI · app-fe) app-lead (Molecule AI · app-lead) app-qa (Molecule AI · app-qa) claude-ceo-assistant claude-ci-reader claude-status-reaper core-be (Molecule AI · core-be) core-devops (Molecule AI · core-devops) core-fe (Molecule AI · core-fe) core-lead (Molecule AI · core-lead) core-offsec (Molecule AI · core-offsec) core-qa (Molecule AI · core-qa) core-security (Molecule AI · core-security) core-uiux (Molecule AI · core-uiux) cp-be (Molecule AI · cp-be) cp-lead (Molecule AI · cp-lead) cp-qa (Molecule AI · cp-qa) cp-security (Molecule AI · cp-security) cui (Zhanlin Cui) dev-lead (Molecule AI · dev-lead) devops-engineer documentation-specialist (Molecule AI · documentation-specialist) fullstack-engineer (Molecule AI · fullstack-engineer) godwin hongming hongming-ceo-delegated hongming-codex-laptop hongming-kimi-laptop hongming-pc2 hongming-personal infra-lead (Molecule AI · infra-lead) infra-runtime-be (Molecule AI · infra-runtime-be) infra-sre (Molecule AI · infra-sre) integration-tester (Molecule AI · integration-tester) mc-drift-bot molecule-code-reviewer plugin-dev (Molecule AI · plugin-dev) pm publish-runtime-bot pypi-publisher (Molecule AI PyPI Publisher (RFC#596)) release-manager (Molecule AI · release-manager) sdk-dev (Molecule AI · sdk-dev) sdk-lead (Molecule AI · sdk-lead) sop-drift-bot sop-tier-bot (SOP Tier-Check Bot) technical-writer (Molecule AI · technical-writer) triage-operator (Molecule AI · triage-operator)
No Assignees core-devops
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: molecule-ai/molecule-core#1686
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?