feat: add desktop control tools #63

Merged

hongming merged 1 commits from feat/desktop-control-tools into main 2026-05-25 09:20:48 +00:00

Conversation 4 Commits 1 Files Changed 10

+511 -7

hongming commented 2026-05-25 09:06:14 +00:00

Owner

Copy Link

Copy Source

Adds native desktop-control MCP tools for display-enabled workspaces.\n\nChanges:\n- Adds desktop_status, desktop_screenshot, desktop_click, desktop_type, desktop_key, desktop_open_url.\n- Includes display tools in the runtime capabilities/system prompt helpers.\n- Bumps runtime to 0.3.0 for the additive desktop-control API.\n\nVerification:\n- PYTHONPATH=$PWD pytest -q -> 355 passed\n- Live path check: /workspace is shared into the host chroot, so screenshots written by host display tools are visible to the runtime container.

Adds native desktop-control MCP tools for display-enabled workspaces.\n\nChanges:\n- Adds desktop_status, desktop_screenshot, desktop_click, desktop_type, desktop_key, desktop_open_url.\n- Includes display tools in the runtime capabilities/system prompt helpers.\n- Bumps runtime to 0.3.0 for the additive desktop-control API.\n\nVerification:\n- PYTHONPATH=$PWD pytest -q -> 355 passed\n- Live path check: /workspace is shared into the host chroot, so screenshots written by host display tools are visible to the runtime container.

hongming added 1 commit 2026-05-25 09:06:15 +00:00

feat: add desktop control tools 8a1e51d087

app-fe force-pushed feat/desktop-control-tools from 8a1e51d087 to 14221793bc 2026-05-25 09:15:55 +00:00 Compare

infra-runtime-be approved these changes 2026-05-25 09:18:27 +00:00

infra-runtime-be left a comment

Member

Copy Link

Copy Source

Reviewed latest desktop-control runtime patch. Security blockers were addressed: no shell URL launch, desktop tools are gated by display.control, screenshots no longer write directly to workspace from host root, and browser launch drops to ubuntu. Local full suite: 364 passed.

Reviewed latest desktop-control runtime patch. Security blockers were addressed: no shell URL launch, desktop tools are gated by display.control, screenshots no longer write directly to workspace from host root, and browser launch drops to ubuntu. Local full suite: 364 passed.

infra-runtime-be approved these changes 2026-05-25 09:18:28 +00:00

infra-runtime-be left a comment

Member

Copy Link

Copy Source

Reviewed latest desktop-control runtime patch. Security blockers were addressed: no shell URL launch, desktop tools are gated by display.control, screenshots no longer write directly to workspace from host root, and browser launch drops to ubuntu. Local full suite: 364 passed.

Reviewed latest desktop-control runtime patch. Security blockers were addressed: no shell URL launch, desktop tools are gated by display.control, screenshots no longer write directly to workspace from host root, and browser launch drops to ubuntu. Local full suite: 364 passed.

dev-lead approved these changes 2026-05-25 09:19:24 +00:00

dev-lead left a comment

Member

Copy Link

Copy Source

Runtime API review approved. The patch is additive, tests cover registration/RBAC/security regressions, and local full suite passes.

Runtime API review approved. The patch is additive, tests cover registration/RBAC/security regressions, and local full suite passes.

dev-lead approved these changes 2026-05-25 09:19:24 +00:00

dev-lead left a comment

Member

Copy Link

Copy Source

Runtime API review approved. The patch is additive, tests cover registration/RBAC/security regressions, and local full suite passes.

Runtime API review approved. The patch is additive, tests cover registration/RBAC/security regressions, and local full suite passes.

hongming merged commit 71287efb6d into main 2026-05-25 09:20:48 +00:00

hongming referenced this issue from a commit 2026-05-25 09:20:50 +00:00

Merge pull request #63 from feat/desktop-control-tools

Sign in to join this conversation.

Reviewers

No Reviewers

infra-runtime-be

dev-lead

Labels

Clear labels

triaged

PM triaged

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

agent-dev-a agent-dev-b agent-pm agent-researcher agent-reviewer agent-reviewer-1 agent-reviewer-cr2 app-fe (Molecule AI · app-fe) app-lead (Molecule AI · app-lead) app-qa (Molecule AI · app-qa) claude-ceo-assistant claude-ci-reader core-be (Molecule AI · core-be) core-devops (Molecule AI · core-devops) core-fe (Molecule AI · core-fe) core-lead (Molecule AI · core-lead) core-offsec (Molecule AI · core-offsec) core-qa (Molecule AI · core-qa) core-security (Molecule AI · core-security) core-uiux (Molecule AI · core-uiux) cp-be (Molecule AI · cp-be) cp-lead (Molecule AI · cp-lead) cp-qa (Molecule AI · cp-qa) cp-security (Molecule AI · cp-security) cui (Zhanlin Cui) dev-lead (Molecule AI · dev-lead) devops-engineer documentation-specialist (Molecule AI · documentation-specialist) fullstack-engineer (Molecule AI · fullstack-engineer) godwin hongming hongming-ceo-delegated hongming-codex-laptop hongming-kimi-laptop hongming-pc2 hongming-personal infra-lead (Molecule AI · infra-lead) infra-runtime-be (Molecule AI · infra-runtime-be) infra-sre (Molecule AI · infra-sre) integration-tester (Molecule AI · integration-tester) molecule-code-reviewer molecule-runtime-release-bot (Molecule Runtime Release Bot) plugin-dev (Molecule AI · plugin-dev) pm publish-runtime-bot pypi-publisher (Molecule AI PyPI Publisher (RFC#596)) release-manager (Molecule AI · release-manager) sdk-dev (Molecule AI · sdk-dev) sdk-lead (Molecule AI · sdk-lead) sop-tier-bot (SOP Tier-Check Bot) technical-writer (Molecule AI · technical-writer) triage-operator (Molecule AI · triage-operator)

No Assignees

4 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: molecule-ai/molecule-ai-workspace-runtime#63