molecule-core/docs/workspace-runtime-package.md

Workspace Runtime PyPI Package

Overview

The shared workspace runtime infrastructure has one editable source and one published artifact:

1. Source of truth (monorepo, editable): workspace/ — every runtime change lands here. Edit it like any other monorepo code.
2. Published artifact (PyPI, generated): molecule-ai-workspace-runtime — produced by .github/workflows/publish-runtime.yml on every runtime-vX.Y.Z tag push. Do NOT edit this independently — it gets overwritten on every publish.

The legacy sibling repo molecule-ai-workspace-runtime (the GitHub repo, as distinct from the PyPI package) is no longer the source-of-truth and should be treated as a publish artifact only. It can be archived or used as a read-only mirror.

Why this shape

The 8 workspace template repos (claude-code, langgraph, hermes, etc.) each build their own Docker image and pip install molecule-ai-workspace-runtime from PyPI. PyPI is the right distribution channel — semver, reproducible builds, no submodule dance per-repo. But the runtime ALSO needs to evolve in lock-step with the platform's wire protocol (queue shape, A2A metadata, event payloads). Shipping cross-cutting protocol changes as separate runtime + platform PRs in two repos creates ordering pain and broken intermediate states.

The monorepo + auto-publish split gives both: edit cross-cutting changes in one PR, publish the runtime artifact via a tag.

What's in the package

Everything in workspace/*.py plus the adapters/, builtin_tools/, plugins_registry/, policies/, skill_loader/ subpackages. Build artifacts (Dockerfile, *.sh, pytest.ini, requirements.txt) are excluded.

The build script rewrites bare imports so the published package is a proper Python namespace:

# In monorepo workspace/:
from a2a_client import discover_peer
from builtin_tools.memory import store

# In published molecule_runtime/ (auto-rewritten at publish time):
from molecule_runtime.a2a_client import discover_peer
from molecule_runtime.builtin_tools.memory import store

The closed allowlist of rewritten module names lives in scripts/build_runtime_package.py (TOP_LEVEL_MODULES + SUBPACKAGES). Add a new top-level module to workspace/? Add it to the allowlist in the same PR.

Adapter repos

Each of the 8 adapter template repos contains:

• adapter.py — runtime-specific Adapter class
• requirements.txt — molecule-ai-workspace-runtime>=0.1.X + adapter deps
• Dockerfile — standalone image with ENV ADAPTER_MODULE=adapter and ENTRYPOINT ["molecule-runtime"]

Adapter	Repo
claude-code	https://github.com/Molecule-AI/molecule-ai-workspace-template-claude-code
langgraph	https://github.com/Molecule-AI/molecule-ai-workspace-template-langgraph
crewai	https://github.com/Molecule-AI/molecule-ai-workspace-template-crewai
autogen	https://github.com/Molecule-AI/molecule-ai-workspace-template-autogen
deepagents	https://github.com/Molecule-AI/molecule-ai-workspace-template-deepagents
hermes	https://github.com/Molecule-AI/molecule-ai-workspace-template-hermes
gemini-cli	https://github.com/Molecule-AI/molecule-ai-workspace-template-gemini-cli
openclaw	https://github.com/Molecule-AI/molecule-ai-workspace-template-openclaw

Adapter discovery (ADAPTER_MODULE)

Standalone adapter repos set ENV ADAPTER_MODULE=adapter in their Dockerfile. The runtime's get_adapter() checks this env var first:

# In molecule_runtime/adapters/__init__.py
def get_adapter(runtime: str) -> type[BaseAdapter]:
    adapter_module = os.environ.get("ADAPTER_MODULE")
    if adapter_module:
        mod = importlib.import_module(adapter_module)
        return getattr(mod, "Adapter")
    raise KeyError(...)

Publishing a new version

# From any local checkout of monorepo, after merging your runtime change:
git tag runtime-v0.1.6
git push origin runtime-v0.1.6

The publish-runtime workflow takes over — checks out the tag, runs scripts/build_runtime_package.py --version 0.1.6, builds wheel + sdist, runs a smoke import to catch broken rewrites, and uploads to PyPI via the PyPA Trusted Publisher action (OIDC). No static API token is stored in this repo — PyPI verifies the workflow's OIDC claim against the trusted-publisher config registered for molecule-ai-workspace-runtime.

For dev/test releases without tagging, dispatch the workflow manually with an explicit version (e.g. 0.1.6.dev1 — PEP 440 dev/rc/post forms are accepted).

After publish, the 8 template repos pick up the new version on their next :latest rebuild. To force-pull immediately, bump the pin in each template's requirements.txt.

End-to-end CD chain

The full chain from monorepo merge → workspace containers running new code:

1. Merge PR with workspace/ changes to main
   ↓
2. .github/workflows/auto-tag-runtime.yml fires
   ↓ reads PR labels (release:major/minor) or defaults to patch
   ↓ pushes runtime-vX.Y.Z tag
   ↓
3. .github/workflows/publish-runtime.yml fires (on the tag)
   ↓ builds wheel via scripts/build_runtime_package.py
   ↓ smoke-imports the wheel
   ↓ uploads to PyPI
   ↓ cascade job fires repository_dispatch (event-type: runtime-published)
   ↓ to all 8 workspace-template-* repos
   ↓
4. Each template's publish-image.yml fires (on repository_dispatch)
   ↓ rebuilds Dockerfile (which pip-installs the new PyPI version)
   ↓ pushes ghcr.io/molecule-ai/workspace-template-<runtime>:latest
   ↓
5. Production hosts run scripts/refresh-workspace-images.sh
   OR an operator hits POST /admin/workspace-images/refresh on the platform
   ↓ docker pull all 8 :latest tags
   ↓ remove + force-recreate any running ws-* containers using a refreshed image
   ↓ canvas re-provisions the workspaces on next interaction

Steps 1-4 are fully automated. Step 5 is one-click: a single curl or shell command. SaaS deployments typically wire step 5 into their normal deploy pipeline (every release pulls fresh images on every host); local dev fires it manually after a runtime release lands.

Auth

PyPI publishing uses Trusted Publisher (OIDC) — no static token in the monorepo. The trusted-publisher config on PyPI binds the molecule-ai-workspace-runtime project to this repo's publish-runtime.yml workflow + pypi-publish environment. Rotation is moot: there is no shared secret to rotate.

Required secrets

Secret	Where	Why
TEMPLATE_DISPATCH_TOKEN	molecule-core repo	Fine-grained PAT with actions:write on the 8 template repos. Without it the cascade job warns and exits clean — PyPI still publishes; templates just don't auto-rebuild.

Step 5 specifics

Local dev (compose stack):

bash scripts/refresh-workspace-images.sh                  # all runtimes
bash scripts/refresh-workspace-images.sh --runtime claude-code
bash scripts/refresh-workspace-images.sh --no-recreate    # pull only, leave containers

Via platform admin endpoint (any deploy):

curl -X POST "$PLATFORM/admin/workspace-images/refresh"
curl -X POST "$PLATFORM/admin/workspace-images/refresh?runtime=claude-code"
curl -X POST "$PLATFORM/admin/workspace-images/refresh?recreate=false"

The endpoint pulls + recreates from inside the platform container, so it needs Docker socket access (the compose stack mounts /var/run/docker.sock already) AND GHCR auth on the host's docker config (docker login ghcr.io once per host). On a fresh host without GHCR auth, the pull step warns per runtime and the response surfaces the failures.

Local dev (build the package without publishing)

python3 scripts/build_runtime_package.py --version 0.1.0-local --out /tmp/runtime-build
cd /tmp/runtime-build
python -m build              # produces dist/*.whl + dist/*.tar.gz
pip install dist/*.whl       # install into a venv to test locally

This is the same pipeline CI runs. Use it to validate import-rewrite correctness before pushing a runtime-v* tag.

Writing a new adapter

Use the GitHub template repo Molecule-AI/molecule-ai-workspace-template-starter — it ships with the canonical Dockerfile + adapter.py skeleton + config.yaml schema + the repository_dispatch: [runtime-published] cascade receiver already wired up. No follow-up setup PR required.

# Replace <runtime> with your runtime slug (lowercase, hyphenated).
gh repo create Molecule-AI/molecule-ai-workspace-template-<runtime> \
  --template Molecule-AI/molecule-ai-workspace-template-starter \
  --public \
  --description "Molecule AI workspace template: <runtime>"

git clone https://github.com/Molecule-AI/molecule-ai-workspace-template-<runtime>
cd molecule-ai-workspace-template-<runtime>

Then fill in the TODO markers in:

File	What to fill in
adapter.py	Rename class to <Runtime>Adapter. Fill in name(), display_name(), description(), get_config_schema(). Implement setup() and create_executor().
requirements.txt	Add your runtime's pip dependencies (e.g. langgraph, crewai, claude-agent-sdk).
Dockerfile	Add runtime-specific apt deps (most runtimes don't need any). Replace ENTRYPOINT only if you need custom boot logic.
config.yaml	Update top-level name/runtime/description. Add the models your runtime supports to models[].
system-prompt.md	Default agent prompt.

After git push:

1. The template's publish-image.yml builds + pushes ghcr.io/molecule-ai/workspace-template-<runtime>:latest automatically.
2. The next runtime-vX.Y.Z tag on molecule-core cascades a repository_dispatch event into your new template, rebuilding the image against the latest runtime — no setup PR required.
3. Register the runtime name in the platform's RuntimeImages map (in workspace-server/internal/provisioner/provisioner.go) so it's selectable in the canvas.

When the starter itself needs to evolve

If the canonical shape changes (e.g. config.yaml schema gets a new field, the BaseAdapter interface adds a method, the reusable CI workflow signature changes), update the starter first. Existing templates can either migrate at their own pace or be touched in a coordinated cleanup PR. Either way, future templates pick up the new shape from day one.

Migration note

Prior to this workflow, the runtime was duplicated across monorepo workspace/ AND a sibling repo molecule-ai-workspace-runtime, with no sync mechanism. That caused 30+ files to drift between the two trees and tonight's chat-leak / queued-classification fixes existed only in the monorepo copy until manually ported.

If you have an old local checkout of molecule-ai-workspace-runtime, treat it as outdated. The monorepo workspace/ is now authoritative; the PyPI artifact is rebuilt from it on every runtime-v* tag.