molecule-ai/molecule-ci
35
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(validate-workspace-template): strict drift gate + canonical-fetch workflow

Browse Source 
P6 Phase 1: enforce the workspace-template contract via CI on every
template-repo push, eliminating the slow drift that produced 8
copies of a 28-line Dockerfile in different states of decay.

The previous validator (50 lines, soft warnings only) couldn't
catch the cache-trap pattern (Dockerfile missing ARG RUNTIME_VERSION)
that silently shipped the previous runtime wheel during cascade
publishes — observed five times in a row on 2026-04-27. Hardened
into structural checks that fail CI, not just warn:

  - Dockerfile must base on python:3.11-slim
  - Dockerfile must declare ARG RUNTIME_VERSION AND reference
    ${RUNTIME_VERSION} in a RUN block (the arg has to be in the
    layer's command line for docker to hash it into the cache key)
  - Dockerfile must create the agent uid-1000 user (Claude Code
    refuses --dangerously-skip-permissions as root for safety)
  - Dockerfile must end at molecule-runtime — directly via
    ENTRYPOINT or via a wrapper script that exec's it (claude-code
    has entrypoint.sh for gosu drop-priv; hermes has start.sh to
    boot the hermes-agent daemon first; both are allowed)
  - config.yaml must have name + runtime + integer
    template_schema_version. Quoted "1" fails — observed previously
    in a copy-pasted template that the YAML loader turned into str
  - requirements.txt must declare molecule-ai-workspace-runtime

Also fixed: the original validator's warning telling adapter.py
NOT to import molecule_runtime was backwards — that's the
canonical package name post-#87. Now it warns on the legacy
molecule_ai prefix instead.

Reusable workflow change: instead of running
.molecule-ci/scripts/validate-workspace-template.py (a per-template
vendored copy that drifts as the validator evolves), the workflow
now checks out molecule-ci itself into .molecule-ci-canonical and
runs the canonical script from there. Single source of truth —
every template runs the SAME contract on every CI run. The legacy
.molecule-ci/scripts/ directories in each template repo can be
deleted in a Phase 2 cleanup PR.

14 unit tests pin the contract:
  - canonical template passes
  - claude-code-style custom entrypoint passes when the wrapper
    exec's molecule-runtime
  - 5 Dockerfile drift modes each error individually
  - 3 config.yaml drift modes each error/warn
  - requirements.txt missing-runtime errors
  - legacy molecule_ai import warns
  - regression cover: modern molecule_runtime import does NOT
    trigger the (deleted) backwards warning

All 8 production template repos pass the new contract today —
this PR locks in the current good state, it does not force any
template-repo edits.

Contract documented at docs/template-contract.md so the rules are
discoverable without reading the validator.
This commit is contained in:
Hongming Wang 2026-04-27 14:50:55 -07:00
parent 9c7f4f5542
commit 73102cdaa9
5 changed files with 741 additions and 72 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
.github/workflows/validate-workspace-template.yml vendored
View File

@ -8,14 +8,25 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
# Calling template repo (Dockerfile + config.yaml + adapter.py).
- uses: actions/checkout@v4
# Canonical validator script lives in molecule-ci, fetched fresh on
# every run. The previous setup expected `.molecule-ci/scripts/` to
# be vendored INTO each template repo, which drifted across the 8
# template repos as the validator evolved. Single source of truth
# eliminates that drift class entirely — every template runs the
# same canonical contract check on every CI run.
- uses: actions/checkout@v4
with:
repository: Molecule-AI/molecule-ci
path: .molecule-ci-canonical
- uses: actions/setup-python@v5
with:
python-version: "3.11"
cache: "pip"
cache-dependency-path: .molecule-ci/scripts/requirements.txt
cache-dependency-path: .molecule-ci-canonical/.molecule-ci/scripts/requirements.txt
- run: pip install pyyaml -q
- run: python3 .molecule-ci/scripts/validate-workspace-template.py
- run: python3 .molecule-ci-canonical/scripts/validate-workspace-template.py
- name: Docker build smoke test
if: hashFiles('Dockerfile') != ''
run: docker build -t template-test . --no-cache 2>&1 | tail -5 && echo "✓ Docker build succeeded"

228
.molecule-ci/scripts/validate-workspace-template.py
View File

@ -1,47 +1,205 @@
#!/usr/bin/env python3
"""Validate a Molecule AI workspace template repo."""
import os, sys, yaml
"""Prototype of the beefed-up validate-workspace-template.py.
errors = []
Run from a template repo's root. Surfaces hard structural drift in
Dockerfile + config.yaml + requirements.txt against the canonical
contract. Replaces the existing soft-warnings-only validator at
molecule-ci/scripts/validate-workspace-template.py.
"""
import os, re, sys
import yaml
if not os.path.isfile("config.yaml"):
print("::error::config.yaml not found at repo root")
sys.exit(1)
ERRORS: list[str] = []
WARNINGS: list[str] = []
with open("config.yaml") as f:
config = yaml.safe_load(f)
def err(msg: str) -> None:
ERRORS.append(msg)
if not config.get("name"):
errors.append("Missing required field: name")
if not config.get("runtime"):
errors.append("Missing required field: runtime")
def warn(msg: str) -> None:
WARNINGS.append(msg)
known = {"langgraph", "claude-code", "crewai", "autogen", "deepagents", "hermes", "gemini-cli", "openclaw"}
runtime = config.get("runtime", "")
if runtime and runtime not in known:
print(f"::warning::Runtime '{runtime}' is not in the known set. OK for custom runtimes.")
# Check for legacy imports
if os.path.isfile("adapter.py"):
with open("adapter.py") as f:
content = f.read()
if "molecule_runtime" in content:
print("::warning::adapter.py imports 'molecule_runtime' — legacy import, use 'molecule_ai' or platform SDK")
# ───────────────────────────────────────────────────────────── Dockerfile
# Check for missing molecule-ai-workspace-runtime dependency hint
if os.path.isfile("Dockerfile"):
with open("Dockerfile") as f:
content = f.read()
if "molecule-ai-workspace-runtime" not in content:
print("::warning::Dockerfile does not reference 'molecule-ai-workspace-runtime' — may need base runtime package")
def check_dockerfile() -> None:
if not os.path.isfile("Dockerfile"):
warn("no Dockerfile — skipping container drift checks (library-only template?)")
return
df = open("Dockerfile").read()
sv = config.get("template_schema_version")
if sv is None:
errors.append("Missing template_schema_version (add: template_schema_version: 1)")
if not re.search(r"^FROM python:3\.11-slim\b", df, re.MULTILINE):
err("Dockerfile: must base on `FROM python:3.11-slim` — see contract doc")
if errors:
for e in errors:
if not re.search(r"^ARG RUNTIME_VERSION", df, re.MULTILINE):
err(
"Dockerfile: missing `ARG RUNTIME_VERSION=`. "
"This arg invalidates the pip-install cache when the cascade "
"publishes a new wheel; without it, the cascade silently ships "
"the previous runtime (cache trap observed 2026-04-27, 5x in a row)."
)
if "molecule-ai-workspace-runtime" not in df and not (
os.path.isfile("requirements.txt")
and "molecule-ai-workspace-runtime" in open("requirements.txt").read()
):
err("Dockerfile + requirements.txt: must install `molecule-ai-workspace-runtime`")
if "${RUNTIME_VERSION}" not in df and "$RUNTIME_VERSION" not in df:
err(
"Dockerfile: must reference `${RUNTIME_VERSION}` in a pip install RUN block. "
'Pattern: `if [ -n "${RUNTIME_VERSION}" ]; then '
'pip install --no-cache-dir --upgrade "molecule-ai-workspace-runtime==${RUNTIME_VERSION}"; fi`'
)
if not re.search(r"useradd[^\n]*\bagent\b", df):
err(
"Dockerfile: must create the `agent` user "
"(`RUN useradd -u 1000 -m -s /bin/bash agent`). "
"Runtime drops to uid 1000; without it, claude-code refuses "
"`--dangerously-skip-permissions` for safety."
)
has_direct_entrypoint = bool(
re.search(r'(ENTRYPOINT|CMD)\s*\[?\s*"?molecule-runtime"?', df)
)
has_custom_entrypoint = bool(
re.search(r'ENTRYPOINT\s*\[?\s*"?(/?[\w./-]*entrypoint\.sh|/?[\w./-]*start\.sh)', df)
)
if not has_direct_entrypoint and not has_custom_entrypoint:
err(
"Dockerfile: must end at `molecule-runtime` "
"(`ENTRYPOINT [\"molecule-runtime\"]` or via custom "
"entrypoint.sh / start.sh that exec's molecule-runtime)"
)
if has_custom_entrypoint:
m = re.search(r'ENTRYPOINT\s*\[?\s*"?(/?[\w./-]+)', df)
if m:
ep_in_image = m.group(1).lstrip("/")
ep_local = os.path.basename(ep_in_image)
if os.path.isfile(ep_local):
if "molecule-runtime" not in open(ep_local).read():
err(
f"Dockerfile uses ENTRYPOINT [{ep_in_image}] but "
f"{ep_local} does not exec `molecule-runtime`"
)
else:
warn(
f"Dockerfile points ENTRYPOINT at {ep_in_image} but "
f"{ep_local} not found in repo root — verify it's COPYed in"
)
# ───────────────────────────────────────────────────────────── config.yaml
KNOWN_RUNTIMES = {
"langgraph",
"claude-code",
"crewai",
"autogen",
"deepagents",
"hermes",
"gemini-cli",
"openclaw",
}
REQUIRED_KEYS = ["name", "runtime", "template_schema_version"]
OPTIONAL_KEYS = [
"description",
"version",
"tier",
"model",
"models",
"runtime_config",
"env",
"skills",
"tools",
"a2a",
"delegation",
"prompt_files",
"bridge",
"governance",
]
def check_config_yaml() -> None:
if not os.path.isfile("config.yaml"):
err("config.yaml: missing at repo root")
return
with open("config.yaml") as f:
try:
config = yaml.safe_load(f)
except yaml.YAMLError as e:
err(f"config.yaml: invalid YAML — {e}")
return
if not isinstance(config, dict):
err(f"config.yaml: root must be a mapping, got {type(config).__name__}")
return
for key in REQUIRED_KEYS:
if key not in config:
err(f"config.yaml: missing required key `{key}`")
runtime = config.get("runtime")
if runtime and runtime not in KNOWN_RUNTIMES:
warn(
f"config.yaml: runtime `{runtime}` not in known set "
f"{sorted(KNOWN_RUNTIMES)} — OK for custom runtimes; "
f"if canonical, add it to KNOWN_RUNTIMES in validate-workspace-template.py"
)
sv = config.get("template_schema_version")
if sv is not None and not isinstance(sv, int):
err(
f"config.yaml: template_schema_version must be int, "
f"got {type(sv).__name__}={sv!r}"
)
unknown = set(config.keys()) - set(REQUIRED_KEYS) - set(OPTIONAL_KEYS)
if unknown:
warn(
f"config.yaml: unknown top-level keys {sorted(unknown)} — "
f"may be drift. If intentional, add them to OPTIONAL_KEYS."
)
# ───────────────────────────────────────────────────────────── requirements.txt
def check_requirements() -> None:
if not os.path.isfile("requirements.txt"):
warn("no requirements.txt — Dockerfile must install runtime by other means")
return
reqs = open("requirements.txt").read()
if "molecule-ai-workspace-runtime" not in reqs:
err("requirements.txt: must declare `molecule-ai-workspace-runtime` as a dependency")
# ───────────────────────────────────────────────────────────── adapter.py
def check_adapter() -> None:
if not os.path.isfile("adapter.py"):
warn("no adapter.py — runtime will use the default langgraph executor from the wheel")
return
content = open("adapter.py").read()
# The original validator's warning ("don't import molecule_runtime") was
# backwards — that's the canonical package name. The previous check shipped
# for ~2 weeks producing false-positive warnings. Removed.
if re.search(r"\bfrom molecule_ai\b|\bimport molecule_ai\b", content):
warn(
"adapter.py imports `molecule_ai` — that's a pre-#87 package name; "
"use `molecule_runtime`"
)
def main() -> None:
check_dockerfile()
check_config_yaml()
check_requirements()
check_adapter()
for w in WARNINGS:
print(f"::warning::{w}")
for e in ERRORS:
print(f"::error::{e}")
sys.exit(1)
if ERRORS:
sys.exit(1)
print(f"✓ Template validation passed ({len(WARNINGS)} warning(s))")
print(f"✓ config.yaml valid: {config['name']} (runtime: {config.get('runtime')})")
if __name__ == "__main__":
main()

67
docs/template-contract.md Normal file
View File

@ -0,0 +1,67 @@
# Workspace Template Contract
Hard rules every `molecule-ai-workspace-template-*` repo must satisfy. Enforced by `scripts/validate-workspace-template.py` on every CI run via the reusable `validate-workspace-template.yml` workflow.
The contract exists because the 8 template repos were extracted from a single monolithic Dockerfile pre-#87, and have drifted as each was edited piecemeal since. Without this gate, a 28-line cascade-friendly Dockerfile in one repo silently regresses to a 25-line non-cache-friendly one in another, and the next runtime publish ships the previous wheel from a stale layer (cache trap observed five times in a row on 2026-04-27).
## Dockerfile
| Rule | Why |
|---|---|
| `FROM python:3.11-slim` | Single base everywhere — keeps apt + pip behaviour identical and lets us reason about CVE patches on one base. |
| `ARG RUNTIME_VERSION=` declared | The arg invalidates the pip-install layer's cache key whenever the cascade publishes a new wheel. Without it the cache hit replays the previous runtime. |
| `${RUNTIME_VERSION}` referenced in a `RUN` | Just declaring the ARG isn't enough — it has to be in the layer's command line so docker hashes it. Pattern: `if [ -n "${RUNTIME_VERSION}" ]; then pip install --no-cache-dir --upgrade "molecule-ai-workspace-runtime==${RUNTIME_VERSION}"; fi` |
| `RUN useradd -u 1000 -m -s /bin/bash agent` | The runtime drops to uid 1000 before exec'ing the SDK. Claude Code refuses `--dangerously-skip-permissions` as root for safety. The `/workspace` volume is also chown'd to 1000 by the platform provisioner. |
| `ENTRYPOINT ["molecule-runtime"]` *or* a wrapper script that exec's `molecule-runtime` | Single entrypoint means the platform's container-restart contract is uniform across templates. Wrapper scripts are allowed (claude-code has `entrypoint.sh` for gosu drop-priv; hermes has `start.sh` to boot the hermes-agent daemon first). |
| `molecule-ai-workspace-runtime` listed in `requirements.txt` (or installed in the Dockerfile directly) | The runtime wheel is the contract — without it the container has no A2A server, no heartbeat, no MCP bridge. |
## config.yaml
| Required key | Type | Notes |
|---|---|---|
| `name` | str | Human-readable; appears on the canvas card. |
| `runtime` | str | Must be one of: `langgraph`, `claude-code`, `crewai`, `autogen`, `deepagents`, `hermes`, `gemini-cli`, `openclaw`. Custom runtimes warn but are allowed. |
| `template_schema_version` | int | Currently `1`. Bump when adding a key that changes how the platform consumes config.yaml. **Must be int**, not string — a quoted `"1"` will fail validation. |
| Optional key | Notes |
|---|---|
| `description` | Free text, surfaces on canvas. |
| `version`, `tier` | int, controls platform-side rollout gating. |
| `model`, `models` | Either a single model id or a list of model ids the agent may use. |
| `runtime_config` | Nested block of runtime-specific settings (used by claude-code, gemini-cli, hermes). |
| `env`, `skills`, `tools`, `a2a`, `delegation`, `prompt_files`, `bridge`, `governance` | Optional feature blocks. Add new keys to `OPTIONAL_KEYS` in the validator when introducing them. |
Unknown top-level keys produce a warning (not an error) so accidental drift is visible without blocking.
## adapter.py
Optional. When present, `adapter.py` should:
- Import `BaseAdapter` from `molecule_runtime.adapter_base`.
- Override `setup()` and `create_executor()` for the runtime's specific entry point.
The pre-#87 import path (`molecule_ai`) produces a warning if it appears.
## requirements.txt
Must declare `molecule-ai-workspace-runtime` (with a version pin or floor).
## CI
Every template repo's `.github/workflows/ci.yml` should be a one-liner that calls the canonical reusable workflow:
```yaml
name: CI
on: [push, pull_request]
jobs:
validate:
uses: Molecule-AI/molecule-ci/.github/workflows/validate-workspace-template.yml@main
```
The reusable workflow checks out `molecule-ci` itself (into `.molecule-ci-canonical`) and runs the canonical `validate-workspace-template.py` from there — so no per-repo vendoring of the script is needed. The legacy `.molecule-ci/scripts/` directory in each template repo is being phased out.
## Adding a new runtime
1. Add the runtime name to `KNOWN_RUNTIMES` in `scripts/validate-workspace-template.py`.
2. Add the runtime + image ref to `RuntimeImages` in `molecule-core/workspace-server/internal/provisioner/provisioner.go`.
3. Stand up the `molecule-ai-workspace-template-<runtime>` repo from the existing template-of-templates pattern (issue #105 covers this).
4. Confirm CI green on the new repo before opening it for general use.

275
scripts/test_validate_workspace_template.py Normal file
View File

@ -0,0 +1,275 @@
"""Tests for validate-workspace-template.py — pin the drift contract.
Each test materialises a tiny template directory in a tmpdir, runs the
validator's check functions in-process, and asserts on the captured
ERRORS / WARNINGS lists. The 8 template repos in the wild are the
ground-truth integration test (CI runs this validator against each on
push), but those repos can change at any time. These tests pin the
contract itself so a refactor of the validator can't silently weaken
it.
Important: the validator was chosen to be import-safe (no top-level
side effects), so the test patches the cwd via os.chdir into tmpdirs.
The module's ERRORS/WARNINGS lists are reset at the start of each
test via _reset_validator_state().
"""
from __future__ import annotations
import importlib.util
import os
from pathlib import Path
import pytest
VALIDATOR_PATH = Path(__file__).resolve().parent / "validate-workspace-template.py"
def _load_validator():
"""Load the validator module by path (its filename has a hyphen so
we can't `import validate-workspace-template` directly)."""
spec = importlib.util.spec_from_file_location("validator", VALIDATOR_PATH)
assert spec is not None and spec.loader is not None
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)
return mod
@pytest.fixture
def validator(monkeypatch):
"""Fresh validator module per test, cwd pinned to tmpdir below."""
mod = _load_validator()
mod.ERRORS.clear()
mod.WARNINGS.clear()
return mod
def _good_dockerfile() -> str:
"""Canonical Dockerfile that should pass every check."""
return (
"FROM python:3.11-slim\n"
"ARG RUNTIME_VERSION=\n"
"RUN useradd -u 1000 -m -s /bin/bash agent\n"
"WORKDIR /app\n"
"COPY requirements.txt .\n"
'RUN pip install -r requirements.txt && \\\n'
' if [ -n "${RUNTIME_VERSION}" ]; then \\\n'
' pip install --upgrade "molecule-ai-workspace-runtime==${RUNTIME_VERSION}"; \\\n'
' fi\n'
'ENTRYPOINT ["molecule-runtime"]\n'
)
def _good_config_yaml() -> str:
return (
"name: test-template\n"
"runtime: claude-code\n"
"template_schema_version: 1\n"
"description: A test template\n"
"tier: 1\n"
)
def _good_requirements_txt() -> str:
return "molecule-ai-workspace-runtime>=0.1.0\n"
def _materialise(tmp_path: Path, dockerfile: str | None = None,
config_yaml: str | None = None,
requirements: str | None = None,
adapter_py: str | None = None) -> None:
if dockerfile is not None:
(tmp_path / "Dockerfile").write_text(dockerfile)
if config_yaml is not None:
(tmp_path / "config.yaml").write_text(config_yaml)
if requirements is not None:
(tmp_path / "requirements.txt").write_text(requirements)
if adapter_py is not None:
(tmp_path / "adapter.py").write_text(adapter_py)
# ───────────────────────────────────────────────────────── happy paths
def test_canonical_template_passes(validator, tmp_path, monkeypatch):
_materialise(
tmp_path,
dockerfile=_good_dockerfile(),
config_yaml=_good_config_yaml(),
requirements=_good_requirements_txt(),
)
monkeypatch.chdir(tmp_path)
validator.check_dockerfile()
validator.check_config_yaml()
validator.check_requirements()
validator.check_adapter()
assert validator.ERRORS == [], validator.ERRORS
def test_custom_entrypoint_script_passes_when_it_execs_runtime(validator, tmp_path, monkeypatch):
"""claude-code style: ENTRYPOINT [/entrypoint.sh] + entrypoint.sh
that exec's molecule-runtime at the end. Must pass."""
df = (
"FROM python:3.11-slim\n"
"ARG RUNTIME_VERSION=\n"
"RUN useradd -u 1000 -m -s /bin/bash agent\n"
"COPY requirements.txt .\n"
'RUN pip install -r requirements.txt && \\\n'
' if [ -n "${RUNTIME_VERSION}" ]; then \\\n'
' pip install --upgrade "molecule-ai-workspace-runtime==${RUNTIME_VERSION}"; \\\n'
' fi\n'
"COPY entrypoint.sh /entrypoint.sh\n"
'ENTRYPOINT ["/entrypoint.sh"]\n'
)
ep = (
"#!/bin/sh\n"
"set -e\n"
'# drop privileges then exec the runtime\n'
'exec gosu agent molecule-runtime "$@"\n'
)
_materialise(
tmp_path,
dockerfile=df,
config_yaml=_good_config_yaml(),
requirements=_good_requirements_txt(),
)
(tmp_path / "entrypoint.sh").write_text(ep)
monkeypatch.chdir(tmp_path)
validator.check_dockerfile()
assert validator.ERRORS == [], validator.ERRORS
# ───────────────────────────────────────────────────────── Dockerfile drift
def test_wrong_base_image_errors(validator, tmp_path, monkeypatch):
df = _good_dockerfile().replace("python:3.11-slim", "python:3.10-alpine")
_materialise(tmp_path, dockerfile=df, config_yaml=_good_config_yaml(),
requirements=_good_requirements_txt())
monkeypatch.chdir(tmp_path)
validator.check_dockerfile()
assert any("FROM python:3.11-slim" in e for e in validator.ERRORS)
def test_missing_arg_runtime_version_errors(validator, tmp_path, monkeypatch):
"""Without ARG RUNTIME_VERSION, the cascade rebuild silently ships
the previous runtime — the cache trap that bit us 5x on 2026-04-27."""
df = _good_dockerfile().replace("ARG RUNTIME_VERSION=\n", "")
_materialise(tmp_path, dockerfile=df, config_yaml=_good_config_yaml(),
requirements=_good_requirements_txt())
monkeypatch.chdir(tmp_path)
validator.check_dockerfile()
assert any("ARG RUNTIME_VERSION" in e for e in validator.ERRORS)
def test_missing_runtime_version_in_run_block_errors(validator, tmp_path, monkeypatch):
"""ARG declared but NEVER referenced in a RUN — same cache-trap,
different shape. Pin both."""
df = (
"FROM python:3.11-slim\n"
"ARG RUNTIME_VERSION=\n"
"RUN useradd -u 1000 -m -s /bin/bash agent\n"
"RUN pip install molecule-ai-workspace-runtime\n"
'ENTRYPOINT ["molecule-runtime"]\n'
)
_materialise(tmp_path, dockerfile=df, config_yaml=_good_config_yaml(),
requirements=_good_requirements_txt())
monkeypatch.chdir(tmp_path)
validator.check_dockerfile()
assert any("RUNTIME_VERSION" in e and "RUN block" in e for e in validator.ERRORS)
def test_missing_agent_user_errors(validator, tmp_path, monkeypatch):
df = _good_dockerfile().replace("RUN useradd -u 1000 -m -s /bin/bash agent\n", "")
_materialise(tmp_path, dockerfile=df, config_yaml=_good_config_yaml(),
requirements=_good_requirements_txt())
monkeypatch.chdir(tmp_path)
validator.check_dockerfile()
assert any("agent" in e for e in validator.ERRORS)
def test_missing_entrypoint_errors(validator, tmp_path, monkeypatch):
df = _good_dockerfile().replace('ENTRYPOINT ["molecule-runtime"]\n', "")
_materialise(tmp_path, dockerfile=df, config_yaml=_good_config_yaml(),
requirements=_good_requirements_txt())
monkeypatch.chdir(tmp_path)
validator.check_dockerfile()
assert any("molecule-runtime" in e and ("ENTRYPOINT" in e or "entrypoint" in e)
for e in validator.ERRORS)
# ───────────────────────────────────────────────────────── config.yaml drift
def test_missing_required_keys_errors(validator, tmp_path, monkeypatch):
cfg = "description: only description, no name/runtime/version\n"
_materialise(tmp_path, dockerfile=_good_dockerfile(), config_yaml=cfg,
requirements=_good_requirements_txt())
monkeypatch.chdir(tmp_path)
validator.check_config_yaml()
missing_msgs = [e for e in validator.ERRORS if "missing required key" in e]
assert len(missing_msgs) >= 3 # name, runtime, template_schema_version
def test_string_template_schema_version_errors(validator, tmp_path, monkeypatch):
cfg = (
"name: t\n"
"runtime: claude-code\n"
'template_schema_version: "1"\n' # str, not int
)
_materialise(tmp_path, dockerfile=_good_dockerfile(), config_yaml=cfg,
requirements=_good_requirements_txt())
monkeypatch.chdir(tmp_path)
validator.check_config_yaml()
assert any("template_schema_version must be int" in e for e in validator.ERRORS)
def test_unknown_runtime_warns_not_errors(validator, tmp_path, monkeypatch):
cfg = _good_config_yaml().replace("claude-code", "my-experimental-runtime")
_materialise(tmp_path, dockerfile=_good_dockerfile(), config_yaml=cfg,
requirements=_good_requirements_txt())
monkeypatch.chdir(tmp_path)
validator.check_config_yaml()
assert any("not in known set" in w for w in validator.WARNINGS)
assert validator.ERRORS == [] # custom runtimes are allowed
def test_unknown_top_level_keys_warn(validator, tmp_path, monkeypatch):
cfg = _good_config_yaml() + "weird_drift_key: something\n"
_materialise(tmp_path, dockerfile=_good_dockerfile(), config_yaml=cfg,
requirements=_good_requirements_txt())
monkeypatch.chdir(tmp_path)
validator.check_config_yaml()
assert any("unknown top-level keys" in w and "weird_drift_key" in w
for w in validator.WARNINGS)
# ───────────────────────────────────────────────────────── requirements.txt
def test_missing_runtime_in_requirements_errors(validator, tmp_path, monkeypatch):
_materialise(tmp_path, dockerfile=_good_dockerfile(), config_yaml=_good_config_yaml(),
requirements="fastapi\n")
monkeypatch.chdir(tmp_path)
validator.check_requirements()
assert any("molecule-ai-workspace-runtime" in e for e in validator.ERRORS)
# ───────────────────────────────────────────────────────── adapter.py
def test_legacy_molecule_ai_import_warns(validator, tmp_path, monkeypatch):
"""Pre-#87 package was named differently. Catch any laggards."""
adapter = "from molecule_ai.adapter_base import BaseAdapter\n"
_materialise(tmp_path, adapter_py=adapter)
monkeypatch.chdir(tmp_path)
validator.check_adapter()
assert any("molecule_ai" in w for w in validator.WARNINGS)
def test_modern_molecule_runtime_import_does_not_warn(validator, tmp_path, monkeypatch):
"""Regression cover: the original validator's warning ('don't import
molecule_runtime') was BACKWARDS — that's the canonical name now.
Pin that the new validator does NOT emit a false positive."""
adapter = "from molecule_runtime.adapter_base import BaseAdapter\n"
_materialise(tmp_path, adapter_py=adapter)
monkeypatch.chdir(tmp_path)
validator.check_adapter()
legacy_warnings = [w for w in validator.WARNINGS if "molecule_ai" in w]
assert legacy_warnings == [], legacy_warnings

228
scripts/validate-workspace-template.py
View File

@ -1,47 +1,205 @@
#!/usr/bin/env python3
"""Validate a Molecule AI workspace template repo."""
import os, sys, yaml
"""Prototype of the beefed-up validate-workspace-template.py.
errors = []
Run from a template repo's root. Surfaces hard structural drift in
Dockerfile + config.yaml + requirements.txt against the canonical
contract. Replaces the existing soft-warnings-only validator at
molecule-ci/scripts/validate-workspace-template.py.
"""
import os, re, sys
import yaml
if not os.path.isfile("config.yaml"):
print("::error::config.yaml not found at repo root")
sys.exit(1)
ERRORS: list[str] = []
WARNINGS: list[str] = []
with open("config.yaml") as f:
config = yaml.safe_load(f)
def err(msg: str) -> None:
ERRORS.append(msg)
if not config.get("name"):
errors.append("Missing required field: name")
if not config.get("runtime"):
errors.append("Missing required field: runtime")
def warn(msg: str) -> None:
WARNINGS.append(msg)
known = {"langgraph", "claude-code", "crewai", "autogen", "deepagents", "hermes", "gemini-cli", "openclaw"}
runtime = config.get("runtime", "")
if runtime and runtime not in known:
print(f"::warning::Runtime '{runtime}' is not in the known set. OK for custom runtimes.")
# Check for legacy imports
if os.path.isfile("adapter.py"):
with open("adapter.py") as f:
content = f.read()
if "molecule_runtime" in content:
print("::warning::adapter.py imports 'molecule_runtime' — legacy import, use 'molecule_ai' or platform SDK")
# ───────────────────────────────────────────────────────────── Dockerfile
# Check for missing molecule-ai-workspace-runtime dependency hint
if os.path.isfile("Dockerfile"):
with open("Dockerfile") as f:
content = f.read()
if "molecule-ai-workspace-runtime" not in content:
print("::warning::Dockerfile does not reference 'molecule-ai-workspace-runtime' — may need base runtime package")
def check_dockerfile() -> None:
if not os.path.isfile("Dockerfile"):
warn("no Dockerfile — skipping container drift checks (library-only template?)")
return
df = open("Dockerfile").read()
sv = config.get("template_schema_version")
if sv is None:
errors.append("Missing template_schema_version (add: template_schema_version: 1)")
if not re.search(r"^FROM python:3\.11-slim\b", df, re.MULTILINE):
err("Dockerfile: must base on `FROM python:3.11-slim` — see contract doc")
if errors:
for e in errors:
if not re.search(r"^ARG RUNTIME_VERSION", df, re.MULTILINE):
err(
"Dockerfile: missing `ARG RUNTIME_VERSION=`. "
"This arg invalidates the pip-install cache when the cascade "
"publishes a new wheel; without it, the cascade silently ships "
"the previous runtime (cache trap observed 2026-04-27, 5x in a row)."
)
if "molecule-ai-workspace-runtime" not in df and not (
os.path.isfile("requirements.txt")
and "molecule-ai-workspace-runtime" in open("requirements.txt").read()
):
err("Dockerfile + requirements.txt: must install `molecule-ai-workspace-runtime`")
if "${RUNTIME_VERSION}" not in df and "$RUNTIME_VERSION" not in df:
err(
"Dockerfile: must reference `${RUNTIME_VERSION}` in a pip install RUN block. "
'Pattern: `if [ -n "${RUNTIME_VERSION}" ]; then '
'pip install --no-cache-dir --upgrade "molecule-ai-workspace-runtime==${RUNTIME_VERSION}"; fi`'
)
if not re.search(r"useradd[^\n]*\bagent\b", df):
err(
"Dockerfile: must create the `agent` user "
"(`RUN useradd -u 1000 -m -s /bin/bash agent`). "
"Runtime drops to uid 1000; without it, claude-code refuses "
"`--dangerously-skip-permissions` for safety."
)
has_direct_entrypoint = bool(
re.search(r'(ENTRYPOINT|CMD)\s*\[?\s*"?molecule-runtime"?', df)
)
has_custom_entrypoint = bool(
re.search(r'ENTRYPOINT\s*\[?\s*"?(/?[\w./-]*entrypoint\.sh|/?[\w./-]*start\.sh)', df)
)
if not has_direct_entrypoint and not has_custom_entrypoint:
err(
"Dockerfile: must end at `molecule-runtime` "
"(`ENTRYPOINT [\"molecule-runtime\"]` or via custom "
"entrypoint.sh / start.sh that exec's molecule-runtime)"
)
if has_custom_entrypoint:
m = re.search(r'ENTRYPOINT\s*\[?\s*"?(/?[\w./-]+)', df)
if m:
ep_in_image = m.group(1).lstrip("/")
ep_local = os.path.basename(ep_in_image)
if os.path.isfile(ep_local):
if "molecule-runtime" not in open(ep_local).read():
err(
f"Dockerfile uses ENTRYPOINT [{ep_in_image}] but "
f"{ep_local} does not exec `molecule-runtime`"
)
else:
warn(
f"Dockerfile points ENTRYPOINT at {ep_in_image} but "
f"{ep_local} not found in repo root — verify it's COPYed in"
)
# ───────────────────────────────────────────────────────────── config.yaml
KNOWN_RUNTIMES = {
"langgraph",
"claude-code",
"crewai",
"autogen",
"deepagents",
"hermes",
"gemini-cli",
"openclaw",
}
REQUIRED_KEYS = ["name", "runtime", "template_schema_version"]
OPTIONAL_KEYS = [
"description",
"version",
"tier",
"model",
"models",
"runtime_config",
"env",
"skills",
"tools",
"a2a",
"delegation",
"prompt_files",
"bridge",
"governance",
]
def check_config_yaml() -> None:
if not os.path.isfile("config.yaml"):
err("config.yaml: missing at repo root")
return
with open("config.yaml") as f:
try:
config = yaml.safe_load(f)
except yaml.YAMLError as e:
err(f"config.yaml: invalid YAML — {e}")
return
if not isinstance(config, dict):
err(f"config.yaml: root must be a mapping, got {type(config).__name__}")
return
for key in REQUIRED_KEYS:
if key not in config:
err(f"config.yaml: missing required key `{key}`")
runtime = config.get("runtime")
if runtime and runtime not in KNOWN_RUNTIMES:
warn(
f"config.yaml: runtime `{runtime}` not in known set "
f"{sorted(KNOWN_RUNTIMES)} — OK for custom runtimes; "
f"if canonical, add it to KNOWN_RUNTIMES in validate-workspace-template.py"
)
sv = config.get("template_schema_version")
if sv is not None and not isinstance(sv, int):
err(
f"config.yaml: template_schema_version must be int, "
f"got {type(sv).__name__}={sv!r}"
)
unknown = set(config.keys()) - set(REQUIRED_KEYS) - set(OPTIONAL_KEYS)
if unknown:
warn(
f"config.yaml: unknown top-level keys {sorted(unknown)} — "
f"may be drift. If intentional, add them to OPTIONAL_KEYS."
)
# ───────────────────────────────────────────────────────────── requirements.txt
def check_requirements() -> None:
if not os.path.isfile("requirements.txt"):
warn("no requirements.txt — Dockerfile must install runtime by other means")
return
reqs = open("requirements.txt").read()
if "molecule-ai-workspace-runtime" not in reqs:
err("requirements.txt: must declare `molecule-ai-workspace-runtime` as a dependency")
# ───────────────────────────────────────────────────────────── adapter.py
def check_adapter() -> None:
if not os.path.isfile("adapter.py"):
warn("no adapter.py — runtime will use the default langgraph executor from the wheel")
return
content = open("adapter.py").read()
# The original validator's warning ("don't import molecule_runtime") was
# backwards — that's the canonical package name. The previous check shipped
# for ~2 weeks producing false-positive warnings. Removed.
if re.search(r"\bfrom molecule_ai\b|\bimport molecule_ai\b", content):
warn(
"adapter.py imports `molecule_ai` — that's a pre-#87 package name; "
"use `molecule_runtime`"
)
def main() -> None:
check_dockerfile()
check_config_yaml()
check_requirements()
check_adapter()
for w in WARNINGS:
print(f"::warning::{w}")
for e in ERRORS:
print(f"::error::{e}")
sys.exit(1)
if ERRORS:
sys.exit(1)
print(f"✓ Template validation passed ({len(WARNINGS)} warning(s))")
print(f"✓ config.yaml valid: {config['name']} (runtime: {config.get('runtime')})")
if __name__ == "__main__":
main()