infra-runtime-be
1c33df62a4
meta-ci-advisory / meta (advisory) (push) Skipped
publish-image / Resolve runtime version (push) Skipped
publish-image / Classify change (decouple publish from promote) (push) Skipped
publish-image / Build & push workspace-template-hermes image (push) Skipped
publish-image / Promote runtime_image_pins (CP admin) (staging-api.moleculesai.app, staging) (push) Skipped
publish-image / Verify runtime_image_pins moved (fail-loud) (staging-api.moleculesai.app, staging) (push) Skipped
Secret scan / Scan diff for credential-shaped strings (push) Skipped
sync-providers-yaml / Compare synced providers.yaml against controlplane canonical (push) Skipped
verify-providers-projection / Regenerate projection, fail on drift, assert registry ⊆ template (push) Skipped
sync-providers-yaml / Compare synced providers.yaml against controlplane canonical (pull_request) Skipped
meta-ci-advisory / meta (advisory) (pull_request) Successful in 4s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 3s
CI / Template validation (static) (pull_request) Successful in 17s
verify-providers-projection / Regenerate projection, fail on drift, assert registry ⊆ template (pull_request) Successful in 14s
CI / Template validation (static) (push) Successful in 33s
CI / Adapter-socket conformance (pull_request) Successful in 39s
CI / Adapter-socket conformance (push) Successful in 48s
CI / Shell unit tests (pull_request) Successful in 43s
CI / Shell unit tests (push) Successful in 34s
CI / Gateway live round-trip (pinned hermes) (pull_request) Successful in 1m3s
CI / Gateway live round-trip (pinned hermes) (push) Successful in 1m15s
CI / Template validation (runtime) (pull_request) Successful in 5m3s
CI / Template validation (runtime) (push) Successful in 4m50s
CI / T4 tier-4 conformance (live) (pull_request) Successful in 5m10s
CI / T4 tier-4 conformance (live) (push) Successful in 4m52s
CI / validate (push) Successful in 1s
CI / all-required (push) Successful in 1s
CI / validate (pull_request) Successful in 2s
CI / all-required (pull_request) Successful in 1s
1151 lines
62 KiB
YAML
1151 lines
62 KiB
YAML
name: CI
|
|
|
|
# Repository-local validation pipeline. It fetches the canonical validator
|
|
# from molecule-ci at a reviewed immutable commit on each run, while keeping
|
|
# this repository's protected context names and runner requirements explicit.
|
|
#
|
|
# Four-surface migration audit (feedback_gitea_actions_migration_audit_pattern):
|
|
# 1. YAML — no `workflow_dispatch.inputs`; no `merge_group`; preserved
|
|
# `on: [push, pull_request]` from the original. Added workflow-level
|
|
# env.GITHUB_SERVER_URL (feedback_act_runner_github_server_url).
|
|
# 2. Cache — `actions/setup-python` `cache: pip` preserved; works against
|
|
# Gitea's built-in cache server when runner.cache is configured.
|
|
# 3. Token — uses auto-injected GITHUB_TOKEN (Gitea-aliased). Validator
|
|
# job needs only `contents: read` (no write to issues/PRs).
|
|
# 4. Docs — anonymous exact-SHA fetch of public molecule-ci with credential
|
|
# helpers disabled.
|
|
#
|
|
# Avoid job-level `if:` guards on protected contexts: skipped required jobs
|
|
# can remain pending in the branch-protection view. Runtime validation runs
|
|
# after static validation via `needs:` so protected contexts complete.
|
|
#
|
|
# Cross-links:
|
|
# - internal#326 — parent tracking issue
|
|
# - molecule-ai/molecule-ci/.gitea/workflows/validate-workspace-template.yml — pattern source
|
|
# - molecule-ai/molecule-core/.gitea/workflows/ci.yml — Gitea port style reference
|
|
|
|
on: [push, pull_request]
|
|
|
|
# Defense-in-depth de-dup ONLY (the t4-conformance unique-name fix is the
|
|
# actual fail-closed primitive against the shared-host-daemon race; see
|
|
# that job). Scope per workflow + ref + EVENT so the push run and the
|
|
# pull_request run of the same internal-PR commit get DISTINCT groups —
|
|
# they must both complete (each emits its own required-status context;
|
|
# feedback_gitea_gate_check_required_list_not_combined_status). Never
|
|
# per-SHA-global: that silently cross-cancels legit required checks
|
|
# (feedback_concurrency_group_per_sha). cancel-in-progress:false so an
|
|
# in-flight live T4 probe is never aborted mid-assertion (a cancelled
|
|
# privileged probe would look like a gate failure / flake); a newer push
|
|
# to the same ref+event simply queues behind it.
|
|
concurrency:
|
|
group: ci-${{ github.workflow }}-${{ github.event_name }}-${{ github.ref }}
|
|
cancel-in-progress: false
|
|
|
|
env:
|
|
# Belt-and-suspenders against the runner-default trap
|
|
# (feedback_act_runner_github_server_url). Runners are configured
|
|
# with this env via /opt/molecule/runners/config.yaml runner.envs,
|
|
# but pinning at the workflow level protects against a runner
|
|
# regenerated without the config file.
|
|
GITHUB_SERVER_URL: https://git.moleculesai.app
|
|
# Canonical validation tooling is reviewed and content-addressed.
|
|
MOLECULE_CI_REF: 11b8598e5c0b3f0b1031733a8d5f6bc238f146a4
|
|
# HERMETIC dep policy (CI-robustness): do NOT set PIP_INDEX_URL=private —
|
|
# it force-routes EVERY pip install (pytest, pyyaml, jsonschema, …) at the
|
|
# private index, which serves only molecules-workspace-runtime → 404s all
|
|
# public tooling (the Adapter-unit-tests / Template-validation reds). Public
|
|
# tooling resolves from the default (PyPI) index; the private runtime — which
|
|
# is ABSENT from public PyPI, so no dependency-confusion — is pulled via an
|
|
# EXPLICIT --extra-index-url on the runtime install only. The legacy name
|
|
# molecule-ai-workspace-runtime IS squatted on public PyPI (0.1.999999); we
|
|
# install the CURRENT dist name molecules-workspace-runtime (private-only).
|
|
MOLECULE_PRIVATE_INDEX: https://git.moleculesai.app/api/packages/molecule-ai/pypi/simple/
|
|
|
|
# Defense-in-depth on the GITHUB_TOKEN scope. The validate-runtime job
|
|
# runs untrusted-by-design code from the calling repo — pip-installs
|
|
# requirements.txt (post-install hooks), imports adapter.py, and
|
|
# docker-builds the Dockerfile. Each primitive can execute arbitrary
|
|
# code with the token in env. Pinning `contents: read` means the worst
|
|
# a malicious template PR can do with the token is read public repo
|
|
# state — no write to issues, no push to branches, no comment-spam.
|
|
permissions:
|
|
contents: read
|
|
|
|
jobs:
|
|
validate-static:
|
|
name: Template validation (static)
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 5
|
|
steps:
|
|
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
with:
|
|
persist-credentials: false
|
|
# Canonical validator script lives in molecule-ci. Fetch the reviewed
|
|
# commit anonymously without consulting runner credential helpers.
|
|
- name: Fetch immutable molecule-ci canonical scripts
|
|
run: |
|
|
set -euo pipefail
|
|
CI_ROOT=".molecule-ci-canonical"
|
|
mkdir "$CI_ROOT"
|
|
git -C "$CI_ROOT" init -q
|
|
git -C "$CI_ROOT" remote add origin https://git.moleculesai.app/molecule-ai/molecule-ci.git
|
|
fetched=false
|
|
for attempt in 1 2 3; do
|
|
if GIT_ASKPASS=/bin/false GIT_TERMINAL_PROMPT=0 git -c credential.helper= -c http.userAgent=curl/8.4.0 -C "$CI_ROOT" fetch --no-tags --depth 1 origin "$MOLECULE_CI_REF"; then
|
|
fetched=true
|
|
break
|
|
fi
|
|
sleep "$attempt"
|
|
done
|
|
if [ "$fetched" != true ]; then
|
|
echo "::error::immutable molecule-ci validator fetch failed"
|
|
exit 1
|
|
fi
|
|
git -C "$CI_ROOT" checkout -q --detach FETCH_HEAD
|
|
ACTUAL_CI_REF="$(git -C "$CI_ROOT" rev-parse HEAD)"
|
|
if [ "$ACTUAL_CI_REF" != "$MOLECULE_CI_REF" ]; then
|
|
echo "::error::immutable molecule-ci validator ref mismatch"
|
|
exit 1
|
|
fi
|
|
- uses: actions/setup-python@v5
|
|
with:
|
|
python-version: "3.11"
|
|
# Secret scan — the most important check. Always runs, including
|
|
# on fork PRs (no third-party code executes here).
|
|
- name: Check for secrets
|
|
run: |
|
|
python3 - << 'PYEOF'
|
|
import os, re, sys
|
|
from pathlib import Path
|
|
|
|
PATTERNS = [
|
|
re.compile(r'''["']sk-ant-[a-zA-Z0-9]{50,}["']'''),
|
|
re.compile(r'''["']ghp_[a-zA-Z0-9]{36,}["']'''),
|
|
re.compile(r'''["']AKIA[A-Z0-9]{16}["']'''),
|
|
re.compile(r'''["'][a-zA-Z0-9/+=]{40}["']'''),
|
|
re.compile(r'''["']sk_test_[a-zA-Z0-9]{24,}["']'''),
|
|
re.compile(r'''["']Bearer\s+[a-zA-Z0-9_.-]{20,}["']'''),
|
|
re.compile(r'''ghp_[a-zA-Z0-9]{36,}'''),
|
|
re.compile(r'''sk-ant-[a-zA-Z0-9]{50,}'''),
|
|
]
|
|
SKIP_DIRS = {'.molecule-ci', '.molecule-ci-canonical', '.git', 'node_modules', '__pycache__'}
|
|
EXTENSIONS = {'.yaml', '.yml', '.md', '.py', '.sh'}
|
|
|
|
def is_false_positive(line):
|
|
ctx = line.lower()
|
|
return '...' in ctx or '<example' in ctx or '</example' in ctx
|
|
|
|
# A 40-char git object id is not a credential. The generic
|
|
# ["'][a-zA-Z0-9/+=]{40}["'] rule above cannot tell a base64-ish
|
|
# 40-char blob from a SHA-1, and this repo pins upstream commits by
|
|
# SHA on purpose (ARG HERMES_COMMIT, ARG HERMES_PLATFORM_MOLECULE_A2A_REF,
|
|
# and the anchors in tests/ that must match them). Exempting at the
|
|
# MATCH level rather than reaching for the line-level `...` escape is
|
|
# deliberate: `...` disables every pattern for the whole line, so a
|
|
# real key sharing a line with a SHA would sail through.
|
|
#
|
|
# Narrow on purpose — exactly 40 chars, lowercase hex only. None of
|
|
# the credential formats above live in that alphabet: a base64 token
|
|
# of length 40 that happens to be all-lowercase-hex has ~1e-9 odds,
|
|
# and every named format (sk-ant-, ghp_, AKIA, sk_test_, Bearer) is
|
|
# matched by its own prefix rule regardless.
|
|
GIT_SHA1 = re.compile(r'\A["\'][0-9a-f]{40}["\']\Z')
|
|
|
|
def is_git_sha(matched_text):
|
|
return bool(GIT_SHA1.match(matched_text))
|
|
|
|
root = Path(os.environ.get('GITHUB_WORKSPACE', '.'))
|
|
warnings = []
|
|
for dirpath, dirnames, filenames in os.walk(root):
|
|
dirnames[:] = [d for d in dirnames if d not in SKIP_DIRS]
|
|
for filename in filenames:
|
|
if Path(filename).suffix not in EXTENSIONS:
|
|
continue
|
|
filepath = Path(dirpath) / filename
|
|
try:
|
|
with open(filepath, 'r', encoding='utf-8', errors='ignore') as f:
|
|
for lineno, line in enumerate(f.readlines(), 1):
|
|
for pattern in PATTERNS:
|
|
for match in pattern.finditer(line):
|
|
if is_false_positive(line):
|
|
continue
|
|
if is_git_sha(match.group(0)):
|
|
continue
|
|
warnings.append(f" {filepath}:{lineno}: {match.group(0)[:40]}...")
|
|
except Exception:
|
|
pass
|
|
|
|
if warnings:
|
|
print("::error::Potential secret found in committed files:")
|
|
for w in warnings:
|
|
print(w)
|
|
sys.exit(1)
|
|
else:
|
|
print("::notice::No secrets detected")
|
|
PYEOF
|
|
# Static-only validator — file existence checks, YAML parse,
|
|
# AST inspection of adapter.py (no import). Doesn't execute any
|
|
# third-party code; safe on fork PRs.
|
|
- run: pip install pyyaml jsonschema packaging -q
|
|
- name: Reject legacy declared-plugin installer
|
|
run: |
|
|
python3 - << 'PYEOF'
|
|
from pathlib import Path
|
|
from packaging.requirements import Requirement
|
|
from packaging.version import Version
|
|
|
|
boot_scripts = [
|
|
path for name in ("entrypoint.sh", "start.sh")
|
|
if (path := Path(name)).exists()
|
|
]
|
|
executable_lines = "\n".join(
|
|
line
|
|
for path in boot_scripts
|
|
for line in path.read_text(encoding="utf-8").splitlines()
|
|
if not line.lstrip().startswith("#")
|
|
)
|
|
forbidden = (
|
|
"MOLECULE_DECLARED_PLUGINS",
|
|
"molecule_runtime.plugin_sources",
|
|
"install_declared_plugins",
|
|
"for _plg_src in $MOLECULE_DECLARED_PLUGINS",
|
|
"/archive/${_plg_ref}.tar.gz",
|
|
"Authorization: token ${MOLECULE_TEMPLATE_REPO_TOKEN}",
|
|
'mkdir -p "/configs/plugins/$_plg_name"',
|
|
'cp -a "$_plg_dir/." "/configs/plugins/$_plg_name/"',
|
|
)
|
|
found = [fragment for fragment in forbidden if fragment in executable_lines]
|
|
if found:
|
|
raise SystemExit(
|
|
"legacy privileged declared-plugin installer detected: "
|
|
+ ", ".join(found)
|
|
)
|
|
|
|
runtime_version = Version(Path(".runtime-version").read_text().strip())
|
|
hardened_version = Version("0.4.0")
|
|
runtime_requirement = next(
|
|
Requirement(line)
|
|
for line in Path("requirements.txt").read_text().splitlines()
|
|
if line.strip().startswith("molecules-workspace-runtime")
|
|
)
|
|
if runtime_version < hardened_version:
|
|
raise SystemExit("runtime pin predates hardened plugin installer")
|
|
if runtime_version not in runtime_requirement.specifier:
|
|
raise SystemExit("runtime pin is outside the template requirement")
|
|
hardened_floor = any(
|
|
specifier.operator in {">=", ">", "==", "===", "~="}
|
|
and Version(specifier.version) >= hardened_version
|
|
for specifier in runtime_requirement.specifier
|
|
)
|
|
if not hardened_floor:
|
|
raise SystemExit("template requirement lacks a hardened runtime floor")
|
|
if Version("0.3.125") in runtime_requirement.specifier:
|
|
raise SystemExit("template requirement permits vulnerable runtime")
|
|
PYEOF
|
|
- run: python3 .molecule-ci-canonical/scripts/validate-workspace-template.py --static-only
|
|
# Audit E1: scripts/molecule-askpass is byte-copied across every workspace
|
|
# template (no shared base image yet). Pin its sha256 so an unintentional
|
|
# edit — or a hand-copy that drifts from its peers — reds this PR. All
|
|
# templates pin the SAME value; a legitimate change must update the file
|
|
# AND this pin in every template together (which a reviewer sees). Always
|
|
# fires (validate-static has no paths filter / fork guard) and feeds the
|
|
# required `validate` aggregate, so it is a hard gate with no secrets.
|
|
- name: askpass helper drift guard (audit E1)
|
|
run: |
|
|
expected=7def3e86a3edf8ff28018081f14a6e390790cb102fd06c27c3f7040b07464303
|
|
test -f scripts/molecule-askpass || { echo "::error::scripts/molecule-askpass is missing"; exit 1; }
|
|
actual=$(sha256sum scripts/molecule-askpass | cut -d' ' -f1)
|
|
if [ "$actual" != "$expected" ]; then
|
|
echo "::error::scripts/molecule-askpass drifted from the pinned cross-template sha256."
|
|
echo " expected $expected"
|
|
echo " actual $actual"
|
|
echo " If this change is intentional, apply it identically to EVERY workspace template and bump this pin in each template's ci.yml."
|
|
exit 1
|
|
fi
|
|
echo "askpass pin OK ($actual)"
|
|
|
|
validate-runtime:
|
|
name: Template validation (runtime)
|
|
# internal#512 follow-on (class defect): pin to `docker-host` so this
|
|
# job (which `docker build`s the template image) lands on Linux
|
|
# docker-host runners. The bare `ubuntu-latest` label is also
|
|
# advertised by hongming-pc-runner-* (Windows act_runner v1.0.3),
|
|
# where `docker build` fails non-deterministically. Mirror of the
|
|
# codex sibling fix (template-codex#9) and internal#512.
|
|
runs-on: docker-host
|
|
timeout-minutes: 15
|
|
needs: validate-static
|
|
steps:
|
|
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
with:
|
|
persist-credentials: false
|
|
- name: Skip executable validation for external forks
|
|
if: github.event.pull_request.head.repo.fork == true
|
|
run: echo "::notice::dependency installation, adapter import, and Docker execution are disabled for external forks"
|
|
- name: Fetch immutable molecule-ci canonical scripts
|
|
if: github.event.pull_request.head.repo.fork != true
|
|
run: |
|
|
set -euo pipefail
|
|
CI_ROOT=".molecule-ci-canonical"
|
|
mkdir "$CI_ROOT"
|
|
git -C "$CI_ROOT" init -q
|
|
git -C "$CI_ROOT" remote add origin https://git.moleculesai.app/molecule-ai/molecule-ci.git
|
|
fetched=false
|
|
for attempt in 1 2 3; do
|
|
if GIT_ASKPASS=/bin/false GIT_TERMINAL_PROMPT=0 git -c credential.helper= -c http.userAgent=curl/8.4.0 -C "$CI_ROOT" fetch --no-tags --depth 1 origin "$MOLECULE_CI_REF"; then
|
|
fetched=true
|
|
break
|
|
fi
|
|
sleep "$attempt"
|
|
done
|
|
if [ "$fetched" != true ]; then
|
|
echo "::error::immutable molecule-ci validator fetch failed"
|
|
exit 1
|
|
fi
|
|
git -C "$CI_ROOT" checkout -q --detach FETCH_HEAD
|
|
ACTUAL_CI_REF="$(git -C "$CI_ROOT" rev-parse HEAD)"
|
|
if [ "$ACTUAL_CI_REF" != "$MOLECULE_CI_REF" ]; then
|
|
echo "::error::immutable molecule-ci validator ref mismatch"
|
|
exit 1
|
|
fi
|
|
- uses: actions/setup-python@v5
|
|
with:
|
|
python-version: "3.11"
|
|
cache: "pip"
|
|
cache-dependency-path: requirements.txt
|
|
- if: github.event.pull_request.head.repo.fork != true
|
|
run: pip install pyyaml jsonschema -q
|
|
# Install the template's runtime dependencies so the validator's
|
|
# check_adapter_runtime_load() can import adapter.py the same way
|
|
# the workspace container does at boot. Without this, a
|
|
# syntactically-valid adapter that ImportErrors on a missing
|
|
# transitive dep would build clean and crash on first user prompt.
|
|
- if: github.event.pull_request.head.repo.fork != true && hashFiles('requirements.txt') != ''
|
|
run: pip install -q --extra-index-url "$MOLECULE_PRIVATE_INDEX" -r requirements.txt
|
|
- if: github.event.pull_request.head.repo.fork != true && hashFiles('requirements.txt') == ''
|
|
run: pip install -q --extra-index-url "$MOLECULE_PRIVATE_INDEX" molecules-workspace-runtime
|
|
- if: github.event.pull_request.head.repo.fork != true
|
|
run: python3 .molecule-ci-canonical/scripts/validate-workspace-template.py
|
|
- name: Docker build smoke test
|
|
if: github.event.pull_request.head.repo.fork != true && hashFiles('Dockerfile') != ''
|
|
run: |
|
|
set -euo pipefail
|
|
# Graceful skip when the runner's job-container can't reach the
|
|
# Docker daemon (e.g. /var/run/docker.sock not mounted into the
|
|
# act job container, or the in-container uid not in the docker
|
|
# group). Without this guard, CI stays red even when the
|
|
# template's Dockerfile is fine — see internal#222 for the
|
|
# proper runner-config fix.
|
|
if ! docker info >/dev/null 2>&1; then
|
|
echo "::warning::docker daemon unreachable from runner job container — skipping Docker build smoke (runner-config gap, not a template issue)."
|
|
exit 0
|
|
fi
|
|
if [ ! -s .runtime-version ]; then
|
|
echo "::error::.runtime-version is missing or empty"
|
|
exit 1
|
|
fi
|
|
if [ -z "${GITHUB_RUN_ID:-}" ] || [ -z "${GITHUB_RUN_ATTEMPT:-}" ]; then
|
|
echo "::error::GITHUB_RUN_ID and GITHUB_RUN_ATTEMPT are required for an isolated Docker tag"
|
|
exit 1
|
|
fi
|
|
EXPECTED_RUNTIME_VERSION="$(tr -d '[:space:]' < .runtime-version)"
|
|
SMOKE_TAG="molecule-ai-workspace-hermes-smoke-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}"
|
|
trap 'docker image rm -f "$SMOKE_TAG" >/dev/null 2>&1 || true' EXIT
|
|
# Redirect, do not pipe: `tail` must not be able to truncate the cause
|
|
# or own the exit status. On failure the WHOLE log is printed -- a fixed
|
|
# window is what made #342 undiagnosable, and buildkit interleaves
|
|
# parallel stages so the failing layer can sit far from the end.
|
|
BUILD_LOG="$(mktemp)"
|
|
rc=0
|
|
docker build --build-arg RUNTIME_VERSION="$EXPECTED_RUNTIME_VERSION" -t "$SMOKE_TAG" . --no-cache >"$BUILD_LOG" 2>&1 || rc=$?
|
|
if [ "$rc" -ne 0 ]; then
|
|
echo "::group::image build output ($(wc -l <"$BUILD_LOG") lines, build failed)"
|
|
cat "$BUILD_LOG"
|
|
echo "::endgroup::"
|
|
echo "::error::image build failed (exit $rc)"
|
|
exit "$rc"
|
|
fi
|
|
tail -5 "$BUILD_LOG"
|
|
ACTUAL_RUNTIME_VERSION="$(
|
|
docker run --rm --entrypoint python3 "$SMOKE_TAG" \
|
|
-c 'from importlib.metadata import version; print(version("molecules-workspace-runtime"))'
|
|
)"
|
|
if [ "$ACTUAL_RUNTIME_VERSION" != "$EXPECTED_RUNTIME_VERSION" ]; then
|
|
echo "::error::image runtime $ACTUAL_RUNTIME_VERSION does not match .runtime-version $EXPECTED_RUNTIME_VERSION"
|
|
exit 1
|
|
fi
|
|
# hermes' terminal tool runs agent commands from a `bash -l` snapshot
|
|
# as the agent user with HOME=/tmp, and Debian's /etc/profile resets
|
|
# PATH there. The bundled node/npx must still resolve in that shell.
|
|
if ! LOGIN_NODE="$(docker run --rm --user 1000:1000 -e HOME=/tmp --entrypoint bash "$SMOKE_TAG" -lc 'command -v node && command -v npx' 2>&1)"; then
|
|
echo "::error::bundled node/npx do not resolve in an agent login shell, so the terminal tool cannot run node: ${LOGIN_NODE}"
|
|
exit 1
|
|
fi
|
|
echo "agent login shell resolves bundled node: ${LOGIN_NODE//$'\n'/ }"
|
|
echo "Docker build succeeded with exact runtime $ACTUAL_RUNTIME_VERSION"
|
|
|
|
# --- Layer-3: real T4 tier-4 conformance gate (RFC internal#456 §11) ---
|
|
# NOT a string-match. Builds the actual image, runs it under the EXACT
|
|
# flags the controlplane provisioner emits for tier-4
|
|
# (userdata_containerized.go @ec2384c: --privileged --pid=host
|
|
# -v /:/host -v /var/run/docker.sock:/var/run/docker.sock), then
|
|
# asserts BOTH properties on the RUNNING container, atomically
|
|
# (RFC §10 — either failing fails the build):
|
|
# (a) the uid-1000 agent can attain host root
|
|
# (sudo nsenter --target 1 --mount --pid -- id -u == 0)
|
|
# (b) the REAL start.sh boot path execs molecule-runtime as uid 1000
|
|
# — the property that determines /configs/.auth_token ownership
|
|
# in SaaS/tenant mode, where the runtime (not the platform)
|
|
# writes the token via save_token(). Asserted by running the
|
|
# real entrypoint (smoke-mode) with a PATH shim that records the
|
|
# terminal-exec uid. NOT a self-chown tautology — fails closed if
|
|
# the `exec gosu agent` (PR#24 b682444) ever regresses to root.
|
|
# The flags are not hard-coded blind: they are the documented
|
|
# provisioner contract; drift is caught because the controlplane
|
|
# string-match unit test (userdata_t4_privileged_test.go) guards the
|
|
# emission side and this gate guards the runtime side. Mirrors the
|
|
# already-live-verified claude-code template gate (12dd604) verbatim.
|
|
t4-conformance:
|
|
name: T4 tier-4 conformance (live)
|
|
# internal#512 follow-on (class defect): pin to `docker-host` so the
|
|
# T4 live probe (`docker build` + `docker run -d` + privileged
|
|
# `docker exec ... sudo -n nsenter --target 1 ...`) lands on the
|
|
# Linux docker-host runners. The bare `ubuntu-latest` label is
|
|
# also advertised by hongming-pc-runner-* (Windows act_runner v1.0.3)
|
|
# where the privileged docker exec chain cannot work. Mirror of
|
|
# template-codex#9 and internal#512.
|
|
runs-on: docker-host
|
|
timeout-minutes: 15
|
|
needs: validate-static
|
|
env:
|
|
MOLECULE_CI_REF: 11b8598e5c0b3f0b1031733a8d5f6bc238f146a4
|
|
# Runs after static validation. Avoid job-level if: guards here so the
|
|
# protected context always reaches a terminal state.
|
|
steps:
|
|
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
with:
|
|
persist-credentials: false
|
|
- name: Skip privileged conformance for external forks
|
|
if: github.event.pull_request.head.repo.fork == true
|
|
run: echo "::notice::privileged T4 validation is disabled for external forks"
|
|
- name: Fetch immutable molecule-ci MCP proof tools
|
|
if: github.event.pull_request.head.repo.fork != true
|
|
run: |
|
|
set -euo pipefail
|
|
: "${RUNNER_TEMP:?RUNNER_TEMP is required}"
|
|
T4_SUFFIX="${GITHUB_RUN_ID:-local}-${GITHUB_RUN_ATTEMPT:-1}"
|
|
CI_ROOT="$RUNNER_TEMP/molecule-ci-mcp-e2e-${T4_SUFFIX}"
|
|
ATTESTATION="$RUNNER_TEMP/mcp-pin-attestation-${T4_SUFFIX}.json"
|
|
ATTESTATION_TMP="${ATTESTATION}.tmp"
|
|
MCP_ATTESTATION_SHA256="${ATTESTATION}.sha256"
|
|
VERIFIER="$CI_ROOT/scripts/mcp_built_image_e2e.py"
|
|
mkdir "$CI_ROOT"
|
|
KEEP_MCP_PROOF=0
|
|
cleanup_mcp_proof_fetch() {
|
|
if [ "$KEEP_MCP_PROOF" -ne 1 ]; then
|
|
rm -rf -- "$CI_ROOT"
|
|
rm -f -- "$ATTESTATION" "$ATTESTATION_TMP" "$MCP_ATTESTATION_SHA256"
|
|
fi
|
|
}
|
|
trap cleanup_mcp_proof_fetch EXIT
|
|
git init -q "$CI_ROOT"
|
|
git -C "$CI_ROOT" remote add origin \
|
|
https://git.moleculesai.app/molecule-ai/molecule-ci.git
|
|
fetched=false
|
|
for attempt in 1 2 3; do
|
|
if GIT_ASKPASS=/bin/false GIT_TERMINAL_PROMPT=0 \
|
|
git -c credential.helper= -c http.userAgent=curl/8.4.0 \
|
|
-C "$CI_ROOT" fetch -q --no-tags --depth 1 origin "$MOLECULE_CI_REF"; then
|
|
fetched=true
|
|
break
|
|
fi
|
|
sleep "$attempt"
|
|
done
|
|
if [ "$fetched" != true ]; then
|
|
echo "::error::immutable molecule-ci verifier fetch failed"
|
|
exit 1
|
|
fi
|
|
git -C "$CI_ROOT" checkout -q --detach FETCH_HEAD
|
|
test "$(git -C "$CI_ROOT" rev-parse HEAD)" = "$MOLECULE_CI_REF"
|
|
test -f "$CI_ROOT/scripts/mcp_pin_lockstep.py"
|
|
test ! -L "$CI_ROOT/scripts/mcp_pin_lockstep.py"
|
|
test -f "$VERIFIER"
|
|
test ! -L "$VERIFIER"
|
|
git -C "$CI_ROOT" diff --quiet --no-ext-diff --no-textconv "$MOLECULE_CI_REF" -- scripts/mcp_pin_lockstep.py scripts/mcp_built_image_e2e.py
|
|
python3 "$CI_ROOT/scripts/mcp_pin_lockstep.py" --repo-root . --json \
|
|
> "$ATTESTATION_TMP"
|
|
test -s "$ATTESTATION_TMP"
|
|
mv "$ATTESTATION_TMP" "$ATTESTATION"
|
|
sha256sum "$ATTESTATION" > "$MCP_ATTESTATION_SHA256"
|
|
KEEP_MCP_PROOF=1
|
|
- name: Build the runtime image
|
|
id: build
|
|
if: github.event.pull_request.head.repo.fork != true
|
|
run: |
|
|
set -euo pipefail
|
|
: "${RUNNER_TEMP:?RUNNER_TEMP is required}"
|
|
T4_SUFFIX="${GITHUB_RUN_ID:-local}-${GITHUB_RUN_ATTEMPT:-1}"
|
|
CI_ROOT="$RUNNER_TEMP/molecule-ci-mcp-e2e-${T4_SUFFIX}"
|
|
ATTESTATION="$RUNNER_TEMP/mcp-pin-attestation-${T4_SUFFIX}.json"
|
|
MCP_ATTESTATION_SHA256="${ATTESTATION}.sha256"
|
|
VERIFIER="$CI_ROOT/scripts/mcp_built_image_e2e.py"
|
|
T4_TAG="t4-conformance-test:${GITHUB_RUN_ID:-local}-${GITHUB_RUN_ATTEMPT:-1}"
|
|
KEEP_T4_IMAGE=0
|
|
cleanup_t4_build() {
|
|
if [ "$KEEP_T4_IMAGE" -ne 1 ]; then
|
|
docker image rm -f "$T4_TAG" >/dev/null 2>&1 || true
|
|
rm -rf -- "$CI_ROOT"
|
|
rm -f -- "$ATTESTATION" "$MCP_ATTESTATION_SHA256"
|
|
fi
|
|
}
|
|
trap cleanup_t4_build EXIT
|
|
if ! docker info >/dev/null 2>&1; then
|
|
echo "::error::docker daemon unreachable — T4 conformance gate CANNOT verify host-root reach. This is a hard gate; failing closed (do NOT treat as skip). Fix runner-config (internal#222) to unblock."
|
|
exit 1
|
|
fi
|
|
# Per-run-unique image tag + container name. This repo triggers
|
|
# `on: [push, pull_request]`, so a single push to an internal PR
|
|
# branch fires BOTH a `push` run AND a `pull_request` run for the
|
|
# SAME commit. The runners share one host Docker daemon
|
|
# (--network host + /var/run/docker.sock bind), so a STATIC
|
|
# `--name t4probe` / `-t t4-conformance-test` collides across the
|
|
# two concurrent same-commit runs: whichever event creates the
|
|
# container first wins, the other dies with
|
|
# "docker: Error response from daemon: Conflict. The container
|
|
# name "/t4probe" is already in use ..." (exit 125)
|
|
# — a FALSE NEGATIVE (the probe never runs; the build that wins
|
|
# the race passes all assertions). Scope every host-daemon object
|
|
# to GITHUB_RUN_ID + GITHUB_RUN_ATTEMPT so push and pull_request
|
|
# runs of the same SHA are isolated. This does NOT weaken the
|
|
# gate — the same privileged container, the same real start.sh
|
|
# uid probe, and the same host-root assertions still run, just
|
|
# under collision-proof names. (RFC internal#456 §11;
|
|
# feedback_concurrency_group_per_sha — unique-name is the
|
|
# fail-closed primitive, concurrency is only de-dup.)
|
|
# Deterministic function of run-id/attempt — recomputed
|
|
# identically in the next step (no cross-step $GITHUB_ENV
|
|
# round-trip, which can be fragile under act_runner).
|
|
if [ ! -s "$ATTESTATION" ]; then
|
|
echo "::error::immutable MCP attestation is missing or empty"
|
|
exit 1
|
|
fi
|
|
test -f "$VERIFIER"
|
|
test ! -L "$VERIFIER"
|
|
git -C "$CI_ROOT" diff --quiet --no-ext-diff --no-textconv "$MOLECULE_CI_REF" -- scripts/mcp_pin_lockstep.py scripts/mcp_built_image_e2e.py
|
|
sha256sum --check "$MCP_ATTESTATION_SHA256"
|
|
EXPECTED_RUNTIME_VERSION="$(
|
|
python3 - "$CI_ROOT" "$ATTESTATION" <<'PY'
|
|
import sys
|
|
from pathlib import Path
|
|
|
|
sys.path.insert(0, str(Path(sys.argv[1]) / "scripts"))
|
|
from mcp_built_image_e2e import load_attestation
|
|
|
|
with Path(sys.argv[2]).open("rb") as stream:
|
|
sys.stdout.write(load_attestation(stream).runtime_version)
|
|
PY
|
|
)"
|
|
# Redirect, do not pipe: `tail` must not be able to truncate the cause
|
|
# or own the exit status. On failure the WHOLE log is printed -- a fixed
|
|
# window is what made #342 undiagnosable, and buildkit interleaves
|
|
# parallel stages so the failing layer can sit far from the end.
|
|
BUILD_LOG="$(mktemp)"
|
|
rc=0
|
|
docker build --build-arg RUNTIME_VERSION="$EXPECTED_RUNTIME_VERSION" -t "$T4_TAG" . --no-cache >"$BUILD_LOG" 2>&1 || rc=$?
|
|
if [ "$rc" -ne 0 ]; then
|
|
echo "::group::image build output ($(wc -l <"$BUILD_LOG") lines, build failed)"
|
|
cat "$BUILD_LOG"
|
|
echo "::endgroup::"
|
|
echo "::error::image build failed (exit $rc)"
|
|
exit "$rc"
|
|
fi
|
|
tail -5 "$BUILD_LOG"
|
|
ACTUAL_RUNTIME_VERSION="$(
|
|
docker run --rm --entrypoint python3 "$T4_TAG" \
|
|
-c 'from importlib.metadata import version; print(version("molecules-workspace-runtime"))'
|
|
)"
|
|
if [ "$ACTUAL_RUNTIME_VERSION" != "$EXPECTED_RUNTIME_VERSION" ]; then
|
|
echo "::error::T4 image runtime $ACTUAL_RUNTIME_VERSION does not match attested runtime $EXPECTED_RUNTIME_VERSION"
|
|
exit 1
|
|
fi
|
|
KEEP_T4_IMAGE=1
|
|
- name: Verify management MCP in the final image
|
|
if: github.event.pull_request.head.repo.fork != true
|
|
run: |
|
|
set -euo pipefail
|
|
: "${RUNNER_TEMP:?RUNNER_TEMP is required}"
|
|
T4_SUFFIX="${GITHUB_RUN_ID:-local}-${GITHUB_RUN_ATTEMPT:-1}"
|
|
CI_ROOT="$RUNNER_TEMP/molecule-ci-mcp-e2e-${T4_SUFFIX}"
|
|
ATTESTATION="$RUNNER_TEMP/mcp-pin-attestation-${T4_SUFFIX}.json"
|
|
MCP_ATTESTATION_SHA256="${ATTESTATION}.sha256"
|
|
VERIFIER="$CI_ROOT/scripts/mcp_built_image_e2e.py"
|
|
MCP_E2E_LOG="$RUNNER_TEMP/mcp-built-image-e2e-${T4_SUFFIX}.log"
|
|
T4_TAG="t4-conformance-test:${GITHUB_RUN_ID:-local}-${GITHUB_RUN_ATTEMPT:-1}"
|
|
MCP_VERIFY_CONTAINER="mcp-built-image-e2e-${GITHUB_RUN_ID:-local}-${GITHUB_RUN_ATTEMPT:-1}"
|
|
KEEP_T4_IMAGE=0
|
|
cleanup_mcp_e2e() {
|
|
docker rm -f "$MCP_VERIFY_CONTAINER" >/dev/null 2>&1 || true
|
|
rm -rf -- "$CI_ROOT"
|
|
rm -f -- "$ATTESTATION" "$MCP_ATTESTATION_SHA256" "$MCP_E2E_LOG"
|
|
if [ "$KEEP_T4_IMAGE" -ne 1 ]; then
|
|
docker image rm -f "$T4_TAG" >/dev/null 2>&1 || true
|
|
fi
|
|
}
|
|
trap cleanup_mcp_e2e EXIT
|
|
test -s "$ATTESTATION"
|
|
test -f "$VERIFIER"
|
|
test ! -L "$VERIFIER"
|
|
docker rm -f "$MCP_VERIFY_CONTAINER" >/dev/null 2>&1 || true
|
|
docker create --interactive --name "$MCP_VERIFY_CONTAINER" \
|
|
--network none \
|
|
--user 1000:1000 --workdir /tmp \
|
|
--cap-drop ALL --security-opt no-new-privileges \
|
|
--pids-limit 128 --memory 768m --cpus 1 \
|
|
--tmpfs /tmp:size=64m \
|
|
--env MOLECULE_PREBAKE_NODE_BIN=/home/agent/.hermes/node/bin \
|
|
--entrypoint python3 "$T4_TAG" \
|
|
/mcp_built_image_e2e.py >/dev/null
|
|
git -C "$CI_ROOT" diff --quiet --no-ext-diff --no-textconv "$MOLECULE_CI_REF" -- scripts/mcp_pin_lockstep.py scripts/mcp_built_image_e2e.py
|
|
docker cp "$VERIFIER" \
|
|
"$MCP_VERIFY_CONTAINER:/mcp_built_image_e2e.py"
|
|
sha256sum --check "$MCP_ATTESTATION_SHA256"
|
|
docker start --attach --interactive "$MCP_VERIFY_CONTAINER" \
|
|
< "$ATTESTATION" \
|
|
| tee "$MCP_E2E_LOG"
|
|
docker rm "$MCP_VERIFY_CONTAINER" >/dev/null
|
|
grep -qxF 'mcp-built-image-e2e:sentinel:executed' "$MCP_E2E_LOG"
|
|
KEEP_T4_IMAGE=1
|
|
- name: Run under EXACT tier-4 provisioner flags + assert host-root reach AND token agent-ownership
|
|
if: github.event.pull_request.head.repo.fork != true
|
|
run: |
|
|
set -euo pipefail
|
|
# EXACT flags from controlplane userdata_containerized.go
|
|
# (tier-4 emission @ec2384c). The Hermes start.sh entrypoint
|
|
# boots the hermes gateway + molecule-runtime and wants live
|
|
# creds; we only need the container up long enough to probe,
|
|
# so override the entrypoint with a sleep (same approach as
|
|
# the claude-code gate) and exercise the agent context directly.
|
|
# Per-run-unique container name (see Build step rationale): the
|
|
# shared host Docker daemon is concurrently used by the SAME
|
|
# commit's push + pull_request runs. A static name collides and
|
|
# false-negatives. T4_PROBE is unique per run-id/attempt.
|
|
# Recompute the SAME deterministic tag the build step used
|
|
# (pure function of run-id/attempt — no cross-step env reliance).
|
|
T4_TAG="t4-conformance-test:${GITHUB_RUN_ID:-local}-${GITHUB_RUN_ATTEMPT:-1}"
|
|
T4_PROBE="t4probe-${GITHUB_RUN_ID:-local}-${GITHUB_RUN_ATTEMPT:-1}"
|
|
CAPLESS_PROBE="capless-liveness-${GITHUB_RUN_ID:-local}-${GITHUB_RUN_ATTEMPT:-1}"
|
|
trap 'docker rm -f "$CAPLESS_PROBE" "$T4_PROBE" >/dev/null 2>&1 || true; docker rmi -f "$T4_TAG" >/dev/null 2>&1 || true' EXIT
|
|
|
|
echo "=== capability-independent child liveness under the local workspace profile ==="
|
|
docker rm -f "$CAPLESS_PROBE" >/dev/null 2>&1 || true
|
|
docker run --rm \
|
|
--name "$CAPLESS_PROBE" \
|
|
--cap-drop ALL \
|
|
--cap-add CHOWN --cap-add DAC_OVERRIDE --cap-add FOWNER \
|
|
--cap-add SETGID --cap-add SETUID \
|
|
--security-opt no-new-privileges \
|
|
--pids-limit 512 \
|
|
--entrypoint /bin/bash \
|
|
"$T4_TAG" -c '
|
|
set -euo pipefail
|
|
. /app/scripts/process-liveness.sh
|
|
gosu agent timeout 2s sleep 30 &
|
|
child_pid=$!
|
|
sleep 0.2
|
|
process_is_running "$child_pid" || {
|
|
echo "capability-independent liveness falsely reported the uid-1000 child dead" >&2
|
|
exit 1
|
|
}
|
|
set +e
|
|
wait "$child_pid"
|
|
child_rc=$?
|
|
set -e
|
|
[ "$child_rc" -eq 124 ] || {
|
|
echo "unexpected bounded child exit: $child_rc" >&2
|
|
exit 1
|
|
}
|
|
if process_is_running "$child_pid"; then
|
|
echo "capability-independent liveness reported the exited child alive" >&2
|
|
exit 1
|
|
fi
|
|
'
|
|
|
|
# Defensive pre-removal of a LEAKED container from a crashed
|
|
# prior attempt of THIS SAME run-id only (the name is run-id
|
|
# scoped, so this can never reap a concurrent foreign run's
|
|
# probe). This is cleanup of our own debris, NOT a gate bypass —
|
|
# no `|| true` on the real assertions below; failure still fails.
|
|
docker rm -f "$T4_PROBE" >/dev/null 2>&1 || true
|
|
docker run -d \
|
|
--name "$T4_PROBE" \
|
|
--network host \
|
|
--privileged \
|
|
--pid=host \
|
|
-v /:/host \
|
|
-v /var/run/docker.sock:/var/run/docker.sock \
|
|
--entrypoint /bin/sh \
|
|
"$T4_TAG" -c 'sleep 600' >/dev/null
|
|
|
|
echo "=== (b) token agent-ownership: assert the REAL start.sh boot path execs molecule-runtime as uid 1000 ==="
|
|
# ROOT-CAUSE-ACCURATE assertion (RFC internal#456 §10, Hermes
|
|
# list_peers-401 class). In SaaS/tenant mode (prod Hermes,
|
|
# ws-tenant-*) the platform does NOT pre-seed /configs/.auth_token
|
|
# — workspace_provision.go issueAndInjectToken() returns early in
|
|
# saasMode(), there is no #1877 volume pre-write and no #418 tar
|
|
# re-injection. The token is delivered in the /registry/register
|
|
# response body and persisted IN-CONTAINER by the runtime itself
|
|
# via platform_auth.save_token() (main.py:364), which does
|
|
# os.open(.auth_token, O_CREAT, 0o600) — the file lands owned by
|
|
# WHATEVER UID molecule-runtime runs as. The pinned PR#23 image
|
|
# (git 66b7565) ran `exec molecule-runtime` as ROOT → token
|
|
# root:root 600 → uid-1000 a2a_mcp_server EACCES → empty bearer →
|
|
# HTTP 401 → list_peers "No peers found". PR#24 (b682444) fixed it
|
|
# by changing the terminal exec to `exec gosu agent ...
|
|
# molecule-runtime` (uid 1000 → token agent-owned), exactly the
|
|
# claude-code proven pattern.
|
|
#
|
|
# The previous version of this step `touch`ed + `chown`ed the
|
|
# token ITSELF and then stat'd it — a tautology that proved
|
|
# nothing about the boot path and would stay GREEN if the gosu
|
|
# exec ever regressed back to root. Instead, run the REAL
|
|
# entrypoint (smoke-mode short-circuits the gateway/creds) with a
|
|
# shim on PATH in place of molecule-runtime that records the uid
|
|
# the terminal `exec` actually confers, then assert it is 1000.
|
|
# This fails CLOSED on any future revert of the `gosu agent` exec
|
|
# — the property that genuinely prevents the 401.
|
|
# Build the probe script on the host (no nested-quote fragility),
|
|
# copy it in, run it. The shim replaces `molecule-runtime` on
|
|
# PATH; start.sh's terminal `exec gosu agent env HOME=/tmp
|
|
# CONFIGS_DIR=/configs molecule-runtime` resolves to it (the
|
|
# `env` form preserves PATH — it is not `env -i`), so the shim
|
|
# records the uid the real terminal exec confers. Smoke mode
|
|
# makes start.sh take its line-23 short-circuit (same `exec gosu
|
|
# agent ... molecule-runtime` codepath, no gateway/creds needed).
|
|
cat > t4_token_uid_probe.sh <<'PROBE'
|
|
#!/bin/sh
|
|
set -e
|
|
mkdir -p /shim
|
|
printf '%s\n' '#!/bin/sh' 'id -u > /shim/runtime_uid' > /shim/molecule-runtime
|
|
chmod +x /shim/molecule-runtime
|
|
rm -f /shim/runtime_uid
|
|
chown -R agent:agent /shim
|
|
MOLECULE_SMOKE_MODE=1 PATH="/shim:$PATH" /usr/local/bin/start.sh > /shim/start.log 2>&1 || true
|
|
PROBE
|
|
docker cp t4_token_uid_probe.sh "$T4_PROBE":/shim_probe.sh
|
|
docker exec "$T4_PROBE" sh /shim_probe.sh
|
|
RUNTIME_UID=$(docker exec "$T4_PROBE" cat /shim/runtime_uid 2>/dev/null || echo MISSING)
|
|
echo "molecule-runtime terminal-exec uid = $RUNTIME_UID"
|
|
if [ "$RUNTIME_UID" != "1000" ]; then
|
|
echo "::error::T4 contract violated: start.sh execs molecule-runtime as uid=$RUNTIME_UID (expected 1000). In SaaS mode platform_auth.save_token() then writes /configs/.auth_token as that uid; uid!=1000 re-ships the Hermes list_peers-401 class (RFC internal#456 §10). The 'exec gosu agent ... molecule-runtime' (PR#24 b682444) must NOT regress."
|
|
docker exec "$T4_PROBE" sh -c 'tail -20 /shim/start.log' || true
|
|
exit 1
|
|
fi
|
|
|
|
echo "=== (a) host-root reach AS THE uid-1000 AGENT (not root) ==="
|
|
# Run as the agent user (uid 1000), exactly as gosu would.
|
|
AGENT_HOSTROOT_UID=$(docker exec -u agent "$T4_PROBE" sudo -n nsenter --target 1 --mount --pid -- id -u)
|
|
echo "agent->host-root id -u = $AGENT_HOSTROOT_UID"
|
|
if [ "$AGENT_HOSTROOT_UID" != "0" ]; then
|
|
echo "::error::T4 contract violated: uid-1000 agent could NOT attain host root via 'sudo nsenter --target 1' (got uid=$AGENT_HOSTROOT_UID). T4 escalation leg ABSENT/broken."
|
|
exit 1
|
|
fi
|
|
# Defense-in-depth: host-filesystem write+readback through /host
|
|
# from the agent, proving real host reach (not just a namespace
|
|
# trick on an isolated PID 1). The probe file path is ALSO
|
|
# per-run-unique: /host maps to the SHARED host /, so a static
|
|
# /host/tmp/.t4-conformance-probe would let the concurrent
|
|
# same-commit push+pull_request runs clobber/rm each other's
|
|
# marker mid-readback (a second false-negative of the same
|
|
# class). Run-id scoping keeps each run asserting exactly the
|
|
# marker it wrote — still fully fail-closed.
|
|
PROBE_FILE="/host/tmp/.t4-conformance-probe-${GITHUB_RUN_ID:-local}-${GITHUB_RUN_ATTEMPT:-1}"
|
|
MARKER="t4-conformance-$(date +%s)-$RANDOM"
|
|
docker exec -u agent "$T4_PROBE" sudo -n sh -c "echo $MARKER > $PROBE_FILE"
|
|
READBACK=$(docker exec -u agent "$T4_PROBE" sudo -n cat "$PROBE_FILE")
|
|
docker exec -u agent "$T4_PROBE" sudo -n rm -f "$PROBE_FILE"
|
|
if [ "$READBACK" != "$MARKER" ]; then
|
|
echo "::error::T4 host-fs write+readback through /host failed (got '$READBACK' expected '$MARKER')."
|
|
exit 1
|
|
fi
|
|
echo "::notice::T4 tier-4 conformance PASS — uid-1000 agent reaches host root AND start.sh execs molecule-runtime as uid 1000 (so SaaS-mode save_token() writes /configs/.auth_token agent-owned; list_peers-401 class closed), both atomically."
|
|
|
|
# Aggregator that emits a single `validate` check name — matches the
|
|
# historical required-check name on this repo's branch protection.
|
|
validate:
|
|
name: validate
|
|
runs-on: ubuntu-latest
|
|
needs: [validate-static, validate-runtime, t4-conformance, conformance, shell-tests, gateway-live-roundtrip]
|
|
timeout-minutes: 1
|
|
steps:
|
|
- name: Aggregate
|
|
run: |
|
|
static="${{ needs.validate-static.result }}"
|
|
runtime="${{ needs.validate-runtime.result }}"
|
|
t4="${{ needs.t4-conformance.result }}"
|
|
conformance="${{ needs.conformance.result }}"
|
|
shell="${{ needs.shell-tests.result }}"
|
|
roundtrip="${{ needs.gateway-live-roundtrip.result }}"
|
|
echo "validate-static: $static"
|
|
echo "validate-runtime: $runtime"
|
|
echo "t4-conformance: $t4"
|
|
echo "conformance: $conformance"
|
|
echo "shell-tests: $shell"
|
|
echo "gateway-roundtrip:$roundtrip"
|
|
if [ "$static" != "success" ]; then
|
|
echo "::error::validate-static did not succeed: $static"
|
|
exit 1
|
|
fi
|
|
# Preserve compatibility with older runs that skipped runtime validation.
|
|
if [ "$runtime" != "success" ] && [ "$runtime" != "skipped" ]; then
|
|
echo "::error::validate-runtime did not succeed: $runtime"
|
|
exit 1
|
|
fi
|
|
is_fork_pr="${{ github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == true }}"
|
|
# Gateway live round-trip — explicit hard gate here too (review
|
|
# wf_3a7b849d #13: this aggregate assigned $roundtrip but never
|
|
# checked it, so a future `if: always()` on validate would silently
|
|
# drop the gate from the historical required-check). `skipped` only
|
|
# on fork PRs (the job self-skips there, #12).
|
|
if [ "$roundtrip" != "success" ]; then
|
|
if [ "$roundtrip" = "skipped" ] && [ "$is_fork_pr" = "true" ]; then
|
|
echo "::notice::gateway-live-roundtrip skipped on fork PR — allowing validate to pass."
|
|
else
|
|
echo "::error::gateway-live-roundtrip did not succeed: $roundtrip"
|
|
exit 1
|
|
fi
|
|
fi
|
|
# T4 conformance is a HARD gate on internal (non-fork) PRs
|
|
# and main pushes. `skipped` is only acceptable on fork PRs.
|
|
# Any other non-success fails the build: "verified" T4 requires
|
|
# this live gate green, never inference.
|
|
if [ "$t4" != "success" ]; then
|
|
if [ "$t4" = "skipped" ] && [ "$is_fork_pr" = "true" ]; then
|
|
echo "::notice::t4-conformance skipped on fork PR — allowing aggregate to pass."
|
|
else
|
|
echo "::error::t4-conformance did not succeed: $t4 — T4 host-root reach / token-ownership not verified on a live container. Failing closed (RFC internal#456 §11)."
|
|
exit 1
|
|
fi
|
|
fi
|
|
# ADR-004 §4 adapter-socket conformance is a HARD gate: the SDK-owned
|
|
# AdapterConformance battery (tests/test_conformance.py) must pass on
|
|
# every push/PR so a hermes-only socket regression can't ship green
|
|
# (audit finding CI2). No fork carve-out — it runs no untrusted code.
|
|
if [ "$conformance" != "success" ]; then
|
|
echo "::error::conformance did not succeed: $conformance — ADR-004 adapter-socket conformance not green (audit CI2)."
|
|
exit 1
|
|
fi
|
|
if [ "$shell" != "success" ] && [ "$shell" != "skipped" ]; then
|
|
echo "::error::shell-tests did not succeed: $shell"
|
|
exit 1
|
|
fi
|
|
echo "::notice::Template validation aggregate passed (static=$static, runtime=$runtime, t4=$t4, conformance=$conformance, shell=$shell)"
|
|
|
|
# New single-sentinel context for branch protection (internal#349).
|
|
# Mirrors the existing `validate` aggregate but posts under the standardized
|
|
# `all-required` name so the org-wide hard-gate can require one context per
|
|
# repo without enumerating every individual CI job.
|
|
all-required:
|
|
name: all-required
|
|
runs-on: ubuntu-latest
|
|
needs: [validate-static, validate-runtime, t4-conformance, conformance, shell-tests, gateway-live-roundtrip]
|
|
if: always()
|
|
timeout-minutes: 1
|
|
steps:
|
|
- name: Verify all required jobs passed
|
|
run: |
|
|
static="${{ needs.validate-static.result }}"
|
|
runtime="${{ needs.validate-runtime.result }}"
|
|
t4="${{ needs.t4-conformance.result }}"
|
|
conformance="${{ needs.conformance.result }}"
|
|
shell="${{ needs.shell-tests.result }}"
|
|
roundtrip="${{ needs.gateway-live-roundtrip.result }}"
|
|
echo "validate-static: $static"
|
|
echo "validate-runtime: $runtime"
|
|
echo "t4-conformance: $t4"
|
|
echo "conformance: $conformance"
|
|
echo "shell-tests: $shell"
|
|
if [ "$static" != "success" ]; then
|
|
echo "::error::validate-static did not succeed: $static"
|
|
exit 1
|
|
fi
|
|
if [ "$runtime" != "success" ] && [ "$runtime" != "skipped" ]; then
|
|
echo "::error::validate-runtime did not succeed: $runtime"
|
|
exit 1
|
|
fi
|
|
# T4 conformance is a HARD gate on internal (non-fork) PRs
|
|
# and main pushes. `skipped` is only acceptable on fork PRs.
|
|
# Any other non-success fails the build: "verified" T4 requires
|
|
# this live gate green, never inference.
|
|
is_fork_pr="${{ github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == true }}"
|
|
if [ "$t4" != "success" ]; then
|
|
if [ "$t4" = "skipped" ] && [ "$is_fork_pr" = "true" ]; then
|
|
echo "::notice::t4-conformance skipped on fork PR — allowing all-required to pass."
|
|
else
|
|
echo "::error::t4-conformance did not succeed: $t4 — T4 host-root reach / token-ownership not verified on a live container. Failing closed (RFC internal#456 §11)."
|
|
exit 1
|
|
fi
|
|
fi
|
|
# Gateway live round-trip — HARD gate (2026-07-23 hermes-0.19
|
|
# incident: four behavior changes shipped green because no job
|
|
# booted the real pinned gateway with the plugin and traffic).
|
|
# `skipped` is only acceptable on fork PRs (its steps no-op there).
|
|
if [ "$roundtrip" != "success" ]; then
|
|
if [ "$roundtrip" = "skipped" ] && [ "$is_fork_pr" = "true" ]; then
|
|
echo "::notice::gateway-live-roundtrip skipped on fork PR — allowing all-required to pass."
|
|
else
|
|
echo "::error::gateway-live-roundtrip did not succeed: $roundtrip — the pinned hermes gateway + a2a plugin round-trip is not verified."
|
|
exit 1
|
|
fi
|
|
fi
|
|
# ADR-004 §4 adapter-socket conformance — HARD gate (audit CI2). No
|
|
# fork carve-out: it executes no untrusted code, so it runs the same
|
|
# on fork PRs, internal PRs and main pushes.
|
|
if [ "$conformance" != "success" ]; then
|
|
echo "::error::conformance did not succeed: $conformance — ADR-004 adapter-socket conformance not green (audit CI2)."
|
|
exit 1
|
|
fi
|
|
if [ "$shell" != "success" ] && [ "$shell" != "skipped" ]; then
|
|
echo "::error::shell-tests did not succeed: $shell"
|
|
exit 1
|
|
fi
|
|
echo "::notice::All required CI jobs passed (static=$static, runtime=$runtime, t4=$t4, conformance=$conformance, shell=$shell)"
|
|
|
|
gateway-live-roundtrip:
|
|
# The gate the 2026-07-23 hermes-0.19 incident demanded: boot the REAL
|
|
# pinned hermes gateway (installed at the Dockerfile's HERMES_COMMIT,
|
|
# so every upstream pin bump exercises the NEW ref pre-merge)
|
|
# with the REAL a2a plugin (at HERMES_PLATFORM_MOLECULE_A2A_REF) and
|
|
# run scripts/e2e_full_chain.py: executor -> /a2a/inbound -> gateway
|
|
# pipeline -> stub LLM -> plugin send() -> executor callback, TWICE
|
|
# (the second message guards the 0.19 pairing-policy active-session
|
|
# drop). Catches, per-PR, all four break classes that shipped green on
|
|
# 2026-07-22/23: plugins.enabled opt-in, adapter ABI drift
|
|
# (connect kwarg), authz drops, and — via the shell unit test in
|
|
# shell-tests — the eager-MCP reconcile mechanics. No docker needed:
|
|
# the gateway runs as a subprocess against a temp HERMES_HOME.
|
|
name: Gateway live round-trip (pinned hermes)
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 12
|
|
needs: validate-static
|
|
# review wf_3a7b849d #12: skip the WHOLE job on fork PRs so it reports
|
|
# result="skipped" (not "success" from no-op'd steps) to the aggregates —
|
|
# otherwise all-required's `roundtrip == skipped` fork carve-out is
|
|
# unreachable and the check log falsely claims a fork's pinned round-trip
|
|
# was verified. Runs on internal PRs (fork=false) and pushes (no PR object).
|
|
if: ${{ github.event_name != 'pull_request' || github.event.pull_request.head.repo.fork != true }}
|
|
steps:
|
|
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
with:
|
|
persist-credentials: false
|
|
- name: Skip executable round-trip for external forks
|
|
if: github.event.pull_request.head.repo.fork == true
|
|
run: echo "::notice::gateway live round-trip is disabled for external forks"
|
|
- uses: actions/setup-python@v5
|
|
with:
|
|
python-version: "3.11"
|
|
- name: Install pinned hermes + a2a plugin + runtime deps
|
|
if: github.event.pull_request.head.repo.fork != true
|
|
run: |
|
|
set -euo pipefail
|
|
HERMES_COMMIT=$(sed -n 's/^ARG HERMES_COMMIT=//p' Dockerfile | head -1)
|
|
A2A_REF=$(sed -n 's/^ARG HERMES_PLATFORM_MOLECULE_A2A_REF=//p' Dockerfile | head -1)
|
|
# review wf_3a7b849d #6: pin the runtime to .runtime-version, the
|
|
# EXACT version the shipped image is built and asserted against
|
|
# (docker build --build-arg RUNTIME_VERSION + the importlib equality
|
|
# check). Installing it unpinned made this "pinned stack" gate resolve
|
|
# a DIFFERENT runtime than the deliverable — so a prepare-dependent
|
|
# break in the pinned 0.4.x could ship green, and an unrelated newer
|
|
# release could red every template PR.
|
|
RUNTIME_VERSION=$(tr -d '[:space:]' < .runtime-version)
|
|
test -n "$HERMES_COMMIT" && test -n "$A2A_REF" && test -n "$RUNTIME_VERSION"
|
|
echo "pinned hermes-agent@$HERMES_COMMIT a2a-plugin@$A2A_REF runtime==$RUNTIME_VERSION"
|
|
# Install from the SAME immutable git ref the image pins (2026-09-03).
|
|
# It used to be `hermes-agent==$HERMES_VERSION` from PyPI; that pin
|
|
# named the wheel in site-packages, which the shipped image does not
|
|
# import (~/.local/bin/hermes puts the checkout at sys.path[0]) — so
|
|
# this gate was conformance-testing code the fleet does not run.
|
|
#
|
|
# It must be an EDITABLE install from a checkout, not `pip install
|
|
# <git-ref>`: upstream's build backend refuses to build a wheel or
|
|
# sdist at all ("Building wheels or sdists for hermes-agent is not
|
|
# supported. Hermes is distributed via the shell installer, Docker
|
|
# image, or Nix"), so the package form fails outright. This mirrors
|
|
# what install.sh does in the image (`uv pip install -e .` from the
|
|
# pinned checkout), which is also why the image was never affected.
|
|
git clone -q --filter=blob:none --no-checkout \
|
|
https://github.com/NousResearch/hermes-agent.git /tmp/hermes-src
|
|
git -C /tmp/hermes-src checkout -q --detach "$HERMES_COMMIT"
|
|
# Same reasoning as the Dockerfile's assertion: verify the ref rather
|
|
# than trust the fetch, so this gate cannot silently test other code.
|
|
ACTUAL_COMMIT="$(git -C /tmp/hermes-src rev-parse HEAD)"
|
|
if [ "$ACTUAL_COMMIT" != "$HERMES_COMMIT" ]; then
|
|
echo "::error::hermes checkout is at $ACTUAL_COMMIT, expected $HERMES_COMMIT"
|
|
exit 1
|
|
fi
|
|
pip install -q -e /tmp/hermes-src
|
|
pip install -q aiohttp
|
|
pip install -q "git+https://git.moleculesai.app/molecule-ai/hermes-platform-molecule-a2a.git@${A2A_REF}#egg=hermes-platform-molecule-a2a"
|
|
pip install -q --extra-index-url "$MOLECULE_PRIVATE_INDEX" "molecules-workspace-runtime==$RUNTIME_VERSION"
|
|
# The image build runs vendor/hermes-agent/install.sh instead of
|
|
# downloading it (Dockerfile, above ARG HERMES_COMMIT). The upstream
|
|
# clone from the previous step is at HERMES_COMMIT, so compare git blob
|
|
# ids: equal ids mean byte-identical files. A HERMES_COMMIT bump that
|
|
# does not refresh the vendored copy fails here.
|
|
- name: Vendored installer is upstream scripts/install.sh at HERMES_COMMIT
|
|
if: github.event.pull_request.head.repo.fork != true
|
|
run: |
|
|
set -euo pipefail
|
|
HERMES_COMMIT=$(sed -n 's/^ARG HERMES_COMMIT=//p' Dockerfile | head -1)
|
|
test -n "$HERMES_COMMIT"
|
|
UPSTREAM_BLOB="$(git -C /tmp/hermes-src rev-parse "$HERMES_COMMIT:scripts/install.sh")"
|
|
VENDORED_BLOB="$(git hash-object vendor/hermes-agent/install.sh)"
|
|
if [ "$UPSTREAM_BLOB" != "$VENDORED_BLOB" ]; then
|
|
echo "::error::vendor/hermes-agent/install.sh is blob $VENDORED_BLOB, but scripts/install.sh at $HERMES_COMMIT is blob $UPSTREAM_BLOB. Refresh the copy (vendor/hermes-agent/README.md)."
|
|
exit 1
|
|
fi
|
|
echo "vendored installer == upstream scripts/install.sh at $HERMES_COMMIT (blob $VENDORED_BLOB)"
|
|
- name: Run the full-chain round-trip against the pinned gateway
|
|
if: github.event.pull_request.head.repo.fork != true
|
|
run: |
|
|
set -euo pipefail
|
|
HERMES_BIN="$(command -v hermes)"
|
|
export HERMES_BIN
|
|
python scripts/e2e_full_chain.py
|
|
|
|
shell-tests:
|
|
# Shell-level unit tests for scripts/*.sh. Keep every helper test wired
|
|
# into CI; the suite completes in under five seconds on a fresh runner.
|
|
name: Shell unit tests
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 3
|
|
needs: validate-static
|
|
steps:
|
|
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
with:
|
|
persist-credentials: false
|
|
- name: Skip executable tests for external forks
|
|
if: github.event.pull_request.head.repo.fork == true
|
|
run: echo "::notice::repository shell tests are disabled for external forks"
|
|
- uses: actions/setup-python@v5
|
|
with:
|
|
python-version: "3.11"
|
|
# PyYAML is required by load-workspace-config.sh's python helper.
|
|
# In production it's transitive via molecule-ai-workspace-runtime;
|
|
# in this minimal CI env we install it explicitly so the YAML path
|
|
# is exercised instead of the script's silent ImportError fallback.
|
|
- if: github.event.pull_request.head.repo.fork != true
|
|
run: pip install -q pyyaml
|
|
- if: github.event.pull_request.head.repo.fork != true
|
|
run: bash scripts/test-derive-provider.sh
|
|
- if: github.event.pull_request.head.repo.fork != true
|
|
run: bash scripts/test-derive-platform-llm.sh
|
|
- if: github.event.pull_request.head.repo.fork != true
|
|
run: bash scripts/test-derive-minimax-bridge.sh
|
|
- if: github.event.pull_request.head.repo.fork != true
|
|
run: bash scripts/test-install-prefix-strip.sh
|
|
- if: github.event.pull_request.head.repo.fork != true
|
|
run: bash scripts/test-load-workspace-config.sh
|
|
- if: github.event.pull_request.head.repo.fork != true
|
|
run: bash scripts/test-default-model-selection.sh
|
|
- if: github.event.pull_request.head.repo.fork != true
|
|
run: bash scripts/test-mcp-configs-dir.sh
|
|
- if: github.event.pull_request.head.repo.fork != true
|
|
run: bash scripts/test-nonroot-boot.sh
|
|
- if: github.event.pull_request.head.repo.fork != true
|
|
run: bash scripts/test-process-liveness.sh
|
|
- if: github.event.pull_request.head.repo.fork != true
|
|
run: bash scripts/test-mcp-reconcile-watch.sh
|
|
- if: github.event.pull_request.head.repo.fork != true
|
|
run: bash scripts/test-log-mirror.sh
|
|
|
|
# --- ADR-004 §4 adapter-socket conformance (audit finding CI2) ---
|
|
# This repo SHIPS tests/test_conformance.py — a ~5-line subclass of the
|
|
# SDK-owned battery molecule_plugin.adapter_conformance.AdapterConformance
|
|
# pointed at this repo's Adapter — but until now NO CI job installed the SDK
|
|
# or ran pytest, so a hermes-only adapter-socket regression (path/render/
|
|
# present/enumerate/persona seam) shipped GREEN. The other three templates
|
|
# (codex/openclaw/claude-code) already run their conformance suite; this job
|
|
# closes the hermes gap and is wired into BOTH required-check aggregates
|
|
# (`validate` + `all-required`) so branch protection actually enforces it.
|
|
#
|
|
# SCOPE: this job runs adapter conformance plus the narrow release/runtime
|
|
# supply-chain contracts, not the whole tests/ dir. The adapter-socket suite
|
|
# (16 cases) is the CI2 target and is green.
|
|
#
|
|
# 2026-09-03: tests/test_executor_plugin_path.py used to be excluded here
|
|
# because it "currently fails independently (a2a extract_message_text /
|
|
# MagicMock-context version sensitivity)". That debt is now PAID — the
|
|
# runtime's canonical extract_message_text probes `.message` first to detect
|
|
# a RequestContext, and a bare MagicMock auto-creates that attribute, so
|
|
# every wire-level test read as a context wrapping a phantom message, got
|
|
# empty text, and returned before dispatching. Pinning `.message = None` on
|
|
# the mocked messages restores all 19. The file is folded in, as that comment
|
|
# anticipated. tests/test_vendor_branding_neutralized.py is folded in too:
|
|
# it is the guard that turns an upstream pin bump red until someone re-reads
|
|
# prompt_builder.py, and it cannot do that from outside CI.
|
|
conformance:
|
|
name: Adapter-socket conformance
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 5
|
|
steps:
|
|
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
with:
|
|
persist-credentials: false
|
|
- name: Skip executable conformance for external forks
|
|
if: github.event.pull_request.head.repo.fork == true
|
|
run: echo "::notice::adapter imports and repository conformance tests are disabled for external forks"
|
|
- uses: actions/setup-python@v5
|
|
with:
|
|
python-version: "3.11"
|
|
# The conformance test imports the hermes Adapter (adapter.py), which
|
|
# imports molecule_runtime.adapters.base, so the runtime dist must be
|
|
# present. Install it exactly like validate-runtime does: explicit
|
|
# --extra-index-url for the private Gitea dist (hermetic dep policy —
|
|
# no global PIP_INDEX_URL; see the env: block), never routing public
|
|
# tooling at the private index.
|
|
- if: github.event.pull_request.head.repo.fork != true && hashFiles('requirements.txt') != ''
|
|
run: pip install -q --extra-index-url "$MOLECULE_PRIVATE_INDEX" -r requirements.txt
|
|
# requirements-dev.txt pins pytest + pytest-asyncio and the molecule-ai-sdk
|
|
# (molecule_plugin) that ships the AdapterConformance battery that
|
|
# tests/test_conformance.py inherits. The SDK is pinned to a reviewed
|
|
# immutable commit there, so
|
|
# the private-index --extra-index-url is applied for its own private
|
|
# transitive deps.
|
|
- if: github.event.pull_request.head.repo.fork != true
|
|
run: pip install -q --extra-index-url "$MOLECULE_PRIVATE_INDEX" -r requirements-dev.txt
|
|
# test_vendor_branding_neutralized.py and test_executor_plugin_path.py
|
|
# were NOT in this list (2026-09-03). That is how both rotted unnoticed:
|
|
# * the vendor-branding guard exists specifically to turn an upstream
|
|
# pin bump RED until someone re-reads prompt_builder.py — it cannot
|
|
# do that from outside CI, and upstream had meanwhile added a THIRD
|
|
# vendor constant the neutralizer did not know about;
|
|
# * every wire-level test in the executor file had silently stopped
|
|
# exercising the transport (runtime's extract_message_text changed
|
|
# shape-detection; a MagicMock message now reads as a RequestContext
|
|
# and yields empty text, so execute() short-circuited before
|
|
# dispatching). 19 of them failed on a pristine tree.
|
|
# Both are green and in the list now. A guard nobody runs is not a guard.
|
|
- name: Run adapter and private-runtime contracts
|
|
if: github.event.pull_request.head.repo.fork != true
|
|
run: >-
|
|
python3 -m pytest
|
|
tests/test_conformance.py
|
|
tests/test_release_contracts.py
|
|
tests/test_prepare_runtime_requirements.py
|
|
tests/test_plugin_installer_cutover.py
|
|
tests/test_ci_runtime_image_pin.py
|
|
tests/test_ci_molecule_ci_fetch.py
|
|
tests/test_toolsearch_concierge_scope.py
|
|
tests/test_github_cli_present.py
|
|
tests/test_context_overflow_autoheal.py
|
|
tests/test_cron_approval_mode.py
|
|
tests/test_memory_skill_knobs.py
|
|
tests/test_session_reset_policy.py
|
|
tests/test_bundled_node_path.py
|
|
tests/test_executor_plugin_path.py
|
|
tests/test_vendor_branding_neutralized.py
|
|
tests/test_vendored_hermes_installer.py
|
|
tests/test_start_sh_tmpdir.py
|
|
tests/test_turn_activity_plugin.py
|
|
-v
|