From ec78c7637bce3bf6d4ddf9d4965282512102deaa Mon Sep 17 00:00:00 2001 From: Hongming Wang Date: Thu, 30 Apr 2026 23:54:57 -0700 Subject: [PATCH] docs(workspace-runtime): clarify what the boot-smoke gate does NOT prove MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A green gate means imports are healthy enough that executor.execute() reaches its body. It does NOT prove that execute() produces the right output: adapters with real I/O inside execute() (subprocess to a gateway, httpx call upstream) time out under the 5s harness window, and the gate treats a clean timeout as success. Surfaced while running publish-image across all 8 templates: the openclaw smoke "passed" with timing-out behavior in execute() because OpenClawA2AExecutor proxies to a subprocess that doesn't exist in the smoke env. Reading the green check, future operators might over-trust it as a runtime-correctness signal — it isn't. Add a "What the gate does NOT prove" subsection so readers don't mistake the import-regression coverage for an integration test. Co-Authored-By: Claude Opus 4.7 (1M context) --- content/docs/agent-runtime/workspace-runtime.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/content/docs/agent-runtime/workspace-runtime.md b/content/docs/agent-runtime/workspace-runtime.md index 8b87721..da331e2 100644 --- a/content/docs/agent-runtime/workspace-runtime.md +++ b/content/docs/agent-runtime/workspace-runtime.md @@ -113,6 +113,14 @@ fi - `adapter.create_executor()` runs - `executor.execute()` is invoked once against a stub `RequestContext`/`EventQueue` with `MOLECULE_SMOKE_TIMEOUT_SECS` (default 5s); a clean timeout exits 0, an import error exits non-zero +### What the gate does NOT prove + +A green gate means **"imports are healthy enough that `executor.execute()` reaches its body"** — that's the regression class the gate exists to catch (lazy `from x import y` inside an `if`-branch, or `importlib.import_module()` on a path that breaks after a wheel bump). + +It does **not** prove that `execute()` produces the right output for real input. Adapters that make real I/O calls inside `execute()` (subprocess to a gateway, httpx call to an upstream LLM) will time out under the harness's default 5s window, and the gate treats a clean timeout as success. The stub `RequestContext` carries an empty user message and the harness never inspects what `execute()` writes back. + +If you need correctness coverage, write a separate integration test that runs the workspace against real or mocked infrastructure — the smoke gate is a strict subset. + ### Stub env the smoke harness sets | Var | Value |