Compare commits

...

3 Commits

Author SHA1 Message Date
d4d3306150 fix(workspace): inject plugins_registry into sys.modules before loading adapters (closes #296)
Some checks failed
sop-tier-check / tier-check (pull_request) Failing after 3s
Secret scan / Scan diff for credential-shaped strings (pull_request) Failing after 58s
audit-force-merge / audit (pull_request) Successful in 2s
Plugin adapters in molecule-skill-* repos do:
  from plugins_registry.builtins import AgentskillsAdaptor as Adaptor

But _load_module_from_path() used exec_module() with a fresh module
namespace that did NOT have plugins_registry or its submodules in sys.modules,
causing:
  ModuleNotFoundError: No module named 'plugins_registry'

Fix: before exec_module(), import and register plugins_registry + all three
submodules (builtins, protocol, raw_drop) in sys.modules so adapter imports
resolve correctly.  Follows the Option 1 recommendation from issue #296.

Also adds test_resolve_plugin.py verifying the fix for both the
AgentskillsAdaptor import and the full InstallContext/resolve/protocol import.

Closes #296.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-10 14:17:16 +00:00
de9f46ea30 Merge pull request '[release-blocker] fix(ci): retry git clone in clone-manifest.sh (publish-workspace-server-image OOM flake)' (#298) from fix/publish-workspace-server-ci-clone-manifest-retry into staging
Some checks are pending
Secret scan / Scan diff for credential-shaped strings (push) Waiting to run
2026-05-10 12:44:35 +00:00
7ff5622a42 [infra-lead-agent] fix(ci): retry git clone in clone-manifest.sh (publish-workspace-server-image flake)
Some checks failed
Secret scan / Scan diff for credential-shaped strings (pull_request) Failing after 1s
sop-tier-check / tier-check (pull_request) Failing after 1s
audit-force-merge / audit (pull_request) Failing after 2s
The publish-workspace-server-image / build-and-push job clones the full
manifest (~36 repos) serially in the "Pre-clone manifest deps" step on a
memory-constrained Gitea Actions runner. Under host memory pressure the
OOM killer SIGKILLs git-remote-https mid-clone:

  cloning .../molecule-ai-plugin-molecule-skill-code-review.git ...
  error: git-remote-https died of signal 9
  fatal: the remote end hung up unexpectedly
    Failure - Main Pre-clone manifest deps
  exitcode '128': failure

Observed in run 4622 (2026-05-10, staging HEAD b5d2ab88) — died on the
14th of 36 clones, which red-lights CI and wedges staging→main.

Wrap each `git clone` in clone-manifest.sh with bounded retry + backoff
(3 attempts, 3s/6s), wiping any partial checkout between tries. A single
transient SIGKILL / network blip no longer fails the whole tenant image
rebuild. Benefits every caller of the script (publish-workspace-server-image,
harness-replays, Dockerfile builds, local quickstart).

This is a mitigation; the durable fix is more runner RAM/swap on the
operator host — tracked separately with Infra-SRE.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-10 11:58:09 +00:00
3 changed files with 121 additions and 5 deletions

View File

@ -37,6 +37,50 @@ PLUGINS_DIR="${4:?Missing plugins dir}"
EXPECTED=0
CLONED=0
# clone_one_with_retry — clone a single repo, retrying on transient failure.
#
# Why: the publish-workspace-server-image (and harness-replays) CI jobs
# clone the full manifest (~36 repos) serially on a memory-constrained
# Gitea Actions runner. Under host memory pressure the OOM killer
# occasionally SIGKILLs git-remote-https mid-clone:
#
# error: git-remote-https died of signal 9
# fatal: the remote end hung up unexpectedly
#
# (observed in publish-workspace-server-image run 4622 on 2026-05-10 — the
# job died on the 14th of 36 clones, which wedged staging→main). One
# transient SIGKILL / network blip would otherwise fail the whole tenant
# image rebuild. Retrying after a short backoff lets the pressure subside.
# The durable fix is more runner RAM/swap (tracked with Infra-SRE); this
# just stops a single flake from being release-blocking.
#
# Args: <target_dir> <name> <clone_url> <display_url> <ref>
clone_one_with_retry() {
local tdir="$1" name="$2" url="$3" display="$4" ref="$5"
local attempt=1 max_attempts=3 backoff
while : ; do
# A killed attempt can leave a partial directory behind; git clone
# refuses a non-empty target, so wipe it before each try.
rm -rf "$tdir/$name"
if [ "$ref" = "main" ]; then
if git clone --depth=1 -q "$url" "$tdir/$name"; then return 0; fi
else
if git clone --depth=1 -q --branch "$ref" "$url" "$tdir/$name"; then return 0; fi
fi
if [ "$attempt" -ge "$max_attempts" ]; then
echo "::error::clone failed after ${max_attempts} attempts: ${display}" >&2
return 1
fi
backoff=$((attempt * 3)) # 3s, then 6s
echo " ⚠ clone attempt ${attempt}/${max_attempts} failed for ${display} — retrying in ${backoff}s" >&2
sleep "$backoff"
attempt=$((attempt + 1))
done
}
clone_category() {
local category="$1"
local target_dir="$2"
@ -82,11 +126,7 @@ clone_category() {
fi
echo " cloning $display_url -> $target_dir/$name (ref=$ref)"
if [ "$ref" = "main" ]; then
git clone --depth=1 -q "$clone_url" "$target_dir/$name"
else
git clone --depth=1 -q --branch "$ref" "$clone_url" "$target_dir/$name"
fi
clone_one_with_retry "$target_dir" "$name" "$clone_url" "$display_url" "$ref"
CLONED=$((CLONED + 1))
i=$((i + 1))
done

View File

@ -51,6 +51,22 @@ class AdaptorSource:
def _load_module_from_path(module_name: str, path: Path):
"""Import a Python file by absolute path. Returns the module or None on failure."""
# Ensure the plugins_registry package and its submodules are importable in the
# fresh module namespace created by module_from_spec(). Plugin adapters
# (molecule-skill-*/adapters/*.py) use "from plugins_registry.builtins import ..."
# which requires plugins_registry and its submodules to already be in sys.modules.
# We import and register them before exec_module so the plugin's own
# from ... import statements resolve correctly.
import sys
import plugins_registry
sys.modules.setdefault("plugins_registry", plugins_registry)
for _sub in ("builtins", "protocol", "raw_drop"):
try:
sub = importlib.import_module(f"plugins_registry.{_sub}")
sys.modules.setdefault(f"plugins_registry.{_sub}", sub)
except Exception:
# Submodule may not exist in all versions; skip if absent.
pass
spec = importlib.util.spec_from_file_location(module_name, path)
if spec is None or spec.loader is None:
return None

View File

@ -0,0 +1,60 @@
"""Tests for _load_module_from_path sys.modules injection fix (issue #296).
Verifies that plugin adapters using "from plugins_registry.builtins import ..."
can be loaded via _load_module_from_path() without ModuleNotFoundError.
"""
import sys
import tempfile
import os
from pathlib import Path
# Ensure the plugins_registry package is importable
import plugins_registry
from plugins_registry import _load_module_from_path
def test_load_adapter_with_plugins_registry_import():
"""Plugin adapter using 'from plugins_registry.builtins import ...' loads cleanly."""
# Write a temp adapter file that does the exact import from the bug report.
with tempfile.NamedTemporaryFile(
mode="w", suffix=".py", delete=False, dir=tempfile.gettempdir()
) as f:
f.write("from plugins_registry.builtins import AgentskillsAdaptor as Adaptor\n")
f.write("assert Adaptor is not None\n")
adapter_path = Path(f.name)
try:
module = _load_module_from_path("test_adapter", adapter_path)
assert module is not None, "module should load without error"
assert hasattr(module, "Adaptor"), "module should expose Adaptor"
finally:
os.unlink(adapter_path)
def test_load_adapter_with_full_plugins_registry_import():
"""Plugin adapter using 'from plugins_registry import ...' loads cleanly."""
with tempfile.NamedTemporaryFile(
mode="w", suffix=".py", delete=False, dir=tempfile.gettempdir()
) as f:
f.write("from plugins_registry import InstallContext, resolve\n")
f.write("from plugins_registry.protocol import PluginAdaptor\n")
f.write("assert InstallContext is not None\n")
f.write("assert resolve is not None\n")
f.write("assert PluginAdaptor is not None\n")
adapter_path = Path(f.name)
try:
module = _load_module_from_path("test_adapter_full", adapter_path)
assert module is not None, "module should load without error"
assert hasattr(module, "InstallContext"), "module should expose InstallContext"
assert hasattr(module, "resolve"), "module should expose resolve"
assert hasattr(module, "PluginAdaptor"), "module should expose PluginAdaptor"
finally:
os.unlink(adapter_path)
if __name__ == "__main__":
test_load_adapter_with_plugins_registry_import()
test_load_adapter_with_full_plugins_registry_import()
print("ALL TESTS PASS")