molecule-ai/molecule-core
52
0
Fork 2
Code Issues 183 Pull Requests 16 Actions 2 Packages Projects Releases Wiki Activity

test(templates): CI vs local environment divergence in TestTemplatesList_* #1813

New Issue
Closed
opened 2026-05-24 23:00:38 +00:00 by hongming · 1 comment
hongming commented 2026-05-24 23:00:38 +00:00
Owner
Copy Link
Copy Source

Summary

TestTemplatesList_* in workspace-server/internal/handlers/templates_test.go pass in CI but fail on a fresh local checkout. The test handler logs runtime registry: manifest.json not found, using fallback allowlist (8 entries) locally, then rejects the test fixtures' empty runtime: field via the post-a5211050 runtime-allowlist filter. In CI the same code paths see a manifest.json with a more permissive runtime set (apparently includes empty) and the tests pass.

This is environment divergence, not a test or filter bug. Was flagged as acceptance criterion #3 on issue #1778 ("Document or remove the runtime-registry fallback-allowlist drift").

Reproduction

cd ~/molecule-core/workspace-server
go test -run '^TestTemplatesList_WithTemplates$' -count=1 ./internal/handlers/
# FAIL — fixtures missing runtime, fallback allowlist rejects empty

The same test passes in CI / Platform (Go) workflow against the same source commit.

Hypothesis

  1. CI's working tree has manifest.json committed or generated by a pre-test step. Local checkouts don't.
  2. The fallback allowlist (fallbackRuntimes in runtime_registry.go) is stricter than whatever manifest.json provides.

Fix options

Option Approach
A Commit a manifest.json to the repo with a known set of supported runtimes. Tests use the same list locally + CI.
B Make fallbackRuntimes accept empty runtime (the legacy backward-compat case) so the fallback is functionally equivalent to the manifest.
C Have tests load a fixture manifest.json explicitly, never the fallback.

Recommend C — most explicit, no production behavior change, scoped to tests.

Acceptance

  • go test -count=1 ./internal/handlers/ passes on a fresh local checkout (no manifest.json built)
  • CI continues to pass
  • No production behavior change

Discovered during

2026-05-24 memory-system session, fix for #1778 (templates_test fixtures) acknowledged this as a separate environment-divergence concern.

## Summary `TestTemplatesList_*` in `workspace-server/internal/handlers/templates_test.go` pass in CI but fail on a fresh local checkout. The test handler logs `runtime registry: manifest.json not found, using fallback allowlist (8 entries)` locally, then rejects the test fixtures' empty `runtime:` field via the post-`a5211050` runtime-allowlist filter. In CI the same code paths see a manifest.json with a more permissive runtime set (apparently includes empty) and the tests pass. This is environment divergence, not a test or filter bug. Was flagged as acceptance criterion #3 on issue #1778 ("Document or remove the runtime-registry fallback-allowlist drift"). ## Reproduction ```bash cd ~/molecule-core/workspace-server go test -run '^TestTemplatesList_WithTemplates$' -count=1 ./internal/handlers/ # FAIL — fixtures missing runtime, fallback allowlist rejects empty ``` The same test passes in `CI / Platform (Go)` workflow against the same source commit. ## Hypothesis 1. CI's working tree has `manifest.json` committed or generated by a pre-test step. Local checkouts don't. 2. The fallback allowlist (`fallbackRuntimes` in `runtime_registry.go`) is stricter than whatever manifest.json provides. ## Fix options | Option | Approach | |---|---| | A | Commit a `manifest.json` to the repo with a known set of supported runtimes. Tests use the same list locally + CI. | | B | Make `fallbackRuntimes` accept empty runtime (the legacy backward-compat case) so the fallback is functionally equivalent to the manifest. | | C | Have tests load a fixture `manifest.json` explicitly, never the fallback. | Recommend C — most explicit, no production behavior change, scoped to tests. ## Acceptance - [ ] `go test -count=1 ./internal/handlers/` passes on a fresh local checkout (no manifest.json built) - [ ] CI continues to pass - [ ] No production behavior change ## Discovered during 2026-05-24 memory-system session, fix for #1778 (templates_test fixtures) acknowledged this as a separate environment-divergence concern.
hongming commented 2026-05-24 23:06:52 +00:00
Author
Owner
Copy Link
Copy Source

Closing as not-reproducible

Re-checking on current main HEAD (post-#1777, #1786, #1809):

$ cd ~/molecule-core/workspace-server
$ go test -run '^TestTemplatesList_' -count=1 ./internal/handlers/
2026/05/24 16:05:58 runtime registry: manifest.json not found, using fallback allowlist (8 entries)
ok  	github.com/Molecule-AI/molecule-monorepo/platform/internal/handlers	0.587s

All TestTemplatesList_* pass green on a local checkout WITH the "manifest.json not found" log line — meaning the fallback allowlist IS being used and the fixtures pass against it.

What I'd missed: PR #1777 (a492d11, before this session) had already aligned the fixtures with the strict runtime allowlist by adding runtime: claude-code to each fixture. My earlier observation of the failure was on the pre-#1777 / pre-#1786 state. The fix has been in for hours.

There's no environment divergence to chase. The fallback allowlist works for both CI and local since the fixtures all declare a known runtime now.

Filing this issue was based on a stale read. Closing.

## Closing as not-reproducible Re-checking on current main HEAD (post-#1777, #1786, #1809): ```bash $ cd ~/molecule-core/workspace-server $ go test -run '^TestTemplatesList_' -count=1 ./internal/handlers/ 2026/05/24 16:05:58 runtime registry: manifest.json not found, using fallback allowlist (8 entries) ok github.com/Molecule-AI/molecule-monorepo/platform/internal/handlers 0.587s ``` All `TestTemplatesList_*` pass green on a local checkout WITH the "manifest.json not found" log line — meaning the fallback allowlist IS being used and the fixtures pass against it. What I'd missed: PR #1777 (`a492d11`, before this session) had already aligned the fixtures with the strict runtime allowlist by adding `runtime: claude-code` to each fixture. My earlier observation of the failure was on the pre-#1777 / pre-#1786 state. The fix has been in for hours. There's no environment divergence to chase. The fallback allowlist works for both CI and local since the fixtures all declare a known runtime now. Filing this issue was based on a stale read. Closing.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
approved
area/ci
CI/CD pipeline issues
 do-not-auto-merge
Opt out of autonomous merge-queue merging
 do-not-merge
hold from auto-merge (design review in progress)
 kind/infrastructure
Infrastructure-related issues
 merge-queue
Ready for serialized Gitea merge queue
 merge-queue-hold
Temporarily hold PR in merge queue
 needs-engineer
platform/go
Go platform test issues
 ready-to-build
release-blocker
Blocks the staging→main promotion / a release
 release-test
security
test-label-sre
tier:high
High risk per dev-sop §SOP-6 — ceo only, 24h cooldown
 tier:low
Low risk per dev-sop §SOP-6 — engineers/managers/ceo can approve
 tier:medium
Medium risk per dev-sop §SOP-6 — managers/ceo can approve
 triage-test
test
 wip
Work in progress — do not auto-merge
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
agent-dev-a agent-dev-b agent-pm agent-researcher agent-reviewer agent-reviewer-1 agent-reviewer-cr2 app-fe (Molecule AI · app-fe) app-lead (Molecule AI · app-lead) app-qa (Molecule AI · app-qa) claude-ceo-assistant claude-ci-reader claude-status-reaper core-be (Molecule AI · core-be) core-devops (Molecule AI · core-devops) core-fe (Molecule AI · core-fe) core-lead (Molecule AI · core-lead) core-offsec (Molecule AI · core-offsec) core-qa (Molecule AI · core-qa) core-security (Molecule AI · core-security) core-uiux (Molecule AI · core-uiux) cp-be (Molecule AI · cp-be) cp-lead (Molecule AI · cp-lead) cp-qa (Molecule AI · cp-qa) cp-security (Molecule AI · cp-security) cui (Zhanlin Cui) dev-lead (Molecule AI · dev-lead) devops-engineer documentation-specialist (Molecule AI · documentation-specialist) fullstack-engineer (Molecule AI · fullstack-engineer) godwin hongming hongming-ceo-delegated hongming-codex-laptop hongming-kimi-laptop hongming-pc2 hongming-personal infra-lead (Molecule AI · infra-lead) infra-runtime-be (Molecule AI · infra-runtime-be) infra-sre (Molecule AI · infra-sre) integration-tester (Molecule AI · integration-tester) mc-drift-bot molecule-code-reviewer plugin-dev (Molecule AI · plugin-dev) pm publish-runtime-bot pypi-publisher (Molecule AI PyPI Publisher (RFC#596)) release-manager (Molecule AI · release-manager) sdk-dev (Molecule AI · sdk-dev) sdk-lead (Molecule AI · sdk-lead) sop-drift-bot sop-tier-bot (SOP Tier-Check Bot) technical-writer (Molecule AI · technical-writer) triage-operator (Molecule AI · triage-operator)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: molecule-ai/molecule-core#1813
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?