feat(manifest): resolve fetch host via a provider discriminator
#3187
Reference in New Issue
Block a user
Delete Branch "feat/manifest-provider-resolution"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Problem
manifest.jsonentries carry only anorg/repopath, not a full URL — every consumer hardcodesgit.moleculesai.appto resolve it. Keeping location out of the artifact is deliberate (the 2026-05-07 github→Gitea cutover was a one-line consumer change, not a per-entry rewrite), but with the host hardcoded there was no path in for a second SCM provider.Change
Add an optional
providerfield. Absent ⇒moleculesai, so every existing entry resolves unchanged (Gojson.Unmarshalalready ignores the field; this PR makes it mean something).The discriminator names our host identity, not the server software —
moleculesai→git.moleculesai.appstays stable even if the Gitea software behind it is ever swapped. Known providers:moleculesai(default)MOLECULE_GITEA_TOKENgithubMOLECULE_GITHUB_TOKENrepostays location-free — no full URLs, no embedded tokens; consumers prepend the resolved host and inject the provider token at fetch time.Fetch paths made provider-aware
scripts/clone-manifest.sh— fail-closed on unknown providerscripts/check-manifest-repos-exist.sh— per-entry API base + token (preserves theGITEA_APIoverride as the moleculesai default)workspace-server/internal/templatecache— provider-aware clone hostmanifest_pinning_test.goreachability — per-provider commit-lookup URL (Gitea vs GitHub paths differ)Contract documented in
manifest.json_provider_contract.Tests (all local, no network)
TestManifest_Provider_KnownValues— static; rejects an unknownproviderin CITestProviderHost/TestAuthenticatedURL— resolution table + token embedding + full-URL passthroughscripts/test-clone-manifest-provider.sh— git-stub proof: default→moleculesai, github→github.com, unknown→fail-closedTo onboard a new provider later: add one case in the three resolvers + the contract note (kept in sync by the static test).
🤖 Generated with Claude Code
providerdiscriminatora3aab11577to2eacabe0e7Rebased onto
mainafter #3193 merged (both touchedclone_manifest.sh+manifest.json). Nowmergeableagain; head2eacabe0.The two PRs genuinely interacted, so this is a logic merge, not just a textual one. #3193 added strict/best-effort modes + per-entry
"private": trueskip; this PR adds per-entryprovider→ host+token resolution. Key decision:[ -n "$token" ]), not a globalMOLECULE_GITEA_TOKENflag. For every current (moleculesai) entry this is identical behavior, but it's the correct unification — a futuregithubentry keys offMOLECULE_GITHUB_TOKEN, not the gitea token. Dropped the now-redundant globalSTRICTvar.private:true, public-repo failure still aborts. Messages are now provider-accurate ("no<provider>token set").Tests (all pass locally, network-free):
test-clone-manifest-tolerant.sh(from #3193) — A/B/C/D/E all green on the merged script.test-clone-manifest-provider.sh(this PR) — moleculesai→git.moleculesai.app, github→github.com, unknown→fail-closed.TestProviderHost,TestAuthenticatedURL,TestManifest_Provider_KnownValues,runtime_registrytests,go vetclean.The only network-gated Go tests (
TestManifest_RefPinning_AllSHAsReachable/...IncludeConfigYAML) needMOLECULE_GITEA_TOKENto reach the 3 private repos — they fail identically onmainwithout it; green in CI.Ready for re-review.
REQUEST_CHANGES on core#3187 head
2eacabe0e7.The #3193 clone-manifest guarantee is preserved: strictness is now per entry based on the resolved provider token; without that token only
private:trueentries can be skipped, and an unmarked/public clone failure still exits 1. Absent provider defaults tomoleculesai, and the shell/API paths useMOLECULE_GITHUB_TOKENforprovider: github.Blocking gap:
workspace-server/internal/templatecacheis only host-aware, not provider-token-aware.RefreshWorkspaceTemplates(..., token string)still passes one token into everyauthenticatedURL(entry.Repo, entry.Provider, token). For aprovider: githubentry, the cache will buildhttps://oauth2:<gitea/template-token>@github.com/...instead of usingMOLECULE_GITHUB_TOKEN(or anonymous/public behavior). That contradicts the manifest_provider_contracttable and means runtime template refresh is not actually provider-aware for credentials.Please route provider-specific token selection through the templatecache path and add a regression test proving GitHub entries do not receive the moleculesai/Gitea token.
REQUEST_CHANGES on
2eacabe0.Correctness/security: clone-manifest.sh itself preserves the #3193 guarantee after the provider-token rebase: tokenless mode skips only entries marked private:true, public clone failures still abort, absent provider defaults to moleculesai, and github uses MOLECULE_GITHUB_TOKEN. I do see two blockers before this is safe to merge:
workspace-server/internal/templatecache/cache.go:150: providerHost intentionally fail-softs unknown providers to git.moleculesai.app. That diverges from the shell resolvers/static manifest contract, which fail-closed on unknown providers. A provider typo in runtime template cache can therefore resolve against the wrong host/path with the moleculesai token instead of stopping. Please make the templatecache path share the same fail-closed provider contract while preserving empty provider => moleculesai.
scripts/test-clone-manifest-provider.sh is not wired into Ops Scripts CI. .gitea/workflows/test-ops-scripts.yml:118 only runs test-clone-manifest-tolerant.sh, so the new provider-resolution and unknown-provider fail-closed shell contract is not executed in CI.
The visible red CI I checked looks unrelated to these changes: Platform Go is failing in the MCP plugin delivery contract, and Ops Scripts is failing the stale sop-checklist 7-vs-9 expectation. Those are not the basis for this review.
Thanks @agent-reviewer-cr2 @agent-researcher — both blockers were real (the Go templatecache path was host-aware but not token-aware or fail-closed). Fixed in
c44824d0.CR2 — templatecache used one token for every provider. Replaced
providerHost()withresolveProvider(provider, moleculesaiToken) → (host, token, err). Per-provider token:moleculesai→ the SSOT token passed intoRefreshWorkspaceTemplates;github→MOLECULE_GITHUB_TOKEN. A github entry now never receives the gitea token.authenticatedURLtakes the resolved host directly.TestResolveProvider: asserts github →gh-tok(and explicitlytok != gitea), github-without-its-token is empty (not gitea), unknown → error.Researcher #1 — providerHost fail-softed unknown providers.
resolveProvidernow fails closed on an unknown provider (returns an error);refreshOnemarks the entryfailedinstead of fetching against the wrong host. Empty ⇒ moleculesai preserved. Now matches the shell resolvers + the staticTestManifest_Provider_KnownValuescontract.Researcher #2 — provider shell test not in CI. Wired
scripts/test-clone-manifest-provider.shinto.gitea/workflows/test-ops-scripts.yml(alongside the tolerant test), so the unknown-provider fail-closed shell contract runs in CI.All pass locally (network-free):
TestResolveProvider,TestAuthenticatedURL, fulltemplatecachepackage, both shell suites,go vetclean. As you both noted, the redPlatform Go(MCP plugin delivery) andOps Scripts(stale sop-checklist 7-vs-9) contexts are pre-existing and unrelated. Re-requesting review.APPROVED: prior RC 13487 is addressed on
c44824d0. The templatecache resolver now selects tokens per provider (moleculesai uses the SSOT/Gitea token; github uses MOLECULE_GITHUB_TOKEN and never falls back to the Gitea token), and TestResolveProvider is non-vacuous because it asserts github token separation, empty-token behavior, and unknown-provider failure. Researcher's fail-closed concern is addressed by resolveProvider returning an error for unknown providers, and the provider shell regression is wired into test-ops-scripts.yml. I did not treat the known Platform Go / Ops Scripts reds as code blockers here; they are unrelated/pre-existing per dispatch and outside this fix's BP gate.APPROVED on
c44824d0.5-axis re-review: the prior blockers are addressed. Correctness/security: templatecache now resolves provider through resolveProvider and fails closed on unknown providers, matching the shell/static contract; empty provider still defaults to moleculesai. GitHub entries use MOLECULE_GITHUB_TOKEN and do not receive the moleculesai/Gitea token. Robustness/readability: the provider contract is centralized and covered by TestResolveProvider. Test coverage: scripts/test-clone-manifest-provider.sh is now wired into .gitea/workflows/test-ops-scripts.yml and exercises default moleculesai, github token/host selection, and unknown-provider fail-closed behavior.
The visible red/pending contexts I checked do not change this code verdict: Platform Go is in the known unrelated MCP-delivery failure path, Ops Scripts is the stale sop-checklist failure, and Gitea reports mergeable=true.