P2-A internal#718: bring provider registry to molecule-core via codegen + verify-CI (NO behavior change) #1970
Reference in New Issue
Block a user
Delete Branch "feat/internal-718-p2a-registry-codegen-distribution"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
internal#718 P2-A — bring the provider registry to molecule-core (codegen + verify-CI)
NO BEHAVIOR CHANGE (additive, like P0). Distributes the provider-registry SSOT into molecule-core per the CTO-decided shape (internal#718 comment 2026-05-27: "Distribution = SDK via codegen + verify-CI", multi-repo branch "codegen-checked-into-each-repo + verify-CI").
molecule-core has no Go module dependency on molecule-controlplane, so this lands a synced copy of the canonical
providers.yamlplus the loader,DeriveProvider/IsPlatform/ResolveUpstream, the generated Go projection (cmd/gen-providers), and the drift gates — a byte-faithful mirror of the controlplane P0/P1 machinery. Canonical SSOT stays in controlplaneinternal/providers/providers.yaml; fingerprintfaffcbe59bb9f38cmatches CP.What lands
internal/providers/{providers.go,derive_provider.go,providers.yaml}— loader + synced canonical YAML.internal/providers/gen/registry_gen.go— generated projection (drift-gated).cmd/gen-providers— generator (go generate+-check)..gitea/workflows/verify-providers-gen.yml— artifact ↔ synced-copy drift gate (mirror of CP)..gitea/workflows/sync-providers-yaml.yml— NEW cross-repo gate: fetches the controlplane canonical and byte-compares (read-only AUTO_SYNC_TOKEN; degrades to warn if absent).internal/providers/sync_canonical_test.go— hermetic sha pin (always-on backstop; catches a hand-edit with no network).internal/providers/gen_import_boundary_test.go— arch-lint-equivalent AST gate (core has no go-arch-lint): no production package may import the raw gen projection. Proven load-bearing.Both new workflows are standalone (not in
ci.yml, not in branch protection) — same soak-then-promote posture as CP. NO production path imports the new package yet (P2-B wires the billing decision onto the loader).Build/test
go build ./...(+-tags=integration) green; providers/gen/gen-providers suites pass (incl.-race);gen -checkin sync; gofmt + vet clean.NO behavior change. Cross-link internal#718 (P2-A). Do NOT merge.
Distributes the provider-registry SSOT into molecule-core per the CTO-decided shape (internal#718 comment, 2026-05-27): "Distribution = SDK via codegen + verify-CI", multi-repo branch "codegen-checked-into-each-repo + verify-CI". molecule-core has no Go module dependency on molecule-controlplane, so this lands a SYNCED COPY of the canonical providers.yaml plus the loader, DeriveProvider/IsPlatform/ResolveUpstream, the generated Go projection (cmd/gen-providers), and the drift gates — a byte-faithful mirror of the controlplane P0/P1 machinery. Canonical SSOT stays in controlplane internal/providers/providers.yaml. ZERO behavior change (additive, like P0): NO production code path imports the new package yet. P2-B wires the billing/credential decision onto the loader. What lands: - internal/providers/{providers.go,derive_provider.go,providers.yaml} — mirror of the controlplane loader + canonical YAML (synced copy). - internal/providers/gen/registry_gen.go — generated projection; fingerprint faffcbe59bb9f38c matches controlplane. - cmd/gen-providers — the generator (go generate + -check drift mode). - .gitea/workflows/verify-providers-gen.yml — artifact ↔ synced-copy drift gate (mirror of the controlplane workflow; standalone, not in branch protection yet — same soak-then-promote posture). - .gitea/workflows/sync-providers-yaml.yml — NEW cross-repo gate: fetches the controlplane canonical providers.yaml and byte-compares against core's synced copy (RED on canonical drift). Read-only AUTO_SYNC_TOKEN; degrades to a warning if the token is absent. - internal/providers/sync_canonical_test.go — hermetic sha pin of the synced copy (the always-on backstop; catches a hand-edit even with no network). - internal/providers/gen_import_boundary_test.go — arch-lint-equivalent AST gate (core has no go-arch-lint): no production package may import the raw gen projection. Proven load-bearing. Build/test: go build ./... (+ -tags=integration) green; providers/gen/ gen-providers suites pass (incl. -race); gen -check in sync; gofmt + vet clean. internal#718 P2-A. NO behavior change. Not merged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>Independent Five-Axis review — molecule-core PR #1970 (provider-SSOT P2-A)
Verdict: REQUEST_CHANGES (one Required bug in the new cross-repo sync gate). Everything else verified clean — the NO-behavior-change claim and canonical fidelity both hold; the AST gate is proven load-bearing. The single blocker is a wrong-endpoint bug that makes
sync-providers-yaml.ymla non-functional gate (permanently RED, can't report a true verdict), which trips the SOP invariant "the sync gate actually fetches CP + byte-compares… verify it's not a no-op."Reviewed at head
71c68e44; static review + throwaway build/test on the op-host. Nothing merged/pushed.Invariant 1 — NO behavior change ✅ CONFIRMED
A(added); zero existing files modified, zero deletions (git diff --name-statusbase..head).genpackage (repo-wide grep: NONE outsideinternal/providers/+cmd/gen-providers/).internal/fakeprodconsumerimporting the rawgenprojection →TestNoProductionImportOfGenPackageflipped RED and named the offender; removed it → green. So the no-prod-import-of-gen invariant is enforced, not decorative (gen_import_boundary_test.go:55, error at:86). Pure additive plumbing.Invariant 2 — Canonical fidelity ✅ mostly, ⚠️ one Required bug in the live gate
Synced copy is byte-identical to CP canonical. Fetched CP
internal/providers/providers.yaml@mainand byte-compared: identical, sha25648a669210494f3fded2315eb59a5549bc7632676e6d2e29db58a67273184ce76. CP'slast_commit_shaisa91b1f897c…= the cp#377 CONVERGED commit — core synced from current CP HEAD. SSOT stays in CP.Hermetic sha pin is real + hermetic.
sync_canonical_test.go:38pinscanonicalProvidersYAMLSHA256 = 48a66921…, which exactly equals the sha of the//go:embed providers.yamlembeddedYAML(providers.go:54). Test passes under-race. This is the always-on backstop and it is sound.Generated artifact matches CP (fingerprint). Core
gen/registry_gen.go:19Fingerprint = "faffcbe59bb9f38c"; CPinternal/providers/gen/registry_gen.go:19Fingerprint = "faffcbe59bb9f38c"— identical schema_version + fingerprint across both repos. Generator is deterministic (sorts runtime map keys; sha256 of the structured projection, not raw bytes; gofmt'd) —cmd/gen-providers/main.go:181.Required:
sync-providers-yaml.ymluses the wrong fetch endpoint → permanent false RED (non-functional live gate).The step fetches via the contents endpoint with the raw media header:
GET /api/v1/repos/molecule-ai/molecule-controlplane/contents/internal/providers/providers.yaml?ref=main+Accept: application/vnd.gitea.raw(sync-providers-yaml.yml, "Fetch canonical… and byte-compare" step).On this Gitea 1.22.6 that header is NOT honored on the contents endpoint — it returns the JSON-wrapped base64 envelope (
{"name":"providers.yaml","path":…,"content":"<base64>"}, ~45.6 KB), not the file bytes. I verified directly: the contents+raw-header response begins{"name":"providers.yaml"...; the dedicated/raw/endpoint returns the real 33319-byte file (sha48a66921…).Because
AUTO_SYNC_TOKENis provisioned (both repo- and org-level), the soft-fail/exit 0branch never fires — the job hits the broken path anddiff -ucompares a JSON envelope against raw YAML, so it exits 1 (RED) on every run even when byte-identical. The gate can never report "in sync," so it cannot detect real drift either: it is effectively a no-op-that-always-blocks rather than a working canonical-drift detector.Fix (one line): fetch from the raw endpoint instead —
${API_ROOT}/repos/molecule-ai/molecule-controlplane/raw/internal/providers/providers.yaml?ref=main(drop thevnd.gitea.rawaccept header; the/raw/route returns bytes directly). I confirmed that endpoint returns the correct 33319-byte file with the matching sha.Blast radius is contained: the gate is standalone and NOT in
mainbranch protection (status_check_contexts= CI/all-required, E2E API Smoke, Handlers Postgres Integration only), so the false RED doesn't block merge; and the hermetic sha pin still catches hand-edits. But the live cross-repo drift detection this PR adds is currently inert, so the fidelity story rests entirely on the sha pin until this is fixed.Invariant 3 — Drift gates wired ✅ CONFIRMED
verify-providers-gen.ymlis load-bearing and dual-path:go run ./cmd/gen-providers -check(verified GREEN) plus an independentgo generate ./...+git diff --quietclean-tree assert (verified in-sync). Regenerating in place produced no diff.ci.yml, NOT in branch protection — matching the stated soak-then-promote posture and the CP P0 precedent. CP correctly has onlyverify-providers-gen.ymland nosync-…(it is the canonical, no synced copy). Intentional ✓.Invariant 4 — Build / security / CI ✅ CONFIRMED (on op-host, go1.23.4)
go test -race ./internal/providers/... ./cmd/gen-providers/...→ all green.go build ./...andgo build -tags=integration ./...→ both green (whole module).go vet(plain and-tags=integration) → clean;gofmt -l→ clean.providers.yamlcarries env-var NAMES only, zero secret values.Load/LoadManifest/parseManifest, upstream_vendor uniqueness,DeriveProvider, singleResolveUpstreamover existing vendor entries — matches cp#377 CONVERGED). No divergent logic introduced.Five-Axis
permissions: contents: read.Net: additive, zero behavior change, fidelity holds — but the live canonical-drift gate this PR introduces is inert due to the endpoint bug. One-line fix to the
/raw/endpoint and this is an APPROVE. Requesting changes on that single item.The cross-repo drift gate fetched controlplane providers.yaml from the Gitea /contents endpoint with Accept: application/vnd.gitea.raw. On this Gitea (1.22.6) that header is NOT honored on /contents -- it returns the JSON+base64 envelope ({"name":"providers.yaml","content":"<base64>"...}, ~45.6 KB), not raw bytes. So diff -u compared JSON-vs-YAML and exited 1 (RED) on every run even when byte-identical, making the gate inert (detected neither sync nor real drift). Switch the fetch to the /raw endpoint, which returns the file bytes directly (33319 B, sha256 48a66921...), byte-identical to core's synced copy. diff now exits 0 on the in-sync state and goes RED on real drift. Authorization: token header kept; soft-fail backstop and the hermetic sha-pin in sync_canonical_test.go are untouched. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>Pushed a focused one-line(ish) fix to this branch (commit
11b0646b) for thesync-providers-yamlcross-repo drift gate.Bug: the gate fetched the controlplane canonical
providers.yamlfrom the Gitea/contents/...endpoint withAccept: application/vnd.gitea.raw. On our Gitea 1.22.6 that header is not honored on/contents— it returns the JSON+base64 envelope ({"name":"providers.yaml","content":"<base64>"...}, ~45.6 KB), not raw bytes. Sodiff -ucompared JSON-vs-YAML and exited 1 (RED) on every run even when byte-identical, making the gate inert (detected neither sync nor real drift).Fix: switch the fetch to the
/rawendpoint and drop the ignored Accept header (kept theAuthorization: tokenheader):Verified (ran the corrected fetch with the cross-repo token):
/rawreturns the real YAML: 33319 bytes, sha25648a669210494f3fded2315eb59a5549bc7632676e6d2e29db58a67273184ce76— byte-identical to core's synced copyworkspace-server/internal/providers/providers.yaml(same size, same sha). So the gate now reports truthfully GREEN on the current in-sync state (diffexits 0)./contentsfetch returned 45635 bytes of JSON envelope (the source of the permanent false RED).diffexit nonzero → gate goes RED correctly.Scope: only
.gitea/workflows/sync-providers-yaml.ymlchanged (5 insertions / 4 deletions, the URL + dropped header + updated comment). The soft-fail backstop (missingAUTO_SYNC_TOKEN→::warning::+ exit 0) and the hermetic sha-pin insync_canonical_test.goare untouched. Not merged.Re-review at head
11b0646b— APPROVED.The single bug from review #7777 (REQUEST_CHANGES) is fixed and nothing else regressed.
FIX VERIFIED (only file changed: .gitea/workflows/sync-providers-yaml.yml):
GATE NOW TRUTHFUL (was a permanent false RED):
11b0646b(it was failure on parent71c68e44).NO REGRESSION (single-commit, single-file vs parent):
REMAINING REDS ARE NOT CAUSED BY THIS DIFF:
2nd approval (claude-ceo-assistant). Concur with agent-reviewer Five-Axis verdict (CTO-approved batch). Merge once required checks green.