feat(validate): platform-model SSOT drift gate (template ⊆ manifest) #24

2026-05-27T09:30:23Z

hongming commented

2026-05-27 09:30:23 +00:00

Summary

Adds the SSOT drift gate for platform-managed LLM models (RFC internal#580 Option C). A template's config.yaml provider: platform models must be a SUBSET of the controlplane providers manifest's platform set for that runtime (internal/providers/providers.yaml runtimes block). Pairs with the controlplane manifest SSOT (cp #359) + the deploy-time platform-models e2e smoke.

Why

Offering a provider: platform model the SSOT doesn't declare risks shipping an unservable option — the SEO 1033 / "Exception: success" class. This gate keeps each template's platform offering honest against the single source of truth.

How

check_platform_models() in validate-workspace-template.py (runs in static + full mode).
Manifest fetch is best-effort: PROVIDERS_MANIFEST_FILE for tests/offline, else a blobless sparse git clone of just providers.yaml via the runner's ambient git creds. Fetch failure -> WARN + skip (no coupling to controlplane uptime; the e2e smoke is the hard backstop). Runtime absent from manifest -> WARN + skip.

Test plan

5 new pytest cases (subset/superset/none/unreachable/runtime-absent)
full suite: 35 passed, no regression

## Summary Adds the SSOT drift gate for platform-managed LLM models (RFC internal#580 Option C). A template's config.yaml `provider: platform` models must be a SUBSET of the controlplane providers manifest's platform set for that runtime (internal/providers/providers.yaml runtimes block). Pairs with the controlplane manifest SSOT (cp #359) + the deploy-time platform-models e2e smoke. ## Why Offering a `provider: platform` model the SSOT doesn't declare risks shipping an unservable option — the SEO 1033 / "Exception: success" class. This gate keeps each template's platform offering honest against the single source of truth. ## How - `check_platform_models()` in validate-workspace-template.py (runs in static + full mode). - Manifest fetch is best-effort: `PROVIDERS_MANIFEST_FILE` for tests/offline, else a blobless sparse `git` clone of just providers.yaml via the runner's ambient git creds. Fetch failure -> WARN + skip (no coupling to controlplane uptime; the e2e smoke is the hard backstop). Runtime absent from manifest -> WARN + skip. ## Test plan - [x] 5 new pytest cases (subset/superset/none/unreachable/runtime-absent) - [x] full suite: 35 passed, no regression

hongming added 1 commit 2026-05-27 09:30:23 +00:00

feat(validate): platform-model SSOT drift gate (template ⊆ manifest)

CI / Workflow YAML lint (pull_request) Successful in 5s

Details

CI / Python script lint (pull_request) Successful in 46s

Details

CI / Secrets scan (pull_request) Successful in 1m9s

Details

0bc7eed063

A template's config.yaml `provider: platform` models must be a SUBSET of the
controlplane providers manifest's platform set for that runtime
(internal/providers/providers.yaml runtimes block — the SSOT per RFC#580
Option C). Offering a platform model the SSOT doesn't declare risks an
unservable option (the SEO 1033 / "Exception: success" class).

- check_platform_models(): extracts the template's provider:platform model
  ids, fetches the manifest, errors on any id not in the runtime's manifest
  platform set.
- Manifest fetch is best-effort: PROVIDERS_MANIFEST_FILE (local path) for
  tests/offline, else a blobless sparse `git` clone of just providers.yaml
  using the runner's ambient git creds (same access as the molecule-ci
  clone). On fetch failure -> WARN + skip, so template CI isn't coupled to
  controlplane reachability; the deploy-time platform-models e2e smoke is the
  hard backstop.
- Runtime absent from the manifest -> WARN + skip (can't gate a runtime the
  SSOT doesn't define).
- 5 tests: subset passes, non-manifest model errors, no-platform skips,
  unreachable warns-not-errors, runtime-absent warns.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

hongming referenced this issue from a commit

2026-05-27 10:25:44 +00:00

fix(validate): drift-gate manifest fetch was always failing (review #24)

hongming added 1 commit 2026-05-27 10:25:44 +00:00

fix(validate): drift-gate manifest fetch was always failing (review #24 )

CI / Workflow YAML lint (pull_request) Successful in 10s

Details

CI / Python script lint (pull_request) Successful in 34s

Details

CI / Secrets scan (pull_request) Successful in 1m21s

Details

b5046e5141

Independent review caught that the live fetch path silently degraded to
always-WARN-skip — the gate never actually blocked:
- sparse-checkout cone mode (the default) takes DIRECTORY paths, not file
  paths, so `set internal/providers/providers.yaml` failed ("not a
  directory") -> CalledProcessError -> swallowed -> WARN-skip every real CI
  run. Use the containing directory `internal/providers`; the file read
  narrows it.
- the clone/sparse path had ZERO test coverage (all 5 tests used
  PROVIDERS_MANIFEST_FILE), which is exactly why the bug shipped green. Add
  two tests that exercise the REAL clone against a local file:// repo: one
  proving subset-pass, one proving it DETECTS drift via the live path.
- log git stderr on clone failure so future breakage is visible, not a
  silent skip.

Full suite: 37 passed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

core-qa approved these changes 2026-05-27 10:39:26 +00:00

core-qa left a comment

Five-Axis (correctness/tests): subset check correct; sparse-checkout cone-mode bug fixed (dir not file path); added real-clone tests (subset-pass + drift-detect) that exercise the live path — the always-skip bug is now covered; full suite 37 green. APPROVED.

core-security approved these changes 2026-05-27 10:39:26 +00:00

core-security left a comment

Security axis: clone uses ambient git creds (no token in URL/args/logs), subprocess arg-list (no shell injection), repo/rel not user-controlled. APPROVED.

hongming merged commit 5b380b1341 into main

2026-05-27 10:58:14 +00:00

hongming referenced this issue from a commit

2026-05-27 10:58:15 +00:00

Merge pull request 'feat(validate): platform-model SSOT drift gate (template ⊆ manifest)' (#24) from feat/platform-models-ssot-drift-gate into main

Sign in to join this conversation.

3 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: molecule-ai/molecule-ci#24