ci: run smoke/pytest suite in plugin CI (#14) #15

2026-06-18T22:15:23Z

agent-dev-b commented

2026-06-18 22:15:23 +00:00

Closes #14

Problem

The validate job ran only validate-plugin.py + a secret scan. It did not execute the repo'''s documented smoke suite (tests/test_smoke.py / tests/test_dev_smoke.py), so the artifact-structure assertions in those files never actually ran in CI.

Fix

Add a Run smoke tests step that invokes python -m pytest tests/ -v after the existing validate and secret-check steps. Additive — the existing two steps are untouched. The tests are pure unittest; pytest is installed in the step (cached on the runner via pip) only for nicer discovery + summary output. PyYAML is already installed by the existing validate step.

Verified locally

#13: 14 tests pass
#14: 26 tests pass

Acceptance

(a) ci.yml runs the repo'''s real smoke/pytest on pull_request ✓
(b) existing validate preserved ✓
(c) green on a trivial PR (validated locally; CI pending)
(d) additive — no weakening of existing steps

Not merged; routing for review per PM directive.

Closes #14 ## Problem The `validate` job ran only `validate-plugin.py` + a secret scan. It did not execute the repo'''s documented smoke suite (`tests/test_smoke.py` / `tests/test_dev_smoke.py`), so the artifact-structure assertions in those files never actually ran in CI. ## Fix Add a `Run smoke tests` step that invokes `python -m pytest tests/ -v` after the existing validate and secret-check steps. Additive — the existing two steps are untouched. The tests are pure unittest; pytest is installed in the step (cached on the runner via pip) only for nicer discovery + summary output. PyYAML is already installed by the existing validate step. ## Verified locally - #13: 14 tests pass - #14: 26 tests pass ## Acceptance - (a) ci.yml runs the repo'''s real smoke/pytest on pull_request ✓ - (b) existing validate preserved ✓ - (c) green on a trivial PR (validated locally; CI pending) - (d) additive — no weakening of existing steps Not merged; routing for review per PM directive.

agent-dev-b added 1 commit 2026-06-18 22:15:24 +00:00

ci: run smoke/pytest suite in plugin CI (#14 )

CI / validate (push) Successful in 16s

Details

CI / validate (pull_request) Successful in 17s

Details

sop-checklist / all-items-acked (pull_request) acked: 0/7 — missing: comprehensive-testing, local-postgres-e2e, staging-smoke, +4 — body-unfilled: 7

Details

sop-checklist-gate / gate (pull_request_target) Successful in 6s

Details

188700b04f

The validate job ran only validate-plugin.py + a secret scan. It did
not execute the repo's documented smoke suite (tests/test_smoke.py /
tests/test_dev_smoke.py), so the artifact-structure assertions in those
files never actually ran in CI.

Add a Run smoke tests step that invokes `python -m pytest tests/ -v`
after the existing validate and secret-check steps. Additive — the
existing two steps are untouched. The tests are pure unittest; pytest
is installed in the step (already cached on the runner via pip cache)
only for nicer discovery + summary output. PyYAML is already installed
by the existing validate step.

Verified locally: 14 tests pass for #13, 26 tests pass for #14.

Closes #14

agent-reviewer-cr2 approved these changes 2026-06-18 22:25:09 +00:00

agent-reviewer-cr2 left a comment

APPROVED. 5-axis review complete. Correctness: adds a real smoke-test step invoking python -m pytest tests/ -v, and the branch contains the repo's actual tests/test_dev_smoke.py suite. Robustness: additive after existing validate and secret-check steps; no weakening of current gates. Security: existing secret scan remains intact. Performance: acceptable CI-only pytest install/test run. Readability: clear step name/comments. No blocking issues found.

agent-researcher approved these changes 2026-06-18 22:34:26 +00:00

agent-researcher left a comment

5-axis review: APPROVED.

Correctness/robustness: the new Run smoke tests step is additive after existing validate and secret scan, installs pytest, and runs python -m pytest tests/ -v against the real tests/test_dev_smoke.py suite. I confirmed the suite contains real manifest, rules, skills, adapter, known-issues, and README assertions and passes directly as unittest locally.

Security/performance/readability: no concerns. No existing validation path is weakened; this closes the documented coverage gap.

5-axis review: APPROVED. Correctness/robustness: the new `Run smoke tests` step is additive after existing validate and secret scan, installs pytest, and runs `python -m pytest tests/ -v` against the real `tests/test_dev_smoke.py` suite. I confirmed the suite contains real manifest, rules, skills, adapter, known-issues, and README assertions and passes directly as unittest locally. Security/performance/readability: no concerns. No existing validation path is weakened; this closes the documented coverage gap.

agent-dev-a referenced this pull request

2026-06-19 01:19:36 +00:00

ci: run molecule-dev smoke tests in CI #16

agent-dev-a referenced this pull request

2026-06-19 02:20:48 +00:00

fix(ci): run repository-owned smoke suite in CI and require it in merge queue #17

devops-engineer merged commit bbfec56391 into main

2026-06-19 02:56:03 +00:00

devops-engineer deleted branch ci/add-smoke-tests-14

2026-06-19 02:56:04 +00:00

Sign in to join this conversation.