test(e2e): wire google-adk into the runtime e2e suite #2012

Merged

hongming merged 2 commits from e2e/google-adk-ci-wiring into main 2026-05-29 23:17:03 +00:00

Conversation 2 Commits 2 Files Changed 3

+48 -8

hongming commented 2026-05-29 21:18:23 +00:00

Owner

Copy Link

Copy Source

What

Wire the google-adk runtime into the e2e suite so it is continuously exercised like the other runtimes. google-adk was fully registered (manifest, provisioner knownRuntimes, canvas, CP runtimeImagePinTemplates + allowedRuntimes) and its image is published + pinned — but it had zero e2e coverage. This closes that gap.

Changes

scripts/test-all-runtimes-a2a-e2e.sh (the all-runtimes A2A parity harness)

• 4 → 5 runtimes. google-adk added to: provision block, provider-key injection (GOOGLE_API_KEY → workspace secret), wait-online loop (already ${!WS_IDS[@]}), A2A round-trip (§4), session-continuity (§5 — ADK InMemorySessionService keyed on A2A context_id).
• SKIP_GOOGLE_ADK guard + a GOOGLE_API_KEY pre-req check, mirroring the existing SKIP_* / OPENROUTER_API_KEY patterns.

.gitea/workflows/e2e-staging-saas.yml + .gitea/workflows/continuous-synth-e2e.yml

• Add the google-adk) per-runtime LLM-key case (expects MOLECULE_STAGING_GOOGLE_API_KEY), the E2E_GOOGLE_API_KEY env wiring, and the gemini model slug. Identical dispatch-gated shape to the existing codex/hermes/langgraph cases (Gitea 1.22.6 drops workflow_dispatch.inputs, so these run via E2E_RUNTIME).

Auth note (deliberate)

PROD disallows API keys (org policy) → Gemini runs via Vertex AI + ADC there. CI uses the keyed AI-Studio path (config model google_genai:gemini-2.5-pro) because it needs no GCP service-account plumbing in the runner. Vertex stays the supported prod path; this is the CI-only keyed path. Documented in-file.

Follow-up (not in this PR)

• MOLECULE_STAGING_GOOGLE_API_KEY repo secret must be set for a green google-adk run (the workflow hard-fails clean with the missing-secret message until then — same as the other runtimes before their keys existed).
• A genuine provisioned-workspace run still needs a tenant on the new workspace-server image (#2003) + staging-CP google-adk pin. The canary fleet is documented as not running (canary-release.md), so that's a separate throwaway-tenant exercise — tracked separately.

🤖 Generated with Claude Code

## What Wire the `google-adk` runtime into the e2e suite so it is continuously exercised **like the other runtimes**. google-adk was fully registered (manifest, provisioner `knownRuntimes`, canvas, CP `runtimeImagePinTemplates` + `allowedRuntimes`) and its image is published + pinned — but it had **zero e2e coverage**. This closes that gap. ## Changes **`scripts/test-all-runtimes-a2a-e2e.sh`** (the all-runtimes A2A parity harness) - 4 → 5 runtimes. google-adk added to: provision block, provider-key injection (`GOOGLE_API_KEY` → workspace secret), wait-online loop (already `${!WS_IDS[@]}`), A2A round-trip (§4), session-continuity (§5 — ADK `InMemorySessionService` keyed on A2A `context_id`). - `SKIP_GOOGLE_ADK` guard + a `GOOGLE_API_KEY` pre-req check, mirroring the existing `SKIP_*` / `OPENROUTER_API_KEY` patterns. **`.gitea/workflows/e2e-staging-saas.yml`** + **`.gitea/workflows/continuous-synth-e2e.yml`** - Add the `google-adk)` per-runtime LLM-key case (expects `MOLECULE_STAGING_GOOGLE_API_KEY`), the `E2E_GOOGLE_API_KEY` env wiring, and the gemini model slug. Identical dispatch-gated shape to the existing `codex`/`hermes`/`langgraph` cases (Gitea 1.22.6 drops `workflow_dispatch.inputs`, so these run via `E2E_RUNTIME`). ## Auth note (deliberate) PROD disallows API keys (org policy) → Gemini runs via **Vertex AI + ADC** there. **CI** uses the keyed **AI-Studio** path (config model `google_genai:gemini-2.5-pro`) because it needs no GCP service-account plumbing in the runner. Vertex stays the supported prod path; this is the CI-only keyed path. Documented in-file. ## Follow-up (not in this PR) - `MOLECULE_STAGING_GOOGLE_API_KEY` repo secret must be set for a green google-adk run (the workflow hard-fails clean with the missing-secret message until then — same as the other runtimes before their keys existed). - A genuine provisioned-workspace run still needs a tenant on the new workspace-server image (#2003) + staging-CP google-adk pin. The canary fleet is documented as **not running** (`canary-release.md`), so that's a separate throwaway-tenant exercise — tracked separately. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

hongming added 1 commit 2026-05-29 21:18:23 +00:00

test(e2e): wire google-adk into the runtime e2e suite ... 1ee864d523

google-adk was registered (manifest, provisioner, canvas, CP pin +
allowlist) but had no e2e coverage. Add it everywhere the other
runtimes sit so it is exercised "like other runtimes":

- scripts/test-all-runtimes-a2a-e2e.sh: provision + provider-key +
  online + A2A round-trip + session-continuity loops now include
  google-adk (5 runtimes). AI-Studio key via GOOGLE_API_KEY → workspace
  secret; SKIP_GOOGLE_ADK guard mirrors the other SKIP_* flags.
- e2e-staging-saas.yml + continuous-synth-e2e.yml: add the
  `google-adk)` per-runtime LLM-key case (expects
  MOLECULE_STAGING_GOOGLE_API_KEY) + E2E_GOOGLE_API_KEY env + the
  gemini model slug. Same dispatch-gated shape as codex/hermes/langgraph
  (Gitea drops workflow_dispatch.inputs, so E2E_RUNTIME-driven).

Auth note: PROD disallows API keys (Vertex+ADC there); CI uses the
keyed AI-Studio path (config model google_genai:gemini-2.5-pro). Vertex
stays the supported prod path. The MOLECULE_STAGING_GOOGLE_API_KEY
secret must be set for a green google-adk run (documented in-file).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

hongming added 1 commit 2026-05-29 21:52:00 +00:00

test(e2e): give google-adk a hermes-class online window ... 947cc730ba

First cold boot of a google-adk workspace pulls a large fresh ADK image;
the default 300s online wait can read a slow first pull as "failed".
Bump google-adk's wait to 180 iters (900s), matching the rationale for
hermes' extended window. No behavior change for other runtimes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

core-devops approved these changes 2026-05-29 23:02:26 +00:00

core-devops left a comment

Member

Copy Link

Copy Source

Review: core-devops (independent — not the author)

APPROVED. Genuine review of the google-adk e2e test-wiring. Findings:

(a) Harness bash (scripts/test-all-runtimes-a2a-e2e.sh) — correct and pattern-consistent.

• bash -n clean; CI "Shellcheck (E2E scripts)" is green.
• GOOGLE_ADK_KEY="${GOOGLE_API_KEY:-}" mirrors PEER_OPENAI_KEY. Provision (provision "ParityGoogleADK" "google-adk" ...), key-injection (set_secret … GOOGLE_API_KEY, double-guarded on WS id + key non-empty), and the §4/§5 loop additions all follow the openclaw/claude-code shape exactly.
• The pre-req guard (lines 57-60) is correctly a separate if keyed on SKIP_GOOGLE_ADK alone — right granularity, since google-adk has its own key rather than sharing the OpenRouter key that the hermes/codex/openclaw OR-guard covers.
• wait-online max=180 for the first cold ADK pull is reasonable (hermes is 120); loop is already ${!WS_IDS[@]}-driven.
• Note (pre-existing, not introduced here): §5 session-continuity asserts "carol" for every runtime including the chat-completions ones that forget between turns — that loop already listed claude-code/codex/openclaw before this PR; google-adk's adapter is expected to retain via InMemorySessionService. Not a defect in this change.

(b) Workflow YAML (e2e-staging-saas.yml + continuous-synth-e2e.yml) — valid + consistent.

• The google-adk) LLM-key case (MOLECULE_STAGING_GOOGLE_API_KEY → E2E_GOOGLE_API_KEY) is byte-identical in shape to the existing codex/hermes cases.
• The E2E_MODEL_SLUG ternary extension is correct: verified in the live run it evaluated to MiniMax-M2 on the default (no-input) path, so the new google-adk branch only fires under E2E_RUNTIME=google-adk.
• "Lint workflow YAML (Gitea-1.22.6-hostile shapes)" is green — no YAML break, no forbidden workflow_dispatch.inputs shape.

(c) Auth model — consistent in the diff and commit body across all three sites: PROD = Vertex AI + ADC (org disallows API keys); CI = keyed AI-Studio path with config model google_genai:gemini-2.5-pro. The workspace secret written is GOOGLE_API_KEY, matching the slug.

CI combined state = failure, but no failure is attributable to this PR:

• qa-review/approved + security-review/approved: normal pre-review red gates (flip to success on approval).
• lint-continue-on-error-tracking: repo-wide — 37 violations across 57 workflow files because tracker mc#774 is now 16 days old (>14d cap). Flags files this PR never touched; the diff adds zero continue-on-error directives. Pre-existing, blocks every open PR until mc#774 is renewed. Not this PR's defect.
• E2E Staging SaaS (full lifecycle): ran on the default claude-code / MiniMax-M2 path (no dispatch input), not google-adk. It failed on a staging-infra flake — workspace 31c8a8cf… "transiently failed — bootstrap-watcher deadline (cp#245)". Unrelated to this wiring.

Follow-up (not blocking, as the PR notes): a fully-green google-adk staging run additionally needs the google-adk template image in the staging ECR account + a staging-CP google-adk pin (#2003 workspace-server image + canary tenant). Confirmed MOLECULE_STAGING_GOOGLE_API_KEY IS already set in the runner env (evaluated to a non-empty secret in the live run), so that follow-up item is satisfied.

Test-only change, no production code touched. Approving.

**Review: core-devops (independent — not the author)** APPROVED. Genuine review of the google-adk e2e test-wiring. Findings: **(a) Harness bash (scripts/test-all-runtimes-a2a-e2e.sh)** — correct and pattern-consistent. - `bash -n` clean; CI "Shellcheck (E2E scripts)" is green. - `GOOGLE_ADK_KEY="${GOOGLE_API_KEY:-}"` mirrors `PEER_OPENAI_KEY`. Provision (`provision "ParityGoogleADK" "google-adk" ...`), key-injection (`set_secret … GOOGLE_API_KEY`, double-guarded on WS id + key non-empty), and the §4/§5 loop additions all follow the openclaw/claude-code shape exactly. - The pre-req guard (lines 57-60) is correctly a *separate* `if` keyed on `SKIP_GOOGLE_ADK` alone — right granularity, since google-adk has its own key rather than sharing the OpenRouter key that the hermes/codex/openclaw OR-guard covers. - wait-online `max=180` for the first cold ADK pull is reasonable (hermes is 120); loop is already `${!WS_IDS[@]}`-driven. - Note (pre-existing, not introduced here): §5 session-continuity asserts "carol" for every runtime including the chat-completions ones that forget between turns — that loop already listed claude-code/codex/openclaw before this PR; google-adk's adapter is expected to retain via InMemorySessionService. Not a defect in this change. **(b) Workflow YAML (e2e-staging-saas.yml + continuous-synth-e2e.yml)** — valid + consistent. - The `google-adk)` LLM-key case (`MOLECULE_STAGING_GOOGLE_API_KEY` → `E2E_GOOGLE_API_KEY`) is byte-identical in shape to the existing codex/hermes cases. - The `E2E_MODEL_SLUG` ternary extension is correct: verified in the live run it evaluated to `MiniMax-M2` on the default (no-input) path, so the new `google-adk` branch only fires under `E2E_RUNTIME=google-adk`. - "Lint workflow YAML (Gitea-1.22.6-hostile shapes)" is green — no YAML break, no forbidden `workflow_dispatch.inputs` shape. **(c) Auth model** — consistent in the diff and commit body across all three sites: PROD = Vertex AI + ADC (org disallows API keys); CI = keyed AI-Studio path with config model `google_genai:gemini-2.5-pro`. The workspace secret written is `GOOGLE_API_KEY`, matching the slug. **CI combined state = failure, but no failure is attributable to this PR:** - `qa-review/approved` + `security-review/approved`: normal pre-review red gates (flip to success on approval). - `lint-continue-on-error-tracking`: repo-wide — 37 violations across 57 workflow files because tracker `mc#774` is now 16 days old (>14d cap). Flags files this PR never touched; the diff adds **zero** `continue-on-error` directives. Pre-existing, blocks every open PR until mc#774 is renewed. Not this PR's defect. - `E2E Staging SaaS (full lifecycle)`: ran on the **default claude-code / MiniMax-M2** path (no dispatch input), not google-adk. It failed on a staging-infra flake — workspace `31c8a8cf…` "transiently failed — bootstrap-watcher deadline (cp#245)". Unrelated to this wiring. **Follow-up (not blocking, as the PR notes):** a fully-green *google-adk* staging run additionally needs the google-adk template image in the staging ECR account + a staging-CP google-adk pin (#2003 workspace-server image + canary tenant). Confirmed `MOLECULE_STAGING_GOOGLE_API_KEY` IS already set in the runner env (evaluated to a non-empty secret in the live run), so that follow-up item is satisfied. Test-only change, no production code touched. Approving.

core-qa approved these changes 2026-05-29 23:16:31 +00:00

core-qa left a comment

Member

Copy Link

Copy Source

Independent 2nd review (QA/coverage lens) — APPROVED

Reviewed the full PR diff against base, plus the script at HEAD 947cc73 and both workflow files. Test-only change; wires google-adk into the runtime E2E suite mirroring the codex/hermes/langgraph pattern.

(a) Consistency — coverage fully wired, not half:
google-adk is present at every site the other runtimes appear in scripts/test-all-runtimes-a2a-e2e.sh:

• pre-req guard + GOOGLE_ADK_KEY capture (L38, L57)
• provision (WS_IDS[google-adk], L176)
• key injection (set_secret … GOOGLE_API_KEY, guarded by WS-id presence, L194-198)
• online-wait loop (auto-included via ${!WS_IDS[@]}, with the 180-iter cold-pull override at L213)
• A2A round-trip loop (L226) and session-continuity loop (L239)
  No gaps.

(b) Guard correctness:
if [ -z "$GOOGLE_ADK_KEY" ] && [ -z "${SKIP_GOOGLE_ADK:-}" ]; then exit 2 — hard-fails loudly when the key is absent and the runtime is not skipped (no silent pass), and is cleanly bypassed by SKIP_GOOGLE_ADK=1 (no spurious fail). The key-injection block is independently guarded by [ -n "${WS_IDS[google-adk]:-}" ], so it no-ops under skip. Correct.

(c) Workflow per-runtime auth cases:
Both continuous-synth-e2e.yml and e2e-staging-saas.yml add a google-adk) case with the correct secret MOLECULE_STAGING_GOOGLE_API_KEY → E2E_GOOGLE_API_KEY, properly ;;-terminated, matching the codex/hermes shape. Both files YAML-parse clean. The E2E_MODEL_SLUG ternary correctly appends google-adk && 'google_genai:gemini-2.5-pro' ahead of the default — verified in the live run log it still evaluates to MiniMax-M2 on the default claude-code path (no regression to the default). Note (non-blocking): workflow_dispatch.inputs are intentionally dropped in the Gitea port, so github.event.inputs.runtime is inert under Gitea today and the google-adk branch is currently only reachable if inputs return — but this exactly mirrors the existing (also-inert) codex/hermes branches, and the author flags it in the inline comment. Consistent with precedent.

(d) Session-continuity assertion justified:
google-adk genuinely preserves cross-turn state via ADK InMemorySessionService keyed on the A2A context_id, so the carol-recall PASS-expectation is legitimate (parity with the hermes plugin path), not a coin-flip. Justified.

Required checks on 947cc73:

• CI / all-required (pull_request) — success ✓
• E2E API Smoke Test / E2E API Smoke Test (pull_request) — success ✓
• Handlers Postgres Integration / Handlers Postgres Integration (pull_request) — success ✓

Non-blocking reds confirmed harmless: lint-continue-on-error-tracking is the known pre-existing mc#774-age failure on untouched files; E2E Staging SaaS (full lifecycle) / E2E Staging SaaS is continue-on-error: true (RFC#219 §1, not the PR gate — pr-validate is, and it is green) and its failure is an unrelated bootstrap-watcher transient on the default claude-code/MiniMax path (cp#245), not the new google-adk wiring; qa-review/security-review are review-gate sentinels that flip green once posted.

Sound and well-scoped. Approving on the QA/coverage axis. Not merging.

**Independent 2nd review (QA/coverage lens) — APPROVED** Reviewed the full PR diff against base, plus the script at HEAD `947cc73` and both workflow files. Test-only change; wires `google-adk` into the runtime E2E suite mirroring the codex/hermes/langgraph pattern. **(a) Consistency — coverage fully wired, not half:** google-adk is present at every site the other runtimes appear in `scripts/test-all-runtimes-a2a-e2e.sh`: - pre-req guard + `GOOGLE_ADK_KEY` capture (L38, L57) - provision (`WS_IDS[google-adk]`, L176) - key injection (`set_secret … GOOGLE_API_KEY`, guarded by WS-id presence, L194-198) - online-wait loop (auto-included via `${!WS_IDS[@]}`, with the 180-iter cold-pull override at L213) - A2A round-trip loop (L226) and session-continuity loop (L239) No gaps. **(b) Guard correctness:** `if [ -z "$GOOGLE_ADK_KEY" ] && [ -z "${SKIP_GOOGLE_ADK:-}" ]; then exit 2` — hard-fails loudly when the key is absent and the runtime is not skipped (no silent pass), and is cleanly bypassed by `SKIP_GOOGLE_ADK=1` (no spurious fail). The key-injection block is independently guarded by `[ -n "${WS_IDS[google-adk]:-}" ]`, so it no-ops under skip. Correct. **(c) Workflow per-runtime auth cases:** Both `continuous-synth-e2e.yml` and `e2e-staging-saas.yml` add a `google-adk)` case with the correct secret `MOLECULE_STAGING_GOOGLE_API_KEY` → `E2E_GOOGLE_API_KEY`, properly `;;`-terminated, matching the codex/hermes shape. Both files YAML-parse clean. The `E2E_MODEL_SLUG` ternary correctly appends `google-adk && 'google_genai:gemini-2.5-pro'` ahead of the default — verified in the live run log it still evaluates to `MiniMax-M2` on the default claude-code path (no regression to the default). Note (non-blocking): `workflow_dispatch.inputs` are intentionally dropped in the Gitea port, so `github.event.inputs.runtime` is inert under Gitea today and the google-adk branch is currently only reachable if inputs return — but this exactly mirrors the existing (also-inert) codex/hermes branches, and the author flags it in the inline comment. Consistent with precedent. **(d) Session-continuity assertion justified:** google-adk genuinely preserves cross-turn state via ADK `InMemorySessionService` keyed on the A2A `context_id`, so the `carol`-recall PASS-expectation is legitimate (parity with the hermes plugin path), not a coin-flip. Justified. **Required checks on 947cc73:** - `CI / all-required (pull_request)` — success ✓ - `E2E API Smoke Test / E2E API Smoke Test (pull_request)` — success ✓ - `Handlers Postgres Integration / Handlers Postgres Integration (pull_request)` — success ✓ Non-blocking reds confirmed harmless: `lint-continue-on-error-tracking` is the known pre-existing mc#774-age failure on untouched files; `E2E Staging SaaS (full lifecycle) / E2E Staging SaaS` is `continue-on-error: true` (RFC#219 §1, not the PR gate — `pr-validate` is, and it is green) and its failure is an unrelated bootstrap-watcher transient on the default claude-code/MiniMax path (cp#245), not the new google-adk wiring; `qa-review`/`security-review` are review-gate sentinels that flip green once posted. Sound and well-scoped. Approving on the QA/coverage axis. Not merging.

hongming merged commit 82bc28a098 into main 2026-05-29 23:17:03 +00:00

hongming referenced this issue from a commit 2026-05-29 23:17:05 +00:00

Merge pull request 'test(e2e): wire google-adk into the runtime e2e suite' (#2012) from e2e/google-adk-ci-wiring into main

Sign in to join this conversation.

Reviewers

No Reviewers

core-devops

core-qa

Labels

Clear labels

area/ci

CI/CD pipeline issues

kind/infrastructure

Infrastructure-related issues

merge-queue

Ready for serialized Gitea merge queue

merge-queue-hold

Temporarily hold PR in merge queue

platform/go

Go platform test issues

release-blocker

Blocks the staging→main promotion / a release

release-test

security

test-label-sre

tier:high

High risk per dev-sop §SOP-6 — ceo only, 24h cooldown

tier:low

Low risk per dev-sop §SOP-6 — engineers/managers/ceo can approve

tier:medium

Medium risk per dev-sop §SOP-6 — managers/ceo can approve

triage-test

test

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

agent-dev-a agent-dev-b agent-pm agent-researcher agent-reviewer agent-reviewer-cr2 app-fe (Molecule AI · app-fe) app-lead (Molecule AI · app-lead) app-qa (Molecule AI · app-qa) claude-ceo-assistant claude-ci-reader claude-status-reaper core-be (Molecule AI · core-be) core-devops (Molecule AI · core-devops) core-fe (Molecule AI · core-fe) core-lead (Molecule AI · core-lead) core-offsec (Molecule AI · core-offsec) core-qa (Molecule AI · core-qa) core-security (Molecule AI · core-security) core-uiux (Molecule AI · core-uiux) cp-be (Molecule AI · cp-be) cp-lead (Molecule AI · cp-lead) cp-qa (Molecule AI · cp-qa) cp-security (Molecule AI · cp-security) cui (Zhanlin Cui) dev-lead (Molecule AI · dev-lead) devops-engineer documentation-specialist (Molecule AI · documentation-specialist) fullstack-engineer (Molecule AI · fullstack-engineer) hongming hongming-ceo-delegated hongming-codex-laptop hongming-kimi-laptop hongming-pc2 infra-lead (Molecule AI · infra-lead) infra-runtime-be (Molecule AI · infra-runtime-be) infra-sre (Molecule AI · infra-sre) integration-tester (Molecule AI · integration-tester) mc-drift-bot molecule-code-reviewer plugin-dev (Molecule AI · plugin-dev) pm publish-runtime-bot pypi-publisher (Molecule AI PyPI Publisher (RFC#596)) release-manager (Molecule AI · release-manager) sdk-dev (Molecule AI · sdk-dev) sdk-lead (Molecule AI · sdk-lead) sop-drift-bot sop-tier-bot (SOP Tier-Check Bot) technical-writer (Molecule AI · technical-writer) triage-operator (Molecule AI · triage-operator)

No Assignees

3 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: molecule-ai/molecule-core#2012