molecule-ai/molecule-core
52
0
Fork 2
Code Issues 184 Pull Requests 16 Actions 1 Packages Projects Releases Wiki Activity

[DRAFT] obs/infra: Railway CP→Loki drain + tenant SSM:SendCommand (#3214) #3221

Closed
agent-dev-a wants to merge 1 commits from draft/obs-railway-loki-drain-3214 into main
pull from: draft/obs-railway-loki-drain-3214
merge into: molecule-ai:main
Conversation 0 Commits 1 Files Changed 3
+227
agent-dev-a commented 2026-06-24 08:42:18 +00:00
Member
Copy Link
Copy Source

DRAFT-ONLY — do NOT merge.

Closes #3214 (owner/infra execution still required).

This PR provides the configuration and policy artifacts the owner needs to close the observability gap described in #3214:

  • infra/obs/railway-loki-drain.alloy — Alloy configuration to receive Railway control-plane logs and ship them to the existing Grafana Cloud Loki stack (ds P8E80F9AEF21F6940).
  • infra/obs/tenant-ssm-sendcommand-policy.json — IAM policy granting the fleet/operator identity ssm:SendCommand (and related actions) on production tenant instances tagged MoleculeTenant=true.
  • infra/obs/WIRING_CHECKLIST.md — step-by-step owner checklist covering Grafana Cloud credentials, Alloy deployment, Railway log-drain wiring, tenant log-shipper reconciliation, and SSM rollout.

Head SHA

822ba81cb62997a75d234b29c4e2d318c95f988c

Next step

Owner reviews the checklist, fills in the actual Loki endpoint/credentials, deploys Alloy, wires the Railway drain, attaches the IAM policy, and verifies end-to-end ingestion + SSM access. Once the infra changes are applied, this draft can be closed or converted to a docs-only merge.

DRAFT-ONLY — do NOT merge. Closes #3214 (owner/infra execution still required). This PR provides the configuration and policy artifacts the owner needs to close the observability gap described in #3214: - `infra/obs/railway-loki-drain.alloy` — Alloy configuration to receive Railway control-plane logs and ship them to the existing Grafana Cloud Loki stack (ds `P8E80F9AEF21F6940`). - `infra/obs/tenant-ssm-sendcommand-policy.json` — IAM policy granting the fleet/operator identity `ssm:SendCommand` (and related actions) on production tenant instances tagged `MoleculeTenant=true`. - `infra/obs/WIRING_CHECKLIST.md` — step-by-step owner checklist covering Grafana Cloud credentials, Alloy deployment, Railway log-drain wiring, tenant log-shipper reconciliation, and SSM rollout. ## Head SHA `822ba81cb62997a75d234b29c4e2d318c95f988c` ## Next step Owner reviews the checklist, fills in the actual Loki endpoint/credentials, deploys Alloy, wires the Railway drain, attaches the IAM policy, and verifies end-to-end ingestion + SSM access. Once the infra changes are applied, this draft can be closed or converted to a docs-only merge.
agent-dev-a added 1 commit 2026-06-24 08:42:18 +00:00
draft(obs): Railway CP→Loki drain config + tenant SSM:SendCommand policy + wiring checklist (#3214)
CI / Python Lint & Test (pull_request) Successful in 6s
Details
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 7s
Details
E2E Peer Visibility (literal MCP list_peers) / detect-changes (pull_request) Successful in 6s
Details
Block integration-tester contamination artifacts / Block staging-trigger / invalid manifest contamination (pull_request) Successful in 9s
Details
Handlers Postgres Integration / detect-changes (pull_request) Successful in 8s
Details
Lint forbidden tenant-env keys / Scan for repo-host token write into tenant workspace surface (pull_request) Successful in 8s
Details
E2E Peer Visibility (literal MCP list_peers) / E2E Peer Visibility (local) (pull_request) Has been skipped
Details
Lint forbidden tenant-env keys / Scan workspace_secrets writers for forbidden env keys (pull_request) Successful in 10s
Details
E2E Peer Visibility (literal MCP list_peers) / E2E Peer Visibility (pull_request) Successful in 7s
Details
E2E Chat / detect-changes (pull_request) Successful in 16s
Details
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 12s
Details
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 4s
Details
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 19s
Details
lint-required-no-paths / lint-required-no-paths (pull_request) Successful in 18s
Details
E2E API Smoke Test / detect-changes (pull_request) Successful in 22s
Details
PR Diff Guard / PR diff guard (pull_request) Successful in 18s
Details
sop-checklist / review-refire (pull_request_target) Has been skipped
Details
qa-review / approved (pull_request_target) Failing after 14s
Details
CI / Detect changes (pull_request) Successful in 28s
Details
E2E Chat / E2E Chat (pull_request) Successful in 6s
Details
template-delivery-e2e / detect-changes (pull_request) Successful in 22s
Details
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 5s
Details
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 4s
Details
reserved-path-review / reserved-path-review (pull_request_target) Successful in 16s
Details
sop-checklist / all-items-acked (pull_request) acked: 0/9 — missing: comprehensive-testing, local-postgres-e2e, staging-smoke, +6 — body-unfilled: comprehensive-testing, local-postgres-e2
Details
CI / Shellcheck (E2E scripts) (pull_request) Successful in 2s
Details
sop-checklist / na-declarations (pull_request) N/A: (none)
Details
CI / Platform (Go) (pull_request) Successful in 5s
Details
template-delivery-e2e / Template-asset delivery (fresh seo-agent — config+prompts via asset channel, seo-all via plugin reconcile) (pull_request) Successful in 2s
Details
CI / Canvas (Next.js) (pull_request) Successful in 3s
Details
security-review / approved (pull_request_target) Failing after 18s
Details
gate-check-v3 / gate-check (pull_request_target) Failing after 25s
Details
sop-checklist / all-items-acked (pull_request_target) Successful in 15s
Details
CI / Canvas Deploy Status (pull_request) Successful in 1s
Details
CI / all-required (pull_request) Successful in 3s
Details
Local Provision Lifecycle E2E / Local Provision Lifecycle E2E (stub) (pull_request) Successful in 42s
Details
Local Provision Lifecycle E2E / Local Provision Lifecycle E2E (real image + MiniMax LLM, advisory) (pull_request) Successful in 2m9s
Details
E2E Staging SaaS (full lifecycle) / pr-validate (pull_request) Waiting to run
Details
E2E Staging SaaS (full lifecycle) / E2E Staging SaaS (pull_request) Waiting to run
Details
E2E Staging SaaS (full lifecycle) / Prune stale e2e DNS records (pull_request) Blocked by required conditions
Details
E2E Staging SaaS (full lifecycle) / E2E Staging Platform Boot (pull_request) Waiting to run
Details
E2E Staging SaaS (full lifecycle) / E2E Staging Concierge user_tasks (pull_request) Waiting to run
Details
E2E Staging SaaS (full lifecycle) / E2E Staging Workspace Requests (core#2606) (pull_request) Waiting to run
Details
E2E Staging SaaS (full lifecycle) / E2E Staging Concierge Creates Workspace (pull_request) Waiting to run
Details
E2E Staging SaaS (full lifecycle) / E2E Staging Concierge (compile+skip) (pull_request) Waiting to run
Details
E2E Staging SaaS (full lifecycle) / E2E Staging Concierge Platform Agent (pull_request) Waiting to run
Details
E2E Staging SaaS (full lifecycle) / E2E Staging Plugin Install Lifecycle (pull_request) Waiting to run
Details
audit-force-merge / audit (pull_request_target) Has been skipped
Details
822ba81cb6 
DRAFT-ONLY — owner/infra execution required.

- infra/obs/railway-loki-drain.alloy: Alloy config to receive Railway logs
  and ship to Grafana Cloud Loki (ds P8E80F9AEF21F6940).
- infra/obs/tenant-ssm-sendcommand-policy.json: IAM policy granting the
  fleet/operator identity ssm:SendCommand on tagged tenant instances.
- infra/obs/WIRING_CHECKLIST.md: step-by-step owner checklist for
  credentials, deployment, Railway drain wiring, and SSM rollout.

Co-Authored-By: Claude <noreply@anthropic.com>
agent-dev-a closed this pull request 2026-06-24 11:50:42 +00:00
Some required checks failed
CI / Python Lint & Test (pull_request) Successful in 6s
Details
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 7s
Details
E2E Peer Visibility (literal MCP list_peers) / detect-changes (pull_request) Successful in 6s
Details
Block integration-tester contamination artifacts / Block staging-trigger / invalid manifest contamination (pull_request) Successful in 9s
Details
Handlers Postgres Integration / detect-changes (pull_request) Successful in 8s
Details
Lint forbidden tenant-env keys / Scan for repo-host token write into tenant workspace surface (pull_request) Successful in 8s
Details
E2E Peer Visibility (literal MCP list_peers) / E2E Peer Visibility (local) (pull_request) Has been skipped
Details
Lint forbidden tenant-env keys / Scan workspace_secrets writers for forbidden env keys (pull_request) Successful in 10s
Details
E2E Peer Visibility (literal MCP list_peers) / E2E Peer Visibility (pull_request) Successful in 7s
Details
E2E Chat / detect-changes (pull_request) Successful in 16s
Details
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 12s
Required
Details
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 4s
Required
Details
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 19s
Details
lint-required-no-paths / lint-required-no-paths (pull_request) Successful in 18s
Details
E2E API Smoke Test / detect-changes (pull_request) Successful in 22s
Details
PR Diff Guard / PR diff guard (pull_request) Successful in 18s
Details
sop-checklist / review-refire (pull_request_target) Has been skipped
Details
qa-review / approved (pull_request_target) Failing after 14s
Required
Details
CI / Detect changes (pull_request) Successful in 28s
Details
E2E Chat / E2E Chat (pull_request) Successful in 6s
Details
template-delivery-e2e / detect-changes (pull_request) Successful in 22s
Details
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 5s
Details
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 4s
Required
Details
reserved-path-review / reserved-path-review (pull_request_target) Successful in 16s
Required
Details
sop-checklist / all-items-acked (pull_request) acked: 0/9 — missing: comprehensive-testing, local-postgres-e2e, staging-smoke, +6 — body-unfilled: comprehensive-testing, local-postgres-e2
Details
CI / Shellcheck (E2E scripts) (pull_request) Successful in 2s
Details
sop-checklist / na-declarations (pull_request) N/A: (none)
Details
CI / Platform (Go) (pull_request) Successful in 5s
Details
template-delivery-e2e / Template-asset delivery (fresh seo-agent — config+prompts via asset channel, seo-all via plugin reconcile) (pull_request) Successful in 2s
Details
CI / Canvas (Next.js) (pull_request) Successful in 3s
Details
security-review / approved (pull_request_target) Failing after 18s
Required
Details
gate-check-v3 / gate-check (pull_request_target) Failing after 25s
Details
sop-checklist / all-items-acked (pull_request_target) Successful in 15s
Details
CI / Canvas Deploy Status (pull_request) Successful in 1s
Details
CI / all-required (pull_request) Successful in 3s
Required
Details
Local Provision Lifecycle E2E / Local Provision Lifecycle E2E (stub) (pull_request) Successful in 42s
Details
Local Provision Lifecycle E2E / Local Provision Lifecycle E2E (real image + MiniMax LLM, advisory) (pull_request) Successful in 2m9s
Details
E2E Staging SaaS (full lifecycle) / pr-validate (pull_request) Waiting to run
Details
E2E Staging SaaS (full lifecycle) / E2E Staging SaaS (pull_request) Waiting to run
Details
E2E Staging SaaS (full lifecycle) / Prune stale e2e DNS records (pull_request) Blocked by required conditions
Details
E2E Staging SaaS (full lifecycle) / E2E Staging Platform Boot (pull_request) Waiting to run
Details
E2E Staging SaaS (full lifecycle) / E2E Staging Concierge user_tasks (pull_request) Waiting to run
Details
E2E Staging SaaS (full lifecycle) / E2E Staging Workspace Requests (core#2606) (pull_request) Waiting to run
Details
E2E Staging SaaS (full lifecycle) / E2E Staging Concierge Creates Workspace (pull_request) Waiting to run
Details
E2E Staging SaaS (full lifecycle) / E2E Staging Concierge (compile+skip) (pull_request) Waiting to run
Details
E2E Staging SaaS (full lifecycle) / E2E Staging Concierge Platform Agent (pull_request) Waiting to run
Details
E2E Staging SaaS (full lifecycle) / E2E Staging Plugin Install Lifecycle (pull_request) Waiting to run
Details
audit-force-merge / audit (pull_request_target) Has been skipped
Details

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
approved
area/ci
CI/CD pipeline issues
 do-not-auto-merge
Opt out of autonomous merge-queue merging
 do-not-merge
hold from auto-merge (design review in progress)
 kind/infrastructure
Infrastructure-related issues
 merge-queue
Ready for serialized Gitea merge queue
 merge-queue-hold
Temporarily hold PR in merge queue
 needs-engineer
platform/go
Go platform test issues
 ready-to-build
release-blocker
Blocks the staging→main promotion / a release
 release-test
security
test-label-sre
tier:high
High risk per dev-sop §SOP-6 — ceo only, 24h cooldown
 tier:low
Low risk per dev-sop §SOP-6 — engineers/managers/ceo can approve
 tier:medium
Medium risk per dev-sop §SOP-6 — managers/ceo can approve
 triage-test
test
 wip
Work in progress — do not auto-merge
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
agent-dev-a agent-dev-b agent-pm agent-researcher agent-reviewer agent-reviewer-1 agent-reviewer-cr2 app-fe (Molecule AI · app-fe) app-lead (Molecule AI · app-lead) app-qa (Molecule AI · app-qa) claude-ceo-assistant claude-ci-reader claude-status-reaper core-be (Molecule AI · core-be) core-devops (Molecule AI · core-devops) core-fe (Molecule AI · core-fe) core-lead (Molecule AI · core-lead) core-offsec (Molecule AI · core-offsec) core-qa (Molecule AI · core-qa) core-security (Molecule AI · core-security) core-uiux (Molecule AI · core-uiux) cp-be (Molecule AI · cp-be) cp-lead (Molecule AI · cp-lead) cp-qa (Molecule AI · cp-qa) cp-security (Molecule AI · cp-security) cui (Zhanlin Cui) dev-lead (Molecule AI · dev-lead) devops-engineer documentation-specialist (Molecule AI · documentation-specialist) fullstack-engineer (Molecule AI · fullstack-engineer) godwin hongming hongming-ceo-delegated hongming-codex-laptop hongming-kimi-laptop hongming-pc2 hongming-personal infra-lead (Molecule AI · infra-lead) infra-runtime-be (Molecule AI · infra-runtime-be) infra-sre (Molecule AI · infra-sre) integration-tester (Molecule AI · integration-tester) mc-drift-bot molecule-code-reviewer plugin-dev (Molecule AI · plugin-dev) pm publish-runtime-bot pypi-publisher (Molecule AI PyPI Publisher (RFC#596)) release-manager (Molecule AI · release-manager) sdk-dev (Molecule AI · sdk-dev) sdk-lead (Molecule AI · sdk-lead) sop-drift-bot sop-tier-bot (SOP Tier-Check Bot) technical-writer (Molecule AI · technical-writer) triage-operator (Molecule AI · triage-operator)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: molecule-ai/molecule-core#3221
Delete Branch "draft/obs-railway-loki-drain-3214"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?