Commit Graph

6 Commits

Author SHA1 Message Date
claude-ceo-assistant
3501e6bfd7 fix(post-suspension): vanity import paths go.moleculesai.app/core/{platform,tests/harness/cp-stub} (closes molecule-ai/internal#71 phase 2)
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (go) (pull_request) Successful in 13s
CodeQL / Analyze (${{ matrix.language }}) (python) (pull_request) Successful in 11s
CodeQL / Analyze (${{ matrix.language }}) (javascript-typescript) (pull_request) Successful in 12s
Check merge_group trigger on required workflows / Required workflows have merge_group trigger (pull_request) Successful in 15s
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 27s
CI / Detect changes (pull_request) Successful in 20s
Retarget main PRs to staging / Retarget to staging (pull_request) Has been skipped
Lint curl status-code capture / Scan workflows for curl status-capture pollution (pull_request) Successful in 15s
E2E API Smoke Test / detect-changes (pull_request) Successful in 51s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 51s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 39s
Handlers Postgres Integration / detect-changes (pull_request) Successful in 51s
Harness Replays / detect-changes (pull_request) Successful in 53s
Runtime PR-Built Compatibility / detect-changes (pull_request) Successful in 48s
Ops Scripts Tests / Ops scripts (unittest) (pull_request) Successful in 1m7s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 31s
Harness Replays / Harness Replays (pull_request) Failing after 1m18s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 2m19s
Runtime PR-Built Compatibility / PR-built wheel + import smoke (pull_request) Successful in 3m14s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 6m1s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 6m47s
CI / Python Lint & Test (pull_request) Successful in 8m16s
CI / Canvas (Next.js) (pull_request) Failing after 9m36s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
CI / Platform (Go) (pull_request) Successful in 12m18s
Migrates the two Go modules under molecule-core off the dead
github.com/Molecule-AI/molecule-monorepo/... identity onto the vanity
host go.moleculesai.app. Also fixes the historical naming
inconsistency where the Gitea repo is molecule-core but the Go module
path said molecule-monorepo.

Module changes:
- workspace-server/go.mod:
    github.com/Molecule-AI/molecule-monorepo/platform
    -> go.moleculesai.app/core/platform
- tests/harness/cp-stub/go.mod:
    github.com/Molecule-AI/molecule-monorepo/tests/harness/cp-stub
    -> go.moleculesai.app/core/tests/harness/cp-stub

Surfaces touched
- 174 *.go files (374 import lines) — every import under
  workspace-server/ + tests/harness/cp-stub/
- 2 Dockerfiles (workspace-server/Dockerfile + Dockerfile.tenant) —
  -ldflags strings updated in lockstep with the module rename so
  buildinfo.GitSHA injection still resolves correctly
- README + docs + scripts + comment URLs to git.moleculesai.app form
- NEW workspace-server/internal/lint/import_path_lint_test.go —
  structural lint gate rejecting future github.com/Molecule-AI/ or
  Molecule-AI/molecule-monorepo references. Identical template to the
  other migration PRs (plugin-gh-identity#3, molecule-cli#2,
  molecule-controlplane#32).

Cross-repo dep allowlist (documented in lint gate)
workspace-server requires molecule-ai-plugin-gh-identity, whose own
vanity migration is PR molecule-ai-plugin-gh-identity#3. Until that PR
merges + a tag is cut at go.moleculesai.app/plugin/gh-identity, the
two locations referencing the legacy github.com path
(workspace-server/go.mod require, cmd/server/main.go import) remain
allowlisted. Follow-up PR drops the allowlist + updates both refs in
one shot once gh-identity is fully migrated.

Test plan
- go build ./... clean for both modules
- go test ./... green except two pre-existing failures
  (TestStartSweeper_RecordsMetricsOnSuccess flaky-on-suite,
  TestLocalResolver_BubblesUpCopyFailure relies on read-only fs perms
  but runs as root on operator host) — both reproduce identically on
  baseline main pre-migration; NOT regressions of this PR
- Mutation-tested: lint gate fails on canaries in .go + .md;
  allowlist correctly suppresses cross-repo dep references in go.mod
  while still flagging unrelated additions

Open dependency
- go.moleculesai.app responder must be deployed before fresh-clone
  external builds resolve the vanity path. Existing CI / Docker builds
  ride pinned go.sum + self-referential module path + responder is
  not on critical path for those.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 22:37:42 +00:00
Hongming Wang
6201d12533 fix(memory-plugin): embed migrations into binary via go:embed
PR #2906 shipped the binary at /memory-plugin without the migrations
directory. The plugin's runMigrations() resolved a relative path
\`cmd/memory-plugin-postgres/migrations\` that exists in the build
context but NOT in the runtime image. Every staging tenant boot
failed with:

  memory-plugin-postgres: migrate: read migrations dir
    "cmd/memory-plugin-postgres/migrations": open
    cmd/memory-plugin-postgres/migrations: no such file or directory

  memory-plugin:  /v1/health never returned 200 after 30s
    — aborting boot

Caught on the staging redeploy fleet job after #2906 merged. Tenants
stayed on the old image (CP redeploy correctly fail-fasted) but the
new image was broken.

Fix: \`//go:embed migrations/*.up.sql\` bundles the migrations into
the binary at build time. No filesystem path dependency at runtime.

  * \`embed.FS\` embeds the .up.sql files alongside the binary.
  * runMigrations() reads from migrationsFS by default;
    MEMORY_PLUGIN_MIGRATIONS_DIR override path preserved for operators
    shipping custom migrations.
  * Names sorted alphabetically — pinned by a test so a future
    \`002_*.up.sql\` is guaranteed to run after \`001_*.up.sql\`.

Tests:
  * TestMigrationsEmbedded_ContainsCreateTable — pins that the embed
    pattern matched files AND those files contain CREATE TABLE
    (catches both empty-pattern and wrong-files-embedded).
  * TestRunMigrationsFromEmbed_OrderingIsAlphabetic — pins sorted
    application order.

Verified locally: \`go build\` succeeds, binary 9.3MB,
\`strings\` shows the embedded SQL.

Refs RFC #2728. Hotfix for #2906.
2026-05-05 11:57:37 -07:00
Hongming Wang
1052f8bdb0 fix(memory-plugin): bind to 127.0.0.1 by default
Self-review of PR #2906 flagged: defaultListenAddr was ":9100" — binds
on every container interface. Inside today's deployment that's moot
(no host port mapping, platform talks over loopback) but it's not
least-privilege. A future Dockerfile edit that publishes the port,
a misconfigured Fly machine, or a future cross-host plugin topology
would expose an unauth'd memory store.

Loopback is the right baseline. Operators with a multi-host topology
already override via MEMORY_PLUGIN_LISTEN_ADDR — that path is unchanged.

Tests:
  * TestLoadConfig_DefaultListenAddrIsLoopback pins the new default.
  * TestLoadConfig_ListenAddrEnvOverride pins the override path so
    operators relying on it don't break.
  * TestLoadConfig_MissingDatabaseURL covers the existing fail-fast.

No prior unit tests existed for loadConfig — boot_e2e_test.go always
sets MEMORY_PLUGIN_LISTEN_ADDR explicitly, so the default was never
exercised by tests. This PR adds that coverage.

Refs RFC #2728. Hardening follow-up to PR #2906.
2026-05-05 11:35:24 -07:00
Hongming Wang
fe7ff5440d Memory v2 fixup Opt-2: add E2E.md operator runbook
Companion to boot_e2e_test.go (just merged). Documents:
  - When the E2E suite runs (build tag + env var)
  - Local run with docker postgres
  - CI integration example (label-gated workflow step)
  - What each test pins
  - Explicit gap list (migration drift, recovery, TTL)
2026-05-04 09:24:16 -07:00
Hongming Wang
5b0a75ab73 Memory v2 fixup Optional-2: real-subprocess boot E2E
Self-review #293. PR-11's E2E test uses sqlmock + httptest —
integration, not E2E. This adds the actual real-subprocess test:
build the binary with `go build`, start it pointing at real postgres,
drive HTTP via the real client.

What in-process tests miss that this catches:
  - Binary build / boot-path panics (env var typos, mixed-key
    interface bugs that only surface when start() runs)
  - Wire encoding bugs that sqlmock smooths over (the pq.Array
    regression from PR-3 development would have been caught here)
  - HTTP+TCP-socket edge cases
  - Real upsert behavior under postgres ON CONFLICT (C1 fix)

Build-tag gated so default CI doesn't require docker:
  go test -tags memory_plugin_e2e -v ./cmd/memory-plugin-postgres/

Tests skip silently when MEMORY_PLUGIN_E2E_DB is unset.

Three tests:
  1. TestE2E_BootAndHealth — capabilities advertised correctly
  2. TestE2E_FullCommitSearchForgetRoundTrip — full agent flow
  3. TestE2E_IdempotencyKey — C1 upsert against real postgres

Plus E2E.md operator runbook with docker quickstart + CI integration
example + explicit statement of what's still uncovered (migration
drift, recovery scenarios, TTL eviction over real time).
2026-05-04 09:23:46 -07:00
Hongming Wang
ff5f4cbf7c Memory v2 PR-3: built-in postgres plugin server + schema migrations
Builds on merged PR-1 (#2729), independent of PR-2/PR-4.

Implements every endpoint of the v1 plugin contract behind an HTTP
server (cmd/memory-plugin-postgres/) backed by postgres. Operators
run this binary next to workspace-server; it's the default
implementation MEMORY_PLUGIN_URL points at.

What ships:
  - cmd/memory-plugin-postgres/main.go: boot, signal-driven shutdown,
    boot-time migrations, configurable LISTEN/DATABASE/MIGRATION_DIR
  - cmd/memory-plugin-postgres/migrations/001_memory_v2.up.sql:
      memory_namespaces (PK on name, kind CHECK, expires_at, metadata)
      memory_records (FK to namespaces with CASCADE, kind+source CHECK,
                      pgvector embedding, FTS tsvector, ivfflat partial
                      index on embedding, partial index on expires_at)
  - internal/memory/pgplugin/store.go: storage layer using lib/pq
  - internal/memory/pgplugin/handlers.go: HTTP layer (no router dep —
    a switch on URL.Path keeps the binary's dep surface tiny)
  - 100% statement coverage on store.go + handlers.go

Schema notes:
  - These tables live next to the plugin binary, NOT in workspace-
    server/migrations/. When operators swap the plugin, these tables
    become orphaned (operator drops manually). Documented in PR-10.
  - Search supports semantic (pgvector cosine) → FTS (>=2 char query)
    → ILIKE (1-char query) → recent-listing (no query), with a TTL
    filter applied uniformly across all paths.
  - DELETE on namespace cascades to memory_records (FK ON DELETE
    CASCADE) — a deleted namespace immediately frees its memories.

Coverage corner cases pinned:
  - Health: ok, degraded (db ping fails), no-ping fn
  - Every CRUD endpoint: happy path, bad name, bad JSON, bad body,
    not-found, store errors, exec/scan/marshal errors
  - Search: FTS, semantic, short-query (ILIKE), no-query (recent),
    kinds filter, store errors, scan errors, mid-iteration row error
  - Routing edge cases: unknown path, empty namespace, unknown sub,
    method-not-allowed, GET on /v1/health (allowed), POST on /v1/health
    (404), GET on /v1/search (404)
  - Helper internals: marshalMetadata (nil/happy/unmarshalable),
    nullTime (nil/non-nil), vectorString (empty/format),
    nullVectorString (empty/non-empty), scanNamespace +
    scanMemory metadata-decode errors

No callers in workspace-server yet; integration starts in PR-5
(MCP handlers wire the plugin client through to MCP tools).
2026-05-04 07:31:56 -07:00