feat(memory-plugin): #1733 A0 — isolate v2 plugin tables under memory_plugin schema
ci-arm64-advisory / fast-checks (pull_request) Waiting to run
Lint shellcheck (arm64 pilot) / shellcheck-arm64 (pilot) (pull_request) Waiting to run
sop-checklist / review-refire (pull_request) Waiting to run
sop-tier-check / tier-check (pull_request) Waiting to run
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 4s
CI / Detect changes (pull_request) Successful in 6s
CI / Python Lint & Test (pull_request) Successful in 3s
E2E API Smoke Test / detect-changes (pull_request) Successful in 6s
E2E Chat / detect-changes (pull_request) Successful in 6s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 6s
E2E Staging SaaS (full lifecycle) / pr-validate (pull_request) Successful in 35s
E2E Staging SaaS (full lifecycle) / E2E Staging SaaS (pull_request) Has been skipped
Handlers Postgres Integration / detect-changes (pull_request) Successful in 5s
Harness Replays / detect-changes (pull_request) Successful in 3s
Lint forbidden tenant-env keys / Scan workspace_secrets writers for forbidden env keys (pull_request) Successful in 3s
Lint no tenant GITEA or GITHUB token write / Scan for repo-host token write into tenant workspace surface (pull_request) Successful in 3s
lint-required-no-paths / lint-required-no-paths (pull_request) Successful in 1m0s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 4s
gate-check-v3 / gate-check (pull_request) Successful in 4s
qa-review / approved (pull_request) Successful in 3s
security-review / approved (pull_request) Failing after 3s
sop-checklist / all-items-acked (pull_request) acked: 0/7 — missing: comprehensive-testing, local-postgres-e2e, staging-smoke, +4 — body-unfilled: comprehensive-testing, local-postgres-e2
sop-checklist / na-declarations (pull_request) N/A: (none)
CI / Canvas (Next.js) (pull_request) Successful in 2s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 2s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 1m34s
E2E Chat / E2E Chat (pull_request) Successful in 3s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 4s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 3s
Harness Replays / Harness Replays (pull_request) Successful in 4s
CI / Platform (Go) (pull_request) Successful in 4m17s
CI / all-required (pull_request) Successful in 21m24s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
ci-arm64-advisory / fast-checks (pull_request) Waiting to run
Lint shellcheck (arm64 pilot) / shellcheck-arm64 (pilot) (pull_request) Waiting to run
sop-checklist / review-refire (pull_request) Waiting to run
sop-tier-check / tier-check (pull_request) Waiting to run
Block internal-flavored paths / Block forbidden paths (pull_request) Successful in 4s
CI / Detect changes (pull_request) Successful in 6s
CI / Python Lint & Test (pull_request) Successful in 3s
E2E API Smoke Test / detect-changes (pull_request) Successful in 6s
E2E Chat / detect-changes (pull_request) Successful in 6s
E2E Staging Canvas (Playwright) / detect-changes (pull_request) Successful in 6s
E2E Staging SaaS (full lifecycle) / pr-validate (pull_request) Successful in 35s
E2E Staging SaaS (full lifecycle) / E2E Staging SaaS (pull_request) Has been skipped
Handlers Postgres Integration / detect-changes (pull_request) Successful in 5s
Harness Replays / detect-changes (pull_request) Successful in 3s
Lint forbidden tenant-env keys / Scan workspace_secrets writers for forbidden env keys (pull_request) Successful in 3s
Lint no tenant GITEA or GITHUB token write / Scan for repo-host token write into tenant workspace surface (pull_request) Successful in 3s
lint-required-no-paths / lint-required-no-paths (pull_request) Successful in 1m0s
Secret scan / Scan diff for credential-shaped strings (pull_request) Successful in 4s
gate-check-v3 / gate-check (pull_request) Successful in 4s
qa-review / approved (pull_request) Successful in 3s
security-review / approved (pull_request) Failing after 3s
sop-checklist / all-items-acked (pull_request) acked: 0/7 — missing: comprehensive-testing, local-postgres-e2e, staging-smoke, +4 — body-unfilled: comprehensive-testing, local-postgres-e2
sop-checklist / na-declarations (pull_request) N/A: (none)
CI / Canvas (Next.js) (pull_request) Successful in 2s
CI / Shellcheck (E2E scripts) (pull_request) Successful in 2s
E2E API Smoke Test / E2E API Smoke Test (pull_request) Successful in 1m34s
E2E Chat / E2E Chat (pull_request) Successful in 3s
E2E Staging Canvas (Playwright) / Canvas tabs E2E (pull_request) Successful in 4s
Handlers Postgres Integration / Handlers Postgres Integration (pull_request) Successful in 3s
Harness Replays / Harness Replays (pull_request) Successful in 4s
CI / Platform (Go) (pull_request) Successful in 4m17s
CI / all-required (pull_request) Successful in 21m24s
CI / Canvas Deploy Reminder (pull_request) Has been skipped
The Memory v2 plugin sidecar already auto-starts on tenant boot (per entrypoint-tenant.sh, since 2026-05-14) when `MEMORY_V2_CUTOVER=true`, which CP sets on every tenant since 2026-05-05 (controlplane ec2.go:2232). Plugin tables today land in the tenant Postgres's `public` schema, sharing the namespace with platform-tenant tables — a soft SSOT violation flagged in #1733. CTO decision 2026-05-23: same Postgres instance, separate schema. Changes: - New `cmd/memory-plugin-postgres/migrations/000_schema_bootstrap.up.sql` + matching `.down.sql`. Migration filenames sort alphabetically (per `main.go:runMigrationsFromEmbed → sort.Strings`), so 000 runs before the existing 001_memory_v2 which assumes the schema exists in search_path. - `entrypoint-tenant.sh`: when defaulting `MEMORY_PLUGIN_DATABASE_URL` from the tenant's `DATABASE_URL`, append `search_path=memory_plugin` (handles both `?` and `&` URL forms). When the operator explicitly sets `MEMORY_PLUGIN_DATABASE_URL` (separate DB entirely), they keep full control — no rewriting. - `migrations_embed_test.go`: relax the per-file assertion (was: every *.up.sql must contain `CREATE TABLE`; now: every file must contain a recognized DDL keyword AND at least one file declares CREATE TABLE). Required because the new 000 file is schema-only. Stage A — verified locally against `pgvector/pgvector:pg16` (the same image CP launches on tenant EC2): - Plugin binary boots, embedded migrations apply in order `000_schema_bootstrap → 001_memory_v2`. - Tables land in `memory_plugin` schema; `public` stays empty. - End-to-end PUT namespace / POST memory / POST search round-trips through the new schema. - The entrypoint URL-mutation logic exercised on 3 cases: DATABASE_URL with `?`, without `?`, and operator-override (no rewrite). - `go test -short -count=1 ./...` green across all packages (workspace-server + plugin). Operational follow-ups (deliberately out of scope for this PR): - Tenants currently running with rows in `public.memory_*` need a one-time migration to `memory_plugin.memory_*`. Handled in #1733 Phase A2 (per-tenant backfill via `cmd/memory-backfill`); not gating this PR because no production tenant has rows in v2 yet (every tenant_workspace_server has been silently falling back to v1 SQL on `agent_memories` since 2026-05-05). - `CREATE EXTENSION vector` (in 001_memory_v2) implicitly lands in the first writable search_path schema — i.e. `memory_plugin` now. Dropping the schema cascade-drops the extension. If a future operator drops the schema intentionally, they must re-`CREATE EXTENSION`. Acceptable for a defensive operator action; flagged in `000_schema_bootstrap.down.sql`. Refs: #1733 (memory SSOT — Phase A0 of revised plan). Followed by A1 (workspace-server: kill v1 fallback, fail-fast on missing plugin) in a separate core PR. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,9 @@
|
||||
-- Reverse of 000_schema_bootstrap.up.sql.
|
||||
--
|
||||
-- Drops the dedicated schema and every plugin object inside it. Operator
|
||||
-- runs this only when intentionally tearing down the v2 plugin store on
|
||||
-- a shared tenant Postgres. Down migrations are NOT auto-applied by the
|
||||
-- plugin (cmd/memory-plugin-postgres/main.go:runMigrations comment) —
|
||||
-- this file is for manual operator-driven cleanup.
|
||||
|
||||
DROP SCHEMA IF EXISTS memory_plugin CASCADE;
|
||||
@@ -0,0 +1,20 @@
|
||||
-- Create the dedicated schema this plugin owns when it shares a Postgres
|
||||
-- with the tenant platform (the default deployment shape — see
|
||||
-- entrypoint-tenant.sh which appends `search_path=memory_plugin` to
|
||||
-- DATABASE_URL when the operator hasn't set MEMORY_PLUGIN_DATABASE_URL
|
||||
-- explicitly).
|
||||
--
|
||||
-- The schema name `memory_plugin` matches the search_path injected by
|
||||
-- the entrypoint; sql.Open's URL controls *where* subsequent CREATE
|
||||
-- TABLE lands (search_path) but does NOT auto-create the target schema,
|
||||
-- so the plugin has to do it itself before 001_memory_v2 runs.
|
||||
--
|
||||
-- Operators who point the plugin at a dedicated database (no shared
|
||||
-- tenant tables) can leave this no-op: CREATE SCHEMA IF NOT EXISTS is
|
||||
-- idempotent and a fresh DB happily owns a `memory_plugin` schema.
|
||||
--
|
||||
-- Migration files are sorted alphabetically (cmd/memory-plugin-postgres/
|
||||
-- main.go:runMigrationsFromEmbed → sort.Strings), so 000_ runs before
|
||||
-- 001_memory_v2 which assumes the schema already exists in search_path.
|
||||
|
||||
CREATE SCHEMA IF NOT EXISTS memory_plugin;
|
||||
@@ -24,6 +24,7 @@ func TestMigrationsEmbedded_ContainsCreateTable(t *testing.T) {
|
||||
}
|
||||
|
||||
var seenUp bool
|
||||
var seenCreateTable bool
|
||||
for _, e := range entries {
|
||||
if e.IsDir() || !strings.HasSuffix(e.Name(), ".up.sql") {
|
||||
continue
|
||||
@@ -34,13 +35,30 @@ func TestMigrationsEmbedded_ContainsCreateTable(t *testing.T) {
|
||||
t.Errorf("read embedded %q: %v", e.Name(), err)
|
||||
continue
|
||||
}
|
||||
if !strings.Contains(string(data), "CREATE TABLE") {
|
||||
t.Errorf("embedded %q has no CREATE TABLE — wrong file embedded?", e.Name())
|
||||
// Each file must contain at least one DDL statement we expect to see
|
||||
// — guards against truncated / empty files that would silently embed.
|
||||
if !containsAnyDDL(string(data)) {
|
||||
t.Errorf("embedded %q contains no recognized DDL (CREATE TABLE/SCHEMA/EXTENSION/INDEX) — wrong or truncated file?", e.Name())
|
||||
}
|
||||
if strings.Contains(string(data), "CREATE TABLE") {
|
||||
seenCreateTable = true
|
||||
}
|
||||
}
|
||||
if !seenUp {
|
||||
t.Fatal("no *.up.sql in embedded migrations — runtime would have no schema to apply")
|
||||
}
|
||||
if !seenCreateTable {
|
||||
t.Fatal("no embedded migration declares CREATE TABLE — the v2 store would have no tables at runtime")
|
||||
}
|
||||
}
|
||||
|
||||
func containsAnyDDL(sql string) bool {
|
||||
for _, kw := range []string{"CREATE TABLE", "CREATE SCHEMA", "CREATE EXTENSION", "CREATE INDEX", "CREATE OR REPLACE", "ALTER TABLE"} {
|
||||
if strings.Contains(sql, kw) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// TestRunMigrationsFromEmbed_OrderingIsAlphabetic pins that we apply
|
||||
|
||||
@@ -38,7 +38,22 @@ if [ "$MEMORY_V2_CUTOVER" = "true" ] || [ -n "$MEMORY_PLUGIN_URL" ]; then
|
||||
memory_plugin_wanted=1
|
||||
fi
|
||||
if [ -z "$MEMORY_PLUGIN_DISABLE" ] && [ -n "$memory_plugin_wanted" ] && [ -n "$DATABASE_URL" ]; then
|
||||
: "${MEMORY_PLUGIN_DATABASE_URL:=$DATABASE_URL}"
|
||||
# Schema isolation (issue #1733): when defaulting from the tenant
|
||||
# DATABASE_URL we co-locate the plugin's tables under a dedicated
|
||||
# `memory_plugin` schema so they never collide with platform-tenant
|
||||
# tables in `public`. The plugin's 000_schema_bootstrap migration
|
||||
# creates the schema; search_path here directs every subsequent CREATE
|
||||
# TABLE / SELECT to land in it.
|
||||
#
|
||||
# Operators who explicitly set MEMORY_PLUGIN_DATABASE_URL (separate DB
|
||||
# entirely) keep full control — search_path is only injected when we
|
||||
# default from DATABASE_URL.
|
||||
if [ -z "$MEMORY_PLUGIN_DATABASE_URL" ]; then
|
||||
case "$DATABASE_URL" in
|
||||
*\?*) MEMORY_PLUGIN_DATABASE_URL="${DATABASE_URL}&search_path=memory_plugin" ;;
|
||||
*) MEMORY_PLUGIN_DATABASE_URL="${DATABASE_URL}?search_path=memory_plugin" ;;
|
||||
esac
|
||||
fi
|
||||
: "${MEMORY_PLUGIN_LISTEN_ADDR:=:9100}"
|
||||
export MEMORY_PLUGIN_DATABASE_URL MEMORY_PLUGIN_LISTEN_ADDR
|
||||
echo "memory-plugin: starting sidecar on $MEMORY_PLUGIN_LISTEN_ADDR" >&2
|
||||
|
||||
Reference in New Issue
Block a user