From f611545ae5a0a84ea77c3b38643198c41a5b017d Mon Sep 17 00:00:00 2001 From: Jean-Luc Makiola Date: Tue, 24 Mar 2026 09:10:48 +0100 Subject: [PATCH] docs(03-01): complete PostgreSQL store and migration infrastructure plan - Add 03-01-SUMMARY.md with full execution record - Update STATE.md: advance to plan 2, record metrics and decisions - Update ROADMAP.md: phase 03 in progress (1/2 plans complete) - Update REQUIREMENTS.md: mark DB-01 and DB-03 complete --- .planning/REQUIREMENTS.md | 8 +- .planning/ROADMAP.md | 2 +- .planning/STATE.md | 25 ++--- .../03-postgresql-support/03-01-SUMMARY.md | 93 +++++++++++++++++++ 4 files changed, 112 insertions(+), 16 deletions(-) create mode 100644 .planning/phases/03-postgresql-support/03-01-SUMMARY.md diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index e7639df..db19275 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -22,9 +22,9 @@ Requirements for this milestone. Each maps to roadmap phases. ### Database -- [ ] **DB-01**: PostgreSQL is supported as an alternative to SQLite via pgx v5 driver +- [x] **DB-01**: PostgreSQL is supported as an alternative to SQLite via pgx v5 driver - [ ] **DB-02**: Database backend is selected via DATABASE_URL env var (present = PostgreSQL, absent = SQLite with DB_PATH) -- [ ] **DB-03**: Existing SQLite users can upgrade without data loss (baseline migration represents current schema) +- [x] **DB-03**: Existing SQLite users can upgrade without data loss (baseline migration represents current schema) ### Bulk Actions @@ -98,9 +98,9 @@ Which phases cover which requirements. Updated during roadmap creation. | REFAC-01 | Phase 2 | Complete | | REFAC-02 | Phase 2 | Complete | | REFAC-03 | Phase 2 | Complete | -| DB-01 | Phase 3 | Pending | +| DB-01 | Phase 3 | Complete | | DB-02 | Phase 3 | Pending | -| DB-03 | Phase 3 | Pending | +| DB-03 | Phase 3 | Complete | | BULK-01 | Phase 4 | Pending | | BULK-02 | Phase 4 | Pending | | SRCH-01 | Phase 4 | Pending | diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 99c5d16..a1298ec 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -61,7 +61,7 @@ Plans: **Plans**: 2 plans Plans: -- [ ] 03-01-PLAN.md — Create PostgresStore (9 Store methods), PostgreSQL migration files, rename RunMigrations to RunSQLiteMigrations, add RunPostgresMigrations +- [x] 03-01-PLAN.md — Create PostgresStore (9 Store methods), PostgreSQL migration files, rename RunMigrations to RunSQLiteMigrations, add RunPostgresMigrations - [ ] 03-02-PLAN.md — Wire DATABASE_URL branching in main.go, fix cross-dialect UNIQUE detection, add Docker Compose postgres profiles, create build-tagged test helper ### Phase 4: UX Improvements diff --git a/.planning/STATE.md b/.planning/STATE.md index b0b02bf..1f10426 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -2,14 +2,14 @@ gsd_state_version: 1.0 milestone: v1.0 milestone_name: milestone -status: Ready to plan -stopped_at: Phase 3 context gathered -last_updated: "2026-03-24T07:46:57.289Z" +status: Ready to execute +stopped_at: Completed 03-01-PLAN.md +last_updated: "2026-03-24T08:10:37.889Z" progress: total_phases: 4 completed_phases: 2 - total_plans: 4 - completed_plans: 4 + total_plans: 6 + completed_plans: 5 --- # Project State @@ -19,12 +19,12 @@ progress: See: .planning/PROJECT.md (updated 2026-03-23) **Core value:** Reliable, persistent visibility into which services need updating — data never disappears, and the dashboard is the one place you trust to show the full picture. -**Current focus:** Phase 02 — backend-refactor +**Current focus:** Phase 03 — postgresql-support ## Current Position -Phase: 3 -Plan: Not started +Phase: 03 (postgresql-support) — EXECUTING +Plan: 2 of 2 ## Performance Metrics @@ -50,6 +50,7 @@ Plan: Not started | Phase 01-data-integrity P02 | 7 | 2 tasks | 2 files | | Phase 02-backend-refactor P01 | 7min | 2 tasks | 7 files | | Phase 02-backend-refactor P02 | 3min | 2 tasks | 4 files | +| Phase 03-postgresql-support P01 | 3min | 2 tasks | 7 files | ## Accumulated Context @@ -70,6 +71,8 @@ Recent decisions affecting current work: - [Phase 02-backend-refactor]: golang-migrate v4.19.1 database/sqlite sub-package confirmed to use modernc.org/sqlite (no CGO); single 0001 baseline migration uses CREATE TABLE IF NOT EXISTS for backward compatibility - [Phase 02-backend-refactor]: Option B for test store access: internal helpers in export_test.go (TestUpsertEvent, TestGetUpdatesMap) instead of exported Store() accessor - keeps store field unexported - [Phase 02-backend-refactor]: NewTestServer pattern: each test gets its own in-memory SQLite DB (RunMigrations + NewSQLiteStore + NewServer) - eliminates shared global state between tests +- [Phase 03-postgresql-support]: PostgresStore uses *sql.DB via pgx/v5/stdlib adapter with no mutex; TEXT timestamps match SQLiteStore scan logic +- [Phase 03-postgresql-support]: CreateTag uses RETURNING id in PostgresStore (pgx does not support LastInsertId); AssignTag uses ON CONFLICT DO UPDATE ### Pending Todos @@ -82,6 +85,6 @@ None yet. ## Session Continuity -Last session: 2026-03-24T07:46:57.286Z -Stopped at: Phase 3 context gathered -Resume file: .planning/phases/03-postgresql-support/03-CONTEXT.md +Last session: 2026-03-24T08:10:37.887Z +Stopped at: Completed 03-01-PLAN.md +Resume file: None diff --git a/.planning/phases/03-postgresql-support/03-01-SUMMARY.md b/.planning/phases/03-postgresql-support/03-01-SUMMARY.md new file mode 100644 index 0000000..48ed358 --- /dev/null +++ b/.planning/phases/03-postgresql-support/03-01-SUMMARY.md @@ -0,0 +1,93 @@ +--- +phase: 03-postgresql-support +plan: "01" +subsystem: persistence +tags: [postgresql, store, migration, pgx] +dependency_graph: + requires: [] + provides: [PostgresStore, RunPostgresMigrations, RunSQLiteMigrations] + affects: [pkg/diunwebhook/migrate.go, pkg/diunwebhook/postgres_store.go] +tech_stack: + added: [github.com/jackc/pgx/v5 v5.9.1, golang-migrate pgx/v5 driver] + patterns: [Store interface implementation, golang-migrate embedded migrations, pgx/v5 stdlib adapter] +key_files: + created: + - pkg/diunwebhook/postgres_store.go + - pkg/diunwebhook/migrations/postgres/0001_initial_schema.up.sql + - pkg/diunwebhook/migrations/postgres/0001_initial_schema.down.sql + modified: + - pkg/diunwebhook/migrate.go + - pkg/diunwebhook/export_test.go + - go.mod + - go.sum +decisions: + - "PostgresStore uses *sql.DB via pgx/v5/stdlib adapter — no native pgx pool, consistent with SQLiteStore pattern" + - "No mutex in PostgresStore — PostgreSQL handles concurrent writes natively (unlike SQLite)" + - "Timestamps stored as TEXT in PostgreSQL schema — matches SQLite scan logic, avoids TIMESTAMPTZ type divergence" + - "CreateTag uses RETURNING id instead of LastInsertId — pgx driver does not support LastInsertId" + - "AssignTag uses ON CONFLICT (image) DO UPDATE instead of INSERT OR REPLACE — standard PostgreSQL upsert" + - "Both migration runners compiled into same binary — no build tags needed (both drivers always present)" +metrics: + duration: "~2.5 minutes" + completed: "2026-03-24T08:09:42Z" + tasks_completed: 2 + files_changed: 7 +--- + +# Phase 03 Plan 01: PostgreSQL Store and Migration Infrastructure Summary + +PostgresStore implementing all 9 Store interface methods using pgx/v5 stdlib adapter, plus PostgreSQL migration infrastructure with RunPostgresMigrations and renamed RunSQLiteMigrations. + +## What Was Built + +### PostgresStore (pkg/diunwebhook/postgres_store.go) +Full implementation of the Store interface for PostgreSQL: +- `NewPostgresStore` constructor with connection pool: `MaxOpenConns(25)`, `MaxIdleConns(5)`, `ConnMaxLifetime(5m)` +- All 9 methods: `UpsertEvent`, `GetUpdates`, `AcknowledgeUpdate`, `ListTags`, `CreateTag`, `DeleteTag`, `AssignTag`, `UnassignTag`, `TagExists` +- PostgreSQL-native SQL: `$1..$15` positional params, `NOW()`, `RETURNING id`, `ON CONFLICT DO UPDATE` +- No `sync.Mutex` — PostgreSQL handles concurrent writes natively + +### PostgreSQL Migrations (pkg/diunwebhook/migrations/postgres/) +- `0001_initial_schema.up.sql`: Creates same 3 tables as SQLite (`updates`, `tags`, `tag_assignments`); uses `SERIAL PRIMARY KEY` for `tags.id`; timestamps remain `TEXT` to match scan logic +- `0001_initial_schema.down.sql`: Drops all 3 tables in dependency order + +### Updated migrate.go +- `RunMigrations` renamed to `RunSQLiteMigrations` +- `RunPostgresMigrations` added using `pgxmigrate` driver with `"pgx5"` database name +- Second `//go:embed migrations/postgres` directive added for `postgresMigrations` + +## Decisions Made + +| Decision | Rationale | +|----------|-----------| +| TEXT timestamps in PostgreSQL schema | Avoids scan divergence with SQLiteStore; both stores parse RFC3339 strings identically | +| RETURNING id in CreateTag | pgx driver does not implement `LastInsertId`; `RETURNING` is the PostgreSQL-idiomatic approach | +| ON CONFLICT (image) DO UPDATE in AssignTag | Replaces SQLite's `INSERT OR REPLACE`; functionally equivalent upsert in standard SQL | +| No mutex in PostgresStore | PostgreSQL connection pool + MVCC handles concurrency; mutex would serialize unnecessarily | +| Both drivers compiled into binary | Simpler than build tags; binary size cost acceptable for a server binary | + +## Deviations from Plan + +### Auto-fixed Issues + +**1. [Rule 1 - Bug] Updated export_test.go to use renamed function** +- **Found during:** Task 1 verification +- **Issue:** `go vet ./pkg/diunwebhook/` failed because `export_test.go` still referenced `RunMigrations` (renamed to `RunSQLiteMigrations`). The plan's acceptance criteria requires `go vet` to exit 0, which takes precedence over the instruction to defer export_test.go changes to Plan 02. +- **Fix:** Updated both `NewTestServer` and `NewTestServerWithSecret` in `export_test.go` to call `RunSQLiteMigrations` +- **Files modified:** `pkg/diunwebhook/export_test.go` +- **Commit:** 95b64b4 + +## Verification Results + +- `go build ./pkg/diunwebhook/` exits 0 +- `go vet ./pkg/diunwebhook/` exits 0 +- PostgreSQL migration UP contains `SERIAL PRIMARY KEY`, all 3 tables +- PostgreSQL migration DOWN contains `DROP TABLE IF EXISTS` for all 3 tables +- `go.mod` contains `github.com/jackc/pgx/v5 v5.9.1` +- `migrate.go` exports both `RunSQLiteMigrations` and `RunPostgresMigrations` + +## Known Stubs + +None — this plan creates implementation code, not UI stubs. + +## Self-Check: PASSED