docs(03-01): complete PostgreSQL store and migration infrastructure plan
- Add 03-01-SUMMARY.md with full execution record - Update STATE.md: advance to plan 2, record metrics and decisions - Update ROADMAP.md: phase 03 in progress (1/2 plans complete) - Update REQUIREMENTS.md: mark DB-01 and DB-03 complete
This commit is contained in:
@@ -22,9 +22,9 @@ Requirements for this milestone. Each maps to roadmap phases.
|
|||||||
|
|
||||||
### Database
|
### Database
|
||||||
|
|
||||||
- [ ] **DB-01**: PostgreSQL is supported as an alternative to SQLite via pgx v5 driver
|
- [x] **DB-01**: PostgreSQL is supported as an alternative to SQLite via pgx v5 driver
|
||||||
- [ ] **DB-02**: Database backend is selected via DATABASE_URL env var (present = PostgreSQL, absent = SQLite with DB_PATH)
|
- [ ] **DB-02**: Database backend is selected via DATABASE_URL env var (present = PostgreSQL, absent = SQLite with DB_PATH)
|
||||||
- [ ] **DB-03**: Existing SQLite users can upgrade without data loss (baseline migration represents current schema)
|
- [x] **DB-03**: Existing SQLite users can upgrade without data loss (baseline migration represents current schema)
|
||||||
|
|
||||||
### Bulk Actions
|
### Bulk Actions
|
||||||
|
|
||||||
@@ -98,9 +98,9 @@ Which phases cover which requirements. Updated during roadmap creation.
|
|||||||
| REFAC-01 | Phase 2 | Complete |
|
| REFAC-01 | Phase 2 | Complete |
|
||||||
| REFAC-02 | Phase 2 | Complete |
|
| REFAC-02 | Phase 2 | Complete |
|
||||||
| REFAC-03 | Phase 2 | Complete |
|
| REFAC-03 | Phase 2 | Complete |
|
||||||
| DB-01 | Phase 3 | Pending |
|
| DB-01 | Phase 3 | Complete |
|
||||||
| DB-02 | Phase 3 | Pending |
|
| DB-02 | Phase 3 | Pending |
|
||||||
| DB-03 | Phase 3 | Pending |
|
| DB-03 | Phase 3 | Complete |
|
||||||
| BULK-01 | Phase 4 | Pending |
|
| BULK-01 | Phase 4 | Pending |
|
||||||
| BULK-02 | Phase 4 | Pending |
|
| BULK-02 | Phase 4 | Pending |
|
||||||
| SRCH-01 | Phase 4 | Pending |
|
| SRCH-01 | Phase 4 | Pending |
|
||||||
|
|||||||
@@ -61,7 +61,7 @@ Plans:
|
|||||||
**Plans**: 2 plans
|
**Plans**: 2 plans
|
||||||
|
|
||||||
Plans:
|
Plans:
|
||||||
- [ ] 03-01-PLAN.md — Create PostgresStore (9 Store methods), PostgreSQL migration files, rename RunMigrations to RunSQLiteMigrations, add RunPostgresMigrations
|
- [x] 03-01-PLAN.md — Create PostgresStore (9 Store methods), PostgreSQL migration files, rename RunMigrations to RunSQLiteMigrations, add RunPostgresMigrations
|
||||||
- [ ] 03-02-PLAN.md — Wire DATABASE_URL branching in main.go, fix cross-dialect UNIQUE detection, add Docker Compose postgres profiles, create build-tagged test helper
|
- [ ] 03-02-PLAN.md — Wire DATABASE_URL branching in main.go, fix cross-dialect UNIQUE detection, add Docker Compose postgres profiles, create build-tagged test helper
|
||||||
|
|
||||||
### Phase 4: UX Improvements
|
### Phase 4: UX Improvements
|
||||||
|
|||||||
@@ -2,14 +2,14 @@
|
|||||||
gsd_state_version: 1.0
|
gsd_state_version: 1.0
|
||||||
milestone: v1.0
|
milestone: v1.0
|
||||||
milestone_name: milestone
|
milestone_name: milestone
|
||||||
status: Ready to plan
|
status: Ready to execute
|
||||||
stopped_at: Phase 3 context gathered
|
stopped_at: Completed 03-01-PLAN.md
|
||||||
last_updated: "2026-03-24T07:46:57.289Z"
|
last_updated: "2026-03-24T08:10:37.889Z"
|
||||||
progress:
|
progress:
|
||||||
total_phases: 4
|
total_phases: 4
|
||||||
completed_phases: 2
|
completed_phases: 2
|
||||||
total_plans: 4
|
total_plans: 6
|
||||||
completed_plans: 4
|
completed_plans: 5
|
||||||
---
|
---
|
||||||
|
|
||||||
# Project State
|
# Project State
|
||||||
@@ -19,12 +19,12 @@ progress:
|
|||||||
See: .planning/PROJECT.md (updated 2026-03-23)
|
See: .planning/PROJECT.md (updated 2026-03-23)
|
||||||
|
|
||||||
**Core value:** Reliable, persistent visibility into which services need updating — data never disappears, and the dashboard is the one place you trust to show the full picture.
|
**Core value:** Reliable, persistent visibility into which services need updating — data never disappears, and the dashboard is the one place you trust to show the full picture.
|
||||||
**Current focus:** Phase 02 — backend-refactor
|
**Current focus:** Phase 03 — postgresql-support
|
||||||
|
|
||||||
## Current Position
|
## Current Position
|
||||||
|
|
||||||
Phase: 3
|
Phase: 03 (postgresql-support) — EXECUTING
|
||||||
Plan: Not started
|
Plan: 2 of 2
|
||||||
|
|
||||||
## Performance Metrics
|
## Performance Metrics
|
||||||
|
|
||||||
@@ -50,6 +50,7 @@ Plan: Not started
|
|||||||
| Phase 01-data-integrity P02 | 7 | 2 tasks | 2 files |
|
| Phase 01-data-integrity P02 | 7 | 2 tasks | 2 files |
|
||||||
| Phase 02-backend-refactor P01 | 7min | 2 tasks | 7 files |
|
| Phase 02-backend-refactor P01 | 7min | 2 tasks | 7 files |
|
||||||
| Phase 02-backend-refactor P02 | 3min | 2 tasks | 4 files |
|
| Phase 02-backend-refactor P02 | 3min | 2 tasks | 4 files |
|
||||||
|
| Phase 03-postgresql-support P01 | 3min | 2 tasks | 7 files |
|
||||||
|
|
||||||
## Accumulated Context
|
## Accumulated Context
|
||||||
|
|
||||||
@@ -70,6 +71,8 @@ Recent decisions affecting current work:
|
|||||||
- [Phase 02-backend-refactor]: golang-migrate v4.19.1 database/sqlite sub-package confirmed to use modernc.org/sqlite (no CGO); single 0001 baseline migration uses CREATE TABLE IF NOT EXISTS for backward compatibility
|
- [Phase 02-backend-refactor]: golang-migrate v4.19.1 database/sqlite sub-package confirmed to use modernc.org/sqlite (no CGO); single 0001 baseline migration uses CREATE TABLE IF NOT EXISTS for backward compatibility
|
||||||
- [Phase 02-backend-refactor]: Option B for test store access: internal helpers in export_test.go (TestUpsertEvent, TestGetUpdatesMap) instead of exported Store() accessor - keeps store field unexported
|
- [Phase 02-backend-refactor]: Option B for test store access: internal helpers in export_test.go (TestUpsertEvent, TestGetUpdatesMap) instead of exported Store() accessor - keeps store field unexported
|
||||||
- [Phase 02-backend-refactor]: NewTestServer pattern: each test gets its own in-memory SQLite DB (RunMigrations + NewSQLiteStore + NewServer) - eliminates shared global state between tests
|
- [Phase 02-backend-refactor]: NewTestServer pattern: each test gets its own in-memory SQLite DB (RunMigrations + NewSQLiteStore + NewServer) - eliminates shared global state between tests
|
||||||
|
- [Phase 03-postgresql-support]: PostgresStore uses *sql.DB via pgx/v5/stdlib adapter with no mutex; TEXT timestamps match SQLiteStore scan logic
|
||||||
|
- [Phase 03-postgresql-support]: CreateTag uses RETURNING id in PostgresStore (pgx does not support LastInsertId); AssignTag uses ON CONFLICT DO UPDATE
|
||||||
|
|
||||||
### Pending Todos
|
### Pending Todos
|
||||||
|
|
||||||
@@ -82,6 +85,6 @@ None yet.
|
|||||||
|
|
||||||
## Session Continuity
|
## Session Continuity
|
||||||
|
|
||||||
Last session: 2026-03-24T07:46:57.286Z
|
Last session: 2026-03-24T08:10:37.887Z
|
||||||
Stopped at: Phase 3 context gathered
|
Stopped at: Completed 03-01-PLAN.md
|
||||||
Resume file: .planning/phases/03-postgresql-support/03-CONTEXT.md
|
Resume file: None
|
||||||
|
|||||||
93
.planning/phases/03-postgresql-support/03-01-SUMMARY.md
Normal file
93
.planning/phases/03-postgresql-support/03-01-SUMMARY.md
Normal file
@@ -0,0 +1,93 @@
|
|||||||
|
---
|
||||||
|
phase: 03-postgresql-support
|
||||||
|
plan: "01"
|
||||||
|
subsystem: persistence
|
||||||
|
tags: [postgresql, store, migration, pgx]
|
||||||
|
dependency_graph:
|
||||||
|
requires: []
|
||||||
|
provides: [PostgresStore, RunPostgresMigrations, RunSQLiteMigrations]
|
||||||
|
affects: [pkg/diunwebhook/migrate.go, pkg/diunwebhook/postgres_store.go]
|
||||||
|
tech_stack:
|
||||||
|
added: [github.com/jackc/pgx/v5 v5.9.1, golang-migrate pgx/v5 driver]
|
||||||
|
patterns: [Store interface implementation, golang-migrate embedded migrations, pgx/v5 stdlib adapter]
|
||||||
|
key_files:
|
||||||
|
created:
|
||||||
|
- pkg/diunwebhook/postgres_store.go
|
||||||
|
- pkg/diunwebhook/migrations/postgres/0001_initial_schema.up.sql
|
||||||
|
- pkg/diunwebhook/migrations/postgres/0001_initial_schema.down.sql
|
||||||
|
modified:
|
||||||
|
- pkg/diunwebhook/migrate.go
|
||||||
|
- pkg/diunwebhook/export_test.go
|
||||||
|
- go.mod
|
||||||
|
- go.sum
|
||||||
|
decisions:
|
||||||
|
- "PostgresStore uses *sql.DB via pgx/v5/stdlib adapter — no native pgx pool, consistent with SQLiteStore pattern"
|
||||||
|
- "No mutex in PostgresStore — PostgreSQL handles concurrent writes natively (unlike SQLite)"
|
||||||
|
- "Timestamps stored as TEXT in PostgreSQL schema — matches SQLite scan logic, avoids TIMESTAMPTZ type divergence"
|
||||||
|
- "CreateTag uses RETURNING id instead of LastInsertId — pgx driver does not support LastInsertId"
|
||||||
|
- "AssignTag uses ON CONFLICT (image) DO UPDATE instead of INSERT OR REPLACE — standard PostgreSQL upsert"
|
||||||
|
- "Both migration runners compiled into same binary — no build tags needed (both drivers always present)"
|
||||||
|
metrics:
|
||||||
|
duration: "~2.5 minutes"
|
||||||
|
completed: "2026-03-24T08:09:42Z"
|
||||||
|
tasks_completed: 2
|
||||||
|
files_changed: 7
|
||||||
|
---
|
||||||
|
|
||||||
|
# Phase 03 Plan 01: PostgreSQL Store and Migration Infrastructure Summary
|
||||||
|
|
||||||
|
PostgresStore implementing all 9 Store interface methods using pgx/v5 stdlib adapter, plus PostgreSQL migration infrastructure with RunPostgresMigrations and renamed RunSQLiteMigrations.
|
||||||
|
|
||||||
|
## What Was Built
|
||||||
|
|
||||||
|
### PostgresStore (pkg/diunwebhook/postgres_store.go)
|
||||||
|
Full implementation of the Store interface for PostgreSQL:
|
||||||
|
- `NewPostgresStore` constructor with connection pool: `MaxOpenConns(25)`, `MaxIdleConns(5)`, `ConnMaxLifetime(5m)`
|
||||||
|
- All 9 methods: `UpsertEvent`, `GetUpdates`, `AcknowledgeUpdate`, `ListTags`, `CreateTag`, `DeleteTag`, `AssignTag`, `UnassignTag`, `TagExists`
|
||||||
|
- PostgreSQL-native SQL: `$1..$15` positional params, `NOW()`, `RETURNING id`, `ON CONFLICT DO UPDATE`
|
||||||
|
- No `sync.Mutex` — PostgreSQL handles concurrent writes natively
|
||||||
|
|
||||||
|
### PostgreSQL Migrations (pkg/diunwebhook/migrations/postgres/)
|
||||||
|
- `0001_initial_schema.up.sql`: Creates same 3 tables as SQLite (`updates`, `tags`, `tag_assignments`); uses `SERIAL PRIMARY KEY` for `tags.id`; timestamps remain `TEXT` to match scan logic
|
||||||
|
- `0001_initial_schema.down.sql`: Drops all 3 tables in dependency order
|
||||||
|
|
||||||
|
### Updated migrate.go
|
||||||
|
- `RunMigrations` renamed to `RunSQLiteMigrations`
|
||||||
|
- `RunPostgresMigrations` added using `pgxmigrate` driver with `"pgx5"` database name
|
||||||
|
- Second `//go:embed migrations/postgres` directive added for `postgresMigrations`
|
||||||
|
|
||||||
|
## Decisions Made
|
||||||
|
|
||||||
|
| Decision | Rationale |
|
||||||
|
|----------|-----------|
|
||||||
|
| TEXT timestamps in PostgreSQL schema | Avoids scan divergence with SQLiteStore; both stores parse RFC3339 strings identically |
|
||||||
|
| RETURNING id in CreateTag | pgx driver does not implement `LastInsertId`; `RETURNING` is the PostgreSQL-idiomatic approach |
|
||||||
|
| ON CONFLICT (image) DO UPDATE in AssignTag | Replaces SQLite's `INSERT OR REPLACE`; functionally equivalent upsert in standard SQL |
|
||||||
|
| No mutex in PostgresStore | PostgreSQL connection pool + MVCC handles concurrency; mutex would serialize unnecessarily |
|
||||||
|
| Both drivers compiled into binary | Simpler than build tags; binary size cost acceptable for a server binary |
|
||||||
|
|
||||||
|
## Deviations from Plan
|
||||||
|
|
||||||
|
### Auto-fixed Issues
|
||||||
|
|
||||||
|
**1. [Rule 1 - Bug] Updated export_test.go to use renamed function**
|
||||||
|
- **Found during:** Task 1 verification
|
||||||
|
- **Issue:** `go vet ./pkg/diunwebhook/` failed because `export_test.go` still referenced `RunMigrations` (renamed to `RunSQLiteMigrations`). The plan's acceptance criteria requires `go vet` to exit 0, which takes precedence over the instruction to defer export_test.go changes to Plan 02.
|
||||||
|
- **Fix:** Updated both `NewTestServer` and `NewTestServerWithSecret` in `export_test.go` to call `RunSQLiteMigrations`
|
||||||
|
- **Files modified:** `pkg/diunwebhook/export_test.go`
|
||||||
|
- **Commit:** 95b64b4
|
||||||
|
|
||||||
|
## Verification Results
|
||||||
|
|
||||||
|
- `go build ./pkg/diunwebhook/` exits 0
|
||||||
|
- `go vet ./pkg/diunwebhook/` exits 0
|
||||||
|
- PostgreSQL migration UP contains `SERIAL PRIMARY KEY`, all 3 tables
|
||||||
|
- PostgreSQL migration DOWN contains `DROP TABLE IF EXISTS` for all 3 tables
|
||||||
|
- `go.mod` contains `github.com/jackc/pgx/v5 v5.9.1`
|
||||||
|
- `migrate.go` exports both `RunSQLiteMigrations` and `RunPostgresMigrations`
|
||||||
|
|
||||||
|
## Known Stubs
|
||||||
|
|
||||||
|
None — this plan creates implementation code, not UI stubs.
|
||||||
|
|
||||||
|
## Self-Check: PASSED
|
||||||
Reference in New Issue
Block a user