From 552817efec650b559491b22a573f38546c3e8095 Mon Sep 17 00:00:00 2001 From: Jean-Luc Makiola Date: Sat, 4 Apr 2026 11:42:01 +0200 Subject: [PATCH] docs(14): capture phase context --- .../14-postgresql-migration/14-CONTEXT.md | 113 ++++++++++++++++++ .../14-DISCUSSION-LOG.md | 90 ++++++++++++++ 2 files changed, 203 insertions(+) create mode 100644 .planning/phases/14-postgresql-migration/14-CONTEXT.md create mode 100644 .planning/phases/14-postgresql-migration/14-DISCUSSION-LOG.md diff --git a/.planning/phases/14-postgresql-migration/14-CONTEXT.md b/.planning/phases/14-postgresql-migration/14-CONTEXT.md new file mode 100644 index 0000000..1ea2d98 --- /dev/null +++ b/.planning/phases/14-postgresql-migration/14-CONTEXT.md @@ -0,0 +1,113 @@ +# Phase 14: PostgreSQL Migration - Context + +**Gathered:** 2026-04-04 +**Status:** Ready for planning + + +## Phase Boundary + +Replace SQLite with PostgreSQL as the sole database. Make all database operations async. Establish PGlite-based test infrastructure. Provide a one-time data migration script and Docker Compose for local Postgres development. + + + + +## Implementation Decisions + +### Migration Strategy +- **D-01:** Clean rewrite of `src/db/schema.ts` using `drizzle-orm/pg-core` (pgTable, serial, text, numeric, timestamp, etc.) — not a conversion of the SQLite schema +- **D-02:** Start fresh Postgres migration history in a new directory (e.g., `drizzle-pg/`) — keep existing `drizzle/` SQLite migrations archived for reference +- **D-03:** `src/db/index.ts` switches from `bun:sqlite` + `drizzle-orm/bun-sqlite` to `drizzle-orm/node-postgres` (or `drizzle-orm/postgres-js`) with async connection + +### Data Migration Script +- **D-04:** Standalone TypeScript script (e.g., `scripts/migrate-sqlite-to-postgres.ts`) that reads from SQLite file and writes to Postgres — not a Drizzle migration +- **D-05:** Script handles type conversions: integer timestamps → proper Postgres `timestamp` columns, `real` weight → `numeric` or `double precision`, text → text +- **D-06:** Script preserves all IDs and foreign key relationships — no ID remapping + +### Test Infrastructure +- **D-07:** `createTestDb()` returns an async PGlite-backed Drizzle instance — same API shape as current, but async +- **D-08:** Per-test fresh PGlite instance with migrations applied (matches current in-memory SQLite pattern, avoids test pollution) +- **D-09:** All service and route tests updated from sync to async database operations + +### Docker Compose +- **D-10:** Separate `docker-compose.dev.yml` for development with Postgres service — keep existing `docker-compose.yml` for production (updated to include Postgres) +- **D-11:** PostgreSQL 16 (latest stable) +- **D-12:** Environment variable `DATABASE_URL` for Postgres connection string (replaces `DATABASE_PATH` for SQLite) + +### Claude's Discretion +- Drizzle Postgres driver choice (`node-postgres` vs `postgres-js`) — pick based on Bun compatibility and async performance +- PGlite configuration details (version, extensions) +- Column type mapping specifics beyond the ones called out (e.g., whether to use `serial` vs `integer().primaryKey()`) +- Migration script error handling and progress reporting +- Whether to use `drizzle-orm/pglite` driver or generic pg driver for tests + + + + +## Canonical References + +**Downstream agents MUST read these before planning or implementing.** + +### Database Schema & Config +- `src/db/schema.ts` — Current SQLite schema (source of truth for tables/columns to migrate) +- `src/db/index.ts` — Current database initialization (bun:sqlite + drizzle) +- `drizzle.config.ts` — Current Drizzle Kit config (sqlite dialect) +- `drizzle/` — Existing SQLite migration files (10 migrations, reference only) + +### Test Infrastructure +- `tests/helpers/db.ts` — Current test database helper (in-memory SQLite, migration application, seed) + +### Services (all need sync → async) +- `src/server/services/*.ts` — 9 service files that use synchronous Drizzle operations +- `src/server/routes/*.ts` — 9 route files that call services + +### Tests (all need updating) +- `tests/services/*.test.ts` — 9 service test files +- `tests/routes/*.test.ts` — 8 route test files +- `tests/mcp/tools.test.ts` — MCP tools test + +### Docker +- `docker-compose.yml` — Current production compose (SQLite volumes, no Postgres) + + + + +## Existing Code Insights + +### Reusable Assets +- Drizzle ORM already in use — schema definition pattern transfers directly to pg-core +- Service layer architecture with DI (db as first param) — makes swapping the db instance straightforward +- Zod schemas in `src/shared/schemas.ts` — validation layer is database-agnostic, no changes needed +- TanStack Query hooks — frontend is fully decoupled from database, no changes needed + +### Established Patterns +- **Service DI pattern**: All services take `db` as first parameter — this means swapping SQLite for Postgres only requires changing what `db` is, not how services use it +- **Sync Drizzle calls**: Current code uses `.run()`, `.get()`, `.all()` synchronously — Postgres requires `.execute()` / await on all queries +- **Test pattern**: `createTestDb()` creates isolated DB, applies migrations, seeds — same pattern works with PGlite +- **Timestamps as integers**: `{ mode: "timestamp" }` on integer columns — Postgres can use native `timestamp` type + +### Integration Points +- `src/db/index.ts` — Single point of database creation (good: only one file to change for connection) +- `src/server/index.ts` — Where db is provided to Hono context via middleware +- `tests/helpers/db.ts` — Single test DB factory (good: only one file to change for test infra) +- `drizzle.config.ts` — Needs dialect change from sqlite to postgresql + + + + +## Specific Ideas + +No specific requirements — open to standard approaches for SQLite-to-Postgres migration with Drizzle ORM. + + + + +## Deferred Ideas + +None — discussion stayed within phase scope + + + +--- + +*Phase: 14-postgresql-migration* +*Context gathered: 2026-04-04* diff --git a/.planning/phases/14-postgresql-migration/14-DISCUSSION-LOG.md b/.planning/phases/14-postgresql-migration/14-DISCUSSION-LOG.md new file mode 100644 index 0000000..f3f8dec --- /dev/null +++ b/.planning/phases/14-postgresql-migration/14-DISCUSSION-LOG.md @@ -0,0 +1,90 @@ +# Phase 14: PostgreSQL Migration - Discussion Log + +> **Audit trail only.** Do not use as input to planning, research, or execution agents. +> Decisions are captured in CONTEXT.md — this log preserves the alternatives considered. + +**Date:** 2026-04-04 +**Phase:** 14-postgresql-migration +**Areas discussed:** Migration strategy, Data migration script, Test infrastructure, Docker Compose layout +**Mode:** --auto (all decisions auto-selected as recommended defaults) + +--- + +## Migration Strategy + +| Option | Description | Selected | +|--------|-------------|----------| +| Clean schema rewrite | Rewrite schema.ts using drizzle-orm/pg-core with fresh migration history | ✓ | +| Convert existing migrations | Transform SQLite migrations to Postgres equivalents | | +| Dual-dialect schema | Maintain both SQLite and Postgres schema definitions | | + +**User's choice:** [auto] Clean schema rewrite (recommended default) +**Notes:** SQLite and Postgres dialects differ enough (type system, auto-increment vs serial, pragma vs native features) that converting migrations is error-prone. Fresh start is cleaner. + +| Option | Description | Selected | +|--------|-------------|----------| +| Fresh Postgres migration history | New directory, archive SQLite migrations | ✓ | +| Convert SQLite migrations | Rewrite each .sql file for Postgres | | + +**User's choice:** [auto] Fresh Postgres migration history (recommended default) +**Notes:** 10 existing SQLite migrations would need manual conversion. Starting fresh avoids dialect translation bugs. + +--- + +## Data Migration Script + +| Option | Description | Selected | +|--------|-------------|----------| +| Standalone TypeScript script | Reads SQLite, writes Postgres, one-time use | ✓ | +| Drizzle migration | Built into the migration pipeline | | +| SQL dump + import | pg_dump-style approach | | + +**User's choice:** [auto] Standalone TypeScript script (recommended default) +**Notes:** One-time operation that doesn't belong in the migration pipeline. Script can handle type conversions explicitly. + +--- + +## Test Infrastructure + +| Option | Description | Selected | +|--------|-------------|----------| +| Per-test PGlite instance | Fresh database per test, migrations applied each time | ✓ | +| Shared PGlite with transaction rollback | One instance, wrap each test in a rolled-back transaction | | +| Shared PGlite with cleanup | One instance, truncate tables between tests | | + +**User's choice:** [auto] Per-test PGlite instance (recommended default) +**Notes:** Matches current in-memory SQLite pattern. Avoids test pollution. PGlite is lightweight enough for per-test instances. + +--- + +## Docker Compose Layout + +| Option | Description | Selected | +|--------|-------------|----------| +| Separate dev compose file | docker-compose.dev.yml with Postgres for development | ✓ | +| Single compose with profiles | Use Docker Compose profiles for dev vs prod | | +| Extend existing compose | Add Postgres to the single docker-compose.yml | | + +**User's choice:** [auto] Separate dev compose file (recommended default) +**Notes:** Separation of concerns. Production compose will also need Postgres eventually but with different configuration. + +| Option | Description | Selected | +|--------|-------------|----------| +| PostgreSQL 16 | Latest stable release | ✓ | +| PostgreSQL 15 | Previous stable | | + +**User's choice:** [auto] PostgreSQL 16 (recommended default) + +--- + +## Claude's Discretion + +- Drizzle Postgres driver choice (node-postgres vs postgres-js) +- PGlite configuration details +- Column type mapping specifics +- Migration script error handling +- Test driver choice for PGlite + +## Deferred Ideas + +None