--- phase: 14-postgresql-migration plan: 01 type: execute wave: 1 depends_on: [] files_modified: - src/db/schema.ts - src/db/index.ts - src/db/migrate.ts - src/db/seed.ts - src/shared/types.ts - tests/helpers/db.ts - drizzle.config.ts - package.json autonomous: true requirements: [DB-01, DB-03] must_haves: truths: - "Schema defines all 12 tables using drizzle-orm/pg-core (pgTable, serial, text, timestamp, etc.)" - "Database connection uses postgres.js driver with DATABASE_URL" - "Test helper creates async PGlite-backed Drizzle instance with migrations applied" - "Drizzle migrations are generated in drizzle-pg/ directory" artifacts: - path: "src/db/schema.ts" provides: "PostgreSQL table definitions" contains: "pgTable" - path: "src/db/index.ts" provides: "Async Postgres connection" contains: "drizzle-orm/postgres-js" - path: "tests/helpers/db.ts" provides: "PGlite test database factory" contains: "drizzle-orm/pglite" - path: "drizzle-pg/" provides: "PostgreSQL migration files" - path: "drizzle.config.ts" provides: "Drizzle Kit config for PostgreSQL" contains: "postgresql" key_links: - from: "tests/helpers/db.ts" to: "src/db/schema.ts" via: "import * as schema" pattern: "import.*schema" - from: "src/db/index.ts" to: "src/db/schema.ts" via: "import * as schema" pattern: "import.*schema" --- Rewrite the database foundation from SQLite to PostgreSQL: schema definitions, database connection, test infrastructure, and Drizzle configuration. Install required packages. Generate the initial PostgreSQL migration. Purpose: Everything else in this phase depends on these files. Schema and DB config must exist before services, routes, or tests can be converted. Output: Working schema.ts (pg-core), index.ts (postgres.js), tests/helpers/db.ts (PGlite), drizzle.config.ts (postgresql), generated migration in drizzle-pg/ @$HOME/.claude/get-shit-done/workflows/execute-plan.md @$HOME/.claude/get-shit-done/templates/summary.md @.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/14-postgresql-migration/14-CONTEXT.md @.planning/phases/14-postgresql-migration/14-RESEARCH.md @src/db/schema.ts @src/db/index.ts @src/db/migrate.ts @src/db/seed.ts @src/shared/types.ts @tests/helpers/db.ts @drizzle.config.ts @package.json Task 1: Install dependencies and rewrite schema + DB config files package.json, src/db/schema.ts, src/db/index.ts, src/db/migrate.ts, src/db/seed.ts, src/shared/types.ts, drizzle.config.ts src/db/schema.ts, src/db/index.ts, src/db/migrate.ts, src/db/seed.ts, src/shared/types.ts, drizzle.config.ts, package.json **Step 1: Install packages** ```bash bun add postgres @electric-sql/pglite bun remove better-sqlite3 @types/better-sqlite3 ``` **Step 2: Rewrite `src/db/schema.ts`** -- Clean rewrite per D-01. Replace all `sqliteTable` with `pgTable`, all imports from `drizzle-orm/sqlite-core` with `drizzle-orm/pg-core`. Column type mapping (apply to ALL 12 tables): - `integer("id").primaryKey({ autoIncrement: true })` -> `serial("id").primaryKey()` - `text("col")` -> `text("col")` (unchanged) - `real("weight_grams")` -> `doublePrecision("weight_grams")` - `real("sort_order")` -> `doublePrecision("sort_order")` - `integer("price_cents")` -> `integer("price_cents")` (unchanged) - `integer("col", { mode: "timestamp" }).$defaultFn(() => new Date())` -> `timestamp("col").notNull().defaultNow()` - `integer("col", { mode: "timestamp" }).notNull()` (no default, e.g., expiresAt) -> `timestamp("col").notNull()` - `integer("used").notNull().default(0)` -> `boolean("used").notNull().default(false)` (oauthCodes table) - `integer("quantity").notNull().default(1)` -> `integer("quantity").notNull().default(1)` (unchanged) Tables to rewrite (12 total): categories, items, threads, threadCandidates, setups, setupItems, settings, users, sessions, apiKeys, oauthClients, oauthCodes, oauthTokens. Import statement: ```typescript import { boolean, doublePrecision, integer, pgTable, serial, text, timestamp } from "drizzle-orm/pg-core"; ``` Preserve ALL foreign key references and cascade rules exactly as they are. Preserve all `.unique()` constraints. Preserve all `.default()` values. For `settings` table: keep `text("key").primaryKey()` (no serial). For `sessions` table: keep `text("id").primaryKey()` (no serial). **Step 3: Rewrite `src/db/index.ts`** per D-03: ```typescript import { drizzle } from "drizzle-orm/postgres-js"; import postgres from "postgres"; import * as schema from "./schema.ts"; const connectionString = process.env.DATABASE_URL || "postgresql://gearbox:gearbox@localhost:5432/gearbox"; const queryClient = postgres(connectionString); export const db = drizzle(queryClient, { schema }); ``` **Step 4: Rewrite `src/db/migrate.ts`**: ```typescript import { drizzle } from "drizzle-orm/postgres-js"; import { migrate } from "drizzle-orm/postgres-js/migrator"; import postgres from "postgres"; const connectionString = process.env.DATABASE_URL || "postgresql://gearbox:gearbox@localhost:5432/gearbox"; const migrationClient = postgres(connectionString, { max: 1 }); const db = drizzle(migrationClient); await migrate(db, { migrationsFolder: "./drizzle-pg" }); await migrationClient.end(); console.log("Migrations applied successfully"); ``` **Step 5: Rewrite `src/db/seed.ts`** to async: ```typescript import { db } from "./index.ts"; import { categories } from "./schema.ts"; export async function seedDefaults() { const existing = await db.select().from(categories); if (existing.length === 0) { await db.insert(categories).values({ name: "Uncategorized", icon: "package", }); } } ``` **Step 6: Update `src/shared/types.ts`** -- No changes needed to the file content itself. The types infer from schema which still exports the same table names. Verify the file still compiles after schema change. **Step 7: Update `drizzle.config.ts`** per D-02: ```typescript import { defineConfig } from "drizzle-kit"; export default defineConfig({ out: "./drizzle-pg", schema: "./src/db/schema.ts", dialect: "postgresql", dbCredentials: { url: process.env.DATABASE_URL || "postgresql://gearbox:gearbox@localhost:5432/gearbox", }, }); ``` grep -q "pgTable" src/db/schema.ts && grep -q "drizzle-orm/pg-core" src/db/schema.ts && grep -q "postgres-js" src/db/index.ts && grep -q "postgresql" drizzle.config.ts && grep -q "async function seedDefaults" src/db/seed.ts && bun run lint 2>&1 | tail -3 && echo "PASS" || echo "FAIL" - src/db/schema.ts contains `import { boolean, doublePrecision, integer, pgTable, serial, text, timestamp } from "drizzle-orm/pg-core"` - src/db/schema.ts contains `pgTable("categories"` and all 12 table definitions use pgTable - src/db/schema.ts does NOT contain `sqliteTable` or `drizzle-orm/sqlite-core` or `real(` or `{ mode: "timestamp" }` - src/db/schema.ts contains `boolean("used")` for oauthCodes table - src/db/schema.ts contains `doublePrecision("weight_grams")` and `doublePrecision("sort_order")` - src/db/schema.ts contains `timestamp("created_at").notNull().defaultNow()` pattern - src/db/index.ts contains `import postgres from "postgres"` and `drizzle-orm/postgres-js` - src/db/index.ts contains `DATABASE_URL` - src/db/index.ts does NOT contain `bun:sqlite` - src/db/migrate.ts contains `drizzle-orm/postgres-js/migrator` and `migrationsFolder: "./drizzle-pg"` - src/db/seed.ts contains `export async function seedDefaults()` - src/db/seed.ts contains `await db.select()` and `await db.insert()` - drizzle.config.ts contains `dialect: "postgresql"` and `out: "./drizzle-pg"` - package.json contains `"postgres"` in dependencies - package.json contains `"@electric-sql/pglite"` in devDependencies or dependencies - package.json does NOT contain `"better-sqlite3"` or `"@types/better-sqlite3"` All 12 tables rewritten with pg-core types. DB connection uses postgres.js. Migrate.ts uses postgres-js migrator. Seed is async. Drizzle config targets postgresql dialect with drizzle-pg/ output. Task 2: Rewrite test helper and generate initial PostgreSQL migration tests/helpers/db.ts, drizzle-pg/ tests/helpers/db.ts, src/db/schema.ts **Step 1: Rewrite `tests/helpers/db.ts`** per D-07 and D-08: ```typescript import { drizzle } from "drizzle-orm/pglite"; import { migrate } from "drizzle-orm/pglite/migrator"; import * as schema from "../../src/db/schema.ts"; export async function createTestDb() { const db = drizzle({ schema }); // Apply migrations from the new PostgreSQL migration directory await migrate(db, { migrationsFolder: "./drizzle-pg" }); // Seed default Uncategorized category await db.insert(schema.categories).values({ name: "Uncategorized", icon: "package" }); return db; } ``` Key changes from current: - Import from `drizzle-orm/pglite` instead of `drizzle-orm/bun-sqlite` - `migrate` from `drizzle-orm/pglite/migrator` instead of `drizzle-orm/bun-sqlite/migrator` - Function is now `async` (returns Promise) - No `Database` import from `bun:sqlite` - No `":memory:"` -- PGlite creates an in-memory Postgres instance by default - Migration folder changed to `./drizzle-pg` - `db.insert(...).values(...).run()` becomes `await db.insert(...).values(...)` **Step 2: Generate initial PostgreSQL migration:** ```bash bunx drizzle-kit generate ``` This reads the updated `drizzle.config.ts` (dialect: "postgresql", schema: src/db/schema.ts) and generates SQL migration files in `drizzle-pg/`. **Step 3: Verify migration was generated and is complete:** ```bash ls drizzle-pg/ cat drizzle-pg/*.sql ``` Confirm the SQL contains `CREATE TABLE` statements for all 12 tables with correct Postgres types (serial, text, timestamp, boolean, double precision, etc.). Count the CREATE TABLE statements -- there must be at least 12 (categories, items, threads, thread_candidates, setups, setup_items, settings, users, sessions, api_keys, oauth_clients, oauth_codes, oauth_tokens). **Step 4: Quick smoke test -- verify PGlite test helper works:** ```bash bun -e " import { createTestDb } from './tests/helpers/db.ts'; const db = await createTestDb(); const cats = await db.select().from((await import('./src/db/schema.ts')).categories); console.log('Categories:', cats.length); if (cats.length !== 1) { console.error('FAIL: expected 1 category'); process.exit(1); } console.log('PGlite test helper works!'); " ``` ls drizzle-pg/*.sql && grep -c "CREATE TABLE" drizzle-pg/*.sql | tail -1 | grep -qE "^drizzle-pg/.*:1[2-9]$|^drizzle-pg/.*:[2-9][0-9]$" || { echo "WARNING: verify CREATE TABLE count manually"; }; grep -q "drizzle-orm/pglite" tests/helpers/db.ts && grep -q "async function createTestDb" tests/helpers/db.ts && bun run lint 2>&1 | tail -3 && echo "PASS" || echo "FAIL" - tests/helpers/db.ts contains `import { drizzle } from "drizzle-orm/pglite"` - tests/helpers/db.ts contains `import { migrate } from "drizzle-orm/pglite/migrator"` - tests/helpers/db.ts contains `export async function createTestDb()` - tests/helpers/db.ts contains `migrationsFolder: "./drizzle-pg"` - tests/helpers/db.ts does NOT contain `bun:sqlite` or `drizzle-orm/bun-sqlite` or `.run()` - drizzle-pg/ directory exists with at least one .sql migration file - Migration SQL contains CREATE TABLE for all 12+ tables (categories, items, threads, thread_candidates, setups, setup_items, settings, users, sessions, api_keys, oauth_clients, oauth_codes, oauth_tokens) - `grep -c "CREATE TABLE" drizzle-pg/*.sql` shows at least 12 CREATE TABLE statements - PGlite smoke test (bun -e script above) exits 0 Test helper returns async PGlite Drizzle instance. Initial PostgreSQL migration generated in drizzle-pg/ with all 12+ CREATE TABLE statements. Smoke test confirms PGlite can apply migrations and seed data. - `grep -r "sqliteTable\|bun:sqlite\|drizzle-orm/sqlite-core\|drizzle-orm/bun-sqlite" src/db/ drizzle.config.ts tests/helpers/db.ts` returns NO matches - `grep -c "pgTable" src/db/schema.ts` returns 12+ (one per table, possibly more from import) - `ls drizzle-pg/*.sql` shows at least one migration file - `grep -c "CREATE TABLE" drizzle-pg/*.sql` shows at least 12 tables - PGlite smoke test exits 0 - `bun run lint` passes All database foundation files rewritten for PostgreSQL. Schema uses pg-core types. DB connection uses postgres.js. Test helper uses PGlite. Initial migration generated with all 12+ tables. No SQLite references remain in these files. Lint passes. After completion, create `.planning/phases/14-postgresql-migration/14-01-SUMMARY.md`