diff --git a/.planning/phases/14-postgresql-migration/14-RESEARCH.md b/.planning/phases/14-postgresql-migration/14-RESEARCH.md new file mode 100644 index 0000000..753fd89 --- /dev/null +++ b/.planning/phases/14-postgresql-migration/14-RESEARCH.md @@ -0,0 +1,574 @@ +# Phase 14: PostgreSQL Migration - Research + +**Researched:** 2026-04-04 +**Domain:** Database migration (SQLite to PostgreSQL), Drizzle ORM, PGlite testing +**Confidence:** HIGH + +## Summary + +This phase replaces the SQLite database with PostgreSQL across the entire stack: schema definitions, database driver, all service/route code (sync to async), test infrastructure (PGlite), data migration script, and Docker Compose for local development. + +The core migration is well-supported by Drizzle ORM, which has first-class drivers for both PostgreSQL (via `postgres` package) and PGlite (for testing). The schema rewrite from `drizzle-orm/sqlite-core` to `drizzle-orm/pg-core` is straightforward -- column type mapping is direct. The bulk of the work is mechanical: adding `await` to ~82 sync `.all()/.get()/.run()` calls across 9 service files, updating 4 transaction usages to async, and updating all 18 test files to use async PGlite-backed databases. + +**Primary recommendation:** Use `postgres` (postgres.js) as the production driver for best Bun compatibility and connection pooling. Use `@electric-sql/pglite` with `drizzle-orm/pglite` for tests. Apply schema in tests via `migrate()` from generated migrations (not `pushSchema`) to match production behavior. + + +## User Constraints (from CONTEXT.md) + +### Locked Decisions +- **D-01:** Clean rewrite of `src/db/schema.ts` using `drizzle-orm/pg-core` (pgTable, serial, text, numeric, timestamp, etc.) -- not a conversion of the SQLite schema +- **D-02:** Start fresh Postgres migration history in a new directory (e.g., `drizzle-pg/`) -- keep existing `drizzle/` SQLite migrations archived for reference +- **D-03:** `src/db/index.ts` switches from `bun:sqlite` + `drizzle-orm/bun-sqlite` to `drizzle-orm/node-postgres` (or `drizzle-orm/postgres-js`) with async connection +- **D-04:** Standalone TypeScript script (e.g., `scripts/migrate-sqlite-to-postgres.ts`) that reads from SQLite file and writes to Postgres -- not a Drizzle migration +- **D-05:** Script handles type conversions: integer timestamps to proper Postgres `timestamp` columns, `real` weight to `numeric` or `double precision`, text to text +- **D-06:** Script preserves all IDs and foreign key relationships -- no ID remapping +- **D-07:** `createTestDb()` returns an async PGlite-backed Drizzle instance -- same API shape as current, but async +- **D-08:** Per-test fresh PGlite instance with migrations applied (matches current in-memory SQLite pattern, avoids test pollution) +- **D-09:** All service and route tests updated from sync to async database operations +- **D-10:** Separate `docker-compose.dev.yml` for development with Postgres service -- keep existing `docker-compose.yml` for production (updated to include Postgres) +- **D-11:** PostgreSQL 16 (latest stable) +- **D-12:** Environment variable `DATABASE_URL` for Postgres connection string (replaces `DATABASE_PATH` for SQLite) + +### Claude's Discretion +- Drizzle Postgres driver choice (`node-postgres` vs `postgres-js`) -- pick based on Bun compatibility and async performance +- PGlite configuration details (version, extensions) +- Column type mapping specifics beyond the ones called out (e.g., whether to use `serial` vs `integer().primaryKey()`) +- Migration script error handling and progress reporting +- Whether to use `drizzle-orm/pglite` driver or generic pg driver for tests + +### Deferred Ideas (OUT OF SCOPE) +None -- discussion stayed within phase scope + + + +## Phase Requirements + +| ID | Description | Research Support | +|----|-------------|------------------| +| DB-01 | Application runs on PostgreSQL instead of SQLite | Schema rewrite (pg-core), driver swap (postgres.js), async service layer | +| DB-02 | All service functions use async database operations | 82 sync calls across 9 services need `await`; 4 transactions need async conversion | +| DB-03 | Test infrastructure uses PGlite instead of bun:sqlite in-memory databases | `@electric-sql/pglite` + `drizzle-orm/pglite` with per-test instances | +| DB-04 | Existing SQLite data can be migrated to Postgres via a one-time script | Standalone script reads SQLite via `bun:sqlite`, writes to Postgres with type conversion | +| DB-05 | Docker Compose provides Postgres for local development | `docker-compose.dev.yml` with PostgreSQL 16, `docker-compose.yml` updated for production | + + +## Standard Stack + +### Core +| Library | Version | Purpose | Why Standard | +|---------|---------|---------|--------------| +| drizzle-orm | 0.45.2 | ORM (already installed, update minor) | Already in use; pg-core module provides PostgreSQL schema/query support | +| drizzle-kit | 0.31.10 | Migration generation (already installed, update minor) | Already in use; supports `postgresql` dialect for migration generation | +| postgres | 3.4.8 | PostgreSQL driver (postgres.js) | Best Bun compatibility, built-in connection pooling, no native bindings needed | +| @electric-sql/pglite | 0.4.3 | In-process WASM Postgres for testing | Real Postgres SQL execution without Docker; per-test isolation in milliseconds | + +### Supporting +| Library | Version | Purpose | When to Use | +|---------|---------|---------|-------------| +| bun:sqlite (built-in) | N/A | Read-only in migration script | Only used by data migration script to read existing SQLite data | + +### Alternatives Considered +| Instead of | Could Use | Tradeoff | +|------------|-----------|----------| +| postgres (postgres.js) | pg (node-postgres) | pg requires `@types/pg`, has native binding option but no benefit on Bun; postgres.js has cleaner API | +| postgres (postgres.js) | bun:sql (Bun SQL) | Bun SQL has known drizzle-kit compatibility issues (push/migrate don't work); not yet mature enough | +| @electric-sql/pglite | Docker Postgres for tests | Docker adds latency, setup complexity; PGlite is zero-config, sub-millisecond startup | + +**Driver recommendation: `postgres` (postgres.js)** +- No native bindings (works on Bun without build tools) +- Built-in connection pooling +- Prepared statements by default +- Drizzle ORM has first-class `drizzle-orm/postgres-js` driver +- Bun SQL driver was considered but drizzle-kit does not fully support it for push/migrate commands yet + +**Installation:** +```bash +bun add postgres @electric-sql/pglite +bun remove better-sqlite3 @types/better-sqlite3 +``` + +Note: `bun:sqlite` is built-in and does not need to be uninstalled -- it remains available for the migration script. `better-sqlite3` and its types are dev dependencies that can be removed since they are no longer needed. + +## Architecture Patterns + +### Recommended Project Structure +``` +src/db/ + schema.ts # Rewritten with drizzle-orm/pg-core (pgTable, serial, text, timestamp, etc.) + index.ts # postgres.js connection + drizzle initialization + migrate.ts # Async migration runner for production startup + seed.ts # Async seed function +drizzle-pg/ # New PostgreSQL migration directory (D-02) +drizzle/ # Archived SQLite migrations (kept for reference) +drizzle.config.ts # Updated: dialect "postgresql", out "./drizzle-pg" +scripts/ + migrate-sqlite-to-postgres.ts # One-time data migration script (D-04) +tests/helpers/ + db.ts # Rewritten: async createTestDb() with PGlite +docker-compose.dev.yml # New: Postgres for local dev +docker-compose.yml # Updated: Postgres for production +``` + +### Pattern 1: PostgreSQL Schema Definition +**What:** Rewrite all tables using `drizzle-orm/pg-core` types +**When to use:** The one-time schema rewrite + +```typescript +// src/db/schema.ts +import { doublePrecision, integer, pgTable, serial, text, timestamp } from "drizzle-orm/pg-core"; + +export const categories = pgTable("categories", { + id: serial("id").primaryKey(), + name: text("name").notNull().unique(), + icon: text("icon").notNull().default("package"), + createdAt: timestamp("created_at").notNull().defaultNow(), +}); + +export const items = pgTable("items", { + id: serial("id").primaryKey(), + name: text("name").notNull(), + weightGrams: doublePrecision("weight_grams"), + priceCents: integer("price_cents"), + categoryId: integer("category_id").notNull().references(() => categories.id), + notes: text("notes"), + productUrl: text("product_url"), + imageFilename: text("image_filename"), + imageSourceUrl: text("image_source_url"), + quantity: integer("quantity").notNull().default(1), + createdAt: timestamp("created_at").notNull().defaultNow(), + updatedAt: timestamp("updated_at").notNull().defaultNow(), +}); +``` + +### Pattern 2: Async Database Connection +**What:** Production database initialization with postgres.js +**When to use:** `src/db/index.ts` + +```typescript +// src/db/index.ts +import { drizzle } from "drizzle-orm/postgres-js"; +import postgres from "postgres"; +import * as schema from "./schema.ts"; + +const queryClient = postgres(process.env.DATABASE_URL!); +export const db = drizzle(queryClient, { schema }); +``` + +### Pattern 3: Async Service Functions +**What:** Convert sync Drizzle calls to async with await +**When to use:** All 9 service files + +```typescript +// BEFORE (SQLite sync): +export function getAllItems(db: Db = prodDb) { + return db.select().from(items).innerJoin(categories, eq(items.categoryId, categories.id)).all(); +} + +// AFTER (PostgreSQL async): +export async function getAllItems(db: Db = prodDb) { + return await db.select().from(items).innerJoin(categories, eq(items.categoryId, categories.id)); +} +``` + +Key differences: +- `.all()` is removed -- Postgres driver returns arrays directly from `await` +- `.get()` is replaced with indexing: `const [result] = await db.select()...` or using `.limit(1)` then `[0]` +- `.run()` is removed -- `await db.delete()...` / `await db.insert()...` is sufficient +- `.returning().get()` becomes `const [result] = await db.insert()...returning()` +- `db.transaction(() => { ... })` becomes `await db.transaction(async (tx) => { ... })` with await inside + +### Pattern 4: PGlite Test Database +**What:** Per-test Postgres instance using PGlite +**When to use:** `tests/helpers/db.ts` + +```typescript +// tests/helpers/db.ts +import { drizzle } from "drizzle-orm/pglite"; +import { migrate } from "drizzle-orm/pglite/migrator"; +import * as schema from "../../src/db/schema.ts"; + +export async function createTestDb() { + const db = drizzle({ schema }); + + // Apply migrations from the new PostgreSQL migration directory + await migrate(db, { migrationsFolder: "./drizzle-pg" }); + + // Seed default category + await db.insert(schema.categories).values({ name: "Uncategorized", icon: "package" }); + + return db; +} +``` + +### Pattern 5: Async Transaction +**What:** Convert sync transactions to async +**When to use:** 4 transaction sites (category delete, setup update, thread resolve/unresolve) + +```typescript +// BEFORE (SQLite sync): +db.transaction(() => { + db.update(items).set({ categoryId: 1 }).where(eq(items.categoryId, id)).run(); + db.delete(categories).where(eq(categories.id, id)).run(); +}); + +// AFTER (PostgreSQL async): +await db.transaction(async (tx) => { + await tx.update(items).set({ categoryId: 1 }).where(eq(items.categoryId, id)); + await tx.delete(categories).where(eq(categories.id, id)); +}); +``` + +### Pattern 6: Drizzle Config for PostgreSQL +**What:** Updated drizzle.config.ts +**When to use:** One-time config update + +```typescript +// drizzle.config.ts +import { defineConfig } from "drizzle-kit"; + +export default defineConfig({ + out: "./drizzle-pg", + schema: "./src/db/schema.ts", + dialect: "postgresql", + dbCredentials: { + url: process.env.DATABASE_URL || "postgresql://gearbox:gearbox@localhost:5432/gearbox", + }, +}); +``` + +### Anti-Patterns to Avoid +- **Mixing sync and async:** Do not leave any `.all()`, `.get()`, `.run()` calls -- they are SQLite-only methods +- **Forgetting await:** Every database call must be awaited; missing awaits will return Promise objects instead of data +- **Using `pushSchema` for tests:** While faster, `pushSchema` from `drizzle-kit/api` does not match production migration behavior -- use `migrate()` to catch migration issues early +- **Integer timestamps in Postgres:** Do not carry over `integer("col", { mode: "timestamp" })` -- use native `timestamp()` type +- **Keeping `bun:sqlite` imports in production code:** Only the migration script should import `bun:sqlite` + +## Don't Hand-Roll + +| Problem | Don't Build | Use Instead | Why | +|---------|-------------|-------------|-----| +| Connection pooling | Custom pool manager | `postgres` built-in pooling | Handles connection limits, idle timeout, reconnection | +| In-memory test DB | Docker Postgres containers | PGlite | Zero setup, sub-ms startup, real Postgres SQL | +| Schema migrations | Manual SQL files | `drizzle-kit generate` | Generates correct DDL from schema diff | +| Data type conversion | Manual column-by-column casting | Drizzle schema + postgres driver auto-coercion | Driver handles JS Date <-> Postgres timestamp, number <-> integer | + +**Key insight:** Drizzle ORM abstracts the SQLite/PostgreSQL differences at the query builder level. The schema definition and driver are the only things that change -- service query logic (select, where, join, insert, etc.) stays identical except for removing sync-only methods. + +## Common Pitfalls + +### Pitfall 1: Missing Await on Database Calls +**What goes wrong:** Route handlers return `Promise` instead of `Item`, leading to empty/broken JSON responses +**Why it happens:** Mechanical conversion misses an `await` in a handler that was previously sync +**How to avoid:** Make route handlers `async` if not already; TypeScript will flag return type mismatches if return types are annotated +**Warning signs:** Tests pass but return `{}` or undefined fields; API returns `{}` + +### Pitfall 2: `.get()` Does Not Exist on PostgreSQL Drizzle +**What goes wrong:** Runtime error: `.get is not a function` +**Why it happens:** `.get()` is a SQLite-only convenience method that returns a single row +**How to avoid:** Replace `.get()` with array destructuring: `const [row] = await db.select()...`; replace `.returning().get()` with `const [row] = await db.insert()...returning()` +**Warning signs:** TypeScript type errors if using strict mode + +### Pitfall 3: `serial` Auto-Increment Behavior in Postgres +**What goes wrong:** Data migration script inserts rows with explicit IDs but the `serial` sequence is not advanced, causing conflicts on next insert +**Why it happens:** PostgreSQL `serial` is backed by a sequence that is only auto-incremented on default inserts -- explicit ID inserts do not update the sequence +**How to avoid:** After data migration, reset sequences: `SELECT setval('table_id_seq', (SELECT MAX(id) FROM table))` +**Warning signs:** Duplicate key errors after migration when creating new records + +### Pitfall 4: Boolean Columns (OAuth `used` Field) +**What goes wrong:** SQLite uses `integer` for boolean (`0`/`1`); Postgres has native `boolean` type +**Why it happens:** Direct schema port without type adjustment +**How to avoid:** Use `boolean("used").notNull().default(false)` in pg-core schema; migration script must convert `0/1` to `false/true` +**Warning signs:** Type errors in OAuth code that checks `=== 0` or `=== 1` + +### Pitfall 5: Transaction Callback Must Be Async +**What goes wrong:** Transaction body runs sync but database calls inside return unresolved promises +**Why it happens:** Forgetting to make the transaction callback `async` and `await` internal operations +**How to avoid:** `await db.transaction(async (tx) => { await tx.update()... })` +**Warning signs:** Empty/partial data writes, no errors thrown + +### Pitfall 6: `createdAt` Default Function Mismatch +**What goes wrong:** `$defaultFn(() => new Date())` in SQLite schema is a JS-side default; Postgres `defaultNow()` is SQL-side +**Why it happens:** Different default mechanisms +**How to avoid:** Use `.defaultNow()` for all timestamp columns in pg-core schema (server-side default is more reliable) +**Warning signs:** Null timestamps when inserting without explicit values + +### Pitfall 7: Test `createTestDb()` Becomes Async +**What goes wrong:** All `beforeEach` blocks that call `createTestDb()` break +**Why it happens:** `createTestDb()` returns a Promise instead of a Drizzle instance +**How to avoid:** `beforeEach(async () => { db = await createTestDb(); })` in all 18 test files +**Warning signs:** `db.select is not a function` errors in every test + +### Pitfall 8: `Db` Type Changes +**What goes wrong:** `type Db = typeof prodDb` in services no longer matches PGlite-created instances in tests +**Why it happens:** `drizzle-orm/postgres-js` and `drizzle-orm/pglite` return different Drizzle instance types +**How to avoid:** Use a shared type or use the generic `PostgresJsDatabase` type that both drivers satisfy. Alternatively, use `ReturnType` from pglite driver which is compatible. +**Warning signs:** TypeScript errors when passing test DB to service functions + +## Code Examples + +### Data Migration Script Structure +```typescript +// scripts/migrate-sqlite-to-postgres.ts +import { Database } from "bun:sqlite"; +import { drizzle } from "drizzle-orm/postgres-js"; +import postgres from "postgres"; +import * as schema from "../src/db/schema.ts"; + +const sqlite = new Database(process.env.SQLITE_PATH || "gearbox.db"); +const pg = postgres(process.env.DATABASE_URL!); +const db = drizzle(pg, { schema }); + +async function migrateTable( + tableName: string, + pgTable: any, + transform: (row: any) => T +) { + const rows = sqlite.query(`SELECT * FROM ${tableName}`).all(); + console.log(`Migrating ${rows.length} ${tableName}...`); + + if (rows.length === 0) return; + + for (const row of rows) { + await db.insert(pgTable).values(transform(row as any)); + } +} + +async function resetSequences() { + const tables = ["categories", "items", "threads", "thread_candidates", + "setups", "setup_items", "users", "api_keys", + "oauth_clients", "oauth_codes", "oauth_tokens"]; + for (const table of tables) { + await pg`SELECT setval('${pg(table)}_id_seq', COALESCE((SELECT MAX(id) FROM ${pg(table)}), 0))`; + } +} + +async function main() { + // Migrate tables in dependency order (parents before children) + // 1. categories, users, settings + // 2. items, threads, sessions, api_keys, oauth_clients + // 3. thread_candidates, setups + // 4. setup_items + // Convert: unix timestamps -> Date objects, integer booleans -> booleans + + await resetSequences(); + await pg.end(); + sqlite.close(); + console.log("Migration complete!"); +} + +main().catch(console.error); +``` + +### Docker Compose Development +```yaml +# docker-compose.dev.yml +services: + postgres: + image: postgres:16-alpine + environment: + POSTGRES_USER: gearbox + POSTGRES_PASSWORD: gearbox + POSTGRES_DB: gearbox + ports: + - "5432:5432" + volumes: + - pgdata-dev:/var/lib/postgresql/data + healthcheck: + test: ["CMD-SHELL", "pg_isready -U gearbox"] + interval: 5s + timeout: 3s + retries: 5 + +volumes: + pgdata-dev: +``` + +### Docker Compose Production (updated) +```yaml +# docker-compose.yml +services: + postgres: + image: postgres:16-alpine + environment: + POSTGRES_USER: gearbox + POSTGRES_PASSWORD: ${POSTGRES_PASSWORD} + POSTGRES_DB: gearbox + volumes: + - pgdata:/var/lib/postgresql/data + healthcheck: + test: ["CMD-SHELL", "pg_isready -U gearbox"] + interval: 10s + timeout: 5s + retries: 5 + + app: + image: gearbox:latest + environment: + DATABASE_URL: postgresql://gearbox:${POSTGRES_PASSWORD}@postgres:5432/gearbox + GEARBOX_URL: ${GEARBOX_URL} + ports: + - "3000:3000" + depends_on: + postgres: + condition: service_healthy + volumes: + - uploads:/app/uploads + +volumes: + pgdata: + uploads: +``` + +### Updated Migration Runner +```typescript +// src/db/migrate.ts +import { drizzle } from "drizzle-orm/postgres-js"; +import { migrate } from "drizzle-orm/postgres-js/migrator"; +import postgres from "postgres"; + +const migrationClient = postgres(process.env.DATABASE_URL!, { max: 1 }); +const db = drizzle(migrationClient); + +await migrate(db, { migrationsFolder: "./drizzle-pg" }); +await migrationClient.end(); + +console.log("Migrations applied successfully"); +``` + +## Column Type Mapping + +| SQLite Column | pg-core Column | Notes | +|---------------|----------------|-------| +| `integer("id").primaryKey({ autoIncrement: true })` | `serial("id").primaryKey()` | `serial` = auto-incrementing 4-byte int | +| `text("name")` | `text("name")` | Identical | +| `real("weight_grams")` | `doublePrecision("weight_grams")` | 8-byte float, matches SQLite `real` precision | +| `integer("price_cents")` | `integer("price_cents")` | Identical | +| `integer("col", { mode: "timestamp" })` | `timestamp("col")` | Native Postgres timestamp; Drizzle returns JS Date | +| `integer("used").default(0)` | `boolean("used").default(false)` | Proper boolean type | +| `real("sort_order")` | `doublePrecision("sort_order")` | Or `real()` (4-byte) -- either works | +| `text("id").primaryKey()` (sessions) | `text("id").primaryKey()` | Identical | +| `text("key").primaryKey()` (settings) | `text("key").primaryKey()` | Identical | + +## State of the Art + +| Old Approach | Current Approach | When Changed | Impact | +|--------------|------------------|--------------|--------| +| `bun:sqlite` sync driver | `postgres` (postgres.js) async driver | This migration | All DB calls become async | +| `drizzle-orm/bun-sqlite` | `drizzle-orm/postgres-js` | This migration | Driver swap in one file | +| In-memory SQLite for tests | PGlite WASM Postgres for tests | This migration | Tests run real Postgres SQL | +| `drizzle-orm/bun-sql` (Bun native) | `postgres` (postgres.js) | N/A | Bun SQL has drizzle-kit incompatibilities; postgres.js is mature | + +## Scope of Change + +Summary of files that need modification: + +| Category | Files | Change Type | +|----------|-------|-------------| +| Schema | `src/db/schema.ts` | Full rewrite (sqlite-core to pg-core) | +| DB config | `src/db/index.ts` | Full rewrite (bun:sqlite to postgres.js) | +| Migrations | `src/db/migrate.ts` | Full rewrite (async, postgres migrator) | +| Seed | `src/db/seed.ts` | Async conversion | +| Drizzle config | `drizzle.config.ts` | Dialect + output path change | +| Services | 9 files in `src/server/services/` | Add async/await to all DB calls (~82 call sites) | +| Routes | 9 files in `src/server/routes/` | Add await to service calls, make handlers async | +| Server entry | `src/server/index.ts` | Async seed call | +| Test helper | `tests/helpers/db.ts` | Full rewrite (PGlite) | +| Service tests | 9 files in `tests/services/` | Async beforeEach + await all assertions | +| Route tests | 8 files in `tests/routes/` | Async createTestApp + await | +| MCP tests | `tests/mcp/tools.test.ts` | Async test DB | +| Docker | `docker-compose.dev.yml` (new), `docker-compose.yml` (new) | Postgres service definitions | +| Dockerfile | `Dockerfile` | Update: copy `drizzle-pg/`, remove SQLite-specific steps | +| Migration script | `scripts/migrate-sqlite-to-postgres.ts` (new) | Data migration | +| Package.json | `package.json` | Add `postgres`, `@electric-sql/pglite`; remove `better-sqlite3` | + +**Total: ~40 files touched, ~2 new files created** + +## Validation Architecture + +### Test Framework +| Property | Value | +|----------|-------| +| Framework | Bun test runner (built-in) | +| Config file | None (uses bun defaults) | +| Quick run command | `bun test tests/services/item.service.test.ts` | +| Full suite command | `bun test tests/` | + +### Phase Requirements to Test Map +| Req ID | Behavior | Test Type | Automated Command | File Exists? | +|--------|----------|-----------|-------------------|-------------| +| DB-01 | App runs on PostgreSQL | integration | `bun test tests/` (all tests use PGlite) | Existing (updated) | +| DB-02 | Async database operations | unit | `bun test tests/services/` | Existing (updated) | +| DB-03 | PGlite test infrastructure | unit | `bun test tests/services/item.service.test.ts -x` | Existing (updated) | +| DB-04 | SQLite data migration script | integration | `bun run scripts/migrate-sqlite-to-postgres.ts` | New (Wave 0) | +| DB-05 | Docker Compose Postgres | smoke | `docker compose -f docker-compose.dev.yml up -d && bun test tests/` | Manual verification | + +### Sampling Rate +- **Per task commit:** `bun test tests/services/item.service.test.ts -x` (fast single-file check) +- **Per wave merge:** `bun test tests/` (full suite) +- **Phase gate:** Full suite green + manual Docker Compose smoke test + +### Wave 0 Gaps +- [ ] `tests/helpers/db.ts` -- must be rewritten to PGlite before any other tests can run +- [ ] Migration files in `drizzle-pg/` -- must be generated before test helper can apply them +- [ ] `scripts/migrate-sqlite-to-postgres.ts` -- new file, needs at least a basic test or manual verification plan + +## Open Questions + +1. **PGlite + Bun test runner performance** + - What we know: PGlite works well with Vitest; Bun test runner is compatible + - What's unclear: Whether Bun's test runner parallel mode causes issues with PGlite WASM initialization + - Recommendation: Start with sequential tests; if slow, investigate parallelization + +2. **`Db` type compatibility between postgres.js and PGlite drivers** + - What we know: Both return Drizzle instances but with different generic type parameters + - What's unclear: Whether the types are structurally compatible without explicit casting + - Recommendation: Define a shared `AppDb` type alias; if types diverge, use a minimal interface or `any` for the DI parameter with runtime compatibility + +3. **Sequence reset in migration script** + - What we know: Explicit ID inserts do not advance Postgres sequences + - What's unclear: Exact syntax for `setval` with dynamic table names via postgres.js + - Recommendation: Use raw SQL via `postgres.unsafe()` or `db.execute(sql\`...\`)` for sequence resets + +## Environment Availability + +| Dependency | Required By | Available | Version | Fallback | +|------------|------------|-----------|---------|----------| +| Docker | Docker Compose dev/prod | Yes | 29.0.0 | -- | +| Docker Compose | Local Postgres service | Yes | 2.40.3 | -- | +| Bun | Runtime | Yes | 1.3.10 | -- | +| PostgreSQL (via Docker) | DB-01, DB-05 | Via Docker | 16-alpine (to pull) | -- | +| psql CLI | Debug/manual verification | No | -- | Use Docker exec or skip | + +**Missing dependencies with no fallback:** None + +**Missing dependencies with fallback:** +- psql CLI not installed locally -- use `docker exec` into Postgres container for manual queries + +## Sources + +### Primary (HIGH confidence) +- [Drizzle ORM PGlite docs](https://orm.drizzle.team/docs/connect-pglite) - Connection setup, migration API +- [Drizzle ORM PostgreSQL docs](https://orm.drizzle.team/docs/get-started-postgresql) - postgres.js and node-postgres driver setup +- [Drizzle ORM pg-core column types](https://orm.drizzle.team/docs/column-types/pg) - Column type definitions +- [Drizzle ORM migrations](https://orm.drizzle.team/docs/migrations) - Programmatic migration execution +- [Drizzle ORM Bun SQL](https://orm.drizzle.team/docs/connect-bun-sql) - Bun SQL driver (evaluated, not recommended) +- Project codebase: `src/db/schema.ts`, `src/db/index.ts`, `tests/helpers/db.ts`, all service files + +### Secondary (MEDIUM confidence) +- [Bun + PostgreSQL compatibility](https://github.com/oven-sh/bun/issues/6555) - Historical postgres.js issues (resolved) +- [drizzle-kit Bun SQL issue #4122](https://github.com/drizzle-team/drizzle-orm/issues/4122) - drizzle-kit push incompatibility with Bun SQL +- [npm registry](https://www.npmjs.com) - Current package versions verified 2026-04-04 + +### Tertiary (LOW confidence) +- [PGlite + Drizzle testing patterns](https://dev.to/benjamindaniel/how-to-test-your-nodejs-postgres-app-using-drizzle-pglite-4fb3) - Community patterns (Vitest-focused, may need adaptation for Bun test runner) + +## Metadata + +**Confidence breakdown:** +- Standard stack: HIGH - Drizzle ORM pg-core and postgres.js are mature, well-documented, verified against official docs +- Architecture: HIGH - Schema mapping is direct; async conversion is mechanical; DI pattern makes driver swap clean +- Pitfalls: HIGH - Based on known SQLite-to-Postgres differences and verified Drizzle API differences +- Testing (PGlite + Bun): MEDIUM - PGlite is well-documented with Vitest; Bun test runner compatibility is inferred but not directly verified + +**Research date:** 2026-04-04 +**Valid until:** 2026-05-04 (stable domain, Drizzle ORM and PGlite are mature)