docs: bring phase 14 planning files into worktree

2026-04-04 12:15:37 +02:00
parent f7c9f3dc94
commit f7048a267a
12 changed files with 2309 additions and 13 deletions
--- a/.planning/phases/14-postgresql-migration/14-05-PLAN.md
+++ b/.planning/phases/14-postgresql-migration/14-05-PLAN.md
@@ -0,0 +1,233 @@
+---
+phase: 14-postgresql-migration
+plan: 05
+type: execute
+wave: 2
+depends_on: [14-01]
+files_modified:
+  - scripts/migrate-sqlite-to-postgres.ts
+autonomous: true
+requirements: [DB-04]
+must_haves:
+  truths:
+    - "Script reads all data from SQLite file and writes it to PostgreSQL"
+    - "Integer timestamps are converted to Date objects for Postgres timestamp columns"
+    - "Boolean integers (0/1) are converted to true/false for Postgres boolean columns"
+    - "All IDs and foreign key relationships are preserved"
+    - "Serial sequences are reset after data migration to avoid duplicate key errors"
+  artifacts:
+    - path: "scripts/migrate-sqlite-to-postgres.ts"
+      provides: "One-time SQLite to Postgres data migration"
+      contains: "migrate-sqlite-to-postgres"
+  key_links:
+    - from: "scripts/migrate-sqlite-to-postgres.ts"
+      to: "src/db/schema.ts"
+      via: "import table definitions for typed inserts"
+      pattern: "import.*schema"
+---
+
+<objective>
+Create the one-time SQLite-to-PostgreSQL data migration script that reads from an existing SQLite database and writes all data into PostgreSQL with proper type conversions.
+
+Purpose: Existing users need to migrate their data from SQLite to Postgres without data loss (DB-04). This is a standalone script run once during the upgrade.
+Output: scripts/migrate-sqlite-to-postgres.ts
+</objective>
+
+<execution_context>
+@$HOME/.claude/get-shit-done/workflows/execute-plan.md
+@$HOME/.claude/get-shit-done/templates/summary.md
+</execution_context>
+
+<context>
+@.planning/PROJECT.md
+@.planning/ROADMAP.md
+@.planning/phases/14-postgresql-migration/14-CONTEXT.md
+@.planning/phases/14-postgresql-migration/14-RESEARCH.md
+@.planning/phases/14-postgresql-migration/14-01-SUMMARY.md
+
+@src/db/schema.ts
+</context>
+
+<interfaces>
+<!-- SQLite schema (current production data format): -->
+<!-- - Timestamps: stored as unix epoch integers (seconds since 1970) -->
+<!-- - Booleans: stored as integers (0 = false, 1 = true), only oauthCodes.used -->
+<!-- - Weights: stored as real (float) -->
+<!-- - IDs: auto-increment integers -->
+<!-- - settings: key (text PK) + value (text) -->
+<!-- - sessions: id (text PK) + userId (int) + expiresAt (int timestamp) -->
+
+<!-- PostgreSQL schema (target format after Plan 01): -->
+<!-- - Timestamps: native timestamp type (JS Date objects) -->
+<!-- - Booleans: native boolean type -->
+<!-- - Weights: doublePrecision -->
+<!-- - IDs: serial (auto-increment with sequence) -->
+
+Tables in dependency order:
+1. categories, users, settings (no foreign keys to other app tables)
+2. items, threads, sessions, apiKeys, oauthClients (FK to categories/users)
+3. threadCandidates, setups, oauthCodes, oauthTokens (FK to threads/etc)
+4. setupItems (FK to setups + items)
+</interfaces>
+
+<tasks>
+
+<task type="auto">
+  <name>Task 1: Create SQLite-to-Postgres migration script</name>
+  <files>scripts/migrate-sqlite-to-postgres.ts</files>
+  <read_first>src/db/schema.ts</read_first>
+  <action>
+Create `scripts/migrate-sqlite-to-postgres.ts` per D-04, D-05, D-06.
+
+```typescript
+// scripts/migrate-sqlite-to-postgres.ts
+import { Database } from "bun:sqlite";
+import { drizzle } from "drizzle-orm/postgres-js";
+import postgres from "postgres";
+import * as schema from "../src/db/schema.ts";
+```
+
+**Environment variables:**
+- `SQLITE_PATH` -- path to SQLite database file (default: `"gearbox.db"`)
+- `DATABASE_URL` -- PostgreSQL connection string (required)
+
+**Structure:**
+1. Open SQLite database read-only
+2. Connect to PostgreSQL via postgres.js + drizzle
+3. Migrate tables in dependency order (parents before children)
+4. Reset all serial sequences after migration
+5. Close both connections
+6. Print summary
+
+**Type conversion functions:**
+```typescript
+function unixToDate(unix: number | null): Date | null {
+  if (unix === null || unix === undefined) return null;
+  return new Date(unix * 1000); // Unix seconds to JS milliseconds
+}
+
+function intToBool(val: number | null): boolean {
+  return val === 1;
+}
+```
+
+**Migration order and transform functions for each table:**
+
+1. **categories** -- `id` (serial), `name`, `icon`, `createdAt` (unixToDate)
+2. **users** -- `id` (serial), `username`, `passwordHash`, `createdAt` (unixToDate)
+3. **settings** -- `key`, `value` (no transforms needed, text PK)
+4. **items** -- `id` (serial), `name`, `weightGrams`, `priceCents`, `categoryId`, `notes`, `productUrl`, `imageFilename`, `imageSourceUrl`, `quantity`, `createdAt` (unixToDate), `updatedAt` (unixToDate)
+5. **threads** -- `id` (serial), `name`, `status`, `resolvedCandidateId`, `categoryId`, `createdAt` (unixToDate), `updatedAt` (unixToDate)
+6. **sessions** -- `id` (text PK), `userId`, `expiresAt` (unixToDate)
+7. **apiKeys** -- `id` (serial), `name`, `keyHash`, `keyPrefix`, `createdAt` (unixToDate)
+8. **oauthClients** -- `id` (serial), `clientId`, `clientName`, `redirectUris`, `createdAt` (unixToDate)
+9. **threadCandidates** -- `id` (serial), all fields, `createdAt`/`updatedAt` (unixToDate), `sortOrder` (keep as number)
+10. **setups** -- `id` (serial), `name`, `createdAt`/`updatedAt` (unixToDate)
+11. **oauthCodes** -- `id` (serial), all fields, `expiresAt` (unixToDate), `used` (intToBool)
+12. **oauthTokens** -- `id` (serial), all fields, `expiresAt`/`refreshExpiresAt`/`createdAt` (unixToDate)
+13. **setupItems** -- `id` (serial), `setupId`, `itemId`, `classification`
+
+**For each table, use this pattern:**
+```typescript
+async function migrateTable(tableName: string, pgTable: any, transform: (row: any) => any) {
+  const rows = sqlite.query(`SELECT * FROM ${tableName}`).all();
+  console.log(`  ${tableName}: ${rows.length} rows`);
+  if (rows.length === 0) return;
+  
+  for (const row of rows) {
+    await db.insert(pgTable).values(transform(row));
+  }
+}
+```
+
+**Sequence reset after all data is migrated:**
+```typescript
+async function resetSequences() {
+  const tablesWithSerial = [
+    "categories", "items", "threads", "thread_candidates",
+    "setups", "setup_items", "users", "api_keys",
+    "oauth_clients", "oauth_codes", "oauth_tokens"
+  ];
+  
+  for (const table of tablesWithSerial) {
+    await sql`SELECT setval(pg_get_serial_sequence('${sql.raw(table)}', 'id'), COALESCE((SELECT MAX(id) FROM ${sql.raw(table)}), 0))`;
+  }
+}
+```
+
+Note: Use `db.execute(sql\`...\`)` from drizzle-orm for raw SQL, or use `pg\`...\`` from the postgres.js client directly. The `sql.raw()` helper is needed for dynamic table names.
+
+**Main function:**
+```typescript
+async function main() {
+  const sqlitePath = process.env.SQLITE_PATH || "gearbox.db";
+  const databaseUrl = process.env.DATABASE_URL;
+  
+  if (!databaseUrl) {
+    console.error("ERROR: DATABASE_URL environment variable is required");
+    process.exit(1);
+  }
+  
+  console.log(`Migrating from SQLite (${sqlitePath}) to PostgreSQL...`);
+  
+  const sqlite = new Database(sqlitePath, { readonly: true });
+  const pg = postgres(databaseUrl);
+  const db = drizzle(pg, { schema });
+  
+  // ... migrate all tables in order ...
+  // ... reset sequences ...
+  
+  await pg.end();
+  sqlite.close();
+  
+  console.log("Migration complete!");
+}
+
+main().catch((err) => {
+  console.error("Migration failed:", err);
+  process.exit(1);
+});
+```
+
+**Error handling per table:** Wrap each table migration in try/catch, log which table failed and which row (by ID if available), then re-throw. This aids debugging partial migrations.
+
+**Add to package.json scripts:**
+```json
+"db:migrate-from-sqlite": "bun run scripts/migrate-sqlite-to-postgres.ts"
+```
+  </action>
+  <verify>
+    <automated>test -f scripts/migrate-sqlite-to-postgres.ts && grep -q "bun:sqlite" scripts/migrate-sqlite-to-postgres.ts && grep -q "postgres" scripts/migrate-sqlite-to-postgres.ts && grep -q "setval" scripts/migrate-sqlite-to-postgres.ts && grep -q "unixToDate\|unix.*Date\|\\* 1000" scripts/migrate-sqlite-to-postgres.ts && bun run lint 2>&1 | tail -3 && echo "PASS" || echo "FAIL"</automated>
+  </verify>
+  <acceptance_criteria>
+    - scripts/migrate-sqlite-to-postgres.ts exists
+    - File imports from `bun:sqlite` (read-only) and `drizzle-orm/postgres-js` and `postgres`
+    - File imports schema from `../src/db/schema.ts`
+    - File contains a unix-to-Date conversion function (multiplies by 1000)
+    - File contains an integer-to-boolean conversion for `used` field
+    - File migrates all 13 tables in dependency order (categories and users before items and threads, etc.)
+    - File contains `setval` calls to reset serial sequences after migration
+    - File reads `DATABASE_URL` from environment and exits with error if missing
+    - File reads `SQLITE_PATH` from environment with default `"gearbox.db"`
+    - File opens SQLite in readonly mode
+    - package.json contains `"db:migrate-from-sqlite"` script
+  </acceptance_criteria>
+  <done>Migration script reads SQLite, writes to Postgres with type conversions, resets sequences. All IDs and FK relationships preserved per D-06.</done>
+</task>
+
+</tasks>
+
+<verification>
+- `bun run scripts/migrate-sqlite-to-postgres.ts --help` or similar does not crash on syntax errors (will fail on missing DATABASE_URL, which is expected)
+- Script contains all 13 table migrations
+- Script resets sequences for all tables with serial IDs
+- `bun run lint` passes
+</verification>
+
+<success_criteria>
+One-time migration script exists, handles all type conversions (timestamps, booleans), preserves IDs, resets sequences. Can be run with `DATABASE_URL=... SQLITE_PATH=... bun run scripts/migrate-sqlite-to-postgres.ts`. Lint passes.
+</success_criteria>
+
+<output>
+After completion, create `.planning/phases/14-postgresql-migration/14-05-SUMMARY.md`
+</output>