docs(14-05): complete SQLite-to-Postgres migration script plan
- SUMMARY.md with execution results - STATE.md updated with plan 05 completion - ROADMAP.md updated with phase 14 progress - DB-04 requirement marked complete
This commit is contained in:
@@ -12,7 +12,7 @@ Requirements for this milestone. Each maps to roadmap phases.
|
|||||||
- [ ] **DB-01**: Application runs on PostgreSQL instead of SQLite
|
- [ ] **DB-01**: Application runs on PostgreSQL instead of SQLite
|
||||||
- [ ] **DB-02**: All service functions use async database operations
|
- [ ] **DB-02**: All service functions use async database operations
|
||||||
- [ ] **DB-03**: Test infrastructure uses PGlite instead of bun:sqlite in-memory databases
|
- [ ] **DB-03**: Test infrastructure uses PGlite instead of bun:sqlite in-memory databases
|
||||||
- [ ] **DB-04**: Existing SQLite data can be migrated to Postgres via a one-time script
|
- [x] **DB-04**: Existing SQLite data can be migrated to Postgres via a one-time script
|
||||||
- [ ] **DB-05**: Docker Compose provides Postgres for local development
|
- [ ] **DB-05**: Docker Compose provides Postgres for local development
|
||||||
|
|
||||||
### Authentication
|
### Authentication
|
||||||
@@ -119,7 +119,7 @@ Which phases cover which requirements. Updated during roadmap creation.
|
|||||||
| DB-01 | Phase 14 | Pending |
|
| DB-01 | Phase 14 | Pending |
|
||||||
| DB-02 | Phase 14 | Pending |
|
| DB-02 | Phase 14 | Pending |
|
||||||
| DB-03 | Phase 14 | Pending |
|
| DB-03 | Phase 14 | Pending |
|
||||||
| DB-04 | Phase 14 | Pending |
|
| DB-04 | Phase 14 | Complete |
|
||||||
| DB-05 | Phase 14 | Pending |
|
| DB-05 | Phase 14 | Pending |
|
||||||
| AUTH-01 | Phase 15 | Pending |
|
| AUTH-01 | Phase 15 | Pending |
|
||||||
| AUTH-02 | Phase 15 | Pending |
|
| AUTH-02 | Phase 15 | Pending |
|
||||||
|
|||||||
@@ -188,7 +188,7 @@ Plans:
|
|||||||
| 11. Candidate Ranking | v1.3 | 2/2 | Complete | 2026-03-16 |
|
| 11. Candidate Ranking | v1.3 | 2/2 | Complete | 2026-03-16 |
|
||||||
| 12. Comparison View | v1.3 | 1/1 | Complete | 2026-03-17 |
|
| 12. Comparison View | v1.3 | 1/1 | Complete | 2026-03-17 |
|
||||||
| 13. Setup Impact Preview | v1.3 | 0/2 | Not started | - |
|
| 13. Setup Impact Preview | v1.3 | 0/2 | Not started | - |
|
||||||
| 14. PostgreSQL Migration | v2.0 | 0/? | Not started | - |
|
| 14. PostgreSQL Migration | v2.0 | 5/6 | In Progress | — |
|
||||||
| 15. External Authentication | v2.0 | 0/? | Not started | - |
|
| 15. External Authentication | v2.0 | 0/? | Not started | - |
|
||||||
| 16. Multi-User Data Model | v2.0 | 0/? | Not started | - |
|
| 16. Multi-User Data Model | v2.0 | 0/? | Not started | - |
|
||||||
| 17. Object Storage | v2.0 | 0/? | Not started | - |
|
| 17. Object Storage | v2.0 | 0/? | Not started | - |
|
||||||
|
|||||||
@@ -2,10 +2,10 @@
|
|||||||
gsd_state_version: 1.0
|
gsd_state_version: 1.0
|
||||||
milestone: v2.0
|
milestone: v2.0
|
||||||
milestone_name: Platform Foundation
|
milestone_name: Platform Foundation
|
||||||
status: planning
|
status: executing
|
||||||
stopped_at: null
|
stopped_at: Completed 14-05-PLAN.md
|
||||||
last_updated: "2026-04-03"
|
last_updated: "2026-04-04T10:28:29Z"
|
||||||
last_activity: 2026-04-03 — v2.0 roadmap created (Phases 14-18)
|
last_activity: 2026-04-04 — Completed 14-05 SQLite-to-Postgres migration script
|
||||||
progress:
|
progress:
|
||||||
total_phases: 5
|
total_phases: 5
|
||||||
completed_phases: 0
|
completed_phases: 0
|
||||||
@@ -26,9 +26,9 @@ See: .planning/PROJECT.md (updated 2026-04-03)
|
|||||||
## Current Position
|
## Current Position
|
||||||
|
|
||||||
Phase: 14 of 18 (PostgreSQL Migration)
|
Phase: 14 of 18 (PostgreSQL Migration)
|
||||||
Plan: 0 of ? in current phase
|
Plan: 5 of 6 in current phase
|
||||||
Status: Ready to plan
|
Status: Executing
|
||||||
Last activity: 2026-04-03 — v2.0 roadmap created (Phases 14-18)
|
Last activity: 2026-04-04 — Completed 14-05 SQLite-to-Postgres migration script
|
||||||
|
|
||||||
Progress: [----------] 0% (v2.0 milestone)
|
Progress: [----------] 0% (v2.0 milestone)
|
||||||
|
|
||||||
@@ -45,7 +45,9 @@ Progress: [----------] 0% (v2.0 milestone)
|
|||||||
|
|
||||||
### Decisions
|
### Decisions
|
||||||
|
|
||||||
Key decisions made during v2.0 planning:
|
Key decisions made during v2.0 execution:
|
||||||
|
- [14-05] Used postgres.js unsafe() for sequence reset DDL instead of drizzle-orm sql template
|
||||||
|
- [14-05] Row-by-row inserts for better error diagnostics during migration
|
||||||
- Platform pivot: single-user to multi-user with discovery-first approach
|
- Platform pivot: single-user to multi-user with discovery-first approach
|
||||||
- External auth provider (self-hosted, open-source) — Logto vs Authentik OPEN decision
|
- External auth provider (self-hosted, open-source) — Logto vs Authentik OPEN decision
|
||||||
- SQLite to Postgres migration — required by auth provider and multi-user concurrency
|
- SQLite to Postgres migration — required by auth provider and multi-user concurrency
|
||||||
@@ -64,6 +66,6 @@ None active.
|
|||||||
|
|
||||||
## Session Continuity
|
## Session Continuity
|
||||||
|
|
||||||
Last session: 2026-04-03
|
Last session: 2026-04-04T10:28:29Z
|
||||||
Stopped at: v2.0 roadmap created with 5 phases (14-18) covering 30 requirements
|
Stopped at: Completed 14-05-PLAN.md
|
||||||
Resume file: None
|
Resume file: None
|
||||||
|
|||||||
93
.planning/phases/14-postgresql-migration/14-05-SUMMARY.md
Normal file
93
.planning/phases/14-postgresql-migration/14-05-SUMMARY.md
Normal file
@@ -0,0 +1,93 @@
|
|||||||
|
---
|
||||||
|
phase: 14-postgresql-migration
|
||||||
|
plan: 05
|
||||||
|
subsystem: database
|
||||||
|
tags: [sqlite, postgres, migration, data-migration, bun-sqlite]
|
||||||
|
|
||||||
|
# Dependency graph
|
||||||
|
requires:
|
||||||
|
- phase: 14-01
|
||||||
|
provides: PostgreSQL schema definitions (Drizzle pgTable)
|
||||||
|
provides:
|
||||||
|
- One-time SQLite-to-PostgreSQL data migration script
|
||||||
|
- db:migrate-from-sqlite npm script
|
||||||
|
affects: [14-06, deployment, upgrade-docs]
|
||||||
|
|
||||||
|
# Tech tracking
|
||||||
|
tech-stack:
|
||||||
|
added: []
|
||||||
|
patterns: [dependency-ordered table migration, unix-to-Date conversion, serial sequence reset]
|
||||||
|
|
||||||
|
key-files:
|
||||||
|
created:
|
||||||
|
- scripts/migrate-sqlite-to-postgres.ts
|
||||||
|
modified:
|
||||||
|
- package.json
|
||||||
|
|
||||||
|
key-decisions:
|
||||||
|
- "Used postgres.js unsafe() for sequence reset instead of drizzle-orm sql template (simpler for raw DDL)"
|
||||||
|
- "Row-by-row insert for error tracing (per-row catch identifies failing record)"
|
||||||
|
|
||||||
|
patterns-established:
|
||||||
|
- "Migration scripts live in scripts/ directory"
|
||||||
|
- "Type conversion helpers (unixToDate, intToBool) for SQLite-to-Postgres data transforms"
|
||||||
|
|
||||||
|
requirements-completed: [DB-04]
|
||||||
|
|
||||||
|
# Metrics
|
||||||
|
duration: 2min
|
||||||
|
completed: 2026-04-04
|
||||||
|
---
|
||||||
|
|
||||||
|
# Phase 14 Plan 05: SQLite-to-Postgres Migration Script Summary
|
||||||
|
|
||||||
|
**One-time data migration script converting all 13 tables from SQLite to PostgreSQL with timestamp/boolean type conversions and serial sequence reset**
|
||||||
|
|
||||||
|
## Performance
|
||||||
|
|
||||||
|
- **Duration:** 2 min
|
||||||
|
- **Started:** 2026-04-04T10:26:29Z
|
||||||
|
- **Completed:** 2026-04-04T10:28:29Z
|
||||||
|
- **Tasks:** 1
|
||||||
|
- **Files modified:** 2
|
||||||
|
|
||||||
|
## Accomplishments
|
||||||
|
- Created standalone migration script that reads SQLite and writes to PostgreSQL
|
||||||
|
- Handles all type conversions: unix epoch integers to Date objects, integer booleans to native booleans
|
||||||
|
- Migrates tables in FK dependency order (4 waves: no-FK, FK-to-parents, FK-to-intermediates, junction tables)
|
||||||
|
- Resets all 11 serial sequences after migration to prevent duplicate key errors
|
||||||
|
- Added `db:migrate-from-sqlite` npm script for easy invocation
|
||||||
|
|
||||||
|
## Task Commits
|
||||||
|
|
||||||
|
Each task was committed atomically:
|
||||||
|
|
||||||
|
1. **Task 1: Create SQLite-to-Postgres migration script** - `b4c3813` (feat)
|
||||||
|
|
||||||
|
## Files Created/Modified
|
||||||
|
- `scripts/migrate-sqlite-to-postgres.ts` - One-time migration script with type conversions and sequence reset
|
||||||
|
- `package.json` - Added db:migrate-from-sqlite script
|
||||||
|
|
||||||
|
## Decisions Made
|
||||||
|
- Used `postgres.js` `unsafe()` for raw `setval` queries instead of drizzle-orm `sql` template -- simpler for dynamic table name interpolation in DDL
|
||||||
|
- Row-by-row inserts instead of bulk for better error diagnostics (each failed row logs its ID)
|
||||||
|
|
||||||
|
## Deviations from Plan
|
||||||
|
|
||||||
|
None - plan executed exactly as written.
|
||||||
|
|
||||||
|
## Issues Encountered
|
||||||
|
- Biome lint flagged unused `sql` import from drizzle-orm (used `pg.unsafe()` instead) and unnecessary suppression comments -- removed both
|
||||||
|
|
||||||
|
## User Setup Required
|
||||||
|
|
||||||
|
None - no external service configuration required.
|
||||||
|
|
||||||
|
## Next Phase Readiness
|
||||||
|
- Migration script ready for use during SQLite-to-Postgres upgrade
|
||||||
|
- Requires DATABASE_URL env var and existing SQLite file
|
||||||
|
- Can be tested against a dev Postgres instance with `docker compose up`
|
||||||
|
|
||||||
|
---
|
||||||
|
*Phase: 14-postgresql-migration*
|
||||||
|
*Completed: 2026-04-04*
|
||||||
Reference in New Issue
Block a user