--- phase: 02-backend-refactor plan: 01 type: execute wave: 1 depends_on: [] files_modified: - pkg/diunwebhook/store.go - pkg/diunwebhook/sqlite_store.go - pkg/diunwebhook/migrate.go - pkg/diunwebhook/migrations/sqlite/0001_initial_schema.up.sql - pkg/diunwebhook/migrations/sqlite/0001_initial_schema.down.sql - go.mod - go.sum autonomous: true requirements: [REFAC-01, REFAC-03] must_haves: truths: - "A Store interface defines all 9 persistence operations with no SQL or *sql.DB in the contract" - "SQLiteStore implements every Store method using raw SQL and a sync.Mutex" - "RunMigrations applies embedded SQL files via golang-migrate and tolerates ErrNoChange" - "Migration 0001 creates the full current schema including acknowledged_at using CREATE TABLE IF NOT EXISTS" - "PRAGMA foreign_keys = ON is set in NewSQLiteStore before any queries" artifacts: - path: "pkg/diunwebhook/store.go" provides: "Store interface with 9 methods" exports: ["Store"] - path: "pkg/diunwebhook/sqlite_store.go" provides: "SQLiteStore struct implementing Store" exports: ["SQLiteStore", "NewSQLiteStore"] - path: "pkg/diunwebhook/migrate.go" provides: "RunMigrations function using golang-migrate + embed.FS" exports: ["RunMigrations"] - path: "pkg/diunwebhook/migrations/sqlite/0001_initial_schema.up.sql" provides: "Baseline schema DDL" contains: "CREATE TABLE IF NOT EXISTS updates" key_links: - from: "pkg/diunwebhook/sqlite_store.go" to: "pkg/diunwebhook/store.go" via: "interface implementation" pattern: "func \\(s \\*SQLiteStore\\)" - from: "pkg/diunwebhook/migrate.go" to: "pkg/diunwebhook/migrations/sqlite/" via: "embed.FS" pattern: "go:embed migrations/sqlite" --- Create the Store interface, SQLiteStore implementation, and golang-migrate migration infrastructure as new files alongside the existing code. Purpose: Establish the persistence abstraction layer and migration system that Plan 02 will wire into the Server struct and handlers. These are additive-only changes -- nothing existing breaks. Output: store.go, sqlite_store.go, migrate.go, migration SQL files, golang-migrate dependency installed. @$HOME/.claude/get-shit-done/workflows/execute-plan.md @$HOME/.claude/get-shit-done/templates/summary.md @.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/02-backend-refactor/02-RESEARCH.md From pkg/diunwebhook/diunwebhook.go: ```go type DiunEvent struct { DiunVersion string `json:"diun_version"` Hostname string `json:"hostname"` Status string `json:"status"` Provider string `json:"provider"` Image string `json:"image"` HubLink string `json:"hub_link"` MimeType string `json:"mime_type"` Digest string `json:"digest"` Created time.Time `json:"created"` Platform string `json:"platform"` Metadata struct { ContainerName string `json:"ctn_names"` ContainerID string `json:"ctn_id"` State string `json:"ctn_state"` Status string `json:"ctn_status"` } `json:"metadata"` } type Tag struct { ID int `json:"id"` Name string `json:"name"` } type UpdateEntry struct { Event DiunEvent `json:"event"` ReceivedAt time.Time `json:"received_at"` Acknowledged bool `json:"acknowledged"` Tag *Tag `json:"tag"` } ``` Task 1: Create Store interface and SQLiteStore implementation pkg/diunwebhook/store.go, pkg/diunwebhook/sqlite_store.go - pkg/diunwebhook/diunwebhook.go (current SQL operations to extract) - .planning/phases/02-backend-refactor/02-RESEARCH.md (Store interface design, SQL operations inventory) **Install golang-migrate dependency first:** ```bash cd /home/jean-luc-makiola/Development/projects/DiunDashboard go get github.com/golang-migrate/migrate/v4@v4.19.1 go get github.com/golang-migrate/migrate/v4/database/sqlite go get github.com/golang-migrate/migrate/v4/source/iofs ``` **Create `pkg/diunwebhook/store.go`** with exactly this interface (per REFAC-01): ```go package diunwebhook // Store defines all persistence operations. Implementations must be safe // for concurrent use from HTTP handlers. type Store interface { UpsertEvent(event DiunEvent) error GetUpdates() (map[string]UpdateEntry, error) AcknowledgeUpdate(image string) (found bool, err error) ListTags() ([]Tag, error) CreateTag(name string) (Tag, error) DeleteTag(id int) (found bool, err error) AssignTag(image string, tagID int) error UnassignTag(image string) error TagExists(id int) (bool, error) } ``` **Create `pkg/diunwebhook/sqlite_store.go`** with `SQLiteStore` struct implementing all 9 Store methods: ```go package diunwebhook import ( "database/sql" "sync" "time" ) type SQLiteStore struct { db *sql.DB mu sync.Mutex } func NewSQLiteStore(db *sql.DB) *SQLiteStore { return &SQLiteStore{db: db} } ``` Move all SQL from current handlers/functions into Store methods: 1. **UpsertEvent** -- move the INSERT...ON CONFLICT from current `UpdateEvent()` function. Keep exact same SQL including `ON CONFLICT(image) DO UPDATE SET` with all 14 columns and `acknowledged_at = NULL`. Use `time.Now().Format(time.RFC3339)` for received_at. Acquire `s.mu.Lock()`. 2. **GetUpdates** -- move the SELECT...LEFT JOIN from current `GetUpdates()` function. Exact same query: `SELECT u.image, u.diun_version, ...` with LEFT JOIN on tag_assignments and tags. Same row scanning logic with `sql.NullInt64`/`sql.NullString` for tag fields. No mutex needed (read-only). 3. **AcknowledgeUpdate** -- move SQL from `DismissHandler`: `UPDATE updates SET acknowledged_at = datetime('now') WHERE image = ?`. Return `(found bool, err error)` where found = RowsAffected() > 0. Acquire `s.mu.Lock()`. 4. **ListTags** -- move SQL from `TagsHandler` GET case: `SELECT id, name FROM tags ORDER BY name`. Return `([]Tag, error)`. No mutex. 5. **CreateTag** -- move SQL from `TagsHandler` POST case: `INSERT INTO tags (name) VALUES (?)`. Return `(Tag{ID: int(lastInsertId), Name: name}, error)`. Acquire `s.mu.Lock()`. 6. **DeleteTag** -- move SQL from `TagByIDHandler`: `DELETE FROM tags WHERE id = ?`. Return `(found bool, err error)` where found = RowsAffected() > 0. Acquire `s.mu.Lock()`. 7. **AssignTag** -- move SQL from `TagAssignmentHandler` PUT case: `INSERT OR REPLACE INTO tag_assignments (image, tag_id) VALUES (?, ?)`. Keep `INSERT OR REPLACE` (correct for SQLite, per research Pitfall 6). Acquire `s.mu.Lock()`. 8. **UnassignTag** -- move SQL from `TagAssignmentHandler` DELETE case: `DELETE FROM tag_assignments WHERE image = ?`. Acquire `s.mu.Lock()`. 9. **TagExists** -- move SQL from `TagAssignmentHandler` PUT check: `SELECT COUNT(*) FROM tags WHERE id = ?`. Return `(bool, error)` where bool = count > 0. No mutex (read-only). **CRITICAL:** `NewSQLiteStore` must run `PRAGMA foreign_keys = ON` on the db connection and `db.SetMaxOpenConns(1)` -- these currently live in `InitDB` and must NOT be lost. Specifically: ```go func NewSQLiteStore(db *sql.DB) *SQLiteStore { db.SetMaxOpenConns(1) // PRAGMA foreign_keys must be set per-connection; with MaxOpenConns(1) this covers all queries db.Exec("PRAGMA foreign_keys = ON") return &SQLiteStore{db: db} } ``` **rows.Close() pattern:** Use `defer rows.Close()` directly (not the verbose closure pattern from the current code). The error from Close() is safe to ignore in read paths. cd /home/jean-luc-makiola/Development/projects/DiunDashboard && go build ./pkg/diunwebhook/ && echo "BUILD OK" - pkg/diunwebhook/store.go contains `type Store interface {` - pkg/diunwebhook/store.go contains exactly these 9 method signatures: UpsertEvent, GetUpdates, AcknowledgeUpdate, ListTags, CreateTag, DeleteTag, AssignTag, UnassignTag, TagExists - pkg/diunwebhook/sqlite_store.go contains `type SQLiteStore struct {` - pkg/diunwebhook/sqlite_store.go contains `func NewSQLiteStore(db *sql.DB) *SQLiteStore` - pkg/diunwebhook/sqlite_store.go contains `db.SetMaxOpenConns(1)` - pkg/diunwebhook/sqlite_store.go contains `PRAGMA foreign_keys = ON` - pkg/diunwebhook/sqlite_store.go contains `func (s *SQLiteStore) UpsertEvent(event DiunEvent) error` - pkg/diunwebhook/sqlite_store.go contains `s.mu.Lock()` (mutex usage in write methods) - pkg/diunwebhook/sqlite_store.go contains `INSERT OR REPLACE INTO tag_assignments` (not ON CONFLICT for this table) - pkg/diunwebhook/sqlite_store.go contains `ON CONFLICT(image) DO UPDATE SET` (UPSERT for updates table) - `go build ./pkg/diunwebhook/` exits 0 Store interface defines 9 methods; SQLiteStore implements all 9 with exact SQL from current handlers; package compiles with no errors Task 2: Create migration infrastructure and SQL files pkg/diunwebhook/migrate.go, pkg/diunwebhook/migrations/sqlite/0001_initial_schema.up.sql, pkg/diunwebhook/migrations/sqlite/0001_initial_schema.down.sql - pkg/diunwebhook/diunwebhook.go (current DDL in InitDB to extract) - .planning/phases/02-backend-refactor/02-RESEARCH.md (RunMigrations pattern, migration file design, Pitfall 2 and 4) **Create migration SQL files:** Create directory `pkg/diunwebhook/migrations/sqlite/`. **`0001_initial_schema.up.sql`** -- Full current schema as a single baseline migration. Use `CREATE TABLE IF NOT EXISTS` for backward compatibility with existing databases (per research recommendation): ```sql CREATE TABLE IF NOT EXISTS updates ( image TEXT PRIMARY KEY, diun_version TEXT NOT NULL DEFAULT '', hostname TEXT NOT NULL DEFAULT '', status TEXT NOT NULL DEFAULT '', provider TEXT NOT NULL DEFAULT '', hub_link TEXT NOT NULL DEFAULT '', mime_type TEXT NOT NULL DEFAULT '', digest TEXT NOT NULL DEFAULT '', created TEXT NOT NULL DEFAULT '', platform TEXT NOT NULL DEFAULT '', ctn_name TEXT NOT NULL DEFAULT '', ctn_id TEXT NOT NULL DEFAULT '', ctn_state TEXT NOT NULL DEFAULT '', ctn_status TEXT NOT NULL DEFAULT '', received_at TEXT NOT NULL, acknowledged_at TEXT ); CREATE TABLE IF NOT EXISTS tags ( id INTEGER PRIMARY KEY AUTOINCREMENT, name TEXT NOT NULL UNIQUE ); CREATE TABLE IF NOT EXISTS tag_assignments ( image TEXT PRIMARY KEY, tag_id INTEGER NOT NULL REFERENCES tags(id) ON DELETE CASCADE ); ``` **`0001_initial_schema.down.sql`** -- Reverse of up migration: ```sql DROP TABLE IF EXISTS tag_assignments; DROP TABLE IF EXISTS tags; DROP TABLE IF EXISTS updates; ``` **Create `pkg/diunwebhook/migrate.go`:** ```go package diunwebhook import ( "database/sql" "embed" "errors" "github.com/golang-migrate/migrate/v4" sqlitemigrate "github.com/golang-migrate/migrate/v4/database/sqlite" "github.com/golang-migrate/migrate/v4/source/iofs" _ "modernc.org/sqlite" ) //go:embed migrations/sqlite var sqliteMigrations embed.FS // RunMigrations applies all pending schema migrations to the given SQLite database. // Returns nil if all migrations applied successfully or if database is already up to date. func RunMigrations(db *sql.DB) error { src, err := iofs.New(sqliteMigrations, "migrations/sqlite") if err != nil { return err } driver, err := sqlitemigrate.WithInstance(db, &sqlitemigrate.Config{}) if err != nil { return err } m, err := migrate.NewWithInstance("iofs", src, "sqlite", driver) if err != nil { return err } if err := m.Up(); err != nil && !errors.Is(err, migrate.ErrNoChange) { return err } return nil } ``` **CRITICAL imports:** - Use `database/sqlite` (NOT `database/sqlite3`) -- the sqlite3 variant requires CGO which is forbidden - Import alias `sqlitemigrate` for `github.com/golang-migrate/migrate/v4/database/sqlite` to avoid collision with the blank import of `modernc.org/sqlite` - The `_ "modernc.org/sqlite"` blank import must be present so the "sqlite" driver is registered for `sql.Open` **After creating files, run:** ```bash cd /home/jean-luc-makiola/Development/projects/DiunDashboard && go mod tidy ``` cd /home/jean-luc-makiola/Development/projects/DiunDashboard && go build ./pkg/diunwebhook/ && go vet ./pkg/diunwebhook/ && echo "BUILD+VET OK" - pkg/diunwebhook/migrate.go contains `//go:embed migrations/sqlite` - pkg/diunwebhook/migrate.go contains `func RunMigrations(db *sql.DB) error` - pkg/diunwebhook/migrate.go contains `!errors.Is(err, migrate.ErrNoChange)` (Pitfall 2 guard) - pkg/diunwebhook/migrate.go contains `database/sqlite` import (NOT `database/sqlite3`) - pkg/diunwebhook/migrations/sqlite/0001_initial_schema.up.sql contains `CREATE TABLE IF NOT EXISTS updates` - pkg/diunwebhook/migrations/sqlite/0001_initial_schema.up.sql contains `CREATE TABLE IF NOT EXISTS tags` - pkg/diunwebhook/migrations/sqlite/0001_initial_schema.up.sql contains `CREATE TABLE IF NOT EXISTS tag_assignments` - pkg/diunwebhook/migrations/sqlite/0001_initial_schema.up.sql contains `acknowledged_at TEXT` (included in baseline, not a separate migration) - pkg/diunwebhook/migrations/sqlite/0001_initial_schema.up.sql contains `ON DELETE CASCADE` - pkg/diunwebhook/migrations/sqlite/0001_initial_schema.down.sql contains `DROP TABLE IF EXISTS` - `go build ./pkg/diunwebhook/` exits 0 - `go vet ./pkg/diunwebhook/` exits 0 - go.mod contains `github.com/golang-migrate/migrate/v4` Migration files exist with full current schema as baseline; RunMigrations function compiles and handles ErrNoChange; golang-migrate v4.19.1 in go.mod; go vet passes - `go build ./pkg/diunwebhook/` compiles without errors (new files coexist with existing code) - `go vet ./pkg/diunwebhook/` reports no issues - `go test ./pkg/diunwebhook/` still passes (existing tests unchanged, new files are additive only) - go.mod contains golang-migrate v4 dependency - No CGO: `go mod graph | grep sqlite3` returns empty (no mattn/go-sqlite3 pulled in) - Store interface with 9 methods exists in store.go - SQLiteStore implements all 9 methods in sqlite_store.go with exact SQL semantics from current handlers - NewSQLiteStore sets PRAGMA foreign_keys = ON and MaxOpenConns(1) - RunMigrations in migrate.go uses golang-migrate + embed.FS + iofs, handles ErrNoChange - Migration 0001 contains full current schema with CREATE TABLE IF NOT EXISTS - All existing tests still pass (no existing code modified) - No CGO dependency introduced After completion, create `.planning/phases/02-backend-refactor/02-01-SUMMARY.md`