421 lines
18 KiB
Markdown
421 lines
18 KiB
Markdown
---
|
|
phase: 03-postgresql-support
|
|
plan: 01
|
|
type: execute
|
|
wave: 1
|
|
depends_on: []
|
|
files_modified:
|
|
- pkg/diunwebhook/postgres_store.go
|
|
- pkg/diunwebhook/migrate.go
|
|
- pkg/diunwebhook/migrations/postgres/0001_initial_schema.up.sql
|
|
- pkg/diunwebhook/migrations/postgres/0001_initial_schema.down.sql
|
|
- go.mod
|
|
- go.sum
|
|
autonomous: true
|
|
requirements: [DB-01, DB-03]
|
|
|
|
must_haves:
|
|
truths:
|
|
- "PostgresStore implements all 9 Store interface methods with PostgreSQL SQL syntax"
|
|
- "PostgreSQL baseline migration creates the same 3 tables as SQLite (updates, tags, tag_assignments)"
|
|
- "RunMigrations is renamed to RunSQLiteMigrations in migrate.go; RunPostgresMigrations exists for PostgreSQL"
|
|
- "Existing SQLite migration path is unchanged (backward compatible)"
|
|
- "Application compiles and all existing tests pass after adding PostgreSQL support code"
|
|
artifacts:
|
|
- path: "pkg/diunwebhook/postgres_store.go"
|
|
provides: "PostgresStore struct implementing Store interface"
|
|
exports: ["PostgresStore", "NewPostgresStore"]
|
|
- path: "pkg/diunwebhook/migrations/postgres/0001_initial_schema.up.sql"
|
|
provides: "PostgreSQL baseline schema"
|
|
contains: "CREATE TABLE IF NOT EXISTS updates"
|
|
- path: "pkg/diunwebhook/migrations/postgres/0001_initial_schema.down.sql"
|
|
provides: "PostgreSQL rollback"
|
|
contains: "DROP TABLE IF EXISTS"
|
|
- path: "pkg/diunwebhook/migrate.go"
|
|
provides: "RunSQLiteMigrations and RunPostgresMigrations functions"
|
|
exports: ["RunSQLiteMigrations", "RunPostgresMigrations"]
|
|
key_links:
|
|
- from: "pkg/diunwebhook/postgres_store.go"
|
|
to: "pkg/diunwebhook/store.go"
|
|
via: "implements Store interface"
|
|
pattern: "func \\(s \\*PostgresStore\\)"
|
|
- from: "pkg/diunwebhook/migrate.go"
|
|
to: "pkg/diunwebhook/migrations/postgres/"
|
|
via: "go:embed directive"
|
|
pattern: "go:embed migrations/postgres"
|
|
---
|
|
|
|
<objective>
|
|
Create the PostgresStore implementation and PostgreSQL migration infrastructure.
|
|
|
|
Purpose: Delivers the core persistence layer for PostgreSQL — all 9 Store methods ported from SQLiteStore with PostgreSQL-native SQL, plus the migration runner and baseline schema. This is the foundation that Plan 02 wires into main.go.
|
|
Output: postgres_store.go, PostgreSQL migration files, updated migrate.go with both RunSQLiteMigrations and RunPostgresMigrations.
|
|
</objective>
|
|
|
|
<execution_context>
|
|
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
|
@$HOME/.claude/get-shit-done/templates/summary.md
|
|
</execution_context>
|
|
|
|
<context>
|
|
@.planning/PROJECT.md
|
|
@.planning/ROADMAP.md
|
|
@.planning/STATE.md
|
|
@.planning/phases/03-postgresql-support/03-CONTEXT.md
|
|
@.planning/phases/03-postgresql-support/03-RESEARCH.md
|
|
|
|
@pkg/diunwebhook/store.go
|
|
@pkg/diunwebhook/sqlite_store.go
|
|
@pkg/diunwebhook/migrate.go
|
|
@pkg/diunwebhook/migrations/sqlite/0001_initial_schema.up.sql
|
|
@pkg/diunwebhook/migrations/sqlite/0001_initial_schema.down.sql
|
|
|
|
<interfaces>
|
|
<!-- Store interface that PostgresStore must implement -->
|
|
From pkg/diunwebhook/store.go:
|
|
```go
|
|
type Store interface {
|
|
UpsertEvent(event DiunEvent) error
|
|
GetUpdates() (map[string]UpdateEntry, error)
|
|
AcknowledgeUpdate(image string) (found bool, err error)
|
|
ListTags() ([]Tag, error)
|
|
CreateTag(name string) (Tag, error)
|
|
DeleteTag(id int) (found bool, err error)
|
|
AssignTag(image string, tagID int) error
|
|
UnassignTag(image string) error
|
|
TagExists(id int) (bool, error)
|
|
}
|
|
```
|
|
|
|
From pkg/diunwebhook/diunwebhook.go:
|
|
```go
|
|
type DiunEvent struct {
|
|
DiunVersion string `json:"diun_version"`
|
|
Hostname string `json:"hostname"`
|
|
Status string `json:"status"`
|
|
Provider string `json:"provider"`
|
|
Image string `json:"image"`
|
|
HubLink string `json:"hub_link"`
|
|
MimeType string `json:"mime_type"`
|
|
Digest string `json:"digest"`
|
|
Created time.Time `json:"created"`
|
|
Platform string `json:"platform"`
|
|
Metadata struct {
|
|
ContainerName string `json:"ctn_names"`
|
|
ContainerID string `json:"ctn_id"`
|
|
State string `json:"ctn_state"`
|
|
Status string `json:"ctn_status"`
|
|
} `json:"metadata"`
|
|
}
|
|
|
|
type Tag struct {
|
|
ID int `json:"id"`
|
|
Name string `json:"name"`
|
|
}
|
|
|
|
type UpdateEntry struct {
|
|
Event DiunEvent `json:"event"`
|
|
ReceivedAt time.Time `json:"received_at"`
|
|
Acknowledged bool `json:"acknowledged"`
|
|
Tag *Tag `json:"tag"`
|
|
}
|
|
```
|
|
|
|
From pkg/diunwebhook/migrate.go:
|
|
```go
|
|
//go:embed migrations/sqlite
|
|
var sqliteMigrations embed.FS
|
|
|
|
func RunMigrations(db *sql.DB) error { ... }
|
|
```
|
|
Import alias: `sqlitemigrate "github.com/golang-migrate/migrate/v4/database/sqlite"`
|
|
</interfaces>
|
|
</context>
|
|
|
|
<tasks>
|
|
|
|
<task type="auto">
|
|
<name>Task 1: Add pgx dependency, create PostgreSQL migrations, update migrate.go</name>
|
|
<read_first>
|
|
- pkg/diunwebhook/migrate.go (current RunMigrations implementation to rename)
|
|
- pkg/diunwebhook/migrations/sqlite/0001_initial_schema.up.sql (schema to translate)
|
|
- pkg/diunwebhook/migrations/sqlite/0001_initial_schema.down.sql (down migration to copy)
|
|
- go.mod (current dependencies)
|
|
</read_first>
|
|
<files>
|
|
pkg/diunwebhook/migrations/postgres/0001_initial_schema.up.sql,
|
|
pkg/diunwebhook/migrations/postgres/0001_initial_schema.down.sql,
|
|
pkg/diunwebhook/migrate.go,
|
|
go.mod,
|
|
go.sum
|
|
</files>
|
|
<action>
|
|
1. Install dependencies:
|
|
```
|
|
go get github.com/jackc/pgx/v5@v5.9.1
|
|
go get github.com/golang-migrate/migrate/v4/database/pgx/v5
|
|
```
|
|
|
|
2. Create `pkg/diunwebhook/migrations/postgres/0001_initial_schema.up.sql` with this exact content:
|
|
```sql
|
|
CREATE TABLE IF NOT EXISTS updates (
|
|
image TEXT PRIMARY KEY,
|
|
diun_version TEXT NOT NULL DEFAULT '',
|
|
hostname TEXT NOT NULL DEFAULT '',
|
|
status TEXT NOT NULL DEFAULT '',
|
|
provider TEXT NOT NULL DEFAULT '',
|
|
hub_link TEXT NOT NULL DEFAULT '',
|
|
mime_type TEXT NOT NULL DEFAULT '',
|
|
digest TEXT NOT NULL DEFAULT '',
|
|
created TEXT NOT NULL DEFAULT '',
|
|
platform TEXT NOT NULL DEFAULT '',
|
|
ctn_name TEXT NOT NULL DEFAULT '',
|
|
ctn_id TEXT NOT NULL DEFAULT '',
|
|
ctn_state TEXT NOT NULL DEFAULT '',
|
|
ctn_status TEXT NOT NULL DEFAULT '',
|
|
received_at TEXT NOT NULL,
|
|
acknowledged_at TEXT
|
|
);
|
|
|
|
CREATE TABLE IF NOT EXISTS tags (
|
|
id SERIAL PRIMARY KEY,
|
|
name TEXT NOT NULL UNIQUE
|
|
);
|
|
|
|
CREATE TABLE IF NOT EXISTS tag_assignments (
|
|
image TEXT PRIMARY KEY,
|
|
tag_id INTEGER NOT NULL REFERENCES tags(id) ON DELETE CASCADE
|
|
);
|
|
```
|
|
Key difference from SQLite: `SERIAL PRIMARY KEY` replaces `INTEGER PRIMARY KEY AUTOINCREMENT` for tags.id. All timestamp columns use TEXT (not TIMESTAMPTZ) to match SQLite scan logic per Pitfall 6 in RESEARCH.md.
|
|
|
|
3. Create `pkg/diunwebhook/migrations/postgres/0001_initial_schema.down.sql` with this exact content:
|
|
```sql
|
|
DROP TABLE IF EXISTS tag_assignments;
|
|
DROP TABLE IF EXISTS tags;
|
|
DROP TABLE IF EXISTS updates;
|
|
```
|
|
|
|
4. Rewrite `pkg/diunwebhook/migrate.go`:
|
|
- Rename `RunMigrations` to `RunSQLiteMigrations` (per RESEARCH.md recommendation)
|
|
- IMPORTANT: Only rename the function definition in migrate.go itself. Do NOT touch cmd/diunwebhook/main.go or pkg/diunwebhook/export_test.go — those call-site renames are handled in Plan 02.
|
|
- Add a second `//go:embed migrations/postgres` directive for `var postgresMigrations embed.FS`
|
|
- Add `RunPostgresMigrations(db *sql.DB) error` using `pgxmigrate "github.com/golang-migrate/migrate/v4/database/pgx/v5"` as the database driver
|
|
- The pgx migrate driver name string for `migrate.NewWithInstance` is `"pgx5"` (NOT "pgx" or "postgres" -- this is the registration name used by golang-migrate's pgx/v5 sub-package)
|
|
- Keep both functions in the same file (both drivers compile into the binary regardless per Pitfall 4 in RESEARCH.md)
|
|
- Full imports for the updated file:
|
|
```go
|
|
import (
|
|
"database/sql"
|
|
"embed"
|
|
"errors"
|
|
|
|
"github.com/golang-migrate/migrate/v4"
|
|
pgxmigrate "github.com/golang-migrate/migrate/v4/database/pgx/v5"
|
|
sqlitemigrate "github.com/golang-migrate/migrate/v4/database/sqlite"
|
|
"github.com/golang-migrate/migrate/v4/source/iofs"
|
|
_ "modernc.org/sqlite"
|
|
)
|
|
```
|
|
- RunPostgresMigrations body follows the exact same pattern as RunSQLiteMigrations but uses `postgresMigrations`, `"migrations/postgres"`, `pgxmigrate.WithInstance`, and `"pgx5"` as the database name
|
|
|
|
5. Because migrate.go renames `RunMigrations` to `RunSQLiteMigrations` but the call sites in main.go and export_test.go still reference the old name, the build will break temporarily. This is expected — Plan 02 (wave 2) updates those call sites. To verify this plan in isolation, the verify command uses `go build ./pkg/diunwebhook/` (package only, not `./...`) and `go vet ./pkg/diunwebhook/`.
|
|
|
|
6. Run `go mod tidy` to clean up go.sum.
|
|
</action>
|
|
<verify>
|
|
<automated>cd /home/jean-luc-makiola/Development/projects/DiunDashboard && go build ./pkg/diunwebhook/ && go vet ./pkg/diunwebhook/</automated>
|
|
</verify>
|
|
<acceptance_criteria>
|
|
- pkg/diunwebhook/migrations/postgres/0001_initial_schema.up.sql contains `SERIAL PRIMARY KEY`
|
|
- pkg/diunwebhook/migrations/postgres/0001_initial_schema.up.sql contains `CREATE TABLE IF NOT EXISTS updates`
|
|
- pkg/diunwebhook/migrations/postgres/0001_initial_schema.up.sql contains `CREATE TABLE IF NOT EXISTS tags`
|
|
- pkg/diunwebhook/migrations/postgres/0001_initial_schema.up.sql contains `CREATE TABLE IF NOT EXISTS tag_assignments`
|
|
- pkg/diunwebhook/migrations/postgres/0001_initial_schema.down.sql contains `DROP TABLE IF EXISTS`
|
|
- pkg/diunwebhook/migrate.go contains `func RunSQLiteMigrations(db *sql.DB) error`
|
|
- pkg/diunwebhook/migrate.go contains `func RunPostgresMigrations(db *sql.DB) error`
|
|
- pkg/diunwebhook/migrate.go contains `//go:embed migrations/postgres`
|
|
- pkg/diunwebhook/migrate.go contains `pgxmigrate "github.com/golang-migrate/migrate/v4/database/pgx/v5"`
|
|
- pkg/diunwebhook/migrate.go contains `"pgx5"` (driver name in NewWithInstance call)
|
|
- go.mod contains `github.com/jackc/pgx/v5`
|
|
- `go build ./pkg/diunwebhook/` exits 0
|
|
- `go vet ./pkg/diunwebhook/` exits 0
|
|
</acceptance_criteria>
|
|
<done>PostgreSQL migration files exist with correct dialect. RunMigrations renamed to RunSQLiteMigrations in migrate.go. RunPostgresMigrations added. pgx/v5 dependency in go.mod. Package builds and vets cleanly.</done>
|
|
</task>
|
|
|
|
<task type="auto">
|
|
<name>Task 2: Create PostgresStore implementing all 9 Store methods</name>
|
|
<read_first>
|
|
- pkg/diunwebhook/store.go (interface contract to implement)
|
|
- pkg/diunwebhook/sqlite_store.go (reference implementation to port)
|
|
- pkg/diunwebhook/diunwebhook.go (DiunEvent, Tag, UpdateEntry type definitions)
|
|
</read_first>
|
|
<files>pkg/diunwebhook/postgres_store.go</files>
|
|
<action>
|
|
Create `pkg/diunwebhook/postgres_store.go` implementing all 9 Store interface methods.
|
|
|
|
Per D-01, D-02: Use `*sql.DB` (from `pgx/v5/stdlib`), not pgx native interface.
|
|
Per D-05: NO mutex -- PostgreSQL handles concurrent writes natively.
|
|
Per D-06: Pool config in constructor: `MaxOpenConns(25)`, `MaxIdleConns(5)`, `ConnMaxLifetime(5 * time.Minute)`.
|
|
Per D-03: Own raw SQL, no shared templates with SQLiteStore.
|
|
|
|
**Struct and constructor:**
|
|
```go
|
|
package diunwebhook
|
|
|
|
import (
|
|
"database/sql"
|
|
"time"
|
|
)
|
|
|
|
type PostgresStore struct {
|
|
db *sql.DB
|
|
}
|
|
|
|
func NewPostgresStore(db *sql.DB) *PostgresStore {
|
|
db.SetMaxOpenConns(25)
|
|
db.SetMaxIdleConns(5)
|
|
db.SetConnMaxLifetime(5 * time.Minute)
|
|
return &PostgresStore{db: db}
|
|
}
|
|
```
|
|
|
|
**Method-by-method port from SQLiteStore with these dialect changes:**
|
|
|
|
1. **UpsertEvent** -- Replace `?` with `$1..$15`, same ON CONFLICT pattern:
|
|
```go
|
|
func (s *PostgresStore) UpsertEvent(event DiunEvent) error {
|
|
_, err := s.db.Exec(`
|
|
INSERT INTO updates (
|
|
image, diun_version, hostname, status, provider,
|
|
hub_link, mime_type, digest, created, platform,
|
|
ctn_name, ctn_id, ctn_state, ctn_status,
|
|
received_at, acknowledged_at
|
|
) VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,NULL)
|
|
ON CONFLICT(image) DO UPDATE SET
|
|
diun_version = EXCLUDED.diun_version,
|
|
hostname = EXCLUDED.hostname,
|
|
status = EXCLUDED.status,
|
|
provider = EXCLUDED.provider,
|
|
hub_link = EXCLUDED.hub_link,
|
|
mime_type = EXCLUDED.mime_type,
|
|
digest = EXCLUDED.digest,
|
|
created = EXCLUDED.created,
|
|
platform = EXCLUDED.platform,
|
|
ctn_name = EXCLUDED.ctn_name,
|
|
ctn_id = EXCLUDED.ctn_id,
|
|
ctn_state = EXCLUDED.ctn_state,
|
|
ctn_status = EXCLUDED.ctn_status,
|
|
received_at = EXCLUDED.received_at,
|
|
acknowledged_at = NULL`,
|
|
event.Image, event.DiunVersion, event.Hostname, event.Status, event.Provider,
|
|
event.HubLink, event.MimeType, event.Digest,
|
|
event.Created.Format(time.RFC3339), event.Platform,
|
|
event.Metadata.ContainerName, event.Metadata.ContainerID,
|
|
event.Metadata.State, event.Metadata.Status,
|
|
time.Now().Format(time.RFC3339),
|
|
)
|
|
return err
|
|
}
|
|
```
|
|
|
|
2. **GetUpdates** -- Identical SQL to SQLiteStore (the SELECT query, JOINs, and COALESCE work in both dialects). Copy the full method body from sqlite_store.go verbatim -- the scan logic, time.Parse, and result building are all the same since timestamps are TEXT columns.
|
|
|
|
3. **AcknowledgeUpdate** -- Replace `datetime('now')` with `NOW()`, `?` with `$1`:
|
|
```go
|
|
res, err := s.db.Exec(`UPDATE updates SET acknowledged_at = NOW() WHERE image = $1`, image)
|
|
```
|
|
Return logic identical to SQLiteStore (check RowsAffected).
|
|
|
|
4. **ListTags** -- Identical SQL (`SELECT id, name FROM tags ORDER BY name`). Copy verbatim from SQLiteStore.
|
|
|
|
5. **CreateTag** -- CRITICAL: Do NOT use `Exec` + `LastInsertId` (pgx does not support LastInsertId). Use `QueryRow` with `RETURNING id`:
|
|
```go
|
|
func (s *PostgresStore) CreateTag(name string) (Tag, error) {
|
|
var id int
|
|
err := s.db.QueryRow(
|
|
`INSERT INTO tags (name) VALUES ($1) RETURNING id`, name,
|
|
).Scan(&id)
|
|
if err != nil {
|
|
return Tag{}, err
|
|
}
|
|
return Tag{ID: id, Name: name}, nil
|
|
}
|
|
```
|
|
|
|
6. **DeleteTag** -- Replace `?` with `$1`:
|
|
```go
|
|
res, err := s.db.Exec(`DELETE FROM tags WHERE id = $1`, id)
|
|
```
|
|
Return logic identical (check RowsAffected).
|
|
|
|
7. **AssignTag** -- Replace `INSERT OR REPLACE` with `INSERT ... ON CONFLICT DO UPDATE`:
|
|
```go
|
|
_, err := s.db.Exec(
|
|
`INSERT INTO tag_assignments (image, tag_id) VALUES ($1, $2)
|
|
ON CONFLICT (image) DO UPDATE SET tag_id = EXCLUDED.tag_id`,
|
|
image, tagID,
|
|
)
|
|
```
|
|
|
|
8. **UnassignTag** -- Replace `?` with `$1`:
|
|
```go
|
|
_, err := s.db.Exec(`DELETE FROM tag_assignments WHERE image = $1`, image)
|
|
```
|
|
|
|
9. **TagExists** -- Replace `?` with `$1`:
|
|
```go
|
|
err := s.db.QueryRow(`SELECT COUNT(*) FROM tags WHERE id = $1`, id).Scan(&count)
|
|
```
|
|
|
|
**IMPORTANT: No mutex.Lock/Unlock anywhere in PostgresStore** (per D-05). No `sync.Mutex` field in the struct.
|
|
</action>
|
|
<verify>
|
|
<automated>cd /home/jean-luc-makiola/Development/projects/DiunDashboard && go build ./pkg/diunwebhook/ && go vet ./pkg/diunwebhook/</automated>
|
|
</verify>
|
|
<acceptance_criteria>
|
|
- pkg/diunwebhook/postgres_store.go contains `type PostgresStore struct`
|
|
- pkg/diunwebhook/postgres_store.go contains `func NewPostgresStore(db *sql.DB) *PostgresStore`
|
|
- pkg/diunwebhook/postgres_store.go contains `func (s *PostgresStore) UpsertEvent(`
|
|
- pkg/diunwebhook/postgres_store.go contains `func (s *PostgresStore) GetUpdates(`
|
|
- pkg/diunwebhook/postgres_store.go contains `func (s *PostgresStore) AcknowledgeUpdate(`
|
|
- pkg/diunwebhook/postgres_store.go contains `func (s *PostgresStore) ListTags(`
|
|
- pkg/diunwebhook/postgres_store.go contains `func (s *PostgresStore) CreateTag(`
|
|
- pkg/diunwebhook/postgres_store.go contains `func (s *PostgresStore) DeleteTag(`
|
|
- pkg/diunwebhook/postgres_store.go contains `func (s *PostgresStore) AssignTag(`
|
|
- pkg/diunwebhook/postgres_store.go contains `func (s *PostgresStore) UnassignTag(`
|
|
- pkg/diunwebhook/postgres_store.go contains `func (s *PostgresStore) TagExists(`
|
|
- pkg/diunwebhook/postgres_store.go contains `RETURNING id` (CreateTag uses QueryRow, not LastInsertId)
|
|
- pkg/diunwebhook/postgres_store.go contains `ON CONFLICT (image) DO UPDATE SET tag_id = EXCLUDED.tag_id` (AssignTag)
|
|
- pkg/diunwebhook/postgres_store.go contains `NOW()` (AcknowledgeUpdate)
|
|
- pkg/diunwebhook/postgres_store.go contains `SetMaxOpenConns(25)` (constructor pool config)
|
|
- pkg/diunwebhook/postgres_store.go does NOT contain `sync.Mutex` (no mutex for PostgreSQL)
|
|
- pkg/diunwebhook/postgres_store.go does NOT contain `mu.Lock` (no mutex)
|
|
- `go build ./pkg/diunwebhook/` exits 0
|
|
- `go vet ./pkg/diunwebhook/` exits 0
|
|
</acceptance_criteria>
|
|
<done>PostgresStore implements all 9 Store interface methods with PostgreSQL-native SQL. No mutex. Pool settings configured. CreateTag uses RETURNING id. AssignTag uses ON CONFLICT DO UPDATE. Code compiles and passes vet.</done>
|
|
</task>
|
|
|
|
</tasks>
|
|
|
|
<verification>
|
|
1. `go build ./pkg/diunwebhook/` succeeds (both stores compile, migrate.go compiles with both drivers)
|
|
2. `go vet ./pkg/diunwebhook/` clean
|
|
3. PostgresStore has all 9 methods matching Store interface (compiler enforces this)
|
|
4. Migration files exist in both `migrations/sqlite/` and `migrations/postgres/`
|
|
5. Note: `go build ./...` and full test suite will fail until Plan 02 updates call sites in main.go and export_test.go that still reference the old `RunMigrations` name. This is expected.
|
|
</verification>
|
|
|
|
<success_criteria>
|
|
- PostgresStore compiles and implements Store interface (go build ./pkg/diunwebhook/ succeeds)
|
|
- PostgreSQL migration creates identical table structure to SQLite (3 tables: updates, tags, tag_assignments)
|
|
- pgx/v5 is in go.mod as a direct dependency
|
|
- migrate.go exports both RunSQLiteMigrations and RunPostgresMigrations
|
|
</success_criteria>
|
|
|
|
<output>
|
|
After completion, create `.planning/phases/03-postgresql-support/03-01-SUMMARY.md`
|
|
</output>
|