From e4d59d4788f4c60c5edd6691b8a5242f6b9fb95a Mon Sep 17 00:00:00 2001 From: Jean-Luc Makiola Date: Mon, 23 Mar 2026 19:45:06 +0100 Subject: [PATCH] docs: complete project research --- .planning/research/ARCHITECTURE.md | 407 +++++++++++++++++++++++++++++ .planning/research/FEATURES.md | 146 +++++++++++ .planning/research/PITFALLS.md | 280 ++++++++++++++++++++ .planning/research/STACK.md | 185 +++++++++++++ .planning/research/SUMMARY.md | 181 +++++++++++++ 5 files changed, 1199 insertions(+) create mode 100644 .planning/research/ARCHITECTURE.md create mode 100644 .planning/research/FEATURES.md create mode 100644 .planning/research/PITFALLS.md create mode 100644 .planning/research/STACK.md create mode 100644 .planning/research/SUMMARY.md diff --git a/.planning/research/ARCHITECTURE.md b/.planning/research/ARCHITECTURE.md new file mode 100644 index 0000000..c8ebc0e --- /dev/null +++ b/.planning/research/ARCHITECTURE.md @@ -0,0 +1,407 @@ +# Architecture Patterns + +**Domain:** Container update dashboard with dual-database support +**Project:** DiunDashboard +**Researched:** 2026-03-23 +**Confidence:** HIGH (based on direct codebase analysis + established Go patterns) + +--- + +## Current Architecture (Before Milestone) + +The app is a single monolithic package (`pkg/diunwebhook/diunwebhook.go`) where database logic and HTTP handlers live in the same file and share package-level globals: + +``` +cmd/diunwebhook/main.go + └── pkg/diunwebhook/diunwebhook.go + ├── package-level var db *sql.DB ← global, opaque + ├── package-level var mu sync.Mutex ← global, opaque + ├── InitDB(), UpdateEvent(), GetUpdates() ← storage functions + └── WebhookHandler, UpdatesHandler, ... ← handlers call db directly +``` + +**The problem for dual-database support:** SQL is written inline in handler functions and storage functions using SQLite-specific syntax: +- `INSERT OR REPLACE` (SQLite only; PostgreSQL uses `INSERT ... ON CONFLICT DO UPDATE`) +- `datetime('now')` (SQLite only; PostgreSQL uses `NOW()`) +- `AUTOINCREMENT` (SQLite only; PostgreSQL uses `SERIAL` or `GENERATED ALWAYS AS IDENTITY`) +- `PRAGMA foreign_keys = ON` (SQLite only; PostgreSQL enforces FKs by default) +- `modernc.org/sqlite` driver import (SQLite only) + +There is no abstraction layer. Adding PostgreSQL directly to the current code would mean `if dialect == "postgres"` branches scattered across 380 lines — unmaintainable. + +--- + +## Recommended Architecture + +### Core Pattern: Repository Interface + +Extract all database operations behind a Go interface. Each database backend implements the interface. The HTTP handlers receive the interface, not a concrete `*sql.DB`. + +``` +cmd/diunwebhook/main.go + ├── reads DB_DRIVER env var ("sqlite" | "postgres") + ├── constructs concrete store (SQLiteStore or PostgresStore) + └── passes store to Server struct + +pkg/diunwebhook/ + ├── store.go ← Store interface definition + ├── sqlite.go ← SQLiteStore implements Store + ├── postgres.go ← PostgresStore implements Store + ├── server.go ← Server struct holds Store, secret; methods = handlers + ├── handlers.go ← HTTP handler methods on Server (no direct DB access) + └── models.go ← DiunEvent, UpdateEntry, Tag structs +``` + +### The Store Interface + +```go +// pkg/diunwebhook/store.go + +type Store interface { + // Lifecycle + Close() error + + // Updates + UpsertEvent(ctx context.Context, event DiunEvent) error + GetAllUpdates(ctx context.Context) (map[string]UpdateEntry, error) + AcknowledgeUpdate(ctx context.Context, image string) (found bool, err error) + AcknowledgeAll(ctx context.Context) error + AcknowledgeByTag(ctx context.Context, tagID int) error + + // Tags + ListTags(ctx context.Context) ([]Tag, error) + CreateTag(ctx context.Context, name string) (Tag, error) + DeleteTag(ctx context.Context, id int) (found bool, err error) + + // Tag assignments + AssignTag(ctx context.Context, image string, tagID int) error + UnassignTag(ctx context.Context, image string) error +} +``` + +**Why this interface boundary:** +- Handlers never import a database driver — they only call `Store` methods. +- Tests inject a fake/in-memory implementation with no database. +- Adding a third backend (e.g., MySQL) requires implementing the interface, not modifying handlers. +- The interface expresses domain intent (`AcknowledgeUpdate`) not SQL mechanics (`UPDATE SET acknowledged_at`). + +### Server Struct (Replaces Package Globals) + +```go +// pkg/diunwebhook/server.go + +type Server struct { + store Store + secret string +} + +func NewServer(store Store, secret string) *Server { + return &Server{store: store, secret: secret} +} + +// Handler methods: func (s *Server) WebhookHandler(w http.ResponseWriter, r *http.Request) +``` + +This addresses the "global mutable state" concern in CONCERNS.md. Multiple instances can coexist (useful for tests). Tests construct `NewServer(fakeStore, "")` without touching a real database. + +--- + +## Component Boundaries + +| Component | Responsibility | Communicates With | Location | +|-----------|---------------|-------------------|----------| +| `main.go` | Read env vars, construct store, wire server, run HTTP | `Server`, `SQLiteStore` or `PostgresStore` | `cmd/diunwebhook/` | +| `Server` | HTTP request lifecycle: parse, validate, delegate, respond | `Store` interface | `pkg/diunwebhook/server.go` | +| `Store` interface | Contract for all persistence operations | Implemented by `SQLiteStore`, `PostgresStore` | `pkg/diunwebhook/store.go` | +| `SQLiteStore` | All SQLite-specific SQL, schema init, migrations | `database/sql` + `modernc.org/sqlite` | `pkg/diunwebhook/sqlite.go` | +| `PostgresStore` | All PostgreSQL-specific SQL, schema init, migrations | `database/sql` + `pgx` stdlib driver | `pkg/diunwebhook/postgres.go` | +| `models.go` | Shared data structs (`DiunEvent`, `UpdateEntry`, `Tag`) | Imported by all components | `pkg/diunwebhook/models.go` | +| Frontend SPA | Visual dashboard, REST polling, drag-and-drop | HTTP API only (`/api/*`) | `frontend/src/` | + +**Strict boundary rules:** +- `Server` never imports `modernc.org/sqlite` or `pgx` — only `Store`. +- `SQLiteStore` and `PostgresStore` never import `net/http`. +- `main.go` is the only place that chooses which backend to construct. +- `models.go` has zero imports beyond stdlib. + +--- + +## Data Flow + +### Webhook Ingestion + +``` +DIUN (external) + POST /webhook + → Server.WebhookHandler + → validate auth header (constant-time compare) + → decode JSON into DiunEvent + → store.UpsertEvent(ctx, event) + → SQLiteStore: INSERT INTO updates ... ON CONFLICT(image) DO UPDATE SET ... + OR + → PostgresStore: INSERT INTO updates ... ON CONFLICT (image) DO UPDATE SET ... + → 200 OK +``` + +Both backends use standard SQL UPSERT syntax (fixing the current `INSERT OR REPLACE` bug). The SQL differs only in timestamp functions and driver-specific syntax, isolated to each store file. + +### Dashboard Polling + +``` +Browser (every 5s) + GET /api/updates + → Server.UpdatesHandler + → store.GetAllUpdates(ctx) + → SQLiteStore: SELECT ... LEFT JOIN ... (SQLite datetime handling) + OR + → PostgresStore: SELECT ... LEFT JOIN ... (PostgreSQL timestamp handling) + → encode map[string]UpdateEntry as JSON + → 200 OK with body +``` + +### Acknowledge Flow + +``` +Browser click + PATCH /api/updates/{image} + → Server.DismissHandler + → extract image from URL path + → store.AcknowledgeUpdate(ctx, image) + → SQLiteStore: UPDATE ... SET acknowledged_at = datetime('now') WHERE image = ? + OR + → PostgresStore: UPDATE ... SET acknowledged_at = NOW() WHERE image = $1 + → if not found: 404; else 204 +``` + +### Startup / Initialization + +``` +main() + → read DB_DRIVER env var ("sqlite" default, "postgres" opt-in) + → if sqlite: NewSQLiteStore(DB_PATH) → opens modernc/sqlite, runs migrations + → if postgres: NewPostgresStore(DSN) → opens pgx driver, runs migrations + → NewServer(store, WEBHOOK_SECRET) + → register handler methods on mux + → srv.ListenAndServe() +``` + +--- + +## Migration Strategy: Dual Schema Management + +Each store manages its own schema independently. No shared migration runner. + +### SQLiteStore migrations + +```go +func (s *SQLiteStore) migrate() error { + // Enable FK enforcement (fixes current bug) + s.db.Exec("PRAGMA foreign_keys = ON") + // Create tables with IF NOT EXISTS + // Apply ALTER TABLE migrations with error-ignore for idempotency + // Future: schema_version table for tracked migrations +} +``` + +### PostgresStore migrations + +```go +func (s *PostgresStore) migrate() error { + // CREATE TABLE IF NOT EXISTS with PostgreSQL syntax + // SERIAL or IDENTITY for auto-increment + // FK enforcement is on by default — no PRAGMA needed + // Timestamp columns as TIMESTAMPTZ not TEXT + // Future: schema_version table for tracked migrations +} +``` + +**Key difference:** SQLite stores timestamps as RFC3339 TEXT (current behavior, must be preserved for backward compatibility). PostgreSQL stores timestamps as `TIMESTAMPTZ`. Each store handles its own serialization/deserialization of `time.Time`. + +--- + +## Patterns to Follow + +### Pattern 1: Constructor-Injected Store + +**What:** `NewServer(store Store, secret string)` — store is a parameter, not a global. + +**When:** Always. This replaces `var db *sql.DB` and `var mu sync.Mutex` package globals. + +**Why:** Enables parallel test execution (each test creates its own `Server` with its own store). Eliminates the "single instance per process" constraint documented in CONCERNS.md. + +### Pattern 2: Context Propagation + +**What:** All `Store` interface methods accept `context.Context` as first argument. + +**When:** From the initial Store interface design — do not add it later. + +**Why:** Enables request cancellation and timeout propagation. PostgreSQL's `pgx` driver uses context natively. Without context, long-running queries cannot be cancelled when the client disconnects. + +### Pattern 3: Driver-Specific SQL Isolated in Store Files + +**What:** Each store file contains all SQL for that backend. No SQL strings in handlers. + +**When:** Any time a handler needs to read or write data — call a Store method instead. + +**Why:** SQLite uses `?` placeholders; PostgreSQL uses `$1, $2`. SQLite uses `datetime('now')`; PostgreSQL uses `NOW()`. SQLite uses `INTEGER PRIMARY KEY AUTOINCREMENT`; PostgreSQL uses `BIGSERIAL`. Mixing these in handler code creates unmaintainable conditional branches. + +### Pattern 4: Idempotent Schema Creation + +**What:** Both store constructors run schema setup on every startup via `CREATE TABLE IF NOT EXISTS`. + +**When:** In `NewSQLiteStore()` and `NewPostgresStore()` constructors. + +**Why:** Preserves current behavior where existing databases are safely upgraded. No external migration tool required for the current scope. + +--- + +## Anti-Patterns to Avoid + +### Anti-Pattern 1: Dialect Switches in Handlers + +**What:** `if s.dialect == "postgres" { query = "..." } else { query = "..." }` inside handler methods. + +**Why bad:** Handlers become aware of database internals. Every handler must be updated when adding a new backend. Tests must cover both branches per handler. + +**Instead:** Move all dialect differences into the Store implementation. Handlers call `store.AcknowledgeUpdate(ctx, image)` — they never see SQL. + +### Anti-Pattern 2: Shared `database/sql` Pool Exposed to Handlers + +**What:** Passing `*sql.DB` directly to handlers (as the current package globals effectively do). + +**Why bad:** Handlers can write arbitrary SQL, bypassing any abstraction. Type system cannot enforce the boundary. + +**Instead:** Expose only the `Store` interface to `Server`. The `*sql.DB` is a private field of `SQLiteStore` / `PostgresStore`. + +### Anti-Pattern 3: Single Store File for Both Backends + +**What:** One `store.go` file with SQLite and PostgreSQL implementations side by side. + +**Why bad:** The two implementations use different drivers, different SQL syntax, different connection setup. Merging them creates a large file with low cohesion. + +**Instead:** `sqlite.go` for `SQLiteStore`, `postgres.go` for `PostgresStore`. Both in `pkg/diunwebhook/` package. Build tags are not needed since both compile — `main.go` chooses at runtime. + +### Anti-Pattern 4: Reusing the Mutex from the Current Code + +**What:** Keeping `var mu sync.Mutex` as a package global once the Store abstraction is introduced. + +**Why bad:** `SQLiteStore` needs its own mutex (SQLite single-writer limitation). `PostgresStore` does not — PostgreSQL has its own concurrency control. Sharing a mutex across backends is wrong for Postgres and forces a false constraint. + +**Instead:** `SQLiteStore` embeds `sync.Mutex` as a private field. `PostgresStore` does not use a mutex — it relies on `pgx`'s connection pool. + +--- + +## Suggested Build Order + +The dependency graph dictates this order. Each step must complete before the next. + +### Step 1: Fix Current SQLite Bugs (prerequisite) + +Fix `INSERT OR REPLACE` → proper UPSERT, add `PRAGMA foreign_keys = ON`. These bugs exist independent of the refactor and will be harder to fix correctly after the abstraction layer is introduced. Do this on the current flat code, with tests confirming the fix. + +**Rationale:** Existing users rely on SQLite working correctly. The refactor must not change behavior — fixing bugs before refactoring means the tests that pass after bugfix become the regression suite for the refactor. + +### Step 2: Extract Models + +Move `DiunEvent`, `UpdateEntry`, `Tag` into `models.go`. No logic changes. This is a safe mechanical split — confirms the package compiles and tests pass after file reorganization. + +**Rationale:** Models are referenced by both Store implementations and by Server. Extracting them first removes a coupling that would otherwise force all files to reference a single monolith. + +### Step 3: Define Store Interface + SQLiteStore + +Define the `Store` interface in `store.go`. Implement `SQLiteStore` in `sqlite.go` by moving all SQL from the current monolith into `SQLiteStore` methods. All existing tests must still pass with zero behavior changes. This step does not add PostgreSQL — it only restructures. + +**Rationale:** Restructuring and new backend introduction must be separate commits. If tests break, the cause is isolated to the refactor, not the PostgreSQL code. + +### Step 4: Introduce Server Struct + +Refactor `pkg/diunwebhook/` to a struct-based design: `Server` with injected `Store`. Update `main.go` to construct `NewServer(store, secret)` and register `s.WebhookHandler` etc. on the mux. All existing tests must still pass. + +**Rationale:** This decouples handler tests from database initialization. Tests can now construct a `Server` with a stub `Store` — faster, no filesystem I/O, parallelisable. + +### Step 5: Implement PostgresStore + +Add `postgres.go` with `PostgresStore` implementing the `Store` interface. Add `pgx` (`github.com/jackc/pgx/v5`) as a dependency using its `database/sql` compatibility shim (`pgx/v5/stdlib`) to avoid changing the `*sql.DB` usage pattern in `SQLiteStore`. Add `DB_DRIVER` env var to `main.go` — `"sqlite"` (default) or `"postgres"`. Add `DATABASE_URL` env var for PostgreSQL DSN. Update `compose.dev.yml` and deployment docs. + +**Rationale:** `pgx/v5/stdlib` registers as a `database/sql` driver, so `PostgresStore` can use the same `*sql.DB` API as `SQLiteStore`. This minimizes the interface surface difference between the two implementations. + +### Step 6: Update Docker Compose and Configuration Docs + +Update `compose.dev.yml` with a `postgres` service profile. Update deployment documentation for PostgreSQL setup. This is explicitly the last step — infrastructure follows working code. + +--- + +## Scalability Considerations + +| Concern | SQLite (current) | PostgreSQL (new) | +|---------|-----------------|-----------------| +| Concurrent writes | Serialized by mutex + `SetMaxOpenConns(1)` | Connection pool, DB-level locking | +| Multiple server instances | Not possible (file lock) | Supported via shared DSN | +| Read performance | `LEFT JOIN` on every poll | Same query; can add indexes | +| Data retention | Unbounded growth | Same; retention policy deferred | +| Connection management | Single connection | `pgx` pool (default 5 conns) | + +For the self-hosted single-user target audience, both backends are more than sufficient. PostgreSQL is recommended when the user already runs a PostgreSQL instance (common in Coolify deployments) to avoid volume-mounting complexity and SQLite file permission issues. + +--- + +## Component Interaction Diagram + +``` +┌─────────────────────────────────────────────────────────┐ +│ cmd/diunwebhook/main.go │ +│ │ +│ DB_DRIVER=sqlite → NewSQLiteStore(DB_PATH) │ +│ DB_DRIVER=postgres → NewPostgresStore(DATABASE_URL) │ +│ │ │ +│ NewServer(store, secret)│ │ +└──────────────────────────┼──────────────────────────────┘ + │ + ▼ +┌──────────────────────────────────────────┐ +│ Server (pkg/diunwebhook/server.go) │ +│ │ +│ store Store ◄──── interface boundary │ +│ secret string │ +│ │ +│ .WebhookHandler() │ +│ .UpdatesHandler() │ +│ .DismissHandler() │ +│ .TagsHandler() │ +│ .TagByIDHandler() │ +│ .TagAssignmentHandler() │ +└──────────────┬───────────────────────────┘ + │ calls Store methods only + ▼ +┌──────────────────────────────────────────┐ +│ Store interface (store.go) │ +│ UpsertEvent / GetAllUpdates / │ +│ AcknowledgeUpdate / ListTags / ... │ +└────────────┬─────────────────┬───────────┘ + │ │ + ▼ ▼ +┌────────────────────┐ ┌──────────────────────┐ +│ SQLiteStore │ │ PostgresStore │ +│ (sqlite.go) │ │ (postgres.go) │ +│ │ │ │ +│ modernc.org/sqlite│ │ pgx/v5/stdlib │ +│ *sql.DB │ │ *sql.DB │ +│ sync.Mutex │ │ (no mutex needed) │ +│ SQLite SQL syntax │ │ PostgreSQL SQL syntax│ +└────────────────────┘ └──────────────────────┘ +``` + +--- + +## Sources + +- Direct analysis of `pkg/diunwebhook/diunwebhook.go` (current monolith) — HIGH confidence +- Direct analysis of `cmd/diunwebhook/main.go` (entry point) — HIGH confidence +- `.planning/codebase/CONCERNS.md` (identified tech debt) — HIGH confidence +- `.planning/PROJECT.md` (constraints: no CGO, backward compat, dual DB) — HIGH confidence +- Go `database/sql` standard library interface pattern — HIGH confidence (well-established Go idiom) +- `pgx/v5/stdlib` compatibility layer for `database/sql` — MEDIUM confidence (standard approach, verify exact import path during implementation) + +--- + +*Architecture research: 2026-03-23* diff --git a/.planning/research/FEATURES.md b/.planning/research/FEATURES.md new file mode 100644 index 0000000..061eada --- /dev/null +++ b/.planning/research/FEATURES.md @@ -0,0 +1,146 @@ +# Feature Landscape + +**Domain:** Container image update monitoring dashboard (self-hosted) +**Researched:** 2026-03-23 +**Confidence note:** Web search and WebFetch tools unavailable in this session. Findings are based on training-data knowledge of Portainer, Watchtower, Dockcheck-web, Diun, Uptime Kuma, and the self-hosted container tooling ecosystem. Confidence levels reflect this constraint. + +--- + +## Table Stakes + +Features users expect from any container monitoring dashboard. Missing any of these and the tool feels unfinished or untrustworthy. + +| Feature | Why Expected | Complexity | Notes | +|---------|--------------|------------|-------| +| Persistent update list (survives page reload, container restart) | Core value prop — the whole point is to not lose track of what needs updating | Low | Already exists but broken by SQLite bugs; fixing it is table stakes | +| Individual acknowledge/dismiss per image | Minimum viable workflow to mark "I dealt with this" | Low | Already exists | +| Bulk acknowledge — dismiss all | Without this, users with 20+ images must click 20+ times; abandonment is near-certain | Medium | Flagged in CONCERNS.md as missing; very high priority | +| Bulk acknowledge — dismiss by group/tag | If you've tagged a group and updated everything in it, dismissing one at a time is painful | Medium | Depends on tag feature existing (already does) | +| Search / filter by image name | Standard affordance in any list of 10+ items | Medium | Missing; flagged in PROJECT.md as active requirement | +| Filter by status (pending update vs acknowledged) | Separating signal from noise is core to the "nag until you fix it" value prop | Low | Missing; complements search | +| New-update indicator (badge, counter, or highlight) | Users need to know at a glance "something new arrived since I last checked" | Medium | Flagged in PROJECT.md as active requirement | +| Page/tab title update count | Gives browser-tab visibility without opening the page — "DiunDashboard (3)" in the tab | Low | Tiny implementation, high perceived value | +| Data integrity across restarts | If the DB loses data on restart, trust collapses | Medium | High-priority bug: INSERT OR REPLACE + missing FK pragma | +| PostgreSQL option for non-SQLite users | Self-hosters who run Postgres expect it as an option for persistent services | High | Flagged in PROJECT.md; dual-DB is the plan | + +--- + +## Differentiators + +Features not universally expected but meaningfully better than the baseline. Build these after table stakes are solid. + +| Feature | Value Proposition | Complexity | Notes | +|---------|-------------------|------------|-------| +| Filter by tag/group | Users who've organized images into groups want to scope their view | Low | Tag infrastructure already exists; filter is a frontend-only change | +| Visual "new since last visit" highlight (session-based) | Distinguish newly arrived updates from ones you've already seen | Medium | Requires client-side tracking of "last seen" timestamp (localStorage) | +| Toast / in-page notification on new update arrival (during polling) | Passive, non-intrusive signal when updates arrive while the tab is open | Medium | Uses existing 5-second poll; could compare prior state | +| Browser notification API on new update | Reaches users when the tab is backgrounded | High | Requires permission prompt; risky UX if over-notified; defer | +| Sort order controls (newest first, image name, registry) | Power-user need once list grows beyond 20 images | Low | Pure frontend sort; no backend change needed | +| Filter by registry | Useful for multi-registry setups | Low | Derived from image name; no schema change needed | +| Keyboard shortcuts (bulk dismiss with keypress, focus search) | Power users strongly value keyboard-driven UIs | Medium | Rarely table stakes for self-hosted tools but appreciated | +| Light / dark theme toggle (currently hardcoded dark) | Respects system preferences; accessibility baseline | Low | Flagged in CONCERNS.md; CSS variable change + prefers-color-scheme | +| Drag handle always visible (not hover-only) | Accessibility: keyboard and touch users need discoverable reordering | Low | Flagged in CONCERNS.md | +| Alternative to drag-and-drop for tag assignment | Dropdown select for assigning tags; removes dependency on pointer hover | Medium | Fixes accessibility gap in CONCERNS.md | +| Data retention / auto-cleanup of old acknowledged entries | Prevents unbounded DB growth over months/years | Medium | Configurable TTL for acknowledged records | + +--- + +## Anti-Features + +Features to deliberately NOT build in this milestone. + +| Anti-Feature | Why Avoid | What to Do Instead | +|--------------|-----------|-------------------| +| Auto-triggering image pulls or container restarts from the dashboard | This app is a viewer, not an orchestrator; acting on the host would require Docker socket access and creates a significant security surface | Remain read-only; users run `docker pull` / Coolify update themselves | +| Notification channel management UI (email, Slack, webhook routing) | DIUN already manages notification channels; duplicating this is wasted effort and creates config drift | Keep DIUN as the notification layer; this dashboard is the persistent record | +| OAuth / multi-user accounts | Single-user self-hosted tool; auth complexity is disproportionate to the use case | Document "don't expose to the public internet"; optional basic auth via reverse proxy is sufficient | +| Real-time WebSocket / SSE updates | The 5-second poll is adequate for this use case; SSE/WS adds complexity without meaningful UX gain for a low-frequency signal | Improve the poll with ETag/If-Modified-Since to reduce wasted bandwidth instead | +| Mobile-native / PWA features | Web-first responsive design is sufficient; self-hosters rarely need a fully offline-capable PWA for an internal tool | Ensure the layout is responsive for mobile browser access | +| Auto-grouping by Docker stack / Compose project | Requires Docker socket access or DIUN metadata changes; significant scope increase | Defer to a dedicated future milestone per PROJECT.md | +| DIUN config management UI | Requires DIUN bundling; out of scope for this milestone | Defer per PROJECT.md | +| Changelog or CVE lookups per image | Valuable but requires external API integrations (Docker Hub, Trivy, etc.); different product scope | Document as a possible future phase | +| Undo for dismiss actions | Adds state complexity; accidental dismisses are recoverable by the next DIUN scan | Keep dismiss as final; communicate this in the UI | + +--- + +## Feature Dependencies + +``` +Data integrity fixes (SQLite upsert + FK pragma) + → must precede all UX features (broken data undermines everything) + +PostgreSQL support + → depends on struct-based refactor (global state → Server struct) + → struct refactor is also a prerequisite for safe parallel tests + +Bulk acknowledge (all) + → no dependencies; purely additive API + frontend work + +Bulk acknowledge (by group) + → depends on tag feature (already exists) + +Search / filter by image name + → no backend dependency; frontend filter on existing GET /api/updates payload + +Filter by status + → no backend dependency; frontend filter + +Filter by tag + → depends on tag data being returned by GET /api/updates (already is) + +New-update indicator (badge/counter) + → depends on frontend comparing poll results across cycles + → no backend change needed + +Page title update count + → depends on update count being derivable from GET /api/updates (already is) + +Toast notification on new arrival + → depends on new-update indicator logic (same poll comparison) + → can share implementation + +Sort controls + → no dependencies; pure frontend + +Data retention / TTL + → depends on PostgreSQL support OR can be added to SQLite path independently + → no frontend dependency + +Light/dark theme + → no dependencies; CSS + localStorage + +Drag handle accessibility fix + → no dependencies + +Alternative tag assignment (dropdown) + → no dependencies +``` + +--- + +## MVP Recommendation for This Milestone + +The milestone goal is: bug fixes, dual DB, and UX improvements (bulk actions, filtering, search, new-update indicators). + +Prioritize in this order: + +1. **Fix SQLite data integrity** (UPSERT + FK pragma) — trust foundation; nothing else matters if data is lost +2. **Bulk acknowledge (all + by group)** — the single highest-impact UX addition; drops manual effort from O(n) to O(1) +3. **Search + filter by name/status/tag** — table stakes for any list of >10 items +4. **New-update indicator + page title count** — completes the "persistent visibility" core value with in-page signal +5. **PostgreSQL support** — requires struct refactor; large but well-scoped; enables users who need it +6. **Light/dark theme + accessibility fixes** — low complexity; removes known complaints + +Defer to next milestone: +- **Data retention / TTL**: Real but not urgent; unbounded growth is a future problem for most users +- **Toast notifications**: Nice to have but the badge + title count cover the signal adequately +- **Alternative tag assignment (dropdown)**: Accessibility improvement but drag-and-drop exists and works +- **Browser notification API**: High complexity, UX risk, very low reward vs. the badge approach + +--- + +## Sources + +- Project context: `.planning/PROJECT.md` (validated requirements and constraints) +- Codebase audit: `.planning/codebase/CONCERNS.md` (confirmed gaps: bulk ops, search, indicators, FK bugs) +- Training-data knowledge of: Portainer CE, Watchtower (no UI), Dockcheck-web, Diun native notifications, Uptime Kuma (comparable self-hosted monitoring dashboard UX patterns) — **MEDIUM confidence** (cannot be verified in this session due to tool restrictions; findings should be spot-checked against current Portainer docs and community forums before roadmap finalization) diff --git a/.planning/research/PITFALLS.md b/.planning/research/PITFALLS.md new file mode 100644 index 0000000..72bd5ac --- /dev/null +++ b/.planning/research/PITFALLS.md @@ -0,0 +1,280 @@ +# Domain Pitfalls + +**Domain:** Go dashboard — SQLite to dual-database (SQLite + PostgreSQL) migration + dashboard UX improvements +**Researched:** 2026-03-23 +**Confidence:** HIGH for SQLite/Go-specific pitfalls (sourced directly from codebase evidence); MEDIUM for PostgreSQL dialect differences (from training knowledge, verified against known Go `database/sql` contract) + +--- + +## Critical Pitfalls + +Mistakes that cause rewrites, data loss, or silent test passes. + +--- + +### Pitfall 1: Leaking SQLite-specific SQL into "shared" query layer + +**What goes wrong:** When adding a PostgreSQL path, developers copy existing SQLite queries and swap the driver — but keep SQLite-isms in the SQL itself. The two most common in this codebase: `datetime('now')` (SQLite built-in, line 225) and `INSERT OR REPLACE` (SQLite only, lines 109 and 352). Both fail silently or loudly on PostgreSQL. PostgreSQL uses `NOW()` and `INSERT ... ON CONFLICT DO UPDATE`. + +**Why it happens:** The queries are embedded as raw strings throughout handler functions rather than in a dedicated SQL layer. Each query must be individually audited and conditionally branched or abstracted. + +**Consequences:** PostgreSQL path silently produces wrong results (`datetime('now')` evaluates as a column name or throws an error) or panics on `INSERT OR REPLACE` (PostgreSQL does not support this syntax at all). + +**Warning signs:** +- Any raw `db.Exec` or `db.Query` call with `datetime(`, `OR REPLACE`, `AUTOINCREMENT`, `PRAGMA`, or `?` placeholders — all must be replaced or branched for PostgreSQL. +- `?` is the SQLite/MySQL placeholder; PostgreSQL requires `$1`, `$2`, etc. + +**Prevention:** +- Define a `Store` interface with methods (`UpsertEvent`, `GetUpdates`, `DismissImage`, etc.) and provide two concrete implementations: `sqliteStore` and `pgStore`. +- Never write raw SQL in HTTP handlers. All SQL lives in the store implementation only. +- Add an integration test that runs against both stores for every write operation; if the schema or SQL diverges the test fails at the driver level. + +**Phase mapping:** Must be resolved before any PostgreSQL code is written — this is the foundational refactor that makes dual-DB possible without a maintenance nightmare. + +--- + +### Pitfall 2: `INSERT OR REPLACE` silently deletes tag assignments before PostgreSQL is even added + +**What goes wrong:** `UpdateEvent()` (line 109) uses `INSERT OR REPLACE INTO updates`. SQLite implements this as DELETE + INSERT when a conflict is found. Because `tag_assignments.image` is a foreign key referencing `updates.image`, the DELETE step removes the child row — unless `PRAGMA foreign_keys = ON` is active (it is not, confirmed at line 58-103). Even with FK enforcement, the CASCADE would delete the assignment rather than preserve it. The result: every time DIUN sends a new event for a tracked image, its tag assignment vanishes. + +**Why it happens:** The intent of `INSERT OR REPLACE` is to update existing rows, but the mechanism is destructive. The UPSERT syntax (`INSERT ... ON CONFLICT(image) DO UPDATE SET ...`) is the correct tool and has been available since SQLite 3.24 (2018). + +**Consequences:** This bug is already in production. Users lose tag assignments every time an image receives a new DIUN event. This directly contributed to the trust erosion described in PROJECT.md. Adding PostgreSQL without fixing this first means the bug ships in both DB paths. + +**Warning signs:** +- Tag assignments disappear after DIUN reports a new update for a previously-tagged image. +- `TestDismissHandler_ReappearsAfterNewWebhook` tests the acknowledged-state reset correctly, but no test asserts that the tag survives a second `UpdateEvent` call on the same image. + +**Prevention:** +- Replace line 109 with: `INSERT INTO updates (...) VALUES (...) ON CONFLICT(image) DO UPDATE SET diun_version=excluded.diun_version, ...` (preserves all other columns, including nothing that touches `tag_assignments`). +- Add `PRAGMA foreign_keys = ON` immediately after `sql.Open` in `InitDB()`. +- Add a regression test: `UpdateEvent` twice on the same image with a tag assigned between calls; assert tag survives. + +**Phase mapping:** Fix before any other work — this is a data-correctness bug affecting existing users. + +--- + +### Pitfall 3: Global package-level state makes database abstraction structurally impossible without a refactor + +**What goes wrong:** The codebase uses `var db *sql.DB` and `var mu sync.Mutex` at package level (lines 48-52). The `InitDB` function sets the global `db`. Adding PostgreSQL means calling a different `sql.Open` and storing it — but there is only one `db` variable. You cannot run SQLite and PostgreSQL tests in the same process, cannot dependency-inject the store into handlers, and cannot test the two stores independently. + +**Why it happens:** The package was written as a single-instance tool, which was appropriate at first. Dual-DB support requires the concept of a "store" that can be swapped — which requires struct-based design. + +**Consequences:** If you try to add PostgreSQL without refactoring, you end up with `if dbType == "postgres" { ... } else { ... }` branches scattered across every handler. This is unmaintainable, untestable, and will break if a third DB is ever added. + +**Warning signs:** +- Any attempt to pass a PostgreSQL `*sql.DB` to the existing handlers requires changing the global variable, which breaks concurrent tests. +- The test file uses `UpdatesReset()` to reset global state between tests — a design smell that signals the global state problem. + +**Prevention:** +- Introduce `type Server struct { store Store; secret string }` where `Store` is an interface. +- Move all handler functions to methods on `Server`. +- `InitDB` becomes a factory: `NewSQLiteStore(path)` or `NewPostgresStore(dsn)` returning the interface. +- Tests construct a fresh `Server` with an in-memory SQLite store; no global state to reset. + +**Phase mapping:** This refactor is the prerequisite for dual-DB. Do it as the first step of the milestone, before any PostgreSQL driver work. + +--- + +### Pitfall 4: Schema migration strategy does not scale to dual-DB or multi-version upgrades + +**What goes wrong:** The current migration strategy is a single silent `ALTER TABLE` at line 87: `_, _ = db.Exec("ALTER TABLE updates ADD COLUMN acknowledged_at TEXT")`. This works for one SQLite column addition but fails in two ways when expanded: (1) PostgreSQL requires different syntax and error handling, (2) there is no version tracking, so there is no way to know which migrations have already run on an existing database. + +**Why it happens:** The approach was acceptable for a single-column addition in a personal project. It does not generalise. + +**Consequences:** +- On PostgreSQL, `ALTER TABLE ... ADD COLUMN IF NOT EXISTS` is available but the silent `_, _` error swallow pattern will hide real migration failures. +- If a second column is added in a future milestone, there is no mechanism to skip it on databases that already have it (SQLite's `IF NOT EXISTS` on `ADD COLUMN` is only available in SQLite 3.37+). +- Existing user databases upgrading from the current version need all migrations to run in order and idempotently. + +**Warning signs:** +- More than one `ALTER TABLE` in `InitDB()`. +- Any `_, _ = db.Exec(...)` where the underscore discards an error on a DDL statement. + +**Prevention:** +- Introduce a `schema_migrations` table with a single `version INTEGER` column. +- Write migrations as numbered functions: `migration001`, `migration002`, etc. +- `InitDB` reads the current version and runs only pending migrations. +- Keep migrations simple: pure SQL, no application logic. +- A lightweight library (`golang-migrate/migrate`) can handle this, but for this project's scale a 30-line hand-rolled runner is sufficient and avoids a new dependency. + +**Phase mapping:** Implement alongside the Store interface refactor. The migration runner must support both SQLite and PostgreSQL SQL dialects. + +--- + +## Moderate Pitfalls + +--- + +### Pitfall 5: PostgreSQL connection pooling behaves differently than SQLite's forced single connection + +**What goes wrong:** The SQLite configuration uses `db.SetMaxOpenConns(1)` to serialize all DB access (line 64). This was the correct choice for SQLite's single-writer model. For PostgreSQL, `MaxOpenConns(1)` is a severe bottleneck and eliminates one of the primary reasons to use PostgreSQL. However, removing the constraint also removes the `sync.Mutex`, which must be eliminated correctly — not just the `SetMaxOpenConns(1)` call. + +**Why it happens:** The mutex was added as belt-and-suspenders to the `SetMaxOpenConns(1)` constraint. For PostgreSQL, transactions handle isolation and the driver manages connection pooling correctly. The mutex is not needed and actively harmful at scale. + +**Consequences:** Keeping `SetMaxOpenConns(1)` on PostgreSQL caps throughput to sequential queries. Removing it without reviewing the mutex usage can cause incorrect locking (the mutex guards writes, but PostgreSQL transactions should guard atomicity instead). + +**Warning signs:** +- The `pgStore` implementation sets `MaxOpenConns(1)` — that is wrong. +- The `pgStore` implementation acquires a `sync.Mutex` around individual `db.Exec` calls instead of using transactions. + +**Prevention:** +- In `sqliteStore`: keep `SetMaxOpenConns(1)` and the mutex (SQLite needs it). +- In `pgStore`: use PostgreSQL's default pooling (`SetMaxOpenConns` appropriate to load, e.g. 10-25), use `db.BeginTx` for operations that require atomicity, no application-level mutex. +- Document the difference in code comments. + +**Phase mapping:** During the `pgStore` implementation phase. + +--- + +### Pitfall 6: Optimistic UI updates in `assignTag` have no rollback on failure + +**What goes wrong:** `assignTag()` in `useUpdates.ts` (lines 60-84) applies the state change optimistically before the API call. If the PUT/DELETE fails, the UI shows the new tag state but the server retained the old one. The next poll at most 5 seconds later will overwrite the optimistic state with the real server state — but during that window the user sees incorrect data. Worse, the error is only `console.error`, so the user gets no feedback that their action failed. + +**Why it happens:** Optimistic updates are a good UX pattern, but require pairing with: (a) rollback on failure, and (b) user-visible error feedback. + +**Consequences:** +- During a 5-second window after a failed tag assignment, the UI shows the wrong tag. +- If the backend is down and the user assigns multiple tags, all changes appear to succeed. The next poll resets all of them silently. + +**Warning signs:** +- No `try/catch` that restores `prev` state on `assignTag` failure. +- No error toast or inline error state for tag assignment failures. + +**Prevention:** +- Capture `prevState` before the optimistic update. +- In the `catch` block: restore `prevState` and surface an error message to the user (inline or toast). +- Example pattern: `const prev = updates[image]; setUpdates(optimistic); try { await api() } catch { setUpdates(restore(prev)); showError() }`. + +**Phase mapping:** Part of the UX improvements phase. + +--- + +### Pitfall 7: Bulk acknowledge actions hitting the backend sequentially instead of in a single operation + +**What goes wrong:** "Dismiss all" and "dismiss by group" are planned features. The naive implementation fires one `PATCH /api/updates/{image}` per image from the frontend. For a user with 30 tracked images, this is 30 sequential API calls. Each call acquires the mutex and executes a SQL UPDATE. This is fine for single-user loads but is the wrong pattern: it creates 30 round trips, 30 DB transactions, and 30 state updates in the React UI (causing 30 re-renders). + +**Why it happens:** The existing dismiss path is single-image by design; bulk is an afterthought unless an explicit bulk endpoint is designed from the start. + +**Consequences:** +- 30 re-renders in rapid succession cause visible UI flickering. +- If one request fails in the middle, some images are acknowledged and others are not, with no clear feedback to the user. + +**Warning signs:** +- A "dismiss all" button that loops over `updates` calling `acknowledge(image)` in sequence or in `Promise.all`. + +**Prevention:** +- Add a `POST /api/updates/acknowledge-bulk` endpoint that accepts an array of image names and wraps all UPDATEs in a single transaction. +- The frontend calls one endpoint and updates state once. +- For "dismiss by group": pass `tag_id` as the filter parameter so the backend does `UPDATE updates SET acknowledged_at = NOW() WHERE image IN (SELECT image FROM tag_assignments WHERE tag_id = ?)`. + +**Phase mapping:** Design the bulk endpoint before implementing the frontend bulk UI; the API contract drives the UI, not the other way around. + +--- + +### Pitfall 8: No rollback path for existing SQLite users upgrading to a version with dual-DB + +**What goes wrong:** When an existing user upgrades their Docker image to the version that includes PostgreSQL support, they continue using SQLite. If the migration runner runs new DDL migrations on their existing SQLite database (e.g., a new column added for PostgreSQL compatibility), and the migration fails silently due to the `_, _` pattern, they are left with a database in an intermediate state. On the next restart the migration runner does not know whether to retry or skip. + +**Why it happens:** No migration version tracking means "already migrated" cannot be distinguished from "never migrated." + +**Consequences:** Database schema becomes inconsistent. Queries that expect the new column fail. The user has no recourse except to delete the database (losing all data) or manually run SQL. + +**Warning signs:** +- `InitDB` has no `SELECT version FROM schema_migrations` step. +- Migration SQL errors are swallowed. + +**Prevention:** +- Implement the versioned migration runner (see Pitfall 4). +- Log migration progress visibly at startup: `INFO: running migration 002 (add_xyz_column)`. +- For the column that already exists implicitly (`acknowledged_at`), migration 001 is `ALTER TABLE updates ADD COLUMN IF NOT EXISTS acknowledged_at TEXT` with the result logged regardless of whether the column existed. + +**Phase mapping:** Part of the store interface refactor phase, before any new schema changes land. + +--- + +## Minor Pitfalls + +--- + +### Pitfall 9: Drag handle invisible by default breaks tag reorganization discoverability + +**What goes wrong:** The `GripVertical` icon in `ServiceCard.tsx` (line 96) has `opacity-0 group-hover:opacity-100`. On touch devices, on keyboard navigation, and for users who do not hover over each card, the drag-to-regroup feature is entirely invisible. Drag-and-drop is the only way to assign a tag to an image (the `assignTag` API is only called from the drag-and-drop handler). + +**Why it happens:** The design prioritized a clean visual for non-interactive browsing, but made the interactive feature undiscoverable. + +**Consequences:** Users who cannot use hover (touch devices, keyboard-only) have no way to reorganize images. As noted in CONCERNS.md, the delete button on `TagSection.tsx` has the same problem. + +**Warning signs:** +- The drag handle has `opacity-0` without a `focus-visible:opacity-100` counterpart. +- No alternative assignment mechanism exists (e.g., a dropdown on the card). + +**Prevention:** +- Make the grip handle always visible at reduced opacity (e.g., `opacity-30 group-hover:opacity-100`), or make it visible on focus. +- Add an accessible fallback: a "Move to group" dropdown on the card's context menu or `...` menu. This also gives keyboard and touch users the ability to assign tags. + +**Phase mapping:** UX improvements phase. Not a blocker for DB work but should be addressed before the milestone closes. + +--- + +### Pitfall 10: `datetime('now')` in DismissHandler produces SQLite-only timestamps + +**What goes wrong:** `DismissHandler` (line 225) writes `acknowledged_at` using `datetime('now')`, a SQLite built-in. This is a SQL dialect issue distinct from the `INSERT OR REPLACE` problem. When the PostgreSQL path is added, this query must become `NOW()` or an application-layer timestamp. + +**Why it happens:** It is a small single-line SQL call, easy to overlook during the migration to dual-DB. + +**Consequences:** `DismissHandler` breaks entirely on PostgreSQL; `datetime('now')` is not a valid PostgreSQL function call and will produce a column-name error. + +**Warning signs:** +- Any raw `datetime(` in query strings. + +**Prevention:** +- In the Store interface, the `DismissImage(image string) error` method takes no timestamp argument — the store implementation generates `NOW()` in SQL or passes `time.Now()` as a parameter from Go. Passing the timestamp from Go (`?` / `$1`) is the most portable approach: both SQLite and PostgreSQL accept a bound `time.Time` value, removing all dialect issues for timestamps. + +**Phase mapping:** Resolve during the `pgStore` implementation. Can be fixed in `sqliteStore` at the same time for consistency. + +--- + +### Pitfall 11: `AUTOINCREMENT` in SQLite schema vs PostgreSQL `SERIAL` or `GENERATED ALWAYS AS IDENTITY` + +**What goes wrong:** The `tags` table uses `INTEGER PRIMARY KEY AUTOINCREMENT` (line 90). PostgreSQL does not have `AUTOINCREMENT`; it uses `SERIAL`, `BIGSERIAL`, or `GENERATED ALWAYS AS IDENTITY`. When writing the `CREATE TABLE` DDL for PostgreSQL, this must be translated. + +**Why it happens:** A detail that is invisible in the SQLite path because `CREATE TABLE IF NOT EXISTS` never re-runs. + +**Consequences:** `CREATE TABLE` fails on PostgreSQL if the SQLite DDL is used verbatim. + +**Warning signs:** +- A single `schema.sql` file used for both databases. + +**Prevention:** +- Store DDL per-driver: `schema_sqlite.sql` and `schema_pg.sql`, or generate DDL in code with driver-specific constants. +- For PostgreSQL, use `id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY`. + +**Phase mapping:** Part of the initial `pgStore` schema setup. + +--- + +## Phase-Specific Warnings + +| Phase Topic | Likely Pitfall | Mitigation | +|---|---|---| +| Fix SQLite bugs (UPSERT + FK enforcement) | INSERT OR REPLACE deletes tag assignments (Pitfall 2) | Use `ON CONFLICT DO UPDATE`; add `PRAGMA foreign_keys = ON` | +| Store interface refactor | Global state prevents dual-DB (Pitfall 3) | Struct-based `Server` with `Store` interface before any PostgreSQL work | +| Migration runner | Silent failures leave DB in unknown state (Pitfalls 4, 8) | Versioned migrations with visible logging; never swallow DDL errors | +| PostgreSQL implementation | SQLite SQL dialect in shared queries (Pitfall 1) | All SQL in store implementations, never in handlers; integration test both stores | +| PostgreSQL connection setup | Single-connection constraint applied to Postgres (Pitfall 5) | `pgStore` uses pooling and transactions, not mutex + `MaxOpenConns(1)` | +| Timestamp writes | `datetime('now')` fails on PostgreSQL (Pitfall 10) | Pass `time.Now()` as a bound parameter from Go instead of using SQL built-ins | +| Schema creation | `AUTOINCREMENT` not valid PostgreSQL syntax (Pitfall 11) | Separate DDL per driver | +| Bulk acknowledge UI | Sequential API calls cause flickering and partial state (Pitfall 7) | Design bulk endpoint first; one API call, one state update | +| Tag UX improvements | Optimistic updates without rollback confuse users (Pitfall 6) | Always pair optimistic updates with `catch` rollback and user-visible error | +| Accessibility improvements | Drag handle invisible; keyboard users cannot reorganize (Pitfall 9) | Always-visible handle at reduced opacity + dropdown alternative | + +--- + +## Sources + +- Codebase analysis: `/pkg/diunwebhook/diunwebhook.go`, lines 48-117, 225, 352 (HIGH confidence — direct code evidence) +- Codebase analysis: `/frontend/src/hooks/useUpdates.ts`, lines 60-84 (HIGH confidence — direct code evidence) +- Codebase analysis: `/frontend/src/components/ServiceCard.tsx`, line 96 (HIGH confidence — direct code evidence) +- `.planning/codebase/CONCERNS.md` — confirmed INSERT OR REPLACE and FK enforcement issues (HIGH confidence — prior audit) +- Go `database/sql` package contract and SQLite vs PostgreSQL dialect differences (MEDIUM confidence — training knowledge, no external verification available; recommend verifying PostgreSQL placeholder syntax `$1` format before implementation) diff --git a/.planning/research/STACK.md b/.planning/research/STACK.md new file mode 100644 index 0000000..337fb6c --- /dev/null +++ b/.planning/research/STACK.md @@ -0,0 +1,185 @@ +# Technology Stack + +**Project:** DiunDashboard — PostgreSQL milestone +**Researched:** 2026-03-23 +**Scope:** Adding PostgreSQL support alongside existing SQLite to a Go 1.26 backend + +--- + +## Recommended Stack + +### PostgreSQL Driver + +| Technology | Version | Purpose | Why | +|------------|---------|---------|-----| +| `github.com/jackc/pgx/v5/stdlib` | v5.9.1 | PostgreSQL `database/sql` driver | The de-facto standard Go PostgreSQL driver. Pure Go. 7,328+ importers. The `stdlib` adapter makes it a drop-in for the existing `*sql.DB` code path. Native pgx interface not needed — this project uses `database/sql` already and has no PostgreSQL-specific features (no LISTEN/NOTIFY, no COPY). | + +**Confidence:** HIGH — Verified via pkg.go.dev (v5.9.1, published 2026-03-22). pgx v5 is the clear community standard; lib/pq is officially in maintenance-only mode. + +**Do NOT use:** +- `github.com/lib/pq` — maintenance-only since 2021; pgx is the successor recommended by the postgres ecosystem. +- Native pgx interface (`pgx.Connect`, `pgxpool.New`) — overkill here; this project only needs standard queries and the existing `*sql.DB` pattern should be preserved for consistency. + +### Database Migration Tool + +| Technology | Version | Purpose | Why | +|------------|---------|---------|-----| +| `github.com/golang-migrate/migrate/v4` | v4.19.1 | Schema migrations for both SQLite and PostgreSQL | Supports both `database/sqlite` (uses `modernc.org/sqlite` — pure Go, no CGO) and `database/pgx/v5` (uses pgx v5). Both drivers are maintained. The existing inline `CREATE TABLE IF NOT EXISTS` + silent `ALTER TABLE` approach does not scale to dual-database support; a proper migration tool is required. | + +**Confidence:** HIGH — Verified via pkg.go.dev. The `database/sqlite` sub-package explicitly uses `modernc.org/sqlite` (pure Go), matching the project's no-CGO constraint. The `database/pgx/v5` sub-package uses pgx v5. + +**Drivers to import:** + +```go +// For SQLite migrations (pure Go, no CGO — matches existing constraint) +_ "github.com/golang-migrate/migrate/v4/database/sqlite" + +// For PostgreSQL migrations (via pgx v5) +_ "github.com/golang-migrate/migrate/v4/database/pgx/v5" + +// Migration source (embedded files) +_ "github.com/golang-migrate/migrate/v4/source/iofs" +``` + +**Do NOT use:** +- `pressly/goose` — Its SQLite dialect documentation does not confirm pure-Go driver support; CGO status is ambiguous. golang-migrate explicitly documents use of `modernc.org/sqlite`. Goose is a fine tool but the CGO uncertainty is a disqualifier for this project. +- `database/sqlite3` variant of golang-migrate — Uses `mattn/go-sqlite3` which requires CGO. Use `database/sqlite` (no `3`) instead. + +### SQLite Driver (Existing — Retain) + +| Technology | Version | Purpose | Why | +|------------|---------|---------|-----| +| `modernc.org/sqlite` | v1.47.0 | Pure-Go SQLite driver | Already in use; must be retained for no-CGO cross-compilation. Current version in go.mod is v1.46.1 — upgrade to v1.47.0 (released 2026-03-17) for latest SQLite 3.51.3 and bug fixes. | + +**Confidence:** HIGH — Verified via pkg.go.dev versions tab. + +--- + +## SQL Dialect Abstraction + +### The Problem + +The existing codebase has four SQLite-specific SQL constructs that break on PostgreSQL: + +| Location | SQLite syntax | PostgreSQL equivalent | +|----------|--------------|----------------------| +| `InitDB` — tags table | `INTEGER PRIMARY KEY AUTOINCREMENT` | `INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY` | +| `UpdateEvent` | `INSERT OR REPLACE INTO updates VALUES (?,...)` | `INSERT INTO updates (...) ON CONFLICT (image) DO UPDATE SET ...` | +| `DismissHandler` | `UPDATE ... SET acknowledged_at = datetime('now')` | `UPDATE ... SET acknowledged_at = NOW()` | +| `TagAssignmentHandler` | `INSERT OR REPLACE INTO tag_assignments` | `INSERT INTO tag_assignments ... ON CONFLICT (image) DO UPDATE SET tag_id = ...` | +| All handlers | `?` positional placeholders | `$1, $2, ...` positional placeholders | + +### Recommended Pattern: Storage Interface + +Extract a `Store` interface in `pkg/diunwebhook/`. Implement it twice: once for SQLite (`sqliteStore`), once for PostgreSQL (`postgresStore`). Both implementations use `database/sql` and raw SQL, but with dialect-appropriate queries. + +```go +// pkg/diunwebhook/store.go +type Store interface { + InitSchema() error + UpdateEvent(event DiunEvent) error + GetUpdates() (map[string]UpdateEntry, error) + DismissUpdate(image string) error + GetTags() ([]Tag, error) + CreateTag(name string) (Tag, error) + DeleteTag(id int) error + AssignTag(image string, tagID int) error + UnassignTag(image string) error +} +``` + +This is a standard Go pattern: define a narrow interface, swap implementations via factory function. The `sync.Mutex` moves into each store implementation (SQLite store keeps `SetMaxOpenConns(1)` + mutex; PostgreSQL store can use a connection pool without a global mutex). + +**Do NOT use:** +- ORM (GORM, ent, sqlc, etc.) — The query set is small and known. An ORM adds a dependency with its own dialect quirks and opaque query generation. Raw SQL with an interface is simpler, easier to test, and matches the existing project style. +- `database/sql` query builder libraries (squirrel, etc.) — Same reasoning; the schema is simple enough that explicit SQL per dialect is more readable and maintainable. + +--- + +## Configuration + +### New Environment Variable + +| Variable | Purpose | Default | +|----------|---------|---------| +| `DATABASE_URL` | PostgreSQL connection string (triggers PostgreSQL mode when set) | — (unset = SQLite mode) | +| `DB_PATH` | SQLite file path (existing) | `./diun.db` | + +**Selection logic:** If `DATABASE_URL` is set, use PostgreSQL. Otherwise, use SQLite with `DB_PATH`. This is the simplest signal — no new `DB_DRIVER` variable needed. + +**PostgreSQL connection string format:** +``` +postgres://user:password@host:5432/dbname?sslmode=disable +``` + +--- + +## Migration File Structure + +``` +migrations/ + 001_initial_schema.up.sql + 001_initial_schema.down.sql + 002_add_acknowledged_at.up.sql + 002_add_acknowledged_at.down.sql +``` + +Each migration file should be valid for **both** SQLite and PostgreSQL — this is achievable for the current schema since: +- `AUTOINCREMENT` can become `INTEGER PRIMARY KEY` (SQLite auto-assigns rowid regardless of keyword; PostgreSQL uses `SERIAL` — requires separate dialect files or a compatibility shim). + +**Revised recommendation:** Use **separate migration directories per dialect** when DDL diverges significantly: + +``` +migrations/ + sqlite/ + 001_initial_schema.up.sql + 002_add_acknowledged_at.up.sql + postgres/ + 001_initial_schema.up.sql + 002_add_acknowledged_at.up.sql +``` + +This is more explicit than trying to share SQL across dialects. golang-migrate supports `iofs` (Go embed) as a source, so both directories can be embedded in the binary. + +--- + +## Full Dependency Changes + +```bash +# Add PostgreSQL driver (via pgx v5 stdlib adapter) +go get github.com/jackc/pgx/v5@v5.9.1 + +# Add migration tool with SQLite (pure Go) and pgx/v5 drivers +go get github.com/golang-migrate/migrate/v4@v4.19.1 + +# Upgrade existing SQLite driver to current version +go get modernc.org/sqlite@v1.47.0 +``` + +No other new dependencies are required. The existing `database/sql` usage throughout the codebase is preserved. + +--- + +## Alternatives Considered + +| Category | Recommended | Alternative | Why Not | +|----------|-------------|-------------|---------| +| PostgreSQL driver | pgx/v5 stdlib | lib/pq | lib/pq is maintenance-only since 2021; pgx is the successor | +| PostgreSQL driver | pgx/v5 stdlib | Native pgx interface | Project uses database/sql; stdlib adapter preserves consistency; no need for PostgreSQL-specific features | +| Migration tool | golang-migrate | pressly/goose | Goose's SQLite CGO status unconfirmed; golang-migrate explicitly uses modernc.org/sqlite | +| Migration tool | golang-migrate | Inline `CREATE TABLE IF NOT EXISTS` | Inline approach cannot handle dual-dialect schema differences or ordered version history | +| Abstraction | Store interface | GORM / ent | Schema is 3 tables; ORM adds complexity without benefit; project already uses raw SQL | +| Abstraction | Store interface | sqlc | Code generation adds a build step and CI dependency; not warranted for this scope | +| Placeholder style | Per-dialect (`?` vs `$1`) | `sqlx` named params | Named params add a new library; explicit per-dialect SQL is clearer and matches project style | + +--- + +## Sources + +- pgx v5.9.1: https://pkg.go.dev/github.com/jackc/pgx/v5@v5.9.1 — HIGH confidence +- pgxpool: https://pkg.go.dev/github.com/jackc/pgx/v5/pgxpool — HIGH confidence +- golang-migrate v4.19.1 sqlite driver (pure Go): https://pkg.go.dev/github.com/golang-migrate/migrate/v4/database/sqlite — HIGH confidence +- golang-migrate v4 pgx/v5 driver: https://pkg.go.dev/github.com/golang-migrate/migrate/v4/database/pgx/v5 — HIGH confidence +- golang-migrate v4 sqlite3 driver (CGO — avoid): https://pkg.go.dev/github.com/golang-migrate/migrate/v4/database/sqlite3 — HIGH confidence +- modernc.org/sqlite v1.47.0: https://pkg.go.dev/modernc.org/sqlite?tab=versions — HIGH confidence +- goose v3.27.0: https://pkg.go.dev/github.com/pressly/goose/v3 — MEDIUM confidence (SQLite CGO status not confirmed in official docs) diff --git a/.planning/research/SUMMARY.md b/.planning/research/SUMMARY.md new file mode 100644 index 0000000..b6953db --- /dev/null +++ b/.planning/research/SUMMARY.md @@ -0,0 +1,181 @@ +# Project Research Summary + +**Project:** DiunDashboard — PostgreSQL milestone + UX improvements +**Domain:** Self-hosted container image update monitoring dashboard (Go backend + React SPA) +**Researched:** 2026-03-23 +**Confidence:** HIGH (stack and architecture sourced from direct codebase analysis and verified package versions; features MEDIUM due to tool restrictions during research) + +## Executive Summary + +DiunDashboard is a self-hosted Go + React dashboard that receives DIUN webhook events and presents a persistent, acknowledgeable list of container images with available updates. The current milestone covers two parallel tracks: (1) fixing active data-correctness bugs and adding PostgreSQL as an alternative to SQLite, and (2) delivering UX improvements users need before the tool is genuinely usable at scale (bulk dismiss, search/filter, new-update indicators). Both tracks have well-understood solutions rooted in established Go patterns — the engineering risk is low provided the work is sequenced correctly. + +The recommended approach is a strict dependency-first build order. The SQLite data-integrity bugs (`INSERT OR REPLACE` silently deleting tag assignments, missing FK pragma) must be fixed before any other work because they undermine trust in the tool and will complicate the subsequent refactor if left in. The backend refactor — introducing a `Store` interface and a `Server` struct to replace package-level globals — is the foundational prerequisite for PostgreSQL support, parallel test execution, and reliable UX features. PostgreSQL is then a clean additive step: implement `PostgresStore`, wire the `DATABASE_URL` env var into `main.go`, and provide dialect-appropriate SQL in the new store file. + +The primary risk is dialect leakage: SQLite-specific SQL (`datetime('now')`, `INSERT OR REPLACE`, `?` placeholders, `AUTOINCREMENT`, `PRAGMA`) scattered across handler functions will silently break on PostgreSQL if the Store interface abstraction is not in place before any PostgreSQL code is written. Secondary risks are a missing versioned migration runner (which leaves existing user databases in an unknown state on upgrade) and bulk dismiss implemented as N sequential API calls rather than a single transactional endpoint. Both risks have well-documented mitigations and are easy to prevent if addressed in the correct phase. + +--- + +## Key Findings + +### Recommended Stack + +The existing stack is largely correct and requires minimal additions. The PostgreSQL driver is `github.com/jackc/pgx/v5/stdlib` (v5.9.1, verified 2026-03-22) — the de-facto community standard. Its `stdlib` adapter makes it a drop-in for the existing `*sql.DB` code path; the native pgx interface is not needed. `lib/pq` is explicitly maintenance-only and must not be used. For schema migrations, `github.com/golang-migrate/migrate/v4` (v4.19.1) supports both the project's `modernc.org/sqlite` (pure-Go, no CGO) and pgx/v5 backends via separately maintained sub-packages. The existing SQLite driver should be upgraded from v1.46.1 to v1.47.0. + +**Core technologies:** +- `github.com/jackc/pgx/v5/stdlib` v5.9.1: PostgreSQL driver — only viable current option; `lib/pq` is maintenance-only +- `github.com/golang-migrate/migrate/v4` v4.19.1: schema migrations — explicit `modernc.org/sqlite` support satisfies no-CGO constraint +- `modernc.org/sqlite` v1.47.0: existing SQLite driver (upgrade from v1.46.1) — must remain for pure-Go cross-compilation + +No ORM. No query-builder library. The query set is 8 operations across 3 tables; raw SQL per store implementation is simpler, easier to audit, and matches the existing project style. + +**Configuration:** `DATABASE_URL` env var (when set, activates PostgreSQL mode). `DB_PATH` retained for SQLite. No separate `DB_DRIVER` variable needed. + +### Expected Features + +The feature research produced a clear priority stack grounded in the documented concerns and self-hosted dashboard conventions. Data integrity is a prerequisite for everything else — broken data collapses user trust faster than any missing feature. + +**Must have (table stakes):** +- SQLite data integrity fix (UPSERT + FK pragma) — existing bug silently deletes tag assignments on every DIUN event +- Bulk acknowledge: dismiss all + dismiss by group — O(n) clicking for 20+ images causes abandonment +- Search + filter by image name, status, and tag — standard affordance for any list exceeding 10 items +- New-update indicator (badge/counter) and page/tab title count — persistent visibility is the core value proposition +- PostgreSQL support — required for users running Coolify or other Postgres-backed infrastructure + +**Should have (differentiators):** +- Toast notification on new update arrival during polling — shares implementation with new-update indicator +- Sort order controls (newest first, by name, by registry) — pure frontend, no backend change +- Light/dark theme toggle — low complexity, removes a known complaint +- Drag handle always visible (accessibility) — currently hover-only, invisible on touch/keyboard +- Optimistic UI rollback on tag assignment failure — current code has no error recovery path + +**Defer (v2+):** +- Data retention / auto-cleanup of acknowledged entries — real concern but not urgent for most users +- Alternative tag assignment dropdown — drag-and-drop exists; dropdown is an accessibility improvement, not a blocker +- Browser notification API — high UX risk, low reward vs. badge approach +- Auto-grouping by Docker stack — requires Docker socket access; different scope entirely + +### Architecture Approach + +The architecture follows a standard Go repository interface pattern. The current monolith (`diunwebhook.go` with package-level `var db *sql.DB` and `var mu sync.Mutex`) is extracted into a `Store` interface implemented by two concrete types (`SQLiteStore`, `PostgresStore`), with HTTP handlers moved to methods on a `Server` struct that holds a `Store`. This pattern eliminates global state, enables parallel tests without resets, and enforces a strict boundary: handlers never see SQL, store implementations never see HTTP. + +**Major components:** +1. `Store` interface (`store.go`) — contract for all persistence; 11 methods covering updates, tags, and assignments +2. `SQLiteStore` (`sqlite.go`) — SQLite-specific SQL, `sync.Mutex`, `SetMaxOpenConns(1)`, `PRAGMA foreign_keys = ON` +3. `PostgresStore` (`postgres.go`) — PostgreSQL-specific SQL, pgx connection pool, no mutex, `db.BeginTx` for atomicity +4. `Server` struct (`server.go`) — holds `Store` and `secret`; all HTTP handlers are methods on `Server` +5. `models.go` — shared `DiunEvent`, `UpdateEntry`, `Tag` structs with no imports beyond stdlib +6. `main.go` — sole location where backend is chosen (`DATABASE_URL` present → PostgreSQL, absent → SQLite) +7. Frontend SPA — unchanged API contract; communicates with backend via `/api/*` only + +**Key pattern: `SQLiteStore` retains `sync.Mutex`; `PostgresStore` does not.** These are structurally different and must not share a mutex. + +**Migration strategy:** Separate DDL per dialect (`migrations/sqlite/` and `migrations/postgres/`). Both embedded in the binary via `//go:embed`. A versioned `schema_migrations` table prevents re-running migrations on existing databases and makes upgrade failures visible. + +### Critical Pitfalls + +1. **SQLite-specific SQL leaking into shared code** — `datetime('now')`, `INSERT OR REPLACE`, `?` placeholders, `AUTOINCREMENT`, and `PRAGMA` all fail on PostgreSQL. Prevention: Store interface forces all SQL into store files; handlers call named methods only; integration tests run both stores. + +2. **`INSERT OR REPLACE` silently deleting tag assignments** — SQLite implements this as DELETE + INSERT, which cascades to `tag_assignments` and erases the user's groupings on every DIUN event. Prevention: replace with `INSERT ... ON CONFLICT(image) DO UPDATE SET ...`; add `PRAGMA foreign_keys = ON`; add regression test asserting tag survives a second `UpdateEvent` call. + +3. **Global package-level state blocks dual-DB without struct refactor** — `var db *sql.DB` at package scope means there is only one DB handle; PostgreSQL cannot be added without introducing `if dbType == "postgres"` branches across every handler. Prevention: `Server` struct with injected `Store` must precede all PostgreSQL work. + +4. **No versioned migration runner** — silent `ALTER TABLE` with discarded errors leaves existing SQLite databases in an unknown state on upgrade. Prevention: `schema_migrations` version table; log every migration attempt; never swallow DDL errors. + +5. **Bulk dismiss implemented as N sequential API calls** — 30 acknowledged images = 30 round trips, 30 mutex acquisitions, 30 React re-renders with potential flickering and partial-state failure. Prevention: design `POST /api/updates/acknowledge-bulk` endpoint first; one call, one transaction, one state update. + +--- + +## Implications for Roadmap + +Based on the dependency graph from feature and architecture research, the milestone decomposes into four phases. The ordering is non-negotiable: each phase is a prerequisite for the next. + +### Phase 1: Data Integrity Fixes +**Rationale:** The `INSERT OR REPLACE` bug is active in production and deletes user data on every DIUN event. Fixing it before the refactor means the bug-fix tests become the regression suite that validates the refactor did not regress behavior. No other work is credible until the data layer is correct. +**Delivers:** Trustworthy persistence — tag assignments survive new DIUN events; FK enforcement works; acknowledged state is preserved correctly. +**Addresses:** Table-stakes feature "Data integrity across restarts"; Pitfalls 2, 10 (timestamp fix can be included here). +**Avoids:** Shipping the bug in both DB paths; losing the fix in refactor noise. +**Research flag:** None needed — the fix is a 3-line SQL change with a clear regression test. Standard patterns apply. + +### Phase 2: Backend Refactor — Store Interface + Server Struct +**Rationale:** The global state architecture makes PostgreSQL support structurally impossible without this refactor. All subsequent work (PostgreSQL implementation, parallel test execution, safer UX features) depends on this change. The refactor must be behavior-neutral — all existing tests pass before PostgreSQL is introduced. +**Delivers:** `Store` interface, `SQLiteStore` implementation, `Server` struct with constructor injection, models in `models.go`. Zero behavior change for existing SQLite users. +**Uses:** Existing `modernc.org/sqlite`; `database/sql` standard library; no new dependencies. +**Implements:** Core architecture pattern from ARCHITECTURE.md; eliminates Pitfall 3 (global state) and Pitfall 4 (migration runner) in one phase. +**Avoids:** Introducing PostgreSQL and refactoring simultaneously (would make failures ambiguous). +**Research flag:** None needed — this is a standard Go repository interface pattern with well-documented prior art. + +### Phase 3: PostgreSQL Support +**Rationale:** With the `Store` interface in place, adding PostgreSQL is additive: write `PostgresStore`, add `pgx/v5/stdlib` as a dependency, add `DATABASE_URL` to `main.go` and Docker Compose. The interface boundary guarantees no SQLite-specific SQL can appear in handlers. +**Delivers:** `PostgresStore` implementing all `Store` methods with PostgreSQL dialect SQL; `DATABASE_URL` env var wired through `main.go`; separate dialect migration files; updated `compose.dev.yml` with optional `postgres` profile; documentation. +**Uses:** `github.com/jackc/pgx/v5/stdlib` v5.9.1; `github.com/golang-migrate/migrate/v4` v4.19.1; separate `migrations/sqlite/` and `migrations/postgres/` directories. +**Avoids:** Pitfalls 1, 5, 8, 10, 11 — all are mitigated by the Store interface + per-dialect SQL + connection pool (no mutex) in `PostgresStore`. +**Research flag:** Verify exact import path for `pgx/v5/stdlib` during implementation. The `database/sql` compatibility layer is standard but the import string should be confirmed against pkg.go.dev before coding. + +### Phase 4: UX Improvements +**Rationale:** These features are independent of the DB work but grouped together because they share the frontend codebase and several features share implementation logic (new-update indicator and toast notification use the same poll-comparison logic; bulk dismiss all and bulk dismiss by group share the same API endpoint design). Deferring UX until after the backend is correct means UX tests run against a trustworthy data layer. +**Delivers:** Bulk acknowledge (all + by group) with a single backend endpoint (`POST /api/updates/acknowledge-bulk`); search and filter by name/status/tag (frontend-only); new-update badge/counter and page title count; light/dark theme toggle; drag handle always-visible fix; optimistic UI rollback with user-visible error on tag assignment failure. +**Uses:** Existing React 19 + Tailwind + shadcn/ui stack; no new frontend dependencies expected. +**Avoids:** Pitfalls 6, 7, 9 — optimistic rollback, bulk endpoint, accessible drag handle. +**Research flag:** None needed for search/filter/theme/accessibility (standard patterns). The bulk acknowledge endpoint needs clear API contract design before frontend implementation begins — define the request/response shape first. + +### Phase Ordering Rationale + +- **Phase 1 before Phase 2:** Bug fix tests become the regression suite; the refactor cannot accidentally regress behavior it has not validated first. +- **Phase 2 before Phase 3:** The Store interface is a structural prerequisite. PostgreSQL added to the monolith produces unmaintainable dialect branches. +- **Phase 3 before Phase 4 (or parallel after Phase 2):** UX features are mostly frontend and do not depend on PostgreSQL. However, the bulk acknowledge endpoint (`AcknowledgeAll`, `AcknowledgeByTag`) must be in the `Store` interface, which is finalized in Phase 2. Phase 4 frontend work can start once Phase 2 is merged; Phase 3 and Phase 4 can proceed in parallel. +- **Never:** Mix refactor and new feature in the same commit. Each phase should be independently reviewable and revertable. + +### Research Flags + +Phases likely needing deeper research during planning: +- **Phase 3 (PostgreSQL):** Verify `pgx/v5/stdlib` import path (`github.com/jackc/pgx/v5/stdlib`) against pkg.go.dev before adding the dependency. Confirm `golang-migrate` `database/sqlite` sub-package still uses `modernc.org/sqlite` (not `mattn/go-sqlite3`) in v4.19.1 — this was verified but should be re-confirmed at time of implementation. + +Phases with standard patterns (skip research-phase): +- **Phase 1 (Bug fixes):** 3-line SQL change with a clear regression test; no research needed. +- **Phase 2 (Refactor):** Standard Go repository interface pattern; no research needed. +- **Phase 4 (UX):** All features use existing stack (React, Tailwind, shadcn/ui); no new technologies introduced. + +--- + +## Confidence Assessment + +| Area | Confidence | Notes | +|------|------------|-------| +| Stack | HIGH | All versions verified via pkg.go.dev at time of research (2026-03-23); pgx v5.9.1 published 2026-03-22; golang-migrate v4.19.1 confirms `modernc.org/sqlite` | +| Features | MEDIUM | Feature priorities derived from direct codebase audit (CONCERNS.md, PROJECT.md) — HIGH confidence; competitive landscape analysis (Portainer, Uptime Kuma patterns) from training data only — MEDIUM | +| Architecture | HIGH | Based on direct analysis of `pkg/diunwebhook/diunwebhook.go` and `cmd/diunwebhook/main.go`; Store interface pattern is a well-established Go idiom with no ambiguity | +| Pitfalls | HIGH (backend) / MEDIUM (frontend) | Backend pitfalls sourced from direct code evidence (line numbers cited); PostgreSQL dialect differences from training knowledge — recommend verifying `$1` placeholder syntax before implementation; frontend pitfalls sourced from direct code analysis | + +**Overall confidence:** HIGH + +### Gaps to Address + +- **PostgreSQL `$1` placeholder syntax:** PITFALLS.md flags this as MEDIUM confidence from training knowledge. Verify against the `pgx/v5/stdlib` documentation before writing any PostgreSQL query strings. +- **`golang-migrate` CGO status at v4.19.1:** Confirmed at research time that `database/sqlite` sub-package uses `modernc.org/sqlite`; re-confirm at implementation time that this has not changed in a patch release. +- **Competitive feature validation:** The UX feature priorities are based on self-hosted dashboard patterns (Portainer, Uptime Kuma) from training data. If the roadmapper wants higher confidence on feature ordering, a quick review of current Portainer CE and Uptime Kuma changelogs would validate the bulk-dismiss and search/filter priorities. +- **`golang-migrate` vs hand-rolled migration runner:** PITFALLS.md notes a 30-line hand-rolled runner is sufficient for this project's scale. STACK.md recommends `golang-migrate`. Either is valid — the roadmap phase should make a decision and commit to one approach before implementation begins to avoid scope creep. + +--- + +## Sources + +### Primary (HIGH confidence) +- `pkg/diunwebhook/diunwebhook.go` (direct codebase analysis, lines 48-117, 225, 352) — dialect issues, global state, INSERT OR REPLACE bug +- `cmd/diunwebhook/main.go` (direct codebase analysis) — entry point, env vars, mux wiring +- `.planning/codebase/CONCERNS.md` (prior audit) — confirmed FK enforcement gap, drag handle, bulk ops missing +- `.planning/PROJECT.md` (requirements source) — confirmed dual-DB requirement, no-CGO constraint, backward compatibility +- https://pkg.go.dev/github.com/jackc/pgx/v5@v5.9.1 — pgx v5 driver, version verified +- https://pkg.go.dev/github.com/golang-migrate/migrate/v4/database/sqlite — pure-Go SQLite sub-package confirmed +- https://pkg.go.dev/github.com/golang-migrate/migrate/v4/database/pgx/v5 — pgx/v5 migration sub-package confirmed +- https://pkg.go.dev/modernc.org/sqlite?tab=versions — v1.47.0 version verified + +### Secondary (MEDIUM confidence) +- Training-data knowledge of `pgx/v5/stdlib` `database/sql` adapter pattern — standard approach, verify import path at implementation +- Training-data knowledge of Portainer CE, Uptime Kuma, Dockcheck-web UX patterns — feature prioritization for self-hosted dashboards + +### Tertiary (LOW confidence) +- None + +--- +*Research completed: 2026-03-23* +*Ready for roadmap: yes*