DiunDashboard/.planning/research/SUMMARY.md

Project Research Summary

Project: DiunDashboard — PostgreSQL milestone + UX improvements Domain: Self-hosted container image update monitoring dashboard (Go backend + React SPA) Researched: 2026-03-23 Confidence: HIGH (stack and architecture sourced from direct codebase analysis and verified package versions; features MEDIUM due to tool restrictions during research)

Executive Summary

DiunDashboard is a self-hosted Go + React dashboard that receives DIUN webhook events and presents a persistent, acknowledgeable list of container images with available updates. The current milestone covers two parallel tracks: (1) fixing active data-correctness bugs and adding PostgreSQL as an alternative to SQLite, and (2) delivering UX improvements users need before the tool is genuinely usable at scale (bulk dismiss, search/filter, new-update indicators). Both tracks have well-understood solutions rooted in established Go patterns — the engineering risk is low provided the work is sequenced correctly.

The recommended approach is a strict dependency-first build order. The SQLite data-integrity bugs (INSERT OR REPLACE silently deleting tag assignments, missing FK pragma) must be fixed before any other work because they undermine trust in the tool and will complicate the subsequent refactor if left in. The backend refactor — introducing a Store interface and a Server struct to replace package-level globals — is the foundational prerequisite for PostgreSQL support, parallel test execution, and reliable UX features. PostgreSQL is then a clean additive step: implement PostgresStore, wire the DATABASE_URL env var into main.go, and provide dialect-appropriate SQL in the new store file.

The primary risk is dialect leakage: SQLite-specific SQL (datetime('now'), INSERT OR REPLACE, ? placeholders, AUTOINCREMENT, PRAGMA) scattered across handler functions will silently break on PostgreSQL if the Store interface abstraction is not in place before any PostgreSQL code is written. Secondary risks are a missing versioned migration runner (which leaves existing user databases in an unknown state on upgrade) and bulk dismiss implemented as N sequential API calls rather than a single transactional endpoint. Both risks have well-documented mitigations and are easy to prevent if addressed in the correct phase.

---

Key Findings

Recommended Stack

The existing stack is largely correct and requires minimal additions. The PostgreSQL driver is github.com/jackc/pgx/v5/stdlib (v5.9.1, verified 2026-03-22) — the de-facto community standard. Its stdlib adapter makes it a drop-in for the existing *sql.DB code path; the native pgx interface is not needed. lib/pq is explicitly maintenance-only and must not be used. For schema migrations, github.com/golang-migrate/migrate/v4 (v4.19.1) supports both the project's modernc.org/sqlite (pure-Go, no CGO) and pgx/v5 backends via separately maintained sub-packages. The existing SQLite driver should be upgraded from v1.46.1 to v1.47.0.

Core technologies:

• github.com/jackc/pgx/v5/stdlib v5.9.1: PostgreSQL driver — only viable current option; lib/pq is maintenance-only
• github.com/golang-migrate/migrate/v4 v4.19.1: schema migrations — explicit modernc.org/sqlite support satisfies no-CGO constraint
• modernc.org/sqlite v1.47.0: existing SQLite driver (upgrade from v1.46.1) — must remain for pure-Go cross-compilation

No ORM. No query-builder library. The query set is 8 operations across 3 tables; raw SQL per store implementation is simpler, easier to audit, and matches the existing project style.

Configuration: DATABASE_URL env var (when set, activates PostgreSQL mode). DB_PATH retained for SQLite. No separate DB_DRIVER variable needed.

Expected Features

The feature research produced a clear priority stack grounded in the documented concerns and self-hosted dashboard conventions. Data integrity is a prerequisite for everything else — broken data collapses user trust faster than any missing feature.

Must have (table stakes):

• SQLite data integrity fix (UPSERT + FK pragma) — existing bug silently deletes tag assignments on every DIUN event
• Bulk acknowledge: dismiss all + dismiss by group — O(n) clicking for 20+ images causes abandonment
• Search + filter by image name, status, and tag — standard affordance for any list exceeding 10 items
• New-update indicator (badge/counter) and page/tab title count — persistent visibility is the core value proposition
• PostgreSQL support — required for users running Coolify or other Postgres-backed infrastructure

Should have (differentiators):

• Toast notification on new update arrival during polling — shares implementation with new-update indicator
• Sort order controls (newest first, by name, by registry) — pure frontend, no backend change
• Light/dark theme toggle — low complexity, removes a known complaint
• Drag handle always visible (accessibility) — currently hover-only, invisible on touch/keyboard
• Optimistic UI rollback on tag assignment failure — current code has no error recovery path

Defer (v2+):

• Data retention / auto-cleanup of acknowledged entries — real concern but not urgent for most users
• Alternative tag assignment dropdown — drag-and-drop exists; dropdown is an accessibility improvement, not a blocker
• Browser notification API — high UX risk, low reward vs. badge approach
• Auto-grouping by Docker stack — requires Docker socket access; different scope entirely

Architecture Approach

The architecture follows a standard Go repository interface pattern. The current monolith (diunwebhook.go with package-level var db *sql.DB and var mu sync.Mutex) is extracted into a Store interface implemented by two concrete types (SQLiteStore, PostgresStore), with HTTP handlers moved to methods on a Server struct that holds a Store. This pattern eliminates global state, enables parallel tests without resets, and enforces a strict boundary: handlers never see SQL, store implementations never see HTTP.

Major components:

1. Store interface (store.go) — contract for all persistence; 11 methods covering updates, tags, and assignments
2. SQLiteStore (sqlite.go) — SQLite-specific SQL, sync.Mutex, SetMaxOpenConns(1), PRAGMA foreign_keys = ON
3. PostgresStore (postgres.go) — PostgreSQL-specific SQL, pgx connection pool, no mutex, db.BeginTx for atomicity
4. Server struct (server.go) — holds Store and secret; all HTTP handlers are methods on Server
5. models.go — shared DiunEvent, UpdateEntry, Tag structs with no imports beyond stdlib
6. main.go — sole location where backend is chosen (DATABASE_URL present → PostgreSQL, absent → SQLite)
7. Frontend SPA — unchanged API contract; communicates with backend via /api/* only

Key pattern: SQLiteStore retains sync.Mutex; PostgresStore does not. These are structurally different and must not share a mutex.

Migration strategy: Separate DDL per dialect (migrations/sqlite/ and migrations/postgres/). Both embedded in the binary via //go:embed. A versioned schema_migrations table prevents re-running migrations on existing databases and makes upgrade failures visible.

Critical Pitfalls

1. SQLite-specific SQL leaking into shared code — datetime('now'), INSERT OR REPLACE, ? placeholders, AUTOINCREMENT, and PRAGMA all fail on PostgreSQL. Prevention: Store interface forces all SQL into store files; handlers call named methods only; integration tests run both stores.

2. INSERT OR REPLACE silently deleting tag assignments — SQLite implements this as DELETE + INSERT, which cascades to tag_assignments and erases the user's groupings on every DIUN event. Prevention: replace with INSERT ... ON CONFLICT(image) DO UPDATE SET ...; add PRAGMA foreign_keys = ON; add regression test asserting tag survives a second UpdateEvent call.

3. Global package-level state blocks dual-DB without struct refactor — var db *sql.DB at package scope means there is only one DB handle; PostgreSQL cannot be added without introducing if dbType == "postgres" branches across every handler. Prevention: Server struct with injected Store must precede all PostgreSQL work.

4. No versioned migration runner — silent ALTER TABLE with discarded errors leaves existing SQLite databases in an unknown state on upgrade. Prevention: schema_migrations version table; log every migration attempt; never swallow DDL errors.

5. Bulk dismiss implemented as N sequential API calls — 30 acknowledged images = 30 round trips, 30 mutex acquisitions, 30 React re-renders with potential flickering and partial-state failure. Prevention: design POST /api/updates/acknowledge-bulk endpoint first; one call, one transaction, one state update.

---

Implications for Roadmap

Based on the dependency graph from feature and architecture research, the milestone decomposes into four phases. The ordering is non-negotiable: each phase is a prerequisite for the next.

Phase 1: Data Integrity Fixes

Rationale: The INSERT OR REPLACE bug is active in production and deletes user data on every DIUN event. Fixing it before the refactor means the bug-fix tests become the regression suite that validates the refactor did not regress behavior. No other work is credible until the data layer is correct. Delivers: Trustworthy persistence — tag assignments survive new DIUN events; FK enforcement works; acknowledged state is preserved correctly. Addresses: Table-stakes feature "Data integrity across restarts"; Pitfalls 2, 10 (timestamp fix can be included here). Avoids: Shipping the bug in both DB paths; losing the fix in refactor noise. Research flag: None needed — the fix is a 3-line SQL change with a clear regression test. Standard patterns apply.

Phase 2: Backend Refactor — Store Interface + Server Struct

Rationale: The global state architecture makes PostgreSQL support structurally impossible without this refactor. All subsequent work (PostgreSQL implementation, parallel test execution, safer UX features) depends on this change. The refactor must be behavior-neutral — all existing tests pass before PostgreSQL is introduced. Delivers: Store interface, SQLiteStore implementation, Server struct with constructor injection, models in models.go. Zero behavior change for existing SQLite users. Uses: Existing modernc.org/sqlite; database/sql standard library; no new dependencies. Implements: Core architecture pattern from ARCHITECTURE.md; eliminates Pitfall 3 (global state) and Pitfall 4 (migration runner) in one phase. Avoids: Introducing PostgreSQL and refactoring simultaneously (would make failures ambiguous). Research flag: None needed — this is a standard Go repository interface pattern with well-documented prior art.

Phase 3: PostgreSQL Support

Rationale: With the Store interface in place, adding PostgreSQL is additive: write PostgresStore, add pgx/v5/stdlib as a dependency, add DATABASE_URL to main.go and Docker Compose. The interface boundary guarantees no SQLite-specific SQL can appear in handlers. Delivers: PostgresStore implementing all Store methods with PostgreSQL dialect SQL; DATABASE_URL env var wired through main.go; separate dialect migration files; updated compose.dev.yml with optional postgres profile; documentation. Uses: github.com/jackc/pgx/v5/stdlib v5.9.1; github.com/golang-migrate/migrate/v4 v4.19.1; separate migrations/sqlite/ and migrations/postgres/ directories. Avoids: Pitfalls 1, 5, 8, 10, 11 — all are mitigated by the Store interface + per-dialect SQL + connection pool (no mutex) in PostgresStore. Research flag: Verify exact import path for pgx/v5/stdlib during implementation. The database/sql compatibility layer is standard but the import string should be confirmed against pkg.go.dev before coding.

Phase 4: UX Improvements

Rationale: These features are independent of the DB work but grouped together because they share the frontend codebase and several features share implementation logic (new-update indicator and toast notification use the same poll-comparison logic; bulk dismiss all and bulk dismiss by group share the same API endpoint design). Deferring UX until after the backend is correct means UX tests run against a trustworthy data layer. Delivers: Bulk acknowledge (all + by group) with a single backend endpoint (POST /api/updates/acknowledge-bulk); search and filter by name/status/tag (frontend-only); new-update badge/counter and page title count; light/dark theme toggle; drag handle always-visible fix; optimistic UI rollback with user-visible error on tag assignment failure. Uses: Existing React 19 + Tailwind + shadcn/ui stack; no new frontend dependencies expected. Avoids: Pitfalls 6, 7, 9 — optimistic rollback, bulk endpoint, accessible drag handle. Research flag: None needed for search/filter/theme/accessibility (standard patterns). The bulk acknowledge endpoint needs clear API contract design before frontend implementation begins — define the request/response shape first.

Phase Ordering Rationale

• Phase 1 before Phase 2: Bug fix tests become the regression suite; the refactor cannot accidentally regress behavior it has not validated first.
• Phase 2 before Phase 3: The Store interface is a structural prerequisite. PostgreSQL added to the monolith produces unmaintainable dialect branches.
• Phase 3 before Phase 4 (or parallel after Phase 2): UX features are mostly frontend and do not depend on PostgreSQL. However, the bulk acknowledge endpoint (AcknowledgeAll, AcknowledgeByTag) must be in the Store interface, which is finalized in Phase 2. Phase 4 frontend work can start once Phase 2 is merged; Phase 3 and Phase 4 can proceed in parallel.
• Never: Mix refactor and new feature in the same commit. Each phase should be independently reviewable and revertable.

Research Flags

Phases likely needing deeper research during planning:

• Phase 3 (PostgreSQL): Verify pgx/v5/stdlib import path (github.com/jackc/pgx/v5/stdlib) against pkg.go.dev before adding the dependency. Confirm golang-migrate database/sqlite sub-package still uses modernc.org/sqlite (not mattn/go-sqlite3) in v4.19.1 — this was verified but should be re-confirmed at time of implementation.

Phases with standard patterns (skip research-phase):

• Phase 1 (Bug fixes): 3-line SQL change with a clear regression test; no research needed.
• Phase 2 (Refactor): Standard Go repository interface pattern; no research needed.
• Phase 4 (UX): All features use existing stack (React, Tailwind, shadcn/ui); no new technologies introduced.

---

Confidence Assessment

Area	Confidence	Notes
Stack	HIGH	All versions verified via pkg.go.dev at time of research (2026-03-23); pgx v5.9.1 published 2026-03-22; golang-migrate v4.19.1 confirms modernc.org/sqlite
Features	MEDIUM	Feature priorities derived from direct codebase audit (CONCERNS.md, PROJECT.md) — HIGH confidence; competitive landscape analysis (Portainer, Uptime Kuma patterns) from training data only — MEDIUM
Architecture	HIGH	Based on direct analysis of pkg/diunwebhook/diunwebhook.go and cmd/diunwebhook/main.go; Store interface pattern is a well-established Go idiom with no ambiguity
Pitfalls	HIGH (backend) / MEDIUM (frontend)	Backend pitfalls sourced from direct code evidence (line numbers cited); PostgreSQL dialect differences from training knowledge — recommend verifying $1 placeholder syntax before implementation; frontend pitfalls sourced from direct code analysis

Overall confidence: HIGH

Gaps to Address

• PostgreSQL $1 placeholder syntax: PITFALLS.md flags this as MEDIUM confidence from training knowledge. Verify against the pgx/v5/stdlib documentation before writing any PostgreSQL query strings.
• golang-migrate CGO status at v4.19.1: Confirmed at research time that database/sqlite sub-package uses modernc.org/sqlite; re-confirm at implementation time that this has not changed in a patch release.
• Competitive feature validation: The UX feature priorities are based on self-hosted dashboard patterns (Portainer, Uptime Kuma) from training data. If the roadmapper wants higher confidence on feature ordering, a quick review of current Portainer CE and Uptime Kuma changelogs would validate the bulk-dismiss and search/filter priorities.
• golang-migrate vs hand-rolled migration runner: PITFALLS.md notes a 30-line hand-rolled runner is sufficient for this project's scale. STACK.md recommends golang-migrate. Either is valid — the roadmap phase should make a decision and commit to one approach before implementation begins to avoid scope creep.

---

Sources

Primary (HIGH confidence)

• pkg/diunwebhook/diunwebhook.go (direct codebase analysis, lines 48-117, 225, 352) — dialect issues, global state, INSERT OR REPLACE bug
• cmd/diunwebhook/main.go (direct codebase analysis) — entry point, env vars, mux wiring
• .planning/codebase/CONCERNS.md (prior audit) — confirmed FK enforcement gap, drag handle, bulk ops missing
• .planning/PROJECT.md (requirements source) — confirmed dual-DB requirement, no-CGO constraint, backward compatibility
• https://pkg.go.dev/github.com/jackc/pgx/v5@v5.9.1 — pgx v5 driver, version verified
• https://pkg.go.dev/github.com/golang-migrate/migrate/v4/database/sqlite — pure-Go SQLite sub-package confirmed
• https://pkg.go.dev/github.com/golang-migrate/migrate/v4/database/pgx/v5 — pgx/v5 migration sub-package confirmed
• https://pkg.go.dev/modernc.org/sqlite?tab=versions — v1.47.0 version verified

Secondary (MEDIUM confidence)

• Training-data knowledge of pgx/v5/stdlib database/sql adapter pattern — standard approach, verify import path at implementation
• Training-data knowledge of Portainer CE, Uptime Kuma, Dockcheck-web UX patterns — feature prioritization for self-hosted dashboards

Tertiary (LOW confidence)

• None

---

Research completed: 2026-03-23 Ready for roadmap: yes