# Project Research Summary

**Project:** DiunDashboard — PostgreSQL milestone + UX improvements
**Domain:** Self-hosted container image update monitoring dashboard (Go backend + React SPA)
**Researched:** 2026-03-23
**Confidence:** HIGH (stack and architecture sourced from direct codebase analysis and verified package versions; features MEDIUM due to tool restrictions during research)

## Executive Summary

DiunDashboard is a self-hosted Go + React dashboard that receives DIUN webhook events and presents a persistent, acknowledgeable list of container images with available updates. The current milestone covers two parallel tracks: (1) fixing active data-correctness bugs and adding PostgreSQL as an alternative to SQLite, and (2) delivering UX improvements users need before the tool is genuinely usable at scale (bulk dismiss, search/filter, new-update indicators). Both tracks have well-understood solutions rooted in established Go patterns — the engineering risk is low provided the work is sequenced correctly.

The recommended approach is a strict dependency-first build order. The SQLite data-integrity bugs (`INSERT OR REPLACE` silently deleting tag assignments, missing FK pragma) must be fixed before any other work because they undermine trust in the tool and will complicate the subsequent refactor if left in. The backend refactor — introducing a `Store` interface and a `Server` struct to replace package-level globals — is the foundational prerequisite for PostgreSQL support, parallel test execution, and reliable UX features. PostgreSQL is then a clean additive step: implement `PostgresStore`, wire the `DATABASE_URL` env var into `main.go`, and provide dialect-appropriate SQL in the new store file.

The primary risk is dialect leakage: SQLite-specific SQL (`datetime('now')`, `INSERT OR REPLACE`, `?` placeholders, `AUTOINCREMENT`, `PRAGMA`) scattered across handler functions will silently break on PostgreSQL if the Store interface abstraction is not in place before any PostgreSQL code is written. Secondary risks are a missing versioned migration runner (which leaves existing user databases in an unknown state on upgrade) and bulk dismiss implemented as N sequential API calls rather than a single transactional endpoint. Both risks have well-documented mitigations and are easy to prevent if addressed in the correct phase.

---

## Key Findings

### Recommended Stack

The existing stack is largely correct and requires minimal additions. The PostgreSQL driver is `github.com/jackc/pgx/v5/stdlib` (v5.9.1, verified 2026-03-22) — the de-facto community standard. Its `stdlib` adapter makes it a drop-in for the existing `*sql.DB` code path; the native pgx interface is not needed. `lib/pq` is explicitly maintenance-only and must not be used. For schema migrations, `github.com/golang-migrate/migrate/v4` (v4.19.1) supports both the project's `modernc.org/sqlite` (pure-Go, no CGO) and pgx/v5 backends via separately maintained sub-packages. The existing SQLite driver should be upgraded from v1.46.1 to v1.47.0.

**Core technologies:**
- `github.com/jackc/pgx/v5/stdlib` v5.9.1: PostgreSQL driver — only viable current option; `lib/pq` is maintenance-only
- `github.com/golang-migrate/migrate/v4` v4.19.1: schema migrations — explicit `modernc.org/sqlite` support satisfies no-CGO constraint
- `modernc.org/sqlite` v1.47.0: existing SQLite driver (upgrade from v1.46.1) — must remain for pure-Go cross-compilation

No ORM. No query-builder library. The query set is 8 operations across 3 tables; raw SQL per store implementation is simpler, easier to audit, and matches the existing project style.

**Configuration:** `DATABASE_URL` env var (when set, activates PostgreSQL mode). `DB_PATH` retained for SQLite. No separate `DB_DRIVER` variable needed.

### Expected Features

The feature research produced a clear priority stack grounded in the documented concerns and self-hosted dashboard conventions. Data integrity is a prerequisite for everything else — broken data collapses user trust faster than any missing feature.

**Must have (table stakes):**
- SQLite data integrity fix (UPSERT + FK pragma) — existing bug silently deletes tag assignments on every DIUN event
- Bulk acknowledge: dismiss all + dismiss by group — O(n) clicking for 20+ images causes abandonment
- Search + filter by image name, status, and tag — standard affordance for any list exceeding 10 items
- New-update indicator (badge/counter) and page/tab title count — persistent visibility is the core value proposition
- PostgreSQL support — required for users running Coolify or other Postgres-backed infrastructure

**Should have (differentiators):**
- Toast notification on new update arrival during polling — shares implementation with new-update indicator
- Sort order controls (newest first, by name, by registry) — pure frontend, no backend change
- Light/dark theme toggle — low complexity, removes a known complaint
- Drag handle always visible (accessibility) — currently hover-only, invisible on touch/keyboard
- Optimistic UI rollback on tag assignment failure — current code has no error recovery path

**Defer (v2+):**
- Data retention / auto-cleanup of acknowledged entries — real concern but not urgent for most users
- Alternative tag assignment dropdown — drag-and-drop exists; dropdown is an accessibility improvement, not a blocker
- Browser notification API — high UX risk, low reward vs. badge approach
- Auto-grouping by Docker stack — requires Docker socket access; different scope entirely

### Architecture Approach

The architecture follows a standard Go repository interface pattern. The current monolith (`diunwebhook.go` with package-level `var db *sql.DB` and `var mu sync.Mutex`) is extracted into a `Store` interface implemented by two concrete types (`SQLiteStore`, `PostgresStore`), with HTTP handlers moved to methods on a `Server` struct that holds a `Store`. This pattern eliminates global state, enables parallel tests without resets, and enforces a strict boundary: handlers never see SQL, store implementations never see HTTP.

**Major components:**
1. `Store` interface (`store.go`) — contract for all persistence; 11 methods covering updates, tags, and assignments
2. `SQLiteStore` (`sqlite.go`) — SQLite-specific SQL, `sync.Mutex`, `SetMaxOpenConns(1)`, `PRAGMA foreign_keys = ON`
3. `PostgresStore` (`postgres.go`) — PostgreSQL-specific SQL, pgx connection pool, no mutex, `db.BeginTx` for atomicity
4. `Server` struct (`server.go`) — holds `Store` and `secret`; all HTTP handlers are methods on `Server`
5. `models.go` — shared `DiunEvent`, `UpdateEntry`, `Tag` structs with no imports beyond stdlib
6. `main.go` — sole location where backend is chosen (`DATABASE_URL` present → PostgreSQL, absent → SQLite)
7. Frontend SPA — unchanged API contract; communicates with backend via `/api/*` only

**Key pattern: `SQLiteStore` retains `sync.Mutex`; `PostgresStore` does not.** These are structurally different and must not share a mutex.

**Migration strategy:** Separate DDL per dialect (`migrations/sqlite/` and `migrations/postgres/`). Both embedded in the binary via `//go:embed`. A versioned `schema_migrations` table prevents re-running migrations on existing databases and makes upgrade failures visible.

### Critical Pitfalls

1. **SQLite-specific SQL leaking into shared code** — `datetime('now')`, `INSERT OR REPLACE`, `?` placeholders, `AUTOINCREMENT`, and `PRAGMA` all fail on PostgreSQL. Prevention: Store interface forces all SQL into store files; handlers call named methods only; integration tests run both stores.

2. **`INSERT OR REPLACE` silently deleting tag assignments** — SQLite implements this as DELETE + INSERT, which cascades to `tag_assignments` and erases the user's groupings on every DIUN event. Prevention: replace with `INSERT ... ON CONFLICT(image) DO UPDATE SET ...`; add `PRAGMA foreign_keys = ON`; add regression test asserting tag survives a second `UpdateEvent` call.

3. **Global package-level state blocks dual-DB without struct refactor** — `var db *sql.DB` at package scope means there is only one DB handle; PostgreSQL cannot be added without introducing `if dbType == "postgres"` branches across every handler. Prevention: `Server` struct with injected `Store` must precede all PostgreSQL work.

4. **No versioned migration runner** — silent `ALTER TABLE` with discarded errors leaves existing SQLite databases in an unknown state on upgrade. Prevention: `schema_migrations` version table; log every migration attempt; never swallow DDL errors.

5. **Bulk dismiss implemented as N sequential API calls** — 30 acknowledged images = 30 round trips, 30 mutex acquisitions, 30 React re-renders with potential flickering and partial-state failure. Prevention: design `POST /api/updates/acknowledge-bulk` endpoint first; one call, one transaction, one state update.

---

## Implications for Roadmap

Based on the dependency graph from feature and architecture research, the milestone decomposes into four phases. The ordering is non-negotiable: each phase is a prerequisite for the next.

### Phase 1: Data Integrity Fixes
**Rationale:** The `INSERT OR REPLACE` bug is active in production and deletes user data on every DIUN event. Fixing it before the refactor means the bug-fix tests become the regression suite that validates the refactor did not regress behavior. No other work is credible until the data layer is correct.
**Delivers:** Trustworthy persistence — tag assignments survive new DIUN events; FK enforcement works; acknowledged state is preserved correctly.
**Addresses:** Table-stakes feature "Data integrity across restarts"; Pitfalls 2, 10 (timestamp fix can be included here).
**Avoids:** Shipping the bug in both DB paths; losing the fix in refactor noise.
**Research flag:** None needed — the fix is a 3-line SQL change with a clear regression test. Standard patterns apply.

### Phase 2: Backend Refactor — Store Interface + Server Struct
**Rationale:** The global state architecture makes PostgreSQL support structurally impossible without this refactor. All subsequent work (PostgreSQL implementation, parallel test execution, safer UX features) depends on this change. The refactor must be behavior-neutral — all existing tests pass before PostgreSQL is introduced.
**Delivers:** `Store` interface, `SQLiteStore` implementation, `Server` struct with constructor injection, models in `models.go`. Zero behavior change for existing SQLite users.
**Uses:** Existing `modernc.org/sqlite`; `database/sql` standard library; no new dependencies.
**Implements:** Core architecture pattern from ARCHITECTURE.md; eliminates Pitfall 3 (global state) and Pitfall 4 (migration runner) in one phase.
**Avoids:** Introducing PostgreSQL and refactoring simultaneously (would make failures ambiguous).
**Research flag:** None needed — this is a standard Go repository interface pattern with well-documented prior art.

### Phase 3: PostgreSQL Support
**Rationale:** With the `Store` interface in place, adding PostgreSQL is additive: write `PostgresStore`, add `pgx/v5/stdlib` as a dependency, add `DATABASE_URL` to `main.go` and Docker Compose. The interface boundary guarantees no SQLite-specific SQL can appear in handlers.
**Delivers:** `PostgresStore` implementing all `Store` methods with PostgreSQL dialect SQL; `DATABASE_URL` env var wired through `main.go`; separate dialect migration files; updated `compose.dev.yml` with optional `postgres` profile; documentation.
**Uses:** `github.com/jackc/pgx/v5/stdlib` v5.9.1; `github.com/golang-migrate/migrate/v4` v4.19.1; separate `migrations/sqlite/` and `migrations/postgres/` directories.
**Avoids:** Pitfalls 1, 5, 8, 10, 11 — all are mitigated by the Store interface + per-dialect SQL + connection pool (no mutex) in `PostgresStore`.
**Research flag:** Verify exact import path for `pgx/v5/stdlib` during implementation. The `database/sql` compatibility layer is standard but the import string should be confirmed against pkg.go.dev before coding.

### Phase 4: UX Improvements
**Rationale:** These features are independent of the DB work but grouped together because they share the frontend codebase and several features share implementation logic (new-update indicator and toast notification use the same poll-comparison logic; bulk dismiss all and bulk dismiss by group share the same API endpoint design). Deferring UX until after the backend is correct means UX tests run against a trustworthy data layer.
**Delivers:** Bulk acknowledge (all + by group) with a single backend endpoint (`POST /api/updates/acknowledge-bulk`); search and filter by name/status/tag (frontend-only); new-update badge/counter and page title count; light/dark theme toggle; drag handle always-visible fix; optimistic UI rollback with user-visible error on tag assignment failure.
**Uses:** Existing React 19 + Tailwind + shadcn/ui stack; no new frontend dependencies expected.
**Avoids:** Pitfalls 6, 7, 9 — optimistic rollback, bulk endpoint, accessible drag handle.
**Research flag:** None needed for search/filter/theme/accessibility (standard patterns). The bulk acknowledge endpoint needs clear API contract design before frontend implementation begins — define the request/response shape first.

### Phase Ordering Rationale

- **Phase 1 before Phase 2:** Bug fix tests become the regression suite; the refactor cannot accidentally regress behavior it has not validated first.
- **Phase 2 before Phase 3:** The Store interface is a structural prerequisite. PostgreSQL added to the monolith produces unmaintainable dialect branches.
- **Phase 3 before Phase 4 (or parallel after Phase 2):** UX features are mostly frontend and do not depend on PostgreSQL. However, the bulk acknowledge endpoint (`AcknowledgeAll`, `AcknowledgeByTag`) must be in the `Store` interface, which is finalized in Phase 2. Phase 4 frontend work can start once Phase 2 is merged; Phase 3 and Phase 4 can proceed in parallel.
- **Never:** Mix refactor and new feature in the same commit. Each phase should be independently reviewable and revertable.

### Research Flags

Phases likely needing deeper research during planning:
- **Phase 3 (PostgreSQL):** Verify `pgx/v5/stdlib` import path (`github.com/jackc/pgx/v5/stdlib`) against pkg.go.dev before adding the dependency. Confirm `golang-migrate` `database/sqlite` sub-package still uses `modernc.org/sqlite` (not `mattn/go-sqlite3`) in v4.19.1 — this was verified but should be re-confirmed at time of implementation.

Phases with standard patterns (skip research-phase):
- **Phase 1 (Bug fixes):** 3-line SQL change with a clear regression test; no research needed.
- **Phase 2 (Refactor):** Standard Go repository interface pattern; no research needed.
- **Phase 4 (UX):** All features use existing stack (React, Tailwind, shadcn/ui); no new technologies introduced.

---

## Confidence Assessment

| Area | Confidence | Notes |
|------|------------|-------|
| Stack | HIGH | All versions verified via pkg.go.dev at time of research (2026-03-23); pgx v5.9.1 published 2026-03-22; golang-migrate v4.19.1 confirms `modernc.org/sqlite` |
| Features | MEDIUM | Feature priorities derived from direct codebase audit (CONCERNS.md, PROJECT.md) — HIGH confidence; competitive landscape analysis (Portainer, Uptime Kuma patterns) from training data only — MEDIUM |
| Architecture | HIGH | Based on direct analysis of `pkg/diunwebhook/diunwebhook.go` and `cmd/diunwebhook/main.go`; Store interface pattern is a well-established Go idiom with no ambiguity |
| Pitfalls | HIGH (backend) / MEDIUM (frontend) | Backend pitfalls sourced from direct code evidence (line numbers cited); PostgreSQL dialect differences from training knowledge — recommend verifying `$1` placeholder syntax before implementation; frontend pitfalls sourced from direct code analysis |

**Overall confidence:** HIGH

### Gaps to Address

- **PostgreSQL `$1` placeholder syntax:** PITFALLS.md flags this as MEDIUM confidence from training knowledge. Verify against the `pgx/v5/stdlib` documentation before writing any PostgreSQL query strings.
- **`golang-migrate` CGO status at v4.19.1:** Confirmed at research time that `database/sqlite` sub-package uses `modernc.org/sqlite`; re-confirm at implementation time that this has not changed in a patch release.
- **Competitive feature validation:** The UX feature priorities are based on self-hosted dashboard patterns (Portainer, Uptime Kuma) from training data. If the roadmapper wants higher confidence on feature ordering, a quick review of current Portainer CE and Uptime Kuma changelogs would validate the bulk-dismiss and search/filter priorities.
- **`golang-migrate` vs hand-rolled migration runner:** PITFALLS.md notes a 30-line hand-rolled runner is sufficient for this project's scale. STACK.md recommends `golang-migrate`. Either is valid — the roadmap phase should make a decision and commit to one approach before implementation begins to avoid scope creep.

---

## Sources

### Primary (HIGH confidence)
- `pkg/diunwebhook/diunwebhook.go` (direct codebase analysis, lines 48-117, 225, 352) — dialect issues, global state, INSERT OR REPLACE bug
- `cmd/diunwebhook/main.go` (direct codebase analysis) — entry point, env vars, mux wiring
- `.planning/codebase/CONCERNS.md` (prior audit) — confirmed FK enforcement gap, drag handle, bulk ops missing
- `.planning/PROJECT.md` (requirements source) — confirmed dual-DB requirement, no-CGO constraint, backward compatibility
- https://pkg.go.dev/github.com/jackc/pgx/v5@v5.9.1 — pgx v5 driver, version verified
- https://pkg.go.dev/github.com/golang-migrate/migrate/v4/database/sqlite — pure-Go SQLite sub-package confirmed
- https://pkg.go.dev/github.com/golang-migrate/migrate/v4/database/pgx/v5 — pgx/v5 migration sub-package confirmed
- https://pkg.go.dev/modernc.org/sqlite?tab=versions — v1.47.0 version verified

### Secondary (MEDIUM confidence)
- Training-data knowledge of `pgx/v5/stdlib` `database/sql` adapter pattern — standard approach, verify import path at implementation
- Training-data knowledge of Portainer CE, Uptime Kuma, Dockcheck-web UX patterns — feature prioritization for self-hosted dashboards

### Tertiary (LOW confidence)
- None

---
*Research completed: 2026-03-23*
*Ready for roadmap: yes*