408 lines
19 KiB
Markdown
408 lines
19 KiB
Markdown
# Architecture Patterns
|
|
|
|
**Domain:** Container update dashboard with dual-database support
|
|
**Project:** DiunDashboard
|
|
**Researched:** 2026-03-23
|
|
**Confidence:** HIGH (based on direct codebase analysis + established Go patterns)
|
|
|
|
---
|
|
|
|
## Current Architecture (Before Milestone)
|
|
|
|
The app is a single monolithic package (`pkg/diunwebhook/diunwebhook.go`) where database logic and HTTP handlers live in the same file and share package-level globals:
|
|
|
|
```
|
|
cmd/diunwebhook/main.go
|
|
└── pkg/diunwebhook/diunwebhook.go
|
|
├── package-level var db *sql.DB ← global, opaque
|
|
├── package-level var mu sync.Mutex ← global, opaque
|
|
├── InitDB(), UpdateEvent(), GetUpdates() ← storage functions
|
|
└── WebhookHandler, UpdatesHandler, ... ← handlers call db directly
|
|
```
|
|
|
|
**The problem for dual-database support:** SQL is written inline in handler functions and storage functions using SQLite-specific syntax:
|
|
- `INSERT OR REPLACE` (SQLite only; PostgreSQL uses `INSERT ... ON CONFLICT DO UPDATE`)
|
|
- `datetime('now')` (SQLite only; PostgreSQL uses `NOW()`)
|
|
- `AUTOINCREMENT` (SQLite only; PostgreSQL uses `SERIAL` or `GENERATED ALWAYS AS IDENTITY`)
|
|
- `PRAGMA foreign_keys = ON` (SQLite only; PostgreSQL enforces FKs by default)
|
|
- `modernc.org/sqlite` driver import (SQLite only)
|
|
|
|
There is no abstraction layer. Adding PostgreSQL directly to the current code would mean `if dialect == "postgres"` branches scattered across 380 lines — unmaintainable.
|
|
|
|
---
|
|
|
|
## Recommended Architecture
|
|
|
|
### Core Pattern: Repository Interface
|
|
|
|
Extract all database operations behind a Go interface. Each database backend implements the interface. The HTTP handlers receive the interface, not a concrete `*sql.DB`.
|
|
|
|
```
|
|
cmd/diunwebhook/main.go
|
|
├── reads DB_DRIVER env var ("sqlite" | "postgres")
|
|
├── constructs concrete store (SQLiteStore or PostgresStore)
|
|
└── passes store to Server struct
|
|
|
|
pkg/diunwebhook/
|
|
├── store.go ← Store interface definition
|
|
├── sqlite.go ← SQLiteStore implements Store
|
|
├── postgres.go ← PostgresStore implements Store
|
|
├── server.go ← Server struct holds Store, secret; methods = handlers
|
|
├── handlers.go ← HTTP handler methods on Server (no direct DB access)
|
|
└── models.go ← DiunEvent, UpdateEntry, Tag structs
|
|
```
|
|
|
|
### The Store Interface
|
|
|
|
```go
|
|
// pkg/diunwebhook/store.go
|
|
|
|
type Store interface {
|
|
// Lifecycle
|
|
Close() error
|
|
|
|
// Updates
|
|
UpsertEvent(ctx context.Context, event DiunEvent) error
|
|
GetAllUpdates(ctx context.Context) (map[string]UpdateEntry, error)
|
|
AcknowledgeUpdate(ctx context.Context, image string) (found bool, err error)
|
|
AcknowledgeAll(ctx context.Context) error
|
|
AcknowledgeByTag(ctx context.Context, tagID int) error
|
|
|
|
// Tags
|
|
ListTags(ctx context.Context) ([]Tag, error)
|
|
CreateTag(ctx context.Context, name string) (Tag, error)
|
|
DeleteTag(ctx context.Context, id int) (found bool, err error)
|
|
|
|
// Tag assignments
|
|
AssignTag(ctx context.Context, image string, tagID int) error
|
|
UnassignTag(ctx context.Context, image string) error
|
|
}
|
|
```
|
|
|
|
**Why this interface boundary:**
|
|
- Handlers never import a database driver — they only call `Store` methods.
|
|
- Tests inject a fake/in-memory implementation with no database.
|
|
- Adding a third backend (e.g., MySQL) requires implementing the interface, not modifying handlers.
|
|
- The interface expresses domain intent (`AcknowledgeUpdate`) not SQL mechanics (`UPDATE SET acknowledged_at`).
|
|
|
|
### Server Struct (Replaces Package Globals)
|
|
|
|
```go
|
|
// pkg/diunwebhook/server.go
|
|
|
|
type Server struct {
|
|
store Store
|
|
secret string
|
|
}
|
|
|
|
func NewServer(store Store, secret string) *Server {
|
|
return &Server{store: store, secret: secret}
|
|
}
|
|
|
|
// Handler methods: func (s *Server) WebhookHandler(w http.ResponseWriter, r *http.Request)
|
|
```
|
|
|
|
This addresses the "global mutable state" concern in CONCERNS.md. Multiple instances can coexist (useful for tests). Tests construct `NewServer(fakeStore, "")` without touching a real database.
|
|
|
|
---
|
|
|
|
## Component Boundaries
|
|
|
|
| Component | Responsibility | Communicates With | Location |
|
|
|-----------|---------------|-------------------|----------|
|
|
| `main.go` | Read env vars, construct store, wire server, run HTTP | `Server`, `SQLiteStore` or `PostgresStore` | `cmd/diunwebhook/` |
|
|
| `Server` | HTTP request lifecycle: parse, validate, delegate, respond | `Store` interface | `pkg/diunwebhook/server.go` |
|
|
| `Store` interface | Contract for all persistence operations | Implemented by `SQLiteStore`, `PostgresStore` | `pkg/diunwebhook/store.go` |
|
|
| `SQLiteStore` | All SQLite-specific SQL, schema init, migrations | `database/sql` + `modernc.org/sqlite` | `pkg/diunwebhook/sqlite.go` |
|
|
| `PostgresStore` | All PostgreSQL-specific SQL, schema init, migrations | `database/sql` + `pgx` stdlib driver | `pkg/diunwebhook/postgres.go` |
|
|
| `models.go` | Shared data structs (`DiunEvent`, `UpdateEntry`, `Tag`) | Imported by all components | `pkg/diunwebhook/models.go` |
|
|
| Frontend SPA | Visual dashboard, REST polling, drag-and-drop | HTTP API only (`/api/*`) | `frontend/src/` |
|
|
|
|
**Strict boundary rules:**
|
|
- `Server` never imports `modernc.org/sqlite` or `pgx` — only `Store`.
|
|
- `SQLiteStore` and `PostgresStore` never import `net/http`.
|
|
- `main.go` is the only place that chooses which backend to construct.
|
|
- `models.go` has zero imports beyond stdlib.
|
|
|
|
---
|
|
|
|
## Data Flow
|
|
|
|
### Webhook Ingestion
|
|
|
|
```
|
|
DIUN (external)
|
|
POST /webhook
|
|
→ Server.WebhookHandler
|
|
→ validate auth header (constant-time compare)
|
|
→ decode JSON into DiunEvent
|
|
→ store.UpsertEvent(ctx, event)
|
|
→ SQLiteStore: INSERT INTO updates ... ON CONFLICT(image) DO UPDATE SET ...
|
|
OR
|
|
→ PostgresStore: INSERT INTO updates ... ON CONFLICT (image) DO UPDATE SET ...
|
|
→ 200 OK
|
|
```
|
|
|
|
Both backends use standard SQL UPSERT syntax (fixing the current `INSERT OR REPLACE` bug). The SQL differs only in timestamp functions and driver-specific syntax, isolated to each store file.
|
|
|
|
### Dashboard Polling
|
|
|
|
```
|
|
Browser (every 5s)
|
|
GET /api/updates
|
|
→ Server.UpdatesHandler
|
|
→ store.GetAllUpdates(ctx)
|
|
→ SQLiteStore: SELECT ... LEFT JOIN ... (SQLite datetime handling)
|
|
OR
|
|
→ PostgresStore: SELECT ... LEFT JOIN ... (PostgreSQL timestamp handling)
|
|
→ encode map[string]UpdateEntry as JSON
|
|
→ 200 OK with body
|
|
```
|
|
|
|
### Acknowledge Flow
|
|
|
|
```
|
|
Browser click
|
|
PATCH /api/updates/{image}
|
|
→ Server.DismissHandler
|
|
→ extract image from URL path
|
|
→ store.AcknowledgeUpdate(ctx, image)
|
|
→ SQLiteStore: UPDATE ... SET acknowledged_at = datetime('now') WHERE image = ?
|
|
OR
|
|
→ PostgresStore: UPDATE ... SET acknowledged_at = NOW() WHERE image = $1
|
|
→ if not found: 404; else 204
|
|
```
|
|
|
|
### Startup / Initialization
|
|
|
|
```
|
|
main()
|
|
→ read DB_DRIVER env var ("sqlite" default, "postgres" opt-in)
|
|
→ if sqlite: NewSQLiteStore(DB_PATH) → opens modernc/sqlite, runs migrations
|
|
→ if postgres: NewPostgresStore(DSN) → opens pgx driver, runs migrations
|
|
→ NewServer(store, WEBHOOK_SECRET)
|
|
→ register handler methods on mux
|
|
→ srv.ListenAndServe()
|
|
```
|
|
|
|
---
|
|
|
|
## Migration Strategy: Dual Schema Management
|
|
|
|
Each store manages its own schema independently. No shared migration runner.
|
|
|
|
### SQLiteStore migrations
|
|
|
|
```go
|
|
func (s *SQLiteStore) migrate() error {
|
|
// Enable FK enforcement (fixes current bug)
|
|
s.db.Exec("PRAGMA foreign_keys = ON")
|
|
// Create tables with IF NOT EXISTS
|
|
// Apply ALTER TABLE migrations with error-ignore for idempotency
|
|
// Future: schema_version table for tracked migrations
|
|
}
|
|
```
|
|
|
|
### PostgresStore migrations
|
|
|
|
```go
|
|
func (s *PostgresStore) migrate() error {
|
|
// CREATE TABLE IF NOT EXISTS with PostgreSQL syntax
|
|
// SERIAL or IDENTITY for auto-increment
|
|
// FK enforcement is on by default — no PRAGMA needed
|
|
// Timestamp columns as TIMESTAMPTZ not TEXT
|
|
// Future: schema_version table for tracked migrations
|
|
}
|
|
```
|
|
|
|
**Key difference:** SQLite stores timestamps as RFC3339 TEXT (current behavior, must be preserved for backward compatibility). PostgreSQL stores timestamps as `TIMESTAMPTZ`. Each store handles its own serialization/deserialization of `time.Time`.
|
|
|
|
---
|
|
|
|
## Patterns to Follow
|
|
|
|
### Pattern 1: Constructor-Injected Store
|
|
|
|
**What:** `NewServer(store Store, secret string)` — store is a parameter, not a global.
|
|
|
|
**When:** Always. This replaces `var db *sql.DB` and `var mu sync.Mutex` package globals.
|
|
|
|
**Why:** Enables parallel test execution (each test creates its own `Server` with its own store). Eliminates the "single instance per process" constraint documented in CONCERNS.md.
|
|
|
|
### Pattern 2: Context Propagation
|
|
|
|
**What:** All `Store` interface methods accept `context.Context` as first argument.
|
|
|
|
**When:** From the initial Store interface design — do not add it later.
|
|
|
|
**Why:** Enables request cancellation and timeout propagation. PostgreSQL's `pgx` driver uses context natively. Without context, long-running queries cannot be cancelled when the client disconnects.
|
|
|
|
### Pattern 3: Driver-Specific SQL Isolated in Store Files
|
|
|
|
**What:** Each store file contains all SQL for that backend. No SQL strings in handlers.
|
|
|
|
**When:** Any time a handler needs to read or write data — call a Store method instead.
|
|
|
|
**Why:** SQLite uses `?` placeholders; PostgreSQL uses `$1, $2`. SQLite uses `datetime('now')`; PostgreSQL uses `NOW()`. SQLite uses `INTEGER PRIMARY KEY AUTOINCREMENT`; PostgreSQL uses `BIGSERIAL`. Mixing these in handler code creates unmaintainable conditional branches.
|
|
|
|
### Pattern 4: Idempotent Schema Creation
|
|
|
|
**What:** Both store constructors run schema setup on every startup via `CREATE TABLE IF NOT EXISTS`.
|
|
|
|
**When:** In `NewSQLiteStore()` and `NewPostgresStore()` constructors.
|
|
|
|
**Why:** Preserves current behavior where existing databases are safely upgraded. No external migration tool required for the current scope.
|
|
|
|
---
|
|
|
|
## Anti-Patterns to Avoid
|
|
|
|
### Anti-Pattern 1: Dialect Switches in Handlers
|
|
|
|
**What:** `if s.dialect == "postgres" { query = "..." } else { query = "..." }` inside handler methods.
|
|
|
|
**Why bad:** Handlers become aware of database internals. Every handler must be updated when adding a new backend. Tests must cover both branches per handler.
|
|
|
|
**Instead:** Move all dialect differences into the Store implementation. Handlers call `store.AcknowledgeUpdate(ctx, image)` — they never see SQL.
|
|
|
|
### Anti-Pattern 2: Shared `database/sql` Pool Exposed to Handlers
|
|
|
|
**What:** Passing `*sql.DB` directly to handlers (as the current package globals effectively do).
|
|
|
|
**Why bad:** Handlers can write arbitrary SQL, bypassing any abstraction. Type system cannot enforce the boundary.
|
|
|
|
**Instead:** Expose only the `Store` interface to `Server`. The `*sql.DB` is a private field of `SQLiteStore` / `PostgresStore`.
|
|
|
|
### Anti-Pattern 3: Single Store File for Both Backends
|
|
|
|
**What:** One `store.go` file with SQLite and PostgreSQL implementations side by side.
|
|
|
|
**Why bad:** The two implementations use different drivers, different SQL syntax, different connection setup. Merging them creates a large file with low cohesion.
|
|
|
|
**Instead:** `sqlite.go` for `SQLiteStore`, `postgres.go` for `PostgresStore`. Both in `pkg/diunwebhook/` package. Build tags are not needed since both compile — `main.go` chooses at runtime.
|
|
|
|
### Anti-Pattern 4: Reusing the Mutex from the Current Code
|
|
|
|
**What:** Keeping `var mu sync.Mutex` as a package global once the Store abstraction is introduced.
|
|
|
|
**Why bad:** `SQLiteStore` needs its own mutex (SQLite single-writer limitation). `PostgresStore` does not — PostgreSQL has its own concurrency control. Sharing a mutex across backends is wrong for Postgres and forces a false constraint.
|
|
|
|
**Instead:** `SQLiteStore` embeds `sync.Mutex` as a private field. `PostgresStore` does not use a mutex — it relies on `pgx`'s connection pool.
|
|
|
|
---
|
|
|
|
## Suggested Build Order
|
|
|
|
The dependency graph dictates this order. Each step must complete before the next.
|
|
|
|
### Step 1: Fix Current SQLite Bugs (prerequisite)
|
|
|
|
Fix `INSERT OR REPLACE` → proper UPSERT, add `PRAGMA foreign_keys = ON`. These bugs exist independent of the refactor and will be harder to fix correctly after the abstraction layer is introduced. Do this on the current flat code, with tests confirming the fix.
|
|
|
|
**Rationale:** Existing users rely on SQLite working correctly. The refactor must not change behavior — fixing bugs before refactoring means the tests that pass after bugfix become the regression suite for the refactor.
|
|
|
|
### Step 2: Extract Models
|
|
|
|
Move `DiunEvent`, `UpdateEntry`, `Tag` into `models.go`. No logic changes. This is a safe mechanical split — confirms the package compiles and tests pass after file reorganization.
|
|
|
|
**Rationale:** Models are referenced by both Store implementations and by Server. Extracting them first removes a coupling that would otherwise force all files to reference a single monolith.
|
|
|
|
### Step 3: Define Store Interface + SQLiteStore
|
|
|
|
Define the `Store` interface in `store.go`. Implement `SQLiteStore` in `sqlite.go` by moving all SQL from the current monolith into `SQLiteStore` methods. All existing tests must still pass with zero behavior changes. This step does not add PostgreSQL — it only restructures.
|
|
|
|
**Rationale:** Restructuring and new backend introduction must be separate commits. If tests break, the cause is isolated to the refactor, not the PostgreSQL code.
|
|
|
|
### Step 4: Introduce Server Struct
|
|
|
|
Refactor `pkg/diunwebhook/` to a struct-based design: `Server` with injected `Store`. Update `main.go` to construct `NewServer(store, secret)` and register `s.WebhookHandler` etc. on the mux. All existing tests must still pass.
|
|
|
|
**Rationale:** This decouples handler tests from database initialization. Tests can now construct a `Server` with a stub `Store` — faster, no filesystem I/O, parallelisable.
|
|
|
|
### Step 5: Implement PostgresStore
|
|
|
|
Add `postgres.go` with `PostgresStore` implementing the `Store` interface. Add `pgx` (`github.com/jackc/pgx/v5`) as a dependency using its `database/sql` compatibility shim (`pgx/v5/stdlib`) to avoid changing the `*sql.DB` usage pattern in `SQLiteStore`. Add `DB_DRIVER` env var to `main.go` — `"sqlite"` (default) or `"postgres"`. Add `DATABASE_URL` env var for PostgreSQL DSN. Update `compose.dev.yml` and deployment docs.
|
|
|
|
**Rationale:** `pgx/v5/stdlib` registers as a `database/sql` driver, so `PostgresStore` can use the same `*sql.DB` API as `SQLiteStore`. This minimizes the interface surface difference between the two implementations.
|
|
|
|
### Step 6: Update Docker Compose and Configuration Docs
|
|
|
|
Update `compose.dev.yml` with a `postgres` service profile. Update deployment documentation for PostgreSQL setup. This is explicitly the last step — infrastructure follows working code.
|
|
|
|
---
|
|
|
|
## Scalability Considerations
|
|
|
|
| Concern | SQLite (current) | PostgreSQL (new) |
|
|
|---------|-----------------|-----------------|
|
|
| Concurrent writes | Serialized by mutex + `SetMaxOpenConns(1)` | Connection pool, DB-level locking |
|
|
| Multiple server instances | Not possible (file lock) | Supported via shared DSN |
|
|
| Read performance | `LEFT JOIN` on every poll | Same query; can add indexes |
|
|
| Data retention | Unbounded growth | Same; retention policy deferred |
|
|
| Connection management | Single connection | `pgx` pool (default 5 conns) |
|
|
|
|
For the self-hosted single-user target audience, both backends are more than sufficient. PostgreSQL is recommended when the user already runs a PostgreSQL instance (common in Coolify deployments) to avoid volume-mounting complexity and SQLite file permission issues.
|
|
|
|
---
|
|
|
|
## Component Interaction Diagram
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────┐
|
|
│ cmd/diunwebhook/main.go │
|
|
│ │
|
|
│ DB_DRIVER=sqlite → NewSQLiteStore(DB_PATH) │
|
|
│ DB_DRIVER=postgres → NewPostgresStore(DATABASE_URL) │
|
|
│ │ │
|
|
│ NewServer(store, secret)│ │
|
|
└──────────────────────────┼──────────────────────────────┘
|
|
│
|
|
▼
|
|
┌──────────────────────────────────────────┐
|
|
│ Server (pkg/diunwebhook/server.go) │
|
|
│ │
|
|
│ store Store ◄──── interface boundary │
|
|
│ secret string │
|
|
│ │
|
|
│ .WebhookHandler() │
|
|
│ .UpdatesHandler() │
|
|
│ .DismissHandler() │
|
|
│ .TagsHandler() │
|
|
│ .TagByIDHandler() │
|
|
│ .TagAssignmentHandler() │
|
|
└──────────────┬───────────────────────────┘
|
|
│ calls Store methods only
|
|
▼
|
|
┌──────────────────────────────────────────┐
|
|
│ Store interface (store.go) │
|
|
│ UpsertEvent / GetAllUpdates / │
|
|
│ AcknowledgeUpdate / ListTags / ... │
|
|
└────────────┬─────────────────┬───────────┘
|
|
│ │
|
|
▼ ▼
|
|
┌────────────────────┐ ┌──────────────────────┐
|
|
│ SQLiteStore │ │ PostgresStore │
|
|
│ (sqlite.go) │ │ (postgres.go) │
|
|
│ │ │ │
|
|
│ modernc.org/sqlite│ │ pgx/v5/stdlib │
|
|
│ *sql.DB │ │ *sql.DB │
|
|
│ sync.Mutex │ │ (no mutex needed) │
|
|
│ SQLite SQL syntax │ │ PostgreSQL SQL syntax│
|
|
└────────────────────┘ └──────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## Sources
|
|
|
|
- Direct analysis of `pkg/diunwebhook/diunwebhook.go` (current monolith) — HIGH confidence
|
|
- Direct analysis of `cmd/diunwebhook/main.go` (entry point) — HIGH confidence
|
|
- `.planning/codebase/CONCERNS.md` (identified tech debt) — HIGH confidence
|
|
- `.planning/PROJECT.md` (constraints: no CGO, backward compat, dual DB) — HIGH confidence
|
|
- Go `database/sql` standard library interface pattern — HIGH confidence (well-established Go idiom)
|
|
- `pgx/v5/stdlib` compatibility layer for `database/sql` — MEDIUM confidence (standard approach, verify exact import path during implementation)
|
|
|
|
---
|
|
|
|
*Architecture research: 2026-03-23*
|