From 9ac8410239371f8763da0ed327f1143fc6c1181c Mon Sep 17 00:00:00 2001 From: Jean-Luc Makiola Date: Sun, 5 Apr 2026 11:55:05 +0200 Subject: [PATCH] docs(17): capture phase context --- .../phases/17-object-storage/17-CONTEXT.md | 117 ++++++++++++++++++ .../17-object-storage/17-DISCUSSION-LOG.md | 64 ++++++++++ 2 files changed, 181 insertions(+) create mode 100644 .planning/phases/17-object-storage/17-CONTEXT.md create mode 100644 .planning/phases/17-object-storage/17-DISCUSSION-LOG.md diff --git a/.planning/phases/17-object-storage/17-CONTEXT.md b/.planning/phases/17-object-storage/17-CONTEXT.md new file mode 100644 index 0000000..5fefc7b --- /dev/null +++ b/.planning/phases/17-object-storage/17-CONTEXT.md @@ -0,0 +1,117 @@ +# Phase 17: Object Storage - Context + +**Gathered:** 2026-04-05 +**Status:** Ready for planning + + +## Phase Boundary + +Move image storage from local filesystem (`uploads/` directory) to MinIO (S3-compatible object storage). All image uploads go to MinIO. Existing images are migrated. Image URLs are served via presigned URLs or a proxy. Docker Compose includes MinIO for local development. + + + + +## Implementation Decisions + +### S3 Client +- **D-01:** Use `@aws-sdk/client-s3` (AWS SDK v3) for MinIO communication. Works with any S3-compatible service. Tree-shakeable, well-maintained. +- **D-02:** Use `@aws-sdk/s3-request-presigner` for generating presigned URLs. + +### Storage Service +- **D-03:** Create `src/server/services/storage.service.ts` — thin wrapper around S3 SDK with functions: `uploadImage(buffer, filename, contentType)`, `deleteImage(filename)`, `getImageUrl(filename)`. +- **D-04:** `getImageUrl()` returns a presigned URL with configurable expiry (default 1 hour). No proxy — client fetches directly from MinIO. +- **D-05:** Environment variables: `S3_ENDPOINT`, `S3_ACCESS_KEY`, `S3_SECRET_KEY`, `S3_BUCKET` (default: `gearbox-images`), `S3_REGION` (default: `us-east-1`). + +### Image Upload Changes +- **D-06:** `POST /api/images` and `POST /api/images/from-url` upload to MinIO instead of local filesystem. Same filename pattern (UUID-based). +- **D-07:** `fetchImageFromUrl()` in image.service.ts uploads the fetched buffer to MinIO instead of writing to disk. +- **D-08:** Remove the static file serving for `/uploads/*` from the server — images are served via presigned URLs from MinIO. + +### Image URL Serving +- **D-09:** When returning item/candidate data with `imageFilename`, the API resolves it to a presigned MinIO URL. Add a `imageUrl` field to API responses (or replace `imageFilename` with the URL). +- **D-10:** Client components use the presigned URL directly. No changes to image display components beyond using the URL field. + +### Migration +- **D-11:** One-time migration script (`scripts/migrate-images-to-minio.ts`) reads all files from `uploads/`, uploads each to MinIO bucket, verifies upload, logs progress. +- **D-12:** No filename changes during migration — existing `imageFilename` values in the database remain valid as MinIO object keys. + +### Docker Compose +- **D-13:** MinIO service in docker-compose.yml (both dev and prod) with automatic bucket creation on startup. +- **D-14:** Dev compose uses fixed credentials for simplicity. Prod compose uses env vars. + +### Claude's Discretion +- Presigned URL expiry duration (1h default, configurable) +- Whether to add a GET /api/images/:filename proxy endpoint as fallback +- MinIO Docker image version +- Bucket policy (private with presigned URLs vs public-read) +- Whether to delete local files after successful migration +- Error handling strategy for upload failures + + + + +## Canonical References + +**Downstream agents MUST read these before planning or implementing.** + +### Image Handling (to be refactored) +- `src/server/services/image.service.ts` — Current local file storage logic +- `src/server/routes/images.ts` — Upload endpoints (file + URL) +- `src/server/index.ts` — Static file serving for uploads/ + +### Schema (imageFilename columns) +- `src/db/schema.ts` — `imageFilename` on items and threadCandidates tables + +### Docker +- `docker-compose.yml` — Production compose (add MinIO) +- `docker-compose.dev.yml` — Dev compose (add MinIO) +- `.env.example` — Add S3 env vars + +### Client (image display) +- `src/client/lib/api.ts` — API wrapper +- `src/client/components/` — Components that display images + +### Requirements +- `.planning/REQUIREMENTS.md` — IMG-01 through IMG-04 + + + + +## Existing Code Insights + +### Reusable Assets +- `image.service.ts` — refactor to use storage service instead of local fs +- UUID filename generation pattern — keep as-is for MinIO object keys +- Content type validation — keep in image routes + +### Established Patterns +- Service DI pattern (db, userId params) — storage service is stateless, no db needed +- Async operations — all storage ops will be async (already async pattern) +- Environment config via `process.env` — same for S3 config + +### Integration Points +- `src/server/routes/images.ts` — Replace Bun.write with storage.upload +- `src/server/services/image.service.ts` — Replace Bun.write with storage.upload +- `src/server/index.ts` — Remove static file serving for uploads/ +- API response serialization — add imageUrl field from presigned URL + + + + +## Specific Ideas + +No specific requirements — open to standard approaches for S3-compatible object storage with MinIO. + + + + +## Deferred Ideas + +None — discussion stayed within phase scope + + + +--- + +*Phase: 17-object-storage* +*Context gathered: 2026-04-05* diff --git a/.planning/phases/17-object-storage/17-DISCUSSION-LOG.md b/.planning/phases/17-object-storage/17-DISCUSSION-LOG.md new file mode 100644 index 0000000..f912389 --- /dev/null +++ b/.planning/phases/17-object-storage/17-DISCUSSION-LOG.md @@ -0,0 +1,64 @@ +# Phase 17: Object Storage - Discussion Log + +> **Audit trail only.** Do not use as input to planning, research, or execution agents. + +**Date:** 2026-04-05 +**Phase:** 17-object-storage +**Areas discussed:** S3 Client, URL Strategy, Storage Abstraction, Migration Approach +**Mode:** --auto --batch + +--- + +## S3 Client + +| Option | Description | Selected | +|--------|-------------|----------| +| @aws-sdk/client-s3 | Official AWS SDK v3, S3-compatible, tree-shakeable | ✓ | +| minio-js | MinIO-specific client | | +| undici/fetch with S3 API | Raw HTTP calls | | + +**User's choice:** @aws-sdk/client-s3 (auto-selected) + +--- + +## URL Strategy + +| Option | Description | Selected | +|--------|-------------|----------| +| Presigned URLs | MinIO generates time-limited signed URLs, client fetches directly | ✓ | +| Proxy through GearBox API | Server fetches from MinIO, streams to client | | +| Public bucket | No auth needed, direct MinIO URLs | | + +**User's choice:** Presigned URLs (auto-selected) + +--- + +## Storage Abstraction + +| Option | Description | Selected | +|--------|-------------|----------| +| Thin storage service | Single file wrapping S3 SDK with upload/delete/getUrl | ✓ | +| Full abstraction layer | Interface with local/S3 implementations | | + +**User's choice:** Thin storage service (auto-selected) + +--- + +## Migration Approach + +| Option | Description | Selected | +|--------|-------------|----------| +| One-time script | Reads uploads/, uploads to MinIO, same filenames | ✓ | +| Lazy migration | Upload to MinIO on first access, fallback to local | | + +**User's choice:** One-time script (auto-selected) + +--- + +## Claude's Discretion + +- Presigned URL expiry, proxy fallback, MinIO version, bucket policy, cleanup strategy + +## Deferred Ideas + +None