docs(17): create phase plan for object storage migration
This commit is contained in:
@@ -161,7 +161,11 @@ Plans:
|
||||
2. All previously uploaded images are accessible after migration to MinIO (no broken images)
|
||||
3. Image URLs work correctly in all views (collection, planning, setups, comparison table)
|
||||
4. Docker Compose includes MinIO for local development with no manual bucket setup required
|
||||
**Plans**: TBD
|
||||
**Plans:** 3 plans
|
||||
Plans:
|
||||
- [ ] 17-01-PLAN.md — Storage service abstraction + Docker Compose MinIO infrastructure
|
||||
- [ ] 17-02-PLAN.md — Server-side image handling refactoring (routes, services, MCP tools)
|
||||
- [ ] 17-03-PLAN.md — Client component updates + image migration script
|
||||
|
||||
### Phase 18: Global Items & Public Profiles
|
||||
**Goal**: Users can discover gear through a global catalog and share their setups publicly via profile pages
|
||||
@@ -196,5 +200,5 @@ Plans:
|
||||
| 14. PostgreSQL Migration | v2.0 | 0/? | Not started | - |
|
||||
| 15. External Authentication | v2.0 | 0/? | Not started | - |
|
||||
| 16. Multi-User Data Model | v2.0 | 2/4 | Complete | 2026-04-05 |
|
||||
| 17. Object Storage | v2.0 | 0/? | Not started | - |
|
||||
| 17. Object Storage | v2.0 | 0/3 | Not started | - |
|
||||
| 18. Global Items & Public Profiles | v2.0 | 0/? | Not started | - |
|
||||
|
||||
194
.planning/phases/17-object-storage/17-01-PLAN.md
Normal file
194
.planning/phases/17-object-storage/17-01-PLAN.md
Normal file
@@ -0,0 +1,194 @@
|
||||
---
|
||||
phase: 17-object-storage
|
||||
plan: 01
|
||||
type: execute
|
||||
wave: 1
|
||||
depends_on: []
|
||||
files_modified:
|
||||
- src/server/services/storage.service.ts
|
||||
- tests/services/storage.service.test.ts
|
||||
- docker-compose.yml
|
||||
- docker-compose.dev.yml
|
||||
- .env.example
|
||||
autonomous: true
|
||||
requirements: [IMG-01, IMG-04]
|
||||
|
||||
must_haves:
|
||||
truths:
|
||||
- "Storage service can upload a buffer to S3-compatible storage"
|
||||
- "Storage service can delete an object from S3-compatible storage"
|
||||
- "Storage service can generate a presigned URL for an object"
|
||||
- "Docker Compose starts MinIO with automatic bucket creation"
|
||||
artifacts:
|
||||
- path: "src/server/services/storage.service.ts"
|
||||
provides: "S3 storage abstraction"
|
||||
exports: ["uploadImage", "deleteImage", "getImageUrl"]
|
||||
- path: "tests/services/storage.service.test.ts"
|
||||
provides: "Storage service unit tests with mocked S3Client"
|
||||
- path: "docker-compose.dev.yml"
|
||||
provides: "MinIO service for local development"
|
||||
contains: "minio"
|
||||
- path: "docker-compose.yml"
|
||||
provides: "MinIO service for production"
|
||||
contains: "minio"
|
||||
key_links:
|
||||
- from: "src/server/services/storage.service.ts"
|
||||
to: "@aws-sdk/client-s3"
|
||||
via: "S3Client with forcePathStyle"
|
||||
pattern: "forcePathStyle.*true"
|
||||
- from: "docker-compose.dev.yml"
|
||||
to: "minio-init"
|
||||
via: "mc init container creates bucket"
|
||||
pattern: "mc mb.*gearbox-images"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Create the S3 storage service abstraction and Docker Compose MinIO infrastructure.
|
||||
|
||||
Purpose: Establish the foundation that all subsequent image storage refactoring depends on. The storage service wraps @aws-sdk/client-s3 and the Docker config ensures MinIO is available for development and production.
|
||||
Output: storage.service.ts with uploadImage/deleteImage/getImageUrl, unit tests, MinIO in both Docker Compose files.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@$HOME/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/PROJECT.md
|
||||
@.planning/ROADMAP.md
|
||||
@.planning/STATE.md
|
||||
@.planning/phases/17-object-storage/17-CONTEXT.md
|
||||
@.planning/phases/17-object-storage/17-RESEARCH.md
|
||||
|
||||
@src/server/services/image.service.ts
|
||||
@docker-compose.yml
|
||||
@docker-compose.dev.yml
|
||||
@.env.example
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Install S3 SDK and create storage service</name>
|
||||
<files>src/server/services/storage.service.ts, tests/services/storage.service.test.ts</files>
|
||||
<read_first>
|
||||
- src/server/services/image.service.ts (current local file storage pattern)
|
||||
- .planning/phases/17-object-storage/17-RESEARCH.md (Pattern 1: S3 Client Singleton, Pattern 2: Presigned URL Injection)
|
||||
</read_first>
|
||||
<action>
|
||||
1. Install dependencies: `bun add @aws-sdk/client-s3 @aws-sdk/s3-request-presigner`
|
||||
|
||||
2. Create `src/server/services/storage.service.ts` per D-03, D-04, D-05:
|
||||
- Create S3Client singleton at module level with `forcePathStyle: true` (REQUIRED for MinIO per research pitfall 4)
|
||||
- Config from env vars: `S3_ENDPOINT`, `S3_ACCESS_KEY`, `S3_SECRET_KEY`, `S3_BUCKET` (default: "gearbox-images"), `S3_REGION` (default: "us-east-1")
|
||||
- `uploadImage(buffer: Buffer | ArrayBuffer, filename: string, contentType: string): Promise<void>` — uses PutObjectCommand
|
||||
- `deleteImage(filename: string): Promise<void>` — uses DeleteObjectCommand
|
||||
- `getImageUrl(filename: string): Promise<string>` — uses GetObjectCommand + getSignedUrl with configurable expiry (default 1h per D-04, configurable via `S3_PRESIGN_EXPIRY` env var)
|
||||
- Export a helper: `async function withImageUrl<T extends { imageFilename: string | null }>(record: T): Promise<T & { imageUrl: string | null }>` — returns null imageUrl when imageFilename is null, presigned URL otherwise (per D-09)
|
||||
- Export a batch helper: `async function withImageUrls<T extends { imageFilename: string | null }>(records: T[]): Promise<(T & { imageUrl: string | null })[]>` — uses Promise.all for parallelism per research pitfall 5
|
||||
|
||||
3. Create `tests/services/storage.service.test.ts`:
|
||||
- Mock @aws-sdk/client-s3 S3Client.send method using `mock.module` from bun:test
|
||||
- Mock @aws-sdk/s3-request-presigner getSignedUrl
|
||||
- Test uploadImage calls PutObjectCommand with correct Bucket, Key, Body, ContentType
|
||||
- Test deleteImage calls DeleteObjectCommand with correct Bucket, Key
|
||||
- Test getImageUrl calls getSignedUrl and returns the result
|
||||
- Test withImageUrl returns null imageUrl when imageFilename is null
|
||||
- Test withImageUrl returns presigned URL when imageFilename is present
|
||||
- Test withImageUrls processes arrays correctly
|
||||
</action>
|
||||
<verify>
|
||||
<automated>bun test tests/services/storage.service.test.ts</automated>
|
||||
</verify>
|
||||
<acceptance_criteria>
|
||||
- grep -q "uploadImage" src/server/services/storage.service.ts
|
||||
- grep -q "deleteImage" src/server/services/storage.service.ts
|
||||
- grep -q "getImageUrl" src/server/services/storage.service.ts
|
||||
- grep -q "withImageUrl" src/server/services/storage.service.ts
|
||||
- grep -q "withImageUrls" src/server/services/storage.service.ts
|
||||
- grep -q "forcePathStyle.*true" src/server/services/storage.service.ts
|
||||
- grep -q "S3_ENDPOINT" src/server/services/storage.service.ts
|
||||
- grep -q "PutObjectCommand" src/server/services/storage.service.ts
|
||||
- grep -q "getSignedUrl" src/server/services/storage.service.ts
|
||||
- bun test tests/services/storage.service.test.ts passes
|
||||
</acceptance_criteria>
|
||||
<done>Storage service exports uploadImage, deleteImage, getImageUrl, withImageUrl, withImageUrls. All unit tests pass with mocked S3 client.</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 2: Add MinIO to Docker Compose and update env config</name>
|
||||
<files>docker-compose.yml, docker-compose.dev.yml, .env.example</files>
|
||||
<read_first>
|
||||
- docker-compose.yml (current production compose)
|
||||
- docker-compose.dev.yml (current dev compose)
|
||||
- .env.example (current env vars)
|
||||
- .planning/phases/17-object-storage/17-RESEARCH.md (Pattern 3: Docker Compose Init Container)
|
||||
</read_first>
|
||||
<action>
|
||||
1. Update `docker-compose.dev.yml` per D-13, D-14:
|
||||
- Add `minio` service using `quay.io/minio/minio:RELEASE.2025-09-07T16-13-09Z` (pinned per research — do NOT use latest or Docker Hub)
|
||||
- Command: `server /data --console-address ":9001"`
|
||||
- Environment: `MINIO_ROOT_USER: minioadmin`, `MINIO_ROOT_PASSWORD: minioadmin` (fixed creds for dev per D-14)
|
||||
- Ports: 9000:9000 (API), 9001:9001 (console)
|
||||
- Volume: `minio-data-dev:/data`
|
||||
- Healthcheck: `["CMD", "mc", "ready", "local"]` interval 5s, timeout 3s, retries 5
|
||||
- Add `minio-init` service using `quay.io/minio/mc:latest`
|
||||
- depends_on minio with condition: service_healthy
|
||||
- entrypoint shell script: set alias, create bucket `gearbox-images` with --ignore-existing, exit 0
|
||||
- Add `minio-data-dev` to volumes section
|
||||
- Add S3 env vars to app service (if app service exists in dev compose) — if not, these will be in the shell env
|
||||
- Add comment noting MinIO GitHub repo archived Feb 2026, S3 API abstraction makes provider swappable
|
||||
|
||||
2. Update `docker-compose.yml` (production) per D-13, D-14:
|
||||
- Add `minio` service same image, same healthcheck
|
||||
- Environment: `MINIO_ROOT_USER: ${S3_ACCESS_KEY}`, `MINIO_ROOT_PASSWORD: ${S3_SECRET_KEY}` (env vars for prod per D-14)
|
||||
- Ports: 9000:9000 only (no console in prod)
|
||||
- Volume: `minio-data:/data`
|
||||
- Add `minio-init` same pattern but using `${S3_ACCESS_KEY:-minioadmin}` and `${S3_SECRET_KEY:-minioadmin}` for mc alias
|
||||
- Add S3 env vars to app service: S3_ENDPOINT=http://minio:9000, S3_ACCESS_KEY, S3_SECRET_KEY, S3_BUCKET=gearbox-images
|
||||
- Remove `uploads:/app/uploads` volume from app service (per D-08 — no more local file serving)
|
||||
- Remove `uploads` from volumes section
|
||||
- Add `minio-data` to volumes section
|
||||
- app service depends_on should include minio (service_healthy)
|
||||
|
||||
3. Update `.env.example`:
|
||||
- Add S3 section with: S3_ENDPOINT, S3_ACCESS_KEY, S3_SECRET_KEY, S3_BUCKET, S3_REGION
|
||||
- Include defaults as comments (endpoint: http://localhost:9000, bucket: gearbox-images, region: us-east-1)
|
||||
</action>
|
||||
<verify>
|
||||
<automated>grep -q "minio" docker-compose.dev.yml && grep -q "minio" docker-compose.yml && grep -q "S3_ENDPOINT" .env.example && echo "PASS"</automated>
|
||||
</verify>
|
||||
<acceptance_criteria>
|
||||
- grep -q "quay.io/minio/minio:RELEASE.2025-09-07" docker-compose.dev.yml
|
||||
- grep -q "minio-init" docker-compose.dev.yml
|
||||
- grep -q "gearbox-images" docker-compose.dev.yml
|
||||
- grep -q "quay.io/minio/minio:RELEASE.2025-09-07" docker-compose.yml
|
||||
- grep -q "minio-init" docker-compose.yml
|
||||
- grep -q "S3_ENDPOINT" docker-compose.yml
|
||||
- grep -q "S3_ACCESS_KEY" .env.example
|
||||
- grep -q "S3_SECRET_KEY" .env.example
|
||||
- grep -q "S3_BUCKET" .env.example
|
||||
</acceptance_criteria>
|
||||
<done>Both Docker Compose files include MinIO with automatic bucket creation. Production uses env vars, dev uses fixed credentials. .env.example documents all S3 variables.</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<verification>
|
||||
- `bun test tests/services/storage.service.test.ts` passes
|
||||
- `grep -r "forcePathStyle" src/server/services/storage.service.ts` confirms MinIO compatibility
|
||||
- `docker compose -f docker-compose.dev.yml config` validates dev compose syntax
|
||||
- `docker compose config` validates prod compose syntax
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
- Storage service exists with all 5 exported functions (uploadImage, deleteImage, getImageUrl, withImageUrl, withImageUrls)
|
||||
- Unit tests pass with mocked S3 client
|
||||
- Both Docker Compose files include MinIO + init container
|
||||
- .env.example documents S3 configuration
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
After completion, create `.planning/phases/17-object-storage/17-01-SUMMARY.md`
|
||||
</output>
|
||||
231
.planning/phases/17-object-storage/17-02-PLAN.md
Normal file
231
.planning/phases/17-object-storage/17-02-PLAN.md
Normal file
@@ -0,0 +1,231 @@
|
||||
---
|
||||
phase: 17-object-storage
|
||||
plan: 02
|
||||
type: execute
|
||||
wave: 2
|
||||
depends_on: ["17-01"]
|
||||
files_modified:
|
||||
- src/server/services/image.service.ts
|
||||
- src/server/routes/images.ts
|
||||
- src/server/routes/items.ts
|
||||
- src/server/routes/threads.ts
|
||||
- src/server/routes/setups.ts
|
||||
- src/server/index.ts
|
||||
- src/server/mcp/tools/items.ts
|
||||
- src/server/mcp/tools/threads.ts
|
||||
- src/server/mcp/tools/images.ts
|
||||
- tests/services/image.service.test.ts
|
||||
- tests/routes/images.test.ts
|
||||
autonomous: true
|
||||
requirements: [IMG-01, IMG-03]
|
||||
|
||||
must_haves:
|
||||
truths:
|
||||
- "Image upload via POST /api/images stores file in MinIO, not local filesystem"
|
||||
- "Image upload via POST /api/images/from-url stores fetched image in MinIO"
|
||||
- "Deleting an item or candidate with an image deletes the image from MinIO"
|
||||
- "API responses include imageUrl field with presigned URLs for items and candidates"
|
||||
- "Static file serving for /uploads/* is removed from the server"
|
||||
- "MCP tools use storage service for image operations"
|
||||
artifacts:
|
||||
- path: "src/server/services/image.service.ts"
|
||||
provides: "URL-based image fetch using storage service"
|
||||
- path: "src/server/routes/images.ts"
|
||||
provides: "Image upload routes using storage service"
|
||||
- path: "src/server/index.ts"
|
||||
provides: "Server entry without /uploads/* static serving"
|
||||
key_links:
|
||||
- from: "src/server/routes/images.ts"
|
||||
to: "src/server/services/storage.service.ts"
|
||||
via: "uploadImage() call"
|
||||
pattern: "import.*uploadImage.*storage"
|
||||
- from: "src/server/routes/items.ts"
|
||||
to: "src/server/services/storage.service.ts"
|
||||
via: "deleteImage() for cleanup, withImageUrl/withImageUrls for responses"
|
||||
pattern: "import.*deleteImage.*storage"
|
||||
- from: "src/server/routes/threads.ts"
|
||||
to: "src/server/services/storage.service.ts"
|
||||
via: "deleteImage() and withImageUrl for candidate images"
|
||||
pattern: "import.*storage"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Refactor all server-side image handling to use the S3 storage service instead of local filesystem.
|
||||
|
||||
Purpose: Replace every Bun.write, unlink, and /uploads/ reference on the server with storage service calls. Enrich API responses with presigned URLs so clients can fetch images directly from MinIO.
|
||||
Output: All server image operations go through storage.service.ts. API responses include imageUrl field. Static /uploads/* serving removed.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@$HOME/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/PROJECT.md
|
||||
@.planning/ROADMAP.md
|
||||
@.planning/phases/17-object-storage/17-CONTEXT.md
|
||||
@.planning/phases/17-object-storage/17-RESEARCH.md
|
||||
@.planning/phases/17-object-storage/17-01-SUMMARY.md
|
||||
|
||||
@src/server/services/image.service.ts
|
||||
@src/server/routes/images.ts
|
||||
@src/server/routes/items.ts
|
||||
@src/server/routes/threads.ts
|
||||
@src/server/index.ts
|
||||
@src/server/mcp/tools/images.ts
|
||||
@src/server/mcp/tools/items.ts
|
||||
@src/server/mcp/tools/threads.ts
|
||||
|
||||
<interfaces>
|
||||
<!-- From Plan 01 - storage.service.ts exports -->
|
||||
From src/server/services/storage.service.ts:
|
||||
```typescript
|
||||
export async function uploadImage(buffer: Buffer | ArrayBuffer, filename: string, contentType: string): Promise<void>;
|
||||
export async function deleteImage(filename: string): Promise<void>;
|
||||
export async function getImageUrl(filename: string): Promise<string>;
|
||||
export async function withImageUrl<T extends { imageFilename: string | null }>(record: T): Promise<T & { imageUrl: string | null }>;
|
||||
export async function withImageUrls<T extends { imageFilename: string | null }>(records: T[]): Promise<(T & { imageUrl: string | null })[]>;
|
||||
```
|
||||
</interfaces>
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Refactor image service and image routes to use storage service</name>
|
||||
<files>src/server/services/image.service.ts, src/server/routes/images.ts, tests/services/image.service.test.ts, tests/routes/images.test.ts</files>
|
||||
<read_first>
|
||||
- src/server/services/storage.service.ts (created in Plan 01 — the storage API)
|
||||
- src/server/services/image.service.ts (current local fs logic to replace)
|
||||
- src/server/routes/images.ts (current upload routes)
|
||||
- tests/services/image.service.test.ts (existing tests to update)
|
||||
- tests/routes/images.test.ts (existing tests to update)
|
||||
</read_first>
|
||||
<action>
|
||||
1. Refactor `src/server/services/image.service.ts` per D-07:
|
||||
- Remove `mkdir` and `Bun.write` imports
|
||||
- Remove `uploadsDir` parameter from `fetchImageFromUrl`
|
||||
- Import `uploadImage` from `./storage.service`
|
||||
- After fetching and validating the image buffer, call `await uploadImage(Buffer.from(buffer), filename, contentType)` instead of `Bun.write`
|
||||
- Keep ALL validation logic unchanged (URL parsing, protocol check, content type, size limits, timeout)
|
||||
- Keep UUID filename generation unchanged per D-12
|
||||
|
||||
2. Refactor `src/server/routes/images.ts` per D-06:
|
||||
- Remove `mkdir`, `join`, `Bun.write` usage
|
||||
- Import `uploadImage` from `../services/storage.service`
|
||||
- In POST `/` handler: after validation, call `await uploadImage(Buffer.from(buffer), filename, file.type)` instead of mkdir + Bun.write
|
||||
- In POST `/from-url` handler: no changes needed (delegates to image.service.ts which is already refactored)
|
||||
- Keep content type validation and size validation unchanged
|
||||
- Keep filename generation pattern unchanged
|
||||
|
||||
3. Update `tests/services/image.service.test.ts`:
|
||||
- Mock the storage.service module using `mock.module` so `uploadImage` is a mock function
|
||||
- Update assertions: verify uploadImage was called with correct buffer, filename, contentType instead of checking Bun.write
|
||||
- Remove any assertions about local filesystem writes
|
||||
|
||||
4. Update `tests/routes/images.test.ts`:
|
||||
- Mock storage.service module
|
||||
- Update assertions to verify uploadImage calls instead of filesystem writes
|
||||
- Test that POST /api/images returns { filename } with 201 status
|
||||
- Test that POST /api/images/from-url returns { filename, sourceUrl } with 201 status
|
||||
</action>
|
||||
<verify>
|
||||
<automated>bun test tests/services/image.service.test.ts tests/routes/images.test.ts</automated>
|
||||
</verify>
|
||||
<acceptance_criteria>
|
||||
- grep -q "import.*uploadImage.*storage" src/server/services/image.service.ts
|
||||
- grep -qv "Bun.write" src/server/services/image.service.ts
|
||||
- grep -qv "mkdir" src/server/services/image.service.ts
|
||||
- grep -q "import.*uploadImage.*storage" src/server/routes/images.ts
|
||||
- grep -qv "Bun.write" src/server/routes/images.ts
|
||||
- grep -qv "mkdir" src/server/routes/images.ts
|
||||
- bun test tests/services/image.service.test.ts passes
|
||||
- bun test tests/routes/images.test.ts passes
|
||||
</acceptance_criteria>
|
||||
<done>Image service and routes use storage service for uploads. No local filesystem writes remain. Tests pass with mocked storage.</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 2: Refactor item/thread/setup routes, remove static serving, update MCP tools</name>
|
||||
<files>src/server/routes/items.ts, src/server/routes/threads.ts, src/server/routes/setups.ts, src/server/index.ts, src/server/mcp/tools/items.ts, src/server/mcp/tools/threads.ts, src/server/mcp/tools/images.ts</files>
|
||||
<read_first>
|
||||
- src/server/services/storage.service.ts (withImageUrl, withImageUrls, deleteImage APIs)
|
||||
- src/server/routes/items.ts (unlink usage on delete, response patterns)
|
||||
- src/server/routes/threads.ts (unlink usage on delete, response patterns)
|
||||
- src/server/routes/setups.ts (response patterns for setup items with images)
|
||||
- src/server/index.ts (serveStatic for /uploads/*)
|
||||
- src/server/mcp/tools/items.ts (MCP tool response patterns)
|
||||
- src/server/mcp/tools/threads.ts (MCP tool response patterns)
|
||||
- src/server/mcp/tools/images.ts (fetchImageFromUrl usage)
|
||||
</read_first>
|
||||
<action>
|
||||
1. Refactor `src/server/routes/items.ts` per D-08, D-09:
|
||||
- Remove `unlink` and `join` imports related to uploads
|
||||
- Import `deleteImage`, `withImageUrl`, `withImageUrls` from `../services/storage.service`
|
||||
- On item delete: replace `unlink(join("uploads", deleted.imageFilename))` with `await deleteImage(deleted.imageFilename)`
|
||||
- On GET single item: wrap response with `withImageUrl()` before returning
|
||||
- On GET list items: wrap response array with `withImageUrls()` before returning
|
||||
- Keep try/catch around deleteImage (missing object is not an error, same as current pattern)
|
||||
|
||||
2. Refactor `src/server/routes/threads.ts` per D-08, D-09:
|
||||
- Remove `unlink` and `join` imports related to uploads
|
||||
- Import `deleteImage`, `withImageUrl`, `withImageUrls` from `../services/storage.service`
|
||||
- On thread delete (where candidate images are cleaned up): replace `unlink(join("uploads", filename))` with `await deleteImage(filename)` in the loop
|
||||
- On candidate delete: replace `unlink(join("uploads", deleted.imageFilename))` with `await deleteImage(deleted.imageFilename)`
|
||||
- On GET thread with candidates: enrich candidate records with `withImageUrls()` before returning
|
||||
- On GET thread list: if threads include image data, enrich accordingly
|
||||
|
||||
3. Refactor `src/server/routes/setups.ts` per D-09:
|
||||
- Import `withImageUrls` from `../services/storage.service`
|
||||
- On GET setup detail (which includes items with imageFilename): enrich the items array with `withImageUrls()` before returning
|
||||
- On GET setup list: if list includes items with images, enrich accordingly
|
||||
|
||||
4. Update `src/server/index.ts` per D-08:
|
||||
- Remove the line `app.use("/uploads/*", serveStatic({ root: "./" }))` entirely
|
||||
- Remove `serveStatic` import if no longer used elsewhere (check — it IS still used for production SPA serving)
|
||||
- Actually: `serveStatic` is still used for SPA serving in production. Only remove the `/uploads/*` line.
|
||||
|
||||
5. Update MCP tools per D-09:
|
||||
- `src/server/mcp/tools/items.ts`: After getting items from service, enrich with `withImageUrl`/`withImageUrls` before returning in tool response
|
||||
- `src/server/mcp/tools/threads.ts`: After getting thread with candidates, enrich candidate images with `withImageUrls`
|
||||
- `src/server/mcp/tools/images.ts`: No changes needed if it calls fetchImageFromUrl (already refactored in Task 1). Verify it does not reference local filesystem directly.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>grep -rn "unlink.*uploads\|Bun\.write.*uploads\|/uploads/" src/server/ | grep -v node_modules | grep -v "\.test\." && echo "FAIL: still has uploads references" || echo "PASS: no uploads references in server"</automated>
|
||||
</verify>
|
||||
<acceptance_criteria>
|
||||
- grep -qv "unlink.*uploads" src/server/routes/items.ts
|
||||
- grep -q "deleteImage" src/server/routes/items.ts
|
||||
- grep -q "withImageUrl" src/server/routes/items.ts
|
||||
- grep -qv "unlink.*uploads" src/server/routes/threads.ts
|
||||
- grep -q "deleteImage" src/server/routes/threads.ts
|
||||
- grep -qv 'uploads/\*.*serveStatic' src/server/index.ts
|
||||
- grep -q "withImageUrl" src/server/mcp/tools/items.ts
|
||||
- No remaining references to "unlink.*uploads" or "/uploads/" in src/server/ (excluding test files)
|
||||
</acceptance_criteria>
|
||||
<done>All server routes use storage service for image deletion and URL generation. Static /uploads/* serving removed. MCP tools return presigned URLs. Zero local filesystem image references remain in server code.</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<verification>
|
||||
- `grep -rn "/uploads/" src/server/ | grep -v node_modules` returns NO matches (all server references removed)
|
||||
- `grep -rn "Bun.write" src/server/ | grep -v node_modules` returns NO image-related matches
|
||||
- `grep -rn "unlink.*uploads" src/server/` returns NO matches
|
||||
- `bun test tests/services/image.service.test.ts tests/routes/images.test.ts` passes
|
||||
- `bun run lint` passes (no unused imports from removed code)
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
- All image uploads go through storage.service.ts (no Bun.write to uploads/)
|
||||
- All image deletions go through storage.service.ts (no unlink of uploads/)
|
||||
- All API responses with images include imageUrl presigned URL field
|
||||
- Static /uploads/* serving removed from server
|
||||
- MCP tools return presigned URLs
|
||||
- All existing image-related tests pass with mocked storage
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
After completion, create `.planning/phases/17-object-storage/17-02-SUMMARY.md`
|
||||
</output>
|
||||
221
.planning/phases/17-object-storage/17-03-PLAN.md
Normal file
221
.planning/phases/17-object-storage/17-03-PLAN.md
Normal file
@@ -0,0 +1,221 @@
|
||||
---
|
||||
phase: 17-object-storage
|
||||
plan: 03
|
||||
type: execute
|
||||
wave: 2
|
||||
depends_on: ["17-01"]
|
||||
files_modified:
|
||||
- src/client/components/ImageUpload.tsx
|
||||
- src/client/components/ItemCard.tsx
|
||||
- src/client/components/CandidateCard.tsx
|
||||
- src/client/components/CandidateListItem.tsx
|
||||
- src/client/components/ComparisonTable.tsx
|
||||
- src/client/routes/setups/$setupId.tsx
|
||||
- scripts/migrate-images-to-minio.ts
|
||||
autonomous: true
|
||||
requirements: [IMG-02, IMG-03]
|
||||
|
||||
must_haves:
|
||||
truths:
|
||||
- "Client components display images using presigned URLs from API responses, not /uploads/ paths"
|
||||
- "Migration script uploads all files from uploads/ directory to MinIO bucket"
|
||||
- "Migration script preserves original filenames as MinIO object keys"
|
||||
- "No /uploads/ path references remain in client code"
|
||||
artifacts:
|
||||
- path: "scripts/migrate-images-to-minio.ts"
|
||||
provides: "One-time migration of local images to MinIO"
|
||||
contains: "uploadImage"
|
||||
- path: "src/client/components/ItemCard.tsx"
|
||||
provides: "Item display using imageUrl from API"
|
||||
- path: "src/client/components/CandidateCard.tsx"
|
||||
provides: "Candidate display using imageUrl from API"
|
||||
key_links:
|
||||
- from: "src/client/components/ItemCard.tsx"
|
||||
to: "API response"
|
||||
via: "imageUrl prop instead of /uploads/ path"
|
||||
pattern: "imageUrl"
|
||||
- from: "scripts/migrate-images-to-minio.ts"
|
||||
to: "src/server/services/storage.service.ts"
|
||||
via: "uploadImage() for each file"
|
||||
pattern: "uploadImage"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Update all client components to use presigned URLs and create the image migration script.
|
||||
|
||||
Purpose: Complete the client-side transition from /uploads/ paths to presigned URLs, and provide a one-time migration script for existing images. After this plan, the full stack uses MinIO for image storage.
|
||||
Output: All client image references use imageUrl from API. Migration script ready to run.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@$HOME/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/PROJECT.md
|
||||
@.planning/ROADMAP.md
|
||||
@.planning/phases/17-object-storage/17-CONTEXT.md
|
||||
@.planning/phases/17-object-storage/17-RESEARCH.md
|
||||
@.planning/phases/17-object-storage/17-01-SUMMARY.md
|
||||
|
||||
@src/client/components/ImageUpload.tsx
|
||||
@src/client/components/ItemCard.tsx
|
||||
@src/client/components/CandidateCard.tsx
|
||||
@src/client/components/CandidateListItem.tsx
|
||||
@src/client/components/ComparisonTable.tsx
|
||||
@src/client/routes/setups/$setupId.tsx
|
||||
|
||||
<interfaces>
|
||||
<!-- From Plan 01 - storage.service.ts exports used by migration script -->
|
||||
From src/server/services/storage.service.ts:
|
||||
```typescript
|
||||
export async function uploadImage(buffer: Buffer | ArrayBuffer, filename: string, contentType: string): Promise<void>;
|
||||
```
|
||||
|
||||
<!-- API responses will now include imageUrl alongside imageFilename (from Plan 02) -->
|
||||
<!-- Components receive props like: { imageFilename: string | null, imageUrl: string | null } -->
|
||||
</interfaces>
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Update client components to use imageUrl from API responses</name>
|
||||
<files>src/client/components/ImageUpload.tsx, src/client/components/ItemCard.tsx, src/client/components/CandidateCard.tsx, src/client/components/CandidateListItem.tsx, src/client/components/ComparisonTable.tsx, src/client/routes/setups/$setupId.tsx</files>
|
||||
<read_first>
|
||||
- src/client/components/ImageUpload.tsx (current /uploads/ usage)
|
||||
- src/client/components/ItemCard.tsx (current /uploads/ usage)
|
||||
- src/client/components/CandidateCard.tsx (current /uploads/ usage)
|
||||
- src/client/components/CandidateListItem.tsx (current /uploads/ usage)
|
||||
- src/client/components/ComparisonTable.tsx (current /uploads/ usage)
|
||||
- src/client/routes/setups/$setupId.tsx (current imageFilename usage)
|
||||
</read_first>
|
||||
<action>
|
||||
Per D-10: Client components use the presigned URL directly from API responses.
|
||||
|
||||
The API (refactored in Plan 02) now returns `imageUrl: string | null` alongside `imageFilename: string | null` on every item/candidate record. Components should use `imageUrl` for display.
|
||||
|
||||
1. `src/client/components/ItemCard.tsx`:
|
||||
- Update props interface: add `imageUrl: string | null` alongside existing `imageFilename`
|
||||
- Replace `src={/uploads/${imageFilename}}` with `src={imageUrl}` (which is already a full URL)
|
||||
- Guard: render image only when `imageUrl` is truthy (not when `imageFilename` is truthy)
|
||||
|
||||
2. `src/client/components/CandidateCard.tsx`:
|
||||
- Update props interface: add `imageUrl: string | null`
|
||||
- Replace `src={/uploads/${imageFilename}}` with `src={imageUrl}`
|
||||
- Guard on `imageUrl` instead of `imageFilename`
|
||||
|
||||
3. `src/client/components/CandidateListItem.tsx`:
|
||||
- Props already receive full candidate object. The candidate object now has `imageUrl`.
|
||||
- Replace `src={/uploads/${candidate.imageFilename}}` with `src={candidate.imageUrl}`
|
||||
- Guard on `candidate.imageUrl` instead of `candidate.imageFilename`
|
||||
|
||||
4. `src/client/components/ComparisonTable.tsx`:
|
||||
- Update candidate type in props: add `imageUrl: string | null`
|
||||
- Replace `src={/uploads/${c.imageFilename}}` with `src={c.imageUrl}`
|
||||
- Guard on `c.imageUrl`
|
||||
|
||||
5. `src/client/components/ImageUpload.tsx`:
|
||||
- This shows a preview of the uploaded image. Currently uses `/uploads/${value}` where `value` is the imageFilename.
|
||||
- After upload, the API returns `{ filename }`. The preview needs a URL.
|
||||
- Two approaches: (a) accept an `imageUrl` prop for existing images, or (b) construct a temporary preview from the File object.
|
||||
- RECOMMENDED: Accept both `value` (imageFilename) and `imageUrl` (presigned URL) as props. For NEW uploads, use `URL.createObjectURL(file)` for instant preview. For EXISTING images (editing), use the `imageUrl` prop passed from the parent.
|
||||
- Update: add `imageUrl?: string | null` prop. For preview display: if imageUrl is provided use it, else if a local file was just selected use createObjectURL.
|
||||
- Parents (ItemForm, CandidateForm) will pass `imageUrl` from the record data.
|
||||
|
||||
6. `src/client/routes/setups/$setupId.tsx`:
|
||||
- This renders setup items with images. The setup API response now includes `imageUrl` on each item.
|
||||
- Replace any `imageFilename={item.imageFilename}` prop passing with `imageUrl={item.imageUrl}` (and keep imageFilename for form data if needed).
|
||||
|
||||
7. Update parent form components that pass imageFilename to ImageUpload:
|
||||
- `src/client/components/ItemForm.tsx`: Pass `imageUrl` from item record to ImageUpload component
|
||||
- `src/client/components/CandidateForm.tsx`: Pass `imageUrl` from candidate record to ImageUpload component
|
||||
|
||||
IMPORTANT: Do NOT break the upload flow. When a new image is uploaded via POST /api/images, the response still returns `{ filename }`. The imageUrl will be available on subsequent GET requests. For immediate preview after upload, use a local object URL or re-fetch.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>grep -rn "/uploads/" src/client/ | grep -v node_modules && echo "FAIL: still has /uploads/ references" || echo "PASS: no /uploads/ references in client"</automated>
|
||||
</verify>
|
||||
<acceptance_criteria>
|
||||
- grep -rn "/uploads/" src/client/ returns NO matches
|
||||
- grep -q "imageUrl" src/client/components/ItemCard.tsx
|
||||
- grep -q "imageUrl" src/client/components/CandidateCard.tsx
|
||||
- grep -q "imageUrl" src/client/components/CandidateListItem.tsx
|
||||
- grep -q "imageUrl" src/client/components/ComparisonTable.tsx
|
||||
- grep -q "imageUrl" src/client/components/ImageUpload.tsx
|
||||
</acceptance_criteria>
|
||||
<done>All 6 client components and the setup route use imageUrl from API responses. Zero /uploads/ path references remain in client code.</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 2: Create image migration script</name>
|
||||
<files>scripts/migrate-images-to-minio.ts</files>
|
||||
<read_first>
|
||||
- src/server/services/storage.service.ts (uploadImage API from Plan 01)
|
||||
- .planning/phases/17-object-storage/17-RESEARCH.md (migration section, D-11, D-12)
|
||||
</read_first>
|
||||
<action>
|
||||
Per D-11, D-12: Create `scripts/migrate-images-to-minio.ts` — a one-time migration script.
|
||||
|
||||
1. Create the script:
|
||||
- Import `uploadImage` from `../src/server/services/storage.service`
|
||||
- Import `readdir` and `readFile` from `node:fs/promises`
|
||||
- Import `join`, `extname` from `node:path`
|
||||
- Define UPLOADS_DIR = "uploads"
|
||||
- Content type mapping: `.jpg`/`.jpeg` -> `image/jpeg`, `.png` -> `image/png`, `.webp` -> `image/webp`
|
||||
|
||||
2. Main function:
|
||||
- Check if uploads/ directory exists. If not, log "No uploads directory found. Nothing to migrate." and exit 0.
|
||||
- Read all files from uploads/ directory (readdir)
|
||||
- Filter to only image files (.jpg, .jpeg, .png, .webp)
|
||||
- Log: "Found {N} images to migrate"
|
||||
- For each file:
|
||||
a. Read file contents with `Bun.file(path).arrayBuffer()`
|
||||
b. Determine content type from extension
|
||||
c. Call `await uploadImage(Buffer.from(buffer), filename, contentType)` — filename is just the basename, per D-12 (no path changes)
|
||||
d. Log success: "Migrated: {filename}"
|
||||
e. On error: log error and continue (don't abort entire migration for one failure)
|
||||
- Track success/failure counts
|
||||
- Log summary: "Migration complete: {success}/{total} files migrated, {failed} failures"
|
||||
- If any failures, log: "Re-run this script to retry failed uploads"
|
||||
- Do NOT delete original files per discretion recommendation. Log: "Original files preserved in uploads/. Delete manually after verifying: rm -rf uploads/"
|
||||
|
||||
3. Run with: `bun run scripts/migrate-images-to-minio.ts`
|
||||
- Script should be self-contained, executable directly with bun
|
||||
- Requires S3 env vars to be set (S3_ENDPOINT, S3_ACCESS_KEY, S3_SECRET_KEY)
|
||||
</action>
|
||||
<verify>
|
||||
<automated>test -f scripts/migrate-images-to-minio.ts && grep -q "uploadImage" scripts/migrate-images-to-minio.ts && grep -q "readdir" scripts/migrate-images-to-minio.ts && echo "PASS"</automated>
|
||||
</verify>
|
||||
<acceptance_criteria>
|
||||
- test -f scripts/migrate-images-to-minio.ts
|
||||
- grep -q "uploadImage" scripts/migrate-images-to-minio.ts
|
||||
- grep -q "readdir" scripts/migrate-images-to-minio.ts
|
||||
- grep -q "image/jpeg" scripts/migrate-images-to-minio.ts
|
||||
- grep -q "image/png" scripts/migrate-images-to-minio.ts
|
||||
- grep -q "uploads" scripts/migrate-images-to-minio.ts
|
||||
- Script does NOT contain "unlink" or "rm" calls (per discretion: do not auto-delete)
|
||||
</acceptance_criteria>
|
||||
<done>Migration script reads all images from uploads/, uploads each to MinIO preserving filenames, logs progress, does not delete originals.</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<verification>
|
||||
- `grep -rn "/uploads/" src/client/` returns NO matches
|
||||
- `grep -rn "/uploads/" src/server/` returns NO matches (verified in Plan 02)
|
||||
- `scripts/migrate-images-to-minio.ts` exists and contains uploadImage, readdir
|
||||
- `bun run lint` passes on all modified files
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
- All client components use imageUrl from API responses (zero /uploads/ references)
|
||||
- Migration script exists and handles: directory check, file reading, S3 upload, error handling, progress logging
|
||||
- Migration script preserves original filenames as object keys (per D-12)
|
||||
- Migration script does not auto-delete originals
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
After completion, create `.planning/phases/17-object-storage/17-03-SUMMARY.md`
|
||||
</output>
|
||||
Reference in New Issue
Block a user