diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index 60f849b..844e18a 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -25,11 +25,11 @@ Requirements for Public Discovery milestone. Each maps to roadmap phases. ### Catalog Enrichment -- [ ] **CATL-01**: Global items have attribution fields (sourceUrl, manufacturer, imageCredit, imageSourceUrl) -- [ ] **CATL-02**: Global items have a unique constraint on (brand, model) preventing duplicates +- [x] **CATL-01**: Global items have attribution fields (sourceUrl, manufacturer, imageCredit, imageSourceUrl) +- [x] **CATL-02**: Global items have a unique constraint on (brand, model) preventing duplicates - [ ] **CATL-03**: Catalog detail pages display image attribution with credit and source link - [ ] **CATL-04**: Bulk import API endpoint accepts multiple catalog items in one request -- [ ] **CATL-05**: Bulk import uses upsert semantics (ON CONFLICT update, not fail) +- [x] **CATL-05**: Bulk import uses upsert semantics (ON CONFLICT update, not fail) ### Agent Seeding Tools @@ -122,11 +122,11 @@ Which phases cover which requirements. Updated during roadmap creation. | PUBL-04 | Phase 24 | Complete | | PUBL-05 | Phase 24 | Complete | | INFR-01 | Phase 24 | Complete | -| CATL-01 | Phase 25 | Pending | -| CATL-02 | Phase 25 | Pending | +| CATL-01 | Phase 25 | Complete | +| CATL-02 | Phase 25 | Complete | | CATL-03 | Phase 25 | Pending | | CATL-04 | Phase 25 | Pending | -| CATL-05 | Phase 25 | Pending | +| CATL-05 | Phase 25 | Complete | | SEED-01 | Phase 25 | Pending | | SEED-02 | Phase 25 | Pending | | SEED-03 | Phase 25 | Pending | diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 1d71665..52adea7 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -105,7 +105,7 @@ Plans: **Plans**: 2 plans Plans: -- [ ] 25-01-PLAN.md — Schema migration (attribution columns + unique constraint) and upsert service layer +- [x] 25-01-PLAN.md — Schema migration (attribution columns + unique constraint) and upsert service layer - [ ] 25-02-PLAN.md — HTTP upsert routes, MCP catalog tools, and client attribution display ### Phase 26: Discovery Landing Page @@ -149,7 +149,7 @@ Plans: | 22. Add-from-Catalog & Thread Integration | v2.0 | 2/2 | Complete | 2026-04-06 | | 23. Manual Entry Fallback | v2.0 | 1/1 | Complete | 2026-04-06 | | 24. Public Access & Infrastructure | v2.1 | 2/2 | Complete | 2026-04-10 | -| 25. Catalog Enrichment & Agent Tools | v2.1 | 0/2 | Not started | - | +| 25. Catalog Enrichment & Agent Tools | v2.1 | 1/2 | In Progress| | | 26. Discovery Landing Page | v2.1 | 0/TBD | Not started | - | ## Backlog diff --git a/.planning/STATE.md b/.planning/STATE.md index 60efd95..ef12e4e 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -2,15 +2,15 @@ gsd_state_version: 1.0 milestone: v2.1 milestone_name: Public Discovery -status: verifying -stopped_at: Phase 25 context gathered -last_updated: "2026-04-10T08:33:11.968Z" +status: executing +stopped_at: Completed 25-01-PLAN.md +last_updated: "2026-04-10T08:59:46.905Z" last_activity: 2026-04-10 progress: total_phases: 6 completed_phases: 1 - total_plans: 2 - completed_plans: 2 + total_plans: 4 + completed_plans: 3 percent: 0 --- @@ -21,13 +21,13 @@ progress: See: .planning/PROJECT.md (updated 2026-04-09) **Core value:** Help people make better gear decisions — discover what others use, compare real-world data, and see how a potential buy affects your setup before committing. -**Current focus:** Phase 24 — public-access-infrastructure +**Current focus:** Phase 25 — catalog-enrichment-agent-tools ## Current Position -Phase: 999.1 -Plan: Not started -Status: Phase complete — ready for verification +Phase: 25 (catalog-enrichment-agent-tools) — EXECUTING +Plan: 2 of 2 +Status: Ready to execute Last activity: 2026-04-10 Progress: [░░░░░░░░░░] 0% @@ -64,6 +64,8 @@ v2.1 decisions: - [Phase 24-public-access-infrastructure]: Browse tier 120/min, detail tier 60/min — same limits for auth and anon users - [Phase 24]: Both auth prompt CTAs go to /login — Logto handles sign-in and sign-up at the same OIDC endpoint - [Phase 24]: Soft navigate() replaces hard window.location.href for private route redirect — defers until auth resolves +- [Phase 25-catalog-enrichment-agent-tools]: Three-way tag sync: undefined=leave untouched, []=clear all, [names]=replace — enables selective tag updates from catalog agents +- [Phase 25-catalog-enrichment-agent-tools]: unique(brand, model) constraint on globalItems: enables safe ON CONFLICT DO UPDATE for catalog enrichment agents ### Pending Todos @@ -75,6 +77,6 @@ None. ## Session Continuity -Last session: 2026-04-10T08:33:11.966Z -Stopped at: Phase 25 context gathered -Resume file: .planning/phases/25-catalog-enrichment-agent-tools/25-CONTEXT.md +Last session: 2026-04-10T08:59:46.903Z +Stopped at: Completed 25-01-PLAN.md +Resume file: None diff --git a/.planning/phases/25-catalog-enrichment-agent-tools/25-01-SUMMARY.md b/.planning/phases/25-catalog-enrichment-agent-tools/25-01-SUMMARY.md new file mode 100644 index 0000000..618b0cc --- /dev/null +++ b/.planning/phases/25-catalog-enrichment-agent-tools/25-01-SUMMARY.md @@ -0,0 +1,132 @@ +--- +phase: 25-catalog-enrichment-agent-tools +plan: 01 +subsystem: database +tags: [drizzle, postgres, zod, catalog, upsert, attribution] + +# Dependency graph +requires: [] +provides: + - globalItems table with sourceUrl, imageCredit, imageSourceUrl attribution columns + - unique constraint on (brand, model) in globalItems table + - migration 0003_loving_serpent_society.sql + - upsertGlobalItemSchema and bulkUpsertGlobalItemsSchema Zod schemas + - UpsertGlobalItemInput and BulkUpsertGlobalItemsInput TypeScript types + - upsertGlobalItem service function with tag sync + - bulkUpsertGlobalItems service function with transaction atomicity +affects: + - 25-02 (HTTP routes and MCP tools will call these service functions) + +# Tech tracking +tech-stack: + added: [] + patterns: + - onConflictDoUpdate with multi-column target for brand+model upsert + - syncGlobalItemTags helper using delete-then-insert in transaction + - tag create-if-not-exists via onConflictDoUpdate on tags.name + +key-files: + created: + - drizzle-pg/0003_loving_serpent_society.sql + modified: + - src/db/schema.ts + - src/shared/schemas.ts + - src/shared/types.ts + - src/server/services/global-item.service.ts + - tests/services/global-item.service.test.ts + +key-decisions: + - "Unique constraint on (brand, model): enables safe ON CONFLICT DO UPDATE for catalog enrichment" + - "Tags sync: undefined=leave untouched, []=clear all, [names]=replace — three-way tag handling" + - "Migration 0003 fixed: drizzle-kit generated spurious duplicate DDL; trimmed to only new changes" + +patterns-established: + - "upsertGlobalItem pattern: check existence before upsert to track created vs updated" + - "syncGlobalItemTags: delete existing links, then create-if-not-exists tags and insert links" + +requirements-completed: [CATL-01, CATL-02, CATL-05] + +# Metrics +duration: 3min +completed: 2026-04-10 +--- + +# Phase 25 Plan 01: Catalog Enrichment Data Layer Summary + +**globalItems attribution columns (sourceUrl, imageCredit, imageSourceUrl) with unique(brand, model) constraint, upsertGlobalItem/bulkUpsertGlobalItems service functions, and Zod schemas — 21 tests passing** + +## Performance + +- **Duration:** ~3 min +- **Started:** 2026-04-10T08:55:26Z +- **Completed:** 2026-04-10T08:58:39Z +- **Tasks:** 2 +- **Files modified:** 5 + +## Accomplishments + +- Added three attribution columns to globalItems table with unique(brand, model) constraint and generated migration +- Implemented upsertGlobalItem with onConflictDoUpdate, three-way tag sync, and created/updated tracking +- Implemented bulkUpsertGlobalItems processing arrays in a single atomic transaction +- Defined upsertGlobalItemSchema and bulkUpsertGlobalItemsSchema Zod validation schemas +- All 21 tests pass (13 pre-existing + 8 new upsert operation tests) + +## Task Commits + +Each task was committed atomically: + +1. **Task 1: Schema migration — attribution columns + unique constraint** - `39ef9cc` (feat) +2. **Task 2: TDD RED — failing tests** - `9093a2c` (test) +3. **Task 2: TDD GREEN — Zod schemas, service functions, tests passing** - `c8ebbf8` (feat) + +_Note: TDD tasks have multiple commits (test RED → feat GREEN)_ + +## Files Created/Modified + +- `src/db/schema.ts` - Added sourceUrl, imageCredit, imageSourceUrl columns and unique().on(brand, model) constraint to globalItems +- `drizzle-pg/0003_loving_serpent_society.sql` - Migration adding 3 columns + unique constraint +- `src/shared/schemas.ts` - Added upsertGlobalItemSchema and bulkUpsertGlobalItemsSchema +- `src/shared/types.ts` - Added UpsertGlobalItemInput and BulkUpsertGlobalItemsInput types +- `src/server/services/global-item.service.ts` - Added upsertGlobalItem, bulkUpsertGlobalItems, syncGlobalItemTags +- `tests/services/global-item.service.test.ts` - Added 8 upsert operation tests + +## Decisions Made + +- **Three-way tag handling**: `undefined` leaves existing tags untouched, `[]` clears all tags, `[names]` replaces tags. This allows callers to selectively update tags without clobbering existing data. +- **Unique constraint on (brand, model)**: Required for ON CONFLICT DO UPDATE semantics. Without it, duplicate inserts would fail rather than update. +- **Created/updated tracking via pre-check**: The service checks for existing row before upsert to accurately report created vs updated counts, since ON CONFLICT DO UPDATE doesn't distinguish via returning rows alone. + +## Deviations from Plan + +### Auto-fixed Issues + +**1. [Rule 1 - Bug] Fixed drizzle-kit generated spurious duplicate DDL in migration 0003** +- **Found during:** Task 1 (schema migration) +- **Issue:** drizzle-kit generated a migration that re-created global_item_tags, re-added FKs on items and thread_candidates, and re-added oauth_codes.user_id — all already present in migration 0002. PGlite tests failed with "relation already exists". +- **Fix:** Trimmed migration 0003 to only include the three new ALTER TABLE ADD COLUMN statements and the unique constraint. +- **Files modified:** drizzle-pg/0003_loving_serpent_society.sql +- **Verification:** 21 tests pass including all new upsert tests +- **Committed in:** c8ebbf8 (Task 2 feat commit) + +--- + +**Total deviations:** 1 auto-fixed (Rule 1 - bug in generated migration) +**Impact on plan:** Fix was necessary for test correctness. No scope creep. + +## Issues Encountered + +- drizzle-kit migration generation included duplicate DDL from prior migrations — likely a state tracking issue in the drizzle-kit snapshots. Fixed by manually editing the migration to contain only the new changes. + +## User Setup Required + +None - no external service configuration required. Production database push (`bun run db:push`) will apply the migration when the database is available. + +## Next Phase Readiness + +- Data layer complete: globalItems has attribution columns, unique constraint, and upsert service functions +- Plan 02 (HTTP routes + MCP tools) can now import upsertGlobalItem and bulkUpsertGlobalItems directly +- No blockers + +--- +*Phase: 25-catalog-enrichment-agent-tools* +*Completed: 2026-04-10*