---
phase: 25-catalog-enrichment-agent-tools
plan: 01
subsystem: database
tags: [drizzle, postgres, zod, catalog, upsert, attribution]

# Dependency graph
requires: []
provides:
  - globalItems table with sourceUrl, imageCredit, imageSourceUrl attribution columns
  - unique constraint on (brand, model) in globalItems table
  - migration 0003_loving_serpent_society.sql
  - upsertGlobalItemSchema and bulkUpsertGlobalItemsSchema Zod schemas
  - UpsertGlobalItemInput and BulkUpsertGlobalItemsInput TypeScript types
  - upsertGlobalItem service function with tag sync
  - bulkUpsertGlobalItems service function with transaction atomicity
affects:
  - 25-02 (HTTP routes and MCP tools will call these service functions)

# Tech tracking
tech-stack:
  added: []
  patterns:
    - onConflictDoUpdate with multi-column target for brand+model upsert
    - syncGlobalItemTags helper using delete-then-insert in transaction
    - tag create-if-not-exists via onConflictDoUpdate on tags.name

key-files:
  created:
    - drizzle-pg/0003_loving_serpent_society.sql
  modified:
    - src/db/schema.ts
    - src/shared/schemas.ts
    - src/shared/types.ts
    - src/server/services/global-item.service.ts
    - tests/services/global-item.service.test.ts

key-decisions:
  - "Unique constraint on (brand, model): enables safe ON CONFLICT DO UPDATE for catalog enrichment"
  - "Tags sync: undefined=leave untouched, []=clear all, [names]=replace — three-way tag handling"
  - "Migration 0003 fixed: drizzle-kit generated spurious duplicate DDL; trimmed to only new changes"

patterns-established:
  - "upsertGlobalItem pattern: check existence before upsert to track created vs updated"
  - "syncGlobalItemTags: delete existing links, then create-if-not-exists tags and insert links"

requirements-completed: [CATL-01, CATL-02, CATL-05]

# Metrics
duration: 3min
completed: 2026-04-10
---

# Phase 25 Plan 01: Catalog Enrichment Data Layer Summary

**globalItems attribution columns (sourceUrl, imageCredit, imageSourceUrl) with unique(brand, model) constraint, upsertGlobalItem/bulkUpsertGlobalItems service functions, and Zod schemas — 21 tests passing**

## Performance

- **Duration:** ~3 min
- **Started:** 2026-04-10T08:55:26Z
- **Completed:** 2026-04-10T08:58:39Z
- **Tasks:** 2
- **Files modified:** 5

## Accomplishments

- Added three attribution columns to globalItems table with unique(brand, model) constraint and generated migration
- Implemented upsertGlobalItem with onConflictDoUpdate, three-way tag sync, and created/updated tracking
- Implemented bulkUpsertGlobalItems processing arrays in a single atomic transaction
- Defined upsertGlobalItemSchema and bulkUpsertGlobalItemsSchema Zod validation schemas
- All 21 tests pass (13 pre-existing + 8 new upsert operation tests)

## Task Commits

Each task was committed atomically:

1. **Task 1: Schema migration — attribution columns + unique constraint** - `39ef9cc` (feat)
2. **Task 2: TDD RED — failing tests** - `9093a2c` (test)
3. **Task 2: TDD GREEN — Zod schemas, service functions, tests passing** - `c8ebbf8` (feat)

_Note: TDD tasks have multiple commits (test RED → feat GREEN)_

## Files Created/Modified

- `src/db/schema.ts` - Added sourceUrl, imageCredit, imageSourceUrl columns and unique().on(brand, model) constraint to globalItems
- `drizzle-pg/0003_loving_serpent_society.sql` - Migration adding 3 columns + unique constraint
- `src/shared/schemas.ts` - Added upsertGlobalItemSchema and bulkUpsertGlobalItemsSchema
- `src/shared/types.ts` - Added UpsertGlobalItemInput and BulkUpsertGlobalItemsInput types
- `src/server/services/global-item.service.ts` - Added upsertGlobalItem, bulkUpsertGlobalItems, syncGlobalItemTags
- `tests/services/global-item.service.test.ts` - Added 8 upsert operation tests

## Decisions Made

- **Three-way tag handling**: `undefined` leaves existing tags untouched, `[]` clears all tags, `[names]` replaces tags. This allows callers to selectively update tags without clobbering existing data.
- **Unique constraint on (brand, model)**: Required for ON CONFLICT DO UPDATE semantics. Without it, duplicate inserts would fail rather than update.
- **Created/updated tracking via pre-check**: The service checks for existing row before upsert to accurately report created vs updated counts, since ON CONFLICT DO UPDATE doesn't distinguish via returning rows alone.

## Deviations from Plan

### Auto-fixed Issues

**1. [Rule 1 - Bug] Fixed drizzle-kit generated spurious duplicate DDL in migration 0003**
- **Found during:** Task 1 (schema migration)
- **Issue:** drizzle-kit generated a migration that re-created global_item_tags, re-added FKs on items and thread_candidates, and re-added oauth_codes.user_id — all already present in migration 0002. PGlite tests failed with "relation already exists".
- **Fix:** Trimmed migration 0003 to only include the three new ALTER TABLE ADD COLUMN statements and the unique constraint.
- **Files modified:** drizzle-pg/0003_loving_serpent_society.sql
- **Verification:** 21 tests pass including all new upsert tests
- **Committed in:** c8ebbf8 (Task 2 feat commit)

---

**Total deviations:** 1 auto-fixed (Rule 1 - bug in generated migration)
**Impact on plan:** Fix was necessary for test correctness. No scope creep.

## Issues Encountered

- drizzle-kit migration generation included duplicate DDL from prior migrations — likely a state tracking issue in the drizzle-kit snapshots. Fixed by manually editing the migration to contain only the new changes.

## User Setup Required

None - no external service configuration required. Production database push (`bun run db:push`) will apply the migration when the database is available.

## Next Phase Readiness

- Data layer complete: globalItems has attribution columns, unique constraint, and upsert service functions
- Plan 02 (HTTP routes + MCP tools) can now import upsertGlobalItem and bulkUpsertGlobalItems directly
- No blockers

---
*Phase: 25-catalog-enrichment-agent-tools*
*Completed: 2026-04-10*