diff --git a/docs/SYNC.md b/docs/SYNC.md index 4938dd8..3a1c685 100644 --- a/docs/SYNC.md +++ b/docs/SYNC.md @@ -11,18 +11,39 @@ > That document decided **where task data lives**; this one decides **how it gets > to a server**. > -> Status: **draft / decision document.** Nothing here is built. Where a question -> is already answered by shipped code, it is marked ✅ and the code is named. +> Status: **draft / decision document.** No sync code is built — no `dav4jvm`, +> no `ical4j`, no `AccountManager`, no adapter, nothing in the manifest. Where a +> question is already answered by shipped code, it is marked ✅ and the code is +> named. > -> ⚠️ **The storage question this document declared settled was reopened, and the -> answer changed.** Agendula is building its own Room store and deleting the -> vendored provider — see [`STORAGE-DECISION.md`](STORAGE-DECISION.md) and +> ⚠️ **The storage question this document declared settled was reopened, the +> answer changed, and the change has since shipped.** Agendula owns a Room store +> and the vendored provider is deleted — see +> [`STORAGE-DECISION.md`](STORAGE-DECISION.md) and > [`OWN-STORE.md`](OWN-STORE.md). Roughly sixteen of the findings below are -> **provider-imposed** and disappear with it; `OWN-STORE.md` § *Effects on the -> sync plan* lists them item by item. Everything platform-level (the targetSdk 34 -> sync gate, the stub adapter, credential storage, Play compliance) and everything -> protocol-level (discovery, RFC 6578, conditional PUT, conflict policy, the -> server-reality table) is unaffected and remains the plan. +> **provider-imposed** and went with it; `OWN-STORE.md` § *Effects on the sync +> plan* lists them item by item. They are kept here, marked, because the External +> path still runs on that provider and because the next person would otherwise +> re-derive them. Everything platform-level (the targetSdk 34 sync gate, the stub +> adapter, credential storage, Play compliance) and everything protocol-level +> (discovery, RFC 6578, conditional PUT, conflict policy, the server-reality +> table) is unaffected and remains the plan. +> +> **What owning the store already settled, in shipped code** (2026-09): +> +> | Was a sync deliverable | Now | +> |---|---| +> | Phase 0 — UIDs at creation | ✅ `uid` is `NOT NULL`, minted on insert in both modes | +> | Phase 0 — backup / prune safety | ✅ backup rules cover the WAL, `ON_STOP` checkpoint, restore test | +> | Phase 1 — the recurrence representation | ✅ `RRULE`/`RDATE`/`EXDATE` stored raw, expanded at read (`RecurrenceExpander`), `RECURRENCE-ID` overrides sharing the master's UID | +> | Phase 3 — Local→Synced migration | ✅ **gone**: `task_lists.account_id` is a nullable FK, so attaching an account is one `UPDATE` | +> | Phase 3 — recurring-completion model | ✅ model **(a)** is what the store writes; the provider's model (d) left with it | +> | Sync bookkeeping columns | ✅ `href`, `etag`, `sync_token`, `is_dirty`, `is_deleted` exist in the v1 schema | +> +> Still nothing but design: phase 1's mapper (`ICalendarWriter` writes VTODO for +> **export only** — no parser, no unknown-property round-trip), phase 2 auth, +> phase 3's engine, phase 4 hardening. Phase 0's licence-attribution screen is +> also not built. Scope: self-hosted CalDAV first (Nextcloud), F-Droid and Play, MIT license. @@ -32,31 +53,34 @@ Scope: self-hosted CalDAV first (Nextcloud), F-Droid and Play, MIT license. | Question | Direction | |---|---| -| Where does task data live? | Our vendored `:provider` — **settled, shipped** | +| Where does task data live? | ⚠️ **Changed since this table was written.** Our own Room store — `:provider` is deleted (`OWN-STORE.md`) | | Who syncs it? | Agendula, via its own sync adapter | | Account model | `AccountManager` **+ a real (stub) sync adapter** — ⚠️ the hybrid without one does not work | | Scheduling | WorkManager, triggered *through* the sync framework | | Protocol library | `dav4jvm` (MPL-2.0) — ⚠️ costs more than the first draft assumed | | Self-signed certs | `cert4android` — ⚠️ **MPL-2.0, not GPLv3.** The first draft rejected it on a false premise | -| iCalendar | In-house mapper over `ical4j`; **no `synctools`** (GPLv3) | +| iCalendar | In-house mapper over `ical4j`; **no `synctools`** (GPLv3). ⚠️ Now VTODO ↔ **Room entities**, not `TaskContract` | | Recurrence | Read/write `RRULE`/`RDATE`/`EXDATE` directly — ⚠️ **not** the Instances table | | Primary read path | ⚠️ `REPORT calendar-query` (VTODO filter, no time-range); `sync-collection` is the optimisation | -| Recurring completion | ⚠️ Accept all four models; `:provider` has **already chosen model (d)** for us | +| Recurring completion | ⚠️ Accept all four models on read; **we write (a)** — the store forks a `RECURRENCE-ID` override sharing the master's UID. The provider's model (d) left with the provider | | Sign-in | Nextcloud Login Flow v2 + generic CalDAV discovery + Digest | | Conflict policy | `If-Match`; on 412 the server wins, local copy preserved | | Reference implementations | jtx Board and DAVx5 — read, never link against (GPLv3) | | # | Phase | Deliverable | Effort | |---|---|---|---| -| 0 | **Groundwork** | UIDs at creation, backup/prune safety, licence-attribution screen, Java-21 decision | 1 week | -| 1 | Mapper | VTODO ↔ `TaskContract`, unknown-property round-trip, fixture corpus | 2–3 weeks | +| 0 | **Groundwork** | ~~UIDs at creation~~ ✅, ~~backup/prune safety~~ ✅, licence-attribution screen ⬜, Java-21 decision ⬜ | ~1 week → days | +| 1 | Mapper | VTODO ↔ **Room entities**, unknown-property round-trip, fixture corpus. ⚠️ `ICalendarWriter` is the export half only — write-only, and it drops what it does not model | 2–3 weeks | | 2 | Auth | Discovery, Login Flow v2, Digest, credential storage, cert trust | 1.5–2 weeks | -| 3 | Engine | `calendar-query` baseline, `sync-collection` optimisation, full reconciliation, conflicts, scheduling, **Local→Synced migration** | 5–6 weeks | +| 3 | Engine | `calendar-query` baseline, `sync-collection` optimisation, full reconciliation, conflicts, scheduling, ~~**Local→Synced migration**~~ ✅ gone — `account_id` is a nullable FK | 4–5 weeks | | 4 | Hardening | Per-server trap matrix, error UX, re-auth, Play compliance | 2–3 weeks | -⚠️ **Revised upward from the first draft's 8–9 weeks to 11.5–15.** Phase 0 is new; -the migration in phase 3 was previously believed not to exist at all; and the -engine grew a second sync path plus a permanent reconciliation pass. +⚠️ **Revised upward from the first draft's 8–9 weeks to 11.5–15**, then back +down. Phase 0 is new; the migration in phase 3 was previously believed not to +exist at all; and the engine grew a second sync path plus a permanent +reconciliation pass. Owning the store then deleted 2.5–4 weeks of it +(`OWN-STORE.md` § *Effects on the sync plan*) — the migration, the recurrence +representation and phase 0's data work are done — leaving roughly **8–11 weeks**. **Calibration, for sanity:** Evolution shipped RFC 6578 in **June 2026** against a request open since 2019. vdirsyncer has declined to implement it for twelve @@ -98,8 +122,15 @@ machinery we already run), but "for free" was too generous. ## What the provider actually gives us -⚠️ **This section is almost entirely rewritten.** Every line is verified against -`provider/src/main/java/`. +⚠️ **This section is now history for the sync plan.** Every line was verified +against `provider/src/main/java/`, which no longer exists — the vendored provider +is deleted and our own store is what sync will run against. Nothing here +constrains the adapter any more. + +It is kept, not deleted, for two reasons: **External mode still talks to exactly +this code** in OpenTasks and tasks.org, so the app layer still lives with these +rules; and if External mode is ever retired (open question 3), this is the record +of what was being given up. | Mechanism | Reality | |---|---| @@ -148,11 +179,17 @@ machinery we already run), but "for free" was too generous. --- -## ⚠️ The migration that was believed not to exist +## ⚠️ The migration that was believed not to exist — and then stopped existing + +**Resolved: the original assertion is true again, because the constraint that +broke it was the provider's.** Our own `task_lists.account_id` is a nullable FK +from v1, so attaching an account to a list is one `UPDATE` and no task moves. +Phase 3 does not carry this deliverable. The rest of this section is the record +of why it was believed to, and still describes External mode exactly. The first draft, `STORAGE-AND-SYNC.md` and `ARCHITECTURE.md` all asserted: *"Synced is Local with an account attached, so switching on sync is not a -migration."* **That is false.** +migration."* **That was false against the dmfs provider.** `processors/lists/Validating.java:68-76` throws on any attempt to change a task list's `ACCOUNT_NAME` or `ACCOUNT_TYPE` — both are write-once, and the contract @@ -233,6 +270,12 @@ nothing registered to receive it. ### ⚠️ Auto Backup will arm `cleanUpLists` into a data-loss path +✅ **Closed.** Both rule sets are now explicit and name our own database with its +WAL sidecars, the app checkpoints on `ON_STOP`, and a restore test covers the WAL +case in both directions (`OWN-STORE.md` phase 6). `cleanUpLists` was the +provider's, and left with it. The original finding, which still describes what an +External-mode user's provider app does: + `backup_rules.xml` and `data_extraction_rules.xml` are both **empty rule sets**, and `allowBackup="true"`. An empty set means Auto Backup's default: databases included. So the provider's `tasks.db` is backed up and restored — while @@ -420,8 +463,12 @@ phase-4 detail. - Consider tasks.org's escape hatch: a per-account **"let the server schedule recurring tasks"** switch. -Open question 4 — but now with a default: **write (a) if we own the whole path; -accept that `:provider`'s `Detaching` pushes us toward (d) unless we bypass it.** +~~Open question 4~~ — **decided and shipped: we write (a).** We do own the whole +path now, so the `Detaching` caveat is moot. `RoomTasksDataSource`'s +`setCompletedInstance` and `updateInstance` both fork a `RECURRENCE-ID` override +sharing the master's UID, and the master stays open — jtx Board's and +Thunderbird's model, and the one that maps onto CalDAV without invention. The +adapter must still **read** all four models, which is unchanged. > **Process note.** During this research a summarising fetch **fabricated a > verbatim RFC 5545 sentence** ("A 'to-do' calendar component without the @@ -471,6 +518,12 @@ This roughly dissolves the self-signed-cert line item in phase 4. ### ⚠️ lib-recur is a version trap, not a free dependency +✅ **Resolved by deleting the other side of the trap.** `:app` declares lib-recur +0.12.2 directly and `RecurrenceExpander` uses it; the vendored provider whose +iterators would have stopped compiling no longer exists, so the version is ours +alone to move. The `RecurrenceSet` removal in 0.16.0 is now a plain upgrade +question, not a build-breaking one. The original finding: + The first draft said "already in the build at 0.12.2 — no new dependency". Both halves are wrong. `provider/build.gradle.kts:56` declares it `implementation`, not `api`, so it is **not** on `:app`'s compile classpath. And lib-recur **0.16.0 @@ -1143,20 +1196,20 @@ everything about storage modes. target and freezes the API churn — it looks better than it did. Before phase 2. 2. **Conflict policy** — preserve-local-on-412, or documented LWW? 3. **External mode** — survives, or becomes an importer? Before phase 1. -4. **Canonical recurring-completion behaviour** — and specifically, ⚠️ **do we - honour `:provider`'s `Detaching` processor (model d), or bypass it and write - `RECURRENCE-ID` overrides (model a)?** No longer an open-ended taste question: - our storage layer already answered it and we have to ratify or override that. - Moved up to **phase 1**. +4. ~~**Canonical recurring-completion behaviour**~~ **closed:** the store writes + model (a), `RECURRENCE-ID` overrides sharing the master's UID. Read all four. 5. **The DAVx5 enum ask** — worth filing, and what compatibility we owe if it - lands. -6. ⚠️ **New: does Local→Synced migrate, or do synced lists start empty?** See - [the migration section](#-the-migration-that-was-believed-not-to-exist). -7. ⚠️ **New: lib-recur — pin at 0.12.2, or rewrite the provider's iterators?** + lands. ⚠️ Reshaped: it now means "sync into an app that publishes no provider". +6. ~~**Does Local→Synced migrate, or do synced lists start empty?**~~ **closed:** + neither — attaching an account to a list is an `UPDATE`, so there is nothing + to migrate. +7. ~~**lib-recur — pin at 0.12.2, or rewrite the provider's iterators?**~~ + **closed** with the provider's deletion; `:app` owns the version. Answered elsewhere and **not** open: the account model (`AccountManager` **plus a stub sync adapter**), `ical4android` (superseded by `synctools`, GPLv3), and the -storage question. +storage question — which was reopened once, answered the other way, and is now +shipped. --- @@ -1165,7 +1218,12 @@ storage question. - **Depending on DAVx5 for sync.** Settled in `STORAGE-AND-SYNC.md`. - **`synctools` / `ical4android`.** GPLv3. The temptation recurs because it does exactly the right mapping against exactly our schema. -- **Rewriting storage to Room before sync exists.** [See above](#settled--the-storage-question-is-not-reopened-here). +- ~~**Rewriting storage to Room before sync exists.**~~ ⚠️ **This one was + revisited, and it was right to.** The phase-1 audit measured the provider's + sync bookkeeping — the reason it was kept — and found most of it broken, absent + or unusable (the table above). Reasoning in + [`STORAGE-DECISION.md`](STORAGE-DECISION.md). Kept here as a reminder that a + dead end is only dead against the evidence that closed it. - ⚠️ **AccountManager + WorkManager with no registered sync adapter.** Not a design choice — a silent no-op at targetSdk ≥ 34. - ⚠️ **Writing through the `instances` URI as a sync adapter.** The flag is