makiolaj/agendula
1
0
Fork 0
Code Issues Pull Requests Actions Releases 7 Activity

docs(sync): bring SYNC.md in line with owning the store

Browse Source 
The document was written against the vendored provider and still said the
storage question was settled that way. Owning the store answered several
of its open questions and deleted others outright, so the corrections are
marked inline the way the rest of the file marks them, rather than
quietly rewritten.

Closed: the Local->Synced migration (account_id is a nullable FK, so
attaching an account is an UPDATE), the recurring-completion model (the
store writes model (a), RECURRENCE-ID overrides sharing the master's
UID), the Auto Backup / cleanUpLists data-loss path, and the lib-recur
version trap. Phase 0's UIDs-at-creation and backup safety are shipped.
The provider-mechanism table is kept as External-mode history rather than
deleted — that code still runs in OpenTasks and tasks.org.

Effort restated: 11.5-15 weeks minus the 2.5-4 owning the store removes,
so roughly 8-11. Also states plainly at the top that no sync code exists
and that ICalendarWriter is the export half of the mapper only.
This commit is contained in:
Jean-Luc Makiola
2026-09-04 15:53:01 +02:00
parent ec2e2eb59d
commit fdbc236ab4
1 changed files with 94 additions and 36 deletions
Download Patch File Download Diff File Expand all files Collapse all files

130
docs/SYNC.md
View File

@@ -11,18 +11,39 @@
> That document decided **where task data lives**; this one decides **how it gets
> to a server**.
>
> Status: **draft / decision document.** Nothing here is built. Where a question
> is already answered by shipped code, it is marked ✅ and the code is named.
> Status: **draft / decision document.** No sync code is built — no `dav4jvm`,
> no `ical4j`, no `AccountManager`, no adapter, nothing in the manifest. Where a
> question is already answered by shipped code, it is marked ✅ and the code is
> named.
>
> ⚠️ **The storage question this document declared settled was reopened, and the
> answer changed.** Agendula is building its own Room store and deleting the
> vendored provider — see [`STORAGE-DECISION.md`](STORAGE-DECISION.md) and
> ⚠️ **The storage question this document declared settled was reopened, the
> answer changed, and the change has since shipped.** Agendula owns a Room store
> and the vendored provider is deleted — see
> [`STORAGE-DECISION.md`](STORAGE-DECISION.md) and
> [`OWN-STORE.md`](OWN-STORE.md). Roughly sixteen of the findings below are
> **provider-imposed** and disappear with it; `OWN-STORE.md` § *Effects on the
> sync plan* lists them item by item. Everything platform-level (the targetSdk 34
> sync gate, the stub adapter, credential storage, Play compliance) and everything
> protocol-level (discovery, RFC 6578, conditional PUT, conflict policy, the
> server-reality table) is unaffected and remains the plan.
> **provider-imposed** and went with it; `OWN-STORE.md` § *Effects on the sync
> plan* lists them item by item. They are kept here, marked, because the External
> path still runs on that provider and because the next person would otherwise
> re-derive them. Everything platform-level (the targetSdk 34 sync gate, the stub
> adapter, credential storage, Play compliance) and everything protocol-level
> (discovery, RFC 6578, conditional PUT, conflict policy, the server-reality
> table) is unaffected and remains the plan.
>
> **What owning the store already settled, in shipped code** (2026-09):
>
> | Was a sync deliverable | Now |
> |---|---|
> | Phase 0 — UIDs at creation | ✅ `uid` is `NOT NULL`, minted on insert in both modes |
> | Phase 0 — backup / prune safety | ✅ backup rules cover the WAL, `ON_STOP` checkpoint, restore test |
> | Phase 1 — the recurrence representation | ✅ `RRULE`/`RDATE`/`EXDATE` stored raw, expanded at read (`RecurrenceExpander`), `RECURRENCE-ID` overrides sharing the master's UID |
> | Phase 3 — Local→Synced migration | ✅ **gone**: `task_lists.account_id` is a nullable FK, so attaching an account is one `UPDATE` |
> | Phase 3 — recurring-completion model | ✅ model **(a)** is what the store writes; the provider's model (d) left with it |
> | Sync bookkeeping columns | ✅ `href`, `etag`, `sync_token`, `is_dirty`, `is_deleted` exist in the v1 schema |
>
> Still nothing but design: phase 1's mapper (`ICalendarWriter` writes VTODO for
> **export only** — no parser, no unknown-property round-trip), phase 2 auth,
> phase 3's engine, phase 4 hardening. Phase 0's licence-attribution screen is
> also not built.
Scope: self-hosted CalDAV first (Nextcloud), F-Droid and Play, MIT license.
@@ -32,31 +53,34 @@ Scope: self-hosted CalDAV first (Nextcloud), F-Droid and Play, MIT license.
| Question | Direction |
|---|---|
| Where does task data live? | Our vendored `:provider` — **settled, shipped** |
| Where does task data live? | ⚠️ **Changed since this table was written.** Our own Room store — `:provider` is deleted (`OWN-STORE.md`) |
| Who syncs it? | Agendula, via its own sync adapter |
| Account model | `AccountManager` **+ a real (stub) sync adapter** — ⚠️ the hybrid without one does not work |
| Scheduling | WorkManager, triggered *through* the sync framework |
| Protocol library | `dav4jvm` (MPL-2.0) — ⚠️ costs more than the first draft assumed |
| Self-signed certs | `cert4android` — ⚠️ **MPL-2.0, not GPLv3.** The first draft rejected it on a false premise |
| iCalendar | In-house mapper over `ical4j`; **no `synctools`** (GPLv3) |
| iCalendar | In-house mapper over `ical4j`; **no `synctools`** (GPLv3). ⚠️ Now VTODO ↔ **Room entities**, not `TaskContract` |
| Recurrence | Read/write `RRULE`/`RDATE`/`EXDATE` directly — ⚠️ **not** the Instances table |
| Primary read path | ⚠️ `REPORT calendar-query` (VTODO filter, no time-range); `sync-collection` is the optimisation |
| Recurring completion | ⚠️ Accept all four models; `:provider` has **already chosen model (d)** for us |
| Recurring completion | ⚠️ Accept all four models on read; **we write (a)** — the store forks a `RECURRENCE-ID` override sharing the master's UID. The provider's model (d) left with the provider |
| Sign-in | Nextcloud Login Flow v2 + generic CalDAV discovery + Digest |
| Conflict policy | `If-Match`; on 412 the server wins, local copy preserved |
| Reference implementations | jtx Board and DAVx5 — read, never link against (GPLv3) |
| # | Phase | Deliverable | Effort |
|---|---|---|---|
| 0 | **Groundwork** | UIDs at creation, backup/prune safety, licence-attribution screen, Java-21 decision | 1 week |
| 1 | Mapper | VTODO ↔ `TaskContract`, unknown-property round-trip, fixture corpus | 2–3 weeks |
| 0 | **Groundwork** | ~~UIDs at creation~~ ✅, ~~backup/prune safety~~ ✅, licence-attribution screen ⬜, Java-21 decision ⬜ | ~1 week → days |
| 1 | Mapper | VTODO ↔ **Room entities**, unknown-property round-trip, fixture corpus. ⚠️ `ICalendarWriter` is the export half only — write-only, and it drops what it does not model | 2–3 weeks |
| 2 | Auth | Discovery, Login Flow v2, Digest, credential storage, cert trust | 1.5–2 weeks |
| 3 | Engine | `calendar-query` baseline, `sync-collection` optimisation, full reconciliation, conflicts, scheduling, **Local→Synced migration** | 5–6 weeks |
| 3 | Engine | `calendar-query` baseline, `sync-collection` optimisation, full reconciliation, conflicts, scheduling, ~~**Local→Synced migration**~~ ✅ gone — `account_id` is a nullable FK | 4–5 weeks |
| 4 | Hardening | Per-server trap matrix, error UX, re-auth, Play compliance | 2–3 weeks |
⚠️ **Revised upward from the first draft's 8–9 weeks to 11.5–15.** Phase 0 is new;
the migration in phase 3 was previously believed not to exist at all; and the
engine grew a second sync path plus a permanent reconciliation pass.
⚠️ **Revised upward from the first draft's 8–9 weeks to 11.5–15**, then back
down. Phase 0 is new; the migration in phase 3 was previously believed not to
exist at all; and the engine grew a second sync path plus a permanent
reconciliation pass. Owning the store then deleted 2.5–4 weeks of it
(`OWN-STORE.md` § *Effects on the sync plan*) — the migration, the recurrence
representation and phase 0's data work are done — leaving roughly **8–11 weeks**.
**Calibration, for sanity:** Evolution shipped RFC 6578 in **June 2026** against
a request open since 2019. vdirsyncer has declined to implement it for twelve
@@ -98,8 +122,15 @@ machinery we already run), but "for free" was too generous.
## What the provider actually gives us
⚠️ **This section is almost entirely rewritten.** Every line is verified against
`provider/src/main/java/`.
⚠️ **This section is now history for the sync plan.** Every line was verified
against `provider/src/main/java/`, which no longer exists — the vendored provider
is deleted and our own store is what sync will run against. Nothing here
constrains the adapter any more.
It is kept, not deleted, for two reasons: **External mode still talks to exactly
this code** in OpenTasks and tasks.org, so the app layer still lives with these
rules; and if External mode is ever retired (open question 3), this is the record
of what was being given up.
| Mechanism | Reality |
|---|---|
@@ -148,11 +179,17 @@ machinery we already run), but "for free" was too generous.
---
## ⚠️ The migration that was believed not to exist
## ⚠️ The migration that was believed not to exist — and then stopped existing
**Resolved: the original assertion is true again, because the constraint that
broke it was the provider's.** Our own `task_lists.account_id` is a nullable FK
from v1, so attaching an account to a list is one `UPDATE` and no task moves.
Phase 3 does not carry this deliverable. The rest of this section is the record
of why it was believed to, and still describes External mode exactly.
The first draft, `STORAGE-AND-SYNC.md` and `ARCHITECTURE.md` all asserted:
*"Synced is Local with an account attached, so switching on sync is not a
migration."* **That is false.**
migration."* **That was false against the dmfs provider.**
`processors/lists/Validating.java:68-76` throws on any attempt to change a task
list's `ACCOUNT_NAME` or `ACCOUNT_TYPE` — both are write-once, and the contract
@@ -233,6 +270,12 @@ nothing registered to receive it.
### ⚠️ Auto Backup will arm `cleanUpLists` into a data-loss path
✅ **Closed.** Both rule sets are now explicit and name our own database with its
WAL sidecars, the app checkpoints on `ON_STOP`, and a restore test covers the WAL
case in both directions (`OWN-STORE.md` phase 6). `cleanUpLists` was the
provider's, and left with it. The original finding, which still describes what an
External-mode user's provider app does:
`backup_rules.xml` and `data_extraction_rules.xml` are both **empty rule sets**,
and `allowBackup="true"`. An empty set means Auto Backup's default: databases
included. So the provider's `tasks.db` is backed up and restored — while
@@ -420,8 +463,12 @@ phase-4 detail.
- Consider tasks.org's escape hatch: a per-account **"let the server schedule
recurring tasks"** switch.
Open question 4 — but now with a default: **write (a) if we own the whole path;
accept that `:provider`'s `Detaching` pushes us toward (d) unless we bypass it.**
~~Open question 4~~ — **decided and shipped: we write (a).** We do own the whole
path now, so the `Detaching` caveat is moot. `RoomTasksDataSource`'s
`setCompletedInstance` and `updateInstance` both fork a `RECURRENCE-ID` override
sharing the master's UID, and the master stays open — jtx Board's and
Thunderbird's model, and the one that maps onto CalDAV without invention. The
adapter must still **read** all four models, which is unchanged.
> **Process note.** During this research a summarising fetch **fabricated a
> verbatim RFC 5545 sentence** ("A 'to-do' calendar component without the
@@ -471,6 +518,12 @@ This roughly dissolves the self-signed-cert line item in phase 4.
### ⚠️ lib-recur is a version trap, not a free dependency
✅ **Resolved by deleting the other side of the trap.** `:app` declares lib-recur
0.12.2 directly and `RecurrenceExpander` uses it; the vendored provider whose
iterators would have stopped compiling no longer exists, so the version is ours
alone to move. The `RecurrenceSet` removal in 0.16.0 is now a plain upgrade
question, not a build-breaking one. The original finding:
The first draft said "already in the build at 0.12.2 — no new dependency". Both
halves are wrong. `provider/build.gradle.kts:56` declares it `implementation`, not
`api`, so it is **not** on `:app`'s compile classpath. And lib-recur **0.16.0
@@ -1143,20 +1196,20 @@ everything about storage modes.
target and freezes the API churn — it looks better than it did. Before phase 2.
2. **Conflict policy** — preserve-local-on-412, or documented LWW?
3. **External mode** — survives, or becomes an importer? Before phase 1.
4. **Canonical recurring-completion behaviour** — and specifically, ⚠️ **do we
honour `:provider`'s `Detaching` processor (model d), or bypass it and write
`RECURRENCE-ID` overrides (model a)?** No longer an open-ended taste question:
our storage layer already answered it and we have to ratify or override that.
Moved up to **phase 1**.
4. ~~**Canonical recurring-completion behaviour**~~ **closed:** the store writes
model (a), `RECURRENCE-ID` overrides sharing the master's UID. Read all four.
5. **The DAVx5 enum ask** — worth filing, and what compatibility we owe if it
lands.
6. ⚠️ **New: does Local→Synced migrate, or do synced lists start empty?** See
[the migration section](#-the-migration-that-was-believed-not-to-exist).
7. ⚠️ **New: lib-recur — pin at 0.12.2, or rewrite the provider's iterators?**
lands. ⚠️ Reshaped: it now means "sync into an app that publishes no provider".
6. ~~**Does Local→Synced migrate, or do synced lists start empty?**~~ **closed:**
neither — attaching an account to a list is an `UPDATE`, so there is nothing
to migrate.
7. ~~**lib-recur — pin at 0.12.2, or rewrite the provider's iterators?**~~
**closed** with the provider's deletion; `:app` owns the version.
Answered elsewhere and **not** open: the account model (`AccountManager` **plus a
stub sync adapter**), `ical4android` (superseded by `synctools`, GPLv3), and the
storage question.
storage question — which was reopened once, answered the other way, and is now
shipped.
---
@@ -1165,7 +1218,12 @@ storage question.
- **Depending on DAVx5 for sync.** Settled in `STORAGE-AND-SYNC.md`.
- **`synctools` / `ical4android`.** GPLv3. The temptation recurs because it does
exactly the right mapping against exactly our schema.
- **Rewriting storage to Room before sync exists.** [See above](#settled--the-storage-question-is-not-reopened-here).
- ~~**Rewriting storage to Room before sync exists.**~~ ⚠️ **This one was
revisited, and it was right to.** The phase-1 audit measured the provider's
sync bookkeeping — the reason it was kept — and found most of it broken, absent
or unusable (the table above). Reasoning in
[`STORAGE-DECISION.md`](STORAGE-DECISION.md). Kept here as a reminder that a
dead end is only dead against the evidence that closed it.
- ⚠️ **AccountManager + WorkManager with no registered sync adapter.** Not a
design choice — a silent no-op at targetSdk ≥ 34.
- ⚠️ **Writing through the `instances` URI as a sync adapter.** The flag is