The parts of chunk 5 that a build can verify. What is left needs a device or a live server, and is listed in docs/SYNC-PLAN.md rather than guessed at. - Attribution screen in Settings. dav4jvm is vendored, which makes MPL-2.0 §3.2(a) ours rather than a dependency's, so its row points at PROVENANCE.md next to upstream. Hand-maintained: generators read POM metadata, which routinely names a non-SPDX licence and a licence URL that 404s. - Revocation both ways. A 401 marks the account, stops it before the next request reaches the network, and takes it off the schedule from outside the worker — Nextcloud throttles then 429s per source IP, so a timer on a dead app password degrades the user's other clients. On removal, a bounded best-effort DELETE of the app password, or uninstalling never revokes it. - Play compliance: docs/PRIVACY.md linked in the app, declaring Collected and not Shared; an option to delete the account's tasks from the device too; REQUEST_IGNORE_BATTERY_OPTIMIZATIONS confirmed absent. - The server trap matrix as far as a protocol mock reaches, with four tests left @Ignore'd and their reasons written out. - docs/SYNC-PLAN.md records what moves to floret-kit, so that branch is a file move rather than a rediscovery. /code-review high raised 9 findings, all fixed. Three were serious: app-password revocation was aimed at the principal URL and revoked nothing; opening the app put accounts a 401 had stopped back on the timer, because KEEP does not keep cancelled work; and the incremental path advanced the sync token past bodies a failed multiget never applied. Also: four scalars were emitted twice whenever their residue copy survived, which the round-trip corpus could not see. Not done, and needing you: the live server matrix, releaseTest on device, the restore-onto-a-fresh-device check, cert4android, and MKCALENDAR feature detection. Chunk 2's on-device review is still outstanding.
908 lines
52 KiB
Markdown
908 lines
52 KiB
Markdown
# Agendula — CalDAV sync: the build plan
|
||
|
||
**Branch:** `feat/caldav-sync`, cut from `release/v1.0.0`.
|
||
**Design:** [`SYNC.md`](SYNC.md) — the decisions, the traps and the audit trail
|
||
behind them. This document does **not** repeat it. It sequences it.
|
||
**Shape:** five chunks, one branch, one PR at the end.
|
||
|
||
`SYNC.md` is a 1,200-line decision document with roughly forty ⚠️ findings
|
||
scattered through it, most of which are "the obvious implementation is wrong,
|
||
here is why". That is the right shape for a design document and the wrong shape
|
||
for a work queue. This is the work queue.
|
||
|
||
---
|
||
|
||
## Decisions taken before the first line of code
|
||
|
||
`SYNC.md` left three questions open and named two of them as blocking. All three
|
||
are now answered, plus one the phases imply. **These are not reopened by the
|
||
implementation chunks** — if a chunk finds evidence that overturns one, it stops
|
||
and says so rather than quietly picking the other branch.
|
||
|
||
| # | Question | Answer | Consequence |
|
||
|---|---|---|---|
|
||
| 1 | dav4jvm + cert4android distribution, and the Java-21 wall | **Vendor a known-good tree**, recompiled at our Java 17 target | `FAIL_ON_PROJECT_REPOS` stays; no Ktor/guava/xpp3 tail; MPL-2.0 headers retained and the attribution screen becomes mandatory, not optional. Freezes the API churn that shipped two breaking majors nineteen days apart |
|
||
| 2 | Conflict policy on 412 | **Server wins, local edit discarded** | DAVx5's policy. Terminates, always. The discarded edit is *surfaced* — a sync report the user can read — because silence is what makes this policy feel like data loss. The draft's "preserve as a duplicate" stays withdrawn: RFC 4791 §4.1 makes it unuploadable |
|
||
| 3 | External mode's future | **Kept, unchanged** | The mapper and the UI stay dual-capable. Retiring it is a separate decision that does not block sync, and it ships and works today |
|
||
| 4 | iCalendar library | **None — a hand-rolled content-line model.** ⚠️ Overturned during chunk 1; `SYNC.md` § *Libraries* assumed a typed library was the only option | A raw `(name, params, value)` content-line tree is what the unknown-property requirement actually wants: a typed model normalises away exactly what must survive. `ICalendarWriter` already folds in 204 lines, `lib-recur` is already a direct `:app` dependency and does `RRULE`, and `java.time` is native at minSdk 29 — so ical4j's 2.2 MB of duplicated zone data, its production-exhausting `ZoneRulesProvider`, its mandatory properties/cache/registry shims and its release-only R8 failure buy nothing. VTIMEZONE bodies are already on the round-trip allowlist, so they round-trip opaquely. **Cost accepted: we own the lexer.** biweekly was rejected on the same ground as before |
|
||
|
||
---
|
||
|
||
## Scope boundaries
|
||
|
||
**Not in this branch, deliberately:**
|
||
|
||
- **Extraction into floret-kit.** `SYNC.md` § *What lands in floret-kit* is right
|
||
that `core-dav` and `core-sync` belong there and that Calendula needs the same
|
||
primitives. Extracting across a submodule boundary while the API is still
|
||
moving costs more than it buys. **Mitigation, which is a real constraint on
|
||
every chunk:** the DAV and iCalendar layers take no task-domain type and no
|
||
Android UI type, so the later extraction is a file move. Chunk 5 records what
|
||
moves.
|
||
- **Retiring External mode**, per decision 3.
|
||
- **A release.** Nothing here bumps `versionCode` / `versionName`. Sync ships
|
||
when it works against real servers, and that is a decision taken after the
|
||
device review, not inside this plan.
|
||
|
||
**In scope but not verifiable by CI:** everything that needs a real server. The
|
||
per-server trap matrix (`SYNC.md` § *Server reality*) is written as named,
|
||
skipped integration tests in chunks 3–4 and turned on by hand against a live
|
||
Nextcloud/Radicale/Baïkal in chunk 5.
|
||
|
||
---
|
||
|
||
## The five chunks
|
||
|
||
| # | Chunk | Rests on | The thing it must not get wrong |
|
||
|---|---|---|---|
|
||
| 1 | **Mapper** — VTODO ↔ Room | nothing | Unknown properties survive a read-modify-write cycle |
|
||
| 2a | **Vendored DAV foundation** | nothing | Vendor a tree we can actually maintain, and know which of its defects are real |
|
||
| 2b | **Discovery and auth protocol** | 2a | The two collection filters are inversions of the obvious rule |
|
||
| 2c | **The Android account layer** | 2b | Registering the sync adapter, or every `ContentResolver` sync call is a silent no-op |
|
||
| 2d | **Account-add UI** | 2c | Never send a credential into an unvalidated redirect chain |
|
||
| 3 | **Engine core** — read, write, conflicts | 1, 2 | A failed resource must not fail the collection |
|
||
| 4 | **Incremental sync + scheduling** | 3 | Never persist a sync token before the bodies it covers are applied |
|
||
| 5 | **Hardening, compliance, real servers** | 1–4 | Ship no licence violation, and no retry loop on a dead app password |
|
||
|
||
Chunks 1 and 2 are genuinely independent of each other; the ordering between
|
||
them is arbitrary and 1 goes first only because it is the one with a
|
||
specification that can be written down before any network exists.
|
||
|
||
---
|
||
|
||
## Chunk 1 — the mapper
|
||
|
||
**Goal:** a bidirectional VTODO ↔ Room-entity mapper whose *specification is its
|
||
fixture corpus*. Today `ICalendarWriter` is the export half only — write-only,
|
||
hand-rolled, and it silently drops everything it does not model.
|
||
|
||
**The corpus comes first.** Not as a testing courtesy: `SYNC.md` § *Unknown
|
||
properties* is explicit that the fixtures that catch bugs are the adversarial
|
||
ones, and a mapper written before its corpus gets a corpus that ratifies it.
|
||
Minimum set, each one a named failure the design predicts:
|
||
|
||
- a `CLASS:CONFIDENTIAL` task as Nextcloud rewrites it out of a **shared**
|
||
calendar — the one that destroys data on write-back
|
||
- a master plus `RECURRENCE-ID` overrides in one resource
|
||
- unknown properties nested **inside a `VALARM`** — the case a flat
|
||
property-per-row model cannot represent
|
||
- an entirely unknown sub-component
|
||
- a `TZID` the device tzdb does not know
|
||
- a UID containing `/` and `@`
|
||
- one exceeding `CALDAV:max-resource-size`
|
||
- `RRULE` + `DUE` with no `DTSTART`, which has no well-defined `RECURRENCE-ID`
|
||
- `X-MOZ-LASTACK` / `X-MOZ-SNOOZE-TIME` (dropping them causes Thunderbird alarm
|
||
storms) and `X-APPLE-SORT-ORDER`
|
||
- all four recurring-completion models on read (a/b/c/d — we write (a))
|
||
|
||
**Round-trip assertion, stated the way `SYNC.md` corrects it.** Byte-stability is
|
||
unachievable and asserting it produces a corpus that gets normalised until it
|
||
tests nothing. Re-parse both sides into a canonical multiset of
|
||
`(component path, property name, params as a sorted map, unfolded unescaped value)`
|
||
and assert equality **modulo an enumerated allowlist**: `PRODID`, `DTSTAMP`,
|
||
`LAST-MODIFIED`, `SEQUENCE`, VTIMEZONE bodies, fold positions, parameter quoting.
|
||
Byte-equality is asserted where it means something — the unfolded, unescaped
|
||
value octets of every untouched property, and full parameter preservation
|
||
including unknown parameters.
|
||
|
||
**Storage for the unknown half.** A new table, and therefore a **schema
|
||
migration to v2** with an exported schema and a `MigrationTestHelper` test, per
|
||
the repo's existing pattern in `app/schemas/`. It must represent nesting
|
||
(`component path`), ordering, and opaque unknown sub-components. Cap the total
|
||
at ~25 kB per resource, as DAVx5 does, for `CursorWindow` and
|
||
`CALDAV:max-resource-size` both.
|
||
|
||
**Deliverables**
|
||
|
||
- fixture corpus under `app/src/test/resources/vtodo/`, with a README naming what
|
||
each fixture is for
|
||
- `VTodoParser` / `VTodoSerialiser` behind one interface, no Android types
|
||
- unknown-property table, Room migration v1→v2, migration test
|
||
- the `SYNC.md` § *The rest of the minefield* rules as individual tests:
|
||
`SEQUENCE` is preserve-verbatim; `DTSTAMP` regenerates but `LAST-MODIFIED`
|
||
does not; `COMPLETED` is UTC-only; `RELATED-TO;RELTYPE=PARENT` points *up*;
|
||
`PRIORITY` 0 = undefined; a `TRIGGER;RELATED=END` alarm without `DUE` is
|
||
invalid and must be caught **before** PUT, because ordinary UI actions reach it
|
||
- `ICalendarWriter` either becomes the serialiser or is deleted in its favour —
|
||
two VTODO writers in one app is the defect this chunk exists to prevent
|
||
- ical4j's `ical4j.properties`, `MapTimeZoneCache` and registry shim, plus its R8
|
||
keep rules (`-keep class net.fortuna.ical4j.** { *; }` + the `-dontwarn` set),
|
||
because that failure is `release`-only
|
||
|
||
**Also settle here:** the `VALARM` ownership collision. Either the adapter owns
|
||
alarms and `AndroidTasksDataSource.setAlarm` stops bulk-deleting property rows,
|
||
or reminders stay local-only and are never serialised. Picking neither is how one
|
||
local reminder edit destroys every server-side alarm on that task.
|
||
|
||
**Done when:** the corpus round-trips, the migration test passes both directions,
|
||
and no fixture loses an unknown property.
|
||
|
||
### Settled while building it
|
||
|
||
- ⚠️ **No schema migration was needed.** `TaskEntity.unknown_properties` already
|
||
exists at v1, reserved for exactly this and written by nothing. The residue
|
||
fills it. It holds leftover properties followed by whole sub-component blocks —
|
||
`BEGIN`/`END` lines are content lines like any other, so nesting and unknown
|
||
sub-components fall out of the same representation, which is what the plan
|
||
thought needed a second table.
|
||
- **`VALARM` ownership: neither side writes the other's alarms.** `VALARM`s
|
||
round-trip in the residue and are never authored; local reminders live in
|
||
`task_alarms` and are never serialised. The collision `SYNC.md` describes was
|
||
`AndroidTasksDataSource.setAlarm` bulk-deleting *provider property rows* — in
|
||
the own-store world the two stores are disjoint and neither can destroy the
|
||
other. Merging them is a later decision; the point is that it is now a
|
||
decision, not an accident.
|
||
- **The mapper claims a property only when it can reproduce it exactly.**
|
||
Everything else stays in the residue: a value it cannot parse or that is out of
|
||
range (`PRIORITY:11`, `PERCENT-COMPLETE:150`, `SEQUENCE:x`, `STATUS:X-DEFERRED`),
|
||
a time it can read but not reproduce (floating, or a `TZID` this device's tzdb
|
||
lacks), and a `DTSTART`/`DUE` pair that disagrees on value type or timezone —
|
||
one `is_all_day` flag and one `timezone` column cannot author both, and
|
||
flattening the odd one out destroys it. Clamping or defaulting any of these
|
||
would be a silent rewrite of somebody's data. `SUPPRESSED_BY_RESIDUE` makes
|
||
"present in the residue" mean "do not author this".
|
||
- ⚠️ **Suppression is conditional, or editing breaks.** A residue property
|
||
suppresses its column only while the two still agree. If the user edits a field
|
||
whose stamp was unreproducible, the stale residue copy is evicted and the
|
||
column is authored — otherwise changing the due date of an imported task would
|
||
silently do nothing on the server. `RELATED-TO` is the deliberate exception: a
|
||
parent we have not fetched is not the same as no parent, so an unresolvable
|
||
link is kept rather than destroyed.
|
||
- **One normalisation is allowed beyond the stated allowlist**, and it is an
|
||
equivalence rather than a concession: an absent `STATUS` and
|
||
`STATUS:NEEDS-ACTION` mean the same thing for a to-do. Any other `STATUS`
|
||
compares exactly, so losing a completion still fails the corpus.
|
||
|
||
---
|
||
|
||
## Chunk 2a — the vendored DAV foundation
|
||
|
||
⚠️ **Split out of chunk 2 while building it.** Vendoring is a self-contained
|
||
piece with its own verification — upstream's suite has to pass unmodified before
|
||
anything is built on top — and bundling it with discovery and auth would have
|
||
produced one commit nobody could review.
|
||
|
||
**Version:** dav4jvm **2.2.1**, the last OkHttp release. 3.0.0 deleted the OkHttp
|
||
package for Ktor and 4.x requires Java 21, and `SYNC.md`'s entire auth section is
|
||
written in OkHttp terms — preemptive Basic via an `Interceptor`, OkHttp stripping
|
||
`Authorization` on cross-host redirects, `BasicDigestAuthHandler` because OkHttp
|
||
has no Digest. Taking 4.x would have invalidated all of it. A plain JVM module,
|
||
not an Android library: the tree has zero Android imports and must keep it that
|
||
way, or the floret-kit extraction stops being a file move.
|
||
|
||
Full record in [`dav/PROVENANCE.md`](../dav/PROVENANCE.md). Of the two defects
|
||
`SYNC.md` said we would inherit, **one was not real** — 2.2.1 does follow 303 —
|
||
and the other (`dav4jvm#209`, permanent redirects never reaching the caller) is
|
||
fixed. A third, which the audit did not have, was found by writing the tests
|
||
upstream never wrote: **HTTP dates were parsed and formatted in the device's
|
||
local zone**, putting every `getlastmodified` out by the local UTC offset.
|
||
|
||
**Done when:** upstream's 63 tests pass unmodified, `commons-lang3` and the xpp3
|
||
runtime dependency are gone, and `:app` links against the module.
|
||
|
||
---
|
||
|
||
## Chunk 2b — the discovery and auth protocol
|
||
|
||
⚠️ **Split again while building it.** The seam is the platform: everything here
|
||
is protocol, testable on a JVM against MockWebServer, and none of it touches
|
||
Android. Keystore, `AccountManager`, Custom Tabs and the account-add UI are a
|
||
different kind of work with a different kind of verification — including the
|
||
on-device review — so they are chunk 2c.
|
||
|
||
Lands in a new `:caldav` module: MIT, plain JVM, `api`-depending on `:dav`. A
|
||
separate module from the vendored MPL tree keeps the licences unmixed and makes
|
||
"no Android types" a compile-time guarantee rather than a discipline.
|
||
|
||
**One dependency added:** `dnsjava` 3.6.3 (BSD-3) for the SRV/TXT half of
|
||
RFC 6764. Android's own `DnsResolver` is callback-only and cannot do the
|
||
`TXT path=` lookup, and JNDI's DNS provider does not exist on Android at all.
|
||
It is what DAVx5 uses. Behind a `DnsResolver` interface so the pipeline is
|
||
testable without a network.
|
||
|
||
**Done when:** the trap table is executable — Posteo's SRV-only 8443, GMX's
|
||
`TXT path=`, null SRV targets, Google's dead-end record, SOGo's triple
|
||
resourcetype, Nextcloud's trashed calendar, the absent/empty component set, and
|
||
`<D:unauthenticated/>` as a 200 that means failure.
|
||
|
||
---
|
||
|
||
## Chunk 2c — the Android account layer
|
||
|
||
Everything that needs the platform and nothing that needs a screen: Keystore
|
||
credentials, `AccountManager`, the stub sync adapter, the manifest, and the
|
||
network security config. The account-add UI is 2d — it is a design task, it
|
||
needs the `material-3` skill, and it is the piece that needs an on-device review.
|
||
|
||
### ⚠️ A stub `ContentProvider` turned out to be required
|
||
|
||
Not in the original plan, and it is load-bearing. A sync adapter registers
|
||
against a **content authority**, and Agendula publishes no provider — `:provider`
|
||
was deleted when we took our own Room store. With no authority there is nothing
|
||
for `<sync-adapter android:contentAuthority>` to name, nothing for
|
||
`requestSync` to address, and nothing for Settings to render a switch against.
|
||
`SyncStubProvider` stores nothing and exists solely to hold up that end.
|
||
|
||
### Other things settled here
|
||
|
||
- **The account type and authority are per build variant**, generated by
|
||
`resValue`. Two installs cannot own the same account type, so a debug build
|
||
sharing the release build's would fight it. `SyncContractTest` asserts the
|
||
Kotlin constants and the generated strings still agree — drift between them is
|
||
invisible at build time and surfaces as an account the framework will not
|
||
trigger.
|
||
- **The credential blob is the one thing excluded from backup.** It lives in its
|
||
own DataStore file for exactly that reason. Keystore keys are non-exportable,
|
||
so a restored ciphertext is permanently undecryptable; excluding it means the
|
||
user signs in again, which is the honest outcome. Excluding the database or all
|
||
of DataStore would trade a latent bug for a live one.
|
||
- ⚠️ **The network security config trusts the whole user CA store, for all
|
||
traffic.** `SYNC.md` specifies it, and without it a correctly installed private
|
||
CA is not trusted at all — which is the self-hosting case. But `base-config` is
|
||
the widest form: any CA in the user store can intercept the CalDAV connection
|
||
and read the app password from the `Authorization` header. **Chunk 5's
|
||
cert4android should replace this block, not sit beside it** — its per-connection
|
||
approval is the narrow version of the same capability.
|
||
- **`FOREGROUND_SERVICE` appears in the merged manifest** without being declared,
|
||
from `work-runtime`. That is not the FGS route: below API 31 WorkManager
|
||
implements expedited work with a foreground service, and minSdk is 29. Noted in
|
||
the manifest because it shows in F-Droid's permission diff.
|
||
|
||
**Not run:** the instrumented tests here (`CredentialStoreTest`,
|
||
`SyncContractTest`) compile but have not been executed — `CLAUDE.md` reserves
|
||
device interaction for when it is explicitly asked for. They need a device run
|
||
before chunk 2 is done.
|
||
|
||
---
|
||
|
||
## Chunk 2d — the account-add UI
|
||
|
||
One flow, one back-stack entry, as a stepper rather than four destinations — the
|
||
steps are not independently reachable, and "back" from the browser step means
|
||
abandoning a server-side flow rather than popping a screen. It hangs off Settings
|
||
using the same sliding-section pattern Storage → Export already uses.
|
||
|
||
⚠️ **`PreemptiveBasicInterceptor` was deleted here**, not kept. The vendored
|
||
`BasicDigestAuthHandler` already sends Basic preemptively over HTTPS, and it also
|
||
does Digest (Baïkal defaults to it and OkHttp has none), caches which scheme
|
||
worked, and restricts by *registrable domain* rather than exact host — which is
|
||
what a cross-host home set needs, since iCloud puts the principal on
|
||
`caldav.icloud.com` and the home set on `pNN-caldav.icloud.com`. Shipping two
|
||
implementations of preemptive Basic is the same defect chunk 1 removed when
|
||
`ICalendarWriter` stopped carrying its own folding.
|
||
|
||
**A seam was added for testability, after the review found eight issues in one
|
||
untested ViewModel.** `CalDavGateway` and `AccountCreator` put the network and
|
||
the database behind interfaces, so the sign-in state machine — which decides
|
||
where five discovery outcomes lead, when a one-shot app password is spent, and
|
||
which host a credential is scoped to — is exercised without a server, a
|
||
database, a Keystore or an `AccountManager`.
|
||
|
||
**Still open, and small:** the system entry point (Settings → Accounts → Add
|
||
account → Agendula) refuses with a message pointing at the in-app flow rather
|
||
than driving it. Wiring it means `MainActivity` handling
|
||
`SyncAuthenticator.ACTION_ADD_ACCOUNT` and answering the
|
||
`AccountAuthenticatorResponse`.
|
||
|
||
**Done when:** the flow has had an on-device review and an explicit go-ahead —
|
||
`CLAUDE.md`'s rule, and this is the case it exists for.
|
||
|
||
---
|
||
|
||
## Chunk 2d — the account-add UI (original outline)
|
||
|
||
**Goal:** the app can add a CalDAV account and list its VTODO collections. No
|
||
syncing yet.
|
||
|
||
**Vendoring, first.** A new Gradle module holding dav4jvm + cert4android sources
|
||
at our Java 17 target, MPL-2.0 headers intact, with a `PROVENANCE.md` recording
|
||
the upstream commit — the repo already has this pattern from `:provider`. Fix the
|
||
two inherited defects while vendoring, since we now can: **303 is not handled**
|
||
(RFC 6764 §5 names it explicitly) and **issue #209** mutates `location` in place
|
||
so permanent redirects never reach the caller.
|
||
|
||
**Discovery** is the nine-step pipeline in `SYNC.md` § *Generic CalDAV
|
||
discovery*, including every step marked ⚠️ MISSING there. The two filter
|
||
corrections are the important part and both are inversions of the obvious:
|
||
|
||
- **`supported-calendar-component-set` absent means "supports everything."**
|
||
Request properties **by name** — RFC 4791 §5.2.3 says the property SHOULD NOT
|
||
come back from an allprop request. Treat empty as all.
|
||
- **Classify on `resourcetype` as an unordered set, with a positive test for
|
||
`CALDAV:calendar`** — not by excluding `schedule-outbox`, because SOGo's main
|
||
personal calendar reports `collection` + `calendar` + `schedule-outbox`
|
||
simultaneously for every non-Apple client, which is exactly what we are.
|
||
|
||
The discovery traps table (Posteo SRV-only on 8443, GMX/Web.de `path=`, null SRV
|
||
target, Google's dead-end SRV, iCloud/Zoho well-known 401, `dav.runbox.com`'s
|
||
HTTPS→HTTP downgrade, `<D:unauthenticated/>` as a 200 that means failure,
|
||
cross-host home sets) becomes a table-driven unit test over recorded responses.
|
||
|
||
**Nextcloud Login Flow v2** with all five corrections: `POST` form-encoded (a
|
||
`GET` is 405); an explicit `User-Agent`, because it becomes the app password's
|
||
name in the user's security settings and `okhttp/4.12.0` defeats the entire point
|
||
of the flow; 404 means *pending only* and anything that is neither 404 nor 200
|
||
stops the poll; validate the `endpoint` origin and refuse an `http` downgrade
|
||
outright; and `loginName` is **not** the uid — it is the Basic auth username and
|
||
nothing else. Custom Tabs needs the `<queries>` entry, the `ACTION_VIEW`
|
||
fallback, `{token, endpoint, deadline}` persisted before launch, and an explicit
|
||
"I finished / Cancel" affordance.
|
||
|
||
**Auth is not just Basic.** Digest (Baïkal defaults to it, OkHttp has none),
|
||
Basic sent preemptively via an `Interceptor` gated to HTTPS and the account's own
|
||
origin, and domain-detection at account-add time for the three servers whose real
|
||
error is not "wrong password": Fastmail (app password; Basic plan has no CalDAV),
|
||
iCloud (app-specific password, 2FA to mint one), Google (OAuth2-only — refuse
|
||
with an explanation, `SYNC.md` drops it as a target).
|
||
|
||
**Credentials:** Keystore `AES/GCM/NoPadding`, blob in DataStore. Do not call
|
||
`setUserAuthenticationRequired`; do **not** set `setUnlockedDeviceRequired`
|
||
(breaks background sync). `AEADBadTagException` and
|
||
`KeyPermanentlyInvalidatedException` mean *re-authenticate*, not crash. Exclude
|
||
the blob — and only the blob — from Auto Backup: a restored ciphertext is
|
||
permanently undecryptable.
|
||
|
||
**The account plumbing, which is the part that silently does nothing if skipped.**
|
||
`AccountManager` **plus a registered `AbstractThreadedSyncAdapter`** whose
|
||
`onPerformSync` enqueues a WorkManager job and waits. Without a registered
|
||
adapter, `hasAuthorityAccess()` makes every `ContentResolver` sync API a silent
|
||
no-op at targetSdk ≥ 34 — no exception, no log, and it passes on a Robolectric
|
||
shadow. Manifest: `INTERNET`, `READ_SYNC_SETTINGS`, `WRITE_SYNC_SETTINGS`,
|
||
`ACCESS_NETWORK_STATE`; the authenticator `<service>` exported and guarded by
|
||
`android.permission.ACCOUNT_MANAGER` (`ACCOUNT_AUTHENTICATOR` does not exist).
|
||
Ship an in-app sync button, since "Sync now" stays greyed out under
|
||
`userVisible="false"`. Add `androidx.work`, `androidx.hilt:hilt-work` and
|
||
`androidx.browser` to the catalogue, with the `HiltWorkerFactory`, the removed
|
||
default `WorkManagerInitializer` and an `@EarlyEntryPoint` for the authenticator.
|
||
|
||
**Also:** ship a `network-security-config` with `<certificates src="user"/>`, or
|
||
a user who correctly installs their private CA is still not trusted.
|
||
|
||
**UI in this chunk needs on-device review before it can be called done** — the
|
||
account-add flow is the first screen a self-hoster meets, and per `CLAUDE.md`
|
||
nothing UI-facing gets tagged before that review.
|
||
|
||
**Done when:** an account can be added against a real Nextcloud and its task
|
||
lists appear, the account shows in system Settings with a working sync switch,
|
||
and the discovery trap table is green.
|
||
|
||
---
|
||
|
||
## Chunk 3 — engine core
|
||
|
||
**Goal:** a full bidirectional sync that is correct but not yet clever — no
|
||
sync-collection, no scheduling beyond a manual trigger.
|
||
|
||
**Read path.** `REPORT calendar-query` with a VTODO comp-filter and **no
|
||
time-range**, matching DAVx5 — which deliberately does not use RFC 6578 for
|
||
tasks, and omits the time-range because some servers return no tasks without a
|
||
time at all. Then `calendar-multiget` in batches, **matching returned hrefs
|
||
against what was asked for**, because real servers reply with responses for
|
||
unrelated URLs. Never request `calendar-data` inside a REPORT that is not
|
||
sanctioned to carry it.
|
||
|
||
**Write path.** `If-None-Match: *` on create, `If-Match` on update and DELETE.
|
||
Send `Prefer: handling=strict` to sabre-based servers — the cheapest single fix
|
||
in the whole audit, because it preserves both our bytes and the ETag. Request
|
||
`Accept-Encoding: identity`. Strip `W/` and keep a weak flag; **if a PUT returns
|
||
no ETag or a weak one, discard it and re-fetch.**
|
||
|
||
**412 means three different things** and they get three code paths: on create it
|
||
means the filename is taken (re-fetch, adopt if the UID matches, else regenerate
|
||
as a UUID); on update it means the server is newer (decision 2: server wins,
|
||
report the discard); on update where the resource is *gone* it is spec-correct
|
||
and not a conflict at all (`HEAD` to disambiguate — 404 is delete-vs-edit).
|
||
|
||
**Never write back a body whose ETag does not match the hash of what we
|
||
downloaded.** This is the Nextcloud shared-calendar landmine and it is the single
|
||
most destructive bug available in this chunk: `CalendarObject::get()` strips
|
||
`VALARM` on read-only shares and reduces `CLASS:CONFIDENTIAL` to a VEVENT-shaped
|
||
whitelist that deletes `DUE`, `STATUS`, `COMPLETED`, `PERCENT-COMPLETE`,
|
||
`PRIORITY` and `RELATED-TO` — while leaving the ETag untouched.
|
||
|
||
**Isolation, in two layers.** A failed collection must not fail the account, and
|
||
— its twin, which the draft omitted — **a failed resource must not fail the
|
||
collection.** Per-resource quarantine with a failure counter, *not* backoff: a
|
||
single HTTP 400 has halted all calendar sync in DAVx5 for weeks. 507 MUST NOT be
|
||
auto-retried (RFC 4918 §11.5), and 5xx is not safely retryable either — a
|
||
contradictory `RRULE`/`EXDATE` pair returns 500 from Nextcloud and will do so
|
||
forever.
|
||
|
||
**Client-side validation before PUT**, because sabre returns 415 for things
|
||
ordinary UI actions produce: `DUE` before `DTSTART`, value-type mismatch between
|
||
them, a present `METHOD`, multiple UIDs, mixed component types in one resource.
|
||
|
||
**href ≠ UID.** Sanitise filenames on vdirsyncer's rule — `a–zA–Z0–9_.-+`,
|
||
**excluding `@`** — cap the basename near 200 bytes, fall back to a UUID.
|
||
DELETE: conditional on `If-Match`, 404/410 count as success, and a locally
|
||
deleted resource that was never uploaded is never DELETEd.
|
||
|
||
**Local change detection** is `is_dirty = 1 OR is_deleted = 1` scoped by list —
|
||
the columns already exist in the v1 schema. Every downstream insert writes
|
||
`is_dirty = 0` explicitly.
|
||
|
||
**Done when:** a task created in Nextcloud web appears locally and vice versa;
|
||
a concurrent edit resolves server-wins with a visible report; a quarantined
|
||
resource does not stop its collection.
|
||
|
||
### Settled while building it
|
||
|
||
**⚠️ The mapper emitted `TZID` with nothing to resolve it.** Chunk 1 writes
|
||
`DTSTART;TZID=Europe/Berlin:…`, and RFC 5545 §3.2.19 requires a `TZID` without a
|
||
leading solidus to reference a `VTIMEZONE` **in the same object**. Nothing
|
||
generated one — `VTIMEZONE` sits at `VCALENDAR` level, outside the `VTODO` whose
|
||
residue the mapper stores, so it was outside the round-trip guarantee entirely.
|
||
Left alone, every zoned task would have been malformed, and `Prefer:
|
||
handling=strict` — which chunk 3 sends precisely so servers stop repairing our
|
||
bytes — turns malformed into rejected. `VTimeZones` now generates the definition
|
||
from `java.time`'s own rules, which also keeps the zone we *write* in agreement
|
||
with the zone we compute occurrences in. **Cost accepted: only the currently
|
||
effective rules are emitted, not the historical transition table.** Tasks are
|
||
dated now or later.
|
||
|
||
**java.time does not encode "the last Sunday" as `-1`.** It arrives as
|
||
`dayOfMonthIndicator = 25` — "the first Sunday on or after the 25th" — and only
|
||
the fact that a seven-day window ending on the last day of the month *is* the
|
||
last occurrence identifies it. The obvious `-1` test never fires for any European
|
||
zone, and the fallback emits a seven-value `BYMONTHDAY` list where every other
|
||
client writes `BYDAY=-1SU`.
|
||
|
||
**`GetETag.eTag` has already been parsed.** dav4jvm strips the `W/` marker and
|
||
records it in a separate field, so feeding its value back through an ETag parser
|
||
reports **every weak tag as strong** — the exact failure the weak flag exists to
|
||
prevent. `ETag.from(property)` and `ETag.parse(header)` are now different
|
||
functions for that reason, and a test pins the difference.
|
||
|
||
**The discard is reported when the replacement arrives, not when the 412 does.**
|
||
Announcing it at the conflict — and clearing `is_dirty` there — makes the loss
|
||
real if the replacement download then fails, with nothing left to retry. The row
|
||
now stays dirty until the download overwrites it.
|
||
|
||
**A `SyncStore` seam, not Robolectric.** The reconciliation is where local edits
|
||
are discarded, tombstones swept and conflicts resolved; proving that needs a test
|
||
double for eight methods, not an in-memory Room plus a new test runtime.
|
||
|
||
**The mutilation guard is scoped to what is actually reachable.** Nextcloud
|
||
reduces `CLASS:CONFIDENTIAL` objects served from a *share* while leaving the ETag
|
||
intact; `VALARM` stripping happens on read-only shares, which are never written
|
||
to anyway. So the refusal is: shared collection + confidential + already on the
|
||
server. The collection's `read-only` and `shared` flags are re-read on every sync
|
||
rather than trusted from account-add, because ACL churn is silent.
|
||
|
||
**Manual trigger.** `ContentResolver.requestSync` is gated behind
|
||
`hasAuthorityAccess()` at our targetSdk and returns silently when it refuses, so
|
||
the in-app button enqueues the WorkManager job directly via `SyncTrigger`. The
|
||
sync adapter uses the same path.
|
||
|
||
### Found by the review, and worth naming
|
||
|
||
**⚠️ An empty listing would have hard-deleted the whole list.** The sweep took
|
||
"absent from `remote.list()`" as proof of a server-side delete, and its evidence
|
||
is a `calendar-query` with a `VTODO` comp-filter — the filter whose mishandling
|
||
is the reason the query carries no time-range in the first place. A server
|
||
answering it with an empty *successful* multistatus is indistinguishable from an
|
||
empty collection, and one such answer destroyed every task in that list in one
|
||
pass. **An empty listing now never sweeps.** Cost accepted: a collection
|
||
genuinely emptied on the server keeps its local rows until one task reappears
|
||
there. That is recoverable by hand; the other error is not.
|
||
|
||
**Quarantine did not cover creates.** Failures on a resource with no href yet
|
||
were counted under its UID and read back under its href, so nothing ever read
|
||
them: a new task the server rejects permanently was re-PUT on every sync forever
|
||
— the exact DAVx5 failure the counter exists to prevent — while the unread keys
|
||
grew in DataStore without bound. Resources are now keyed by href *or* `uid:…`,
|
||
and a success clears both.
|
||
|
||
**A `RELATED-TO` deleted on the server never unparented anything.** Only present
|
||
parents were recorded, and `upsert` carried the old `parent_id` forward — which
|
||
the writer then resolved straight back into a `RELATED-TO`, re-uploading the link
|
||
the user had deleted elsewhere.
|
||
|
||
**Two validators disagreed about what a DATE is.** `ResourceValidator` tested
|
||
only `VALUE=DATE` while the mapper also reads a bare eight-digit value as one, so
|
||
a residue `DTSTART:20260101` beside an authored `DUE;VALUE=DATE:20260102` was
|
||
rejected locally and never left the device — a case `VTodoMapperTest` already had
|
||
a test for. The per-component rules now delegate to `VTodoMapper.validate`, which
|
||
also restores the `TRIGGER;RELATED=END` check that the new validator had silently
|
||
dropped and left as dead code.
|
||
|
||
**An unresolvable `TZID` produced a resource with no definition for it.** A
|
||
Windows zone name from another client survives in the residue, and `java.time`
|
||
cannot regenerate a `VTIMEZONE` for it — so with `Prefer: handling=strict` the
|
||
upload becomes an unexplained rejection. It is now refused by name.
|
||
**Open for chunk 5:** preserving the server's own `VCALENDAR`-level `VTIMEZONE`
|
||
would fix it properly, and needs somewhere to put residue that is not the
|
||
`VTODO`'s.
|
||
|
||
**Three narrower ones, all fixed:** quarantine counts were a global
|
||
read-modify-write, so two accounts syncing at once discarded each other's
|
||
(now merged inside `edit`); `NeedsSignIn` and `Misconfigured` never called
|
||
`recordSync`, leaving the accounts screen reporting "synced 5 minutes ago" for an
|
||
account that cannot sync at all; and `apply()` re-read every row in the list once
|
||
per downloaded resource.
|
||
|
||
---
|
||
|
||
## Chunk 4 — incremental sync and scheduling
|
||
|
||
**Goal:** `sync-collection` as the optimisation on top of chunk 3's baseline, and
|
||
sync that runs by itself.
|
||
|
||
Every numbered finding in `SYNC.md` § *RFC 6578* is a test in this chunk. The
|
||
four that are not optional:
|
||
|
||
1. **Invalidation has no status code.** Ignore the status; match
|
||
`<D:valid-sync-token/>` anywhere in the body on **any** 4xx. Observed in the
|
||
wild: 403 (sabre), 400 (Google), 409 (Radicale), 412 (Evolution accepts it).
|
||
Thunderbird accepts only 400 and therefore never recovers from the 403 most of
|
||
the self-hosted world emits.
|
||
2. **Initial sync must not report deletions**, so mark-and-sweep is mandatory and
|
||
`initialIncomplete` must be persisted *atomically with the token* — otherwise a
|
||
resumed partial sync sweeps against an incomplete set and **deletes live
|
||
data**.
|
||
3. **Persist the token only after the bodies are applied**, per page. The RFC's
|
||
own Appendix B has this backwards. Under WorkManager, process death mid-sync is
|
||
routine.
|
||
4. **A token the server accepts over a change log it already pruned** returns 207,
|
||
zero changes, and no error. RFC 6578 gives no signal for it. The only
|
||
mitigation is **periodic full reconciliation** (PROPFIND `Depth: 1` + ETag
|
||
diff) on a slow cadence regardless of the token — so chunk 3's full path is a
|
||
permanent safety net, not a fallback.
|
||
|
||
Plus: no `DAV:limit` (Nextcloud regressed it to a localised HTML error page);
|
||
507 on the SELF href is truncation, 507 as the outer status means retry without
|
||
the limit; an iteration cap **and** a no-progress guard, because the RFC never
|
||
requires the token to advance; tokens are opaque and **never reused across
|
||
collections**, keyed `(accountId, collectionUrl)`; deletion is
|
||
`<D:status>404</D:status>` at *`<D:response>`* level, not inside a `propstat`; a
|
||
207 with no `<D:sync-token>` degrades to PROPFIND rather than throwing; and
|
||
`sync-collection` **never reports collection property changes**, so displayname,
|
||
colour and read-only status refresh only via PROPFIND on the home set.
|
||
|
||
**Three membership edge cases** that each look like a bug: create-then-delete
|
||
between syncs is reported as removed (the delete handler no-ops on an unseen
|
||
href); delete-then-recreate at the same URI is reported as *changed* (re-read the
|
||
UID from the body); and ACL churn may be reported as removal, so **apply a sanity
|
||
threshold before acting on a large delete batch.**
|
||
|
||
**Scheduling has a ceiling.** Make sync chunked and resumable — persist the
|
||
cursor per collection so a killed worker resumes rather than restarts. Periodic
|
||
sync is a plain `PeriodicWorkRequest`, no foreground service. "Sync now" from a
|
||
visible screen uses `setExpedited(RUN_AS_NON_EXPEDITED_WORK_REQUEST)`, and
|
||
`getForegroundInfo` is implemented **unconditionally** — omitting it crashes
|
||
below API 31 and we support 29. Android 15 forbids starting a `dataSync` FGS from
|
||
`BOOT_COMPLETED`, and we have a boot receiver. Sync hard on app open and on
|
||
connectivity-regained, and be honest in the UI about cadence: in the `rare` and
|
||
`restricted` buckets network access is off entirely and the worst case is
|
||
genuinely "once overnight".
|
||
|
||
**Done when:** a token invalidation on any of the four status codes recovers; a
|
||
process kill mid-page loses nothing and duplicates nothing; the periodic worker
|
||
runs and the manual trigger is instant.
|
||
|
||
### Settled while building it
|
||
|
||
**`initialIncomplete` turned out not to need a column — or to exist.** The plan
|
||
called for it to be persisted atomically with the token, to stop a resumed
|
||
partial sync sweeping against an incomplete set. But the mark-and-sweep it
|
||
protects *is* chunk 3's full path, so the rule that removes the whole hazard is
|
||
simpler: **a token is adopted only when a full reconciliation completes.** A run
|
||
killed part-way stores no token and the next one starts over. No second column,
|
||
nothing to write atomically with anything.
|
||
|
||
**The token is taken from the PROPFIND that *precedes* the read, not after it.**
|
||
Both are imprecise; only one is imprecise safely. A token minted after the
|
||
listing silently swallows everything that changed while we were reading it; one
|
||
minted before merely re-reports those changes next time, and a re-report costs an
|
||
ETag comparison.
|
||
|
||
**Where the cadence clock lives.** `task_lists.sync_token` is domain state and
|
||
stays in Room. "When did this collection last reconcile in full" is scheduling
|
||
bookkeeping, so it is in DataStore (`SyncCadenceStore`) — and deliberately
|
||
outside any backup, because a restored "checked recently" on a fresh install is
|
||
exactly the stale trust the full path exists to break.
|
||
|
||
**⚠️ `ForegroundInfo` must carry no `foregroundServiceType`.** Implementing
|
||
`getForegroundInfo` is mandatory — `setExpedited` falls back to a foreground
|
||
service below API 31 and the default implementation throws, so a worker without
|
||
it crashes on every API 29/30 device. But passing
|
||
`FOREGROUND_SERVICE_TYPE_DATA_SYNC` re-introduces the entire tail the design
|
||
rejected: the `FOREGROUND_SERVICE_DATA_SYNC` permission, Android 15's
|
||
six-hours-per-24 budget with its fatal `RemoteServiceException`, and a Play video
|
||
demo per declared type. Lint caught it, correctly. The type is unnecessary
|
||
anyway: above API 30 `setExpedited` uses an expedited job and never calls
|
||
`getForegroundInfo`, and types only became mandatory at API 34.
|
||
|
||
**No boot receiver for sync.** WorkManager restores its own schedule across a
|
||
reboot, and Android 15 forbids starting a `dataSync` FGS from `BOOT_COMPLETED` —
|
||
so a design that needed one would have had no way to run. App open calls
|
||
`rescheduleAll()` instead, which is idempotent under `KEEP` and repairs the one
|
||
case WorkManager cannot: a schedule lost to "clear app data" or to a restore.
|
||
|
||
**Expedited is spent on the button and nowhere else.** The quota is per-app and
|
||
exhaustible; a background trigger that spends it leaves none for the trigger the
|
||
user is actually watching.
|
||
|
||
### Found by the review, and worth naming
|
||
|
||
**⚠️ The cadence clock was in the backed-up DataStore — while its own KDoc, and
|
||
this document, claimed it was not.** Auto Backup includes `datastore/`, and only
|
||
`agendula_credentials` was excluded. Restored onto a new device it would say
|
||
"reconciled minutes ago", so the restored sync token would be trusted for another
|
||
day: exactly the silently pruned change log the full path exists to catch. Both
|
||
sync-state stores now live in their own `agendula_sync_state` file, excluded
|
||
alongside the credentials — a restored quarantine count is the same class of lie,
|
||
silently skipping resources that were never tried on this device.
|
||
|
||
**The incremental download path had no write-phase guard.** `downloadPhase` skips
|
||
a row that is still dirty; `downloadChanged` did not. A share demoted to
|
||
read-only leaves the upload refused and the row dirty, and the next change-log
|
||
entry for it overwrote the user's unsent edit with no `DiscardedEdit` at all. The
|
||
guard cannot simply be copied, either — the 412 path *deliberately* leaves the
|
||
row dirty and waits for the download — so it is "dirty **and not awaiting a
|
||
refetch**".
|
||
|
||
**A change log describes the past, and `applyRemovals` treated it as the
|
||
present.** An entry predating this run's own upload deleted the row we had just
|
||
written, leaving an orphan on the server and nothing on the device. It now
|
||
respects `touched`, and reports a lost edit the way the sweep does.
|
||
|
||
**An abandoned incremental attempt poisoned a successful run.** `ChangeSet.Failed`
|
||
and the mass-removal refusal both set `report.failure`, which the full fallback
|
||
never cleared — so a server answering `sync-collection` with a bare 400 refused
|
||
the new token, skipped the cadence record, and showed "last sync didn't finish"
|
||
on every other sync while nothing was wrong with the data. A fallback that
|
||
succeeds now demotes the attempt to `incrementalNote`.
|
||
|
||
**`refetch` was silently dropped on a successful incremental run**, so a PUT
|
||
accepted without a strong ETag never got its canonical body back on a server that
|
||
does not echo our own writes into its change log.
|
||
|
||
**Two of my own patches were wrong in ways the compiler could not see.** `fatal()`
|
||
was written with its KDoc and never called — a `python` patch aborted on its
|
||
second assertion after the first had already matched, so nothing was written, and
|
||
the function sat there describing a bug it was not preventing. And the
|
||
`delete-then-recreate` lookup read the href index *after* the same index had been
|
||
re-pointed at the rows just written, so it never found the displaced row. The
|
||
first was caught by the review, the second by a test.
|
||
|
||
**Two smaller ones:** the displaced lookup reintroduced the per-resource
|
||
full-table scan the comment twelve lines above it forbids, and `MainActivity`
|
||
re-ran the app-open sync on every rotation, theme switch and locale change.
|
||
|
||
---
|
||
|
||
## Chunk 5 — hardening, compliance, real servers
|
||
|
||
**Goal:** the things that are not features and are not optional.
|
||
|
||
**The server matrix, turned on.** Nextcloud, Radicale, Baïkal (**on Digest**),
|
||
SOGo, Fastmail, iCloud — each per-server trap from `SYNC.md` § *Server reality*
|
||
as a named test. Two must be explicit because both silently destroy data:
|
||
**round-tripping a task from a shared Nextcloud calendar**, and **an
|
||
ETag-unchanged body change on SOGo** (its ETag is a row-version counter and the
|
||
body is regenerated per principal). Also: Nextcloud's per-calendar UID uniqueness
|
||
(409 `no-uid-conflict`), its trashbin renaming hrefs to `<name>-deleted.ics` so
|
||
delete→recreate→delete returns 403, MKCALENDAR rate-limited 10/hour; SOGo never
|
||
invalidating a token and using second-granularity ones; Radicale never enforcing
|
||
`supported-calendar-component-set`. Feature-detect collection creation via
|
||
OPTIONS and **disable the "new task list" UI where neither MKCALENDAR nor
|
||
extended MKCOL exists** (iCloud is MKCOL-only, Google has neither, Posteo
|
||
disables it).
|
||
|
||
**Revocation, both directions.** On 401: stop that account immediately, mark
|
||
`NEEDS_REAUTH`, notify with a deep link into the login flow, and **do not retry
|
||
on a timer** — Nextcloud's brute-force protection throttles then 429s *per source
|
||
IP*, so a retry loop on a dead app password takes down the user's other Nextcloud
|
||
clients and looks like we broke their server. Distinguish 401 from 403 from
|
||
429/503 (honour `Retry-After`). On account removal: best-effort
|
||
`DELETE /ocs/v2.php/core/apppassword` with `OCS-APIRequest: true`, or
|
||
uninstalling never revokes access.
|
||
|
||
**The backup path, which arms itself the day this ships.** Post-restore
|
||
reconciliation must **offer to re-attach the account**, never prune. Pruning is
|
||
event-driven off `AccountManager`'s account-removed broadcast, not off "absent
|
||
from the visible set". Device-verify by restoring onto a fresh device and
|
||
confirming nothing is deleted — `SYNC.md` is explicit that excluding the database
|
||
from backup is the wrong fix, because Auto Backup is currently Local mode's only
|
||
safety net.
|
||
|
||
**Licence attribution — a plain violation until it exists.** MPL-2.0 §3.2(a)
|
||
(dav4jvm, cert4android) requires telling recipients how to obtain source; §3.4
|
||
requires retaining file headers; BSD-3 (ical4j) requires reproducing the
|
||
copyright notice in binary distributions; Apache-2.0 §4(d) propagates NOTICE.
|
||
Settings exposes only our own MIT `LICENSE` today. Note ical4j's POM declares a
|
||
non-SPDX licence name and a `LICENSE` URL that 404s, so generators produce empty
|
||
output — this one is written by hand.
|
||
|
||
**Play compliance.** A privacy policy, linked in Console **and in the app**.
|
||
File as **Collected, not Shared**, encrypted in transit — "not collected" is not
|
||
defensible, because Play defines collection as transmitting off-device
|
||
irrespective of recipient. Ship a "Remove account and delete local data" action
|
||
even though the Account Deletion policy does not apply. **Do not ship
|
||
`REQUEST_IGNORE_BATTERY_OPTIMIZATIONS` in the Play build** — generic server sync
|
||
is not on the acceptable-use list; use
|
||
`ACTION_IGNORE_BATTERY_OPTIMIZATION_SETTINGS`.
|
||
|
||
**Minified build.** `:app` runs R8 + resource shrinking in `release`, and
|
||
`proguard-rules.pro` already documents two pruning outages. Verify on the
|
||
`releaseTest` build type, on device.
|
||
|
||
**Record the extraction.** A short section listing what moves to floret-kit
|
||
(`core-dav`, `core-sync`, Login Flow v2, credential storage, the licence screen)
|
||
and what stays app-local (the mapper, conflict policy and its UI, storage modes),
|
||
so the follow-up branch is a file move and not a rediscovery.
|
||
|
||
**Done when:** the matrix is green where a server exists to run it against, the
|
||
attribution screen ships, and a `releaseTest` build syncs on a real device.
|
||
|
||
### What shipped, and what is still owed
|
||
|
||
**Shipped in the chunk-5 commit** — everything verifiable without a device or a
|
||
live server:
|
||
|
||
- **Attribution.** `Settings → Open source licenses`, hand-maintained. Not a
|
||
generated list: POM metadata routinely names a non-SPDX licence and points at a
|
||
URL that 404s, so generators produce confidently empty output. dav4jvm is
|
||
vendored, which makes MPL-2.0 §3.2(a) *ours* rather than a dependency's — the
|
||
row names `dav/PROVENANCE.md` alongside upstream, because §3.2(a) is about
|
||
where a recipient of the binary can obtain the source of what we modified.
|
||
- **Revocation, both directions.** A 401 escalates from the collection to the
|
||
account, marks it, **cancels its periodic work**, and stops. Cancelling is the
|
||
part that matters: Nextcloud throttles and then 429s per source IP, so a timer
|
||
on a dead app password degrades every other Nextcloud client on that network.
|
||
On removal, `DELETE /ocs/v2.php/core/apppassword` with `OCS-APIRequest: true`,
|
||
best effort — without it, uninstalling never revokes anything and the password
|
||
outlives the app in the user's device list.
|
||
- **Play compliance.** A privacy policy in `docs/PRIVACY.md`, linked **in the
|
||
app** (the half that is usually missed) and declaring *Collected, not Shared* —
|
||
"not collected" is indefensible when Play defines collection as transmission
|
||
off-device regardless of recipient. A "delete this account's tasks from this
|
||
device too" option on removal. `REQUEST_IGNORE_BATTERY_OPTIMIZATIONS` was
|
||
already absent and stays absent.
|
||
- **The server matrix**, as far as a protocol mock can carry it: Nextcloud's
|
||
`no-uid-conflict`, its trashbin 403 on a reused href, and share detection;
|
||
SOGo's second-granularity opaque tokens and its triple resource type; Radicale
|
||
advertising `sync-collection` without implementing it.
|
||
|
||
**Still owed, and blocked on things a loop cannot do:**
|
||
|
||
1. **The live matrix.** Four tests are `@Ignore`d with their reasons written out
|
||
rather than deleted: Baïkal on Digest, a `CLASS:CONFIDENTIAL` task on a shared
|
||
Nextcloud calendar, an ETag-unchanged body change on SOGo, and Nextcloud's
|
||
MKCALENDAR rate limit. Each names why a mock cannot stand in for it.
|
||
2. **`releaseTest` on device**, and the R8 outages `proguard-rules.pro` already
|
||
documents.
|
||
3. **The restore path**, which needs a restore onto a fresh device — the one
|
||
verification `SYNC.md` calls for explicitly and the one that cannot be faked.
|
||
4. **cert4android**, which replaces the current blanket user-CA trust and is a
|
||
UI-bearing dependency.
|
||
5. **Collection creation feature detection** (OPTIONS → MKCALENDAR / extended
|
||
MKCOL) and disabling the "new task list" affordance where neither exists.
|
||
Deferred because the app has no CalDAV-side list-creation UI yet, so there is
|
||
nothing to disable — it becomes real the moment that UI does.
|
||
|
||
### Found by the review, and worth naming
|
||
|
||
**⚠️ Revocation was aimed at the principal URL and revoked nothing.** The OCS
|
||
endpoint sits at the *server root*; the principal is
|
||
`…/remote.php/dav/principals/users/alice/`, so appending to it produced a path
|
||
that 404s on every server, every time — and the failure was swallowed as
|
||
best-effort. The whole feature was a no-op. The root is now derived by cutting at
|
||
`remote.php`, which is right for a subpath install too, with the origin as the
|
||
fallback.
|
||
|
||
**⚠️ Opening the app resurrected accounts a 401 had just stopped.**
|
||
`ExistingPeriodicWorkPolicy.KEEP` only keeps work that is *unfinished*, and
|
||
CANCELLED counts as finished — so `rescheduleAll()` re-enqueued the very request
|
||
the stop had removed, and put a dead app password back on a four-hour loop
|
||
against a server that throttles by source IP. Exactly what the stop exists to
|
||
prevent.
|
||
|
||
**⚠️ The cursor advanced past bodies that were never applied.** A network drop
|
||
mid-multiget set `report.failure` and returned, and the incremental path stored
|
||
the new token anyway — a permanent hole in the collection, invisible until the
|
||
next full reconciliation a day later. `runFull` had this guard from the start;
|
||
the incremental path did not.
|
||
|
||
**Stopping an account cancelled the worker doing the stopping.** `syncTrigger.cancel`
|
||
kills both unique names, one of which is the WorkSpec currently executing —
|
||
WorkManager interrupts the coroutine, so `NeedsSignIn` was never returned and the
|
||
adapter saw CANCELLED rather than FAILED. The flag now does the work: `sync()`
|
||
refuses before touching the network, and the schedule is cancelled from outside
|
||
any worker.
|
||
|
||
**"Sign in again" was a dead end.** `create` is the only path that writes a
|
||
credential and it rejected the duplicate name, so a user with a revoked app
|
||
password could only remove the account — discarding the choice to keep its lists.
|
||
It now re-authenticates in place, but *only* for an account already marked as
|
||
needing sign-in, so a healthy credential can never be silently overwritten.
|
||
|
||
**Four scalars were emitted twice.** `SEQUENCE`, `PRIORITY`, `PERCENT-COMPLETE`
|
||
and `CLASS` reach the residue verbatim when unparseable, and `write` authors them
|
||
from their columns unconditionally — so a task read from `SEQUENCE:x` round-tripped
|
||
carrying both `SEQUENCE:0` and `SEQUENCE:x`. Invisible to the corpus, which
|
||
filters `SEQUENCE` out of comparison entirely. Now suppressed, with a test that
|
||
counts occurrences rather than comparing canonically.
|
||
|
||
**A solidus-prefixed `TZID` was rejected as undefined** — the rule's own comment
|
||
says §3.2.19 exempts it. Thunderbird writes
|
||
`/mozilla.org/20050126_1/Europe/Berlin` on every zoned task, so the validator
|
||
would have quarantined every Lightning-authored task permanently.
|
||
|
||
**Two smaller ones:** a resource the query lists but the multiget will not return
|
||
was re-requested for ever with nothing counting it, and account removal blocked
|
||
on a 30-second connect timeout before anything visible happened.
|
||
|
||
## What moves to floret-kit
|
||
|
||
Recorded here so the follow-up branch is a file move rather than a rediscovery.
|
||
The rule that made it a move rather than a rewrite held: **no task-domain type
|
||
and no Android UI type crossed into the DAV or CalDAV layers.**
|
||
|
||
| Moves | Why it generalises |
|
||
|---|---|
|
||
| `:dav` (vendored dav4jvm) → `core-dav` | Not app-specific in any way; Calendula needs the identical tree |
|
||
| `:caldav` — discovery, `CalendarCollection`, `sync-collection`, quirks | Pure protocol, plain JVM, no Android types |
|
||
| `NextcloudLoginFlow` | Nothing about it is task-shaped |
|
||
| `CredentialStore`, `CalDavAccounts`, `SyncAuthenticator` | Keystore + `AccountManager` plumbing, identical for any CalDAV client |
|
||
| `LicencesScreen` / `Attribution` | Every app in the family owes the same notices |
|
||
|
||
| Stays app-local | Why |
|
||
|---|---|
|
||
| `VTodoMapper`, `CalendarResource`, `ResourceValidator` | VTODO ↔ *our* Room schema; the residue contract is ours |
|
||
| `CollectionSyncer`, `SyncReport`, conflict policy and its UI | Encodes decision 2, which is a product decision |
|
||
| `SyncEngine`, `SyncWorker`, `SyncTrigger` | Bound to our entities and our scheduling choices |
|
||
| Storage modes, External mode | Agendula-specific by construction |
|
||
|
||
---
|
||
|
||
## Definition of done, every chunk
|
||
|
||
1. It builds: `./gradlew :app:assembleDebug`.
|
||
2. Unit tests pass:
|
||
`./gradlew :app:testDebugUnitTest :dav:test :caldav:test`. ⚠️ Name the plain
|
||
JVM modules explicitly — `testDebugUnitTest` is an Android-variant task and
|
||
does not exist on them, so their suites would otherwise be compiled by nobody
|
||
and run by nobody. Add each new module here **and** to CI.
|
||
3. Lint passes: `./gradlew :app:lintDebug`. Added after chunk 1 shipped a literal
|
||
byte-order mark that builds and tests both accepted and CI's lint step would
|
||
have rejected.
|
||
4. `/code-review high` on the chunk's diff, with the findings either fixed or
|
||
answered in the commit message. For a chunk that vendors code, scope the
|
||
review to the files we wrote — reviewing thousands of lines of verbatim
|
||
upstream is noise, and its defects belong in `PROVENANCE.md`.
|
||
5. One commit per chunk, referencing the chunk number. No release, no tag.
|
||
|
||
Chunk 2 additionally does not count as done until the account-add UI has had an
|
||
**on-device review with an explicit go-ahead** — `CLAUDE.md`'s rule, and this is
|
||
exactly the case it exists for.
|
||
|
||
## What stops the loop
|
||
|
||
- A `SYNC.md` decision that the evidence overturns. Say so; do not pick the other
|
||
branch silently.
|
||
- Anything requiring a real server that is not reachable — write the test, mark
|
||
it skipped, name it in the commit, move on.
|
||
- Anything that would need a device beyond `adb install`.
|