The parts of chunk 5 that a build can verify. What is left needs a device or a live server, and is listed in docs/SYNC-PLAN.md rather than guessed at. - Attribution screen in Settings. dav4jvm is vendored, which makes MPL-2.0 §3.2(a) ours rather than a dependency's, so its row points at PROVENANCE.md next to upstream. Hand-maintained: generators read POM metadata, which routinely names a non-SPDX licence and a licence URL that 404s. - Revocation both ways. A 401 marks the account, stops it before the next request reaches the network, and takes it off the schedule from outside the worker — Nextcloud throttles then 429s per source IP, so a timer on a dead app password degrades the user's other clients. On removal, a bounded best-effort DELETE of the app password, or uninstalling never revokes it. - Play compliance: docs/PRIVACY.md linked in the app, declaring Collected and not Shared; an option to delete the account's tasks from the device too; REQUEST_IGNORE_BATTERY_OPTIMIZATIONS confirmed absent. - The server trap matrix as far as a protocol mock reaches, with four tests left @Ignore'd and their reasons written out. - docs/SYNC-PLAN.md records what moves to floret-kit, so that branch is a file move rather than a rediscovery. /code-review high raised 9 findings, all fixed. Three were serious: app-password revocation was aimed at the principal URL and revoked nothing; opening the app put accounts a 401 had stopped back on the timer, because KEEP does not keep cancelled work; and the incremental path advanced the sync token past bodies a failed multiget never applied. Also: four scalars were emitted twice whenever their residue copy survived, which the round-trip corpus could not see. Not done, and needing you: the live server matrix, releaseTest on device, the restore-onto-a-fresh-device check, cert4android, and MKCALENDAR feature detection. Chunk 2's on-device review is still outstanding.
52 KiB
Agendula — CalDAV sync: the build plan
Branch: feat/caldav-sync, cut from release/v1.0.0.
Design: SYNC.md — the decisions, the traps and the audit trail
behind them. This document does not repeat it. It sequences it.
Shape: five chunks, one branch, one PR at the end.
SYNC.md is a 1,200-line decision document with roughly forty ⚠️ findings
scattered through it, most of which are "the obvious implementation is wrong,
here is why". That is the right shape for a design document and the wrong shape
for a work queue. This is the work queue.
Decisions taken before the first line of code
SYNC.md left three questions open and named two of them as blocking. All three
are now answered, plus one the phases imply. These are not reopened by the
implementation chunks — if a chunk finds evidence that overturns one, it stops
and says so rather than quietly picking the other branch.
| # | Question | Answer | Consequence |
|---|---|---|---|
| 1 | dav4jvm + cert4android distribution, and the Java-21 wall | Vendor a known-good tree, recompiled at our Java 17 target | FAIL_ON_PROJECT_REPOS stays; no Ktor/guava/xpp3 tail; MPL-2.0 headers retained and the attribution screen becomes mandatory, not optional. Freezes the API churn that shipped two breaking majors nineteen days apart |
| 2 | Conflict policy on 412 | Server wins, local edit discarded | DAVx5's policy. Terminates, always. The discarded edit is surfaced — a sync report the user can read — because silence is what makes this policy feel like data loss. The draft's "preserve as a duplicate" stays withdrawn: RFC 4791 §4.1 makes it unuploadable |
| 3 | External mode's future | Kept, unchanged | The mapper and the UI stay dual-capable. Retiring it is a separate decision that does not block sync, and it ships and works today |
| 4 | iCalendar library | None — a hand-rolled content-line model. ⚠️ Overturned during chunk 1; SYNC.md § Libraries assumed a typed library was the only option |
A raw (name, params, value) content-line tree is what the unknown-property requirement actually wants: a typed model normalises away exactly what must survive. ICalendarWriter already folds in 204 lines, lib-recur is already a direct :app dependency and does RRULE, and java.time is native at minSdk 29 — so ical4j's 2.2 MB of duplicated zone data, its production-exhausting ZoneRulesProvider, its mandatory properties/cache/registry shims and its release-only R8 failure buy nothing. VTIMEZONE bodies are already on the round-trip allowlist, so they round-trip opaquely. Cost accepted: we own the lexer. biweekly was rejected on the same ground as before |
Scope boundaries
Not in this branch, deliberately:
- Extraction into floret-kit.
SYNC.md§ What lands in floret-kit is right thatcore-davandcore-syncbelong there and that Calendula needs the same primitives. Extracting across a submodule boundary while the API is still moving costs more than it buys. Mitigation, which is a real constraint on every chunk: the DAV and iCalendar layers take no task-domain type and no Android UI type, so the later extraction is a file move. Chunk 5 records what moves. - Retiring External mode, per decision 3.
- A release. Nothing here bumps
versionCode/versionName. Sync ships when it works against real servers, and that is a decision taken after the device review, not inside this plan.
In scope but not verifiable by CI: everything that needs a real server. The
per-server trap matrix (SYNC.md § Server reality) is written as named,
skipped integration tests in chunks 3–4 and turned on by hand against a live
Nextcloud/Radicale/Baïkal in chunk 5.
The five chunks
| # | Chunk | Rests on | The thing it must not get wrong |
|---|---|---|---|
| 1 | Mapper — VTODO ↔ Room | nothing | Unknown properties survive a read-modify-write cycle |
| 2a | Vendored DAV foundation | nothing | Vendor a tree we can actually maintain, and know which of its defects are real |
| 2b | Discovery and auth protocol | 2a | The two collection filters are inversions of the obvious rule |
| 2c | The Android account layer | 2b | Registering the sync adapter, or every ContentResolver sync call is a silent no-op |
| 2d | Account-add UI | 2c | Never send a credential into an unvalidated redirect chain |
| 3 | Engine core — read, write, conflicts | 1, 2 | A failed resource must not fail the collection |
| 4 | Incremental sync + scheduling | 3 | Never persist a sync token before the bodies it covers are applied |
| 5 | Hardening, compliance, real servers | 1–4 | Ship no licence violation, and no retry loop on a dead app password |
Chunks 1 and 2 are genuinely independent of each other; the ordering between them is arbitrary and 1 goes first only because it is the one with a specification that can be written down before any network exists.
Chunk 1 — the mapper
Goal: a bidirectional VTODO ↔ Room-entity mapper whose specification is its
fixture corpus. Today ICalendarWriter is the export half only — write-only,
hand-rolled, and it silently drops everything it does not model.
The corpus comes first. Not as a testing courtesy: SYNC.md § Unknown
properties is explicit that the fixtures that catch bugs are the adversarial
ones, and a mapper written before its corpus gets a corpus that ratifies it.
Minimum set, each one a named failure the design predicts:
- a
CLASS:CONFIDENTIALtask as Nextcloud rewrites it out of a shared calendar — the one that destroys data on write-back - a master plus
RECURRENCE-IDoverrides in one resource - unknown properties nested inside a
VALARM— the case a flat property-per-row model cannot represent - an entirely unknown sub-component
- a
TZIDthe device tzdb does not know - a UID containing
/and@ - one exceeding
CALDAV:max-resource-size RRULE+DUEwith noDTSTART, which has no well-definedRECURRENCE-IDX-MOZ-LASTACK/X-MOZ-SNOOZE-TIME(dropping them causes Thunderbird alarm storms) andX-APPLE-SORT-ORDER- all four recurring-completion models on read (a/b/c/d — we write (a))
Round-trip assertion, stated the way SYNC.md corrects it. Byte-stability is
unachievable and asserting it produces a corpus that gets normalised until it
tests nothing. Re-parse both sides into a canonical multiset of
(component path, property name, params as a sorted map, unfolded unescaped value)
and assert equality modulo an enumerated allowlist: PRODID, DTSTAMP,
LAST-MODIFIED, SEQUENCE, VTIMEZONE bodies, fold positions, parameter quoting.
Byte-equality is asserted where it means something — the unfolded, unescaped
value octets of every untouched property, and full parameter preservation
including unknown parameters.
Storage for the unknown half. A new table, and therefore a schema
migration to v2 with an exported schema and a MigrationTestHelper test, per
the repo's existing pattern in app/schemas/. It must represent nesting
(component path), ordering, and opaque unknown sub-components. Cap the total
at ~25 kB per resource, as DAVx5 does, for CursorWindow and
CALDAV:max-resource-size both.
Deliverables
- fixture corpus under
app/src/test/resources/vtodo/, with a README naming what each fixture is for VTodoParser/VTodoSerialiserbehind one interface, no Android types- unknown-property table, Room migration v1→v2, migration test
- the
SYNC.md§ The rest of the minefield rules as individual tests:SEQUENCEis preserve-verbatim;DTSTAMPregenerates butLAST-MODIFIEDdoes not;COMPLETEDis UTC-only;RELATED-TO;RELTYPE=PARENTpoints up;PRIORITY0 = undefined; aTRIGGER;RELATED=ENDalarm withoutDUEis invalid and must be caught before PUT, because ordinary UI actions reach it ICalendarWritereither becomes the serialiser or is deleted in its favour — two VTODO writers in one app is the defect this chunk exists to prevent- ical4j's
ical4j.properties,MapTimeZoneCacheand registry shim, plus its R8 keep rules (-keep class net.fortuna.ical4j.** { *; }+ the-dontwarnset), because that failure isrelease-only
Also settle here: the VALARM ownership collision. Either the adapter owns
alarms and AndroidTasksDataSource.setAlarm stops bulk-deleting property rows,
or reminders stay local-only and are never serialised. Picking neither is how one
local reminder edit destroys every server-side alarm on that task.
Done when: the corpus round-trips, the migration test passes both directions, and no fixture loses an unknown property.
Settled while building it
- ⚠️ No schema migration was needed.
TaskEntity.unknown_propertiesalready exists at v1, reserved for exactly this and written by nothing. The residue fills it. It holds leftover properties followed by whole sub-component blocks —BEGIN/ENDlines are content lines like any other, so nesting and unknown sub-components fall out of the same representation, which is what the plan thought needed a second table. VALARMownership: neither side writes the other's alarms.VALARMs round-trip in the residue and are never authored; local reminders live intask_alarmsand are never serialised. The collisionSYNC.mddescribes wasAndroidTasksDataSource.setAlarmbulk-deleting provider property rows — in the own-store world the two stores are disjoint and neither can destroy the other. Merging them is a later decision; the point is that it is now a decision, not an accident.- The mapper claims a property only when it can reproduce it exactly.
Everything else stays in the residue: a value it cannot parse or that is out of
range (
PRIORITY:11,PERCENT-COMPLETE:150,SEQUENCE:x,STATUS:X-DEFERRED), a time it can read but not reproduce (floating, or aTZIDthis device's tzdb lacks), and aDTSTART/DUEpair that disagrees on value type or timezone — oneis_all_dayflag and onetimezonecolumn cannot author both, and flattening the odd one out destroys it. Clamping or defaulting any of these would be a silent rewrite of somebody's data.SUPPRESSED_BY_RESIDUEmakes "present in the residue" mean "do not author this". - ⚠️ Suppression is conditional, or editing breaks. A residue property
suppresses its column only while the two still agree. If the user edits a field
whose stamp was unreproducible, the stale residue copy is evicted and the
column is authored — otherwise changing the due date of an imported task would
silently do nothing on the server.
RELATED-TOis the deliberate exception: a parent we have not fetched is not the same as no parent, so an unresolvable link is kept rather than destroyed. - One normalisation is allowed beyond the stated allowlist, and it is an
equivalence rather than a concession: an absent
STATUSandSTATUS:NEEDS-ACTIONmean the same thing for a to-do. Any otherSTATUScompares exactly, so losing a completion still fails the corpus.
Chunk 2a — the vendored DAV foundation
⚠️ Split out of chunk 2 while building it. Vendoring is a self-contained piece with its own verification — upstream's suite has to pass unmodified before anything is built on top — and bundling it with discovery and auth would have produced one commit nobody could review.
Version: dav4jvm 2.2.1, the last OkHttp release. 3.0.0 deleted the OkHttp
package for Ktor and 4.x requires Java 21, and SYNC.md's entire auth section is
written in OkHttp terms — preemptive Basic via an Interceptor, OkHttp stripping
Authorization on cross-host redirects, BasicDigestAuthHandler because OkHttp
has no Digest. Taking 4.x would have invalidated all of it. A plain JVM module,
not an Android library: the tree has zero Android imports and must keep it that
way, or the floret-kit extraction stops being a file move.
Full record in dav/PROVENANCE.md. Of the two defects
SYNC.md said we would inherit, one was not real — 2.2.1 does follow 303 —
and the other (dav4jvm#209, permanent redirects never reaching the caller) is
fixed. A third, which the audit did not have, was found by writing the tests
upstream never wrote: HTTP dates were parsed and formatted in the device's
local zone, putting every getlastmodified out by the local UTC offset.
Done when: upstream's 63 tests pass unmodified, commons-lang3 and the xpp3
runtime dependency are gone, and :app links against the module.
Chunk 2b — the discovery and auth protocol
⚠️ Split again while building it. The seam is the platform: everything here
is protocol, testable on a JVM against MockWebServer, and none of it touches
Android. Keystore, AccountManager, Custom Tabs and the account-add UI are a
different kind of work with a different kind of verification — including the
on-device review — so they are chunk 2c.
Lands in a new :caldav module: MIT, plain JVM, api-depending on :dav. A
separate module from the vendored MPL tree keeps the licences unmixed and makes
"no Android types" a compile-time guarantee rather than a discipline.
One dependency added: dnsjava 3.6.3 (BSD-3) for the SRV/TXT half of
RFC 6764. Android's own DnsResolver is callback-only and cannot do the
TXT path= lookup, and JNDI's DNS provider does not exist on Android at all.
It is what DAVx5 uses. Behind a DnsResolver interface so the pipeline is
testable without a network.
Done when: the trap table is executable — Posteo's SRV-only 8443, GMX's
TXT path=, null SRV targets, Google's dead-end record, SOGo's triple
resourcetype, Nextcloud's trashed calendar, the absent/empty component set, and
<D:unauthenticated/> as a 200 that means failure.
Chunk 2c — the Android account layer
Everything that needs the platform and nothing that needs a screen: Keystore
credentials, AccountManager, the stub sync adapter, the manifest, and the
network security config. The account-add UI is 2d — it is a design task, it
needs the material-3 skill, and it is the piece that needs an on-device review.
⚠️ A stub ContentProvider turned out to be required
Not in the original plan, and it is load-bearing. A sync adapter registers
against a content authority, and Agendula publishes no provider — :provider
was deleted when we took our own Room store. With no authority there is nothing
for <sync-adapter android:contentAuthority> to name, nothing for
requestSync to address, and nothing for Settings to render a switch against.
SyncStubProvider stores nothing and exists solely to hold up that end.
Other things settled here
- The account type and authority are per build variant, generated by
resValue. Two installs cannot own the same account type, so a debug build sharing the release build's would fight it.SyncContractTestasserts the Kotlin constants and the generated strings still agree — drift between them is invisible at build time and surfaces as an account the framework will not trigger. - The credential blob is the one thing excluded from backup. It lives in its own DataStore file for exactly that reason. Keystore keys are non-exportable, so a restored ciphertext is permanently undecryptable; excluding it means the user signs in again, which is the honest outcome. Excluding the database or all of DataStore would trade a latent bug for a live one.
- ⚠️ The network security config trusts the whole user CA store, for all
traffic.
SYNC.mdspecifies it, and without it a correctly installed private CA is not trusted at all — which is the self-hosting case. Butbase-configis the widest form: any CA in the user store can intercept the CalDAV connection and read the app password from theAuthorizationheader. Chunk 5's cert4android should replace this block, not sit beside it — its per-connection approval is the narrow version of the same capability. FOREGROUND_SERVICEappears in the merged manifest without being declared, fromwork-runtime. That is not the FGS route: below API 31 WorkManager implements expedited work with a foreground service, and minSdk is 29. Noted in the manifest because it shows in F-Droid's permission diff.
Not run: the instrumented tests here (CredentialStoreTest,
SyncContractTest) compile but have not been executed — CLAUDE.md reserves
device interaction for when it is explicitly asked for. They need a device run
before chunk 2 is done.
Chunk 2d — the account-add UI
One flow, one back-stack entry, as a stepper rather than four destinations — the steps are not independently reachable, and "back" from the browser step means abandoning a server-side flow rather than popping a screen. It hangs off Settings using the same sliding-section pattern Storage → Export already uses.
⚠️ PreemptiveBasicInterceptor was deleted here, not kept. The vendored
BasicDigestAuthHandler already sends Basic preemptively over HTTPS, and it also
does Digest (Baïkal defaults to it and OkHttp has none), caches which scheme
worked, and restricts by registrable domain rather than exact host — which is
what a cross-host home set needs, since iCloud puts the principal on
caldav.icloud.com and the home set on pNN-caldav.icloud.com. Shipping two
implementations of preemptive Basic is the same defect chunk 1 removed when
ICalendarWriter stopped carrying its own folding.
A seam was added for testability, after the review found eight issues in one
untested ViewModel. CalDavGateway and AccountCreator put the network and
the database behind interfaces, so the sign-in state machine — which decides
where five discovery outcomes lead, when a one-shot app password is spent, and
which host a credential is scoped to — is exercised without a server, a
database, a Keystore or an AccountManager.
Still open, and small: the system entry point (Settings → Accounts → Add
account → Agendula) refuses with a message pointing at the in-app flow rather
than driving it. Wiring it means MainActivity handling
SyncAuthenticator.ACTION_ADD_ACCOUNT and answering the
AccountAuthenticatorResponse.
Done when: the flow has had an on-device review and an explicit go-ahead —
CLAUDE.md's rule, and this is the case it exists for.
Chunk 2d — the account-add UI (original outline)
Goal: the app can add a CalDAV account and list its VTODO collections. No syncing yet.
Vendoring, first. A new Gradle module holding dav4jvm + cert4android sources
at our Java 17 target, MPL-2.0 headers intact, with a PROVENANCE.md recording
the upstream commit — the repo already has this pattern from :provider. Fix the
two inherited defects while vendoring, since we now can: 303 is not handled
(RFC 6764 §5 names it explicitly) and issue #209 mutates location in place
so permanent redirects never reach the caller.
Discovery is the nine-step pipeline in SYNC.md § Generic CalDAV
discovery, including every step marked ⚠️ MISSING there. The two filter
corrections are the important part and both are inversions of the obvious:
supported-calendar-component-setabsent means "supports everything." Request properties by name — RFC 4791 §5.2.3 says the property SHOULD NOT come back from an allprop request. Treat empty as all.- Classify on
resourcetypeas an unordered set, with a positive test forCALDAV:calendar— not by excludingschedule-outbox, because SOGo's main personal calendar reportscollection+calendar+schedule-outboxsimultaneously for every non-Apple client, which is exactly what we are.
The discovery traps table (Posteo SRV-only on 8443, GMX/Web.de path=, null SRV
target, Google's dead-end SRV, iCloud/Zoho well-known 401, dav.runbox.com's
HTTPS→HTTP downgrade, <D:unauthenticated/> as a 200 that means failure,
cross-host home sets) becomes a table-driven unit test over recorded responses.
Nextcloud Login Flow v2 with all five corrections: POST form-encoded (a
GET is 405); an explicit User-Agent, because it becomes the app password's
name in the user's security settings and okhttp/4.12.0 defeats the entire point
of the flow; 404 means pending only and anything that is neither 404 nor 200
stops the poll; validate the endpoint origin and refuse an http downgrade
outright; and loginName is not the uid — it is the Basic auth username and
nothing else. Custom Tabs needs the <queries> entry, the ACTION_VIEW
fallback, {token, endpoint, deadline} persisted before launch, and an explicit
"I finished / Cancel" affordance.
Auth is not just Basic. Digest (Baïkal defaults to it, OkHttp has none),
Basic sent preemptively via an Interceptor gated to HTTPS and the account's own
origin, and domain-detection at account-add time for the three servers whose real
error is not "wrong password": Fastmail (app password; Basic plan has no CalDAV),
iCloud (app-specific password, 2FA to mint one), Google (OAuth2-only — refuse
with an explanation, SYNC.md drops it as a target).
Credentials: Keystore AES/GCM/NoPadding, blob in DataStore. Do not call
setUserAuthenticationRequired; do not set setUnlockedDeviceRequired
(breaks background sync). AEADBadTagException and
KeyPermanentlyInvalidatedException mean re-authenticate, not crash. Exclude
the blob — and only the blob — from Auto Backup: a restored ciphertext is
permanently undecryptable.
The account plumbing, which is the part that silently does nothing if skipped.
AccountManager plus a registered AbstractThreadedSyncAdapter whose
onPerformSync enqueues a WorkManager job and waits. Without a registered
adapter, hasAuthorityAccess() makes every ContentResolver sync API a silent
no-op at targetSdk ≥ 34 — no exception, no log, and it passes on a Robolectric
shadow. Manifest: INTERNET, READ_SYNC_SETTINGS, WRITE_SYNC_SETTINGS,
ACCESS_NETWORK_STATE; the authenticator <service> exported and guarded by
android.permission.ACCOUNT_MANAGER (ACCOUNT_AUTHENTICATOR does not exist).
Ship an in-app sync button, since "Sync now" stays greyed out under
userVisible="false". Add androidx.work, androidx.hilt:hilt-work and
androidx.browser to the catalogue, with the HiltWorkerFactory, the removed
default WorkManagerInitializer and an @EarlyEntryPoint for the authenticator.
Also: ship a network-security-config with <certificates src="user"/>, or
a user who correctly installs their private CA is still not trusted.
UI in this chunk needs on-device review before it can be called done — the
account-add flow is the first screen a self-hoster meets, and per CLAUDE.md
nothing UI-facing gets tagged before that review.
Done when: an account can be added against a real Nextcloud and its task lists appear, the account shows in system Settings with a working sync switch, and the discovery trap table is green.
Chunk 3 — engine core
Goal: a full bidirectional sync that is correct but not yet clever — no sync-collection, no scheduling beyond a manual trigger.
Read path. REPORT calendar-query with a VTODO comp-filter and no
time-range, matching DAVx5 — which deliberately does not use RFC 6578 for
tasks, and omits the time-range because some servers return no tasks without a
time at all. Then calendar-multiget in batches, matching returned hrefs
against what was asked for, because real servers reply with responses for
unrelated URLs. Never request calendar-data inside a REPORT that is not
sanctioned to carry it.
Write path. If-None-Match: * on create, If-Match on update and DELETE.
Send Prefer: handling=strict to sabre-based servers — the cheapest single fix
in the whole audit, because it preserves both our bytes and the ETag. Request
Accept-Encoding: identity. Strip W/ and keep a weak flag; if a PUT returns
no ETag or a weak one, discard it and re-fetch.
412 means three different things and they get three code paths: on create it
means the filename is taken (re-fetch, adopt if the UID matches, else regenerate
as a UUID); on update it means the server is newer (decision 2: server wins,
report the discard); on update where the resource is gone it is spec-correct
and not a conflict at all (HEAD to disambiguate — 404 is delete-vs-edit).
Never write back a body whose ETag does not match the hash of what we
downloaded. This is the Nextcloud shared-calendar landmine and it is the single
most destructive bug available in this chunk: CalendarObject::get() strips
VALARM on read-only shares and reduces CLASS:CONFIDENTIAL to a VEVENT-shaped
whitelist that deletes DUE, STATUS, COMPLETED, PERCENT-COMPLETE,
PRIORITY and RELATED-TO — while leaving the ETag untouched.
Isolation, in two layers. A failed collection must not fail the account, and
— its twin, which the draft omitted — a failed resource must not fail the
collection. Per-resource quarantine with a failure counter, not backoff: a
single HTTP 400 has halted all calendar sync in DAVx5 for weeks. 507 MUST NOT be
auto-retried (RFC 4918 §11.5), and 5xx is not safely retryable either — a
contradictory RRULE/EXDATE pair returns 500 from Nextcloud and will do so
forever.
Client-side validation before PUT, because sabre returns 415 for things
ordinary UI actions produce: DUE before DTSTART, value-type mismatch between
them, a present METHOD, multiple UIDs, mixed component types in one resource.
href ≠ UID. Sanitise filenames on vdirsyncer's rule — a–zA–Z0–9_.-+,
excluding @ — cap the basename near 200 bytes, fall back to a UUID.
DELETE: conditional on If-Match, 404/410 count as success, and a locally
deleted resource that was never uploaded is never DELETEd.
Local change detection is is_dirty = 1 OR is_deleted = 1 scoped by list —
the columns already exist in the v1 schema. Every downstream insert writes
is_dirty = 0 explicitly.
Done when: a task created in Nextcloud web appears locally and vice versa; a concurrent edit resolves server-wins with a visible report; a quarantined resource does not stop its collection.
Settled while building it
⚠️ The mapper emitted TZID with nothing to resolve it. Chunk 1 writes
DTSTART;TZID=Europe/Berlin:…, and RFC 5545 §3.2.19 requires a TZID without a
leading solidus to reference a VTIMEZONE in the same object. Nothing
generated one — VTIMEZONE sits at VCALENDAR level, outside the VTODO whose
residue the mapper stores, so it was outside the round-trip guarantee entirely.
Left alone, every zoned task would have been malformed, and Prefer: handling=strict — which chunk 3 sends precisely so servers stop repairing our
bytes — turns malformed into rejected. VTimeZones now generates the definition
from java.time's own rules, which also keeps the zone we write in agreement
with the zone we compute occurrences in. Cost accepted: only the currently
effective rules are emitted, not the historical transition table. Tasks are
dated now or later.
java.time does not encode "the last Sunday" as -1. It arrives as
dayOfMonthIndicator = 25 — "the first Sunday on or after the 25th" — and only
the fact that a seven-day window ending on the last day of the month is the
last occurrence identifies it. The obvious -1 test never fires for any European
zone, and the fallback emits a seven-value BYMONTHDAY list where every other
client writes BYDAY=-1SU.
GetETag.eTag has already been parsed. dav4jvm strips the W/ marker and
records it in a separate field, so feeding its value back through an ETag parser
reports every weak tag as strong — the exact failure the weak flag exists to
prevent. ETag.from(property) and ETag.parse(header) are now different
functions for that reason, and a test pins the difference.
The discard is reported when the replacement arrives, not when the 412 does.
Announcing it at the conflict — and clearing is_dirty there — makes the loss
real if the replacement download then fails, with nothing left to retry. The row
now stays dirty until the download overwrites it.
A SyncStore seam, not Robolectric. The reconciliation is where local edits
are discarded, tombstones swept and conflicts resolved; proving that needs a test
double for eight methods, not an in-memory Room plus a new test runtime.
The mutilation guard is scoped to what is actually reachable. Nextcloud
reduces CLASS:CONFIDENTIAL objects served from a share while leaving the ETag
intact; VALARM stripping happens on read-only shares, which are never written
to anyway. So the refusal is: shared collection + confidential + already on the
server. The collection's read-only and shared flags are re-read on every sync
rather than trusted from account-add, because ACL churn is silent.
Manual trigger. ContentResolver.requestSync is gated behind
hasAuthorityAccess() at our targetSdk and returns silently when it refuses, so
the in-app button enqueues the WorkManager job directly via SyncTrigger. The
sync adapter uses the same path.
Found by the review, and worth naming
⚠️ An empty listing would have hard-deleted the whole list. The sweep took
"absent from remote.list()" as proof of a server-side delete, and its evidence
is a calendar-query with a VTODO comp-filter — the filter whose mishandling
is the reason the query carries no time-range in the first place. A server
answering it with an empty successful multistatus is indistinguishable from an
empty collection, and one such answer destroyed every task in that list in one
pass. An empty listing now never sweeps. Cost accepted: a collection
genuinely emptied on the server keeps its local rows until one task reappears
there. That is recoverable by hand; the other error is not.
Quarantine did not cover creates. Failures on a resource with no href yet
were counted under its UID and read back under its href, so nothing ever read
them: a new task the server rejects permanently was re-PUT on every sync forever
— the exact DAVx5 failure the counter exists to prevent — while the unread keys
grew in DataStore without bound. Resources are now keyed by href or uid:…,
and a success clears both.
A RELATED-TO deleted on the server never unparented anything. Only present
parents were recorded, and upsert carried the old parent_id forward — which
the writer then resolved straight back into a RELATED-TO, re-uploading the link
the user had deleted elsewhere.
Two validators disagreed about what a DATE is. ResourceValidator tested
only VALUE=DATE while the mapper also reads a bare eight-digit value as one, so
a residue DTSTART:20260101 beside an authored DUE;VALUE=DATE:20260102 was
rejected locally and never left the device — a case VTodoMapperTest already had
a test for. The per-component rules now delegate to VTodoMapper.validate, which
also restores the TRIGGER;RELATED=END check that the new validator had silently
dropped and left as dead code.
An unresolvable TZID produced a resource with no definition for it. A
Windows zone name from another client survives in the residue, and java.time
cannot regenerate a VTIMEZONE for it — so with Prefer: handling=strict the
upload becomes an unexplained rejection. It is now refused by name.
Open for chunk 5: preserving the server's own VCALENDAR-level VTIMEZONE
would fix it properly, and needs somewhere to put residue that is not the
VTODO's.
Three narrower ones, all fixed: quarantine counts were a global
read-modify-write, so two accounts syncing at once discarded each other's
(now merged inside edit); NeedsSignIn and Misconfigured never called
recordSync, leaving the accounts screen reporting "synced 5 minutes ago" for an
account that cannot sync at all; and apply() re-read every row in the list once
per downloaded resource.
Chunk 4 — incremental sync and scheduling
Goal: sync-collection as the optimisation on top of chunk 3's baseline, and
sync that runs by itself.
Every numbered finding in SYNC.md § RFC 6578 is a test in this chunk. The
four that are not optional:
- Invalidation has no status code. Ignore the status; match
<D:valid-sync-token/>anywhere in the body on any 4xx. Observed in the wild: 403 (sabre), 400 (Google), 409 (Radicale), 412 (Evolution accepts it). Thunderbird accepts only 400 and therefore never recovers from the 403 most of the self-hosted world emits. - Initial sync must not report deletions, so mark-and-sweep is mandatory and
initialIncompletemust be persisted atomically with the token — otherwise a resumed partial sync sweeps against an incomplete set and deletes live data. - Persist the token only after the bodies are applied, per page. The RFC's own Appendix B has this backwards. Under WorkManager, process death mid-sync is routine.
- A token the server accepts over a change log it already pruned returns 207,
zero changes, and no error. RFC 6578 gives no signal for it. The only
mitigation is periodic full reconciliation (PROPFIND
Depth: 1+ ETag diff) on a slow cadence regardless of the token — so chunk 3's full path is a permanent safety net, not a fallback.
Plus: no DAV:limit (Nextcloud regressed it to a localised HTML error page);
507 on the SELF href is truncation, 507 as the outer status means retry without
the limit; an iteration cap and a no-progress guard, because the RFC never
requires the token to advance; tokens are opaque and never reused across
collections, keyed (accountId, collectionUrl); deletion is
<D:status>404</D:status> at <D:response> level, not inside a propstat; a
207 with no <D:sync-token> degrades to PROPFIND rather than throwing; and
sync-collection never reports collection property changes, so displayname,
colour and read-only status refresh only via PROPFIND on the home set.
Three membership edge cases that each look like a bug: create-then-delete between syncs is reported as removed (the delete handler no-ops on an unseen href); delete-then-recreate at the same URI is reported as changed (re-read the UID from the body); and ACL churn may be reported as removal, so apply a sanity threshold before acting on a large delete batch.
Scheduling has a ceiling. Make sync chunked and resumable — persist the
cursor per collection so a killed worker resumes rather than restarts. Periodic
sync is a plain PeriodicWorkRequest, no foreground service. "Sync now" from a
visible screen uses setExpedited(RUN_AS_NON_EXPEDITED_WORK_REQUEST), and
getForegroundInfo is implemented unconditionally — omitting it crashes
below API 31 and we support 29. Android 15 forbids starting a dataSync FGS from
BOOT_COMPLETED, and we have a boot receiver. Sync hard on app open and on
connectivity-regained, and be honest in the UI about cadence: in the rare and
restricted buckets network access is off entirely and the worst case is
genuinely "once overnight".
Done when: a token invalidation on any of the four status codes recovers; a process kill mid-page loses nothing and duplicates nothing; the periodic worker runs and the manual trigger is instant.
Settled while building it
initialIncomplete turned out not to need a column — or to exist. The plan
called for it to be persisted atomically with the token, to stop a resumed
partial sync sweeping against an incomplete set. But the mark-and-sweep it
protects is chunk 3's full path, so the rule that removes the whole hazard is
simpler: a token is adopted only when a full reconciliation completes. A run
killed part-way stores no token and the next one starts over. No second column,
nothing to write atomically with anything.
The token is taken from the PROPFIND that precedes the read, not after it. Both are imprecise; only one is imprecise safely. A token minted after the listing silently swallows everything that changed while we were reading it; one minted before merely re-reports those changes next time, and a re-report costs an ETag comparison.
Where the cadence clock lives. task_lists.sync_token is domain state and
stays in Room. "When did this collection last reconcile in full" is scheduling
bookkeeping, so it is in DataStore (SyncCadenceStore) — and deliberately
outside any backup, because a restored "checked recently" on a fresh install is
exactly the stale trust the full path exists to break.
⚠️ ForegroundInfo must carry no foregroundServiceType. Implementing
getForegroundInfo is mandatory — setExpedited falls back to a foreground
service below API 31 and the default implementation throws, so a worker without
it crashes on every API 29/30 device. But passing
FOREGROUND_SERVICE_TYPE_DATA_SYNC re-introduces the entire tail the design
rejected: the FOREGROUND_SERVICE_DATA_SYNC permission, Android 15's
six-hours-per-24 budget with its fatal RemoteServiceException, and a Play video
demo per declared type. Lint caught it, correctly. The type is unnecessary
anyway: above API 30 setExpedited uses an expedited job and never calls
getForegroundInfo, and types only became mandatory at API 34.
No boot receiver for sync. WorkManager restores its own schedule across a
reboot, and Android 15 forbids starting a dataSync FGS from BOOT_COMPLETED —
so a design that needed one would have had no way to run. App open calls
rescheduleAll() instead, which is idempotent under KEEP and repairs the one
case WorkManager cannot: a schedule lost to "clear app data" or to a restore.
Expedited is spent on the button and nowhere else. The quota is per-app and exhaustible; a background trigger that spends it leaves none for the trigger the user is actually watching.
Found by the review, and worth naming
⚠️ The cadence clock was in the backed-up DataStore — while its own KDoc, and
this document, claimed it was not. Auto Backup includes datastore/, and only
agendula_credentials was excluded. Restored onto a new device it would say
"reconciled minutes ago", so the restored sync token would be trusted for another
day: exactly the silently pruned change log the full path exists to catch. Both
sync-state stores now live in their own agendula_sync_state file, excluded
alongside the credentials — a restored quarantine count is the same class of lie,
silently skipping resources that were never tried on this device.
The incremental download path had no write-phase guard. downloadPhase skips
a row that is still dirty; downloadChanged did not. A share demoted to
read-only leaves the upload refused and the row dirty, and the next change-log
entry for it overwrote the user's unsent edit with no DiscardedEdit at all. The
guard cannot simply be copied, either — the 412 path deliberately leaves the
row dirty and waits for the download — so it is "dirty and not awaiting a
refetch".
A change log describes the past, and applyRemovals treated it as the
present. An entry predating this run's own upload deleted the row we had just
written, leaving an orphan on the server and nothing on the device. It now
respects touched, and reports a lost edit the way the sweep does.
An abandoned incremental attempt poisoned a successful run. ChangeSet.Failed
and the mass-removal refusal both set report.failure, which the full fallback
never cleared — so a server answering sync-collection with a bare 400 refused
the new token, skipped the cadence record, and showed "last sync didn't finish"
on every other sync while nothing was wrong with the data. A fallback that
succeeds now demotes the attempt to incrementalNote.
refetch was silently dropped on a successful incremental run, so a PUT
accepted without a strong ETag never got its canonical body back on a server that
does not echo our own writes into its change log.
Two of my own patches were wrong in ways the compiler could not see. fatal()
was written with its KDoc and never called — a python patch aborted on its
second assertion after the first had already matched, so nothing was written, and
the function sat there describing a bug it was not preventing. And the
delete-then-recreate lookup read the href index after the same index had been
re-pointed at the rows just written, so it never found the displaced row. The
first was caught by the review, the second by a test.
Two smaller ones: the displaced lookup reintroduced the per-resource
full-table scan the comment twelve lines above it forbids, and MainActivity
re-ran the app-open sync on every rotation, theme switch and locale change.
Chunk 5 — hardening, compliance, real servers
Goal: the things that are not features and are not optional.
The server matrix, turned on. Nextcloud, Radicale, Baïkal (on Digest),
SOGo, Fastmail, iCloud — each per-server trap from SYNC.md § Server reality
as a named test. Two must be explicit because both silently destroy data:
round-tripping a task from a shared Nextcloud calendar, and an
ETag-unchanged body change on SOGo (its ETag is a row-version counter and the
body is regenerated per principal). Also: Nextcloud's per-calendar UID uniqueness
(409 no-uid-conflict), its trashbin renaming hrefs to <name>-deleted.ics so
delete→recreate→delete returns 403, MKCALENDAR rate-limited 10/hour; SOGo never
invalidating a token and using second-granularity ones; Radicale never enforcing
supported-calendar-component-set. Feature-detect collection creation via
OPTIONS and disable the "new task list" UI where neither MKCALENDAR nor
extended MKCOL exists (iCloud is MKCOL-only, Google has neither, Posteo
disables it).
Revocation, both directions. On 401: stop that account immediately, mark
NEEDS_REAUTH, notify with a deep link into the login flow, and do not retry
on a timer — Nextcloud's brute-force protection throttles then 429s per source
IP, so a retry loop on a dead app password takes down the user's other Nextcloud
clients and looks like we broke their server. Distinguish 401 from 403 from
429/503 (honour Retry-After). On account removal: best-effort
DELETE /ocs/v2.php/core/apppassword with OCS-APIRequest: true, or
uninstalling never revokes access.
The backup path, which arms itself the day this ships. Post-restore
reconciliation must offer to re-attach the account, never prune. Pruning is
event-driven off AccountManager's account-removed broadcast, not off "absent
from the visible set". Device-verify by restoring onto a fresh device and
confirming nothing is deleted — SYNC.md is explicit that excluding the database
from backup is the wrong fix, because Auto Backup is currently Local mode's only
safety net.
Licence attribution — a plain violation until it exists. MPL-2.0 §3.2(a)
(dav4jvm, cert4android) requires telling recipients how to obtain source; §3.4
requires retaining file headers; BSD-3 (ical4j) requires reproducing the
copyright notice in binary distributions; Apache-2.0 §4(d) propagates NOTICE.
Settings exposes only our own MIT LICENSE today. Note ical4j's POM declares a
non-SPDX licence name and a LICENSE URL that 404s, so generators produce empty
output — this one is written by hand.
Play compliance. A privacy policy, linked in Console and in the app.
File as Collected, not Shared, encrypted in transit — "not collected" is not
defensible, because Play defines collection as transmitting off-device
irrespective of recipient. Ship a "Remove account and delete local data" action
even though the Account Deletion policy does not apply. Do not ship
REQUEST_IGNORE_BATTERY_OPTIMIZATIONS in the Play build — generic server sync
is not on the acceptable-use list; use
ACTION_IGNORE_BATTERY_OPTIMIZATION_SETTINGS.
Minified build. :app runs R8 + resource shrinking in release, and
proguard-rules.pro already documents two pruning outages. Verify on the
releaseTest build type, on device.
Record the extraction. A short section listing what moves to floret-kit
(core-dav, core-sync, Login Flow v2, credential storage, the licence screen)
and what stays app-local (the mapper, conflict policy and its UI, storage modes),
so the follow-up branch is a file move and not a rediscovery.
Done when: the matrix is green where a server exists to run it against, the
attribution screen ships, and a releaseTest build syncs on a real device.
What shipped, and what is still owed
Shipped in the chunk-5 commit — everything verifiable without a device or a live server:
- Attribution.
Settings → Open source licenses, hand-maintained. Not a generated list: POM metadata routinely names a non-SPDX licence and points at a URL that 404s, so generators produce confidently empty output. dav4jvm is vendored, which makes MPL-2.0 §3.2(a) ours rather than a dependency's — the row namesdav/PROVENANCE.mdalongside upstream, because §3.2(a) is about where a recipient of the binary can obtain the source of what we modified. - Revocation, both directions. A 401 escalates from the collection to the
account, marks it, cancels its periodic work, and stops. Cancelling is the
part that matters: Nextcloud throttles and then 429s per source IP, so a timer
on a dead app password degrades every other Nextcloud client on that network.
On removal,
DELETE /ocs/v2.php/core/apppasswordwithOCS-APIRequest: true, best effort — without it, uninstalling never revokes anything and the password outlives the app in the user's device list. - Play compliance. A privacy policy in
docs/PRIVACY.md, linked in the app (the half that is usually missed) and declaring Collected, not Shared — "not collected" is indefensible when Play defines collection as transmission off-device regardless of recipient. A "delete this account's tasks from this device too" option on removal.REQUEST_IGNORE_BATTERY_OPTIMIZATIONSwas already absent and stays absent. - The server matrix, as far as a protocol mock can carry it: Nextcloud's
no-uid-conflict, its trashbin 403 on a reused href, and share detection; SOGo's second-granularity opaque tokens and its triple resource type; Radicale advertisingsync-collectionwithout implementing it.
Still owed, and blocked on things a loop cannot do:
- The live matrix. Four tests are
@Ignored with their reasons written out rather than deleted: Baïkal on Digest, aCLASS:CONFIDENTIALtask on a shared Nextcloud calendar, an ETag-unchanged body change on SOGo, and Nextcloud's MKCALENDAR rate limit. Each names why a mock cannot stand in for it. releaseTeston device, and the R8 outagesproguard-rules.proalready documents.- The restore path, which needs a restore onto a fresh device — the one
verification
SYNC.mdcalls for explicitly and the one that cannot be faked. - cert4android, which replaces the current blanket user-CA trust and is a UI-bearing dependency.
- Collection creation feature detection (OPTIONS → MKCALENDAR / extended MKCOL) and disabling the "new task list" affordance where neither exists. Deferred because the app has no CalDAV-side list-creation UI yet, so there is nothing to disable — it becomes real the moment that UI does.
Found by the review, and worth naming
⚠️ Revocation was aimed at the principal URL and revoked nothing. The OCS
endpoint sits at the server root; the principal is
…/remote.php/dav/principals/users/alice/, so appending to it produced a path
that 404s on every server, every time — and the failure was swallowed as
best-effort. The whole feature was a no-op. The root is now derived by cutting at
remote.php, which is right for a subpath install too, with the origin as the
fallback.
⚠️ Opening the app resurrected accounts a 401 had just stopped.
ExistingPeriodicWorkPolicy.KEEP only keeps work that is unfinished, and
CANCELLED counts as finished — so rescheduleAll() re-enqueued the very request
the stop had removed, and put a dead app password back on a four-hour loop
against a server that throttles by source IP. Exactly what the stop exists to
prevent.
⚠️ The cursor advanced past bodies that were never applied. A network drop
mid-multiget set report.failure and returned, and the incremental path stored
the new token anyway — a permanent hole in the collection, invisible until the
next full reconciliation a day later. runFull had this guard from the start;
the incremental path did not.
Stopping an account cancelled the worker doing the stopping. syncTrigger.cancel
kills both unique names, one of which is the WorkSpec currently executing —
WorkManager interrupts the coroutine, so NeedsSignIn was never returned and the
adapter saw CANCELLED rather than FAILED. The flag now does the work: sync()
refuses before touching the network, and the schedule is cancelled from outside
any worker.
"Sign in again" was a dead end. create is the only path that writes a
credential and it rejected the duplicate name, so a user with a revoked app
password could only remove the account — discarding the choice to keep its lists.
It now re-authenticates in place, but only for an account already marked as
needing sign-in, so a healthy credential can never be silently overwritten.
Four scalars were emitted twice. SEQUENCE, PRIORITY, PERCENT-COMPLETE
and CLASS reach the residue verbatim when unparseable, and write authors them
from their columns unconditionally — so a task read from SEQUENCE:x round-tripped
carrying both SEQUENCE:0 and SEQUENCE:x. Invisible to the corpus, which
filters SEQUENCE out of comparison entirely. Now suppressed, with a test that
counts occurrences rather than comparing canonically.
A solidus-prefixed TZID was rejected as undefined — the rule's own comment
says §3.2.19 exempts it. Thunderbird writes
/mozilla.org/20050126_1/Europe/Berlin on every zoned task, so the validator
would have quarantined every Lightning-authored task permanently.
Two smaller ones: a resource the query lists but the multiget will not return was re-requested for ever with nothing counting it, and account removal blocked on a 30-second connect timeout before anything visible happened.
What moves to floret-kit
Recorded here so the follow-up branch is a file move rather than a rediscovery. The rule that made it a move rather than a rewrite held: no task-domain type and no Android UI type crossed into the DAV or CalDAV layers.
| Moves | Why it generalises |
|---|---|
:dav (vendored dav4jvm) → core-dav |
Not app-specific in any way; Calendula needs the identical tree |
:caldav — discovery, CalendarCollection, sync-collection, quirks |
Pure protocol, plain JVM, no Android types |
NextcloudLoginFlow |
Nothing about it is task-shaped |
CredentialStore, CalDavAccounts, SyncAuthenticator |
Keystore + AccountManager plumbing, identical for any CalDAV client |
LicencesScreen / Attribution |
Every app in the family owes the same notices |
| Stays app-local | Why |
|---|---|
VTodoMapper, CalendarResource, ResourceValidator |
VTODO ↔ our Room schema; the residue contract is ours |
CollectionSyncer, SyncReport, conflict policy and its UI |
Encodes decision 2, which is a product decision |
SyncEngine, SyncWorker, SyncTrigger |
Bound to our entities and our scheduling choices |
| Storage modes, External mode | Agendula-specific by construction |
Definition of done, every chunk
- It builds:
./gradlew :app:assembleDebug. - Unit tests pass:
./gradlew :app:testDebugUnitTest :dav:test :caldav:test. ⚠️ Name the plain JVM modules explicitly —testDebugUnitTestis an Android-variant task and does not exist on them, so their suites would otherwise be compiled by nobody and run by nobody. Add each new module here and to CI. - Lint passes:
./gradlew :app:lintDebug. Added after chunk 1 shipped a literal byte-order mark that builds and tests both accepted and CI's lint step would have rejected. /code-review highon the chunk's diff, with the findings either fixed or answered in the commit message. For a chunk that vendors code, scope the review to the files we wrote — reviewing thousands of lines of verbatim upstream is noise, and its defects belong inPROVENANCE.md.- One commit per chunk, referencing the chunk number. No release, no tag.
Chunk 2 additionally does not count as done until the account-add UI has had an
on-device review with an explicit go-ahead — CLAUDE.md's rule, and this is
exactly the case it exists for.
What stops the loop
- A
SYNC.mddecision that the evidence overturns. Say so; do not pick the other branch silently. - Anything requiring a real server that is not reachable — write the test, mark it skipped, name it in the commit, move on.
- Anything that would need a device beyond
adb install.