The document was written against the vendored provider and still said the storage question was settled that way. Owning the store answered several of its open questions and deleted others outright, so the corrections are marked inline the way the rest of the file marks them, rather than quietly rewritten. Closed: the Local->Synced migration (account_id is a nullable FK, so attaching an account is an UPDATE), the recurring-completion model (the store writes model (a), RECURRENCE-ID overrides sharing the master's UID), the Auto Backup / cleanUpLists data-loss path, and the lib-recur version trap. Phase 0's UIDs-at-creation and backup safety are shipped. The provider-mechanism table is kept as External-mode history rather than deleted — that code still runs in OpenTasks and tasks.org. Effort restated: 11.5-15 weeks minus the 2.5-4 owning the store removes, so roughly 8-11. Also states plainly at the top that no sync code exists and that ICalendarWriter is the export half of the mapper only.
1244 lines
76 KiB
Markdown
1244 lines
76 KiB
Markdown
# Agendula — CalDAV sync
|
||
|
||
> Design notes for Agendula's own sync adapter. Drafted 2026-08-13; **audited the
|
||
> same day** against the platform, the vendored provider source, and the current
|
||
> state of every library named. The audit refuted or corrected a substantial part
|
||
> of the first draft — the corrections are marked ⚠️ **inline and kept visible**
|
||
> rather than quietly rewritten, because most of them are things the next person
|
||
> would otherwise assume again.
|
||
>
|
||
> This is step 5 of [`STORAGE-AND-SYNC.md`](STORAGE-AND-SYNC.md)'s sequencing.
|
||
> That document decided **where task data lives**; this one decides **how it gets
|
||
> to a server**.
|
||
>
|
||
> Status: **draft / decision document.** No sync code is built — no `dav4jvm`,
|
||
> no `ical4j`, no `AccountManager`, no adapter, nothing in the manifest. Where a
|
||
> question is already answered by shipped code, it is marked ✅ and the code is
|
||
> named.
|
||
>
|
||
> ⚠️ **The storage question this document declared settled was reopened, the
|
||
> answer changed, and the change has since shipped.** Agendula owns a Room store
|
||
> and the vendored provider is deleted — see
|
||
> [`STORAGE-DECISION.md`](STORAGE-DECISION.md) and
|
||
> [`OWN-STORE.md`](OWN-STORE.md). Roughly sixteen of the findings below are
|
||
> **provider-imposed** and went with it; `OWN-STORE.md` § *Effects on the sync
|
||
> plan* lists them item by item. They are kept here, marked, because the External
|
||
> path still runs on that provider and because the next person would otherwise
|
||
> re-derive them. Everything platform-level (the targetSdk 34 sync gate, the stub
|
||
> adapter, credential storage, Play compliance) and everything protocol-level
|
||
> (discovery, RFC 6578, conditional PUT, conflict policy, the server-reality
|
||
> table) is unaffected and remains the plan.
|
||
>
|
||
> **What owning the store already settled, in shipped code** (2026-09):
|
||
>
|
||
> | Was a sync deliverable | Now |
|
||
> |---|---|
|
||
> | Phase 0 — UIDs at creation | ✅ `uid` is `NOT NULL`, minted on insert in both modes |
|
||
> | Phase 0 — backup / prune safety | ✅ backup rules cover the WAL, `ON_STOP` checkpoint, restore test |
|
||
> | Phase 1 — the recurrence representation | ✅ `RRULE`/`RDATE`/`EXDATE` stored raw, expanded at read (`RecurrenceExpander`), `RECURRENCE-ID` overrides sharing the master's UID |
|
||
> | Phase 3 — Local→Synced migration | ✅ **gone**: `task_lists.account_id` is a nullable FK, so attaching an account is one `UPDATE` |
|
||
> | Phase 3 — recurring-completion model | ✅ model **(a)** is what the store writes; the provider's model (d) left with it |
|
||
> | Sync bookkeeping columns | ✅ `href`, `etag`, `sync_token`, `is_dirty`, `is_deleted` exist in the v1 schema |
|
||
>
|
||
> Still nothing but design: phase 1's mapper (`ICalendarWriter` writes VTODO for
|
||
> **export only** — no parser, no unknown-property round-trip), phase 2 auth,
|
||
> phase 3's engine, phase 4 hardening. Phase 0's licence-attribution screen is
|
||
> also not built.
|
||
|
||
Scope: self-hosted CalDAV first (Nextcloud), F-Droid and Play, MIT license.
|
||
|
||
---
|
||
|
||
## The plan, in short
|
||
|
||
| Question | Direction |
|
||
|---|---|
|
||
| Where does task data live? | ⚠️ **Changed since this table was written.** Our own Room store — `:provider` is deleted (`OWN-STORE.md`) |
|
||
| Who syncs it? | Agendula, via its own sync adapter |
|
||
| Account model | `AccountManager` **+ a real (stub) sync adapter** — ⚠️ the hybrid without one does not work |
|
||
| Scheduling | WorkManager, triggered *through* the sync framework |
|
||
| Protocol library | `dav4jvm` (MPL-2.0) — ⚠️ costs more than the first draft assumed |
|
||
| Self-signed certs | `cert4android` — ⚠️ **MPL-2.0, not GPLv3.** The first draft rejected it on a false premise |
|
||
| iCalendar | In-house mapper over `ical4j`; **no `synctools`** (GPLv3). ⚠️ Now VTODO ↔ **Room entities**, not `TaskContract` |
|
||
| Recurrence | Read/write `RRULE`/`RDATE`/`EXDATE` directly — ⚠️ **not** the Instances table |
|
||
| Primary read path | ⚠️ `REPORT calendar-query` (VTODO filter, no time-range); `sync-collection` is the optimisation |
|
||
| Recurring completion | ⚠️ Accept all four models on read; **we write (a)** — the store forks a `RECURRENCE-ID` override sharing the master's UID. The provider's model (d) left with the provider |
|
||
| Sign-in | Nextcloud Login Flow v2 + generic CalDAV discovery + Digest |
|
||
| Conflict policy | `If-Match`; on 412 the server wins, local copy preserved |
|
||
| Reference implementations | jtx Board and DAVx5 — read, never link against (GPLv3) |
|
||
|
||
| # | Phase | Deliverable | Effort |
|
||
|---|---|---|---|
|
||
| 0 | **Groundwork** | ~~UIDs at creation~~ ✅, ~~backup/prune safety~~ ✅, licence-attribution screen ⬜, Java-21 decision ⬜ | ~1 week → days |
|
||
| 1 | Mapper | VTODO ↔ **Room entities**, unknown-property round-trip, fixture corpus. ⚠️ `ICalendarWriter` is the export half only — write-only, and it drops what it does not model | 2–3 weeks |
|
||
| 2 | Auth | Discovery, Login Flow v2, Digest, credential storage, cert trust | 1.5–2 weeks |
|
||
| 3 | Engine | `calendar-query` baseline, `sync-collection` optimisation, full reconciliation, conflicts, scheduling, ~~**Local→Synced migration**~~ ✅ gone — `account_id` is a nullable FK | 4–5 weeks |
|
||
| 4 | Hardening | Per-server trap matrix, error UX, re-auth, Play compliance | 2–3 weeks |
|
||
|
||
⚠️ **Revised upward from the first draft's 8–9 weeks to 11.5–15**, then back
|
||
down. Phase 0 is new; the migration in phase 3 was previously believed not to
|
||
exist at all; and the engine grew a second sync path plus a permanent
|
||
reconciliation pass. Owning the store then deleted 2.5–4 weeks of it
|
||
(`OWN-STORE.md` § *Effects on the sync plan*) — the migration, the recurrence
|
||
representation and phase 0's data work are done — leaving roughly **8–11 weeks**.
|
||
|
||
**Calibration, for sanity:** Evolution shipped RFC 6578 in **June 2026** against
|
||
a request open since 2019. vdirsyncer has declined to implement it for twelve
|
||
years. Thunderbird still carries unlanded patches for one of its error paths.
|
||
This is not a phase that gets shortened by trying harder.
|
||
|
||
---
|
||
|
||
## Settled — the storage question is not reopened here
|
||
|
||
A later working draft re-argued storage as a fresh choice between *bundle the
|
||
dmfs provider* and *a Room-native store with a `TaskContract` facade*, and
|
||
recommended Room. Not adopted; this is the audit trail so it does not come back.
|
||
|
||
The provider shipped in `f978c37`. Three of the arguments against it do not
|
||
survive contact with the repository:
|
||
|
||
1. **"`synctools` is GPLv3 and propagates to the app."** Misattributed.
|
||
`:provider` depends on `jems`, `rfc5545-datetime` and `lib-recur` — all
|
||
Apache-2.0. `synctools` is a *mapper* choice, equally avoidable either way.
|
||
2. **"Cursor access everywhere; the widget pays the Calendula cost."** Calendula
|
||
queries `CalendarProvider` **cross-process**; ours is same-package, same-uid,
|
||
so `ContentResolver` returns the local provider instance — no Binder hop, no
|
||
`CursorWindow` marshalling. The observer→Flow seam already exists.
|
||
3. **"The dmfs schema can't hold what we need."** The `Properties` table takes
|
||
arbitrary mimetypes — `PropertyHandlerFactory.get` falls through to
|
||
`DEFAULT_PROPERTY_HANDLER`, and `PropertyHandler.insert` is a bare
|
||
`db.insert` with no validation, over sixteen real `data0`–`data15` TEXT
|
||
columns. That is the mechanism for unknown-property round-tripping, this
|
||
document's single most important correctness requirement.
|
||
|
||
**But the audit also weakened the positive case.** The first draft claimed the
|
||
provider hands us the sync bookkeeping for free. It hands us *some* of it, with
|
||
sharp edges — see the next section. Room would have been the wrong trade anyway
|
||
(it buys Flow ergonomics we already have and costs the tombstone/instance
|
||
machinery we already run), but "for free" was too generous.
|
||
|
||
---
|
||
|
||
## What the provider actually gives us
|
||
|
||
⚠️ **This section is now history for the sync plan.** Every line was verified
|
||
against `provider/src/main/java/`, which no longer exists — the vendored provider
|
||
is deleted and our own store is what sync will run against. Nothing here
|
||
constrains the adapter any more.
|
||
|
||
It is kept, not deleted, for two reasons: **External mode still talks to exactly
|
||
this code** in OpenTasks and tasks.org, so the app layer still lives with these
|
||
rules; and if External mode is ever retired (open question 3), this is the record
|
||
of what was being given up.
|
||
|
||
| Mechanism | Reality |
|
||
|---|---|
|
||
| `CALLER_IS_SYNCADAPTER` | Works on `tasks` and `tasklists`. ⚠️ **Ignored on the `instances` URI** — `processors/instances/TaskValueDelegate` hardcodes `false` on every delegation |
|
||
| `_DIRTY` on user writes | ⚠️ **Set on insert/update only, never on delete.** `AutoCompleting.delete` skips `updateFields`; `TaskCommitProcessor.delete` sets only `_DELETED` |
|
||
| `_DIRTY` default | ⚠️ **Defaults to `1`** (`TaskDatabaseHelper:456`). Every downstream insert must write `_dirty=0` explicitly or it uploads straight back |
|
||
| `TaskLists._DIRTY` | ⚠️ **A monotonic counter, not a flag** — a trigger does `_dirty = _dirty + new._dirty + new._deleted` and nothing ever decrements it |
|
||
| `_DELETED` tombstones | ✅ Real. Soft-delete for account-backed tasks, hard delete for sync adapters *and* for the local account. Hidden from non-sync queries |
|
||
| Instances | ⚠️ **Materialises exactly one upcoming occurrence** (`UPCOMING_INSTANCE_COUNT_LIMIT = 1`), on write only, never re-expanded as time passes. Useless as a recurrence representation for sync |
|
||
| `_UID` | ⚠️ **Settable by anyone on insert** — restricted to sync adapters on *update* only. Never generated by the provider |
|
||
| `SYNC1`–`SYNC8`, `_SYNC_ID`, `SYNC_VERSION` | Exist and are free — but ⚠️ **`Moving` nulls all of them on a list move while keeping `_UID`**, leaving a live row and a tombstone sharing one UID |
|
||
| Per-collection sync state | ⚠️ **Does not exist.** The `SyncState` table is one blob **per account**, `db.replace`d |
|
||
| Read-only collections | ⚠️ **Cannot be represented.** `ACCESS_LEVEL` is inert — the contract says "not used yet", and `Validating.java:60` still carries upstream's `// TODO: ensure that the list is writable` |
|
||
| Account scoping on delete | ⚠️ The provider **requires** account params on a sync-adapter task delete and then **ignores them** (`TaskProvider.java:793` — upstream `// TODO`) |
|
||
|
||
### The consequences, as rules
|
||
|
||
1. **The upload query is `_dirty = 1 OR _deleted = 1`.** Deleting is not dirtying.
|
||
2. **Every downstream insert sets `_dirty = 0` explicitly.**
|
||
3. **Name the squats now**, because none of these columns exist: CTag →
|
||
`TaskLists.SYNC_VERSION`; per-task ETag → `Tasks.SYNC_VERSION`; href →
|
||
`Tasks._SYNC_ID`; per-collection sync-token → a `TaskLists.SYNC*` slot (the
|
||
`SyncState` blob is per-account and cannot hold it).
|
||
4. **Never write through the `instances` URI.** Read and write `RRULE`, `RDATE`
|
||
and `EXDATE` on `tasks` — they are raw TEXT and round-trip cleanly.
|
||
5. **Scope every adapter delete by `list_id` yourself.**
|
||
6. **Sequence DELETE before PUT on a list move**, or the two rows sharing a
|
||
`_UID` collide on the server.
|
||
7. **Read-only collections are enforced in the app layer**, not the provider.
|
||
8. **Soft-deleted subtasks lose their `Relation` rows** before the adapter sees
|
||
the tombstone (`Reparenting.unlinkParent` runs regardless of `isSyncAdapter`).
|
||
Either cache the relation or accept that `RELATED-TO` is unrecoverable on
|
||
delete.
|
||
|
||
### Two collisions with the shipped app layer
|
||
|
||
- **`AndroidTasksDataSource.setAlarm` deletes every alarm property row on the
|
||
task** before re-inserting. Once the adapter round-trips `VALARM`s, one local
|
||
reminder edit destroys all server-side alarms on that task. Decide ownership:
|
||
either the adapter owns `VALARM`s and the app stops bulk-deleting, or reminders
|
||
stay local-only and are never serialised.
|
||
- **`updateInstance` routes through the instances URI**, which per the table
|
||
above is permanently non-sync-adapter and forks override rows. Those overrides
|
||
arrive `_dirty=1` with no `_uid`/`_sync_id` and must be uploaded as
|
||
`RECURRENCE-ID` components.
|
||
|
||
---
|
||
|
||
## ⚠️ The migration that was believed not to exist — and then stopped existing
|
||
|
||
**Resolved: the original assertion is true again, because the constraint that
|
||
broke it was the provider's.** Our own `task_lists.account_id` is a nullable FK
|
||
from v1, so attaching an account to a list is one `UPDATE` and no task moves.
|
||
Phase 3 does not carry this deliverable. The rest of this section is the record
|
||
of why it was believed to, and still describes External mode exactly.
|
||
|
||
The first draft, `STORAGE-AND-SYNC.md` and `ARCHITECTURE.md` all asserted:
|
||
*"Synced is Local with an account attached, so switching on sync is not a
|
||
migration."* **That was false against the dmfs provider.**
|
||
|
||
`processors/lists/Validating.java:68-76` throws on any attempt to change a task
|
||
list's `ACCOUNT_NAME` or `ACCOUNT_TYPE` — both are write-once, and the contract
|
||
documents them as such. Local lists live under `org.dmfs.account.LOCAL` and can
|
||
never be re-pointed at a real account.
|
||
|
||
Turning on sync therefore means, as a real deliverable with its own tests:
|
||
|
||
1. Create new lists under the account (sync-adapter insert; account params come
|
||
from the **URI**, never the values, and are frozen thereafter).
|
||
2. `UPDATE list_id` on every task — this *is* permitted, and it is the one path
|
||
the provider gives us.
|
||
3. Assign `_UID`s (or better: have them already, see phase 0).
|
||
4. Delete the old local lists (sync-adapter only).
|
||
5. Re-point `DEFAULT_LIST_ID`, per-list reminder overrides in DataStore, and
|
||
every scheduled alarm.
|
||
|
||
Not modelling `SYNCED` as a third `StorageMode` is still right — it is derived
|
||
state. But the migration it implied away is real, and step 2's `Moving` processor
|
||
nulls `_sync_id`/`sync_version`/`SYNC1`–`SYNC8` and clones a tombstone as it goes.
|
||
|
||
**Alternative worth considering:** don't migrate. Synced lists are always *new*
|
||
lists, and moving local tasks into them is an explicit user action with a visible
|
||
UI. Cheaper, more honest, and it never silently rewrites the user's data.
|
||
|
||
---
|
||
|
||
## Account model — and the trap under it
|
||
|
||
**Decision: `AccountManager` with our own account type, plus a registered
|
||
`<sync-adapter>` service whose entire job is to enqueue a WorkManager job.**
|
||
|
||
⚠️ **The first draft's justification was circular** and its architecture did not
|
||
work. Both corrected:
|
||
|
||
### The justification
|
||
|
||
The draft argued AccountManager was *required* because the provider prunes lists
|
||
whose `ACCOUNT_TYPE` has no authenticator in this package. That is backwards —
|
||
`PROVENANCE.md` change 1 made the prunable set **empty by construction**
|
||
precisely so that shipping no authenticator prunes nothing. The provider imposes
|
||
no requirement at all. tasks.org proves the alternative: no AccountManager, Room
|
||
accounts, pure WorkManager, `GET_ACCOUNTS` stripped with `tools:node="remove"`.
|
||
|
||
The real reasons, which are still good ones: a stable account identity that
|
||
third-party engines can address (the DAVx5 ask, open question 5), presence in
|
||
system Settings, and the sync framework as a change-trigger.
|
||
|
||
### The trap ⚠️
|
||
|
||
`ContentService.hasAuthorityAccess()` gates `requestSync`, `setSyncAutomatically`,
|
||
`addPeriodicSync`, `setIsSyncable`, `getSyncStatus` and seven more behind a
|
||
compat change `@EnabledAfter(TIRAMISU)` — **on for targetSdk ≥ 34**, which we
|
||
are. With no package registering a sync adapter for our authority, every one of
|
||
those calls **returns silently**: no exception, no log. It is documented on no
|
||
Android behaviour-changes page.
|
||
|
||
So "AccountManager accounts for visibility + WorkManager for scheduling + no sync
|
||
adapter" — exactly what the first draft described — yields:
|
||
|
||
- every `ContentResolver` sync API a no-op that passes on a Robolectric shadow,
|
||
- an account in system Settings permanently reading **"Sync off for all items"**,
|
||
- a **greyed-out "Sync now"**, because `enabledSyncNowMenu()` needs at least one
|
||
checked authority switch.
|
||
|
||
**Fix:** register a real `AbstractThreadedSyncAdapter` whose `onPerformSync`
|
||
enqueues a WorkManager job and waits — DAVx5's own comment: *"We use the sync
|
||
adapter framework only for the trigger, actual syncing is implemented with
|
||
WorkManager."* Declare `READ_SYNC_SETTINGS` / `WRITE_SYNC_SETTINGS`. Ship an
|
||
in-app sync button too, since "Sync now" stays greyed out under
|
||
`userVisible="false"`.
|
||
|
||
**Free trigger already running:** `TaskProvider.syncToNetwork()` returns `true`
|
||
unconditionally, so `SQLiteContentProvider` already fires
|
||
`notifyChange(uri, null, syncToNetwork=true)` on every user write. The system is
|
||
*already* requesting a sync for our authority on each edit — there is simply
|
||
nothing registered to receive it.
|
||
|
||
### ⚠️ Auto Backup will arm `cleanUpLists` into a data-loss path
|
||
|
||
✅ **Closed.** Both rule sets are now explicit and name our own database with its
|
||
WAL sidecars, the app checkpoints on `ON_STOP`, and a restore test covers the WAL
|
||
case in both directions (`OWN-STORE.md` phase 6). `cleanUpLists` was the
|
||
provider's, and left with it. The original finding, which still describes what an
|
||
External-mode user's provider app does:
|
||
|
||
`backup_rules.xml` and `data_extraction_rules.xml` are both **empty rule sets**,
|
||
and `allowBackup="true"`. An empty set means Auto Backup's default: databases
|
||
included. So the provider's `tasks.db` is backed up and restored — while
|
||
AccountManager accounts, which live in `/data/system_ce/`, are not.
|
||
|
||
`TaskProvider.onCreate` registers `addOnAccountsUpdatedListener(…, updateImmediately=true)`.
|
||
On the first callback after a restore, `cleanUpLists` sees lists carrying our
|
||
`ACCOUNT_TYPE` with no matching account and **deletes them, and their tasks,
|
||
silently** — cascading through `task_list_cleanup_trigger`. Exactly the failure
|
||
`PROVENANCE.md` change 1 exists to prevent, reintroduced from behind.
|
||
|
||
Latent today (no authenticator ⇒ nothing prunable); live the day phase 3 ships.
|
||
|
||
**Do not fix this by excluding the database from backup.** That was the audit's
|
||
suggestion and it is wrong for us: Local-mode data lives in exactly one place,
|
||
and Auto Backup is currently its only automatic safety net — removing it to
|
||
protect *synced* lists would trade a latent bug for a live one. Fix it at the
|
||
cause instead:
|
||
|
||
1. Make pruning **event-driven** — react to `AccountManager`'s account-removed
|
||
broadcast, not to "absent from the visible set".
|
||
2. Add a post-restore reconciliation that offers to **re-attach the account**
|
||
rather than deleting.
|
||
3. Exclude only the **Keystore-encrypted credential blob** from backup — a
|
||
restored ciphertext is permanently undecryptable, since Keystore keys are
|
||
non-exportable.
|
||
4. Device-verify: restore a backup onto a fresh device, confirm nothing is pruned.
|
||
|
||
---
|
||
|
||
## VTODO ↔ `TaskContract`
|
||
|
||
In-house, behind an interface, so the iCalendar library stays swappable.
|
||
|
||
### Unknown properties: the one non-negotiable
|
||
|
||
Any `X-` property, unrecognised component or parameter written by another client
|
||
**must survive a read-modify-write cycle unchanged.** Failing this silently
|
||
destroys other people's data and is invisible in our own UI.
|
||
|
||
Storage exists: a `Properties` row with our own `unknown-property` mimetype and
|
||
the serialised property in `DATA0`. Verified: no validation rejects an unknown
|
||
mimetype, the only index is non-unique so repeats are fine, and there is no FTS
|
||
interaction (`updateFTSEntry` is called only from `CategoryHandler`). Note
|
||
`MIMETYPE` is *declared* `INTEGER` while holding strings — harmless under SQLite
|
||
affinity, but don't be alarmed by it. `Tasks.HAS_PROPERTIES` is never set by
|
||
anything; do not filter on it.
|
||
|
||
⚠️ **But "byte-stable" is unachievable as the draft stated it, and stating it that
|
||
way is dangerous** — the corpus would fail on day one and then be normalised
|
||
until the only thing that matters, *no unknown property is dropped*, is no longer
|
||
tested. Six independent reasons a faithful implementation cannot be byte-identical:
|
||
`PRODID` **must** change (emitting another product's is a lie); fold position
|
||
carries no information and a line may split between any two characters; parameter
|
||
quoting is optional (`TZID=Europe/Berlin` ≡ `TZID="Europe/Berlin"`); property
|
||
order within a component is unconstrained, and storing known fields as columns
|
||
destroys the original interleaving **by construction**; `DTSTAMP` is regenerated
|
||
and `VTIMEZONE` re-emitted; and the server will not return what we sent anyway.
|
||
|
||
**Restate it as a semantic round-trip with byte-stable property values.** Re-parse
|
||
both sides into a canonical multiset of
|
||
`(component path, property name, params as a sorted map, unfolded unescaped value)`
|
||
and assert equality **modulo an explicitly enumerated allowlist** — `PRODID`,
|
||
`DTSTAMP`, `LAST-MODIFIED`, `SEQUENCE`, VTIMEZONE bodies, fold positions,
|
||
parameter quoting. Nothing else may differ. Byte-equality is then asserted where
|
||
it means something: the **unfolded, unescaped value octets** of every untouched
|
||
property, plus full parameter preservation including unknown parameters. RFC 5545
|
||
§3.1 is the requirement being encoded: *"Applications MUST preserve the value data
|
||
for x-name and iana-token values that they don't recognize."*
|
||
|
||
⚠️ **And the requirement is necessary but not sufficient.** A client that
|
||
round-trips perfectly, passing every fixture, still destroys the owner's data if
|
||
it PUTs back a body Nextcloud filtered on the way out (see
|
||
[Server reality](#️-server-reality--the-matrix-audited)). Never write back a body
|
||
whose ETag does not match the hash of what we downloaded.
|
||
|
||
**Three gaps in the storage sketch.** A flat property-per-row model does not
|
||
represent unknown properties **nested inside a known sub-component** (an `X-` prop
|
||
on a `VALARM`), or entirely unknown components, or RFC 9074's alarm properties
|
||
(`ACKNOWLEDGED`, `PROXIMITY`) which are what other clients now write. Decide
|
||
between an opaque sub-component blob and reconstructing nesting from
|
||
`DATA0`–`DATA15`. And **set a size cap** — DAVx5 drops unknown properties above
|
||
~25 kB — for two reasons: Android's `CursorWindow` row limit, and
|
||
`CALDAV:max-resource-size`, whose violation is a failed PUT.
|
||
|
||
**Test-first.** The corpus comes before the mapper, and the fixtures that catch
|
||
bugs are the adversarial ones, not a clean server-generated VTODO: a
|
||
`CLASS:CONFIDENTIAL` task fetched from a **shared** Nextcloud calendar; a resource
|
||
carrying a master plus `RECURRENCE-ID` overrides; unknown properties nested inside
|
||
a `VALARM`; a `TZID` the device's tzdb does not know; a UID containing `/` and
|
||
`@`; and one exceeding `max-resource-size`.
|
||
|
||
### The rest of the minefield
|
||
|
||
- `DUE` vs `DTSTART`; `VALUE=DATE` vs `DATE-TIME`; floating times and `TZID`.
|
||
Match the all-day/UTC convention `fix/provider-interaction-review` established.
|
||
- **`STATUS` / `PERCENT-COMPLETE` / `COMPLETED` disagree across clients.** Pick a
|
||
canonical reading, normalise on write only. Local convention to reconcile
|
||
against: the edit form writes `PERCENT_COMPLETE` clamped 0–100 and leaves
|
||
`STATUS` to the complete toggle.
|
||
- `RELATED-TO` for subtask trees, including orphans in another collection. The UI
|
||
nests one level; the *data* must not assume it.
|
||
- `VALARM` — see the `setAlarm` collision above before writing a line of this.
|
||
- `CATEGORIES`, `PRIORITY` (**0 = undefined, 1 = highest**).
|
||
- ⚠️ **`SEQUENCE` is preserve-verbatim, not ours to bump.** It is the *Organizer's*
|
||
revision counter (§3.8.7.4); a client that increments it on every save confuses
|
||
scheduling-aware peers. Related: **`DTSTAMP` is regenerated per serialisation,
|
||
`LAST-MODIFIED` changes only when the data actually did.** Conflating them makes
|
||
every sync look like an edit.
|
||
- ⚠️ **`COMPLETED` MUST be UTC** (§3.8.2.1) — no TZID, no floating, no DATE.
|
||
- ⚠️ **A `VALARM` with `TRIGGER;RELATED=END` requires `DUE`, or `DTSTART` plus
|
||
`DURATION`** (§3.8.6.3). A user clearing the due date on a task that has an
|
||
end-relative reminder produces an invalid resource, permanently rejected. Validate
|
||
before PUT — this is reachable from ordinary UI actions.
|
||
- ⚠️ **`RELATED-TO;RELTYPE` reads backwards to most implementers.** §3.8.4.5:
|
||
`PARENT` means *the referencing component is subordinate to the referenced
|
||
one*. Both Nextcloud Tasks and tasks.org put `RELTYPE=PARENT` on the child
|
||
pointing up — that is the correct reading. Also: the RFC explicitly disclaims
|
||
cascade semantics, so "completing a parent completes its subtasks" is a local
|
||
UI convention peers will not reproduce.
|
||
- ⚠️ **Round-trip `X-MOZ-LASTACK` / `X-MOZ-SNOOZE-TIME` unmodified.** Dropping
|
||
them causes documented **alarm storms** on Thunderbird. Emit RFC 9074
|
||
`ACKNOWLEDGED` for our own writes rather than minting `X-MOZ-*`. Preserve
|
||
`X-APPLE-SORT-ORDER`; never interpret it.
|
||
|
||
### ⚠️ Recurring completion — and the model our provider already chose
|
||
|
||
The draft said "no standard; choose one". That is **confirmed and understated**,
|
||
and the choice is less free than it looked.
|
||
|
||
**How settled the non-standard is.** `draft-ietf-calext-ical-tasks-17` — the
|
||
active IETF work item whose entire purpose is extending VTODO, `Updates: RFC5545`
|
||
— contains the substring **"recur" zero times** in 1,904 lines. It adds
|
||
`SUBSTATE`, `REASON`, `TASK-MODE` and a `VSTATUS` component and leaves this
|
||
untouched. RFC 8984 §5.2.6 (JSCalendar) is the only RFC that addresses it at all,
|
||
and it **blesses two mutually incompatible approaches and declines to pick**.
|
||
RFC 5545 permits `COMPLETED` on a recurring master with no interaction rule —
|
||
undefined, which is worse than forbidden, because every client picks differently
|
||
and all stay conformant.
|
||
|
||
The four models in the wild:
|
||
|
||
| | Model | Who |
|
||
|---|---|---|
|
||
| **a** | Write a `RECURRENCE-ID` override; master stays open | jtx Board, Thunderbird, eM Client |
|
||
| **b** | Advance the master's `DUE`/`DTSTART` in place, clear completion | tasks.org, Evolution, Nextcloud Tasks *in practice* |
|
||
| **c** | `STATUS:COMPLETED` on the master — kills the series | Nextcloud Tasks ≤ 0.17. **Always a bug** |
|
||
| **d** | Detach the completed occurrence as a **new task with a new UID**, and advance the master | **OpenTasks — i.e. our `:provider`** |
|
||
|
||
**The finding that matters to us: `:provider` has already chosen model (d).**
|
||
`processors/instances/Detaching.java` nulls `_UID`, `_SYNC_ID` and every
|
||
`ORIGINAL_INSTANCE_*` on the detached row, and `detachAll` advances the master
|
||
and decrements `RRULE;COUNT`. dmfs did this deliberately — on the record, *"the
|
||
primary reason is to support Apple clients; they don't support overrides"*. So
|
||
our storage layer emits a UID-less orphan plus a moved master, and our sync
|
||
adapter has to either honour that, bypass the processor, or reconcile after it.
|
||
**That is a design constraint we inherited without deciding it**, and it is the
|
||
strongest single argument for treating this as a phase-1 decision rather than a
|
||
phase-4 detail.
|
||
|
||
**Rules the audit establishes, regardless of which model we write:**
|
||
|
||
- **Never write model (c).** Every instance found was filed as a defect;
|
||
Thunderbird fixed it fifteen years ago.
|
||
- **Never do (a) and (b) together.** `RECURRENCE-ID` is *defined* as the
|
||
instance's original `DTSTART`, so advancing the master orphans your own
|
||
override. Nextcloud Tasks 0.18 attempts exactly this — and is saved only by an
|
||
accident: its override write dispatches a Vuex action that **does not exist**,
|
||
which Vuex 4 swallows without throwing. Shipped behaviour is therefore silent
|
||
model (b) with no record the instance was ever completed.
|
||
- **Accept all four models on read, unconditionally**, including an inbound
|
||
master whose `DUE` moved and whose `COMPLETED` vanished. That is not corruption.
|
||
- **Never abort a sync batch on a multi-VTODO resource.** tasks.org returns from
|
||
its whole sync function on one — so a single Thunderbird-completed recurring
|
||
task **stops that entire collection from syncing**, ctag never advances, and
|
||
every other change in the batch is silently lost. Degrade to the master and
|
||
continue.
|
||
- **Keep overrides in the same calendar object resource** (RFC 4791 §4.1).
|
||
- ⚠️ **`RRULE` + `DUE` with no `DTSTART` has no well-defined `RECURRENCE-ID`
|
||
value** — undefined in RFC 5545, ubiquitous in the wild. Synthesising
|
||
`DTSTART := DUE` is the common workaround and is itself the source of visible
|
||
DTSTART/DUE desync between clients. Handle it explicitly.
|
||
- ⚠️ **Repeat-from-completion has no interoperable encoding at all.** Either drop
|
||
it or document that peers read it as repeat-from-due.
|
||
- Consider tasks.org's escape hatch: a per-account **"let the server schedule
|
||
recurring tasks"** switch.
|
||
|
||
~~Open question 4~~ — **decided and shipped: we write (a).** We do own the whole
|
||
path now, so the `Detaching` caveat is moot. `RoomTasksDataSource`'s
|
||
`setCompletedInstance` and `updateInstance` both fork a `RECURRENCE-ID` override
|
||
sharing the master's UID, and the master stays open — jtx Board's and
|
||
Thunderbird's model, and the one that maps onto CalDAV without invention. The
|
||
adapter must still **read** all four models, which is unchanged.
|
||
|
||
> **Process note.** During this research a summarising fetch **fabricated a
|
||
> verbatim RFC 5545 sentence** ("A 'to-do' calendar component without the
|
||
> 'dtstart' property MUST NOT be part of a recurring set") that appears nowhere
|
||
> in the RFC — grep confirms zero hits — along with an invented DTSTART/DUE
|
||
> exclusivity rule. Normative text gets read from the raw RFC, never from a
|
||
> summary. Both fabrications would have inverted a design decision here.
|
||
|
||
---
|
||
|
||
## Libraries
|
||
|
||
⚠️ **The first draft's table had one outright licence error and understated two
|
||
libraries by roughly an order of magnitude.** Re-verified 2026-08-13 against
|
||
published POMs, Gradle module metadata and extracted jars.
|
||
|
||
### Take
|
||
|
||
| Library | Licence | Real cost |
|
||
|---|---|---|
|
||
| **dav4jvm** 4.0.1 | MPL-2.0 | ⚠️ **Ktor-only** (the OkHttp package was deleted in 3.0.0); ⚠️ **requires Java 21 bytecode** — we target 17 everywhere, including all seven floret-kit modules; pulls Ktor (~2.45 MB), `guava-jre` (wrong flavour — force `-android`), and `xpp3` (371 KB, duplicates framework `org.xmlpull.v1`). The library itself is only 433 KB / 246 classes |
|
||
| **cert4android** | ⚠️ **MPL-2.0 — not GPLv3** | Same org, same licence, same JitPack question as dav4jvm. See below |
|
||
| **ical4j** 4.3.0 | BSD-3-Clause | ⚠️ Not "needs desugaring" — `java.time` is native at minSdk 26 and we are 29. Real costs: **2.2 MB of duplicated tz data** in the jar (`zoneinfo/` *and* `zoneinfo-global/`, ~596 `.ics` each), a `ZoneRulesProvider` that pre-allocates 1500 synthetic zone IDs **and exhausts in production**, and a mandatory `ical4j.properties` + `MapTimeZoneCache` + registry shim |
|
||
| **lib-recur** | Apache-2.0 | ⚠️ **Version trap, see below** |
|
||
|
||
### ⚠️ cert4android was rejected on a false premise
|
||
|
||
The first draft listed it as GPLv3 and budgeted a hand-rolled trust-on-first-use
|
||
dialog instead. **It is MPL-2.0** — verbatim MPL text in `LICENSE`, SPDX
|
||
boilerplate in the README, GitHub agrees. The same licence as dav4jvm, which the
|
||
same document accepts two rows above. Two independent audit tracks caught this.
|
||
|
||
That matters because the hand-rolled version is not a dialog:
|
||
|
||
- **A background sync has no UI to show a dialog in.** cert4android's bound
|
||
service + notification approval *is* the library, not an accessory to it.
|
||
- **Network Security Config cannot express runtime trust** — it is a static
|
||
manifest resource. And since API 24, user-installed CAs aren't trusted without
|
||
an NSC entry, so "tell the user to install their CA" fails too.
|
||
- **The 3-arg `checkServerTrusted(chain, authType, host)` is mandatory**;
|
||
2-arg-only TrustManagers have repeatedly broken on OkHttp.
|
||
- **Hostname verification is a second override** and the other thing Play flags.
|
||
- **Play has blocked publishing on unsafe `X509TrustManager` since 2016**, and
|
||
the guidance explicitly names "buggy or incomplete custom verification".
|
||
|
||
This roughly dissolves the self-signed-cert line item in phase 4.
|
||
|
||
### ⚠️ lib-recur is a version trap, not a free dependency
|
||
|
||
✅ **Resolved by deleting the other side of the trap.** `:app` declares lib-recur
|
||
0.12.2 directly and `RecurrenceExpander` uses it; the vendored provider whose
|
||
iterators would have stopped compiling no longer exists, so the version is ours
|
||
alone to move. The `RecurrenceSet` removal in 0.16.0 is now a plain upgrade
|
||
question, not a build-breaking one. The original finding:
|
||
|
||
The first draft said "already in the build at 0.12.2 — no new dependency". Both
|
||
halves are wrong. `provider/build.gradle.kts:56` declares it `implementation`, not
|
||
`api`, so it is **not** on `:app`'s compile classpath. And lib-recur **0.16.0
|
||
removed `RecurrenceSet`**, which the vendored provider uses in
|
||
`TaskInstanceIterable`/`TaskInstanceIterator`. The moment `:app` adds a current
|
||
lib-recur, Gradle's highest-wins resolution upgrades the graph and **the provider
|
||
stops compiling.**
|
||
|
||
Decide explicitly: pin `strictly = "0.12.2"` and accept the known fixes we forgo
|
||
(0.15.1 `FastForwarded`, 0.15.2 empty-`ByDay`), or budget the iterator rewrite
|
||
onto the post-0.16 API. Note upstream is dormant — last release 0.17.1, last
|
||
commit 2024-04.
|
||
|
||
### Consider
|
||
|
||
**biweekly** (BSD-2) is stronger than the first draft credited: 639 KB / 432
|
||
classes, **no bundled tz database at all**, legacy `java.util.Date` so no
|
||
`ZoneRulesProvider` hazard — against ical4j's 2.2 MB of zone data and its
|
||
registry shim. Caveats: last release 0.6.8 (2024-01), still 0.x, mandatory
|
||
`jackson-core` for jCal (excludable), and the missing tz database means it relies
|
||
on `VTIMEZONE` components being present rather than resolving `TZID`s itself —
|
||
a real gap for CalDAV round-tripping.
|
||
|
||
### Do not take
|
||
|
||
| Library | Why not |
|
||
|---|---|
|
||
| **synctools** | **GPLv3.** Does exactly the mapping we need against exactly the schema we run, which makes it the sharpest temptation here. Still a one-way door. (Repo now archived and folded into `davx5-ose` as a module — the standalone coordinate is stale.) |
|
||
| **Android-SingleSignOn** | ⚠️ **GPL-3.0.** The first draft carried it as a harmless optional extra; it is the actual one-way door. It also proxies through the Files app and supports only OCS plus a few WebDAV verbs — not a CalDAV transport |
|
||
| **caldav4j** | Apache-2.0, but server-oriented, last release 2022-01 |
|
||
| **sardine** | Needs JAXB — a non-starter on Android |
|
||
|
||
**Still true:** there is no mature Kotlin-native iCalendar library, and no
|
||
Android-suitable CalDAV client on Maven Central at all. That is *why* the JitPack
|
||
question is unavoidable rather than optional.
|
||
|
||
### On GPL and Play
|
||
|
||
The rejection reasoning is right in effect but was imprecise. MIT **is**
|
||
GPL-compatible; the constraint is on the terms of the distributed binary, not on
|
||
our source headers. And ⚠️ **GPLv3 is not a Play problem** — DAVx5 is GPLv3 and
|
||
ships on Play with 287k+ installs; §6 Installation Information is a hardware
|
||
provision. If the real reason is wanting Agendula to stay permissively
|
||
relicensable, say that, because that is the reason that holds.
|
||
|
||
### ⚠️ Licence obligations we cannot currently discharge
|
||
|
||
Settings exposes only our own MIT `LICENSE`. There is no third-party attribution
|
||
surface and no AboutLibraries in the build. But MPL-2.0 §3.2(a) requires telling
|
||
recipients how to obtain source; §3.4 requires retaining file headers; **BSD-3
|
||
requires reproducing the copyright notice in binary distributions** (that is
|
||
ical4j, and it is not optional); Apache-2.0 §4(d) propagates NOTICE. Shipping any
|
||
of these without an attribution screen is a plain violation, independent of
|
||
copyleft. **Phase 0 work item.** Bonus trap: ical4j's POM declares a non-SPDX
|
||
licence name and a `LICENSE` URL that 404s, so generators produce empty output.
|
||
|
||
### The JitPack question, restated
|
||
|
||
`dav4jvm` and `cert4android` are both `com.github.bitfireAT:*` — JitPack only.
|
||
F-Droid's inclusion policy does trust jitpack.io for freely-licensed artifacts,
|
||
so **F-Droid is not the obstacle; our own `FAIL_ON_PROJECT_REPOS` policy is.**
|
||
But F-Droid's own writing is lukewarm — JitPack "hosts whatever is built from
|
||
GitHub, without checking the license" — and concretely: JitPack **does not sign
|
||
artifacts** (`.asc` 404s; Maven Central's does not), and rebuilds on demand, so a
|
||
coordinate is not immutable. We have no `verification-metadata.xml` today.
|
||
|
||
Weigh against that: **dav4jvm shipped two breaking majors nineteen days apart**
|
||
(3.0.0 OkHttp→Ktor 2026-07-08; 4.0.0 callbacks→coroutines 2026-07-27), which
|
||
argues for vendoring a known-good tree rather than a floating pin — and at 433 KB
|
||
vendoring is far cheaper than depending, once the Ktor/guava/xpp3 tail is counted.
|
||
Open question 1.
|
||
|
||
---
|
||
|
||
## Authentication and discovery
|
||
|
||
### Nextcloud Login Flow v2
|
||
|
||
The protocol description survives audit against the server source: the endpoint,
|
||
the `{poll:{token,endpoint},login}` shape, 404-until-approval, the 20-minute
|
||
lifetime (`lifetime = 1200` in `LoginFlowV2Mapper.php`), and "the 200 is returned
|
||
exactly once" (the mapper deletes the row inside `poll()` before returning). It is
|
||
not deprecated, there is no v3, and OAuth2 is a worse fit. Corrections:
|
||
|
||
- ⚠️ **Poll with `POST`, form-encoded.** A `GET` gets 405. The draft didn't say.
|
||
- ⚠️ **Set an explicit `User-Agent`.** `init()` passes it to `createTokens()`,
|
||
where it becomes the app password's **name** in Settings → Security → Devices &
|
||
sessions. With OkHttp's default the user sees `okhttp/4.12.0` and cannot tell
|
||
what to revoke — defeating the entire point of the flow. (`OCS-APIRequest` is
|
||
*not* needed here; v2 is a Frontpage route.)
|
||
- ⚠️ **404 only means pending.** "Treat anything that isn't a 200 as pending"
|
||
swallows 429 (brute-force protection), 503 (maintenance), Cloudflare challenge
|
||
pages (200 with HTML), and DNS/TLS failure — turning a diagnosable error into a
|
||
20-minute spinner. Require `Content-Type: application/json` before parsing.
|
||
Stop polling on anything that is neither 404 nor 200. Note 404 is *also*
|
||
returned for expired/consumed, so keep tracking the deadline locally.
|
||
- ⚠️ **Validate the `endpoint` origin.** Verbatim is right for the *path*, wrong
|
||
as a blanket rule: it is generated from `overwrite.cli.url` / `overwriteprotocol`
|
||
/ `trusted_proxies`, misconfigured on a large fraction of self-hosted installs.
|
||
Refuse a scheme downgrade to `http` outright — the poll token is exchanged for a
|
||
long-lived app password, so this is a credential-grade secret. If the host
|
||
differs from the one the user typed, confirm explicitly and say *"your server's
|
||
`overwrite.cli.url` is wrong"*, which saves a support round-trip. (The draft's
|
||
"some deployments return 302" is a proxy symptom, not a Nextcloud variant.)
|
||
- ⚠️ **`loginName` is not the uid.** It is what the user typed — possibly an
|
||
email, an LDAP-derived value, or the right name in the wrong case. Use it
|
||
**only** as the Basic auth username; never interpolate
|
||
`remote.php/dav/calendars/<loginName>/`. Discover via `current-user-principal`
|
||
→ `calendar-home-set`, exactly as the generic path already does. This is the
|
||
classic "logged in but no calendars" bug.
|
||
- ⚠️ **Custom Tabs needs four things the draft omitted:** a `<queries>` entry for
|
||
`android.support.customtabs.action.CustomTabsService` (or provider detection
|
||
silently fails on API 30+); a try/catch with an `ACTION_VIEW` fallback
|
||
(`launchUrl` throws `ActivityNotFoundException` with no Custom Tabs browser —
|
||
realistic on GrapheneOS/CalyxOS/AOSP, i.e. disproportionately our users);
|
||
persistence of `{token, endpoint, deadline}` to disk immediately, so process
|
||
death mid-flow is resumable; and an explicit "I finished / Cancel" affordance,
|
||
since Custom Tabs return **no result** when dismissed and Nextcloud's flow ends
|
||
on a "you can close this window" page that never returns to the app.
|
||
|
||
### Generic CalDAV discovery
|
||
|
||
⚠️ **The draft's five steps were the happy path of a much longer pipeline, and
|
||
one of its two filters was inverted.** Corrected version, with live probes run
|
||
2026-08-13:
|
||
|
||
```
|
||
1. Input: email / mailto: / http(s) URL
|
||
2. Base URL typed → PROPFIND Depth:0 on it first (principal, home-set and
|
||
collection can all come back in one response)
|
||
3. Else:
|
||
a. SRV _caldavs._tcp.<domain> — honour RFC 2782 priority/weight,
|
||
honour non-443 ports, target "." = none
|
||
b. TXT _caldavs._tcp.<domain> — parse path= ⚠️ MISSING FROM DRAFT
|
||
c. ladder: [TXT path] → /.well-known/caldav → / ⚠️ "/" MISSING
|
||
4. PROPFIND Depth:0 for DAV:current-user-principal
|
||
- follow 301/302/303/307/308, re-sending PROPFIND and its body
|
||
- relative Location; reject HTTPS→HTTP; cap at 5; PERSIST 301/308
|
||
- 401 → authenticate and retry. NOT a failure ⚠️ MISSING
|
||
- reject <D:unauthenticated/> ⚠️ MISSING
|
||
- OPTIONS gate: DAV: header must contain calendar-access ⚠️ MISSING
|
||
5. PROPFIND Depth:0 on the principal for calendar-home-set
|
||
→ iterate ALL hrefs (0..n); cross-host is normative ⚠️ DRAFT ASSUMED ONE
|
||
6. PROPFIND Depth:1 per home set, requesting properties BY NAME
|
||
→ optionally recurse one level into plain {DAV:}collection members
|
||
7. Classify:
|
||
a. resourcetype as a SET; require CALDAV:calendar ⚠️ MISSING
|
||
b. VTODO test: property ABSENT ⇒ INCLUDE ⚠️ DRAFT INVERTED IT
|
||
c. privilege-set absent ⇒ assume writable; handle 403 on write
|
||
8. sync path: supported-report-set → RFC 6578 keyed on DAV:sync-token
|
||
9. Creation: OPTIONS feature-detect → MKCALENDAR / extended MKCOL / disable UI
|
||
```
|
||
|
||
**The two filter corrections, which are the important part:**
|
||
|
||
- ⚠️ **`supported-calendar-component-set` absent means "supports everything",
|
||
not "supports nothing".** The draft kept only collections whose set *includes*
|
||
VTODO, which silently drops every server that doesn't advertise it. RFC 4791
|
||
§5.2.3 also says the property SHOULD NOT come back from an allprop request —
|
||
so **request properties by name**, or you get none of them. Its grammar is
|
||
`(comp+)`; an empty element is non-conformant, and `dav4jvm`'s parser starts
|
||
all-`false` and would classify it as supporting nothing. Treat empty as all.
|
||
- ⚠️ **Classify on `resourcetype`, with a positive test.** The draft had no
|
||
resourcetype check at all, so a Depth:1 listing on Nextcloud yields inboxes,
|
||
outboxes, notification collections, trash bins and subscriptions as "task
|
||
lists". The test must be **`CALDAV:calendar` is present in the set** — *not*
|
||
exclusion by `schedule-outbox`, because **SOGo's main personal calendar reports
|
||
`collection` + `calendar` + `schedule-outbox` simultaneously** for every
|
||
non-Apple client, which is exactly what we are. The positive rule also keeps
|
||
shared calendars (which add `CS:shared` alongside `CALDAV:calendar`) and drops
|
||
Nextcloud's `nc:deleted-calendar`, which deliberately strips `caldav:calendar`.
|
||
Treat resourcetype as an **unordered set**, never by position.
|
||
|
||
**Discovery traps, live-probed:**
|
||
|
||
| Trap | Detail |
|
||
|---|---|
|
||
| SRV TXT `path=` | Live at **Posteo** (`path=/`), **GMX** and **Web.de** (`path=/begenda/dav/users/`). Skip it and GMX/Web.de land on the wrong path |
|
||
| Posteo | **SRV-only**, on port **8443**; its `/.well-known/caldav` 404s. Hardcoding 443 fails |
|
||
| Null SRV target | `_caldav._tcp.fastmail.com` and `runbox.com` return `0 0 0 .` — "explicitly unavailable" |
|
||
| Google SRV | Returns a valid record pointing at `calendar.google.com`, which **is not a DAV server** (PROPFIND → 405). A strict RFC 6764 client follows it into a dead end for every `@gmail.com` |
|
||
| well-known 401 | **iCloud and Zoho** answer 401 — the endpoint *is* the DAV root and wants auth. RFC-legal; must not be read as failure |
|
||
| Redirect downgrade | `dav.runbox.com` redirects **HTTPS→HTTP** in production today |
|
||
| Method preservation | A generic HTTP stack may legally downgrade 301/302 to GET, silently breaking PROPFIND |
|
||
| `<D:unauthenticated/>` | RFC 5397 §3 — a **200** whose body means auth failed. Without this check a failed login looks like a successful discovery that found nothing |
|
||
| Cross-host home set | Normative (RFC 4791 §6.2.1's own example), and iCloud depends on it: principal on `caldav.icloud.com`, home set on `pNN-caldav.icloud.com`. Allow it, require HTTPS, surface the host change, never send credentials into an unvalidated redirect chain |
|
||
| `caldav.fastmail.com` | `d.fastmail.com` is dead — cert mismatch |
|
||
|
||
⚠️ **Two `dav4jvm` defects we would inherit:** it handles 301/302/307/308 but
|
||
**not 303**, which RFC 6764 §5 names explicitly; and issue #209 — `location` is
|
||
mutated in place, so **permanent redirects never reach the caller**. DAVx5 never
|
||
rewrites its stored collection URL after a 301 and re-follows on every sync.
|
||
Persist the new URL ourselves on 301/308.
|
||
|
||
**Read-only detection is softer than the draft assumed.** RFC 3744 §3.7 defines
|
||
a `DAV:read-current-user-privilege-set` privilege, so a server may legally return
|
||
`current-user-privilege-set` in a **403 propstat**. Default to writable when it is
|
||
absent (as DAVx5 does), expand aggregates yourself (`DAV:write` and `DAV:all`
|
||
imply content-write; some servers don't expand), and prefer sabre's
|
||
`{DAV:}share-access` where offered as the cleanest signal.
|
||
|
||
**Creating lists is not always possible.** MKCALENDAR is only *RECOMMENDED* by
|
||
RFC 4791 §5.3.1. **iCloud is MKCOL-only** (and MKCOL under `…/calendars/<name>/`
|
||
returns 412 while `…/<principal>/<name>/` returns 201); **Google has neither**;
|
||
**Posteo disables it** despite running sabre. Feature-detect via OPTIONS and
|
||
**disable the "new task list" UI** where neither is available. Set
|
||
`supported-calendar-component-set` **at creation — it is protected afterwards**,
|
||
and follow up with an explicit PROPPATCH for `displayname`, which most servers
|
||
ignore in the MKCALENDAR body.
|
||
|
||
**Two more VTODO viability landmines:** Zimbra and OX/mailbox.org restrict tasks
|
||
to dedicated task lists (MKCALENDAR with `[VTODO]`), and **OX rejects recurring
|
||
VTODOs with 400**. And **server-side `CALDAV:expand` on VTODO is broken on
|
||
Nextcloud, Baïkal, SOGo, Radicale and Posteo alike** — expand client-side, which
|
||
`lib-recur` already gives us. Request `max-resource-size` too: violating it is a
|
||
failed PUT, and long `DESCRIPTION`/`ATTACH` payloads reach it.
|
||
|
||
⚠️ **`getctag` is not a cheap pre-check** — `caldav-ctag-03` deprecated it in
|
||
2015 in favour of RFC 6578, and `DAV:sync-token` is itself PROPFIND-able, so the
|
||
same pre-check comes back in the Depth:1 listing we already make. On Nextcloud
|
||
they are literally the same value. Keep `getctag` only as a legacy fallback where
|
||
`supported-report-set` omits `sync-collection`.
|
||
|
||
⚠️ **Auth is not just Basic:**
|
||
|
||
- **Baïkal defaults to Digest** (`dav_auth_type`), and **OkHttp has no Digest
|
||
support** — square/okhttp#205 has been open for years. DAVx5 carries a
|
||
hand-written `BasicDigestAuthHandler` in dav4jvm precisely for this. Take it
|
||
(MPL-2.0, same decision as above) or detect the `WWW-Authenticate: Digest`
|
||
challenge and emit a real error instead of "wrong password". Baïkal is squarely
|
||
in our target audience.
|
||
- **Send Basic preemptively via an `Interceptor`**, gated to HTTPS and the
|
||
account's own origin. OkHttp's `Authenticator` is reactive-only — an extra round
|
||
trip on every request of a PROPFIND-heavy sync, and it never fires at all on
|
||
servers that answer 403/404 without a challenge. Note OkHttp strips
|
||
`Authorization` on cross-host redirects (correct, but it breaks `.well-known`
|
||
discovery across hosts — re-attach only after validating the target).
|
||
- **Fastmail requires an app password** and its Basic plan has no CalDAV at all.
|
||
**iCloud requires an app-specific password** and 2FA to mint one. **Google is
|
||
OAuth2-only** — refuse it with an explanation rather than a 401. Detect these
|
||
by domain at account-add time; "wrong password" that is actually "you used your
|
||
account password" is the single most common support ticket any CalDAV client
|
||
inherits.
|
||
|
||
### Credential storage, rotation, revocation
|
||
|
||
⚠️ `androidx.security:security-crypto` is not "effectively stalled" — it is
|
||
**formally deprecated and terminal**: deprecated at 1.1.0-alpha07 (2025-04),
|
||
shipped deprecated in stable 1.1.0 (2025-07), with release notes saying there
|
||
will be no subsequent releases. Its successor `datastore-tink` is alpha only.
|
||
|
||
⚠️ And the alternative is weaker than implied: **AccountManager stores passwords
|
||
as plain `TEXT`** — no encryption or hashing anywhere in AOSP. FBE plus a
|
||
same-signature check is the whole boundary. That is DAVx5's actual posture and is
|
||
defensible, but state it rather than implying it is secure storage.
|
||
|
||
**Decision:** Keystore `AES/GCM/NoPadding`, blob in DataStore.
|
||
`setUserAuthenticationRequired(false)` is the default — don't call it. Do **not**
|
||
set `setUnlockedDeviceRequired` (breaks background sync). Handle
|
||
`AEADBadTagException` / `KeyPermanentlyInvalidatedException` as *re-authenticate*,
|
||
not as a crash. Note `getUserData` returns null while the device is locked, so a
|
||
boot-triggered sync must wait for unlock.
|
||
|
||
⚠️ **Revocation is bidirectional and the draft had neither direction:**
|
||
|
||
- **On 401: stop syncing that account immediately**, mark `NEEDS_REAUTH`, notify
|
||
with a deep link into the login flow, and **do not retry on a timer**.
|
||
Nextcloud's brute-force protection throttles then 429s **per source IP** — a
|
||
retry loop on a dead app password takes down the user's *other* Nextcloud
|
||
clients on that network and looks like we broke their server. App passwords do
|
||
die in the wild (password change, admin revocation, server bug #39615).
|
||
Distinguish 401 (re-auth) from 403 (forbidden, do not re-auth) from 429/503
|
||
(back off, honour `Retry-After`). Nextcloud returns 401 with
|
||
`PasswordLoginForbidden` when 2FA is on and a real password was used — worth
|
||
detecting for a precise message.
|
||
- **On account removal: call `DELETE /ocs/v2.php/core/apppassword`**
|
||
(this one *does* need `OCS-APIRequest: true`), best-effort. Otherwise
|
||
uninstalling never revokes access, and orphaned entries accumulate that the user
|
||
cannot identify — see the User-Agent point above.
|
||
|
||
---
|
||
|
||
## The sync engine
|
||
|
||
- Collection discovery and refresh; per-collection sync state (in a `TaskLists`
|
||
`SYNC*` slot, per the squat table).
|
||
- ⚠️ **Baseline is `REPORT calendar-query` with a VTODO comp-filter and *no*
|
||
time-range**, matching DAVx5 — which deliberately does not use RFC 6578 for
|
||
tasks, and omits the time-range *"because some servers don't return tasks
|
||
without time at all"*. `sync-collection` is the **optimisation on top**, not
|
||
the primary path. The draft had this the wrong way round.
|
||
- CTag / sync-token loop (RFC 6578 `sync-collection`) where it works. ⚠️ **The
|
||
full-reconciliation path is not a fallback for weak servers — it is a permanent
|
||
safety net on every server**, because a pruned change log behind a still-valid
|
||
token is undetectable (below).
|
||
- Local change detection: `_dirty = 1 OR _deleted = 1`, scoped by `list_id`.
|
||
- `If-Match` conditional PUT.
|
||
- Backoff and partial-failure recovery. A failed collection must not fail the
|
||
account.
|
||
|
||
### ⚠️ RFC 6578, and where every shipping client has bugs
|
||
|
||
The audit read RFC 6578 in full (no errata, fourteen years on) plus the
|
||
w3c-dist-auth threads that are its only authoritative gloss. Calibration first:
|
||
**Evolution shipped `sync-collection` in June 2026** against a request open since
|
||
2019; **vdirsyncer has declined it for twelve years**; **Thunderbird still has
|
||
unlanded patches** for one of the cases below. The library covers about a third.
|
||
|
||
**⚠️ Note DAVx5 does not use RFC 6578 for tasks at all** — its own documentation
|
||
says *CalDAV tasks: use `REPORT calendar-query`*, because a collection
|
||
advertising both VEVENT and VTODO would stream every event change and force a
|
||
fetch to discover it isn't a task. That is a real argument for making
|
||
`calendar-query` our primary path and `sync-collection` the optimisation.
|
||
|
||
1. **Invalidation has no status code.** §3.2 defines the `DAV:valid-sync-token`
|
||
precondition and never assigns an HTTP status. Observed: **403** (sabre ⇒
|
||
Nextcloud, ownCloud, Baïkal, and Radicale 3.1.8), **400** (Google, CalDAV and
|
||
CardDAV), **409** (Radicale, per its maintainer), **412** (accepted by
|
||
Evolution). One server family, two codes across versions.
|
||
**Rule: ignore the status; match `<D:valid-sync-token/>` anywhere in the body
|
||
on any 4xx.** Thunderbird's CardDAV code accepts only 400 and therefore never
|
||
recovers from the 403 that most of the self-hosted world emits.
|
||
2. **Initial sync must not report deletions** (§3.4), so a forced full resync
|
||
cannot learn what was deleted. **Mark-and-sweep is mandatory** — and the
|
||
`initialIncomplete` flag must be persisted *alongside* the token, or a resumed
|
||
partial sync sweeps against an incomplete "present remotely" set and **deletes
|
||
live data**.
|
||
3. **⚠️ Persist the token only after the bodies are applied.** The RFC's own
|
||
Appendix B gets this backwards — it associates the new token with the
|
||
collection *first*, then fetches. Death in between loses those changes
|
||
permanently. Under WorkManager, process death mid-sync is routine, not
|
||
exotic. Persist per page, after step 5, atomically with `initialIncomplete`.
|
||
4. **Worse than invalidation: a token the server still accepts over a change log
|
||
it already pruned.** Returns 207, zero changes, "you're current" — no error,
|
||
no recovery, and **RFC 6578 provides no signal for it.** Nextcloud's
|
||
`totalNumberOfSyncTokensToKeep` defaults to 10,000 and its own admin manual
|
||
warns this "will lead to premature data deletion and synchronization
|
||
problems"; Baïkal #1140 has shipped an empty change set *forever* since 2022.
|
||
**The only mitigation is periodic full reconciliation** (PROPFIND `Depth: 1` +
|
||
ETag diff) on a slow cadence regardless of the token.
|
||
5. **Truncation: detect the 507 on the SELF href, not the error element.**
|
||
§3.6's `DAV:number-of-matches-within-limits` is a SHOULD and sabre omits it
|
||
entirely. Distinguish it from a **507 as the outer HTTP status**, which means
|
||
your `DAV:limit` could not be honoured — retry without the limit, don't page.
|
||
iCloud emits a SELF response with status **200**; ignore that one.
|
||
**Do not send `DAV:limit`** — Nextcloud regressed it to a localised HTML error
|
||
page in 28.0.10/29.0.7/30.0.0. Cap by bytes client-side instead, and still
|
||
implement 507 handling. Add an iteration cap **and** a no-progress guard: the
|
||
RFC never requires the token to advance, and an unchanged token spins forever.
|
||
6. **`supported-report-set` is a hint, not a contract.** Radicale advertised
|
||
`sync-collection` for years without implementing it; Cyrus 3.8 advertises it
|
||
and rejects the mandated empty token. A 207 with no `<D:sync-token>` must
|
||
**degrade to PROPFIND, not throw**.
|
||
7. **Tokens are opaque.** §3.2 says they MUST be URIs; Google, iCloud, fruux
|
||
(`0`), and grommunio all violate it. Never parse or validate. And **never
|
||
reuse a token across collections** — sabre validates only the prefix, then
|
||
returns a wrong-but-plausible delta with no error. Key by
|
||
`(accountId, collectionUrl)`.
|
||
8. **Deletion is `<D:status>404</D:status>` at *`<D:response>`* level.** A 404
|
||
inside a `<D:propstat>` is a missing *property* on a resource that exists.
|
||
Confusing the two nesting levels deletes live data.
|
||
9. **Three membership edge cases** (§3.5): create-then-delete between syncs is
|
||
reported as removed, so the delete handler must no-op on an href it has never
|
||
seen; delete-then-recreate at the same URI is reported as **changed**, so href
|
||
identity is not UID identity — re-read the UID from the body; and **ACL churn
|
||
may be reported as removal**, so toggling a share can look like mass deletion.
|
||
Apply a sanity threshold before acting on a large delete batch.
|
||
10. **`sync-collection` never reports collection property changes.** §3.5.1 keys
|
||
"changed" on an entity tag, and a calendar collection has no entity body. So
|
||
displayname, colour and **read-only status can only be refreshed by PROPFIND
|
||
on the home set** — and on sabre, `CalendarHome` does not implement
|
||
`ISyncCollection` at all, so there is no sync-collection there to use.
|
||
DAVx5 has this exact gap open as its own bug.
|
||
11. **`calendar-data` inside `sync-collection` is sanctioned by neither RFC.**
|
||
RFC 4791 §9.6 says it "is not a WebDAV property"; it works on sabre only
|
||
because that codebase exposes it as one by explicit accident. Request
|
||
`getetag` + `resourcetype`, then batch `calendar-multiget` — **and match the
|
||
returned hrefs against what you asked for**, because real servers reply with
|
||
responses for unrelated URLs.
|
||
12. **`getctag` is formally deprecated** by `caldav-ctag-03` in favour of this
|
||
REPORT — and every shipping client still keeps it as a fallback. Do the same,
|
||
but never compare a ctag to a sync-token.
|
||
|
||
**What `dav4jvm` actually gives us:** spec-correct serialisation, `Depth: 0`,
|
||
`"infinite"` spelled right, a streaming `Flow` with the token arriving as an
|
||
`ExtraProperty`, and typed exceptions. **What it does not:** the truncation loop,
|
||
507 detection, any `valid-sync-token` handling (its `Error.kt` says outright
|
||
*"there is no logic for subclassing errors"*), mark-and-sweep, `initialIncomplete`
|
||
persistence, or multiget orchestration. And its error extraction only parses XML
|
||
content-types within a 20 KB excerpt at depth 1 — so a `<D:error>` served as
|
||
`text/html`, or buried behind a PHP stack trace (exactly what ownCloud and Baïkal
|
||
emit), yields no recovery. Add a raw-body substring fallback, as Evolution does.
|
||
|
||
### ⚠️ Scheduling has a ceiling the draft didn't price
|
||
|
||
"WorkManager with network constraints" was the entire treatment. Reality:
|
||
|
||
- An ordinary worker is documented for **< 10 minutes**. An initial full sync of a
|
||
large collection over a slow homelab link will exceed it. DAVx5's own
|
||
`workerWaitTimeout` is 10 minutes.
|
||
- Escalating to `setForeground` pulls in `FOREGROUND_SERVICE` +
|
||
`FOREGROUND_SERVICE_DATA_SYNC` (missing ⇒ `SecurityException` at targetSdk 34+),
|
||
a `tools:node="merge"` override on WorkManager's own service, and the
|
||
**Android 15 six-hours-per-24 `dataSync` budget** whose failure mode is a fatal
|
||
`RemoteServiceException`. Android 15 also **forbids starting a `dataSync` FGS
|
||
from `BOOT_COMPLETED`** — and we have a boot receiver.
|
||
- **Android 16 removed the shield**: jobs running alongside a foreground service
|
||
now obey the job runtime quota, and the `active` bucket is capped at 20 min /
|
||
rolling 60 min.
|
||
|
||
**Therefore:** make sync **chunked and resumable** — persist the sync-token/ETag
|
||
cursor per collection so a killed worker resumes rather than restarts. Hard socket
|
||
and wall-clock timeouts. Periodic sync is a plain `PeriodicWorkRequest`, no FGS.
|
||
"Sync now" from a visible screen uses `setExpedited(RUN_AS_NON_EXPEDITED_WORK_REQUEST)`
|
||
— and implement `getForegroundInfo` unconditionally, since omitting it crashes
|
||
below API 31 and we support 29. FGS only for user-initiated full syncs, with
|
||
`Service.onTimeout → stopSelf()` as a backstop. Play requires a video demo per
|
||
declared FGS type.
|
||
|
||
⚠️ **Be honest about cadence.** `PeriodicWorkRequest`'s 15-minute floor is
|
||
nominal. In the `rare` and `restricted` buckets network access is disabled
|
||
outright; Doze allows idle apps network roughly **once a day**. Combined with
|
||
unmetered-only, worst case is genuinely "once overnight". Promise eventual
|
||
consistency, and sync hard on app open and on connectivity-regained.
|
||
|
||
### ⚠️ Server reality — the matrix, audited
|
||
|
||
The draft listed six servers as a test matrix. What they actually do:
|
||
|
||
| Server | VTODO | The thing that will bite |
|
||
|---|---|---|
|
||
| **Nextcloud** (sabre) | ✅ | ⚠️ **Never round-trip a body fetched from a shared calendar** — see below. Never send `<D:limit>`. Per-calendar UID uniqueness ⇒ **409 `no-uid-conflict`**. Trashbin renames the href to `<name>-deleted.ics` ⇒ 403 on delete-then-recreate. MKCALENDAR is rate-limited 10/hour ⇒ 429, max 30 calendars ⇒ 403 |
|
||
| **Baïkal** (sabre) | ✅ | Handles `<D:limit>` and 507 **correctly** — the reference implementation for that path. Never prunes its change log, so tokens stay valid forever. Defaults to **Digest** auth |
|
||
| **Radicale** | ✅ | `supported-calendar-component-set` is **never enforced** — a VEVENT PUT into a VTODO-only collection is accepted. Advertises three reports it does not implement. Its VTODO time-range now implements all eight RFC 4791 §9.9 rows — **the widespread "Radicale doesn't do time ranges" claim is stale** |
|
||
| **SOGo** | ✅ | ⚠️ **Never invalidates a sync token** (`valid |= …` makes the check always pass) and tokens are **second-granularity**, so you re-receive up to a second of changes every sync. ⚠️ **The ETag is a row-version counter, and the body is regenerated per-principal** — same ETag, different bytes. Cannot create a VTODO-only collection at all |
|
||
| **Fastmail** | ✅ | ⚠️ **Reframe as supported.** The backend does VTODO fine; their own UI hides task-only calendars by design. Create collections as **mixed `VEVENT,VTODO`** so the list doesn't vanish from Fastmail's UI. Requires an app password; Basic plan has no CalDAV |
|
||
| **iCloud** | ⚠️ | **Reminders left CalDAV at iOS 13.** A new VTODO collection syncs bidirectionally but is **invisible in Reminders.app forever**. Market it as "store tasks in iCloud", never as "sync with Apple Reminders". Cheap detection: no VTODO-capable collection in the home set ⇒ upgraded account |
|
||
| **Google** | ❌ | ⚠️ **Drop it.** First-party docs: *"Doesn't support VTODO or VJOURNAL data"* and no MKCALENDAR. Refuse with an explanation, don't fail with a 401 |
|
||
|
||
**Two data-destruction landmines, both confirmed from server source:**
|
||
|
||
1. **Nextcloud rewrites task bodies on GET from a shared calendar.**
|
||
`CalendarObject::get()` strips `VALARM` on read-only shares, and for
|
||
`CLASS:CONFIDENTIAL` reduces the object to a VEVENT-shaped whitelist that
|
||
**deletes `DUE`, `STATUS`, `COMPLETED`, `PERCENT-COMPLETE`, `PRIORITY` and
|
||
`RELATED-TO`** — every property that makes it a task. **The ETag is left
|
||
untouched**, so `ETag ≠ md5(body)` and re-PUTting what you downloaded destroys
|
||
the task. Baïkal never does this.
|
||
2. **sabre runs vobject `REPAIR` on every PUT** unless you send
|
||
`Prefer: handling=strict` — adding UID/DTSTAMP/PRODID/VERSION — and when it
|
||
modifies the object it **suppresses the ETag response header**, so you must
|
||
re-GET rather than assume. It also 415s on **`DUE` < `DTSTART`**, value-type
|
||
mismatch between them, multiple UIDs, mixed component types in one resource,
|
||
and a present `METHOD`.
|
||
|
||
**Three consequences for the design:**
|
||
|
||
- **`supported-calendar-component-set` is not an invariant.** Unenforced on
|
||
Radicale, discarded by SOGo, immutable on sabre (403 if you try to change it).
|
||
Filter on it, but never rely on it.
|
||
- **VTODO scheduling exists nowhere.** sabre's own docs: *"We don't do VTODO
|
||
scheduling yet, and only support VEVENT."* Treat `ORGANIZER`/`ATTENDEE` on a
|
||
task as inert text to round-trip — which is a mercy, since it also means
|
||
scheduling never rewrites our objects or suppresses our ETags.
|
||
- **Never trust an ETag as a content hash** (SOGo, and Nextcloud shares).
|
||
|
||
### ⚠️ Writing: conditional PUT, and the conflict policy that had to change
|
||
|
||
**`If-None-Match: *` on create. `If-Match` on update and DELETE.** The draft said
|
||
"`If-Match` on every PUT", which omits the creation case entirely — RFC 4791
|
||
§5.3.2 asks for `If-None-Match: *` there, and all three reference clients send it.
|
||
Without it, a filename collision (two devices minting the same UID, or a sanitiser
|
||
folding two UIDs onto one name) makes the second PUT **silently destroy the
|
||
first**, with no ETag to protect it because we have never seen the resource.
|
||
|
||
⚠️ **412 means three different things** and the draft's single rule conflated them:
|
||
|
||
| On | Means | Do |
|
||
|---|---|---|
|
||
| create (`If-None-Match`) | the filename is taken | re-fetch that href; adopt if the UID matches, else regenerate the name as a UUID |
|
||
| update (`If-Match`) | the server has a newer version | conflict resolution, below |
|
||
| update, resource gone | no selected representation, so the condition is false — **spec-correct** (Radicale and DAViCal do this) | `HEAD` to disambiguate; 404 ⇒ delete-vs-edit, not a conflict |
|
||
|
||
⚠️ **The proposed conflict policy is unimplementable and is withdrawn.** The draft
|
||
said the local version would be *"preserved rather than discarded — a duplicate
|
||
task, marked, in the same list."* RFC 4791 §4.1 requires a **UID to be unique
|
||
within a collection**, and every target server enforces it: Nextcloud, Radicale
|
||
and SOGo all answer **409 `CALDAV:no-uid-conflict`**. So the preserved duplicate
|
||
can never be uploaded — same UID fails forever, a new UID forks a task that never
|
||
reconciles. The "visible clutter" the design wanted is either a permanently
|
||
failing row or a permanent fork.
|
||
|
||
**Pick one and write the consequence down** (open question 2, now with real
|
||
options): **server-wins and discard the local edit** — DAVx5's stated policy —
|
||
or **server-wins and fork under a new UID**, marked in the UI, with the new UID
|
||
persisted so the fork is first-class from that moment. Prompting is unavailable;
|
||
a background sync has nobody to ask.
|
||
|
||
⚠️ **The ETag may be weak or absent, and then `If-Match` can never succeed.**
|
||
RFC 4791 §5.3.4: when the server does not store your bytes verbatim, *"a strong
|
||
entity tag MUST NOT be returned"*. RFC 9110 §13.1.1: *"A weak entity-tag cannot be
|
||
used with If-Match."* On sabre this is the **default path** — `validateICalendar`
|
||
runs vobject `REPAIR` unless you send `Prefer: handling=strict`, and
|
||
`Server::createFile` then deliberately withholds the ETag. Worse, **weak ETags
|
||
also arrive from the user's reverse proxy**: any gzip-compressing nginx,
|
||
Cloudflare or Traefik in front of Nextcloud produces them, so the risk tracks the
|
||
user's deployment rather than their server software.
|
||
|
||
Therefore: send **`Prefer: handling=strict`** to sabre-based servers — the
|
||
cheapest single fix in this whole audit, since it preserves both our bytes and
|
||
the ETag. Request `Accept-Encoding: identity`. Strip `W/` and keep a weak flag.
|
||
**If a PUT returns no ETag or a weak one, discard it and re-fetch** for the strong
|
||
validator *and* the server's canonical body. Bound every 412 retry loop.
|
||
|
||
⚠️ **Errors are not one status.** RFC 4791 §5.3.2.1 defines **eleven**
|
||
preconditions, and the one we will hit most is not in the draft at all: **sabre
|
||
returns 415** for a VTODO whose `DUE` precedes `DTSTART`, whose `DUE`/`DTSTART`
|
||
value types disagree, or which carries a `METHOD`. The first two are reachable
|
||
from ordinary UI actions and must be validated client-side. Also: **507 MUST NOT
|
||
be auto-retried** (RFC 4918 §11.5 — quota exhaustion is common on hosted
|
||
Nextcloud, and a generic backoff loop violates the spec), and **5xx is not safely
|
||
retryable either** — a contradictory `RRULE`/`EXDATE` pair returns 500 from
|
||
Nextcloud and will do so forever.
|
||
|
||
So line-for-line with "a failed collection must not fail the account", add its
|
||
twin: ⚠️ **a failed resource must not fail the collection.** Per-resource
|
||
quarantine with a failure counter, not backoff. A single HTTP 400 has halted all
|
||
calendar sync in DAVx5 for weeks.
|
||
|
||
**DELETE needs the same care:** conditional on `If-Match`; **404/410 count as
|
||
success**; a resource deleted locally that was never uploaded is never DELETEd.
|
||
And Nextcloud's trashbin renames the href to `<name>-deleted.ics`, so
|
||
delete → recreate → delete the same href returns **403** — which task apps hit
|
||
constantly, because they reuse hrefs.
|
||
|
||
**href and UID are unrelated.** RFC 4791 §5.3.2 opens by saying the URL *"is
|
||
entirely arbitrary and does not need to bear a specific relationship"* to the
|
||
content, and `.ics` is MAY. Sanitise filenames following vdirsyncer's rule —
|
||
`a–zA–Z0–9_.-+`, **excluding `@`**, because some servers percent-encode it in the
|
||
path and then reject or "repair" the URL, and RFC 4791's own example UID is
|
||
`…@example.com`. Cap the basename around 200 bytes; fall back to a UUID.
|
||
|
||
### Manifest and permissions for phase 3
|
||
|
||
⚠️ The draft named only `INTERNET`. Actually needed: `INTERNET`,
|
||
`READ_SYNC_SETTINGS`, `WRITE_SYNC_SETTINGS`, `ACCESS_NETWORK_STATE` (merged in by
|
||
`work-runtime`, but it shows in F-Droid's permission diff), plus
|
||
`FOREGROUND_SERVICE` + `FOREGROUND_SERVICE_DATA_SYNC` if the FGS route is taken.
|
||
The authenticator `<service>` must be `android:exported="true"` guarded by
|
||
`android:permission="android.permission.ACCOUNT_MANAGER"` — note
|
||
`android.permission.ACCOUNT_AUTHENTICATOR` **does not exist**.
|
||
|
||
Also missing from `libs.versions.toml` entirely: `androidx.work`,
|
||
`androidx.hilt:hilt-work`, `androidx.browser`. With Hilt that means a
|
||
`HiltWorkerFactory`, removing the default `WorkManagerInitializer`, and an
|
||
`@EarlyEntryPoint` for the authenticator service.
|
||
|
||
### Two network facts for homelab users
|
||
|
||
- ⚠️ **Ship a `network-security-config` with `<certificates src="user"/>`.** Since
|
||
Android 7, a user who correctly installs their private CA into Android's store
|
||
is *still* not trusted by apps. Cleartext `http://` is blocked by default since
|
||
API 28; any escape hatch must be a narrow, warned, per-account opt-in — Play's
|
||
User Data policy requires modern cryptography in transit.
|
||
- ⚠️ **Android 17 / targetSdk 37 breaks LAN CalDAV.** Local network protections
|
||
become mandatory: TCP to a local address and `.local` resolution require the
|
||
runtime `ACCESS_LOCAL_NETWORK` permission. The failure mode is a **connection
|
||
timeout, not a `SecurityException`** — "my Nextcloud at 192.168.1.50 just
|
||
hangs", the worst bug-report shape there is. We are safe at targetSdk 36 (which
|
||
gets an implicit grant) and must **not** request it before targeting 37 — but
|
||
`compileSdk` is already 37 and Play's floor rises annually, so this is a
|
||
scheduled break aimed precisely at the self-hosting demographic.
|
||
|
||
---
|
||
|
||
## External mode's future
|
||
|
||
`STORAGE-AND-SYNC.md` keeps Posture A as a user choice; a later draft proposed
|
||
replacing it with a one-time importer. Premature — it argues against something
|
||
shipped and working — but it is the right question one phase early.
|
||
|
||
Once we sync ourselves, External mode's only job is reading tasks in someone
|
||
else's app. Costs are real and permanent: capability divergence (tasks.org's fork
|
||
is DB 22 and lacks `is_recurring`, which is why `TaskMapper.task` derives
|
||
recurrence from `rrule`/`rdate`), per-backend UI degradation forever, two
|
||
dangerous permissions in a static manifest, a doubled device matrix.
|
||
|
||
**Decide before phase 1** — it determines whether the mapper and UI stay
|
||
dual-capable. Open question 3.
|
||
|
||
If retired, the importer spec is sound: read-only, one-time, idempotent; identify
|
||
local lists by `ACCOUNT_TYPE`, treating unknown types as synced; **import the
|
||
`Properties` table, not just `Tasks`** (categories, alarms, `RELATED-TO`, `X-`
|
||
props — the commonly forgotten half); preserve `_UID`s; persist
|
||
`(authority, _ID, uid)` for idempotency; and while scanning, read the *names* of
|
||
synced lists so the UI can say *"these 3 lists come from cloud.example.de — add
|
||
that account to bring them back"*.
|
||
|
||
---
|
||
|
||
## Play compliance
|
||
|
||
⚠️ Absent from the first draft entirely.
|
||
|
||
- **A privacy policy is mandatory regardless of collection**, linked both in Play
|
||
Console and **inside the app**.
|
||
- **"Not collected" is not defensible.** Play defines collection as transmitting
|
||
data off the device *irrespective of recipient*. Neither the on-device nor the
|
||
ephemeral exemption applies, and the E2EE exemption doesn't survive TLS to a
|
||
server that reads plaintext. File **Collected, not Shared**, encrypted in
|
||
transit. There is no credentials category, but "authentication information" is
|
||
explicitly named as personal and sensitive data.
|
||
- **Account Deletion policy does not apply** (offline-created accounts are out of
|
||
scope), but ship a "Remove account and delete local data" action anyway —
|
||
cheap insurance against a reviewer pattern-matching.
|
||
- **Do not ship `REQUEST_IGNORE_BATTERY_OPTIMIZATIONS` in the Play build.**
|
||
Generic server sync is not on the acceptable-use list. Use
|
||
`ACTION_IGNORE_BATTERY_OPTIMIZATION_SETTINGS`. DAVx5 declares the former and is
|
||
F-Droid-safe on it; we would not be.
|
||
|
||
---
|
||
|
||
## What lands in floret-kit
|
||
|
||
| Candidate | Kit module | Note |
|
||
|---|---|---|
|
||
| DAV client + iCalendar parse/serialise | new, e.g. `core-dav` | **the big one.** Calendula needs the same primitives. Design for two consumers from the start |
|
||
| Sync-adapter + authenticator scaffolding | new, e.g. `core-sync` | including the stub-adapter→WorkManager bridge, which is pure mechanics |
|
||
| Nextcloud Login Flow v2 | with `core-dav` | pure protocol, zero task knowledge |
|
||
| Credential storage | kit | Keystore mechanics, not domain |
|
||
| Third-party licence screen | kit | Calendula needs it the moment it takes any of the above |
|
||
|
||
**Stays app-local:** the VTODO ↔ `TaskContract` mapper (domain, and where all the
|
||
judgement calls live), `ICalendarWriter`, conflict policy and its UI, and
|
||
everything about storage modes.
|
||
|
||
---
|
||
|
||
## Test strategy
|
||
|
||
- **Round-trip corpus first** — VTODO fixtures from each target server; assert
|
||
`parse → store → serialise` is byte-stable for untouched properties. The
|
||
mapper's specification, not its regression net.
|
||
- **Server matrix:** Nextcloud, Radicale, Baïkal (**on Digest**), SOGo, Fastmail,
|
||
iCloud — with the per-server traps above as named test cases. Google is out.
|
||
Two that must be explicit tests, because both silently destroy data:
|
||
**round-tripping a task from a shared Nextcloud calendar**, and **an
|
||
ETag-unchanged body change on SOGo**.
|
||
- **Conflict scenarios:** concurrent edit, delete-vs-edit, collection removed
|
||
server-side, credentials revoked mid-sync.
|
||
- **Interop:** edit the same task from Nextcloud web and from tasks.org + DAVx5.
|
||
- **Device, not Robolectric,** for account paths — `ProviderAccountCleanupTest`
|
||
skips on ARM64. Two specific device cases the audit added: **restore a cloud
|
||
backup onto a fresh device** and confirm nothing is pruned; **remove the
|
||
account** and confirm only our lists go.
|
||
- ⚠️ **Minified-build tests.** `:app` runs `isMinifyEnabled` + `isShrinkResources`
|
||
in release, and `proguard-rules.pro` already documents two R8-pruning outages.
|
||
ical4j resolves through seven `META-INF/services` files and instantiates its
|
||
cache from a class-name string — it needs `-keep class net.fortuna.ical4j.** { *; }`
|
||
plus ~8 `-dontwarn` lines, and is effectively unshrinkable. Without them this
|
||
fails in `release` only.
|
||
|
||
---
|
||
|
||
## Open questions
|
||
|
||
1. **dav4jvm + cert4android distribution *and* Java target.** Scoped JitPack,
|
||
vendor, or in-house verbs? Now compounded: 4.x requires **Java 21** and we
|
||
target 17 across `:app` and all of floret-kit. Vendoring recompiles at our own
|
||
target and freezes the API churn — it looks better than it did. Before phase 2.
|
||
2. **Conflict policy** — preserve-local-on-412, or documented LWW?
|
||
3. **External mode** — survives, or becomes an importer? Before phase 1.
|
||
4. ~~**Canonical recurring-completion behaviour**~~ **closed:** the store writes
|
||
model (a), `RECURRENCE-ID` overrides sharing the master's UID. Read all four.
|
||
5. **The DAVx5 enum ask** — worth filing, and what compatibility we owe if it
|
||
lands. ⚠️ Reshaped: it now means "sync into an app that publishes no provider".
|
||
6. ~~**Does Local→Synced migrate, or do synced lists start empty?**~~ **closed:**
|
||
neither — attaching an account to a list is an `UPDATE`, so there is nothing
|
||
to migrate.
|
||
7. ~~**lib-recur — pin at 0.12.2, or rewrite the provider's iterators?**~~
|
||
**closed** with the provider's deletion; `:app` owns the version.
|
||
|
||
Answered elsewhere and **not** open: the account model (`AccountManager` **plus a
|
||
stub sync adapter**), `ical4android` (superseded by `synctools`, GPLv3), and the
|
||
storage question — which was reopened once, answered the other way, and is now
|
||
shipped.
|
||
|
||
---
|
||
|
||
## Dead ends — do not revisit
|
||
|
||
- **Depending on DAVx5 for sync.** Settled in `STORAGE-AND-SYNC.md`.
|
||
- **`synctools` / `ical4android`.** GPLv3. The temptation recurs because it does
|
||
exactly the right mapping against exactly our schema.
|
||
- ~~**Rewriting storage to Room before sync exists.**~~ ⚠️ **This one was
|
||
revisited, and it was right to.** The phase-1 audit measured the provider's
|
||
sync bookkeeping — the reason it was kept — and found most of it broken, absent
|
||
or unusable (the table above). Reasoning in
|
||
[`STORAGE-DECISION.md`](STORAGE-DECISION.md). Kept here as a reminder that a
|
||
dead end is only dead against the evidence that closed it.
|
||
- ⚠️ **AccountManager + WorkManager with no registered sync adapter.** Not a
|
||
design choice — a silent no-op at targetSdk ≥ 34.
|
||
- ⚠️ **Writing through the `instances` URI as a sync adapter.** The flag is
|
||
ignored there; every such write dirties the row and forks an override.
|
||
- ⚠️ **Hand-rolled TrustManager to avoid a GPL licence cert4android does not
|
||
have.**
|
||
|
||
---
|
||
|
||
## Related
|
||
|
||
- [`STORAGE-AND-SYNC.md`](STORAGE-AND-SYNC.md) — where task data lives. This is
|
||
its step 5.
|
||
- [`../provider/PROVENANCE.md`](../provider/PROVENANCE.md) — every deviation from
|
||
upstream dmfs. Change 1 is load-bearing for the account model here.
|
||
- [`ARCHITECTURE.md`](ARCHITECTURE.md) §4 — the data seam the adapter writes
|
||
underneath.
|