Refactor/active sync

[TDannhauer]

Nag: ActiveSync task lists (separate folders, web ↔ mobile device)

Summary

Nag supports web-side create / update / delete of task lists when ActiveSync runs with separate task folders (activesync_no_multiplex). List changes update preferences, invalidate folder hierarchy in the device cache, and rely on the ActiveSync mobile device to run FolderSync (FolderHierarchy:Add / Remove) before item-level SYNC on Tasks:<share-id>.

Companion changes in Horde ActiveSync (Ping.php, Collections.php) are documented in a separate PR; this document covers Nag only.

---

Background: two layers of sync

With activesync_no_multiplex enabled, each synced task list is exposed as its own ActiveSync folder:

Layer	Protocol	What moves
Folder hierarchy	FolderSync	Which task lists exist (Add / Update / Remove)
Items	SYNC on Tasks:<share-id>	Tasks inside one list

Important constraints:

1. sync_lists preference — Nag only exposes lists selected here to ActiveSync (Nag_Api::sources(..., $sync_only = true)). The default list is always included via prefs logic.
2. No server push for new folders — After a web-side list change, the device must run FolderSync; there is no out-of-band notification that creates folders on the mobile device.
3. Separate PHP sessions — Web UI and activesync.php do not share in-memory prefs; writes must hit storage immediately (persistPrefs()).
4. Device cache vs DB state — Per device, Horde keeps a sync cache (folder UIDs, collection synckeys, hierarchy key) and foldersync / collection state in the database. These must stay consistent; aggressive cache wipes cause KEYMISMATCH, full resets, and broken PING.

---

Problem

With separate task folders enabled, web-side create, update, and delete of task lists must propagate to ActiveSync mobile devices. List CRUD must update sync_lists, persist preferences for the next sync request, invalidate folder hierarchy on registered devices, and update the per-device sync cache without breaking foldersync state or PING.

This PR implements that workflow for Nag list CRUD and for manual sync_lists preference changes.

Design constraints

Cache and preference handling follow these rules:

Rule	Rationale
Notify via hierarchy = '0' only	Forces FolderSync on the next device request without deleting stored foldersync state
Do not clear foldersync DB state on list add	Avoids synckey mismatch (KEYMISMATCH) and full device folder cache reset
Do not prune folder cache in notifyActiveSyncOfTaskListChange()	Folder UIDs must remain until FolderSync delivers FolderHierarchy:Remove
On web delete, remove collections only	Stops PING/SYNC on removed lists while folder mappings stay for FolderSync Remove
Prune only from sync_lists on_change	User-driven shrink of synced lists; allowlist is array_values(getSyncLists()) (share ids, not numeric array keys)
persistPrefs() after web writes	Web UI and activesync.php use separate sessions; prefs must be stored before the next sync request
List all devices for auth + original user id	Device rows may be registered under either Horde id; both must be updated

---

Solution overview

sequenceDiagram
    participant Web as Nag web UI
    participant Nag as Nag.php
    participant Prefs as DB prefs
    participant Cache as Device sync cache
    participant Device as ActiveSync mobile device

    Web->>Nag: addTasklist / updateTasklist / deleteTasklist
    Nag->>Prefs: sync_lists, display_tasklists (persistPrefs)
    Nag->>Nag: expireListCache (shares)
    alt delete
        Nag->>Cache: remove collections only (keep folder mapping)
    end
    Nag->>Cache: hierarchy = 0, bump timestamp
    Device->>Cache: FolderSync
    Cache-->>Device: FolderHierarchy Add/Remove
    Device->>Cache: SYNC per Tasks:share-id

Loading

Behaviour:

Event	Nag actions
Create (web or API with synchronize)	Add to sync_lists, persistPrefs, invalidate hierarchy
Update	persistPrefs, invalidate hierarchy
Delete	Update prefs, remove collections from device cache, invalidate hierarchy; keep folder cache until FolderSync Remove
sync_lists pref change	pruneActiveSyncTaskCache(), touchActiveSyncDeviceCaches()

---

File-by-file changes

lib/Nag.php

Central implementation. New and changed behaviour:

Task list CRUD hooks

Method	Change
addTasklist($info, $display, $sync)	Third parameter $sync: when true, adds share id to sync_lists and calls notifyActiveSyncOfTaskListChange(). Web create uses $sync = true.
updateTasklist()	Calls notifyActiveSyncOfTaskListChange() after save (rename/color/description → folder Update on next FolderSync).
deleteTasklist()	Removes list from display_tasklists and sync_lists, deletes share/storage, then removeActiveSyncTaskListCollectionsFromDeviceCache() (collections only), then notifyActiveSyncOfTaskListChange(). Does not remove folder mappings from device cache.

Session and prefs

Method	Purpose
refreshWebSessionState()	Called from initialize(): reloads prefs from storage and expires nag_shares list cache so the web UI sees current data after external changes.
persistPrefs()	$GLOBALS['prefs']->store() — required so ActiveSync requests see web-written sync_lists immediately.
addTasklistToDisplayListsPref() / removeTasklistFromDisplayListsPref()	Maintain display_tasklists without duplicate entries; mirror updates in $GLOBALS['display_tasklists'] when set.
addTasklistToSyncLists() / removeTasklistFromSyncLists()	Maintain serialized sync_lists via _getPrefList / _setPrefList.
getSyncLists()	Returns share ids from pref, filtered to lists that still exist; falls back to default editable list. Drives ActiveSync folder list via Nag_Api::sources(..., true).

ActiveSync notification and cache management

Method	Purpose
notifyActiveSyncOfTaskListChange()	persistPrefs(), expire share cache, requestActiveSyncFolderHierarchySync() only — no pruneActiveSyncTaskCache() here. Optional user notification in Nag UI.
requestActiveSyncFolderHierarchySync()	For each device with non-empty cache: set hierarchy = '0', updateTimestamp(), save(). Skips devices with empty folder/collection cache. Requires activesync_no_multiplex. Does not delete foldersync rows in SQL.
removeActiveSyncTaskListCollectionsFromDeviceCache($tasklistId)	New. Removes SYNC/PING collection entries for Tasks:<id> on all user devices; folders stay for FolderSync Remove. Used from deleteTasklist().
removeActiveSyncTaskListFromDeviceCache($tasklistId)	Full cleanup (folders + collections). Not used on web delete; reserved for exceptional/manual repair.
_purgeActiveSyncTaskListCollections($cache, $tasklistId)	Shared helper: removeCollection($id, true) for matching task collections.
pruneActiveSyncTaskCache($allowedShareIds = null)	Removes folder/collection cache entries not in allowlist. Default allowlist: array_values(self::getSyncLists()) (share ids → Tasks:<id>). Used from prefs on_change only.
touchActiveSyncDeviceCaches()	Bumps cache timestamp on all devices (wakes stale PING / concurrent sync detection). Used after prefs sync_lists change.
_isActiveSyncEnabled()	Checks $GLOBALS['conf']['activesync']['enabled'].
_listActiveSyncDevicesForUser()	Merges listDevices() for auth id and getAuth('original'); deduplicates by device_id + device_user.

---

lib/Api.php

Area	Change
addTasklist(..., $params)	Documents and passes synchronize parameter to Nag::addTasklist(..., true, !empty($params['synchronize'])). ActiveSync / RPC callers that create a list with synchronize => true add it to sync_lists and trigger hierarchy invalidation — same as web create.
sources($writeable, $sync_only)	When $sync_only is true, intersects task lists with Nag::getSyncLists(). This is how the ActiveSync connector builds the server folder list for FolderSync diffs.

No change to delete/update API surface beyond delegating to Nag::updateTasklist / share layer; delete goes through shares/forms calling Nag::deleteTasklist.

---

lib/Form/CreateTaskList.php

execute() calls:

return Nag::addTasklist($info, true, true);

• Second argument: add to display lists.
• Third argument: add to sync_lists and notify ActiveSync.

Ensures a task list created in the web UI is synced to the ActiveSync mobile device without a separate prefs step.

---

lib/Form/DeleteTaskList.php

Unchanged call path: Nag::deleteTasklist($this->_tasklist). Documented here because it is the web delete entry point that triggers collection cleanup + hierarchy invalidation described above.

---

config/prefs.php

Setting	Role
sync_lists	Multi-select of extra task lists to sync (plus default). on_change: ensures default list remains selected; if ActiveSync + activesync_no_multiplex, runs Nag::pruneActiveSyncTaskCache() and Nag::touchActiveSyncDeviceCaches() with user notification. This is the only supported path for shrinking the cached folder set when the user removes lists from sync in prefs UI.
activesync_no_multiplex	Checkbox “Support separate task lists?” — must be enabled for per-list folders and for all Nag ActiveSync cache helpers in this PR.

Prefs group activesync includes both members for the Nag preferences screen.

---

What is intentionally not in this PR

Topic	Where
PING requires FolderSync when hierarchy key missing	ActiveSync Ping.php
Orphan collection cleanup on PING (StateGone)	ActiveSync Collections.php
Horde Core connector multiplex checks	Horde_Core_ActiveSync_Connector (existing)

Enable activesync_no_multiplex in Nag prefs and select lists under sync_lists before testing separate folders.

---

Test plan

• Web create — New list appears in Nag; sync_lists contains share id; after the mobile device syncs, new task folder appears; tasks sync into it.
• Web rename / color — Folder Update on device after FolderSync.
• Web delete — List gone in Nag; device receives Remove on FolderSync; no endless PING for old folder UID; tasks in that folder gone on device.
• Prefs: remove list from sync_lists — Prune drops extra cached folders; FolderSync removes folders not in allowlist.
• Mobile device create (if supported) — addTasklist(..., synchronize => true) adds to sync_lists and invalidates hierarchy.
• Logs — No KEYMISMATCH on web add; stable PING after delete when deployed together with the ActiveSync companion PR.

---

Deployment

• Apply together with the Horde ActiveSync companion PR (Ping.php, Collections.php) for hierarchy checks and orphan collection handling during PING.
• Rename via web updates share metadata and invalidates hierarchy; client behaviour for folder Update depends on the mobile ActiveSync implementation.
• Stale entries in serialized sync_lists for deleted shares are filtered by getSyncLists() at runtime; saving prefs rewrites the stored list.

---

Related configuration

Nag preferences → ActiveSync:
  ☑ Support separate task lists?  (activesync_no_multiplex)
  Select sync_lists …

Horde-wide: $conf['activesync']['enabled'] must be true.

[ralflang]

This PR is part of a companion pair: #25 invalidates hierarchy (sets hierarchy = '0') when task lists change on the web; horde/ActiveSync#32 makes PING
detect that invalidation and return FolderSync-required immediately instead of waiting through the heartbeat. Neither works without the other

They
should ship together.

Noteworthy issues:

Missing try/catch in deleteTasklist() for ActiveSync calls
The getSyncLists() method should use _getPrefList() to match the new pattern
Main problem is performance: The refreshWebSessionState() on every page load is expensive to compute.

[TDannhauer]

performance feedback embraced, issue solved by #28