Retention can delete rows not yet exported by a configured sink (wire wait_for_sink_ack ack-coupled eviction)

[philcunliffe]

Problem

Retention (src/core/cache/retention.js) deletes cache rows purely by age —
"delete rows older than the retention window" (default 30 days, per-dataset
configurable). It has no awareness of how far any sink has progressed, so a sink
that falls further behind than the retention window can have its un-exported
rows position-deleted before it ever ships them: silent data loss.

A healthy sink ships every tick (~1/min) and is never close to the window, so
this does not happen in steady state. It bites only when a sink is broken/down/
misconfigured for longer than the retention window, when the window is set very
short, or when a backlog can't drain inside the window.

This is pre-existing behaviour on master — it is not introduced by
#159. What #159 adds, for the
first time, is the mechanism that makes a fix possible: the row-resident
monotonic _hyp_ingest_seq watermark (LLP 0040). #159 deliberately leaves
retention unchanged to stay small; this issue tracks the follow-up.

Why it's now fixable

The _hyp_ingest_seq watermark + the persisted per-(sink instance, partition)
bookmark mean retention can finally implement "evict only past the minimum
exported watermark across all configured sinks" (wait_for_sink_ack).

The config knob already exists but is parsed-but-unwired today:
normalizeConfig in src/core/cache/retention.js reads wait_for_sink_ack
(annotated @ref LLP 0013#open-question), but nothing consumes it.

Design considerations (why it's a separate PR, not a one-liner)

1. Cross-subsystem read. Retention lives in src/core/cache/; the watermarks
   live under each sink plugin's PluginPaths.stateDir
   (src/core/sinks/incremental.js scopes them per instance). Retention has no
   current path to enumerate sink instances, so wiring this is a new
   cache→sink-state dependency (or an inversion where sinks publish a
   cache-visible min-watermark). That store-location choice should be made
   deliberately.
2. Dead-sink escape hatch (required). A naive "never evict past the slowest
   sink" rule means a permanently dead or mis-subscribed sink wedges
   retention → the cache grows unbounded. A staleness bound is needed so a sink
   that hasn't advanced in too long stops counting toward the eviction floor.
3. Backlog/first-export interaction. A sink with no watermark yet (legacy
   null-seq backlog) has effectively "exported nothing" — the floor must treat
   that correctly without freezing all eviction.

References

• Spec: LLP 0039 — incremental sink reads
• Design: LLP 0040 §6 risk #5 —
  "This design does not change retention — a lagging sink can still have
  un-exported rows pruned (data loss). Decide whether to wire ack-coupled
  eviction alongside this."
• Open question: LLP 0013
  — wait_for_sink_ack evict-on-ack vs evict-on-retention
• Deferred from: Incremental sink reads: per-row ingest-seq watermark for forward + blob sinks (LLP 0039/0040) #159 (kept small)

Acceptance (sketch)

• With wait_for_sink_ack enabled, a row with _hyp_ingest_seq above the
  minimum exported watermark across configured sinks is not evicted, even if
  older than the retention window.
• A sink stale beyond a configured bound stops holding the eviction floor (cache
  growth stays bounded).
• Default behaviour (wait_for_sink_ack off) is unchanged.