docs(#1445): note at-most-once GC concurrency semantics#189
Open
dimitri-yatsenko wants to merge 2 commits into
Open
docs(#1445): note at-most-once GC concurrency semantics#189dimitri-yatsenko wants to merge 2 commits into
dimitri-yatsenko wants to merge 2 commits into
Conversation
Spec-first pair for the Renderable Protocol landing in DataJoint 2.3 (per user direction 2026-06-23, bringing T3.2 back into 2.3 scope). New files: - src/reference/specs/renderable.md — normative spec for the Renderable Protocol. Covers signature, return-value shape constraints (primitives / lists / dicts mapping to Spark ArrayType / StructType / MapType), why the contract is a Protocol rather than an abstract method on Codec, eligibility detection via isinstance, out-of-scope items, and two worked example codec implementations (FloatArrayCodec, Image2DCodec, PointWithLabelCodec). - src/explanation/renderable-codecs.md — explainer. Covers the Bronze/Silver layer model (CDC mirror vs typed silver layer), why <blob@> is bronze-only, what typed renderable codecs are, the design rationale for the Protocol pattern (smaller OSS surface, cleaner opt-in, no churn for existing plugins, structural typing), what's out of scope, and a decision guide for choosing codecs in a new pipeline. Nav entries added: - Reference > Specifications > Type System > Renderable Codec Protocol - Concepts > Storage > Renderable Codecs Implementation (against this spec) follows in datajoint-python; the addition is small (~10 lines: a runtime_checkable Protocol declaration in src/datajoint/rendering.py, re-exported as dj.Renderable). Examples use core DataJoint types (float64, int32) per project convention. Cross-links to codec-api.md (the base Codec interface that renderable codecs extend by composition, not inheritance).
T3.1 deferral holding pattern for 2.3: document current behavior so quarantine-based serialization can land in 2.4 without retracting stronger claims.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
T3.1 of the 2.3 release plan defers the two-phase transaction-safe GC (#1445) to 2.4 (effort + design-needed). To avoid retracting stronger claims later, add a one-paragraph note to
how-to/garbage-collection.mdclarifying that the current GC is at-most-once: an object inserted between scan and delete may be briefly orphaned, and a future release will add quarantine-based serialization.Test plan
mkdocs serve— note renders in the how-to and reads cleanly