Binary format
Every core CRDT type (PeerId, HybridLogicalClock, OperationId,
FugueElementID, VersionVector, Change, Snapshot) has a compact,
self-describing binary representation, through toBytes() / toUint8List()
and a matching fromBytes() / fromUint8List().
This is the canonical format: the persistence adapters store it, and
crdt_socket_sync sends it. It is also a public API you can use directly, to
build your own storage or sync layer. The byte layout and size of each type are
documented in the dartdoc of its toBytes() / fromBytes(), on
pub.dev: a table here would
only drift out of sync.
End to end binary
- Operation payloads inside a
Changeare produced by the handler'sValueCodec<T>(see Your own value types). - Each entry of
Snapshot.datais produced by the handler'sgetSnapshotState(). The built-in handlers reuse the sameValueCodec<T>to encode their items.
So the whole path (operation payload → Change → Snapshot) is binary. JSON
appears only as the default ValueCodec<T>, when you pass none.
Two levels of versioning
Snapshot.schemaVersion (currently 1, added in crdt_lf 4.0.0) covers only
the wrapper: the snapshot id (derived from the version vector), the version
vector, and the framing of the per-handler entries. A reader accepts exactly
this version.
Inside an entry, the layout belongs to the handler that wrote it, and so does its version. A handler can change its layout without moving a byte that any other handler reads.
The rules of a blob version
Every built-in handler follows these rules, and a handler of your own should too.
- The first byte of the blob is its version.
snapshotHeader()returns a builder that already holds it; write the state after it. snapshotBlobVersionis the version this build writes. It defaults to1. Raise it when the layout changes.minReadableSnapshotBlobVersionis the oldest version this build reads. It defaults tosnapshotBlobVersion, so a build reads only what it writes.- The reader accepts the range
minReadableSnapshotBlobVersion..snapshotBlobVersion.readSnapshotHeader(bytes)checks the first byte and returns the version and the offset of the state. A version outside the range throws aFormatException("newer than this build reads" or "older than this build reads"). The blob is refused whole: it is never read as far as it happens to parse. - The handler spec declares the same range.
HandlerFormats(blobVersions: BlobVersionRange(min, max))must match the two getters. An assertion checks it when the handler registers.
int get snapshotBlobVersion => 2;
// Still reads the layout of version 1, and upgrades it.
int get minReadableSnapshotBlobVersion => 1;
Uint8List getSnapshotState() {
final out = snapshotHeader(); // writes the byte `2`
// ... write the state in the layout of version 2
return out.toBytes();
}
MyState _readSnapshot(Uint8List bytes) {
final header = readSnapshotHeader(bytes); // throws outside 1..2
return switch (header.version) {
1 => _readV1(bytes, header.offset),
_ => _readV2(bytes, header.offset),
};
}
Changing a layout
| You want to | Do this | What happens |
|---|---|---|
| change the layout and keep reading the old one | raise snapshotBlobVersion, keep minReadableSnapshotBlobVersion, branch on the version | a build that does not read the new version refuses the new blobs (SnapshotBlobTooNew in the sync handshake) until it updates |
| stop reading an old layout | raise minReadableSnapshotBlobVersion | this build refuses data still in the old layout (SnapshotBlobTooOld in the sync handshake) |
A blob written without snapshotHeader() has no version byte. The handler
still reports the range 1..1, so the sync handshake has nothing real to
compare: such a handler is not protected by the check. Always write the
header.
The Fugue sequence handlers (CRDTFugueTextHandler, CRDTFugueListHandler,
CRDTFugueRichTextHandler) show what that room is for. Their blob groups elements into runs of
consecutive ids from the same peer, instead of one entry per element. It also
carries the deleted elements inside those runs, marked by a bit, so a deletion
keeps its identity and its place instead of leaving a gap.
Changes and blob versions on the wire
Each handler kind states the operation kinds it decodes and the snapshot blob
versions it reads (HandlerFormats). That is what the sync handshake compares,
so a peer that cannot read a document is refused before it breaks: see
Versions and compatibility.