The adapters
The three adapters keep the same contract, so most apps only need the line that opens the database (see Which adapter). This page is for what is specific to each one: setup, what is stored where, and the lower-level storages.
Each adapter is a bridge to a storage package. The adapter stores CRDT changes and snapshots in that package's database, and nothing else. How to install, configure and run the database itself (native libraries, Flutter setup, platforms, file locations) is in the documentation of that package:
| Adapter | Built on | Read its docs for |
|---|---|---|
crdt_lf_sqlite | sqlite3 | the native SQLite library on each platform |
crdt_lf_hive | hive | Hive.init and the storage folder, in Dart and in Flutter |
crdt_lf_drift | drift | opening a database on each platform (QueryExecutor) |
Install and open
- SQLite
- Hive
- Drift
crdt_lf_sqlite stores the documents in a SQLite database opened with
sqlite3. Set up sqlite3 for your
platforms as its documentation explains, then:
dart pub add crdt_lf_sqlite
import 'package:crdt_lf_sqlite/crdt_lf_sqlite.dart';
// Open (or create) a database file...
final backend = CRDTSqlite.open('./my_app_data.db');
// ...or an in-memory database (useful for tests).
final memory = CRDTSqlite.memory();
// ...or a connection you already manage.
final shared = CRDTSqlite.fromDatabase(myDatabase);
backend.close();
crdt_lf_hive stores the documents in Hive
boxes. Initialize Hive (Hive.init with a folder; in Flutter, a folder of the
app) as its documentation explains, then:
dart pub add crdt_lf_hive hive
import 'package:crdt_lf_hive/crdt_lf_hive.dart';
import 'package:hive/hive.dart';
Hive.init('./my_app_data');
// Registers the type adapters of Change and Snapshot. Pass changeTypeId /
// snapshotTypeId when your other Hive adapters already use the default ids.
CRDTHive.initialize();
final backend = await CRDTHive.open();
await backend.close();
crdt_lf_drift stores the documents in a Drift
database. Choose and open the Drift executor for your platforms as its
documentation explains, then:
dart pub add crdt_lf_drift
import 'dart:io';
import 'package:crdt_lf_drift/crdt_lf_drift.dart';
// Open (or create) a database file...
final backend = CRDTDrift.open(File('./my_app_data.db'));
// ...or an in-memory database (useful for tests).
final memory = CRDTDrift.memory();
// ...or a CRDTDriftDatabase on a QueryExecutor of your choice: on the web,
// the executor of a Drift web database.
final custom = CRDTDrift.fromDatabase(CRDTDriftDatabase(myExecutor));
await backend.close();
Then backend.openDocument(id) and the rest of
Persistence work the same on all
three.
How data is stored
- SQLite
- Hive
- Drift
One database file holds every document, in three tables:
CREATE TABLE changes (
document_id TEXT NOT NULL,
author TEXT NOT NULL,
hlc_l INTEGER NOT NULL,
hlc_c INTEGER NOT NULL,
bytes BLOB NOT NULL,
PRIMARY KEY (document_id, author, hlc_l, hlc_c)
);
CREATE TABLE snapshots (
document_id TEXT NOT NULL,
snapshot_id TEXT NOT NULL,
bytes BLOB NOT NULL,
PRIMARY KEY (document_id, snapshot_id)
);
CREATE TABLE peers (
document_id TEXT NOT NULL,
peer_id TEXT NOT NULL,
PRIMARY KEY (document_id)
);
A change is named by its author and its clock, not by one text id. An
OperationId is a peer and a clock, and kept apart SQL can compare it,
which is what a version vector asks. The primary key is then already the index
that comparison wants. The clock takes two columns because l is 48 bits and
c is 16: together they stay inside the 53 bits an integer keeps exactly in
JavaScript.
- Transactions.
storageForDocumenthands back aCRDTSqliteDocumentStorage, which backstransaction()with a real SQLite transaction: a prune drops the covered changes and rewrites the survivors, and either all of it lands or none of it does. It is built on savepoints, so a batch that opens a transaction of its own nests inside instead of failing. - Closing.
close()on a document storage does nothing on purpose: one database file holds every document, so the connection isCRDTSqlite.close()'s to release. - Schema version. The database carries its schema version in
PRAGMA user_version, and this build writes version 2. A database written by0.2.0is version 1: it has nopeerstable and names a change with a singlechange_idcolumn. The first open rebuilds it inside one savepoint, so half a rebuilt table is never left behind. The split reads the old column and not the stored bytes, so a change whose blob this build cannot decode migrates all the same.
Each document gets boxes of its own:
- changes:
{boxName}_{documentId}(defaultchanges_{documentId}); - snapshots:
{boxName}_{documentId}(defaultsnapshots_{documentId}).
Two boxes are shared, one per app:
peer_ids: thePeerIdeach document writes under, keyed by document id. A box of its own would cost an open for a single string. The value is text, so no type adapter and no type id are involved.documents: the list of document ids the backend holds. Hive cannot list its boxes, soCRDTHive.open()keeps this small registry. A document costs one extra row, written with its first write (a change, a snapshot or its identity). A document stored before this registry existed is not on the list until it is opened once; its data is untouched either way.
Change and Snapshot are stored through one TypeAdapter each, as the
binary blob of toBytes().
-
Reads answer at once. A Hive box holds its entries in memory, so
getChanges,getSnapshots,countandcontainsSnapshotare not futures. Writes go through the box journal and stay asynchronous. -
No transactions. Hive has none, so
transaction()just runs its body. That is still conformant: every step the persistence takes is safe to repeat. -
Closing. A box closes with its last user.
close()on aCRDTHiveDocumentStorageor on a backend leaves open the boxes that another storage or backend still uses, so two replicas, or a list and an editor, can share them.deleteDocumentcloses the boxes of that document for everyone.CRDTHive.closeAllBoxes()closes every Hive box the app has open, yours included: use it when the app shuts down. -
Custom names. Every box name can be changed:
final backend = await CRDTHive.open(changesBoxName: 'my_changes',snapshotsBoxName: 'my_snapshots',); -
Type ids. Make sure your own Hive adapters use type ids different from the two of
CRDTHive.initialize().
One database holds every document, in three tables:
changes, keyed by(document_id, author, hlc_l, hlc_c), with the change itself in abytesblob column;snapshots, keyed by(document_id, snapshot_id), same blob column;peers, one row per document: thePeerIdthis device writes it under.
A change is named by its author and its clock, for the same reason as in the SQLite adapter: SQL can compare the two parts, which is what a version vector asks.
- Asynchronous end to end. Every method returns a
Future, soCRDTDocumentPersistence.openSyncdoes not work on it. Useopen, orbackend.openDocument. - Transactions.
storageForDocumenthands back aCRDTDriftDocumentStorage, which backstransaction()withdatabase.transaction(...): a prune lands all at once or not at all. - Closing.
close()on a document storage does nothing on purpose: the connection isCRDTDrift.close()'s to release. - Schema version. The
peerstable arrived with schema version 2. A database written by version 1 gains it on the next open, and its changes and snapshots stay as they are.
The lower-level storages
backend.openDocument uses three storages per document. You can use them
directly, for example to read part of the log or to build your own opening.
- SQLite
- Hive
- Drift
Every method answers on the spot: getChanges() gives a List<Change>,
saveChange() gives void.
final changes = backend.changeStorageForDocument('doc-123');
changes.saveChanges([change1, change2]);
final all = changes.getChanges();
final missing = changes.getChanges(newerThan: theirVersion);
print('Total changes: ${changes.count}');
final snapshots = backend.snapshotStorageForDocument('doc-123');
snapshots.saveSnapshot(snapshot);
final latest = snapshots.getSnapshots();
final peers = backend.peerIdStorageForDocument('doc-123');
final document = CRDTDocument(
documentId: 'doc-123',
peerId: peers.loadOrCreate() as PeerId, // synchronous here
);
The storages open their boxes, so they come back as futures. Reads then answer at once.
final changes = await CRDTHive.openChangeStorageForDocument('doc-123');
await changes.saveChanges([change1, change2]);
final all = changes.getChanges();
final missing = changes.getChanges(newerThan: theirVersion);
print('Total changes: ${changes.count}');
final snapshots = await CRDTHive.openSnapshotStorageForDocument('doc-123');
await snapshots.saveSnapshot(snapshot);
final latest = snapshots.getSnapshots();
final peers = await CRDTHive.openPeerIdStorageForDocument('doc-123');
final document = CRDTDocument(
documentId: 'doc-123',
peerId: await peers.loadOrCreate(),
);
CRDTHive.openStorageForDocument(id) opens the change and snapshot storages at
once. CRDTHive.deleteBox(name) deletes one box.
Every method returns a Future.
final changes = backend.changeStorageForDocument('doc-123');
await changes.saveChanges([change1, change2]);
final all = await changes.getChanges();
final missing = await changes.getChanges(newerThan: theirVersion);
print('Total changes: ${await changes.count}');
final snapshots = backend.snapshotStorageForDocument('doc-123');
await snapshots.saveSnapshot(snapshot);
final latest = await snapshots.getSnapshots();
final peers = backend.peerIdStorageForDocument('doc-123');
final document = CRDTDocument(
documentId: 'doc-123',
peerId: await peers.loadOrCreate(),
);
getChanges takes the two version-vector bounds newerThan and upTo: see
Reading part of the log.
Read the identity before building the document, because the id has to exist
first (see Peer identity).
Examples
- The recipe a text note saved on disk,
with one tab per adapter: one runnable
main.darteach, which is also theexample/of its package. - In Flutter: a note and a list of notes, both on Hive.
- A sync server with a persistent registry, backed by Hive, with clients that sync through it (see Server registry).
- Greyhound Markdown
keeps every room in the browser with
crdt_lf_hive:storage: CRDTHive.openin its replica. Edit a room offline, reload the page, and the text is still there (source).