Persistence
crdt_lf keeps the document in memory. To find it again after a restart, add
a storage adapter. It reads the document back on launch, then follows it
and writes down every change: you write none of that code.
The contract and the adapters
Two layers, in two kinds of package:
crdt_lf_persistenceholds the contract (what a storage must do) and the code that uses it: open a document, follow it, compact it, copy it. It holds no store of its own.- An adapter implements the contract on a database, and re-exports the
whole API. Your
pubspec.yamlnames only the adapter:
| Adapter | Database | Kind |
|---|---|---|
crdt_lf_hive | Hive | key-value boxes (NoSQL) |
crdt_lf_sqlite | sqlite3 | SQL, synchronous |
crdt_lf_drift | Drift | SQL, asynchronous |
All three implement the same CRDTStorageBackend. Code written against it runs
on any of them, so you can change adapter by changing one line: the one that
opens the database. You can even
copy a document from one to another.
Which adapter
Every adapter stores the same thing: each change and each snapshot as an opaque binary blob, plus the identity the device writes under. They differ in the database under them.
| Hive | SQLite | Drift | |
|---|---|---|---|
| Model | NoSQL: two boxes per document | SQL: one file, three tables | SQL: one database, three tables |
| Reads | synchronous (the box is in memory) | synchronous (FFI) | asynchronous |
| Writes | asynchronous (box journal) | synchronous | asynchronous |
| Transactions | none | yes (savepoints) | yes (database.transaction) |
openSync (restore before the first frame) | ✅ | ✅ | — |
| Platforms | Dart VM, Flutter, web | Dart VM, native Flutter | Dart VM, Flutter, web (with a web executor) |
| Pick it when | you need the web, or want no native library | you want SQL with no await in the way | your app already uses Drift |
How to read the trade-offs:
- SQL or NoSQL. The CRDT data is opaque blobs, so you never query it yourself: the choice is about the rest of your app. If it already has a SQL database, or you want one file you can inspect with any SQLite tool, pick a SQL adapter. If you want the lightest setup, with no native library, pick Hive.
- Synchronous or asynchronous. A synchronous backend lets
CRDTDocumentPersistence.openSyncrestore the document before it returns, so a Flutter app draws its first frame from the stored state instead of an empty one. With Drift youawaitthe restore. - Transactions. With transactions, a compaction (a snapshot plus the prune that follows it) lands all at once or not at all. Without them the steps are separate. That is still safe, because every step can be repeated: Hive is conformant too.
Two adapters run in the browser:
- Hive, with no setup: Greyhound Markdown uses it there.
- Drift, through
CRDTDrift.fromDatabaseand a web executor of Drift (see the Drift tab).CRDTDrift.openandCRDTDrift.memoryneed a native database, and throw anUnsupportedErroron the web.
The SQLite adapter runs on the Dart VM and native Flutter only.
None of them fits? Build your own adapter. Before you do, open an issue: an adapter in this repository serves everyone.
Local only
No server to talk to: an adapter, and one call.
- SQLite
- Hive
- Drift
import 'package:crdt_lf/crdt_lf.dart';
import 'package:crdt_lf_sqlite/crdt_lf_sqlite.dart';
final backend = CRDTSqlite.open('note.db');
// Reads the stored note (or starts an empty one), then writes every edit.
final note = await backend.openDocument('note');
final text = CRDTFugueTextHandler(note.document, 'body');
text.insert(text.value.runes.length, 'Hello 🌍\n');
await note.persistence.dispose(); // writes what is still waiting
note.document.dispose();
await note.persistence.storage.close();
backend.close();
import 'package:crdt_lf/crdt_lf.dart';
import 'package:crdt_lf_hive/crdt_lf_hive.dart';
import 'package:hive/hive.dart';
// Once per process, before the first open.
Hive.init('note_db');
CRDTHive.initialize();
final backend = await CRDTHive.open();
// Reads the stored note (or starts an empty one), then writes every edit.
final note = await backend.openDocument('note');
final text = CRDTFugueTextHandler(note.document, 'body');
text.insert(text.value.runes.length, 'Hello 🌍\n');
await note.persistence.dispose(); // writes what is still waiting
note.document.dispose();
await note.persistence.storage.close();
await backend.close();
import 'dart:io';
import 'package:crdt_lf/crdt_lf.dart';
import 'package:crdt_lf_drift/crdt_lf_drift.dart';
final backend = CRDTDrift.open(File('note.db'));
// Reads the stored note (or starts an empty one), then writes every edit.
final note = await backend.openDocument('note');
final text = CRDTFugueTextHandler(note.document, 'body');
text.insert(text.value.runes.length, 'Hello 🌍\n');
await note.persistence.dispose(); // writes what is still waiting
note.document.dispose();
await note.persistence.storage.close();
await backend.close();
Every run reads the note back and adds a line.
The whole opening, in one call
openDocument does three things, in the order they have to happen:
- Identity. It reads the
PeerIdthis device writes the document under (see Peer identity). - Document. It builds the document with that identity.
- Restore. It reads the stored changes and snapshots into the document, then follows it.
final room = await backend.openDocument('note-1');
room.document; // already full, already the author it was last time
room.persistence; // dispose this when you are done
final text = CRDTFugueTextHandler(room.document, 'body');
Build the handlers on the document it hands back. A handler reads its state from the document whenever it is created, so one made after the restore reads exactly what one made before it would have.
| Argument | What it is for |
|---|---|
author | the identity to use for a document that has none stored yet; it is stored |
onDocument | runs before the restore, for what has to exist first: a listener on events, the kinds of nested handlers |
writeDelay, compactAfter, keepSnapshots, onError | as on CRDTDocumentPersistence.open, see How it works |
The identity is always kept. A stored id beats author: it is what the
document already wrote under, and writing under a second one would make one
device look like two peers.
If the restore fails, the document is disposed and the failure is thrown, so you never get half a document back.
Use CRDTDocumentPersistence.open(document, storage) directly where there is no
backend: a single CRDTDocumentStorage you wrote yourself.
Offline-first, with sync
Same call, plus one rule: open the document before you connect.
final room = await backend.openDocument(roomId);
// Only now.
await client.connect();
The restored document is what the sync client reconciles against, so what you
wrote on a plane reaches the server after the restart. Connect first, and the
client catches up from the server before it knows what it already had.
CRDTReplica does both steps, in this order.
Many documents
An app has notes, not a note. The backend is the database itself:
/// A note in the list: its document id and its title.
typedef NoteSummary = ({String id, String title});
/// The notes stored in [backend], without the ones in [except].
Future<List<NoteSummary>> readNotes(
CRDTStorageBackend backend, {
Set<String> except = const {},
}) async {
final notes = <NoteSummary>[];
for (final id in await backend.documentIds) {
if (except.contains(id)) {
continue;
}
// A read-only copy: nothing writes it, so dispose it after reading.
final document = await backend.readDocument(id);
final body = CRDTFugueTextHandler(document, 'body').value;
document.dispose();
final title = body.split('\n').first.trim();
notes.add((id: id, title: title.isEmpty ? 'Empty note' : title));
}
return notes..sort((a, b) => a.id.compareTo(b.id));
}
readDocument reads and never writes: use it for a list or a preview, and
openDocument for the note being edited. backend.deleteDocument(id) removes
the changes, the snapshots and the identity of a note. The
notes recipe is the whole app.
documentIds lists a document once anything about it is stored, its identity
included. A note that was created and never typed into is still there, and
still empty.
What you can call
Most of the API is methods on the backend, so you find them by typing a dot:
| You want to | Call |
|---|---|
| open a document that is going to be edited | backend.openDocument(id) |
| read one without following it (a preview, a list) | backend.readDocument(id) |
| see it as it was at some version | backend.documentAt(id, version) |
| back it up, or move it to another adapter | backend.copyDocumentTo(other, id) |
| stop the log from growing, now | persistence.compact() |
The three read-and-copy calls are also on a single CRDTDocumentStorage
(storage.readDocument(), storage.documentAt(version),
storage.copyTo(other)), for when that is all you hold. compact() lives on
the CRDTDocumentPersistence that follows a document.