Skip to main content

Persistence

crdt_lf keeps the document in memory. To find it again after a restart, add a storage adapter. It reads the document back on launch, then follows it and writes down every change: you write none of that code.

The contract and the adapters​

Two layers, in two kinds of package:

  • crdt_lf_persistence holds the contract (what a storage must do) and the code that uses it: open a document, follow it, compact it, copy it. It holds no store of its own.
  • An adapter implements the contract on a database, and re-exports the whole API. Your pubspec.yaml names only the adapter:
AdapterDatabaseKind
crdt_lf_hiveHivekey-value boxes (NoSQL)
crdt_lf_sqlitesqlite3SQL, synchronous
crdt_lf_driftDriftSQL, asynchronous

All three implement the same CRDTStorageBackend. Code written against it runs on any of them, so you can change adapter by changing one line: the one that opens the database. You can even copy a document from one to another.

Which adapter​

Every adapter stores the same thing: each change and each snapshot as an opaque binary blob, plus the identity the device writes under. They differ in the database under them.

HiveSQLiteDrift
ModelNoSQL: two boxes per documentSQL: one file, three tablesSQL: one database, three tables
Readssynchronous (the box is in memory)synchronous (FFI)asynchronous
Writesasynchronous (box journal)synchronousasynchronous
Transactionsnoneyes (savepoints)yes (database.transaction)
openSync (restore before the first frame)✅✅—
PlatformsDart VM, Flutter, webDart VM, native FlutterDart VM, Flutter, web (with a web executor)
Pick it whenyou need the web, or want no native libraryyou want SQL with no await in the wayyour app already uses Drift

How to read the trade-offs:

  • SQL or NoSQL. The CRDT data is opaque blobs, so you never query it yourself: the choice is about the rest of your app. If it already has a SQL database, or you want one file you can inspect with any SQLite tool, pick a SQL adapter. If you want the lightest setup, with no native library, pick Hive.
  • Synchronous or asynchronous. A synchronous backend lets CRDTDocumentPersistence.openSync restore the document before it returns, so a Flutter app draws its first frame from the stored state instead of an empty one. With Drift you await the restore.
  • Transactions. With transactions, a compaction (a snapshot plus the prune that follows it) lands all at once or not at all. Without them the steps are separate. That is still safe, because every step can be repeated: Hive is conformant too.
Web

Two adapters run in the browser:

  • Hive, with no setup: Greyhound Markdown uses it there.
  • Drift, through CRDTDrift.fromDatabase and a web executor of Drift (see the Drift tab). CRDTDrift.open and CRDTDrift.memory need a native database, and throw an UnsupportedError on the web.

The SQLite adapter runs on the Dart VM and native Flutter only.

None of them fits? Build your own adapter. Before you do, open an issue: an adapter in this repository serves everyone.

Local only​

No server to talk to: an adapter, and one call.

import 'package:crdt_lf/crdt_lf.dart';
import 'package:crdt_lf_sqlite/crdt_lf_sqlite.dart';

final backend = CRDTSqlite.open('note.db');

// Reads the stored note (or starts an empty one), then writes every edit.
final note = await backend.openDocument('note');
final text = CRDTFugueTextHandler(note.document, 'body');

text.insert(text.value.runes.length, 'Hello 🌍\n');

await note.persistence.dispose(); // writes what is still waiting
note.document.dispose();
await note.persistence.storage.close();
backend.close();

Every run reads the note back and adds a line.

The whole opening, in one call​

openDocument does three things, in the order they have to happen:

  1. Identity. It reads the PeerId this device writes the document under (see Peer identity).
  2. Document. It builds the document with that identity.
  3. Restore. It reads the stored changes and snapshots into the document, then follows it.
final room = await backend.openDocument('note-1');

room.document; // already full, already the author it was last time
room.persistence; // dispose this when you are done

final text = CRDTFugueTextHandler(room.document, 'body');

Build the handlers on the document it hands back. A handler reads its state from the document whenever it is created, so one made after the restore reads exactly what one made before it would have.

ArgumentWhat it is for
authorthe identity to use for a document that has none stored yet; it is stored
onDocumentruns before the restore, for what has to exist first: a listener on events, the kinds of nested handlers
writeDelay, compactAfter, keepSnapshots, onErroras on CRDTDocumentPersistence.open, see How it works

The identity is always kept. A stored id beats author: it is what the document already wrote under, and writing under a second one would make one device look like two peers.

If the restore fails, the document is disposed and the failure is thrown, so you never get half a document back.

Use CRDTDocumentPersistence.open(document, storage) directly where there is no backend: a single CRDTDocumentStorage you wrote yourself.

Offline-first, with sync​

Same call, plus one rule: open the document before you connect.

final room = await backend.openDocument(roomId);

// Only now.
await client.connect();

The restored document is what the sync client reconciles against, so what you wrote on a plane reaches the server after the restart. Connect first, and the client catches up from the server before it knows what it already had. CRDTReplica does both steps, in this order.

Many documents​

An app has notes, not a note. The backend is the database itself:

/// A note in the list: its document id and its title.
typedef NoteSummary = ({String id, String title});

/// The notes stored in [backend], without the ones in [except].
Future<List<NoteSummary>> readNotes(
CRDTStorageBackend backend, {
Set<String> except = const {},
}) async {
final notes = <NoteSummary>[];
for (final id in await backend.documentIds) {
if (except.contains(id)) {
continue;
}
// A read-only copy: nothing writes it, so dispose it after reading.
final document = await backend.readDocument(id);
final body = CRDTFugueTextHandler(document, 'body').value;
document.dispose();

final title = body.split('\n').first.trim();
notes.add((id: id, title: title.isEmpty ? 'Empty note' : title));
}
return notes..sort((a, b) => a.id.compareTo(b.id));
}

readDocument reads and never writes: use it for a list or a preview, and openDocument for the note being edited. backend.deleteDocument(id) removes the changes, the snapshots and the identity of a note. The notes recipe is the whole app.

documentIds lists a document once anything about it is stored, its identity included. A note that was created and never typed into is still there, and still empty.

What you can call​

Most of the API is methods on the backend, so you find them by typing a dot:

You want toCall
open a document that is going to be editedbackend.openDocument(id)
read one without following it (a preview, a list)backend.readDocument(id)
see it as it was at some versionbackend.documentAt(id, version)
back it up, or move it to another adapterbackend.copyDocumentTo(other, id)
stop the log from growing, nowpersistence.compact()

The three read-and-copy calls are also on a single CRDTDocumentStorage (storage.readDocument(), storage.documentAt(version), storage.copyTo(other)), for when that is all you hold. compact() lives on the CRDTDocumentPersistence that follows a document.