SQLite Storage - OpenMLS

The SqliteStorageProvider implements the StorageProvider trait using SQLite through the rusqlite crate. This provides persistent, production-ready storage with ACID guarantees.

Overview

SQLite storage persists all OpenMLS data to a SQLite database file. It supports custom serialization codecs and automatic schema migrations.

Installation

Add the SQLite storage crate to your Cargo.toml:

[dependencies]
openmls_sqlite_storage = "0.2"
openmls_traits = "0.2"
rusqlite = "0.31"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"

Basic Usage

use openmls_sqlite_storage::{SqliteStorageProvider, Codec};
use rusqlite::Connection;
use serde::{Serialize, Deserialize};

// Define a codec for serialization
#[derive(Default)]
struct JsonCodec;

impl Codec for JsonCodec {
    type Error = serde_json::Error;

    fn to_vec<T: Serialize>(value: &T) -> Result<Vec<u8>, Self::Error> {
        serde_json::to_vec(value)
    }

    fn from_slice<T: serde::de::DeserializeOwned>(slice: &[u8]) -> Result<T, Self::Error> {
        serde_json::from_slice(slice)
    }
}

// Create storage with file-based database
let connection = Connection::open("openmls.db")?;
let mut storage = SqliteStorageProvider::<JsonCodec, _>::new(connection);

// Run migrations to initialize schema
storage.run_migrations()?;

// Use with OpenMLS operations
storage.write_key_package(&hash_ref, &key_package)?;
let key_package = storage.key_package(&hash_ref)?;

Codec Configuration

The storage provider is generic over the codec used for serialization:

JSON Codec (Recommended)

use serde::{Serialize, de::DeserializeOwned};

#[derive(Default)]
pub struct JsonCodec;

impl Codec for JsonCodec {
    type Error = serde_json::Error;

    fn to_vec<T: Serialize>(value: &T) -> Result<Vec<u8>, Self::Error> {
        serde_json::to_vec(value)
    }

    fn from_slice<T: DeserializeOwned>(slice: &[u8]) -> Result<T, Self::Error> {
        serde_json::from_slice(slice)
    }
}

Binary Codec (More Efficient)

use bincode;

#[derive(Default)]
pub struct BincodeCodec;

impl Codec for BincodeCodec {
    type Error = bincode::Error;

    fn to_vec<T: Serialize>(value: &T) -> Result<Vec<u8>, Self::Error> {
        bincode::serialize(value)
    }

    fn from_slice<T: DeserializeOwned>(slice: &[u8]) -> Result<T, Self::Error> {
        bincode::deserialize(slice)
    }
}

Custom Codec

struct MyCodec;

impl Codec for MyCodec {
    type Error = MyCodecError;

    fn to_vec<T: Serialize>(value: &T) -> Result<Vec<u8>, Self::Error> {
        // Your serialization logic
    }

    fn from_slice<T: DeserializeOwned>(slice: &[u8]) -> Result<T, Self::Error> {
        // Your deserialization logic
    }
}

Database Initialization

File-Based Database

use rusqlite::Connection;

let connection = Connection::open("openmls.db")?;
let mut storage = SqliteStorageProvider::<JsonCodec, _>::new(connection);
storage.run_migrations()?;

In-Memory Database

Useful for testing:

let connection = Connection::open_in_memory()?;
let mut storage = SqliteStorageProvider::<JsonCodec, _>::new(connection);
storage.run_migrations()?;

Custom Connection Parameters

use rusqlite::{Connection, OpenFlags};

let connection = Connection::open_with_flags(
    "openmls.db",
    OpenFlags::SQLITE_OPEN_READ_WRITE | OpenFlags::SQLITE_OPEN_CREATE,
)?;

let mut storage = SqliteStorageProvider::<JsonCodec, _>::new(connection);
storage.run_migrations()?;

Schema Migrations

The storage provider uses the refinery crate for schema migrations:

// Initialize database with latest schema
storage.run_migrations()?;

Migrations are automatically applied when calling run_migrations(). The migration table is named openmls_sqlite_storage_migrations.

Storage Operations

Key Packages

// Write key package
storage.write_key_package(&hash_ref, &key_package)?;

// Read key package
let key_package: Option<KeyPackageBundle> = storage.key_package(&hash_ref)?;

// Delete key package
storage.delete_key_package(&hash_ref)?;

Group State

// Write group state
storage.write_group_state(&group_id, &group_state)?;

// Read group state
let state: Option<MlsGroupState> = storage.group_state(&group_id)?;

// Delete group state
storage.delete_group_state(&group_id)?;

Tree Sync

// Write tree
storage.write_tree(&group_id, &tree)?;

// Read tree
let tree: Option<TreeSync> = storage.tree(&group_id)?;

// Delete tree
storage.delete_tree(&group_id)?;

Proposals

// Queue a proposal
storage.queue_proposal(&group_id, &proposal_ref, &proposal)?;

// Get proposal refs for a group
let refs: Vec<ProposalRef> = storage.queued_proposal_refs(&group_id)?;

// Get all proposals for a group
let proposals: Vec<(ProposalRef, QueuedProposal)> = 
    storage.queued_proposals(&group_id)?;

// Remove specific proposal
storage.remove_proposal(&group_id, &proposal_ref)?;

// Clear all proposals for a group
storage.clear_proposal_queue(&group_id)?;

Encryption Keys

// Write encryption key pair
storage.write_encryption_key_pair(&public_key, &key_pair)?;

// Read encryption key pair
let key_pair: Option<EncryptionKeyPair> = 
    storage.encryption_key_pair(&public_key)?;

// Write epoch key pairs
storage.write_encryption_epoch_key_pairs(
    &group_id,
    &epoch,
    leaf_index,
    &key_pairs,
)?;

// Read epoch key pairs
let key_pairs: Vec<EncryptionKeyPair> = 
    storage.encryption_epoch_key_pairs(&group_id, &epoch, leaf_index)?;

// Delete encryption key pair
storage.delete_encryption_key_pair(&public_key)?;

// Delete epoch key pairs
storage.delete_encryption_epoch_key_pairs(&group_id, &epoch, leaf_index)?;

Signature Keys

// Write signature key pair
storage.write_signature_key_pair(&public_key, &key_pair)?;

// Read signature key pair
let key_pair: Option<SignatureKeyPair> = 
    storage.signature_key_pair(&public_key)?;

// Delete signature key pair
storage.delete_signature_key_pair(&public_key)?;

PSKs (Pre-Shared Keys)

// Write PSK
storage.write_psk(&psk_id, &psk_bundle)?;

// Read PSK
let psk: Option<PskBundle> = storage.psk(&psk_id)?;

// Delete PSK
storage.delete_psk(&psk_id)?;

Group Context

// Write context
storage.write_context(&group_id, &group_context)?;

// Read context
let context: Option<GroupContext> = storage.group_context(&group_id)?;

// Delete context
storage.delete_context(&group_id)?;

Message Secrets

// Write message secrets
storage.write_message_secrets(&group_id, &message_secrets)?;

// Read message secrets
let secrets: Option<MessageSecretsStore> = 
    storage.message_secrets(&group_id)?;

// Delete message secrets
storage.delete_message_secrets(&group_id)?;

Group Epoch Secrets

// Write epoch secrets
storage.write_group_epoch_secrets(&group_id, &epoch_secrets)?;

// Read epoch secrets
let secrets: Option<GroupEpochSecrets> = 
    storage.group_epoch_secrets(&group_id)?;

// Delete epoch secrets
storage.delete_group_epoch_secrets(&group_id)?;

Error Handling

SQLite storage returns rusqlite::Error:

use rusqlite::Error;

match storage.key_package(&hash_ref) {
    Ok(Some(kp)) => println!("Found key package"),
    Ok(None) => println!("Key package not found"),
    Err(Error::QueryReturnedNoRows) => println!("No data"),
    Err(e) => println!("Database error: {}", e),
}

Connection Management

The storage provider can work with different connection types:

Owned Connection

let connection = Connection::open("openmls.db")?;
let storage = SqliteStorageProvider::<JsonCodec, Connection>::new(connection);

Borrowed Connection

let connection = Connection::open("openmls.db")?;
let storage = SqliteStorageProvider::<JsonCodec, &Connection>::new(&connection);

Mutable Reference (for migrations)

let mut connection = Connection::open("openmls.db")?;
let mut storage = SqliteStorageProvider::<JsonCodec, &mut Connection>::new(&mut connection);
storage.run_migrations()?;

Performance Considerations

Batch Operations

For multiple writes, use transactions:

let tx = connection.transaction()?;
let storage = SqliteStorageProvider::<JsonCodec, _>::new(&tx);

for key_package in key_packages {
    storage.write_key_package(&hash_ref, &key_package)?;
}

tx.commit()?;

Indexing

The schema includes indexes on frequently queried columns. Migrations ensure optimal index structure.

WAL Mode

Enable Write-Ahead Logging for better concurrency:

use rusqlite::Connection;

let connection = Connection::open("openmls.db")?;
connection.execute_batch("PRAGMA journal_mode=WAL")?;

Storage Version

The SQLite storage provider tracks its own version:

const STORAGE_PROVIDER_VERSION: u16 = 1;

The storage version must match OpenMLS CURRENT_VERSION. Migrations are required when the version changes.

Database Schema

The storage provider creates these tables:

key_packages - Stores key package bundles
psks - Pre-shared keys
encryption_key_pairs - HPKE encryption key pairs
signature_key_pairs - Signature key pairs
epoch_key_pairs - Epoch-specific encryption keys
group_data - Group context, state, configuration
proposals - Queued proposals by group
own_leaf_nodes - Leaf nodes owned by this client

Limitations

Not supported on wasm32 targets
Requires rusqlite which uses native SQLite library
Single-process access (use WAL mode for multiple connections)

Testing

Use in-memory databases for tests:

#[test]
fn test_storage_operations() {
    let connection = Connection::open_in_memory().unwrap();
    let mut storage = SqliteStorageProvider::<JsonCodec, _>::new(connection);
    storage.run_migrations().unwrap();
    
    // Test operations...
}

Memory Storage - In-memory storage for testing
Custom Storage - Implement your own storage provider

Documentation Index

​Overview

​Installation

​Basic Usage

​Codec Configuration

​JSON Codec (Recommended)

​Binary Codec (More Efficient)

​Custom Codec

​Database Initialization

​File-Based Database

​In-Memory Database

​Custom Connection Parameters

​Schema Migrations

​Storage Operations

​Key Packages

​Group State

​Tree Sync

​Proposals

​Encryption Keys

​Signature Keys

​PSKs (Pre-Shared Keys)

​Group Context

​Message Secrets

​Group Epoch Secrets

​Error Handling

​Connection Management

​Owned Connection

​Borrowed Connection

​Mutable Reference (for migrations)

​Performance Considerations

​Batch Operations

​Indexing

​WAL Mode

​Storage Version

​Database Schema

​Limitations

​Testing

​Related

Overview

Installation

Basic Usage

Codec Configuration

JSON Codec (Recommended)

Binary Codec (More Efficient)

Custom Codec

Database Initialization

File-Based Database

In-Memory Database

Custom Connection Parameters

Schema Migrations

Storage Operations

Key Packages

Group State

Tree Sync

Proposals

Encryption Keys

Signature Keys

PSKs (Pre-Shared Keys)

Group Context

Message Secrets

Group Epoch Secrets

Error Handling

Connection Management

Owned Connection

Borrowed Connection

Mutable Reference (for migrations)

Performance Considerations

Batch Operations

Indexing

WAL Mode

Storage Version

Database Schema

Limitations

Testing

Related