Getting started with desert-rust

desert-rust is a binary serialization library for Rust. It focuses on compact binary data while still allowing compatible changes to structs and enums over time.

The Rust crate is the counterpart of the original Scala desert library. The wire format is intentionally similar, but the API is shaped around Rust traits, derive macros, feature flags, and explicit error handling.

Installation

Add the public crate to Cargo.toml:

[dependencies]
desert_rust = "0.1.8"

Additional codecs are controlled with crate features:

[dependencies]
desert_rust = { version = "0.1.8", features = ["uuid", "chrono", "url"] }

Feature flags exposed by the public crate are:

bigdecimal
bit-vec
chrono
mac_address
nonempty-collections
serde-json
url
uuid

The current desert_core default features already enable bigdecimal, chrono, uuid, nonempty-collections, and serde-json. Features such as url, mac_address, and bit-vec must be enabled explicitly.

Serialize and deserialize a known type

The most direct API works with a Vec<u8> or bytes::Bytes:

use desert_rust::{deserialize, serialize_to_byte_vec, Result};

fn main() -> Result<()> {
    let bytes = serialize_to_byte_vec(&"Hello world".to_string())?;
    let value: String = deserialize(&bytes)?;

    assert_eq!(value, "Hello world");
    Ok(())
}

This works because String implements both BinarySerializer and BinaryDeserializer. Their combination is named BinaryCodec.

Derive codecs for your data

For structs and enums, derive BinaryCodec:

use desert_rust::{deserialize, serialize_to_byte_vec, BinaryCodec, Result};

#[derive(Debug, PartialEq, BinaryCodec)]
struct Point {
    x: i32,
    y: i32,
}

#[derive(Debug, PartialEq, BinaryCodec)]
enum Command {
    Move { to: Point },
    Label(String),
    Stop,
}

fn main() -> Result<()> {
    let command = Command::Move {
        to: Point { x: 10, y: -5 },
    };

    let bytes = serialize_to_byte_vec(&command)?;
    let decoded: Command = deserialize(&bytes)?;

    assert_eq!(decoded, command);
    Ok(())
}

The derive macro generates trait implementations and metadata needed for schema evolution. There is no runtime reflection or registration step for statically known types.

Top-level helper functions

The main helper functions are:

serialize(value, output) writes to any BinaryOutput.
serialize_with_options(value, output, options) does the same with explicit options.
serialize_to_byte_vec(value) returns Vec<u8>.
serialize_to_bytes(value) returns bytes::Bytes.
deserialize::<T>(input) reads T from a byte slice.
deserialize_with_options::<T>(input, options) reads with explicit options.

Scala compatibility option

Rust char normally serializes as a variable-length Unicode scalar value. The Scala library encoded characters as 16-bit Unicode units. Use Options::scala_compatible() when reading or writing data that must match the Scala character encoding:

use desert_rust::{
    deserialize_with_options, serialize_to_byte_vec_with_options, Options, Result,
};

fn main() -> Result<()> {
    let options = Options::scala_compatible();
    let bytes = serialize_to_byte_vec_with_options(&'x', options.clone())?;
    let decoded: char = deserialize_with_options(&bytes, options)?;

    assert_eq!(decoded, 'x');
    Ok(())
}

If a character cannot be represented as a single 16-bit unit in this mode, serialization fails with Error::UnsupportedCharacter.

Building this book locally

The byte-layout examples in this book are generated by the mdbook-desert preprocessor. Build it first, then put Cargo’s debug binary directory on PATH while running mdBook:

cargo build -p desert_book
PATH="$PWD/target/debug:$PATH" mdbook build book

Where to go next

Read Codecs and derivation for built-in types and custom codec implementations. Read Data model evolution before persisting data long term or sending it between independently deployed versions.

Keyboard shortcuts

desert