serde_json to simd-json in streamerOS IPC: hot-path deserialization

•6 min read•Yaseen Khatib · MERN + AI Architect

Why we swapped parsers

Our streamerOS IPC channel pushes a lot of JSON across Unix domain sockets. Even after amortizing framing, we were spending measurable CPU in the deserialize hot path. serde_json is ubiquitous and solid, but its scalar parser and string allocation model were not ideal for our workload, which is:

  • Lots of small control envelopes (~300–800 bytes) with string-heavy fields.
  • Periodic telemetry payloads (5–50 KB) containing arrays of numbers and string tags.

simd-json brings SIMD-accelerated scanning (AVX2 on x86_64, NEON on aarch64) and zero-copy borrowing for strings via in-place unescaping. The gotcha: it requires a mutable byte slice (&mut [u8]) because it rewrites structural characters during parse.

This post shows exactly how we benchmarked the deserialize hot path and integrated simd-json in the IPC layer, with the sharp edges called out.

Setup and feature flags

Cargo.toml snippets:

[dependencies]
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
simd-json = { version = "0.13", features = ["serde_impl"] }

[dev-dependencies]
criterion = "0.5"

[profile.release]
opt-level = 3
lto = "fat"
codegen-units = 1

For best results, compile with your CPU’s SIMD features:

# .cargo/config.toml
[target.x86_64-unknown-linux-gnu]
rustflags = ["-C", "target-cpu=native"]

[target.aarch64-apple-darwin]
rustflags = ["-C", "target-cpu=native"]

simd-json auto-selects the best implementation available at compile/runtime, but if you build with a generic target CPU you may leave performance on the table.

The message shape (borrowable strings)

We made stringy fields borrowable to leverage simd-json’s zero-copy path. serde_json will still allocate Strings for these fields; simd-json can hand out &str pointing into the mutable input buffer.

use serde::Deserialize;
use std::borrow::Cow;

#[derive(Deserialize, Debug)]
#[serde(tag = "kind")]
enum Envelope<'a> {
    Control {
        ts: u64,
        stream_id: u64,
        #[serde(borrow)] op: Cow<'a, str>,
        #[serde(borrow)] node: Cow<'a, str>,
        #[serde(borrow)] tags: Vec<Cow<'a, str>>,
    },
    Telemetry {
        ts: u64,
        stream_id: u64,
        // numerics still allocate; only strings can be borrowed
        samples: Vec<f32>,
        #[serde(borrow)] metric: Cow<'a, str>,
    }
}

Example JSONs representative of our IPC frames:

{"kind":"Control","ts":1710000000000,"stream_id":42,"op":"resume","node":"ingest-a","tags":["vip","cdn-eu"]}

{"kind":"Telemetry","ts":1710000000000,"stream_id":42,"samples":[0.12,0.08,0.10,0.09,0.13,0.11],"metric":"p95_latency"}

Criterion benchmark for the deserialize hot path

Two important constraints:

  1. simd-json requires &mut [u8] and will modify it.
  2. To keep things fair, both benches pay the same buffer materialization cost per iteration (as would happen when the network stack fills a new buffer).
// benches/ipc_deser.rs
use criterion::{black_box, criterion_group, criterion_main, BatchSize, Criterion};
use serde::Deserialize;
use std::borrow::Cow;

#[derive(Deserialize, Debug)]
#[serde(tag = "kind")]
enum Envelope<'a> {
    Control {
        ts: u64,
        stream_id: u64,
        #[serde(borrow)] op: Cow<'a, str>,
        #[serde(borrow)] node: Cow<'a, str>,
        #[serde(borrow)] tags: Vec<Cow<'a, str>>,
    },
    Telemetry {
        ts: u64,
        stream_id: u64,
        samples: Vec<f32>,
        #[serde(borrow)] metric: Cow<'a, str>,
    }
}

static CONTROL_JSON: &str = r#"{
  "kind":"Control",
  "ts":1710000000000,
  "stream_id":42,
  "op":"resume",
  "node":"ingest-a",
  "tags":["vip","cdn-eu","edge-12"]
}"#;

static TELEMETRY_JSON: &str = r#"{
  "kind":"Telemetry",
  "ts":1710000000000,
  "stream_id":42,
  "samples":[0.12,0.08,0.10,0.09,0.13,0.11,0.12,0.10,0.12,0.09,0.11,0.12,0.10,0.09,0.13,0.11],
  "metric":"p95_latency"
}"#;

fn bench_control(c: &mut Criterion) {
    let mut g = c.benchmark_group("control-envelope");

    g.bench_function("serde_json", |b| {
        b.iter_batched(
            || CONTROL_JSON.as_bytes().to_vec(),
            |mut buf| {
                let env: Envelope = serde_json::from_slice(&buf).unwrap();
                black_box(env);
                black_box(buf);
            },
            BatchSize::SmallInput,
        );
    });

    g.bench_function("simd_json", |b| {
        b.iter_batched(
            || CONTROL_JSON.as_bytes().to_vec(),
            |mut buf| {
                let env: Envelope = simd_json::serde::from_slice(&mut buf).unwrap();
                black_box(env);
                black_box(buf);
            },
            BatchSize::SmallInput,
        );
    });

    g.finish();
}

fn bench_telemetry(c: &mut Criterion) {
    let mut g = c.benchmark_group("telemetry-envelope");

    g.bench_function("serde_json", |b| {
        b.iter_batched(
            || TELEMETRY_JSON.as_bytes().to_vec(),
            |mut buf| {
                let env: Envelope = serde_json::from_slice(&buf).unwrap();
                black_box(env);
                black_box(buf);
            },
            BatchSize::SmallInput,
        );
    });

    g.bench_function("simd_json", |b| {
        b.iter_batched(
            || TELEMETRY_JSON.as_bytes().to_vec(),
            |mut buf| {
                let env: Envelope = simd_json::serde::from_slice(&mut buf).unwrap();
                black_box(env);
                black_box(buf);
            },
            BatchSize::SmallInput,
        );
    });

    g.finish();
}

criterion_group!(benches, bench_control, bench_telemetry);
criterion_main!(benches);

Run the bench:

RUSTFLAGS="-C target-cpu=native" cargo bench --bench ipc_deser --release

Sample results (Ryzen 7950X, Rust 1.78, AVX2 enabled)

Workload serde_json (µs) simd-json (µs) Speedup
Control (~420 B) 1.45 0.88 1.65x
Telemetry (~1.1 KB) 5.64 2.37 2.38x

Your numbers will vary with CPU, JSON shape, and memory pressure. As payloads get bigger or string density increases, simd-json tends to pull further ahead.

Integrating simd-json in the IPC hot path

Our frames are length-prefixed and carried over tokio’s UnixStream. The key is to parse in-place from a mutable buffer. bytes::BytesMut is convenient here.

use bytes::{BufMut, BytesMut};
use simd_json::serde as simd_serde;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::UnixStream;

async fn read_frame(stream: &mut UnixStream) -> tokio::io::Result<Envelope<'_>> {
    let mut len_buf = [0u8; 4];
    stream.read_exact(&mut len_buf).await?;
    let len = u32::from_le_bytes(len_buf) as usize;

    let mut buf = BytesMut::with_capacity(len);
    // Safety: reserve ensures capacity; put_bytes initializes
    buf.put_bytes(0, len);
    stream.read_exact(&mut buf[..]).await?;

    // Parse in-place. simd-json will modify the buffer to zero-copy strings.
    let env: Envelope = simd_serde::from_slice(&mut buf[..]).map_err(|e| {
        use std::io::{Error, ErrorKind};
        Error::new(ErrorKind::InvalidData, format!("JSON: {e}"))
    })?;
    Ok(env)
}

Notes:

  • You must not reuse the same buffer contents after simd-json mutates it, except to drop it or refill it. We allocate per frame anyway, so this fits.
  • If your upstream API only hands you &str, you’ll need to copy into a Vec (costly). Prefer receiving into a mutable byte buffer (Vec, BytesMut, or a pooling allocator) to unlock simd-json’s gains.

Where the wins come from

  • SIMD scanning: simd-json uses wide registers to detect structural characters and validate UTF‑8 fast. This accelerates both small and large documents.
  • In-place unescaping: Strings without escapes can be returned as &str pointing into the input, eliminating per-field allocations. In our control messages, this removed multiple allocs (op, node, tags).
  • Fewer bound checks and branches on the hot path, especially noticeable in larger payloads.

Sharp edges and caveats

  • &mut [u8] required: If your pipeline is &str-only, you’ll pay an extra copy. That alone can erase any advantage on tiny messages.
  • Zero-copy only for strings: Numeric arrays must still be parsed and allocated (both parsers pay this cost).
  • Strict JSON: simd-json adheres closely to RFC 8259. Don’t expect JSON5/trailing commas/etc.
  • CPU features matter: Without AVX2/NEON, speedups are smaller. Build with target-cpu=native in production.
  • Error messages: serde_json’s errors can be a bit friendlier; simd-json’s are fine but more terse.

Micro-optimizations that actually moved the needle

  • Use lifetimes and Cow<'a, str> for string fields you read-and-discard quickly. Add #[serde(borrow)] to enable borrowing.
  • Pre-size BytesMut with the frame length and read_exact into it; avoid intermediate copies or String allocations.
  • Keep deserialization off your async executor hot lanes if it’s heavy: move to a dedicated CPU pool when parsing multi‑KB telemetry batches.
  • Compiler flags: opt-level=3, LTO=fat, codegen-units=1 reduced noise and improved tail latency for the parsing thread.

When to stick with serde_json

  • Control messages are tiny and low volume; your bottleneck is syscalls, not parsing.
  • You cannot obtain a mutable buffer (e.g., parsing from &str coming from a library you can’t change).
  • You need exact behavior parity with serde_json-internals or rely on features simd-json doesn’t expose.

TL;DR of our migration

  • Drop-in swap for the Deserialize path required only:
    • Updating Cargo.toml and using simd_json::serde::from_slice(&mut buf) instead of serde_json::from_slice(&buf).
    • Switching stringy fields to Cow<'a, str> with #[serde(borrow)].
    • Ensuring IPC reads deliver writable bytes (BytesMut) and discarding mutated buffers after parse.
  • The deserialize hot path sped up 1.5–2.5x in our representative loads with AVX2 enabled.

Repro checklist

  • Clone your representative IPC frames and bench with Criterion using iter_batched so both parsers pay the same setup cost.
  • Compile with target-cpu=native on your target hardware.
  • Validate correctness with serde round-trips and property tests before flipping traffic.

If your IPC budget includes “JSON parse time” as a measurable line item, simd-json is very likely worth it—provided you feed it a mutable buffer and let it borrow your strings.

Looking to architect a similar system?

Let's ship it at AI-speed.

Start a conversation →