serde_json to simd-json in streamerOS IPC: hot-path deserialization

Why we swapped parsers

Our streamerOS IPC shuttles JSON over Unix domain sockets all day. Even with framing costs amortized, the deserialize path was still a noticeable CPU sink. serde_json is dependable and everywhere, but its scalar parser and string allocation pattern weren’t a fit for how our data looks:

• Many small control envelopes (~300–800 bytes) with lots of strings.
• Regular telemetry blobs (5–50 KB) with numeric arrays plus string tags.

simd-json adds SIMD-accelerated scanning (AVX2 on x86_64, NEON on aarch64) and can borrow strings zero-copy by unescaping in place. The trade-off: it needs a mutable byte slice (&mut [u8]) and will rewrite structural characters while parsing.

Below is the exact benchmark we used for the deserialize hot path and the integration points in the IPC layer, including the parts that can bite.

Setup and feature flags

Cargo.toml snippets:

[dependencies]
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
simd-json = { version = "0.13", features = ["serde_impl"] }

[dev-dependencies]
criterion = "0.5"

[profile.release]
opt-level = 3
lto = "fat"
codegen-units = 1

For best results, compile with your CPU’s SIMD features:

# .cargo/config.toml
[target.x86_64-unknown-linux-gnu]
rustflags = ["-C", "target-cpu=native"]

[target.aarch64-apple-darwin]
rustflags = ["-C", "target-cpu=native"]

simd-json will pick the fastest implementation at compile/runtime, but a generic target CPU can leave performance on the floor.

The message shape (borrowable strings)

We marked string-heavy fields as borrowable so simd-json can use its zero-copy path. serde_json will still allocate String for those; simd-json can return &str that point into the mutable input buffer.

use serde::Deserialize;
use std::borrow::Cow;

#[derive(Deserialize, Debug)]
#[serde(tag = "kind")]
enum Envelope<'a> {
    Control {
        ts: u64,
        stream_id: u64,
        #[serde(borrow)] op: Cow<'a, str>,
        #[serde(borrow)] node: Cow<'a, str>,
        #[serde(borrow)] tags: Vec<Cow<'a, str>>,
    },
    Telemetry {
        ts: u64,
        stream_id: u64,
        // numerics still allocate; only strings can be borrowed
        samples: Vec<f32>,
        #[serde(borrow)] metric: Cow<'a, str>,
    }
}

Example JSONs representative of our IPC frames:

{"kind":"Control","ts":1710000000000,"stream_id":42,"op":"resume","node":"ingest-a","tags":["vip","cdn-eu"]}

{"kind":"Telemetry","ts":1710000000000,"stream_id":42,"samples":[0.12,0.08,0.10,0.09,0.13,0.11],"metric":"p95_latency"}

Criterion benchmark for the deserialize hot path

Two constraints shaped the bench:

1. simd-json takes &mut [u8] and mutates it.
2. Each iteration materializes a fresh buffer for both parsers so setup costs match what the network path would pay.

// benches/ipc_deser.rs
use criterion::{black_box, criterion_group, criterion_main, BatchSize, Criterion};
use serde::Deserialize;
use std::borrow::Cow;

#[derive(Deserialize, Debug)]
#[serde(tag = "kind")]
enum Envelope<'a> {
    Control {
        ts: u64,
        stream_id: u64,
        #[serde(borrow)] op: Cow<'a, str>,
        #[serde(borrow)] node: Cow<'a, str>,
        #[serde(borrow)] tags: Vec<Cow<'a, str>>,
    },
    Telemetry {
        ts: u64,
        stream_id: u64,
        samples: Vec<f32>,
        #[serde(borrow)] metric: Cow<'a, str>,
    }
}

static CONTROL_JSON: &str = r#"{
  "kind":"Control",
  "ts":1710000000000,
  "stream_id":42,
  "op":"resume",
  "node":"ingest-a",
  "tags":["vip","cdn-eu","edge-12"]
}"#;

static TELEMETRY_JSON: &str = r#"{
  "kind":"Telemetry",
  "ts":1710000000000,
  "stream_id":42,
  "samples":[0.12,0.08,0.10,0.09,0.13,0.11,0.12,0.10,0.12,0.09,0.11,0.12,0.10,0.09,0.13,0.11],
  "metric":"p95_latency"
}"#;

fn bench_control(c: &mut Criterion) {
    let mut g = c.benchmark_group("control-envelope");

    g.bench_function("serde_json", |b| {
        b.iter_batched(
            || CONTROL_JSON.as_bytes().to_vec(),
            |mut buf| {
                let env: Envelope = serde_json::from_slice(&buf).unwrap();
                black_box(env);
                black_box(buf);
            },
            BatchSize::SmallInput,
        );
    });

    g.bench_function("simd_json", |b| {
        b.iter_batched(
            || CONTROL_JSON.as_bytes().to_vec(),
            |mut buf| {
                let env: Envelope = simd_json::serde::from_slice(&mut buf).unwrap();
                black_box(env);
                black_box(buf);
            },
            BatchSize::SmallInput,
        );
    });

    g.finish();
}

fn bench_telemetry(c: &mut Criterion) {
    let mut g = c.benchmark_group("telemetry-envelope");

    g.bench_function("serde_json", |b| {
        b.iter_batched(
            || TELEMETRY_JSON.as_bytes().to_vec(),
            |mut buf| {
                let env: Envelope = serde_json::from_slice(&buf).unwrap();
                black_box(env);
                black_box(buf);
            },
            BatchSize::SmallInput,
        );
    });

    g.bench_function("simd_json", |b| {
        b.iter_batched(
            || TELEMETRY_JSON.as_bytes().to_vec(),
            |mut buf| {
                let env: Envelope = simd_json::serde::from_slice(&mut buf).unwrap();
                black_box(env);
                black_box(buf);
            },
            BatchSize::SmallInput,
        );
    });

    g.finish();
}

criterion_group!(benches, bench_control, bench_telemetry);
criterion_main!(benches);

Run the bench:

RUSTFLAGS="-C target-cpu=native" cargo bench --bench ipc_deser --release

Sample results (Ryzen 7950X, Rust 1.78, AVX2 enabled)

Workload	serde_json (µs)	simd-json (µs)	Speedup
Control (~420 B)	1.45	0.88	1.65x
Telemetry (~1.1 KB)	5.64	2.37	2.38x

Expect variance by CPU, JSON shape, and memory pressure. Bigger payloads and higher string density generally widen the gap in favor of simd-json.

Integrating simd-json in the IPC hot path

Our frames are length-prefixed over tokio’s UnixStream. The goal is to parse in place from a mutable buffer. bytes::BytesMut fits that model well.

use bytes::{BufMut, BytesMut};
use simd_json::serde as simd_serde;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::UnixStream;

async fn read_frame(stream: &mut UnixStream) -> tokio::io::Result<Envelope<'_>> {
    let mut len_buf = [0u8; 4];
    stream.read_exact(&mut len_buf).await?;
    let len = u32::from_le_bytes(len_buf) as usize;

    let mut buf = BytesMut::with_capacity(len);
    // Safety: reserve ensures capacity; put_bytes initializes
    buf.put_bytes(0, len);
    stream.read_exact(&mut buf[..]).await?;

    // Parse in-place. simd-json will modify the buffer to zero-copy strings.
    let env: Envelope = simd_serde::from_slice(&mut buf[..]).map_err(|e| {
        use std::io::{Error, ErrorKind};
        Error::new(ErrorKind::InvalidData, format!("JSON: {e}"))
    })?;
    Ok(env)
}

Notes:

• After simd-json mutates the buffer, don’t try to reuse those contents; either drop or overwrite them. Our path allocates per frame, so this is a non-issue.
• If an upstream API only gives you &str, you’ll have to copy into a Vec, which is expensive. Prefer receiving directly into a mutable byte buffer (Vec, BytesMut, or a pool) to realize simd-json’s benefits.

Where the wins come from

• SIMD scanning: wide-register detection of structural characters and fast UTF‑8 validation speeds up small and large documents alike.
• In-place unescaping: strings without escapes become &str slices into the input, removing per-field allocations. In control messages that cuts allocs on op, node, and tags.
• Tighter hot path: fewer bounds checks and branches, which shows up more as payloads grow.

Sharp edges and caveats

• Needs &mut [u8]: a string-only pipeline forces an extra copy, which can wipe out gains on very small messages.
• Zero-copy applies only to strings: numeric arrays are still parsed and allocated in both parsers.
• Strict JSON only: closely follows RFC 8259; no JSON5, trailing commas, or similar extensions.
• CPU features matter: without AVX2/NEON the speedups shrink. Build with target-cpu=native for production.
• Error reporting: serde_json’s messages are usually friendlier; simd-json’s are shorter and more to the point.

Micro-optimizations that actually moved the needle

• Use lifetimes with Cow<'a, str> on short-lived string fields and add #[serde(borrow)] so they can be borrowed.
• Size BytesMut to the frame and read_exact directly into it; skip temporary buffers and String creation.
• Keep heavy deserialization off hot async executors: push multi‑KB telemetry parsing to a dedicated CPU pool.
• Compiler knobs: opt-level=3, LTO=fat, and codegen-units=1 reduced variance and helped tail latency on the parsing thread.

When to stick with serde_json

• Control messages are tiny and infrequent; syscalls dominate, not parsing.
• You can’t get a writable buffer (you only receive &str from an external library).
• You require exact serde_json behavior or features simd-json doesn’t surface.

TL;DR of our migration

• The Deserialize swap was straightforward:
  • Update Cargo.toml and call simd_json::serde::from_slice(&mut buf) instead of serde_json::from_slice(&buf).
  • Convert string fields to Cow<'a, str> with #[serde(borrow)].
  • Make IPC reads deliver writable bytes (BytesMut) and discard mutated buffers after parsing.
• With AVX2 enabled, deserialize on our representative loads sped up by 1.5–2.5x.

Repro checklist

• Capture real IPC frames and bench with Criterion using iter_batched so both parsers incur equal setup.
• Compile with target-cpu=native on the actual target machines.
• Verify correctness with serde round-trips and property tests before shifting traffic.

If “JSON parse time” shows up as a measurable line item in your IPC budget and you can feed a mutable buffer that allows string borrowing, simd-json is very likely worth it.