SIMD sentiment scoring in Rust via WASM workers (no UI jank)

•8 min read•Yaseen Khatib · MERN + AI Architect
#Rust#WebAssembly#AI
Cover illustration: SIMD sentiment scoring in Rust via WASM workers (no UI jank)

Why SIMD for chat sentiment?

You don’t need a transformer to score a chat line. A lexicon with per-token floats plus a couple of lightweight rules handles the jobs that matter in a client: highlight toxic posts, surface excitable threads, tweak notification sounds. The core loop is a sum of small floats across a token list. That work is trivially parallel and maps cleanly to SIMD. Compile it to WebAssembly, run it inside a Web Worker, and you keep keystroke-time analysis off the UI thread.

Below, I’ll cover:

  • A Rust SIMD kernel that sums f32 weights on wasm32 simd128 and x86_64 AVX2.
  • A batch API that reduces FFI chatter.
  • Web Worker plumbing that keeps React/Next/Electron snappy.

The scoring model (simple, cheap, good-enough)

  • Tokenize to words/emojis (in JS or Rust). Map each token to a float from SentiWordNet/AFINN/your own table. Add negation and intensifiers as scalar multipliers where needed.
  • The hot loop is a sum over weights, optionally a dot with per-token factors. It stays branch-free, uses little memory, and is friendly to caches.

If you want VADER-like rules later, keep them around the SIMD sum in scalar code so the vectorized path remains straight-line.

A SIMD kernel in Rust (WASM + AVX2 + scalar fallback)

We’ll implement:

  • sum_f32_simd: vector adds with a horizontal reduction at the end.
  • score_batch_weights: a batch entry point that takes a flat f32 array plus offsets.
// src/lib.rs
use wasm_bindgen::prelude::*;

#[inline]
fn sum_f32_scalar(xs: &[f32]) -> f32 {
    let mut acc = 0.0f32;
    // Manual unroll helps even scalar
    let mut i = 0;
    let len = xs.len();
    while i + 4 <= len {
        acc += xs[i] + xs[i + 1] + xs[i + 2] + xs[i + 3];
        i += 4;
    }
    while i < len { acc += xs[i]; i += 1; }
    acc
}

// wasm32 simd128 path
#[cfg(all(target_arch = "wasm32"))]
#[inline]
unsafe fn sum_f32_wasm128(xs: &[f32]) -> f32 {
    use core::arch::wasm32::*;
    let mut acc = f32x4_splat(0.0);
    let mut i = 0;
    let len = xs.len();
    // Process 4 floats per iteration
    while i + 4 <= len {
        let v = v128_load(xs.as_ptr().add(i) as *const v128);
        acc = f32x4_add(acc, v);
        i += 4;
    }
    // Horizontal add 4 lanes
    let mut sum = f32x4_extract_lane::<0>(acc)
        + f32x4_extract_lane::<1>(acc)
        + f32x4_extract_lane::<2>(acc)
        + f32x4_extract_lane::<3>(acc);
    // Remainder
    while i < len { sum += *xs.get_unchecked(i); i += 1; }
    sum
}

// x86_64 AVX2 path
#[cfg(all(target_arch = "x86_64", target_feature = "avx2"))]
#[inline]
unsafe fn sum_f32_avx2(xs: &[f32]) -> f32 {
    use core::arch::x86_64::*;
    let mut acc = _mm256_setzero_ps();
    let mut i = 0;
    let len = xs.len();
    while i + 8 <= len {
        let v = _mm256_loadu_ps(xs.as_ptr().add(i));
        acc = _mm256_add_ps(acc, v);
        i += 8;
    }
    // Fold 8 lanes -> 4 -> 1
    let hi = _mm256_extractf128_ps(acc, 1);
    let lo = _mm256_castps256_ps128(acc);
    let s128 = _mm_add_ps(lo, hi);                 // 4 lanes
    // Use hadd (SSE3) to horizontally reduce
    let s64 = _mm_hadd_ps(s128, s128);             // [a0+a1, a2+a3, a0+a1, a2+a3]
    let s32 = _mm_hadd_ps(s64, s64);               // [sum, sum, sum, sum]
    let mut sum = _mm_cvtss_f32(s32);
    while i < len { sum += *xs.get_unchecked(i); i += 1; }
    sum
}

#[inline]
fn sum_f32_simd(xs: &[f32]) -> f32 {
    #[cfg(all(target_arch = "wasm32"))]
    unsafe { return sum_f32_wasm128(xs); }

    #[cfg(all(target_arch = "x86_64", target_feature = "avx2"))]
    unsafe { return sum_f32_avx2(xs); }

    // Fallback
    sum_f32_scalar(xs)
}

#[wasm_bindgen]
pub fn score_batch_weights(flat: Vec<f32>, offsets: Vec<u32>) -> Vec<f32> {
    // offsets[i]..offsets[i+1] form a message; final offset == flat.len()
    assert!(offsets.len() >= 2, "need at least start and end offset");
    let mut out = Vec::with_capacity(offsets.len() - 1);

    // Safety: we only take sub-slices within flat bounds
    for w in offsets.windows(2) {
        let a = w[0] as usize;
        let b = w[1] as usize;
        let slice = &flat[a..b];
        let s = sum_f32_simd(slice);
        out.push(s);
    }
    out
}

#[wasm_bindgen]
pub fn score_message_weights(weights: Vec<f32>) -> f32 {
    sum_f32_simd(&weights)
}

Notes:

  • Both wasm32 and x86_64 versions build; only the active target makes it into the artifact. A scalar path is always there as a safe default.
  • The wasm code uses unaligned loads (fine in wasm). AVX2 uses _mm256_loadu_ps for the same reason.

Building for wasm32 with SIMD

# 1) Add target
rustup target add wasm32-unknown-unknown

# 2) Build with simd128 enabled
RUSTFLAGS="-C target-feature=+simd128" \
  cargo build --release --target wasm32-unknown-unknown

# Or use wasm-pack (recommended for bundlers)
cargo install wasm-pack
RUSTFLAGS="-C target-feature=+simd128" \
  wasm-pack build --release --target web

Modern browsers ship wasm SIMD. If you must keep older ones alive, detect support at runtime and fall back to JS or a non-SIMD wasm build.

Wiring a Web Worker in a React/Next chat

We micro-batch messages, compute weights in the worker, and send scores back. To keep the example tight, assume the UI thread already turned tokens into Float32Array weights and ships them to the worker. In production, move tokenization and lookup into the worker too.

// sentiment.worker.ts
// Vite/Next build: let the bundler treat this as a module worker
import init, { score_batch_weights } from './pkg/sentiment_wasm.js';

let ready: Promise<void> | null = null;

self.onmessage = async (ev: MessageEvent) => {
  const { flatWeights, offsets } = ev.data as {
    flatWeights: Float32Array; // concatenated weights
    offsets: Uint32Array;      // message boundaries
  };

  if (!ready) ready = init().then(() => {});
  await ready;

  // Copy into wasm (simple path). Optimize later with shared memory.
  const flat = Array.from(flatWeights);
  const offs = Array.from(offsets);
  const scores = score_batch_weights(flat, offs);
  // scores is a JS array (Vec<f32> from wasm-bindgen)
  (self as any).postMessage({ scores });
};

On the main thread, batch messages over a short window (about 8–16 ms) to spread the copy cost.

// ui-sentiment.ts
const worker = new Worker(new URL('./sentiment.worker.ts', import.meta.url), { type: 'module' });

let batch: number[] = [];      // flat weights
let offsets: number[] = [0];   // starts at 0

function enqueueWeights(weights: number[]) {
  batch.push(...weights);
  offsets.push(batch.length);
}

let scheduled = false;
function flush() {
  if (scheduled) return;
  scheduled = true;
  requestIdleCallback(() => {
    const flat = new Float32Array(batch);
    const offs = new Uint32Array(offsets);
    worker.postMessage({ flatWeights: flat, offsets: offs });
    batch = []; offsets = [0]; scheduled = false;
  }, { timeout: 16 });
}

// Usage per message (post-tokenization and lexicon lookup):
export function scoreMessageSoon(weights: number[]) {
  enqueueWeights(weights);
  flush();
}

worker.onmessage = (ev) => {
  const { scores } = ev.data as { scores: number[] };
  // Bind scores back to messages in the same order
  for (const score of scores) {
    // e.g., dispatch to store, colorize row, etc.
    // store.dispatch(updateMessageScore({ id, score }))
  }
};

These paths copy buffers at the worker boundary and again into wasm memory. For short chat messages, that’s acceptable. If you need zero-copy:

  • Use a SharedArrayBuffer for the flat weights with the right isolation headers, and read it from wasm by pointer.
  • Or allocate in wasm, return a pointer/length, and fill via a view on WebAssembly.Memory (supported by wasm-bindgen).

Negation, intensifiers, and still staying vectorized

Treat scope modifiers as premultipliers in the stream:

  • Map tokens to base weights w[i].
  • Compute m[i] (negation flips sign until punctuation; intensifiers scale by 1.5, etc.).
  • Feed SIMD the product v[i] = m[i] * w[i] and sum v. You can precompute v in scalar code, or add a multiply then add in the kernel. wasm lacks FMA, but f32x4_mul followed by f32x4_add is cheap enough.

Performance characteristics and gotchas

  • Throughput: On an M2 in Safari/Chrome, summing ~64 floats with wasm SIMD often lands around 10–20 GB/s effective bandwidth, with per-message times under a microsecond. AVX2 on desktops is in the same ballpark or faster. JS↔WASM copies and worker hops dominate.
  • Amortize overhead: Batch multiple messages and flatten buffers as shown.
  • Cache friendliness: Keep the lexicon lookup tight. If tokenizing in the worker, store weights contiguously and avoid hashmap work in the hot path.
  • Branchlessness: Keep the SIMD loop free of branches. Handle emoji and negation before the sum.
  • Feature gating: If you expose AVX2 natively, also include SSE2 or scalar. For wasm, browsers without SIMD won’t instantiate; detect and fall back.
  • Numerics: f32 is fine here. For very long texts, consider Kahan or pairwise summation in the scalar tail to trim error. For chat-sized inputs, skip it.

Optional: native Node/Electron path

In Electron, if you want to avoid wasm copies, expose the same kernel with N-API (e.g., napi-rs) and keep the worker-thread architecture. Build with RUSTFLAGS="-C target-cpu=native" to pick up AVX2/FMA on your fleet, or ship multiple ISAs and dispatch at runtime.

What this buys you in a MERN stack

  • React stays responsive because the work runs in a Worker.
  • Mongo/Express aren’t affected; score persistence can stay async.
  • With SSR/Next, run the same Rust crate on the server for backfill and keep the client on wasm for immediate feedback.

Further optimizations

  • Add a fast-path tokenizer (ASCII first, simdutf8 for validation) only if profiles show CPU there.
  • Use wasm-bindgen reference types and WebAssembly.Memory views to reduce copies.
  • If you move to a dot-product over k>1 features, vectorize mul and add and finish with a lane-wise reduction.

Checklist

  • SIMD in Rust via core::arch for wasm32 and AVX2, plus a scalar fallback.
  • Work off the UI thread with a module Worker and micro-batching.
  • Detect features and provide clean fallbacks.
  • Measure with a sampling CPU profiler and performance.now() around worker boundaries; focus on copy costs, not just kernel FLOPs.

You get low-latency sentiment without model serving, lower battery impact, and a UI that keeps its frame budget even when traffic spikes.

Need an engineer who can build this?

I'm Yaseen Khatib — a Senior Full-Stack AI Engineer (MERN + TypeScript) who ships production AI systems solo. Open to senior and lead roles, remote or on-site.

Get in touch →See what I've shipped