Quantum Benchmark Zoo

← All benchmarks

decoder-bench

Error correction

Suite of Stim-generated syndrome traces for scoring classical QEC decoders on accuracy and latency across surface, color, and bivariate-bicycle codes plus lattice surgery.

Measures
Accuracy (logical error rate) and latency of classical QEC decoders on a shared set of Stim-generated syndrome traces
Introduced by
Maurya, Viszlai, Raveendran, Das & Tannu (UW-Madison, UChicago, U. Arizona, UT Austin) (2025)
Also known as
Decoder-Bench, DecoderBench

decoder-bench (styled lowercase in the paper and repository) is a benchmark suite for the classical half of quantum error correction: the decoders that turn measured syndromes into corrections. Introduced by Maurya, Viszlai, Raveendran, Das & Tannu at IEEE IISWC 2025, it standardizes the inputs on which decoder accuracy and speed are compared: previously each decoder paper generated its own, making results hard to line up. It sits in this catalog’s error-correction section, but the object under test is classical decoder software and hardware; no quantum device is measured.

How it works

The suite fixes the workload. Syndrome traces are generated with Stim for memory experiments on surface codes, color codes, and bivariate bicycle (qLDPC) codes, plus surface-code lattice surgery, under code-capacity, phenomenological, and circuit-level noise. Candidate decoders consume the shared traces and are scored on two axes: accuracy, as the logical error rate achieved, and latency, the time to produce corrections. The MIT-licensed harness lives on GitHub and the 2.8 GB trace dataset is archived on Zenodo, so comparisons are reproducible trace-for-trace.

Strengths and limitations

Shared traces make decoder comparisons apples-to-apples, and coverage extends past the usual surface-code memory experiment to lattice surgery and bivariate bicycle codes, where real-time decoding pressure is heading. Scoring latency alongside accuracy reflects the fault-tolerance constraint that decoders must keep pace with syndrome extraction.

Because every trace is simulated, realism goes only as far as the noise models; rankings under Stim circuit-level noise need not carry over to hardware data of the kind behind logical error per round or stability experiments. The primary paper sits behind the IEEE Xplore paywall with no arXiv preprint, though the artifact is fully open. And the suite is young (released August 2025), so adoption beyond the authors’ groups remains to be seen. Not to be confused with qec-lego-bench, a separate decoder benchmarking effort.

Notable results

Presented at IEEE IISWC 2025 (October 2025); the August 2025 code and dataset release remains the current version as of mid-2026.

Key papers

Reference implementations

  • Logical error rate per round : The headline figure of QEC memory experiments: the probability per round of error correction that the decoded logical qubit suffers a logical flip.
  • Stability experiment : The dual of the QEC memory experiment: instead of preserving a logical observable over time, it scores how well syndrome extraction and the decoder determine a product of stabilizers across space.
  • FTPrimitiveBench : Stim-based suite that scores logical error rates of surface-code Clifford primitives (memory, lattice surgery, transversal Hadamard, and the S gate) under hardware-motivated biased noise.