Quantum Benchmark Zoo

← All benchmarks

QED-C Application-Oriented Benchmarks

Application-level

Open-source consortium suite that scores devices on ~16 algorithm workloads, from Bernstein-Vazirani to VQE and MaxCut, via normalized fidelity on width-by-depth volumetric plots.

Measures
Normalized result fidelity per algorithm benchmark as a function of circuit width and depth, shown on volumetric plots, alongside execution-time metrics
Introduced by
QED-C (Quantum Economic Development Consortium) (2021)
Also known as
QED-C Application-Oriented Performance Benchmarks, Application-Oriented Performance Benchmarks, QC-App-Oriented-Benchmarks, QED-C benchmarks

The QED-C application-oriented benchmarks are an open-source suite from the Quantum Economic Development Consortium’s Technical Advisory Committee on Standards and Performance Metrics, first released in 2021 with Thomas Lubinski as lead author (Lubinski et al., published in IEEE Transactions on Quantum Engineering in 2023). Where random-circuit tests score a device in the abstract, this suite asks how well it runs recognizable quantum programs, and it has become a de facto reference point for application-level comparison, to the extent that IonQ’s Algorithmic Qubits metric is a pinned derivative of it.

How it works

The suite comprises roughly 16 algorithm benchmarks: Deutsch-Jozsa, Bernstein-Vazirani, hidden shift, QFT, phase and amplitude estimation, Grover search, HHL, Monte Carlo sampling, Hamiltonian simulation (including HamLib instances), VQE, Shor period finding, MaxCut, hydrogen lattice, and image recognition. Each is swept over problem sizes, and every run’s measured output distribution is compared with the classically computed ideal to produce a normalized result fidelity. Scores are positioned on width-by-depth volumetric plots, so application performance can be read against the circuit shapes a device sustains, and execution-time metrics are recorded alongside. The reference repository (Apache-2.0) implements the suite for Qiskit, Cirq, Amazon Braket, Q#, and CUDA-Q.

Strengths and limitations

Its strengths are vendor neutrality, breadth, and upkeep: the suite is consortium-governed, spans five programming frameworks, remains under active release (v2.0.4, May 2026), and keeps growing through follow-on studies on combinatorial optimization and a continuing series of further application areas (2402.08985, 2409.06919, 2504.09813, 2510.08469). The limitations are structural. Normalized fidelity requires the classically simulated ideal distribution, so verifiable scores stop where classical simulation does; the workloads are simplified, small-instance versions of their namesake applications; and the suite prescribes no single summary number, the gap Algorithmic Qubits fills at the cost of pinning parameters. Peer suites cover adjacent ground: SupermarQ scores feature-mapped applications, while QASMBench offers an unscored circuit collection.

Key papers

Reference implementations

  • SupermarQ : Scalable, hardware-agnostic suite of eight application-level benchmarks, with a six-dimensional feature vector that profiles how each workload stresses a device.
  • Algorithmic Qubits : IonQ's single-number application metric, the largest width at which a six-algorithm circuit suite clears a 1/e fidelity bar, retired in 2025 in favor of industry-standard metrics.
  • Volumetric benchmarking : Sandia framework that generalizes Quantum Volume from square circuits to the full width × depth plane, mapping the frontier of circuit shapes a device can execute.
  • HamLib : More than 1.5 million pre-encoded qubit Hamiltonians (spin models, chemistry, and combinatorial optimization) supplying standardized problem instances for application-level quantum benchmarking.