A Technical Leader's Guide to Evaluating Quantum Claims

The Pitch You’ll Hear

A quantum computing company will tell you three things in the first five minutes: how many qubits their processor has, what problem domain they’re targeting, and a timeline for when quantum will transform your industry. The qubit number will be large and growing. The domain will be yours. The timeline will be two to five years.

None of these three things, taken alone, tells you whether the technology is real, useful, or ready.

A 1,000-qubit machine with 95% gate fidelity is less capable than a 30-qubit machine with 99.9% gate fidelity for almost any computation that matters. A vendor targeting “financial optimization” may have tested their approach on a 10-variable toy problem that a spreadsheet could solve. A “two to five year” timeline may be counting from the arrival of fault-tolerant hardware that nobody has built yet.

This chapter is your filter. It won’t make you a quantum physicist, but it will make you the person in the room who asks the questions that matter.

The Qubit Count Trap

Qubit count is to quantum computing what megapixels were to digital cameras in 2005: a number that marketing departments love and that tells you almost nothing about actual performance.

A qubit count tells you the size of the quantum register. It does not tell you:

How many qubits are usable. Manufacturing defects, frequency collisions, and calibration issues mean that not all physical qubits on a chip perform equally. Some may be effectively unusable for a given computation.

How they connect. A superconducting chip might have 1,000 qubits arranged in a grid where each qubit connects to 2-4 neighbors. A trapped-ion system might have 30 qubits where any qubit can interact with any other. The 30-qubit system can run circuits that the 1,000-qubit system cannot, because the 1,000-qubit system needs dozens of SWAP operations (each introducing errors) to move information across the chip.

How good they are. If each two-qubit gate has a 2% error rate, your 1,000-qubit system can meaningfully execute circuits of perhaps 30-40 gates deep. If each gate has a 0.1% error rate, meaningful circuit depth jumps to 500+. The second system, even with fewer qubits, can solve harder problems.

When a vendor leads with qubit count, the correct response is: “What’s the two-qubit gate fidelity? What’s the connectivity? What’s the coherence time? What quantum volume does the system achieve?”

Benchmarks That Matter

The quantum computing community has developed several benchmarks that attempt to capture system-level performance rather than single-metric bragging rights.

Quantum Volume (QV)

Proposed by IBM in 2019, quantum volume measures the largest square circuit (equal width and depth) that a quantum computer can execute reliably. A system with QV 128 can run a 7-qubit circuit 7 layers deep (2^7 = 128) with the correct heavy output appearing more than two-thirds of the time.

Quantum volume captures the interplay of qubit count, connectivity, gate fidelity, and compiler quality. A higher QV means the system can solve harder problems. Current benchmarks: Quantinuum’s H2 processor holds the highest published QV at 2^20 (1,048,576), reflecting its high gate fidelity and all-to-all connectivity. IBM’s 127-qubit Eagle processors achieve QV 128. IonQ systems have demonstrated QV in the range of 2^15 to 2^25, depending on the configuration.

Limitations: QV saturates for large systems (it’s bounded by the number of qubits), and it uses random circuits that may not reflect the performance on real applications.

Circuit Layer Operations Per Second (CLOPS)

CLOPS measures how fast a system can execute parameterized circuits in a variational loop, including classical processing overhead. This captures the speed of the classical-quantum feedback loop that dominates current NISQ workloads.

A system might have high gate fidelity but terrible CLOPS because of slow classical processing, long measurement times, or inefficient job scheduling. Since variational algorithms require thousands of circuit executions, CLOPS directly predicts how long your computation takes.

Error Per Layered Gate (EPLG)

A newer metric that measures the average error introduced by a single layer of two-qubit gates across the full processor. This captures not just the raw gate fidelity but also crosstalk and correlated errors that appear when many gates execute simultaneously.

EPLG is more informative than isolated gate fidelity because real computations run many gates in parallel. A processor might achieve 99.9% fidelity on an isolated two-qubit gate but only 99.2% EPLG when all qubits are active, because crosstalk degrades performance under load.

Application-Specific Benchmarks

The most useful benchmarks run actual applications and compare the quantum results to the best known classical results.

For chemistry: compute the ground-state energy of a benchmark molecule (like H2O or N2) and compare accuracy and runtime to classical methods (CCSD(T), DMRG, FCI).

For optimization: run a benchmark optimization problem (like Max-Cut on specific graph families) and compare solution quality and time to classical solvers (Gurobi, CPLEX, simulated annealing).

For simulation: simulate a known quantum system (like a transverse-field Ising model) and compare accuracy to exact classical simulation.

These benchmarks are the most honest because they answer the question that actually matters: does the quantum approach produce better results than the best classical alternative?

Evaluating “Quantum Advantage” Claims

When a press release says “quantum advantage achieved,” here’s your checklist.

1. What Is the Classical Baseline?

The single most important question. “Quantum advantage” means faster or better than classical. But faster than which classical method?

Google’s 2019 quantum supremacy claim compared their Sycamore processor to a specific classical simulation strategy. IBM immediately showed that a different classical strategy could solve the same problem much faster. The advantage was real under one comparison and questionable under another.

When evaluating a claim, ask: was the classical baseline the best known classical algorithm for this specific problem, running on appropriate classical hardware (not a laptop, but an HPC system)? Was the classical implementation optimized by someone who actually works on classical algorithms, or was it a naive implementation designed to lose?

2. What Problem Size?

Advantage at a toy scale doesn’t imply advantage at a practical scale. A quantum approach might beat classical methods for 10 variables but lose at 100 variables because the quantum overhead (in circuit depth, error mitigation shots, and post-processing) grows faster than the classical computation time.

Equally, advantage at a practical scale might not exist at a toy scale. The theoretical quantum speedup for certain problems only kicks in above a threshold size. Below that, classical methods win even if quantum wins eventually.

Ask: at what problem size was the comparison made? Is this size commercially relevant? How does the scaling comparison look, not just at one data point, but across a range of sizes?

3. What Error Mitigation Was Used?

Error mitigation techniques (Chapter 4) can dramatically improve quantum results but at the cost of exponentially more measurement shots. A result that required 10 million shots to produce through zero-noise extrapolation is not the same as a result obtained from a single circuit execution.

The total computational cost includes all shots, all classical post-processing for error mitigation, and all overhead. When this is accounted for, some claimed advantages shrink or vanish.

Ask: what error mitigation was applied? How many total shots were required? What is the total wall-clock time including all overhead?

4. Is the Comparison Published and Reproducible?

Claims in peer-reviewed publications carry more weight than press releases. Claims with published code and data that others can verify carry the most weight.

The quantum computing field has a reproducibility challenge. Hardware access is limited, calibration changes daily, and results depend on specific implementation details. A result published with circuit descriptions, transpilation details, error mitigation parameters, and raw measurement data is trustworthy. A result presented in a webinar with “proprietary benchmarks” is not.

Red Flags

“Our system has N qubits” (with no fidelity or benchmark data). They’re selling the megapixel number.

“Applicable to any optimization problem.” No quantum approach is universally applicable. If they can’t name specific problem structures where their approach outperforms classical methods, they don’t have advantage.

“Quantum-inspired algorithms running on classical hardware.” These are classical algorithms that borrow mathematical ideas from quantum computing. They can be genuinely useful, but they’re not quantum computing. If a vendor includes these results in their “quantum” pitch without clear labeling, they’re blurring the line deliberately.

“We’ll achieve quantum advantage in 18 months.” Everyone has been saying this for 10 years. Ask: what specific technical milestone must be reached? What is the current state relative to that milestone? What error correction capability is required?

“Our proprietary approach bypasses the need for error correction.” No. Noise is physics, not engineering. Any claim to avoid error correction entirely should be treated with extreme skepticism. Error mitigation can reduce the impact of noise for shallow circuits, but it has fundamental scaling limits (Chapter 4).

Customer testimonials with no published results. “Fortune 500 company X is using our quantum platform” means they have an account. It does not mean they have useful results. Ask for published case studies with quantitative comparisons.

Green Flags

Published benchmarks across multiple metrics. Vendors who report QV, CLOPS, EPLG, and application benchmarks are being transparent about system-level performance.

Honest limitations. A vendor who says “our system is best suited for problems with structure X, and is not competitive for problems with structure Y” understands their technology. A vendor who claims universal applicability doesn’t.

Classical comparison methodology. Published results that compare against optimized classical baselines, and acknowledge when classical methods win, indicate scientific rigor.

Error correction roadmap. A clear plan for transitioning from NISQ to error-corrected computation, with specific milestones and timelines, shows long-term engineering thinking rather than short-term hype.

Peer-reviewed publications from the team. Research papers in Physical Review, Nature, Science, Quantum, or similar journals indicate that the technical claims have survived expert review.

Open-source tools and transparent pricing. Companies that publish their software frameworks, provide clear documentation, and offer transparent pricing models are building for long-term adoption, not short-term hype cycles.

The Questions to Ask

When a quantum vendor presents to your organization, here’s what to ask.

On hardware: What is your two-qubit gate fidelity? What is your connectivity graph? What are your T1 and T2 times? What quantum volume does your system achieve? What is your EPLG under full load?

On the proposed application: What mathematical formulation are you using for our problem? What is the qubit encoding? What is the circuit depth? How does this scale with problem size? What error mitigation techniques are needed, and what is the sampling overhead?

On advantage claims: What classical method are you comparing against? Who implemented and optimized the classical baseline? At what problem size does the crossover occur? Is the comparison published? Can we reproduce it?

On timeline: What error correction scheme is your roadmap based on? What physical error rate do you need to achieve? When do you project reaching that error rate? What is the contingency if you don’t?

On business model: What does access cost at the scale we’d need? What happens to our computations if your hardware platform changes? Can we run the same code on a simulator for development? What’s the lock-in risk?

The Honest State of Things

Quantum computing is real technology being built by serious engineers and physicists. It is not a scam. It is also not ready for production use in most commercial applications.

The organizations that will benefit most from quantum computing are those that start building understanding now, not by spending millions on quantum hardware access, but by identifying which of their problems have quantum-amenable structure, building the mathematical formulation expertise to translate those problems, establishing clean architectural interfaces for quantum backends, and monitoring the progress of error correction milestones that determine when quantum hardware becomes practically useful.

The vendor that tells you “start now because the advantage is here” is premature for most use cases. The vendor that tells you “ignore quantum, it’s all hype” is wrong about the science and careless about the timeline. The correct position is in between: the technology is real, the timeline is uncertain, and the preparation you do today determines whether you can move fast when the hardware catches up.

The best quantum strategy for most technical leaders in 2025-2026 is to become the person in the room who understands the technology well enough to ask the right questions. That costs nothing and compounds over time.

You’ve just finished the seven chapters that equip you to do exactly that.