Frameworks for Benchmarking Quantum Computing Systems

A core area of our research focuses on the development and application of benchmarking methodologies for quantum computing systems. Benchmarking spans multiple levels, from hardware characterization to system-level and application-oriented evaluation. Our work emphasizes integrated approaches that capture the combined effects of quantum hardware, software stacks, and algorithm design, enabling more realistic and reproducible assessment of performance.

This includes contributions to the QED-C Application-Oriented Performance Benchmarks, an open-source framework designed to evaluate quantum systems using representative algorithms and application patterns. These benchmarks measure metrics such as result fidelity, execution time, and resource utilization across varying problem sizes, providing a practical view of system behavior under realistic workloads.

At the same time, application-oriented benchmarks are only one part of a broader strategy. Their results can depend on implementation details and execution choices, which can limit direct comparability. Our work therefore emphasizes frameworks that support systematic exploration of these factors, enabling more consistent evaluation across platforms.

We also develop measurement and analysis frameworks that capture performance across multiple dimensions, including fidelity, execution time, and scalability. These approaches support both single-circuit evaluation and iterative hybrid algorithms, where tradeoffs between solution quality and computational cost are central, and help move beyond reliance on single-number metrics.

Looking ahead, we are extending this work and contributing toward more rigorous and standardized benchmarking methodologies that integrate application-level and system-level perspectives. By combining open frameworks with high-performance simulation and detailed performance analysis, we aim to support reproducible, scientifically grounded evaluation of quantum computing systems as the field continues to evolve.