Benchmarking IQM Star#

This notebook allows you to run some useful benchmarks for the Star system.

Connect to the backend#

import os
from iqm.qiskit_iqm import IQMProvider
import random

QPU = "deneb" # or any other star QPU
token = ""
os.environ["IQM_TOKEN"] = token

iqm_url = "https://cocos.resonance.meetiqm.com/" + QPU
provider = IQMProvider(iqm_url)
backend = provider.get_backend()

C:\Users\RaphaelBrieger\miniconda3\envs\iqm-benchmarks\Lib\site-packages\iqm\iqm_client\iqm_client.py:140: UserWarning: Your IQM Client version 22.8 was built for a different version of IQM Server. You might encounter issues. For the best experience, consider using a version of IQM Client that satisfies 20.8 <= iqm-client < 22.0.
  warnings.warn(version_incompatibility_msg)

We can access the Star backend and plot its connectivity graph to check that everything is working properly.

import networkx as nx
import matplotlib.pyplot as plt

coupling_map = backend.coupling_map

G = nx.Graph()
G.add_edges_from(coupling_map) 
pos = nx.spring_layout(G, seed=42) 
nx.draw(G, pos, with_labels=True, node_color='lightblue', edge_color='gray', 
        node_size=1000, font_size=10, linewidths=1.5, width=2)
plt.show()

../_images/c7b641bc50387919b49c8965cfea220abf4d8e7459ff5cc76ff4f5f367a46cb1.png

We run the cell below to ignore those warnings that are not critical for the correct run of the benchmarks.

import warnings
warnings.filterwarnings(action="ignore")  

GHZ state fidelity#

The GHZ (Greenberger-Horne-Zeilinger) state is a maximally entangled quantum state that involves three or more qubits, $n$ . It is an equal superposition of all qubits being in state 0 and all qubits being in state 1, i.e., $| G H Z ⟩ = \frac{1}{\sqrt{2}} (| 0 ⟩^{\otimes n} + | 1 ⟩^{\otimes n})$ .

The GHZ state fidelity acts as a witness for genuine multi-qubit entanglement if found to be above $0.5$ . This means that the measurement results cannot be explained without entanglement involving all qubits, so it is a great way to evaluate the “quantumness” of the computer.

The state $ρ_{ideal} = | G H Z ⟩ ⟨ G H Z |$ is a pure state, so in this case the fidelity can be computed as:

F (ideal, measured) = ⟨ G H Z | ρ_{measured} | G H Z ⟩,

where $ρ_{measured}$ is the density matrix given by the actual results of the quantum computer. The ideal GHZ state density matrix entries can be written as $ρ_{i, j} = ⟨ i | ρ_{ideal} | j ⟩$ where $i, j$ are the $n$ basis states ${| 00. .0 ⟩, . . ., | 11. .1 ⟩}$ ; only the corner entries $ρ_{0, 0}, ρ_{0, n}, ρ_{n, 0}$ and $ρ_{n, n}$ are non-zero. This simplifies the process since we only need to measure these four components. In the fidelity formula, all other entries are effectively nullified by the zero entries in the ideal state matrix. To measure the coherences (off-diagonal entries) we use the method of multiple quantum coherences Mooney, 2021.

from iqm.benchmarks.entanglement.ghz import GHZConfiguration, GHZBenchmark

num_qubits = backend.num_qubits
chosen_layout = [list(range(qubits)) for qubits in range(2,num_qubits+1)]
GHZ = GHZConfiguration(
    state_generation_routine="star",
    custom_qubits_array=chosen_layout,
    shots=2000,
    fidelity_routine="coherences", 
    rem=True,
    mit_shots=1000,
)

benchmark_ghz = GHZBenchmark(backend, GHZ)
run_ghz = benchmark_ghz.run()

2025-03-14 16:42:11,866 - iqm.benchmarks.logging_config - INFO - Now generating a 2-qubit GHZ state on qubits [0, 1]
2025-03-14 16:42:11,868 - iqm.benchmarks.logging_config - INFO - Transpiling for backend IQM Backend with optimization level 3, sabre routing method including SQG optimization all circuits
2025-03-14 16:42:12,665 - iqm.benchmarks.logging_config - INFO - Transpiling for backend IQM Backend with optimization level 3, sabre routing method including SQG optimization all circuits
2025-03-14 16:42:12,936 - iqm.benchmarks.logging_config - INFO - Submitting batch with 7 circuits corresponding to qubits [0, 1]
2025-03-14 16:42:13,384 - iqm.benchmarks.logging_config - INFO - Now generating a 3-qubit GHZ state on qubits [0, 1, 2]
2025-03-14 16:42:13,384 - iqm.benchmarks.logging_config - INFO - Transpiling for backend IQM Backend with optimization level 3, sabre routing method including SQG optimization all circuits
2025-03-14 16:42:13,429 - iqm.benchmarks.logging_config - INFO - Transpiling for backend IQM Backend with optimization level 3, sabre routing method including SQG optimization all circuits
2025-03-14 16:42:13,762 - iqm.benchmarks.logging_config - INFO - Submitting batch with 9 circuits corresponding to qubits [0, 1, 2]
2025-03-14 16:42:14,307 - iqm.benchmarks.logging_config - INFO - Now generating a 4-qubit GHZ state on qubits [0, 1, 2, 3]
2025-03-14 16:42:14,310 - iqm.benchmarks.logging_config - INFO - Transpiling for backend IQM Backend with optimization level 3, sabre routing method including SQG optimization all circuits
2025-03-14 16:42:14,413 - iqm.benchmarks.logging_config - INFO - Transpiling for backend IQM Backend with optimization level 3, sabre routing method including SQG optimization all circuits
2025-03-14 16:42:15,066 - iqm.benchmarks.logging_config - INFO - Submitting batch with 11 circuits corresponding to qubits [0, 1, 2, 3]
2025-03-14 16:42:15,497 - iqm.benchmarks.logging_config - INFO - Now generating a 5-qubit GHZ state on qubits [0, 1, 2, 3, 4]
2025-03-14 16:42:15,501 - iqm.benchmarks.logging_config - INFO - Transpiling for backend IQM Backend with optimization level 3, sabre routing method including SQG optimization all circuits
2025-03-14 16:42:15,565 - iqm.benchmarks.logging_config - INFO - Transpiling for backend IQM Backend with optimization level 3, sabre routing method including SQG optimization all circuits
2025-03-14 16:42:16,297 - iqm.benchmarks.logging_config - INFO - Submitting batch with 13 circuits corresponding to qubits [0, 1, 2, 3, 4]
2025-03-14 16:42:16,758 - iqm.benchmarks.logging_config - INFO - Retrieving all counts
2025-03-14 16:42:25,323 - iqm.benchmarks.logging_config - INFO - Applying readout error mitigation
2025-03-14 16:43:06,137 - iqm.benchmarks.logging_config - INFO - Retrieving all counts
2025-03-14 16:43:07,265 - iqm.benchmarks.logging_config - INFO - Applying readout error mitigation
2025-03-14 16:43:12,988 - iqm.benchmarks.logging_config - INFO - Retrieving all counts
2025-03-14 16:43:14,028 - iqm.benchmarks.logging_config - INFO - Applying readout error mitigation
2025-03-14 16:43:20,028 - iqm.benchmarks.logging_config - INFO - Retrieving all counts
2025-03-14 16:43:21,991 - iqm.benchmarks.logging_config - INFO - Applying readout error mitigation

result_ghz = benchmark_ghz.analyze()
result_ghz.plot_all()

../_images/15f675a5b927adf43799313f61df916b21121dc36abfaf0ada5e14caa596dcc6.png

Quantum Volume#

Quantum volume is a single-number metric that was introduced in Cross, 2019. It evaluates the quality of a quantum processor via the largest random square circuit, i.e., with the same number of layers of parallel random 2-qubit unitaries as number of qubits, that it can run successfully.

The success of a run is based on the heavy output probability, which corresponds to the probability of observing heavy outputs, i.e. the measurement outputs that occcur with a probability greater than the median of the distribution. The heavy output generation problem asks if the generated distribution of the random circuit we run contains heavy outputs at least 2/3 of the time (on average) with a high confidence level, typically higher than 97.5%. It can be shown that the heavy output probability for an ideal device is at around 0.85 asymptotically. The quantum volume is then defined as

\log_{2} V_{q} = \underset{n}{argmax} min (n, d (n))

where $n \leq N$ is a number of qubits and $d (n)$ is the achievable depth, i.e. the largest depth such that we are confident the probability of observing a heavy output is greater than 2/3.

from iqm.benchmarks.quantum_volume.quantum_volume import QuantumVolumeConfiguration, QuantumVolumeBenchmark

We define a combination of qubits to test quantum volume on.

chosen_layouts = [[0,2,3]] ## choose the optimal layouts to run
QV = QuantumVolumeConfiguration(
    num_circuits=500, 
    shots=2**8,
    calset_id=None,
    num_sigmas=2,
    choose_qubits_routine="custom",
    custom_qubits_array=chosen_layouts, 
    qiskit_optim_level=3,
    optimize_sqg=True,
    max_gates_per_batch=10_000,
    rem=True,
    mit_shots=1_000,
)

If you want to modify the settings above, please refer to the documentation here.

Warning: The following code cell may take few minutes to run since it will compute the benchmark on all the qubit layouts specified above.

benchmark_qv = QuantumVolumeBenchmark(backend, QV)
run_qv = benchmark_qv.run()

2025-03-14 16:45:14,314 - iqm.benchmarks.logging_config - INFO - Executing QV on qubits [0, 2, 3]
2025-03-14 16:45:15,655 - iqm.benchmarks.logging_config - INFO - Successfully generated all 500 circuits to be executed
2025-03-14 16:45:15,657 - iqm.benchmarks.logging_config - INFO - Will transpile according to "fixed" physical layout
2025-03-14 16:45:15,658 - iqm.benchmarks.logging_config - INFO - Transpiling for backend IQM Backend with optimization level 3, sabre routing method including SQG optimization all circuits
2025-03-14 16:46:04,018 - iqm.benchmarks.logging_config - INFO - Submitting batch with 500 circuits corresponding to qubits [0, 2, 3]
2025-03-14 16:46:04,033 - iqm.benchmarks.logging_config - INFO - max_gates_per_batch restriction: submitting subbatch #1 with 192 circuits corresponding to qubits [0, 2, 3]
2025-03-14 16:46:05,443 - iqm.benchmarks.logging_config - INFO - max_gates_per_batch restriction: submitting subbatch #2 with 192 circuits corresponding to qubits [0, 2, 3]
2025-03-14 16:46:06,786 - iqm.benchmarks.logging_config - INFO - max_gates_per_batch restriction: submitting subbatch #3 with 116 circuits corresponding to qubits [0, 2, 3]
2025-03-14 16:46:07,863 - iqm.benchmarks.logging_config - INFO - Job for layout [0, 2, 3] submitted successfully!
2025-03-14 16:46:07,863 - iqm.benchmarks.logging_config - INFO - Retrieving all counts for [0, 2, 3]
2025-03-14 16:47:24,654 - iqm.benchmarks.logging_config - INFO - Adding counts of [0, 2, 3] run to the dataset
2025-03-14 16:47:40,986 - iqm.benchmarks.logging_config - INFO - QV experiment execution concluded !

result_qv = benchmark_qv.analyze()
for v in result_qv.plots.values():
    display(v)

2025-03-14 16:47:41,326 - iqm.benchmarks.logging_config - INFO - Noiseless simulation and post-processing for layout [0, 2, 3]
2025-03-14 16:47:42,554 - iqm.benchmarks.logging_config - INFO - REM post-processing for layout [0, 2, 3] with 1000 shots

../_images/41368adba2ee8942ea3e347fb711d64d5b2eff5c4f4bc08824173f04da9c60ee.png

../_images/e3065da959fee9e72f80c885b1fe8dae2a9c3b706bd8e51d901a99e779a5bc99.png

Circuit Layer Operations Per Second (CLOPS)#

CLOPS is a metric that estimates the speed at which a quantum computer can execute Quantum Volume (QV) layers of a quantum circuit. That is, the circuits to calculate this benchmark have the same structure as the ones used for QV. Here we follow the definition introduced in (Wack, 2021), but other versions of this benchmark exist.

CLOPS is measured by means of a quantum variational-like protocol, where templates of parametrized QV circuits are assigned random parameters, executed, and outcomes are used as a seed to assign new parameters and repeat the process. The ratio of number of templates ( $M$ ), parameter updates ( $K$ ), measurement shots ( $S$ ) and QV layers ( $\log_{2} QV$ ) with the time taken to run all, constitutes the CLOPS value:

CLOPS = M \times K \times S \times \log_{2} QV / total_time .

Notice that the total CLOPS time includes that of assignment of parameters, submission of circuits and retrieval of results.

from iqm.benchmarks.quantum_volume.clops import CLOPSConfiguration, CLOPSBenchmark, plot_times

CLOPS = CLOPSConfiguration(
    qubits=[0,2,3], # run with the same layout as 
    num_circuits=100,
    num_updates=10, 
    num_shots=100, 
    calset_id=None,
    qiskit_optim_level=3,
    optimize_sqg=True,
    routing_method="sabre",
    physical_layout="fixed",
)

If you want to modify the settings above, please refer to the documentation here.

benchmark_clops = CLOPSBenchmark(backend, CLOPS)
run_clops = benchmark_clops.run()

result_clops = benchmark_clops.analyze()
fig_clops = plot_times(result_clops.dataset, result_clops.observations)
display(fig_clops[1])

Q-Score#

The Q-score measures the maximum number of qubits that can be used effectively to solve the MaxCut combinatorial optimization problem with the Quantum Approximate Optimization Algorithm - Martiel,2021

The graphs chosen for the benchmark are random Erdős-Rényi graphs with 50% edge-probability between nodes. The obtained cost of the solution, i.e. the average number of cut edges, must be above a certain threshold. Specifically, one has to find the cost of a graph to be above $β \geq 0.2$ on a scale where $β = 0$ corresponds to a random solution and $β = 1$ to an ideal solution.

from iqm.benchmarks.optimization.qscore import QScoreConfiguration, QScoreBenchmark

num_qubits = backend.num_qubits
chosen_layout = [list(range(qubits)) for qubits in range(1,num_qubits+1)]
QSCORE = QScoreConfiguration(
    num_instances = 1,
    num_qaoa_layers= 1,
    shots = 10000,
    calset_id=None, 
    min_num_nodes = 2,
    max_num_nodes = None,
    use_virtual_node = True,
    use_classically_optimized_angles = True,
    choose_qubits_routine = "custom",
    custom_qubits_array=chosen_layout,
    seed = random.randint(1, 999999),
    REM = True,
    mit_shots = 1000,
    qpu_topology = "star",
    )

If you want to modify the settings above, please refer to the documentation here.

Warning: The following code cell may take several minutes to run.

benchmark_qscore = QScoreBenchmark(backend, QSCORE)
run_qscore = benchmark_qscore.run()

result_qscore = benchmark_qscore.analyze()

result_qscore.plot_all()

Summary#

import numpy as np

### GHZ
obs_ghz = result_ghz.observations
fidelity = round(min([obs_ghz[i].value for i in range(len(obs_ghz)) if obs_ghz[i].name=='fidelity']),2)

### QV
obs_qv = result_qv.observations
qv = max([obs_qv[i].value for i in range(len(obs_qv)) if obs_qv[i].name=='QV_result'])

### CLOPS
obs_clops = result_clops.observations
clops = max([obs_clops[item]['clops_v']['value'] for item in obs_clops])

### QS 
obs_qs = result_qscore.observations
qs = np.argmin([obs_qs[i].value-0.2 for i in range(len(obs_qs)) if obs_qs[i].name == 'mean_approximation_ratio' and obs_qs[i].value-0.2>0])+2


summary = {'GHZ state fidelity': ['≥ 0.5', fidelity],
    'Quantum Volume': qv, 
    'CLOPS':  clops, 
    'Q-Score':  qs 
}

summary

Benchmarking IQM Star

Contents

Benchmarking IQM Star#

Connect to the backend#

GHZ state fidelity#

Quantum Volume#

Circuit Layer Operations Per Second (CLOPS)#

Q-Score#

Summary#