iqm.benchmarks.benchmark_definition.Benchmark

class iqm.benchmarks.benchmark_definition.Benchmark(backend: str | IQMBackendBase, configuration: BenchmarkConfigurationBase, **kwargs)

Bases: ABC

A base class for running cortex-based Benchmark experiments.

In order to write a new benchmark, it is recommended to derive a class from this baseclass. A new benchmark is defined by deriving from this base class and implementing the execute method. Additionally, a custom analysis for the benchmark can be implemented by giving the pointer to the analysis method in analysis_function field. The given analysis_function should accept AnalysisResult as its input and return the final result.

Attributes

default_options
options

Methods

analysis_function(result)	The default analysis that only pass the result through.
analyze([run_index])	The default analysis for the benchmark.
execute(backend)	Executes the benchmark.
name()	Return the name of the benchmark.
run([calibration_set_id])	Runs the benchmark using the given backend and calibration_set_id.

Parameters:

• backend (str | IQMBackendBase) –

• configuration (BenchmarkConfigurationBase) –

static analysis_function(result: BenchmarkRunResult) → BenchmarkAnalysisResult

The default analysis that only pass the result through.

Parameters:

result (BenchmarkRunResult) –

Return type:

BenchmarkAnalysisResult

abstract classmethod name()

Return the name of the benchmark.

abstract execute(backend: IQMBackend | IQMFacadeBackend | str) → Dataset

Executes the benchmark.

This method should be overridden by deriving classes.

Parameters:

backend (IQMBackend | IQMFacadeBackend | str) – Qiskit backend used to execute benchmarks.

Returns:

An xarray dataset which contains to results of the benchmark execution. The dataset should contain all the information necessary for analysing the benchmark results.

Return type:

Dataset

run(calibration_set_id: str | UUID | None = None) → BenchmarkRunResult

Runs the benchmark using the given backend and calibration_set_id.

Parameters:

calibration_set_id (str | UUID | None) – CalibrationSetId used to initialize the backend or None for the latest calibration set.

Returns:

The result of the benchmark run.

Return type:

RunResult

analyze(run_index=-1) → BenchmarkAnalysisResult

The default analysis for the benchmark.

Internally uses the function defined by the attribute analysis_function to perform the analysis. This function makes a shallow copy of the dataset produced by run. Therefore, it is recommended to not make changes to the data of the dataset but just the structure of the array.

Parameters:

run_index – Index for the run to analyze.

Returns:

An analysis result constructed from the run and updated by the analysis method defined by the analysis_function field.

Return type:

BenchmarkAnalysisResult