API Reference¶

This page documents the main Python modules in Metriq-Gym.

Core Modules¶

metriq_gym.run¶

Main runtime module with dispatch, poll, and upload functions.

Runtime entrypoints for dispatching and managing metriq-gym benchmarks via the CLI.

`dispatch_job(args, job_manager)` ¶

Dispatch a single benchmark configuration to a quantum device.

Parameters:

Name	Type	Description	Default
`args`	`Namespace`	CLI arguments with benchmark config, provider, and device	required
`job_manager`	`JobManager`	Tracks dispatched jobs for later polling	required

Note: Continues processing remaining configs if individual configs fail.

`poll_job(args, job_manager)` ¶

`upload_job(args, job_manager)` ¶

Upload a job's results to a GitHub repo by opening a Pull Request.

`view_job(args, job_manager)` ¶

`load_provider(provider_name)` ¶

Lazy proxy to qbraid.runtime.load_provider.

Exposed at module level so tests can monkeypatch metriq_gym.run.load_provider.

`setup_device(provider_name, device_name)` ¶

Setup a QBraid device with id device_name from specified provider.

Parameters:

Name	Type	Description	Default
`provider_name`	`str`	a metriq-gym supported provider name.	required
`device_name`	`str`	the id of a device supported by the provider.	required

Raises: QBraidSetupError: If no device matching the name is found in the provider.

metriq_gym.cli¶

Command-line interface utilities.

Command-line interface for running Metriq-Gym benchmarks using Typer.

Usage overview

Dispatch a single job: mgym job dispatch path/to/config.json -p -d
Poll latest job and write JSON results: mgym job poll latest --json results.json
Dispatch a suite of jobs: mgym suite dispatch path/to/suite.json -p -d
Poll a suite: mgym suite poll
Dry-run upload (no network): mgym job upload latest --dry-run

`job_delete(job_id=None)` ¶

Delete a job from the local database.

Note: This only removes the job from local tracking. It does not cancel jobs running on quantum hardware.

`job_dispatch(config, provider=None, device=None)` ¶

Dispatch a benchmark job to a quantum device or simulator.

`job_estimate(config, provider=None, device=None)` ¶

Estimate circuit resource requirements before dispatching jobs.

This is especially useful for understanding costs on paid hardware like Quantinuum. For Quantinuum providers, calculates H-series Quantum Credits (HQCs).

`job_main(ctx)` ¶

Show job help when mgym job is invoked without a subcommand.

`job_poll(job_id=None, json_output=None, no_cache=False, include_raw=False)` ¶

Poll job status and retrieve results when complete.

`job_replay(debug_file, json_output=None)` ¶

Replay benchmark computation from a debug file.

This allows recomputing benchmark results locally without access to the original quantum provider, using the raw measurement data captured with --include-raw.

`job_upload(job_id=None, repo='unitaryfoundation/metriq-data', base_branch='main', upload_dir=None, branch_name=None, pr_title=None, pr_body=None, commit_message=None, clone_dir=None, dry_run=False)` ¶

Upload job results to GitHub via pull request.

`job_view(job_id=None)` ¶

View job details and metadata.

`list_jobs(jobs, show_index=False, show_suite_id=True)` ¶

List jobs recorded in the job manager.

Parameters:

Name	Type	Description	Default
`jobs`	`list[MetriqGymJob]`	List of MetriqGymJob instances.	required
`show_index`	`bool`	Whether to show the job index in the output table.	`False`
`show_suite_id`	`bool`	Whether to show the suite ID column.	`True`

`main(ctx)` ¶

Show top-level help when mgym is invoked without a command.

`prompt_for_job(job_id, job_manager)` ¶

Prompt user to select a job if job_id is not provided.

Parameters:

Name	Type	Description	Default
`job_id`	`Optional[str]`	Optional job ID or 'latest'.	required
`job_manager`	`JobManager`	JobManager instance.	required

Returns:

Type	Description
`MetriqGymJob \| None`	Selected MetriqGymJob or None.

`suite_delete(suite_id=None)` ¶

Delete all jobs in a suite from the local database.

`suite_dispatch(suite_config, provider=None, device=None)` ¶

Dispatch a suite of benchmark jobs to a quantum device.

`suite_main(ctx)` ¶

Show suite help when mgym suite is invoked without a subcommand.

`suite_poll(suite_id=None, json_output=None, no_cache=False)` ¶

Poll suite jobs and retrieve results when complete.

`suite_upload(suite_id=None, repo='unitaryfoundation/metriq-data', base_branch='main', upload_dir=None, branch_name=None, pr_title=None, pr_body=None, commit_message=None, clone_dir=None, dry_run=False)` ¶

Upload suite results to GitHub via pull request.

`suite_view(suite_id=None)` ¶

View jobs in a suite.

metriq_gym.job_manager¶

Job tracking and persistence.

Local persistence and helpers for tracking dispatched metriq-gym jobs.

`JobManager` ¶

`update_job(updated_job)` ¶

Persist updated job information to disk.

`MetriqGymJob` `dataclass` ¶

`__post_init__()` ¶

Populate the canonical dataset platform without rewriting runtime identifiers.

provider_name and device_name retain the values needed to address the runtime provider and device. The corresponding platform values are stable dataset keys and may intentionally omit runtime details such as an AWS region.

metriq_gym.schema_validator¶

JSON Schema validation utilities.

Schema loading and validation utilities for metriq-gym benchmark configurations.

`create_pydantic_model(schema)` ¶

Create a Pydantic model from a JSON schema.

`load_and_validate(file_path, schema_dir=DEFAULT_SCHEMA_DIR)` ¶

Load parameters from a JSON file and validate them against the corresponding schema.

Raises a ValidationError if validation fails.

`load_json_file(file_path)` ¶

Load and parse a JSON file.

`load_schema(benchmark_name, schema_dir=DEFAULT_SCHEMA_DIR)` ¶

Load a JSON schema based on the benchmark name.

Uses package resources for installed distributions; falls back to local path.

metriq_gym.suite_parser¶

Benchmark suite parsing.

Parsing helpers for benchmark suite definitions.

`parse_suite_file(path)` ¶

Parse a suite JSON file and return a Suite object.

Key Classes¶

JobManager¶

Manages job lifecycle and persistence.

from metriq_gym.job_manager import JobManager

manager = JobManager()

# Get all jobs
jobs = manager.get_jobs()

# Get specific job
job = manager.get_job("job-id")

# Add a new job
manager.add_job(job)

# Update job status
manager.update_job(job)

MetriqGymJob¶

Represents a benchmark job.

from metriq_gym.job_manager import MetriqGymJob

job = MetriqGymJob(
    id="unique-id",
    job_type=JobType.QUANTUM_VOLUME,
    params={"num_qubits": 5, "shots": 1000},
    provider_name="ibm",
    device_name="ibm_sherbrooke",
    provider_job_ids=["remote-job-id"],
    dispatch_time="2025-01-15T12:00:00",
    app_version="0.3.1",
)

Benchmark Base Classes¶

Benchmark¶

Base class for all benchmarks.

from metriq_gym.benchmarks.benchmark import Benchmark

class MyBenchmark(Benchmark):
    def dispatch_handler(self, device):
        # Create and submit circuits
        pass

    def poll_handler(self, job_data, result_data):
        # Process results
        pass

BenchmarkResult¶

Base class for benchmark results.

from dataclasses import dataclass
from metriq_gym.benchmarks.benchmark import BenchmarkResult

@dataclass
class MyResult(BenchmarkResult):
    metric_value: float

BenchmarkScore¶

Metric with uncertainty.

from metriq_gym.benchmarks.benchmark import BenchmarkScore

score = BenchmarkScore(value=0.95, uncertainty=0.02)

Exporter Classes¶

GitHubPRExporter¶

Exports results to GitHub via pull request.

from metriq_gym.exporters.github_pr_exporter import GitHubPRExporter

exporter = GitHubPRExporter(job, result)
pr_url = exporter.export(
    repo="unitaryfoundation/metriq-data",
    base_branch="main",
    directory="results/",
)

JSONExporter¶

Exports results to local JSON file.

from metriq_gym.exporters.json_exporter import JSONExporter

exporter = JSONExporter(job, result)
exporter.export(output_path="result.json")

Constants¶

JobType¶

Enum of supported benchmark types.

from metriq_gym.constants import JobType

JobType.CLOPS           # "CLOPS"
JobType.BSEQ            # "BSEQ"
JobType.WIT             # "WIT"
# ... etc

SCHEMA_MAPPING¶

Maps JobType to schema files.

from metriq_gym.constants import SCHEMA_MAPPING

schema_file = SCHEMA_MAPPING[JobType.QUANTUM_VOLUME]
# "quantum_volume.schema.json"

Usage Examples¶

Dispatch and Poll¶

from types import SimpleNamespace
from dotenv import load_dotenv
from metriq_gym.run import dispatch_job, poll_job
from metriq_gym.job_manager import JobManager

load_dotenv()
job_manager = JobManager()

# Dispatch
dispatch_config = SimpleNamespace(
    config="metriq_gym/schemas/examples/wit.example.json",
    provider="local",
    device="aer_simulator",
)
dispatch_job(dispatch_config, job_manager)

# Poll
jobs = job_manager.get_jobs()
poll_config = SimpleNamespace(job_id=jobs[-1].id)
poll_job(poll_config, job_manager)

Custom Provider Setup¶

from metriq_gym.run import load_provider, setup_device

# Load provider
provider = load_provider("ibm")

# Get device
device = setup_device("ibm", "ibm_sherbrooke")

# List available devices
devices = provider.get_devices()
for d in devices:
    print(f"{d.id}: {d.status}")

Working with Results¶

from metriq_gym.job_manager import JobManager

manager = JobManager()
job = manager.get_job("job-id")

# Access result data
if job.result_data:
    print(job.result_data)

API Reference¶

Core Modules¶

metriq_gym.run¶

dispatch_job(args, job_manager) ¶

poll_job(args, job_manager) ¶

upload_job(args, job_manager) ¶

view_job(args, job_manager) ¶

load_provider(provider_name) ¶

setup_device(provider_name, device_name) ¶

metriq_gym.cli¶

job_delete(job_id=None) ¶

job_dispatch(config, provider=None, device=None) ¶

job_estimate(config, provider=None, device=None) ¶

job_main(ctx) ¶

job_poll(job_id=None, json_output=None, no_cache=False, include_raw=False) ¶

job_replay(debug_file, json_output=None) ¶

job_upload(job_id=None, repo='unitaryfoundation/metriq-data', base_branch='main', upload_dir=None, branch_name=None, pr_title=None, pr_body=None, commit_message=None, clone_dir=None, dry_run=False) ¶

job_view(job_id=None) ¶

list_jobs(jobs, show_index=False, show_suite_id=True) ¶

main(ctx) ¶

prompt_for_job(job_id, job_manager) ¶

suite_delete(suite_id=None) ¶

suite_dispatch(suite_config, provider=None, device=None) ¶

suite_main(ctx) ¶

suite_poll(suite_id=None, json_output=None, no_cache=False) ¶

suite_upload(suite_id=None, repo='unitaryfoundation/metriq-data', base_branch='main', upload_dir=None, branch_name=None, pr_title=None, pr_body=None, commit_message=None, clone_dir=None, dry_run=False) ¶

suite_view(suite_id=None) ¶

metriq_gym.job_manager¶

JobManager ¶

update_job(updated_job) ¶

MetriqGymJob dataclass ¶

__post_init__() ¶

metriq_gym.schema_validator¶

create_pydantic_model(schema) ¶

load_and_validate(file_path, schema_dir=DEFAULT_SCHEMA_DIR) ¶

load_json_file(file_path) ¶

load_schema(benchmark_name, schema_dir=DEFAULT_SCHEMA_DIR) ¶

metriq_gym.suite_parser¶

parse_suite_file(path) ¶

Key Classes¶

JobManager¶

MetriqGymJob¶

Benchmark Base Classes¶

Benchmark¶

BenchmarkResult¶

BenchmarkScore¶

Exporter Classes¶

GitHubPRExporter¶

JSONExporter¶

Constants¶

JobType¶

SCHEMA_MAPPING¶

Usage Examples¶

Dispatch and Poll¶

Custom Provider Setup¶

Working with Results¶

`dispatch_job(args, job_manager)` ¶

`poll_job(args, job_manager)` ¶

`upload_job(args, job_manager)` ¶

`view_job(args, job_manager)` ¶

`load_provider(provider_name)` ¶

`setup_device(provider_name, device_name)` ¶

`job_delete(job_id=None)` ¶

`job_dispatch(config, provider=None, device=None)` ¶

`job_estimate(config, provider=None, device=None)` ¶

`job_main(ctx)` ¶

`job_poll(job_id=None, json_output=None, no_cache=False, include_raw=False)` ¶

`job_replay(debug_file, json_output=None)` ¶

`job_upload(job_id=None, repo='unitaryfoundation/metriq-data', base_branch='main', upload_dir=None, branch_name=None, pr_title=None, pr_body=None, commit_message=None, clone_dir=None, dry_run=False)` ¶

`job_view(job_id=None)` ¶

`list_jobs(jobs, show_index=False, show_suite_id=True)` ¶

`main(ctx)` ¶

`prompt_for_job(job_id, job_manager)` ¶

`suite_delete(suite_id=None)` ¶

`suite_dispatch(suite_config, provider=None, device=None)` ¶

`suite_main(ctx)` ¶

`suite_poll(suite_id=None, json_output=None, no_cache=False)` ¶

`suite_upload(suite_id=None, repo='unitaryfoundation/metriq-data', base_branch='main', upload_dir=None, branch_name=None, pr_title=None, pr_body=None, commit_message=None, clone_dir=None, dry_run=False)` ¶

`suite_view(suite_id=None)` ¶

`JobManager` ¶

`update_job(updated_job)` ¶

`MetriqGymJob` `dataclass` ¶

`__post_init__()` ¶

`create_pydantic_model(schema)` ¶

`load_and_validate(file_path, schema_dir=DEFAULT_SCHEMA_DIR)` ¶

`load_json_file(file_path)` ¶

`load_schema(benchmark_name, schema_dir=DEFAULT_SCHEMA_DIR)` ¶

`parse_suite_file(path)` ¶