API Reference¶

This page documents the main Python modules in Metriq-Gym.

Core Modules¶

metriq_gym.run¶

Main runtime module with dispatch, poll, and upload functions.

Runtime entrypoints for dispatching and managing metriq-gym benchmarks via the CLI.

`dispatch_job(args, job_manager)` ¶

Dispatch a single benchmark configuration to a quantum device.

Parameters:

Name	Type	Description	Default
`args`	`Namespace`	CLI arguments with benchmark config, provider, and device	required
`job_manager`	`JobManager`	Tracks dispatched jobs for later polling	required

Note: Continues processing remaining configs if individual configs fail.

`poll_job(args, job_manager)` ¶

`upload_job(args, job_manager)` ¶

Upload a job's results to a GitHub repo by opening a Pull Request.

`view_job(args, job_manager)` ¶

`load_provider(provider_name)` ¶

Lazy proxy to qbraid.runtime.load_provider.

Exposed at module level so tests can monkeypatch metriq_gym.run.load_provider.

`setup_device(provider_name, device_name)` ¶

Setup a QBraid device with id device_name from specified provider.

Parameters:

Name	Type	Description	Default
`provider_name`	`str`	a metriq-gym supported provider name.	required
`device_name`	`str`	the id of a device supported by the provider.	required

Raises: QBraidSetupError: If no device matching the name is found in the provider.

metriq_gym.cli¶

Command-line interface utilities.

Command-line interface for running Metriq-Gym benchmarks using Typer.

Usage overview

Dispatch a single job: mgym job dispatch path/to/config.json -p -d
Poll latest job and write JSON results: mgym job poll latest --json results.json
Dispatch a suite of jobs: mgym suite dispatch path/to/suite.json -p -d
Poll a suite: mgym suite poll
Dry-run upload (no network): mgym job upload latest --dry-run

`job_delete(job_id=None)` ¶

Delete a job from the local database.

Note: This only removes the job from local tracking. It does not cancel jobs running on quantum hardware.

`job_dispatch(config, provider=None, device=None)` ¶

Dispatch a benchmark job to a quantum device or simulator.

`job_estimate(config, provider=None, device=None)` ¶

Estimate circuit resource requirements before dispatching jobs.

This is especially useful for understanding costs on paid hardware like Quantinuum. For Quantinuum providers, calculates H-series Quantum Credits (HQCs).

`job_poll(job_id=None, json_output=None, no_cache=False, include_raw=False)` ¶

Poll job status and retrieve results when complete.

`job_replay(debug_file, json_output=None)` ¶

Replay benchmark computation from a debug file.

This allows recomputing benchmark results locally without access to the original quantum provider, using the raw measurement data captured with --include-raw.

`job_upload(job_id=None, repo='unitaryfoundation/metriq-data', base_branch='main', upload_dir=None, branch_name=None, pr_title=None, pr_body=None, commit_message=None, clone_dir=None, dry_run=False)` ¶

Upload job results to GitHub via pull request.

`job_view(job_id=None)` ¶

View job details and metadata.

`list_jobs(jobs, show_index=False, show_suite_id=True)` ¶

List jobs recorded in the job manager.

Parameters:

Name	Type	Description	Default
`jobs`	`list[MetriqGymJob]`	List of MetriqGymJob instances.	required
`show_index`	`bool`	Whether to show the job index in the output table.	`False`
`show_suite_id`	`bool`	Whether to show the suite ID column.	`True`

`prompt_for_job(job_id, job_manager)` ¶

Prompt user to select a job if job_id is not provided.

Parameters:

Name	Type	Description	Default
`job_id`	`Optional[str]`	Optional job ID or 'latest'.	required
`job_manager`	`JobManager`	JobManager instance.	required

Returns:

Type	Description
`MetriqGymJob \| None`	Selected MetriqGymJob or None.

`suite_delete(suite_id=None)` ¶

Delete all jobs in a suite from the local database.

`suite_dispatch(suite_config, provider=None, device=None)` ¶

Dispatch a suite of benchmark jobs to a quantum device.

`suite_poll(suite_id=None, json_output=None, no_cache=False)` ¶

Poll suite jobs and retrieve results when complete.

`suite_upload(suite_id=None, repo='unitaryfoundation/metriq-data', base_branch='main', upload_dir=None, branch_name=None, pr_title=None, pr_body=None, commit_message=None, clone_dir=None, dry_run=False)` ¶

Upload suite results to GitHub via pull request.

`suite_view(suite_id=None)` ¶

View jobs in a suite.

metriq_gym.job_manager¶

Job tracking and persistence.

Local persistence and helpers for tracking dispatched metriq-gym jobs.

`JobManager` ¶

`update_job(updated_job)` ¶

Persist updated job information to disk.

`MetriqGymJob` `dataclass` ¶

`__post_init__()` ¶

Keep platform and provider/device fields in sync on initialization.

If platform is missing, populate from provider_name/device_name.
If platform exists but lacks keys, backfill them from provider/device fields.

metriq_gym.schema_validator¶

JSON Schema validation utilities.

Schema loading and validation utilities for metriq-gym benchmark configurations.

`create_pydantic_model(schema)` ¶

Create a Pydantic model from a JSON schema.

`load_and_validate(file_path, schema_dir=DEFAULT_SCHEMA_DIR)` ¶

Load parameters from a JSON file and validate them against the corresponding schema.

Raises a ValidationError if validation fails.

`load_json_file(file_path)` ¶

Load and parse a JSON file.

`load_schema(benchmark_name, schema_dir=DEFAULT_SCHEMA_DIR)` ¶

Load a JSON schema based on the benchmark name.

Uses package resources for installed distributions; falls back to local path.

metriq_gym.suite_parser¶

Benchmark suite parsing.

Parsing helpers for benchmark suite definitions.

`parse_suite_file(path)` ¶

Parse a suite JSON file and return a Suite object.

Key Classes¶

JobManager¶

Manages job lifecycle and persistence.

from metriq_gym.job_manager import JobManager

manager = JobManager()

# Get all jobs
jobs = manager.get_jobs()

# Get specific job
job = manager.get_job("job-id")

# Add a new job
manager.add_job(job)

# Update job status
manager.update_job(job)

MetriqGymJob¶

Represents a benchmark job.

from metriq_gym.job_manager import MetriqGymJob

job = MetriqGymJob(
    id="unique-id",
    job_type=JobType.QUANTUM_VOLUME,
    params={"num_qubits": 5, "shots": 1000},
    provider_name="ibm",
    device_name="ibm_sherbrooke",
    provider_job_ids=["remote-job-id"],
    dispatch_time="2025-01-15T12:00:00",
    app_version="0.3.1",
)

Benchmark Base Classes¶

Benchmark¶

Base class for all benchmarks.

from metriq_gym.benchmarks.benchmark import Benchmark

class MyBenchmark(Benchmark):
    def dispatch_handler(self, device):
        # Create and submit circuits
        pass

    def poll_handler(self, job_data, result_data):
        # Process results
        pass

BenchmarkResult¶

Base class for benchmark results.

from dataclasses import dataclass
from metriq_gym.benchmarks.benchmark import BenchmarkResult

@dataclass
class MyResult(BenchmarkResult):
    metric_value: float

BenchmarkScore¶

Metric with uncertainty.

from metriq_gym.benchmarks.benchmark import BenchmarkScore

score = BenchmarkScore(value=0.95, uncertainty=0.02)

Exporter Classes¶

GitHubPRExporter¶

Exports results to GitHub via pull request.

from metriq_gym.exporters.github_pr_exporter import GitHubPRExporter

exporter = GitHubPRExporter(job, result)
pr_url = exporter.export(
    repo="unitaryfoundation/metriq-data",
    base_branch="main",
    directory="results/",
)

JSONExporter¶

Exports results to local JSON file.

from metriq_gym.exporters.json_exporter import JSONExporter

exporter = JSONExporter(job, result)
exporter.export(output_path="result.json")

Constants¶

JobType¶

Enum of supported benchmark types.

from metriq_gym.constants import JobType

JobType.CLOPS           # "CLOPS"
JobType.BSEQ            # "BSEQ"
JobType.WIT             # "WIT"
# ... etc

SCHEMA_MAPPING¶

Maps JobType to schema files.

from metriq_gym.constants import SCHEMA_MAPPING

schema_file = SCHEMA_MAPPING[JobType.QUANTUM_VOLUME]
# "quantum_volume.schema.json"

Usage Examples¶

Dispatch and Poll¶

from types import SimpleNamespace
from dotenv import load_dotenv
from metriq_gym.run import dispatch_job, poll_job
from metriq_gym.job_manager import JobManager

load_dotenv()
job_manager = JobManager()

# Dispatch
dispatch_config = SimpleNamespace(
    config="metriq_gym/schemas/examples/wit.example.json",
    provider="local",
    device="aer_simulator",
)
dispatch_job(dispatch_config, job_manager)

# Poll
jobs = job_manager.get_jobs()
poll_config = SimpleNamespace(job_id=jobs[-1].id)
poll_job(poll_config, job_manager)

Custom Provider Setup¶

from metriq_gym.run import load_provider, setup_device

# Load provider
provider = load_provider("ibm")

# Get device
device = setup_device("ibm", "ibm_sherbrooke")

# List available devices
devices = provider.get_devices()
for d in devices:
    print(f"{d.id}: {d.status}")

Working with Results¶

from metriq_gym.job_manager import JobManager

manager = JobManager()
job = manager.get_job("job-id")

# Access result data
if job.result_data:
    print(job.result_data)

API Reference¶

Core Modules¶

metriq_gym.run¶

dispatch_job(args, job_manager) ¶

poll_job(args, job_manager) ¶

upload_job(args, job_manager) ¶

view_job(args, job_manager) ¶

load_provider(provider_name) ¶

setup_device(provider_name, device_name) ¶

metriq_gym.cli¶

job_delete(job_id=None) ¶

job_dispatch(config, provider=None, device=None) ¶

job_estimate(config, provider=None, device=None) ¶

job_poll(job_id=None, json_output=None, no_cache=False, include_raw=False) ¶

job_replay(debug_file, json_output=None) ¶

job_upload(job_id=None, repo='unitaryfoundation/metriq-data', base_branch='main', upload_dir=None, branch_name=None, pr_title=None, pr_body=None, commit_message=None, clone_dir=None, dry_run=False) ¶

job_view(job_id=None) ¶

list_jobs(jobs, show_index=False, show_suite_id=True) ¶

prompt_for_job(job_id, job_manager) ¶

suite_delete(suite_id=None) ¶

suite_dispatch(suite_config, provider=None, device=None) ¶

suite_poll(suite_id=None, json_output=None, no_cache=False) ¶

suite_upload(suite_id=None, repo='unitaryfoundation/metriq-data', base_branch='main', upload_dir=None, branch_name=None, pr_title=None, pr_body=None, commit_message=None, clone_dir=None, dry_run=False) ¶

suite_view(suite_id=None) ¶

metriq_gym.job_manager¶

JobManager ¶

update_job(updated_job) ¶

MetriqGymJob dataclass ¶

__post_init__() ¶

metriq_gym.schema_validator¶

create_pydantic_model(schema) ¶

load_and_validate(file_path, schema_dir=DEFAULT_SCHEMA_DIR) ¶

load_json_file(file_path) ¶

load_schema(benchmark_name, schema_dir=DEFAULT_SCHEMA_DIR) ¶

metriq_gym.suite_parser¶

parse_suite_file(path) ¶

Key Classes¶

JobManager¶

MetriqGymJob¶

Benchmark Base Classes¶

Benchmark¶

BenchmarkResult¶

BenchmarkScore¶

Exporter Classes¶

GitHubPRExporter¶

JSONExporter¶

Constants¶

JobType¶

SCHEMA_MAPPING¶

Usage Examples¶

Dispatch and Poll¶

Custom Provider Setup¶

Working with Results¶

`dispatch_job(args, job_manager)` ¶

`poll_job(args, job_manager)` ¶

`upload_job(args, job_manager)` ¶

`view_job(args, job_manager)` ¶

`load_provider(provider_name)` ¶

`setup_device(provider_name, device_name)` ¶

`job_delete(job_id=None)` ¶

`job_dispatch(config, provider=None, device=None)` ¶

`job_estimate(config, provider=None, device=None)` ¶

`job_poll(job_id=None, json_output=None, no_cache=False, include_raw=False)` ¶

`job_replay(debug_file, json_output=None)` ¶

`job_upload(job_id=None, repo='unitaryfoundation/metriq-data', base_branch='main', upload_dir=None, branch_name=None, pr_title=None, pr_body=None, commit_message=None, clone_dir=None, dry_run=False)` ¶

`job_view(job_id=None)` ¶

`list_jobs(jobs, show_index=False, show_suite_id=True)` ¶

`prompt_for_job(job_id, job_manager)` ¶

`suite_delete(suite_id=None)` ¶

`suite_dispatch(suite_config, provider=None, device=None)` ¶

`suite_poll(suite_id=None, json_output=None, no_cache=False)` ¶

`suite_upload(suite_id=None, repo='unitaryfoundation/metriq-data', base_branch='main', upload_dir=None, branch_name=None, pr_title=None, pr_body=None, commit_message=None, clone_dir=None, dry_run=False)` ¶

`suite_view(suite_id=None)` ¶

`JobManager` ¶

`update_job(updated_job)` ¶

`MetriqGymJob` `dataclass` ¶

`__post_init__()` ¶

`create_pydantic_model(schema)` ¶

`load_and_validate(file_path, schema_dir=DEFAULT_SCHEMA_DIR)` ¶

`load_json_file(file_path)` ¶

`load_schema(benchmark_name, schema_dir=DEFAULT_SCHEMA_DIR)` ¶

`parse_suite_file(path)` ¶