Developer Guide¶

This guide covers setting up a development environment for contributing to Metriq-Gym.

Prerequisites¶

Before you begin, ensure you have:

Python (version 3.12 or newer)
uv for dependency management
Git for version control

Getting Started¶

Clone the Repository¶

Clone with submodules:

git clone --recurse-submodules https://github.com/unitaryfoundation/metriq-gym.git
cd metriq-gym

If you already have a clone, update it:

git pull --recurse-submodules

Install Dependencies¶

Install all dependencies including dev tools:

uv sync --all-groups

This creates a virtual environment in .venv and installs all dependencies.

Activate Environment¶

Either activate the virtual environment:

source .venv/bin/activate

Or use uv run for isolated commands:

uv run pytest

macOS Note¶

macOS users installing optional pyqpanda3 support must install libidn2:

brew reinstall libidn2

Install this before running uv sync to avoid build errors.

Development Workflow¶

Pre-commit Hooks¶

Install pre-commit hooks after cloning:

uv run pre-commit install

This runs linting and formatting automatically on each commit.

Running Tests¶

# All tests
uv run pytest

# Unit tests only
uv run pytest -m "not e2e"

# End-to-end tests only
uv run pytest -m e2e

# Specific test file
uv run pytest tests/test_benchmarks.py

Linting and Formatting¶

# Run ruff linter
uv run ruff check .

# Run ruff formatter
uv run ruff format .

# Check types with mypy
uv run mypy

Building Documentation¶

cd docs
uv run mkdocs serve

Open http://127.0.0.1:8000 to view the documentation locally.

Project Structure¶

metriq-gym/
├── metriq_gym/
│   ├── benchmarks/      # Benchmark implementations
│   ├── exporters/       # Result export (JSON, GitHub PR)
│   ├── local/           # Local simulator provider
│   ├── qplatform/       # Provider-specific device/job helpers
│   ├── schemas/         # JSON schemas and examples
│   ├── cli.py           # CLI argument parsing
│   ├── constants.py     # JobType enum, schema mapping
│   ├── job_manager.py   # Job tracking
│   ├── registry.py      # Benchmark registration
│   └── run.py           # Main entrypoint
├── tests/               # Test suite
├── docs/                # Documentation
├── submodules/          # External dependencies
└── pyproject.toml       # Project configuration

Contributing¶

Contribution Workflow¶

Fork the repository
Create a feature branch
Make your changes
Run tests and linting
Submit a pull request

Commit Style¶

Follow Conventional Commits:

feat: add support for new benchmark type
fix: correct BSEQ calculation
docs: update provider setup guide
refactor: simplify job manager interface

Pull Request Guidelines¶

Rebase onto the latest main before opening a PR
Link related issues or discussions
Include CLI output or screenshots for user-facing changes
Ensure all tests pass
Get approval from maintainers

Code Style¶

The project uses: - Ruff for linting and formatting - mypy for type checking - Line length: 100 characters

Type Annotations¶

Use type annotations for all public functions:

def dispatch_job(config: SimpleNamespace, job_manager: JobManager) -> str:
    """Dispatch a benchmark job.

    Args:
        config: Job configuration with provider, device, and config path
        job_manager: Job tracking instance

    Returns:
        The metriq-gym job ID
    """
    ...

Testing¶

Test Categories¶

Marker	Description
(none)	Unit tests (fast, no external dependencies)
`e2e`	End-to-end tests (may require credentials)

Writing Tests¶

import pytest
from metriq_gym.benchmarks.wit import WIT

def test_wit_dispatch():
    """Test WIT benchmark dispatch creates correct circuits."""
    benchmark = WIT(config)
    result = benchmark.dispatch_handler(mock_device)
    assert result.circuits is not None

@pytest.mark.e2e
def test_wit_full_workflow():
    """Test full WIT workflow on simulator."""
    # This test requires local simulator
    ...

Debugging¶

Verbose Logging¶

Enable debug logging:

import logging
logging.basicConfig(level=logging.DEBUG)

Local Testing¶

Test against the local simulator:

mgym job dispatch metriq_gym/schemas/examples/wit.example.json \
    -p local -d aer_simulator

Release Process¶

Releases are managed by maintainers:

Version is determined by setuptools_scm from git tags
CI builds and tests the wheel and source distribution before publishing to PyPI
Documentation is deployed to GitHub Pages

Distribution checks¶

The Distribution Tests workflow runs on pull requests and pushes to main. Both publishing workflows also call it: a failed build, metadata check, install, or simulator smoke test prevents publishing. Release builds must have the same version as their tag after Python version normalization (for example, v0.5.1-alpha becomes 0.5.1a0). PyPI and TestPyPI receive the exact artifacts that passed these checks.

The wheel is installed in fresh environments on Linux and macOS with Python 3.12 and 3.13. The source distribution is independently installed on Linux with Python 3.12. These installs resolve the dependencies declared in the package, without the development environment or uv.lock.

The smoke script runs outside the checkout and checks the installed version, CLI entry point, JSON schemas, bundled suites, dashboard HTML, and benchmark modules from the submodules. It then dispatches and polls a small local Aer job and a two-job suite, checking their JSON results. No provider credentials are needed.

To reproduce a wheel check locally on Linux or macOS:

uv build
smoke_dir=$(mktemp -d)
uv venv "$smoke_dir/venv" --python 3.12
uv pip install --python "$smoke_dir/venv/bin/python" dist/*.whl
uv pip check --python "$smoke_dir/venv/bin/python"
cp scripts/smoke_distribution.py "$smoke_dir/"
(
    cd "$smoke_dir"
    unset PYTHONPATH
    ./venv/bin/python -I smoke_distribution.py
)

Repeat with a fresh environment and dist/*.tar.gz to check the source distribution. OpenCL must be installed, as for the regular test workflow.

Tag naming requirement¶

Release tags must follow the v<digit> scheme, i.e. v<major>.<minor>.<patch> with an optional pre-release suffix (e.g. v0.7.1, v0.8.0rc1). setuptools_scm is configured to only consider tags matching the glob v[0-9]* when deriving the package version, so:

A release cut from a conforming tag builds a clean version (e.g. 0.7.1) that PyPI accepts.
A release cut from a non-conforming tag (e.g. release-1.0) would fall back to the nearest conforming tag and produce a dev version with a +g<hash> local segment, which PyPI rejects on upload.

Tags that are not releases (e.g. metriq-paper-v1) must not match v[0-9]*; they are ignored by versioning, so they can be created freely.