Adding new runners

Users usually don't want to run long-running or intensive tasks on their own computer. QA-Board will try to dispatch "batches" of runs to async task queues. Those will handle job management, fairness...

Available Runners for distributed computing

• Using the local computer with joblib
• celery
• LSF

List of possible future runner integrations

• at least one cloud-based task runner from AWS/Azure/Google?
• kubernetes to be hype!
• Jenkins used a task runner with a remote-shell job. It's ugly but common...
• ray since it's popular for ML.
• ...are you using something else? Tell us!

Adding a new runner

1. Think about the configuration options needed for your runner.

• [optional] Add CLI options for qa batch --myrunner-param in qaboard/qa.py, and merge them into job_options.
• Add the options to your project's qaboard.yaml:

runners:
  my-runner:
    option: value

2. Implement your runner in qaboard/runners/my_runner.py. It should implement a simple API:

from __future__ import annotations
from typing import Optional, List, Dict, Any

from .base import BaseRunner
from .job import Job
from ..run import RunContext 

class MyRunner(BaseRunner):
  type = "my-runner"

  def __init__(self, run_context : RunContext):
    self.run_context = run_context

  # only those 2 methods are called directly right now
  @staticmethod
  def start_jobs(jobs: List[Job], job_options: Optional[Dict[str, Any]] = None, blocking=True):
      raise NotImplementedError
  @staticmethod
  def stop_jobs(jobs: List[Job], job_options: Optional[Dict[str, Any]] = None):
      raise NotImplementedError

  def start(self, blocking=True):
    raise NotImplementedError

3. Register the runner like the existing ones in qaboard/runners/init.py
4. Happy debugging with qa batch --runner=my-runner!

Look at local.py and lsf.py for examples.

Tip: you can use environment variables like QA_RUNNER=my-runner, QA_MYRUNNER_OPTION...