Multinode

Multinode is a low-friction framework for running asynchronous tasks on the cloud. The framework:

• Provisions compute resources on demand, incurring zero costs when idle - all without you having to worry about cloud API calls or cloud permissions.
• Handles retries, timeouts, concurrency quotas, cancellations and progress monitoring
• Supports distributed tasks of arbitrary complexity, i.e. tasks that spawn parallel subtasks at runtime.
• Runs asynchronous tasks triggered by users of an application - not just offline/scheduled tasks.

Quick start

Deploy the Multinode control plane into your AWS account - see instructions here. (Or contact us if you are interested in a hosted solution.)

Install the Multinode Python package and authenticate with the Multinode control plane.

pip install multinode
multinode login

Define your asynchronous task as a Python function.

# File: tasks/main.py

from multinode import Multinode
from datetime import timedelta

mn = Multinode()

@mn.function(
    cpu=4.0,
    memory="16 GiB",
    max_retries=1,
    max_concurrency=10,
    timeout=timedelta(hours=1)
)
def run_expensive_task(x):
    out =  # ... details of the task ...
    return out

Register the function with the Multinode control plane.

multinode deploy tasks/ --project-name=my_project

Implement the rest of your application, which triggers the asynchronous task by invoking the Python function. (In this particular example, the application is a FastAPI web server.)

# File: application/main.py
# NB can be a different codebase from tasks/

from multinode import get_deployed_function
from fastapi import FastAPI

run_expensive_task = get_deployed_function(
    project_name="my_project",
    function_name="run_expensive_task"
)

app = FastAPI()

@app.post("/task_invocations")
def start_task():
    # The task will run on a *remote* cloud container (provisioned on demand)
    invocation_id = run_expensive_task.start(x=10000)
    return {"invocation_id": invocation_id}

@app.get("/task_invocations/{invocation_id}")
def get_task_status_and_result(invocation_id: str):
    invocation_data = run_expensive_task.get(invocation_id)
    return {
        "status": invocation_data.status,  # e.g. PENDING, RUNNING, SUCCEEDED, FAILED
        "result_if_complete": invocation_data.result
    }

@app.put("/task_invocations/{invocation_id}/cancel")
def cancel_task(invocation_id: str):
    run_expensive_task.cancel(invocation_id)

Advanced usage

• Client reference guide - dependencies, project structure, deployment lifecycle and more.
• An example with distributed compute - subtasks spawned dynamically at runtime.

Architecture

Currently, Multinode runs on AWS, using ECS/Fargate for the asynchronous tasks.

A (slightly simplified) architecture diagram is shown below

With minimal API changes, the framework can be extended to other AWS compute engines (e.g. EC2 with GPUs), to other cloud providers, and to Kubernetes.

We may implement these extensions if there is demand. We also welcome contributions from the open source community in this regard.

Resource provisioning - Multinode vs other solutions

Multinode's approach: Direct resource provisioning. Multinode makes direct API calls to the cloud provider, to provision a new worker for each new task.

Alternative approach: Autoscaling a warm worker pool. Popular alternative frameworks for asynchronous tasks include Celery and Kafka consumers. Applications written in these frameworks usually run on a warm pool of workers. Each worker stays alive between task executions. The number of workers is autoscaled according to some metric (e.g. the number of pending tasks).

Advantages of Multinode's approach:

• Scales up immediately when new tasks are created; scales down immediately when a task finishes.
• No risk of interrupting a task execution when scaling down.

Advantages of the alternative warm-pool-based approach:

• More suitable for processing a higher volume of shorter-lived tasks.
• Can maintain spare capacity to mitigate against cold starts.

Programming language support

Python is the only supported language at the moment.

If you need to invoke a deployed Python function from an application written in another language such as Javascript, then you will need to use the REST API. (Or you can contribute a Javascript client!)

Let us know if you want to define your functions in other languages.