Devnet as a Service

[lgalabru]

Phase 1: Proof of concept

clarinet integrate --k8s can spin up a devnet network on a local K8S cluster (instead of docker).

Phase 2: New stacks-devnet-api component

Architecture

• component stacks-devnet-api:
  • Configured to be ran as a pod
  • Instantiate and manage a set of networks, via a REST API (described above), by issuing k8s commands.
• component stacks-network:
  • Logic from https://github.com/hirosystems/clarinet/blob/develop/components/stacks-network/src/chains_coordinator.rs extracted, can be ran as a pod, within the devnet network.

Endpoints

Create a network

POST /api/v1/networks/

Body

{
    label: string,
    network_id: number,
    stacks_node_wait_time_for_microblocks: number,
    stacks_node_first_attempt_time_ms: number,
    stacks_node_subsequent_attempt_time_ms: number,
    bitcoin_node_username: string,
    bitcoin_node_password: string,
    miner_mnemonic: string,
    miner_derivation_path: string,
    miner_coinbase_recipient: string,
    faucet_mnemonic: string,
    faucet_derivation_path: string,
    bitcoin_controller_block_time: number,
    bitcoin_controller_automining_disabled: boolean,
    disable_bitcoin_explorer: boolean,
    disable_stacks_explorer: boolean,
    disable_stacks_api: boolean,
    epoch_2_0: number,
    epoch_2_05: number,
    epoch_2_1: number,
    epoch_2_2: number,
    pox_2_activation: number,
    pox_2_unlock_height: number,
}

• genesis accounts

Response

{
    network_uuid: "123e4567-e89b-12d3-a456-426614174000",
}

Get network informations

GET /api/v1/network/<network-uuid>

{
    status: "booting" | "running" | "paused" | "terminating",
    stacks_chain_tip: 8000,
    bitcoin_chain_tip: 10000,
    started_at: "2023-04-24T07:20:50.52Z"
}

Manage a network

POST /api/v1/network/<network-uuid>/commands/

Commands available:

{
    action: "pause" 
}

{
    action: "run",
    block_time: "10000"
}

{
    action: "mine_bitcoin_block"
}

{
    action: "mine_stacks_block"
}

HTTP Requests forwarding

POST /api/v1/network/<network-uuid>/bitcoin-node/

GET /api/v1/network/<network-uuid>/stacks-node/v2/info

GET /api/v1/network/<network-uuid>/stacks-api/extended/v1/block/by_height/1200

...

Delete a network

DELETE /api/v1/network/<network-uuid>

Note: spec is under-specified at the moment, @MicaiahReid will take the lead on this project and share his exploration with the team.
Related: https://github.com/hirosystems/artemis/issues/446

Current known steps:

• SPIKE: proof of concept for clarinet integrate running in kubernetes [3 weeks] #973
• Create pods for subnet, stacks api, and stacks/bitcoin explorers #987
• Simplify/strip out unnecessary user config settings for a hosted version (in orchestrator.rs) #988
• Create an image for the simplified orchestrator to be deployed to a k8s pod #989
• Create k8s deployment for new pods #990
• feat: allow stacks-network to be run as standalone chain coordinator #1064

[hstove]

This is fantastic. Would be really helpful for "private" testnet scenarios or even just a public "mocknet".

There is a gap between local devnet and public testnet. Public testnet is great for testing an app/protocol in as realistic an environment as possible, but it has inconsistent block times and generally isn't great for just exposing an app to testers for iterative development. Something like this would be the perfect fit.

[MicaiahReid]

Related: #973

[hugocaillard]

Should we convert to epic (and enable epics on the repo/board)? cc @smcclellan

[smcclellan]

@MicaiahReid All "known steps" in the OC are checked; can we close this? Is any remaining work ticketed?

[MicaiahReid]

@smcclellan most issues are now being tracked at the actual Stacks Devnet API repo, but there's one more PR open here on the clarinet side to merge my changes to clarinet back into master (so we'll no longer have a separate fork).

It's nearing the end of its review, then we should be able to close this issue. I've added this last PR as a task here on this issue.

[MicaiahReid]

All Clarinet-related tasks for the devnet service should be completed! Closing.