Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

better support for running multiple separate brokers #85

Merged
merged 2 commits into from
Sep 17, 2024
Merged

Conversation

tpetr
Copy link
Contributor

@tpetr tpetr commented Sep 12, 2024

This PR lays the foundation for cleanly supporting multiple broker replicas for an organization. The current problems are:

  1. Running identical replicas cause the brokers to fight between one another for who "owns" the peer connection. It's possible that requests can fail during some of the ownership transition periods.
  2. Running multiple replicas with different peer addresses but identical keypairs doen't work -- wireguard expects each keypair to be unique
  3. Running multiple replicas with same peer address but different keypairs would work, but they'd still fight over which peer connection is which

The solution we're going with is to concatenate multiple keys together into one mega key, and then use a new config field (inbound.wireguard.brokerIndex in config or --broker-index on the command line) to select which offset key and peer address to connect with. Benefit here is that it means a) minimial config changes and b) is backwards compatible with old brokers. Only thing we'll need to keep in mind is that folks will have to use our key generation comments -- wg genkey / wg pubkey won't work for multiple brokers.

Tl;dr:

  1. Update genkey command to generate multiple keys concatenated together (default 3)
  2. Update pubkey command to properly handle multiple private keys concatenated together
  3. Update inbound proxy to select appropriate private key and local address depending on broker index value

There are some companion changes that need to be done in the broker gateway, but as long as nobody is using the broker index setting yet, everything should work.

@tpetr tpetr requested a review from zyannes September 12, 2024 21:23
zyannes
zyannes previously approved these changes Sep 13, 2024
cmd/pubkey.go Outdated Show resolved Hide resolved
@tpetr tpetr merged commit 66150c5 into develop Sep 17, 2024
4 checks passed
@tpetr tpetr deleted the tom/index branch September 17, 2024 17:11
tpetr added a commit that referenced this pull request Oct 7, 2024
tpetr added a commit that referenced this pull request Oct 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants