Testnet securely maintain a pool of recovery codes

[CMCDragonkai]

Once we have the ability to use PK_RECOVERY_CODE to automatically bootstrap the PK keynodes, we need to create at least 1 recovery code and 1 root key to be used.

The recovery code must be kept secret. I'll maintain this right now. The root key will be inside AWS's block device mounted into the ECS container and this will be kept safe inside AWS.

The recovery code will need to be used as an environment variable for ECS for the testnet.

Eventually we can store the recovery code inside a running Polykey node, and make use of AWS integrations, like our wiki page: "Service Deployment Secrets with AWS ECS".

Doing this should ensure that we don't need to maintain the volume state mounted in to the ECS container, it just has to be mutable, but it can be deleted, since everything can be recovered.

Tasks

1. - Use pk bootstrap locally to generate a recovery code and root key.
2. - Save the recovery code securely.
3. - Try using pk bootstrap on a different directory and see if the same root key is used. Compare them.
4. - Delete the root key.
5. - Use the recovery code for ECS Task Definition

This will be done for 1 single testnet node. We can scale this up later.

[joshuakarp]

Start date changed from Nov 11th to Nov 17th (based on delays in #231).

[joshuakarp]

Start date changed from Wednesday Nov 17th to Friday Nov 19th (delays in #269, #231, and CLI MR on Gitlab).

[joshuakarp]

Start date changed from Friday Nov 19th to Wednesday Dec 1 (delayed from refactoring work in #283).

[CMCDragonkai]

Going to bootstrap this from a password manager.

[CMCDragonkai]

This no longer blocks #194, as it can be resolved before #285 is done. Testnet nodes are still using simple passwords atm, we need to fix some bugs before setting up all the secrets in AWS for recovery codes. In particular addressing partially #403.

[CMCDragonkai]

This still needs to be done, but we need to work out partially #403, as to whether we have a set of Node IDs in PKI or as a gestalt graph or just as a set.

I prefer going down the route of the gestalt graph. But I think for efficiency sake, we still need to provide a set to the agent software and update the src/config.ts with that set. But still with the ability to discover new node IDs.

Once some level of integration testing is done, then this will become the next priority to have testnet integration tests fully passing @tegefaulkes @emmacasolin

[CMCDragonkai]

See: https://gitlab.com/MatrixAI/Engineering/Polykey/Polykey-Infrastructure/-/issues/6

[CMCDragonkai]

According to https://gitlab.com/MatrixAI/Engineering/Polykey/Polykey-Infrastructure/-/merge_requests/2#note_1145876524, we will inject our recovery codes into AWS secrets manager.

We can use this: https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-secrets-manager/index.html to automate the injection.

That means recovery code generation is done manually and also fixed. It could be done automatically if the infrastructure code had access to the recovery code generators. It would mean bringing in PK's code base and calling generateRecoveryCode. This is only possible once we release the this stable library on NPM. We'll do this after #446.

The NodeId is just the public key then.

Auto generation must be stable. For example if we expect the pool to be 10, and there's 5, then only generate 5 more. If there's 10, we don't delete anything.

Deletion is manual for now.

[CMCDragonkai]

Based on working through #403, the usage of recovery code pool should be placed into the AWS secrets manager. The naming of these secrets should be:

polykey-testnet-agent-1 -> [RecoveryCode, Password, NodeId]
polykey-testnet-agent-2 -> [RecoveryCode, Password, NodeId]

Note that NodeId can be replaced in the future, because it would be deterministically be derived from the RecoveryCode, but we can only do this practically once #446 is merged in otherwise keypair generation will take too long.

[CMCDragonkai]

The secret generation is not automated, we don't have an infrastructure orchestrator at this point in time. But it could be done as part of Polykey-Infrastructure later.

[CMCDragonkai]

@tegefaulkes please link the PR for Polykey-Infrastructure that addresses this.

[tegefaulkes]

The PR hasn't been made yet. I'll link it when it is created.

[tegefaulkes]

No PR was made, It was pushed directly to master. I'm resolving this issue.