feat(quorum): integrate aws secrets manager

This PR will allow the use of the AWS service called Secrets Manager to store sensitive information, similar to how we use HashiCorp Vault for the same purpose. - A guide named "integrate-aws-secrets-manager-with-eks.md" has been introduced to help users securely connect their EKS cluster with Secrets Manager using OIDC. - The Quorum master README has been updated to guide users on how to deploy a network with AWS Secrets Manager. - A Python script has been added that contains the CRUD operation code for AWS Secrets Manager, injecting the script into the container via ConfigMap. - The Quorum Genesis and Node charts code have been updated to support Secrets Manager. fixes #2200 Signed-off-by: saurabhkumarkardam <[email protected]>
hyperledger-bevel · Sep 23, 2024 · ed7bbde · ed7bbde
1 parent 3b10f95
commit ed7bbde
Show file tree

Hide file tree

Showing 14 changed files with 704 additions and 206 deletions.
diff --git a/docs/source/concepts/integrate-aws-secrets-manager-with-eks.md b/docs/source/concepts/integrate-aws-secrets-manager-with-eks.md
@@ -0,0 +1,124 @@
+# `Integrating AWS Secrets Manager with an EKS Cluster`
+
+This guide walks you through the steps needed to create IAM policies, trust policies, and roles to securely integrate AWS Secrets Manager with your EKS cluster. By following these steps, you will enable your EKS workloads to securely access secrets stored in AWS Secrets Manager using OpenID Connect (OIDC) authentication.
+
+## Step 1: Create the `policy.json` File
+
+The `policy.json` file defines the permissions required to interact with AWS Secrets Manager. The IAM policy allows specific actions on all secrets within a region under your account.
+
+### `policy.json` content:
+
+```json
+{
+    "Version": "2012-10-17",
+    "Statement": [
+        {
+            "Effect": "Allow",
+            "Action": [
+                "secretsmanager:UntagResource",
+                "secretsmanager:DescribeSecret",
+                "secretsmanager:PutSecretValue",
+                "secretsmanager:CreateSecret",
+                "secretsmanager:DeleteSecret",
+                "secretsmanager:CancelRotateSecret",
+                "secretsmanager:ListSecretVersionIds",
+                "secretsmanager:UpdateSecret",
+                "secretsmanager:GetResourcePolicy",
+                "secretsmanager:GetSecretValue",
+                "secretsmanager:StopReplicationToReplica",
+                "secretsmanager:ReplicateSecretToRegions",
+                "secretsmanager:RestoreSecret",
+                "secretsmanager:RotateSecret",
+                "secretsmanager:UpdateSecretVersionStage",
+                "secretsmanager:RemoveRegionsFromReplication",
+                "secretsmanager:TagResource"
+            ],
+            "Resource": "arn:aws:secretsmanager:<region>:<account-id>:secret:*"
+        }
+    ]
+}
+```
+
+### Notes:
+- **Replace `<account-id>`**: Your actual AWS account ID.
+- **Replace `<region>`**: AWS region where your secret is stored.
+
+
+## Step 2: Create the Secrets Manager Policy
+
+Now that you have the `policy.json` file, create the IAM policy using the AWS CLI.
+
+### Command:
+
+```bash
+aws iam create-policy --policy-name BevelSecretsManagerAccessPolicy --policy-document file://<path-of-policy.json-file>
+```
+
+### Notes:
+- **Replace `<path-of-policy.json-file>`**: Path to the `policy.json` file created in Step 1.
+- This command creates an IAM policy named `BevelSecretsManagerAccessPolicy` with the specified permissions.
+
+
+## Step 3: Create the `trust-policy.json` file
+
+Creates a trust policy to allow role assumption through OpenID Connect (OIDC).
+
+### `trust-policy.json` content:
+
+```json
+{
+    "Version": "2012-10-17",
+    "Statement": [
+        {
+            "Effect": "Allow",
+            "Principal": {
+                "Federated": "arn:aws:iam::<account-id>:oidc-provider/<OpenID-Connect-provider-URL>"
+            },
+            "Action": "sts:AssumeRoleWithWebIdentity",
+            "Condition": {
+                "StringEquals": {
+                    "<OpenID-Connect-provider-URL>:aud": "sts.amazonaws.com"
+                }
+            }
+        }
+    ]
+}
+```
+
+### Notes:
+- **Replace `<account-id>`**: Use your AWS account ID.
+- **Replace `<OpenID-Connect-provider-URL>`**: 
+  - Insert your OpenID Connect provider URL without the `https://` prefix.
+  - For example, if your provider URL is `https://oidc.eks.eu-north-1.amazonaws.com/id/ABC1234567890`, you would use `oidc.eks.eu-north-1.amazonaws.com/id/ABC1234567890`.
+
+## Step 4: Create an IAM Role
+
+Once the trust policy is configured, create a IAM role that can assume this trust policy.
+
+### Command:
+
+```bash
+aws iam create-role --role-name BevelEKSSecretsRole --assume-role-policy-document file://<path-to-trust-policy.json>
+```
+
+### Notes:
+- **Replace `<path-to-trust-policy.json>`**: Path to the `trust-policy.json` file created in Step 3.
+- **Role Name**: The role created is named `BevelEKSSecretsRole`.
+
+This role now has the ability to assume the trust relationship defined by the OpenID Connect provider.
+
+
+## Step 5: Attach the Secrets Manager Policy to the IAM Role
+
+After creating the role, attach the previously created `BevelSecretsManagerAccessPolicy` to the role `BevelEKSSecretsRole`, granting it the necessary permissions to manage AWS Secrets.
+
+### Command:
+
+```bash
+aws iam attach-role-policy --role-name BevelEKSSecretsRole --policy-arn arn:aws:iam::<account-id>:policy/BevelSecretsManagerAccessPolicy
+```
+
+### Notes:
+- **Replace `<account-id>`**: Ensure you substitute `<account-id>` with your AWS account ID.
+- The policy **`BevelSecretsManagerAccessPolicy`** is now attached to the role **`BevelEKSSecretsRole`**, allowing it to perform actions defined in the `policy.json` file.
+---
diff --git a/platforms/quorum/charts/README.md b/platforms/quorum/charts/README.md
@@ -26,25 +26,27 @@ global:
     role: vault-role
 ```
 
-## Usage
+<hr style="border: 10px solid green;">
 
-### Pre-requisites
+## `Usage`
+
+### Pre-requisites:
 
 - Kubernetes Cluster (either Managed cloud option like EKS or local like minikube)
 - Accessible and unsealed Hahsicorp Vault (if using Vault)
 - Configured Ambassador AES (if using Ambassador as proxy)
 - Update the dependencies
-  ```
-  helm dependency update quorum-genesis
-  helm dependency update quorum-node
-  ```
+   ```
+   helm dependency update quorum-genesis
+   helm dependency update quorum-node
+   ```
 
+<hr style="border: 10px solid green;">
 
 ## `Without Proxy and Vault`
 
 ### 1. Install Genesis Node
 ```bash
-# Install the genesis node
 helm install genesis ./quorum-genesis --namespace supplychain-quo --create-namespace --values ./values/noproxy-and-novault/genesis.yaml
 ```
 
@@ -59,26 +61,68 @@ helm install validator-3 ./quorum-node --namespace supplychain-quo --values ./va
 
 ### 3. Deploy Member and Tessera Node Pair
 ```bash
-# Deploy Quorum and Tessera node pair
-helm install member-1 ./quorum-node --namespace supplychain-quo --values ./values/noproxy-and-novault/txnode.yaml
+helm install member-0 ./quorum-node --namespace supplychain-quo --values ./values/noproxy-and-novault/txnode.yaml
 ```
 
 ### Setting Up Another Member in a Different Namespace
 
 ```bash
-# Get the genesis and static nodes from existing member and and place them in the directory 'besu-genesis/files'
+# Get the genesis and static nodes from existing member and place them in the directory 'besu-genesis/files'
 cd ./quorum-genesis/files/
 kubectl --namespace supplychain-quo get configmap quorum-peers -o jsonpath='{.data.static-nodes\.json}' > static-nodes.json
 kubectl --namespace supplychain-quo get configmap quorum-genesis  -o jsonpath='{.data.genesis\.json}' > genesis.json
 
 # Install secondary genesis node
-helm install genesis ./quorum-genesis --namespace carrier-quo --values ./values/noproxy-and-novault/genesis-sec.yaml
+helm install genesis ./quorum-genesis --namespace carrier-quo --create-namespace --values ./values/noproxy-and-novault/genesis-sec.yaml
 
 # Install secondary member node
-helm install member-2 ./quorum-node --namespace carrier-quo --values ./values/noproxy-and-novault/txnode-sec.yaml
+helm install member-1 ./quorum-node --namespace carrier-quo --values ./values/noproxy-and-novault/txnode-sec.yaml
+```
+
+<hr style="border: 10px solid green;">
+
+## `Without Proxy and AWS-Secret-Manager`
+
+### 1. Prerequisite:
+- To securely integrate AWS Secrets Manager with an EKS Cluster, refer to the guide available [here.](../../../docs/source/concepts/integrate-aws-secrets-manager-with-eks.md)
+- After completing all the steps mentioned in the guide, keep the IAM Role ARN handy. For example, it should look something like this: `arn:aws:iam::<account-id>:role/BevelEKSSecretsRole`
+
+### 2. Install genesis node:
+```bash
+helm install genesis ./quorum-genesis --namespace supplychain-quo --create-namespace --values ./values/noproxy-and-novault/genesis.yaml --set global.cluster.cloudNativeServices=true,global.cluster.secretManagerArn="<YOUR_AWS_SECRET_MANAGER_ROLE_ARN>",global.cluster.secretManagerRegion="<YOUR_AWS_REGION>"
+```
+
+### 3. Install validator nodes:
+```bash
+helm install validator-0 ./quorum-node --namespace supplychain-quo --values ./values/noproxy-and-novault/validator.yaml --set global.cluster.cloudNativeServices=true,global.cluster.secretManagerRegion="<YOUR_AWS_REGION>"
+helm install validator-1 ./quorum-node --namespace supplychain-quo --values ./values/noproxy-and-novault/validator.yaml --set global.cluster.cloudNativeServices=true,global.cluster.secretManagerRegion="<YOUR_AWS_REGION>"
+helm install validator-2 ./quorum-node --namespace supplychain-quo --values ./values/noproxy-and-novault/validator.yaml --set global.cluster.cloudNativeServices=true,global.cluster.secretManagerRegion="<YOUR_AWS_REGION>"
+helm install validator-3 ./quorum-node --namespace supplychain-quo --values ./values/noproxy-and-novault/validator.yaml --set global.cluster.cloudNativeServices=true,global.cluster.secretManagerRegion="<YOUR_AWS_REGION>"
+```
+
+### 4. Install member node:
+```bash
+helm install member-0 ./quorum-node --namespace supplychain-quo --values ./values/noproxy-and-novault/txnode.yaml --set global.cluster.cloudNativeServices=true,global.cluster.secretManagerRegion="<YOUR_AWS_REGION>",tessera.enabled=false
 ```
 
----
+### 5. Setting Up Another Member in a Different Namespace
+
+```bash
+# 5.1. Get the genesis and static nodes from existing member and place them in the directory 'besu-genesis/files'
+cd ./quorum-genesis/files/
+
+kubectl --namespace supplychain-quo get configmap quorum-peers -o jsonpath='{.data.static-nodes\.json}' > static-nodes.json
+
+kubectl --namespace supplychain-quo get configmap quorum-genesis  -o jsonpath='{.data.genesis\.json}' > genesis.json
+
+# 5.2. Install secondary genesis node
+helm install genesis ./quorum-genesis --namespace carrier-quo --create-namespace --values ./values/noproxy-and-novault/genesis-sec.yaml --set global.cluster.cloudNativeServices=true,global.cluster.secretManagerArn="<YOUR_AWS_SECRET_MANAGER_ROLE_ARN>"
+
+# 5.3. Install secondary member node
+helm install member-1 ./quorum-node --namespace carrier-quo --values ./values/noproxy-and-novault/txnode-sec.yaml --set global.cluster.cloudNativeServices=true,global.cluster.secretManagerRegion="<YOUR_AWS_REGION>",tessera.enabled=false
+```
+
+<hr style="border: 10px solid green;">
 
 ## `With Ambassador Proxy and Vault`
 
@@ -127,12 +171,14 @@ kubectl create namespace carrier-quo
 kubectl -n carrier-quo create secret generic roottoken --from-literal=token=<VAULT_ROOT_TOKEN>
 
 # Install secondary genesis node
-helm install genesis ./quorum-genesis --namespace carrier-quo --values ./values/proxy-and-vault/genesis-sec.yaml
+helm install genesis ./quorum-genesis --namespace carrier-quo --create-namespace --values ./values/proxy-and-vault/genesis-sec.yaml
 
 # Install secondary member node
 helm install member-0 ./quorum-node --namespace carrier-quo --values ./values/proxy-and-vault/txnode-sec.yaml --set global.proxy.p2p=15016
 ```
 
+<hr style="border: 10px solid green;">
+
 ## `API call`
 
 Once your services are deployed, they can be accessed using the domain name provided in your `global.proxy.externalUrlSuffix`.
@@ -171,6 +217,8 @@ Once your services are deployed, they can be accessed using the domain name prov
 
    This confirms that your node is syncing as expected.
 
+<hr style="border: 10px solid green;">
+
 ## `Managing IBFT Validators Deployment`
 
 To deploy the proposed validator chart for IBFT, you first need to set up the Quorum DLT network. Below are the steps you can follow:
@@ -203,6 +251,7 @@ To deploy the proposed validator chart for IBFT, you first need to set up the Qu
 
 	Replace `<SOURCE-HOST>` with the appropriate host address.
 
+<hr style="border: 10px solid green;">
 
 ## `Clean-up`