GH Actions Workflow for AWS Authentication (#54)

* Test push

:)

* Init commit for main.yml + email-config.py

main.yml will authenticate AWS with our IAM role, use a dependency to get the name of the latest file pushed to configs directory, and run email-config.py.

Pushing email-config.py as it was in the last commit to my  nataliejschultz:AWS-email-config PR.

Reverting README.md to before test push.

* configs push test

Testing to see if pushing to the configs directory runs the main.yml workflow!

* Updating main.yml

main.yml didn't run on my last push (though I thought it would). Updating when the workflow runs and trying again.

* Another test push

Trying to get workflow to run. If this doesn't work, it might be because my PR is a draft? I've seen conflicting info about how to get a workflow to run on a non-main branch in PR phase.

* Region error

trying to fix `Error: Region is not valid: <"us-west-2"> `

* Push test

Changing file to trigger workflow.

* Fixing typo

Typo was causing secrets access issue.

* Push test

* Changing ARN syntax + push test

Hoping that previous run is just a syntax error that I'm correcting now.

* Changing option in run + push test

`m` is not an option for the command, so I added a different one.

* Creating clients + push test

I'm not sure how to set up the client when authenticating through IAM. Trying this out.

* Modifying cognito client + push test

Adding region to cognito client + moving where environment variable is accessed.

* Combining jobs to pass credentials

I don't think the jobs can be separated if I want to pass the credentials from my AWS auth step to the run step. Going to try combining them and see what happens!

* Fixing dependency

I don't think this dependency worked because of where `id` was in the workflow. Trying again!

* Testing TJ dependency

Testing out another dependency to see if it's better than jitterbit :)

* Push test

Testing out TJ dependency with this push!

* Config file export

Modified the `changed_files` job to include a for loop that exports an environment variable with the name of the changed file. The variable is then passed in while running the python script. Hoping this works!

* Push test

* Typo

Forgot to change value of file from changed_file. Oops.

* Push test

* Adding prints

Adding some prints for troubleshooting to see if the name of the config file is being passed properly or not.

* Push test

* Trying out another way to access config file

Accessing environment variable (hopefully) the right way.

* Trying to get jobs to run sequentially

* Another way to pass config file name

Trying another way to pass the config file name between the two jobs.

* Prints

prints to figure out why the filepath isn't correct

* Another print

Adding another print.

* Config relative file path change

The relative path is slightly different in github actions. Adding an if else line to change depending on -l or -g arg

* Using full path to file

Relative paths seem to have an issue in GHA. trying full path to file.

* Push test after updated settings

Last error was due to AWS permissions not working. Jianli updated the settings, so we're going to check and see if it works now.

* Push test

Trying again per jianli

* Updating config path for -g + removing prints

Last run worked! I removed myself from the user pool, so I'm going to try again with this updated method of getting the config path for -g runs + trying to actually add myself to the user pool.

* Adding welcome template

Forgot that I needed to add the welcome template file! Adding :)

* File not found?

Got `FileNotFoundError: [Errno 2] No such file or directory: 'welcome-template.txt'` on last run. Pushing a change to make sure the file is there and going to try again.

* Push test

* Adding pwd

Added os.getcwd() to see why it's not opening the welcome template file.

* welcome template access

adding a workaround for the filepath issue

* Filepaths again

Trying another fix.

* Previous issue fixed. Now sts client

Adding an empty defintion for sts_client with -g since it has to be passed into the get_verified_arn function.

* Push test

Testing now that the AWS settings have been updated !

* Updating script per changes  +emoving changes to readme

Updating email-config so it's up to date with the other email-config.py in the other PR.

Removing the PR's changes to the Readme (not sure how they got on there)

* Readme

Reverting readme?

* Updating email-config.py

Updating to be up-to-date with latest push in the [other PR](#45)

* Renaming workflow

Giving main.yml a more meaningful name (AWS-auth.yml)

* Push test

testing workflow under new name before removing email :)

* Removing email

Removing email address from wyoming file. Workflow will probably raise an error on this run.

* Trying to fix readme changes

* Trying to remove all changes to readme

I tried git checkout, but that didn't give me a pushable option to remove my previous commit changes. Let's see how this does :)

* Removing duplicate files

Since the final test worked fine, we can remove them from this PR.  They should be merged on the other PR first, and then this PR can be closed.

* rename for demo

* re-adding files for demo

* adding wyoming config file

* Email parsing fix

After demoing with Abby, we found out that the emails weren't being parsed correctly due to differences in our previous configs vs the new generation method. Fixing and testing!

* Delete wyoming.nrel-op.json

* Add wyoming config

* Restoring wyoming original config

Restored the old wyoming config + removing files that were temporarily re-added for the demo

* What happens when modifying two configs in one push?

Trying to see what happens when we update two config files at once. Will the value of the github output be formatted as a string? Will it be a comma separated list?

* echo multiple filename change output

Checking output when we change multiple configs at once

* push test

* Push test

Trying to activate the GH actions workflow

* Changing bash filename handling

Modifying the GHActions script to (potentially) handle multiple changed config files.

* Array appending debugging

Array did not work for bash. Checking out it's being output.

* Array formatting

Modifying array formatting so it's hopefully what I want! Also seeing if I can read the actual output, though it might produce a context error.

* Moving things around

moving some echoes.

* Array syntax

Seeing if the array will echo properly with this syntax change.

* Testing passing two changed files

Modified the GH Actions to be able to pass in and loop over two config files. We'll see if it works by temporarily adding in email-config.py with a relevant print and seeing what happens!

* Syntax?

I'm not sure what went wrong the first time with bash. Adding some echoes to figure it out.

* More subtle bash syntax

As it would turn out, bash scripting is finicky when dealing with arrays. My array is not being sent to the GitHub output properly for some reason. Trying it out by replacing * with @. If that doesn't work, I'm going to try removing the entire [@] section and see what happens.

* More syntax, whoops!

I changed CONFIG_FILE to CONFIG_FILES at one point and didn't change it in the outputs section. Now let's see if it works!

* Bug catching

Caught a bug in my email-config.py script where it wasn't returning is_userpool_exist properly. Let's see if this fixes things.

* Commenting out

Want to merge recent changes to my branch, but don't want to re-run all these configs!

* renaming email config

renaming to merge from main.

* email-config.py bug fix

Fixing a bug I found a few pushes ago.

* Two changed files, removal test

Testing removal of my email from the wyoming pool, in addition to adding myself to nrel-commute pool in the same action.

* See last commit. Running job

Job wasn't picked up on last push. Trying again.

* action run only on push to main

changing branch that workflow runs on push to (main)

* Reverting wyoming config

Reverting Wyoming config file

* WY format

Continuing reverting WY file

* syntax

blank space removal

* Update wyoming.nrel-op.json

Adding blank line after file

* Reverting commute config

Reverting to original commute config

.github/workflows/AWS-auth.yml

@@ -0,0 +1,77 @@
+name: AWS Auth + Welcome Email
+
+on:
+  push:
+    branches: [ main ]
+    paths:
+        'configs/**.json'
+
+env:
+  AWS_ACCT_ID: ${{  secrets.AWS_ACCT_ID  }}
+  AWS_REGION : 'us-west-2'
+  IAM_ROLE: ${{  secrets.ROLE_NAME  }}
+
+permissions:
+      id-token: write
+      contents: read
+jobs:
+  changed_files:
+    runs-on: ubuntu-latest  # windows-latest || macos-latest
+    name: Get config file name
+    outputs:
+      config-file-name: ${{ steps.config-file-name.outputs.CONFIG_FILES}}
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0  # OR "2" -> To retrieve the preceding commit.
+
+      - name: Get changed files
+        id: get-changed-files
+        uses: tj-actions/changed-files@v40
+
+      # NOTE: `since_last_remote_commit: true` is implied by default and falls back to the previous local commit.
+
+      - name: List all changed files
+        id: config-file-name
+        run: |
+            echo ${{ steps.get-changed-files.outputs.all_changed_files }}
+            changedfiles=()
+            for file in ${{ steps.get-changed-files.outputs.all_changed_files }}; do
+              if [[ "$file" == *nrel-op.json ]]; then
+              changedfiles+=("${file}")
+              echo "The name of the config file is: ${file}."
+              fi
+            done
+            echo "final changedfiles array: ${changedfiles[*]}"
+
+            echo "CONFIG_FILES=${changedfiles[*]}" >> "$GITHUB_OUTPUT"
+
+  AssumeRoleAndCallIdentity:
+    name: AWS Authentication + Sending Welcome Email
+    needs: changed_files
+    if: always()
+    runs-on: ubuntu-latest
+    steps:
+      - name: Git clone the repository
+        uses: actions/checkout@v3
+      - name: configure aws credentials
+        uses: aws-actions/[email protected]
+        with:
+          role-to-assume: arn:aws:iam::${{ env.AWS_ACCT_ID }}:role/${{ env.IAM_ROLE }}
+          role-session-name: GitHub_to_AWS_via_FederatedOIDC
+          aws-region: ${{ env.AWS_REGION }}
+      # Hello from AWS: WhoAmI
+      - name: Sts GetCallerIdentity
+        run: |
+          aws sts get-caller-identity --debug
+
+      - name: Install Boto3
+        run: pip install boto3
+
+      - name: Run email-config.py
+        run: |
+          echo "changed files string: ${{ needs.changed_files.outputs.config-file-name }}"
+          for config_file in ${{ needs.changed_files.outputs.config-file-name }}; do 
+            echo "config file name ${config_file}" 
+            python email_automation/email-config.py -g ${config_file}
+          done

email_automation/email-config.py

@@ -59,7 +59,7 @@ def get_userpool_name(pool_name, cognito_client):
     user_pools = [user_pool["Name"] for user_pool in response["UserPools"]]
     is_userpool_exist = True if pool_name in user_pools else False
     user_pool_index = user_pools.index(pool_name) if is_userpool_exist else None
-    pool_id = response["UserPools"][user_pool_index]["Id"]
+    pool_id = response["UserPools"][user_pool_index]["Id"] if is_userpool_exist else None
     return is_userpool_exist, pool_id
 
 def get_users(pool_id, cognito_client):