Skip to content

Commit

Permalink
Read paginated values since we now have more than 60 pools
Browse files Browse the repository at this point in the history
The initial version of this script read the first 60 user pools.
This is because this is a paginated API, with a max value of 60.
However, we now have more than 60 pools, so the new pool that we were trying to
create was not returned in that first batch of 60.

```
    response = cognito_client.list_user_pools(MaxResults=59)

nrelopenpath-prod-unc-ebike does not exist! Try again later.
```

```
    response = cognito_client.list_user_pools(MaxResults=59)

nrelopenpath-prod-unc-ebike does not exist! Try again later.
```

Simply bumping up the max results doesn't work because of the server side limit

```
    response = cognito_client.list_user_pools(MaxResults=61)

botocore.errorfactory.InvalidParameterException: An error occurred (InvalidParameterException) when calling the ListUserPools operation: 1 validation error detected: Value '61' at 'maxResults' failed to satisfy constraint: Member must have value less than or equal to 60
```

Instead, we need to implement pagination (similar to
https://stackoverflow.com/a/64698263) to read all the entries before we try to
find the pool name. We still have < 100 pools so this is fine. But if we get to
10s of thousands of pools, we should switch to a more streaming appraoch in
which we check the results on each page instead of concatenating everything.

Testing done:

```
Received response with len(response["UserPools"])=60 and next_token=REDACTED
Received response with len(response["UserPools"])=21 & next_token=None
[email protected] not in user pool! Creating account...
Account created! Sending welcome email.
[email protected] not in user pool! Creating account...
Account created! Sending welcome email.
[email protected] not in user pool! Creating account...
Account created! Sending welcome email.
```
  • Loading branch information
shankari committed Sep 4, 2024
1 parent 6af23d3 commit a4bdcfd
Showing 1 changed file with 18 additions and 5 deletions.
23 changes: 18 additions & 5 deletions email_automation/email-config.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,14 +52,28 @@
AWS_REGION = os.environ.get("AWS_REGION")
cognito_client = boto3.client('cognito-idp', region_name=AWS_REGION)
sts_client = ''

def read_userpool_obj_list_on_all_pages(cognito_client):
# From https://stackoverflow.com/a/64698263
response = cognito_client.list_user_pools(MaxResults=60)
next_token = response.get("NextToken", None)
print(f'Received response with {len(response["UserPools"])=} and {next_token=}')
user_pool_obj_list = response["UserPools"]
while next_token is not None:
response = cognito_client.list_user_pools(NextToken=next_token, MaxResults=60)
next_token = response.get("NextToken", None)
print(f'Received response with {len(response["UserPools"])=} & {next_token=}')
user_pool_obj_list.extend(response["UserPools"])
return user_pool_obj_list

# Functions
def get_userpool_name(pool_name, cognito_client):
response = cognito_client.list_user_pools(MaxResults=60)
is_userpool_exist = False
user_pools = [user_pool["Name"] for user_pool in response["UserPools"]]
all_user_pools = read_userpool_obj_list_on_all_pages(cognito_client)
is_userpool_exist = False
user_pools = [user_pool["Name"] for user_pool in all_user_pools]
is_userpool_exist = True if pool_name in user_pools else False
user_pool_index = user_pools.index(pool_name) if is_userpool_exist else None
pool_id = response["UserPools"][user_pool_index]["Id"] if is_userpool_exist else None
pool_id = all_user_pools[user_pool_index]["Id"] if is_userpool_exist else None
return is_userpool_exist, pool_id

def get_users(pool_id, cognito_client):
Expand Down Expand Up @@ -182,4 +196,3 @@ def remove_user(pool_id, user):
print(email + " already in user pool!")
else:
print(pool_name + " does not exist! Try again later.")

0 comments on commit a4bdcfd

Please sign in to comment.