[Bug]: Following "Getting Started" documentation on README fails

[dbwiddis]

Describe the bug

I am following the Getting Started documentation on the README:

opensearch-ci/README.md

Lines 32 to 37 in 1590bcc

	## Getting Started
	- Requires [NPM](https://docs.npmjs.com/cli/v7/configuring-npm/install) to be installed
	- Install project dependencies using `npm install` from this project directory
	- Configure [aws credentials](https://docs.aws.amazon.com/cdk/latest/guide/getting_started.html#getting_started_prerequisites)
	- Deploy stacks with `npm run cdk deploy`

The final step results in an exception:

➜  opensearch-ci git:(main) npm run cdk deploy     

> [email protected] cdk
> cdk deploy

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!                                                                                                                      !!
!!  This software has not been tested with node v20.0.0.                                                                !!
!!  Should you encounter odd runtime issues, please try using one of the supported release before filing a bug report.  !!
!!                                                                                                                      !!
!!  This software is currently running on node v20.0.0.                                                                 !!
!!  As of the current release of this software, supported node releases are:                                            !!
!!  - ^18.0.0 (Planned end-of-life: 2025-04-30)                                                                         !!
!!  - ^16.3.0 (Planned end-of-life: 2023-09-11)                                                                         !!
!!  - ^19.0.0 (Planned end-of-life: 2023-06-01)                                                                         !!
!!                                                                                                                      !!
!!  This warning can be silenced by setting the JSII_SILENCE_WARNING_UNTESTED_NODE_VERSION environment variable.        !!
!!                                                                                                                      !!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

/Users/widdisd/git/opensearch-ci/lib/ci-stack.ts:95
      throw new Error('useSsl parameter is required to be set as - true or false');
            ^
Error: useSsl parameter is required to be set as - true or false
    at new CIStack (/Users/widdisd/git/opensearch-ci/lib/ci-stack.ts:95:13)
    at Object.<anonymous> (/Users/widdisd/git/opensearch-ci/bin/ci-stack.ts:20:17)
    at Module._compile (node:internal/modules/cjs/loader:1267:14)
    at Module.m._compile (/Users/widdisd/git/opensearch-ci/node_modules/ts-node/src/index.ts:1056:23)
    at Module._extensions..js (node:internal/modules/cjs/loader:1321:10)
    at Object.require.extensions.<computed> [as .ts] (/Users/widdisd/git/opensearch-ci/node_modules/ts-node/src/index.ts:1059:12)
    at Module.load (node:internal/modules/cjs/loader:1125:32)
    at Function.Module._load (node:internal/modules/cjs/loader:965:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:83:12)
    at main (/Users/widdisd/git/opensearch-ci/node_modules/ts-node/src/bin.ts:198:14)

To reproduce

"From Scratch" on macOS:

1. brew install nvm
2. Update .zshrc per homebrew instructions
3. nvm install node
4. Obtained AWS credentials/keys and set up config files per linked docs
5. npm install -g aws-cdk
6. npm run cdk deploy

Expected behavior

No errors

Screenshots

N/A

Host / Environment

macOS 13.3.1

Additional context

The Getting Started documentation is difficult to follow; it took 2 hours just to get through the 4 steps only to fail:

1. First bullet: the link to NPM in the first step goes to version 7; current is version 9; there is no guidance on which version should be installed, if not the latest
2. Second bullet: took me a while to see that "from this project directory" required cloning the project locally; this should be explicitly stated (and agan in the fourth bullet)
3. Configure aws credentials. Umm, ok? Exactly what am I configuring? The linked document is very broad. It took me bouncing to multiple links from there to figure that (I think) I just needed the ~/.aws/config and ~/.aws/credentials files per samples at those links. I'm not 100% sure I was right.
4. This apparently requires a useSsl setting per the stack trace. There's no documentation of where/how to provide it. Is it supposed to be in the aws config in the previous step? A command line switch?

Relevant log output

No response

[dbwiddis]

(Following up) It looks like the deploy command is run later in the third step under "Dev Deployment". So I guess the documentation bug is that the getting started section implies one should be running that command, rather than a more general "Deploy as described in the next sections."

See "Additional context" above for other unclear things in these docs.

[dbwiddis]

(More confusion) Continuing to document more README updates needed. Dev Deployment step 4 specifies CASC_RELOAD_TOKEN but I'm assuming from my output it's this one (values have been obfuscated):

OpenSearch-CI-Config-Dev.cascSecretValue = arn:aws:secretsmanager:us-east-1:111122223333:secret:CascReloadTokenValueABCDEFG-hijklmnopqrs-TUVWXY

It says to update the password and I'm assuming I put that in the Secret String, but that's not clear from the docs. The field name "CascReloadToken" implies I just take that from the tail-end of the arn (e.g., ABCDEFG-hijklmnopqrs-TUVWXY from above).

The aws command in the example has a leading $ from the command prompt, which doesn't match the rest of the commands and catches we unwary copy-pasters.

The link in that section goes to v1 docs, which say I should be using v2 docs, as well.

[dbwiddis]

After entering what I thought was the correct aws secretsmanager put-secret-value command in the previous step, the resulting error message is "Error loading SSO Token: Token for my-sso does not exist". This implies I did something wrong way back in the extremely vaguely worded "configure aws credentials" step. I had tried a few sso commands there that failed, clearly they were needed.

[jordarlu]

thanks for the issue, let me have a look and update. thanks!

[jordarlu]

Hi, @dbwiddis , I believe we can definitely optimize the README, so, others won't get confused as you and me. ( yes, I was able to reproduce the issue, thanks for laying out the steps you did )

1. I think we should remove the 4th line in Getting Started section because it is supposed to be "getting started" as prerequisite to begin the CDK project. The exact/runnable 'cdk deploy' commands are mentioned in the following up sections.
2. we can move Dev Deployment in front of CI Deployment; so, the user will be able to experience bootstrping and cdk deploy from end to end in Dev before they create their Prod project.
3. I feel the link to AWS CDK getting_started.html is still valid as it is the official AWS CDK prerequisite page; however, we can probably mention the AWS CLI may easy the step by doing 'aws configure' to setup all the security settings required.

To answer your highlighted questions :

First bullet: the link to NPM in the first step goes to version 7; current is version 9; there is no guidance on which version should be installed, if not the latest

Maybe we can use this page instaed : https://docs.npmjs.com/downloading-and-installing-node-js-and-npm. It has the links to the latest LTS version of node including the latest npm.

Second bullet: took me a while to see that "from this project directory" required cloning the project locally; this should be explicitly stated (and agan in the fourth bullet)

yes, I am with you on this, I too feel that we can clearly state cloning this repository to the local project folder, and all the commands mentioned later on shall be run at local project folder.

Configure aws credentials. Umm, ok? Exactly what am I configuring? The linked document is very broad. It took me bouncing to multiple links from there to figure that (I think) I just needed the ~/.aws/config and ~/.aws/credentials files per samples at those links. I'm not 100% sure I was right.

yes, maybe we should recommend installing AWS CLI, and run aws configure to set all the required key, Id, and region.

This apparently requires a useSsl setting per the stack trace. There's no documentation of where/how to provide it. Is it supposed to be in the aws config in the previous step? A command line switch?

we can set this optional argument to be off if needed as metnioned in the later section.

As for the 'aws secretsmanager' step, it means to retrieve the entire secret manager arns from the output of ci-config-stack ( where you can find it by logging in AWS Console, CloudFormation, then, find the stack name, the, the output tab ).
In your case it will be like this :

$aws secretsmanager put-secret-value \
--secret-id arn:aws:secretsmanager:us-east-1:111122223333:secret:CascReloadTokenValueABCDEFG-hijklmnopqrs-TUVWXY \
--secret-string "_any string you like to use here_"

Do you think this help? Let me also chat with team if we can suggest more improvement at once...

thanks,

[jordarlu]

Hi, @peterzhuamazon @gaiksaya @zelinh @rishabh6788 , please have a look and if any comments on the approaching of improving the README content in openseach-ci repo? thanks !!