Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: New environment variable obfuscation functionality #355

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

matglas
Copy link
Contributor

@matglas matglas commented Sep 20, 2024

What this PR does / why we need it

Implementing obfuscation of environment variables. Capturing secret values like tokens and api keys is a security risk and attestation should not hold that information. Although it is important to know which values are used to adjust the behavior for scripts.

Which issue(s) this PR fixes (optional)

Partially fixes #275

Acceptance Criteria Met

  • Docs changes if needed
  • Testing changes if needed
  • All workflow checks passing (automatically enforced)
  • All review conversations resolved (automatically enforced)
  • DCO Sign-off

Special notes for your reviewer:

The default list can be extended with more variable that should be hidden. This can be extended in the cli with flags that add more items to the list. I think that the default obfuscation list does not need to be 'overridden' but only needs extension.

It also obfuscates based on glob. So if an key would be added like REASON_* it will pick up all REASON_TEST etc. variables. There are a few glob keys that are added by default.

*_TOKEN
SECRET_*
*_API_KEY
*_PASSWORD
*_JWT

@matglas matglas force-pushed the env-obfuscate branch 5 times, most recently from c913faa to 39ceb12 Compare September 20, 2024 12:25
@matglas
Copy link
Contributor Author

matglas commented Sep 20, 2024

@jkjell would you be able to do a review?

@kairoaraujo kairoaraujo self-requested a review September 20, 2024 15:20
attestation/environment/obfuscate.go Outdated Show resolved Hide resolved
Comment on lines 26 to 32
return map[string]struct{}{
"*_TOKEN": {},
"SECRET_*": {},
"*_API_KEY": {},
"*_PASSWORD": {},
"*_JWT": {},
}
Copy link
Collaborator

@kairoaraujo kairoaraujo Sep 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I'm missing some context here, but for code maintainability, my suggestion would be:

Detach the

return map[string]struct{}{
"AWS_ACCESS_KEY_ID": {},
"AWS_SECRET_ACCESS_KEY": {},
"AMAZON_AWS_ACCESS_KEY_ID": {},
"AMAZON_AWS_SECRET_ACCESS_KEY": {},
"ALGOLIA_API_KEY": {},
"AZURE_CLIENT_ID": {},

(...)

from the DefaultBlockList and add it to a specific separated file, i.e., sensitive_env_vars.go, and use it for blocklist and obfuscate.
I would also extend this list to have the wildcards you added here.

About the wildcards, I think we should not use *_<WORD> but *<WORD> as it may not start with a prefix i.e. NAME_TOKEN but TOKEN_NAME

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Context: Currently the block list completely removes those variables. The obfuscate will keep them but hide the content.

I am not sure if combining those lists is a good option in that case.

About the wildcard. I can understand. I do think that this obfuscate defaults is just a 'primary' list. And users might need to extend it for their own cases. Actually many of the items in the block list 'could' be removed and be covered by the obfuscate.

As I am thinking about this now it might still be very valuable to extend the blocklist. Because you do not want to expose to consumers where you are using a specific token. 🤔 The obfuscate is basically a stop gap solution. So maybe you are right and we should obfuscate any *<WORD>* like *TOKEN*.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if combining those lists is a good option in that case.

I don't understand why the source list of blocking and obfuscating should be different, as both just have different behaviors when removing or obfuscating the data.

In other words. As a user, I expect them to differ on the "remove"/"obfuscate" my environment variables.
As a user, I shouldn't need to understand that actions uses different criteria

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True. Obfuscate now is complementary to the blocklist without changing the current behavior. But in the end it would be better to use a flag that says e.g. "block and obfuscate by default".

Co-authored-by: Kairo Araujo <[email protected]>
Signed-off-by: Matthias Glastra <[email protected]>
@matglas
Copy link
Contributor Author

matglas commented Sep 27, 2024

With this latest change I go from default blocking behavior to default obfuscate behavior. And the default list
is the same for blocking and obfuscating. Also the blocking will also use glob patterns now.

Introducing several arguments:

--attestor-environment-block-sensitive-vars
Switch from obfuscate to blocking variables which removes them from the output completely.

--attestor-environment-disable-default-sensitive-vars
Disable the default list of sensitive vars and only use the items mentioned by --attestor-environment-sensitive-key.

--attestor-environment-sensitive-key can be repeated
Add keys to the list of sensitive environment keys.

I believe that as we are still in the v0.x.y versions it should not be a problem to change the default
behavior from blocking to obfuscating. The variables will still not be exposed and switching to blocking
is still possible.

@mikhailswift I need a bit of help on the --attestor-environment-sensitive-key and it looks like you worked on the registry.StringSliceConfigOption. But its not capturing the values correctly somehow. Are you able to help me a bit here?

Adding flags to behave blocking or obfuscating and adding new keys.

Signed-off-by: Matthias Glastra <[email protected]>
@matglas
Copy link
Contributor Author

matglas commented Sep 27, 2024

Forget the part about the registry.StringSliceConfigOption. I had a misconfiguration in my debug setup. But I did found another missing part after that 😄 . The function works now properly. It only needs more work on the tests.

@matglas
Copy link
Contributor Author

matglas commented Sep 30, 2024

My last changes fixed the tests. I also renamed block to filter. Which seems to me that it is a much more logical.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feat]: Extensibility for the environment attestor
2 participants