Re-work the way the configuration works

[carlspring]

Task Description

The current way that the configuration is implemented is quite confusing and error-prone to work with. We need to clarify how we'd like this to work, gather the requirements and refactor this.

Current Way It Works

Generic Way

Map<String, Object> env;

String accessKey = System.getenv(ACCESS_KEY);
String secretKey = System.getenv(SECRET_KEY);
String region = System.getenv(REGION);
String protocol = System.getenv(PROTOCOL);

env = ImmutableMap.<String, Object>builder().put(ACCESS_KEY, accessKey)
                                            .put(SECRET_KEY, secretKey)
                                            .put(REGION, region)
                                            .put(PROTOCOL, protocol)
                                            .build();

Production

If there is an amazon.properties on the classpath, this will be used to load the properties. The configuration is loaded using org.carlspring.cloud.storage.s3fs.S3FileSystemProvider.

If not, the generic way can be used.

Tests

If there is an amazon-test.properties on the classpath, this will be used to load the properties. The configuration is loaded using org.carlspring.cloud.storage.s3fs.util.EnvironmentBuilder.

If not, the generic way can be used.

Current Problems

• Production and testing should use the same code for loading properties and setting up the configuration.
• It is not possible to use multiple buckets, region, or credential settings.

Tasks

The following tasks will need to be carried out:

• Define our use cases.
• Collect a list of problems with the current implementation.
• Collect a list of requirements.
  • It must be possible to override the default settings using properties.
  • It must be easy to override the configuration during testing, especially, if we have parallel tests.
  • It must be possible to use in parallel tests.
• Refactor the code
  • Convert all the properties to use dots instead of underscores, as this is the real Java convention. ( Convert the configuration properties to use dots instead of underscores #136 )
  • Make it possible to use settings for multiple buckets, regions, credentials. ( Re-work the configuration properties to support multiple buckets, credentials and regions #137 )
  • Make the loading of the configuration work the same way in production and testing.
  • Improve the validation for the core properties (bucket name, access key, secret key, region, protocol) and fail early with meaningful messages.
• Update the documentation. ( Document the configuration options #114 )

Task Relationships

This task:

• Relates to: Document the configuration options #114

Help

• Our chat channel
• Points of contact:
  • @carlspring
  • @steve-todorov

[carlspring]

@ptirador , @steve-todorov , @elerch, @markjschreiber, @pditommaso ,

Please, feel free to join in and share your thoughts on this topic! :)

@elerch, @markjschreiber: I know you had some first hand experience in this regard, as well as a passionate disliking of how it currently works, which would be quite helpful in defining how to re-implement and do it better.