Conditionally populate resourceLabels to tag field in nf-amazon plugin's AwsBatchTaskHandler for AWS Batch Template Registration

[rbartelme-pivot]

New feature

Proposed change, if resourceLabels are present for a given process. They should populate the AwsBatchTaskHandler's tag's field when Nextflow registers a job definition on AWS Batch.

Usage scenario

Multiple users have asked for this feature to track resource utilization and this can be used by AWS Solutions Architects to manage Nextflow behavior in non-NF Tower systems. This also allows AWS Solutions Architects to track costs, compute utilization, and optimize deployments across AWS Batch's resources like Fargate and EC2.

Suggest implementation

Add a conditional to the nf-amazon plugin's groovy source code where if resourceLabels are present for a given process that is trying to register an AWS Batch template, the resourceLabels contents populate the tags via req.addTagsEntry().

nextflow/plugins/nf-amazon/src/main/nextflow/cloud/aws/batch/AwsBatchTaskHandler.groovy

Line 657 in 711864f

protected String createJobDef(RegisterJobDefinitionRequest req) {

[pditommaso]

It sounds reasonable

[pditommaso]

Discussing a bit about this, this has not been done because the job definition is created by nextflow only the very first, and then re-used for all following runs using the same container.

Adding the labels, it should be recreated ever time a label value change, that may be expected, it could result in a proliferation of batch job definitions, which may not be desirable in common cases