Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use eval to collect version #5834

Open
Tracked by #5828
ewels opened this issue Jun 19, 2024 · 5 comments
Open
Tracked by #5828

Use eval to collect version #5834

ewels opened this issue Jun 19, 2024 · 5 comments

Comments

@ewels
Copy link
Member

ewels commented Jun 19, 2024

Nextflow 24.02.0-edge brings with it two new features which should give much cleaner syntax for collecting tool versions: channel topics using topic (see #4517) and eval. They are designed to work together nicely, but are technically separate features.

The blog post contains a basic example of how this can be used to collect versions:

process FASTQC {
  input:
  tuple val(meta), path(reads)

  output:
  tuple val(meta), path('*.html'), emit: html
  tuple val("${task.process}"), val('fastqc'), eval('fastqc --version'), topic: versions 

  """
  fastqc $reads
  """
}

workflow {
  Channel.topic('versions') 
    | unique()
    | collectFile(name: 'collated_versions.yml')
    | CUSTOM_DUMPSOFTWAREVERSIONS
}

Needs some thought about how we approach these changes - may be best to do a bulk change everywhere in one go, as it affects multiple code locations (modules and pipelines).

@muffato
Copy link
Member

muffato commented Jun 19, 2024

Nextflow 22.04 brings with it

Did you mean 24.04 ?

@ewels
Copy link
Member Author

ewels commented Jun 27, 2024

See comment: #4517 (comment)

@SPPearce
Copy link
Contributor

SPPearce commented Oct 1, 2024

Does this help fix #6696 ?

@asp8200
Copy link
Contributor

asp8200 commented Oct 1, 2024

An example that might need to be taken into consideration is that, in the nf-module for bwa, the command used to retrieve the version information is bwa 2>&1 and that command always returns exit code 1, and so I guess eval('bwa 2>&1') will always fail.

If the result of the version-retrieving command is pipped into sed like here, and the eval-statement over in the fastqc-module in the nf-core/repo would be something like eval('fastqc --version | sed \'/FastQC v/!d; s/.*v//'\')(Not sure about the syntax for escaping the single quotation mark (') but nevermind.), then Nextflow might not detect if the command fastqc --version | sed \'/FastQC v/!d; s/.*v//') has failed, because <some_failing_cmd> | sed '/FastQC v/!d; s/.*v//' seems to always return exit code 0.

@SPPearce
Copy link
Contributor

SPPearce commented Oct 1, 2024

Can we set pipefail globally? Is it already done?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants