Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Workflow output definition (second preview) #5185

Open
wants to merge 16 commits into
base: master
Choose a base branch
from

Conversation

bentsherman
Copy link
Member

@bentsherman bentsherman commented Jul 30, 2024

Close #5103

TODO:

language improvements:

  • support dynamic path
  • move publish options to config
  • remove publish section from process definition

runtime improvements:

  • prevent leading/trailing slashes
  • report unused target names
  • detect file collisions at runtime ?
  • detect outputs that aren't published or used downstream ?

new features (might be handled separately):

  • support JSON index file
  • generate output schema ?
  • include publish targets in inspect command ?

Copy link

netlify bot commented Jul 30, 2024

Deploy Preview for nextflow-docs-staging ready!

Name Link
🔨 Latest commit a2a690b
🔍 Latest deploy log https://app.netlify.com/sites/nextflow-docs-staging/deploys/671274070f4d8b00081ee330
😎 Deploy Preview https://deploy-preview-5185--nextflow-docs-staging.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@bentsherman bentsherman changed the title Finalize workflow output definition Workflow output definition (second preview) Sep 23, 2024
@bentsherman bentsherman marked this pull request as ready for review September 24, 2024 19:42
@bentsherman bentsherman requested a review from a team as a code owner September 24, 2024 19:42
@bentsherman bentsherman requested review from pditommaso and removed request for a team September 24, 2024 19:42
@bentsherman
Copy link
Member Author

Second preview is ready for review, POCs have also been updated:

output {
'fastq' {
path { meta, fastq_1, fastq_2 ->
{ file -> "fastq/${meta.id}/${file.baseName}" }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

<3

Signed-off-by: Ben Sherman <[email protected]>
@nvnieuwk
Copy link

nvnieuwk commented Oct 8, 2024

I just tried the dynamic publishing with some dummy code and it worked perfectly! I'm looking forward to implementing this in my pipelines 🥳

Comment on lines -52 to -86
void contentType(String value) {
setDefault('contentType', value)
}

void contentType(boolean value) {
setDefault('contentType', value)
}

void ignoreErrors(boolean value) {
setDefault('ignoreErrors', value)
}

void mode(String value) {
setDefault('mode', value)
}

void overwrite(boolean value) {
setDefault('overwrite', value)
}

void overwrite(String value) {
setDefault('overwrite', value)
}

void storageClass(String value) {
setDefault('storageClass', value)
}

void tags(Map value) {
setDefault('tags', value)
}

void enabled( boolean value ) {
setDefault('enabled', value)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it still possible to control these at target level?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice, and output top level? only in the config or config and script definition?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

only the config. default output directory is results

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants