Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tool/pipeline wrapping FAQ #5374

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

tool/pipeline wrapping FAQ #5374

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
33a1f8e
enable more cyoa features
hexylena Sep 27, 2024
cb79f5e
Draft tool/pipeline wrapping FAQs
hexylena Sep 27, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion _includes/cyoa-choices.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
<blockquote class="notranslate hands_on gtn-cyoa" id="gtn-cyoa{{ include.disambiguation }}">
<div class="box-title" aria-label="hands-on box: choose your own tutorial"><i class="fas fa-pencil-alt" aria-hidden="true"></i><span class="visually-hidden"></span> Hands-on: Choose Your Own Tutorial</div>
<div class="box-title" aria-label="hands-on box: choose your own tutorial"><i class="fas fa-pencil-alt" aria-hidden="true"></i><span class="visually-hidden"></span> {{ include.title | default: "Hands-on: Choose Your Own Tutorial" }}</div>
{% unless include.brief %}
<p>This is a "Choose Your Own Tutorial" section, where you can select between multiple paths. Click one of the buttons below to select how you want to follow the tutorial</p>
{% endunless %}
{% if include.text %}
<p>
{{ include.text }}
Expand Down
132 changes: 132 additions & 0 deletions topics/dev/faqs/pipeline_wrapping.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
---
title: How to Wrap Your Pipeline
description: Pre-existing pipelines can be wrapped in a variety of ways
area: tool-dev
box_type: tip
layout: faq
contributors: [hexylena]
---

{% include _includes/cyoa-choices.html title="Is this for wide distribution or to help a colleague internally?" disambiguation="distribute" option1="Internal only" option2="We want the world to use this in Galaxy" default="__hidden__" %}

## Pipeline Source

<div class="Internal-only">

{% include _includes/cyoa-choices.html disambiguation="package" title="Which workflow engine is used for this pipeline?" option1="Nextflow" option2="Snakemake" option3="Bash" default="__hidden__" %}

<div class="Nextflow" markdown="1">

You're probably a busy researcher! You may not want to spend a lot of time wrapping this. There is an *easy way out* for you. It's **not best practice**, but maybe you just need to get it done quickly

```xml
<tool name="Nextflow Pipeline: Sarek" id="internal.nextflow.sarek" version="1.0">
<description>runs a specific nextflow pipeline</description>
<requirements>
<requirement type="package" version="24.04.4">nextflow</requirement>
</requirements>
<stdio>
<exit_code range="1:" level="fatal"/>
</stdio>
<command><![CDATA[
nextflow run nf-core/sarek \
--input '$input' \
--outdir outdir
]]></command>
<inputs>
<param label="Input File" type="data" format="csv" name="input"/>
</inputs>
<outputs>
<discover_datasets pattern="(?P&lt;designation&gt;.+)\.(?P&lt;ext&gt;.*)" directory="outdir" />
</outputs>
<help><![CDATA[]]></help>
</tool>
```

</div>

<div class="Snakemake" markdown="1">

You're probably a busy researcher! You may not want to spend a lot of time wrapping this. There is an *easy way out* for you. It's **not best practice**, but maybe you just need to get it done quickly

```xml
<tool name="Snakemake Pipeline" id="internal.snakemake.pipeline" version="1.0" profile="21.05">
<description>runs a specific pipeline</description>
<requirements>
<requirement type="package" version="8.20.5">snakemake</requirement>
</requirements>
<stdio>
<exit_code range="1:" level="fatal"/>
</stdio>
<command><![CDATA[
cp '$__tool_directory__/Snakefile' . ;
snakemake
]]></command>
<inputs>
<param label="Input File" type="data" format="csv" name="input"/>
</inputs>
<outputs>
<discover_datasets pattern="(?P&lt;designation&gt;.+)\.(?P&lt;ext&gt;.*)" directory="outdir" />
</outputs>
<help><![CDATA[]]></help>
</tool>
```

</div>

<div class="Bash" markdown="1">

You're probably a busy researcher! You may not want to spend a lot of time wrapping this. There is an *easy way out* for you. It's **not best practice**, but maybe you just need to get it done quickly

```xml
<tool name="Bash Pipeline" id="internal.bash.pipeline" version="1.0">
<description>runs a specific pipeline</description>
<requirements>
<requirement type="package" version="1.0.0">my-conda-dependency</requirement>
</requirements>
<stdio>
<exit_code range="1:" level="fatal"/>
</stdio>
<command><![CDATA[
# Literally just paste your bash script here
# And add some '$input' and 'outdir' variables
]]></command>
<inputs>
<param label="Input File" type="data" format="csv" name="input"/>
</inputs>
<outputs>
<discover_datasets pattern="(?P&lt;designation&gt;.+)\.(?P&lt;ext&gt;.*)" directory="outdir" />
</outputs>
<help><![CDATA[]]></help>
</tool>
```

<div class="Bash Snakemake Nextflow" markdown="1">

Obviously this will not take optimal advantage of your cluster, you'll have to set the memory for this "tool" to the maximum value that the entire pipeline will need, but it'll get the job done.

Of course if you have time or want the world to use this pipeline in Galaxy, you can go back and follow the <button class="btn btn-primary" onclick="cyoaChoice('We-want-the-world-to-use-this-in-Galaxy', 'gtn-cyoadistribute');">Best Practice workflow</button>

</div>

</div>
</div>


<div class="We-want-the-world-to-use-this-in-Galaxy" markdown="1">

Ah you want the world to use this! Ok, that probably means you'll want to make it a best practice workflow, which means decomposing each and every individual step into the corresponding Galaxy tool.

This may mean updating existing tools to use new flags, or creating new tools entirely.

## Tools

- tools
- are ALL of those tools in Galaxy?
- are SOME in Galaxy?
- are NONE in Galaxy?


We'll start off with what you have

</div>
52 changes: 52 additions & 0 deletions topics/dev/faqs/tool_wrapping.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
---
title: How to Wrap Your Tool
description: There are many different ways to wrap a tool and some of the choices you'll want to make change depending on what you have available.
area: tool-dev
box_type: tip
layout: faq
contributors: [hexylena]
---

## Dependencies

{% include _includes/cyoa-choices.html title="How are your dependencies currently available?" disambiguation="1" option1="Conda" option2="Docker" option3="BioContainers" option4="None of the above" default="__hidden__" %}


<div class="Conda" markdown=1>
Great! You're good to go then! Galaxy natively uses Conda based dependencies.

in your tool XML you'll just need to specify packages and their versions, based on your conda environment.

```xml
<requirements>
<requirement type="package" version="1.0.0">my-package</requirement>
</requirements>
```

</div>

<div class="Docker" markdown=1>

Great! You're good to go then! Galaxy can natively use Docker based dependencies. In your tool XML you'll just need to specify the container and it's version

```xml
<requirements>
<container type="docker">quay.io/galaxy/homer-galaxy:4.11</container>
</requirements>
```

TODO: how to inputs/outputs work? container commands?

</div>

<div class="BioContainers" markdown=1>

If your dependency is a [BioContainers](https://biocontainers.pro/) container, there's a chance it's available in Conda as well, as biocontainers are generated from conda packages automatically. You can check the conda repositories for the upstream package, and if it's available, you can treat it like a conda package:

```xml
<requirements>
<requirement type="package" version="1.0.0">my-package</requirement>
</requirements>
```

</div>
Loading