Skip to content

Commit

Permalink
Merge pull request #194 from JoseEspinosa/merging-template-3.0.2
Browse files Browse the repository at this point in the history
Merging template 3.0.2
  • Loading branch information
JoseEspinosa authored Oct 15, 2024
2 parents 5270d2e + 469bbf4 commit aebbd21
Show file tree
Hide file tree
Showing 17 changed files with 106 additions and 71 deletions.
4 changes: 0 additions & 4 deletions .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ indent_style = space
[*.{md,yml,yaml,html,css,scss,js}]
indent_size = 2


# These files are edited and tested upstream in nf-core/modules
[/modules/nf-core/**]
charset = unset
Expand All @@ -26,12 +25,9 @@ insert_final_newline = unset
trim_trailing_whitespace = unset
indent_style = unset



[/assets/email*]
indent_size = unset


# ignore python and markdown
[*.{py,md}]
indent_style = unset
2 changes: 1 addition & 1 deletion .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ Once there, use `nf-core pipelines schema build` to add to `nextflow_schema.json

### Default processes resource requirements

Sensible defaults for process resource requirements (CPUs / memory / time) for a process should be defined in `conf/base.config`. These should generally be specified generic with `withLabel:` selectors so they can be shared across multiple processes/steps of the pipeline. A nf-core standard set of labels that should be followed where possible can be seen in the [nf-core pipeline template](https://github.com/nf-core/tools/blob/master/nf_core/pipeline-template/conf/base.config), which has the default process as a single core-process, and then different levels of multi-core configurations for increasingly large memory requirements defined with standardised labels.
Sensible defaults for process resource requirements (CPUs / memory / time) for a process should be defined in `conf/base.config`. These should generally be specified generic with `withLabel:` selectors so they can be shared across multiple processes/steps of the pipeline. A nf-core standard set of labels that should be followed where possible can be seen in the [nf-core pipeline template](https://github.com/nf-core/tools/blob/main/nf_core/pipeline-template/conf/base.config), which has the default process as a single core-process, and then different levels of multi-core configurations for increasingly large memory requirements defined with standardised labels.

The process resources can be passed on to the tool dynamically within the process with the `${task.cpus}` and `${task.memory}` variables in the `script:` block.

Expand Down
6 changes: 4 additions & 2 deletions .github/workflows/awsfulltest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@ on:
jobs:
run-platform:
name: Run AWS full tests
if: github.repository == 'nf-core/proteinfold' && github.event.review.state == 'approved'
# run only if the PR is approved by at least 2 reviewers and against the master branch or manually triggered
if: github.repository == 'nf-core/proteinfold' && github.event.review.state == 'approved' && github.event.pull_request.base.ref == 'master' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
# Do a full-scale run on each of the mode
strategy:
Expand All @@ -34,10 +35,11 @@ jobs:
- uses: octokit/[email protected]
id: check_approvals
with:
route: GET /repos/${{ github.repository }}/pulls/${{ github.event.review.number }}/reviews
route: GET /repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}/reviews
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- id: test_variables
if: github.event_name != 'workflow_dispatch'
run: |
JSON_RESPONSE='${{ steps.check_approvals.outputs.data }}'
CURRENT_APPROVALS_COUNT=$(echo $JSON_RESPONSE | jq -c '[.[] | select(.state | contains("APPROVED")) ] | length')
Expand Down
64 changes: 52 additions & 12 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,16 @@ on:

env:
NXF_ANSI_LOG: false
NXF_SINGULARITY_CACHEDIR: ${{ github.workspace }}/.singularity
NXF_SINGULARITY_LIBRARYDIR: ${{ github.workspace }}/.singularity

concurrency:
group: "${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}"
cancel-in-progress: true

jobs:
test:
name: Run pipeline with test data
name: "Run pipeline with test data (${{ matrix.NXF_VER }} | ${{ matrix.profile }} | ${{ matrix.test_profile }})"
# Only run on push if this is the nf-core dev branch (merged PRs)
if: "${{ github.event_name != 'push' || (github.event_name == 'push' && github.repository == 'nf-core/proteinfold') }}"
runs-on: ubuntu-latest
Expand All @@ -27,38 +29,76 @@ jobs:
NXF_VER:
- "24.04.2"
- "latest-everything"
parameters:
profile:
- "conda"
- "docker"
- "singularity"
test_name:
- "test"
test_profile:
- "test"
- "test_alphafold2_split"
- "test_alphafold2_download"
- "test_colabfold_local"
- "test_colabfold_webserver"
- "test_colabfold_download"
- "test_esmfold"

isMaster:
- ${{ github.base_ref == 'master' }}
# Exclude conda and singularity on dev
exclude:
- isMaster: false
profile: "conda"
- isMaster: false
profile: "singularity"
steps:
- name: Check out pipeline code
uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4

- name: Install Nextflow
- name: Set up Nextflow
uses: nf-core/setup-nextflow@v2
with:
version: "${{ matrix.NXF_VER }}"

- name: Disk space cleanup
- name: Set up Apptainer
if: matrix.profile == 'singularity'
uses: eWaterCycle/setup-apptainer@main

- name: Set up Singularity
if: matrix.profile == 'singularity'
run: |
mkdir -p $NXF_SINGULARITY_CACHEDIR
mkdir -p $NXF_SINGULARITY_LIBRARYDIR
- name: Set up Miniconda
if: matrix.profile == 'conda'
uses: conda-incubator/setup-miniconda@a4260408e20b96e80095f42ff7f1a15b27dd94ca # v3
with:
miniconda-version: "latest"
auto-update-conda: true
conda-solver: libmamba
channels: conda-forge,bioconda

- name: Set up Conda
if: matrix.profile == 'conda'
run: |
echo $(realpath $CONDA)/condabin >> $GITHUB_PATH
echo $(realpath python) >> $GITHUB_PATH
- name: Clean up Disk space
uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be # v1.3.1

- name: Run pipeline with test data ${{ matrix.parameters }} (docker)
- name: Run pipeline with test data (docker)
run: |
nextflow run ${GITHUB_WORKSPACE} -profile ${{ matrix.parameters }},docker --outdir ./results
nextflow run ${GITHUB_WORKSPACE} -profile ${{ matrix.test_profile }},docker --outdir ./results
- name: Run pipeline with test data ${{ matrix.parameters }} (singularity)
- name: Run pipeline with test data (singularity)
run: |
nextflow run ${GITHUB_WORKSPACE} -profile ${{ matrix.parameters }},singularity --outdir ./results
nextflow run ${GITHUB_WORKSPACE} -profile ${{ matrix.test_profile }},singularity --outdir ./results
if: "${{ github.base_ref == 'master' }}"

## Warning: Pipeline can not be run with conda
# - name: Run pipeline with test data ${{ matrix.parameters }} (conda)
# ## Warning: Pipeline can not be run with conda
# - name: Run pipeline with test data (conda)
# run: |
# nextflow run ${GITHUB_WORKSPACE} -profile ${{ matrix.parameters }},conda --outdir ./results
# nextflow run ${GITHUB_WORKSPACE} -profile test,conda --outdir ./results
# if: "${{ github.base_ref == 'master' }}"
21 changes: 12 additions & 9 deletions .github/workflows/template_version_comment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,11 @@ jobs:
steps:
- name: Check out pipeline code
uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4
with:
ref: ${{ github.event.pull_request.head.sha }}

- name: Read template version from .nf-core.yml
uses: pietrobolcato/action-read-yaml@1.0.0
uses: nichmor/minimal-read-yaml@v0.0.2
id: read_yml
with:
config: ${{ github.workspace }}/.nf-core.yml
Expand All @@ -24,20 +26,21 @@ jobs:
- name: Check nf-core outdated
id: nf_core_outdated
run: pip list --outdated | grep nf-core
run: echo "OUTPUT=$(pip list --outdated | grep nf-core)" >> ${GITHUB_ENV}

- name: Post nf-core template version comment
uses: mshick/add-pr-comment@b8f338c590a895d50bcbfa6c5859251edc8952fc # v2
if: |
${{ steps.nf_core_outdated.outputs.stdout }} =~ 'nf-core'
contains(env.OUTPUT, 'nf-core')
with:
repo-token: ${{ secrets.NF_CORE_BOT_AUTH_TOKEN }}
allow-repeats: false
message: |
## :warning: Newer version of the nf-core template is available.
Your pipeline is using an old version of the nf-core template: ${{ steps.read_yml.outputs['nf_core_version'] }}.
Please update your pipeline to the latest version.
For more documentation on how to update your pipeline, please see the [nf-core documentation](https://github.com/nf-core/tools?tab=readme-ov-file#sync-a-pipeline-with-the-template) and [Synchronisation documentation](https://nf-co.re/docs/contributing/sync).
> [!WARNING]
> Newer version of the nf-core template is available.
>
> Your pipeline is using an old version of the nf-core template: ${{ steps.read_yml.outputs['nf_core_version'] }}.
> Please update your pipeline to the latest version.
>
> For more documentation on how to update your pipeline, please see the [nf-core documentation](https://github.com/nf-core/tools?tab=readme-ov-file#sync-a-pipeline-with-the-template) and [Synchronisation documentation](https://nf-co.re/docs/contributing/sync).
#
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,4 @@ results/
testing/
testing*
*.pyc
null/
2 changes: 1 addition & 1 deletion .nf-core.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ lint:
- .github/workflows/linting.yml
- .github/CONTRIBUTING.md
multiqc_config: false
nf_core_version: 3.0.0
nf_core_version: 3.0.2
org_path: null
repository_type: pipeline
template:
Expand Down
1 change: 0 additions & 1 deletion .prettierignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

email_template.html
adaptivecard.json
slackreport.json
Expand Down
9 changes: 0 additions & 9 deletions conf/igenomes_ignored.config

This file was deleted.

4 changes: 2 additions & 2 deletions modules.json
Original file line number Diff line number Diff line change
Expand Up @@ -50,12 +50,12 @@
"nf-core": {
"utils_nextflow_pipeline": {
"branch": "master",
"git_sha": "f533459a222ac53eb4c6bb7a5f574e4069197cdb",
"git_sha": "3aa0aec1d52d492fe241919f0c6100ebf0074082",
"installed_by": ["subworkflows"]
},
"utils_nfcore_pipeline": {
"branch": "master",
"git_sha": "f533459a222ac53eb4c6bb7a5f574e4069197cdb",
"git_sha": "1b6b9a3338d011367137808b49b923515080e3ba",
"installed_by": ["subworkflows"]
},
"utils_nfschema_plugin": {
Expand Down
2 changes: 1 addition & 1 deletion modules/nf-core/multiqc/environment.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions modules/nf-core/multiqc/main.nf

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

26 changes: 13 additions & 13 deletions modules/nf-core/multiqc/tests/main.nf.test.snap

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions nextflow.config
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,7 @@ params {
show_hidden = false
version = false
pipelines_testdata_base_path = 'https://raw.githubusercontent.com/nf-core/test-datasets/'

// Config options
config_profile_name = null
config_profile_description = null
Expand All @@ -115,6 +116,7 @@ params {
custom_config_base = "https://raw.githubusercontent.com/nf-core/configs/${params.custom_config_version}"
config_profile_contact = null
config_profile_url = null

// Schema validation default options
validate_params = true

Expand Down Expand Up @@ -254,6 +256,7 @@ includeConfig !System.getenv('NXF_OFFLINE') && params.custom_config_base ? "${pa

// Load nf-core/proteinfold custom profiles from different institutions.
includeConfig !System.getenv('NXF_OFFLINE') && params.custom_config_base ? "${params.custom_config_base}/pipeline/proteinfold.config" : "/dev/null"

// Set default registry for Apptainer, Docker, Podman, Charliecloud and Singularity independent of -profile
// Will not be used unless Apptainer / Docker / Podman / Charliecloud / Singularity are enabled
// Set to your registry if you have a mirror of containers
Expand Down
2 changes: 0 additions & 2 deletions subworkflows/local/utils_nfcore_proteinfold_pipeline/main.nf
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,6 @@ def toolCitationText() {
// Uncomment function in methodsDescriptionText to render in MultiQC report
def citation_text = [
"Tools used in the workflow included:",
"FastQC (Andrews 2010),",
"MultiQC (Ewels et al. 2016)",
"."
].join(' ').trim()
Expand All @@ -174,7 +173,6 @@ def toolBibliographyText() {
// Can use ternary operators to dynamically construct based conditions, e.g. params["run_xyz"] ? "<li>Author (2023) Pub name, Journal, DOI</li>" : "",
// Uncomment function in methodsDescriptionText to render in MultiQC report
def reference_text = [
"<li>Andrews S, (2010) FastQC, URL: https://www.bioinformatics.babraham.ac.uk/projects/fastqc/).</li>",
"<li>Ewels, P., Magnusson, M., Lundin, S., & Käller, M. (2016). MultiQC: summarize analysis results for multiple tools and samples in a single report. Bioinformatics , 32(19), 3047–3048. doi: /10.1093/bioinformatics/btw354</li>"
].join(' ').trim()

Expand Down
22 changes: 12 additions & 10 deletions subworkflows/nf-core/utils_nextflow_pipeline/main.nf

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit aebbd21

Please sign in to comment.