Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New module: checkqc #4158

Merged
merged 79 commits into from
Jul 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
79 commits
Select commit Hold shift + click to select a range
438d3b4
Add template module with nf-core with some modifications
matrulda Oct 16, 2023
7b2dbbf
checkqc: Removed TODOs, added tests (need improvement), copy files wh…
matrulda Oct 17, 2023
90538df
Use nf-tests for checkQC
matrulda Oct 18, 2023
dc76d6f
Change logic of mitohifi/mitohifi api (#4193)
ksenia-krasheninnikova Oct 28, 2023
7e2e138
Sentieon/tnfilter (#4208)
asp8200 Oct 29, 2023
d2c747d
pycoqc: add nf-test (#4238)
koenbossers Oct 30, 2023
7c4ffb4
Update seqkit/fx2tab to version 2.5.1 (#4240)
heuermh Oct 31, 2023
5de8431
FEAT: update bclconvert to 4.2.4 (#4253)
maxulysse Oct 31, 2023
c54c9d5
Add conda to renovate (#4216)
edmundmiller Oct 31, 2023
67aeaf5
chore(deps): update pre-commit hook python-jsonschema/check-jsonschem…
renovate[bot] Oct 31, 2023
a97fbad
chore(deps): update pre-commit hook pre-commit/mirrors-prettier to v3…
renovate[bot] Oct 31, 2023
26954e9
Fix conda declaration (#4252)
nvnieuwk Oct 31, 2023
3fd1004
chore(config): migrate renovate config (#4255)
renovate[bot] Oct 31, 2023
7a57477
chore(deps): update docker.io/continuumio/miniconda3 docker tag to v2…
renovate[bot] Oct 31, 2023
4805ef1
Fix linting test not being run for modules with only nf-test tests (#…
nvnieuwk Oct 31, 2023
9799d88
Nf-test rtgtools/rocplot (#4200)
nvnieuwk Oct 31, 2023
d307124
samtools/merge - emit crai file when cram file created (#4258)
gq1 Oct 31, 2023
1e5d77c
chore(deps): update biocontainers/chromap docker tag to v0.2.5 (#4214)
renovate[bot] Oct 31, 2023
c126028
Use "error" rather than "exit 1", and removed `quay.io/` (#4248)
muffato Oct 31, 2023
08604d0
chore(deps): update biocontainers/goat docker tag to v0.2.5 (#4259)
renovate[bot] Nov 1, 2023
10d6fa4
change database sufix to nin which is more common in v5 and v4 databa…
yumisims Nov 1, 2023
a3917ee
add nf-tests and snapshots for kallisto index & quant (#4250)
sateeshperi Nov 1, 2023
230f8a0
Update genomescope (#4267)
mahesh-panchal Nov 1, 2023
d133bca
nf-test interproscan (#4269)
vagkaratzas Nov 1, 2023
b9813bb
Update busco (#4270)
mahesh-panchal Nov 1, 2023
15815c7
chore(deps): update biocontainers/bowtie2 docker tag to v2.4.5 (#4212)
renovate[bot] Nov 1, 2023
eda0bd0
chore(deps): update biocontainers/checkm-genome docker tag to v1.2.2 …
renovate[bot] Nov 1, 2023
b9085d6
Mitohifi channel input (#4254)
ksenia-krasheninnikova Nov 2, 2023
2fee75d
chore(deps): Bump checkm-genome conda env (#4279)
edmundmiller Nov 2, 2023
689ed92
chore(deps): update biocontainers/htseq docker tag to v2.0.3 (#4266)
renovate[bot] Nov 2, 2023
77d971c
Fix up kallisto (#4271)
pinin4fjords Nov 2, 2023
f6c9cc8
FIX: More coherence in all gatk4spark modules (#4282)
maxulysse Nov 2, 2023
95332f6
update multiqc to 1.17 in custom/dumpsoftwareversions (#4295)
maxulysse Nov 3, 2023
3bd6b8f
Rename utility subworkflows (#4268)
adamrtalbot Nov 3, 2023
b601d83
Ucsc/bedgraphtobigwig nftest (#4136)
laramiellindsey Nov 3, 2023
89b54cf
Update missed renovate environments (#4296)
edmundmiller Nov 3, 2023
5256c9f
build: Add renovate comments to conda packages (#4298)
edmundmiller Nov 3, 2023
d9e2001
Fix conda and renovate (#4299)
edmundmiller Nov 3, 2023
46c5d28
Add renovate config check (#4297)
edmundmiller Nov 3, 2023
dd55c30
add arriba download (#4280)
peterpru Nov 5, 2023
1c0d47d
add agat/convertspgff2tsv module (#4265)
rannick Nov 6, 2023
c98065d
Bug: nf-test bam_sort_stats_samtools (#4190)
Joon-Klaps Nov 6, 2023
e87c53d
new module gridss GeneratePonBedpe (#4196)
kubranarci Nov 6, 2023
15068f2
3167 new module gridss gridss somatic filter (#4247)
kubranarci Nov 6, 2023
0bddc14
Added a meta map for the reference genome of minimap2 (#4285)
muffato Nov 6, 2023
0729b51
Add ngscheckmate bed file to test_data config (#4308)
SPPearce Nov 7, 2023
62322e3
Diamond tools: nf-test, container update, stubs added, makedb meta ad…
vagkaratzas Nov 7, 2023
f1f3a5f
blastp added choice for xml or tsv outputs other than the existing cs…
vagkaratzas Nov 7, 2023
ddff74a
Apply suggestions from code review
matrulda Nov 8, 2023
107dab6
Remove trailing white space
matrulda Nov 8, 2023
0e8d1eb
Merge branch 'master' into checkqc
matrulda Nov 8, 2023
f749839
Apply suggestions from code review
matrulda Nov 8, 2023
0ffb054
Handle Failed QC and add test for it
matrulda Nov 8, 2023
32e1808
Update modules/nf-core/checkqc/main.nf
matrulda Nov 8, 2023
c2ca108
Update modules/nf-core/checkqc/main.nf
matrulda Nov 8, 2023
51e3636
Update to CheckQC v4.0.1
matrulda Mar 19, 2024
55e9b98
Address lint errors and warnings
matrulda Mar 19, 2024
43d6d21
Merge branch 'master' into checkqc
matrulda Mar 19, 2024
048260b
Merge branch 'master' into checkqc
nvnieuwk May 14, 2024
3f660a0
update main.nf.test to new guidelines
nvnieuwk May 14, 2024
9789f75
using an env.yaml for conda
nvnieuwk May 14, 2024
7f2aeca
Merge branch 'master' into checkqc
SPPearce Jun 29, 2024
c9734d3
Merge branch 'master' into checkqc
fmalmeida Jul 12, 2024
659759a
add community wave container
atrigila Jul 17, 2024
e441e06
update snapshot
atrigila Jul 17, 2024
782454a
remove conda config
atrigila Jul 17, 2024
d30a189
add metamap
atrigila Jul 17, 2024
7eaf61a
update test and snapshot
atrigila Jul 17, 2024
fae13f3
Merge branch 'master' into checkqc
atrigila Jul 17, 2024
48da3f9
Merge branch 'master' into checkqc
fmalmeida Jul 17, 2024
5b584d8
add yml for tracking versions
atrigila Jul 18, 2024
70f9b8f
skip checkqc conda profile
fmalmeida Jul 18, 2024
0d07bfd
fix linting
atrigila Jul 18, 2024
d9b2e91
Update test.yml
maxulysse Jul 22, 2024
ba7bc1b
Update .github/workflows/test.yml
maxulysse Jul 22, 2024
d13ce00
Merge branch 'master' into checkqc
maxulysse Jul 22, 2024
0e9516f
Merge branch 'master' into checkqc
atrigila Jul 22, 2024
d2cc55d
add fixme note as requested in slack
atrigila Jul 22, 2024
80bdbac
Merge branch 'master' into checkqc
atrigila Jul 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -544,6 +544,8 @@ jobs:
path: modules/nf-core/cellranger/multi
- profile: conda
path: modules/nf-core/cellranger/vdj
- profile: conda
path: modules/nf-core/checkqc
- profile: conda
path: modules/nf-core/custom/dumpsoftwareversions
- profile: conda
Expand Down
11 changes: 11 additions & 0 deletions modules/nf-core/checkqc/environment.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
name: checkqc
channels:
- conda-forge
- bioconda
- defaults
dependencies:
- python=3.8
- pip
- pip: # FIXME https://github.com/nf-core/modules/issues/5814
- checkqc==3.8.0
- interop==1.2.4
50 changes: 50 additions & 0 deletions modules/nf-core/checkqc/main.nf
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
process CHECKQC {
tag "$meta.id"
label 'process_single'

container "community.wave.seqera.io/library/python_pip_interop_checkqc:d76c912c8fadc561"

input:
tuple val(meta), path(run_dir)
path(checkqc_config)

output:
tuple val(meta), path("*checkqc_report.json"), emit: report
path "versions.yml" , emit: versions

when:
task.ext.when == null || task.ext.when

script:
// Exit if running this module with -profile conda / -profile mamba
if (workflow.profile.tokenize(',').intersect(['conda', 'mamba']).size() >= 1) {
error "CheckQC module does not support Conda yet. Please use Docker / Singularity / Podman instead."
}

def args = task.ext.args ?: ''
def config = checkqc_config ? "--config $checkqc_config" : ''

"""
checkqc \
$args \
$config \
--json \
atrigila marked this conversation as resolved.
Show resolved Hide resolved
$run_dir > checkqc_report.json

cat <<-END_VERSIONS > versions.yml
"${task.process}":
checkqc: \$( checkqc --version | sed -e "s/checkqc, version //g" )
END_VERSIONS
"""

stub:
def args = task.ext.args ?: ''
"""
touch checkqc_report.json

cat <<-END_VERSIONS > versions.yml
"${task.process}":
checkqc: \$( checkqc --version | sed -e "s/checkqc, version //g" )
END_VERSIONS
"""
}
39 changes: 39 additions & 0 deletions modules/nf-core/checkqc/meta.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
---
# yaml-language-server: $schema=https://raw.githubusercontent.com/nf-core/modules/master/modules/yaml-schema.json
name: "checkqc"
description: A simple program to parse Illumina NGS data and check it for quality criteria
keywords:
- QC
- Illumina
- genomics
tools:
- "checkqc":
description: "A simple program to parse Illumina NGS data and check it for quality criteria."
homepage: "https://github.com/Molmed/checkQC"
documentation: "http://checkqc.readthedocs.io/en/latest/"
doi: "10.21105/joss.00556"
licence: ["GPL v3"]

input:
- run_dir:
type: file
description: |
Illumina sequencing run directory
Can be directory or a compressed tar (tar.gz) of the directory
- checkqc_config:
type: file
description: CheckQC configuration file
pattern: "*.{yml,yaml}"

output:
- versions:
type: file
description: File containing software versions
pattern: "versions.yml"
- report:
type: file
description: CheckQC report in json format
pattern: "*checkqc_report.json"

authors:
- "@matrulda"
44 changes: 44 additions & 0 deletions modules/nf-core/checkqc/tests/main.nf.test
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
nextflow_process {

name "Test Process CHECKQC"
script "../main.nf"
process "CHECKQC"
tag "modules"
tag "modules_nfcore"
tag "untar"
tag "checkqc"
matrulda marked this conversation as resolved.
Show resolved Hide resolved

test("homo_sapiens illumina test_interop_bcl2fastqstats_flowcell - QC OK") {
config "./nextflow.config"

setup {
run("UNTAR") {
script "../../untar/main.nf"
process {
"""
input[0] = Channel.of([[id: 'test'], file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/interop_bcl2fastqstats/test_flowcell_stats.tar.gz', checkIfExists: true)])
"""
}
}
}

when {
process {
"""
input[0] = UNTAR.out.untar
input[1] = []
"""
}
}

then {
assertAll(
{ assert process.success },
{ assert snapshot(process.out).match() }
)
}

}
}


35 changes: 35 additions & 0 deletions modules/nf-core/checkqc/tests/main.nf.test.snap
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
{
"homo_sapiens illumina test_interop_bcl2fastqstats_flowcell - QC OK": {
"content": [
{
"0": [
[
{
"id": "test"
},
"checkqc_report.json:md5,c69cb912a8c5bf53544764bd5bd73ae5"
]
],
"1": [
"versions.yml:md5,c37381fc450f4ed00da2987430db8177"
],
"report": [
[
{
"id": "test"
},
"checkqc_report.json:md5,c69cb912a8c5bf53544764bd5bd73ae5"
]
],
"versions": [
"versions.yml:md5,c37381fc450f4ed00da2987430db8177"
]
}
],
"meta": {
"nf-test": "0.8.4",
"nextflow": "24.04.3"
},
"timestamp": "2024-07-17T18:30:52.712542733"
}
}
5 changes: 5 additions & 0 deletions modules/nf-core/checkqc/tests/nextflow.config
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
process {
withName: CHECKQC {
ext.args = '--downgrade-errors ReadsPerSampleHandler'
}
}
2 changes: 2 additions & 0 deletions modules/nf-core/checkqc/tests/tags.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
checkqc:
- modules/nf-core/checkqc/**
2 changes: 2 additions & 0 deletions tests/config/test_data.config
Original file line number Diff line number Diff line change
Expand Up @@ -536,6 +536,8 @@ params {

varlociraptor_scenario = "${params.test_data_base}/data/genomics/homo_sapiens/illumina/varlociraptor/scenario.yml"

test_interop_bcl2fastqstats_flowcell = "${params.test_data_base}/data/genomics/homo_sapiens/illumina/interop_bcl2fastqstats/test_flowcell_stats.tar.gz"

contig_ploidy_priors_table = "${params.test_data_base}/data/genomics/homo_sapiens/illumina/gatk/contig_ploidy_priors_table.tsv"

purecn_ex1_bam = "${params.test_data_base}/data/genomics/homo_sapiens/illumina/purecn/purecn_ex1.bam"
Expand Down
Loading