singularity.cacheDir does not work properly with containers that have ':' in the filename

[stevekm]

Bug report

with my nextflow.config like this;

singularity {
    enabled = true
    autoMounts = true
    cacheDir = "/fsx/home/tools/containers"
}

process {
    executor = 'slurm'
    container = 'nextflow_bash:latest.sif'

    withName: DebugScript {
        container = 'pytorch_pytorch:latest.sif'
    }
}

The required containers exist as such;

$ ll /fsx/home/tools/containers/nextflow_bash:latest.sif
-rw-r--r-- 1 username mygroup 29M Sep 23 16:44 /fsx/home/tools/containers/nextflow_bash:latest.sif

$ ll /fsx/home/tools/containers/pytorch_pytorch:latest.sif
-rw-r--r-- 1 username mygroup 3.5G Sep 23 17:39 /fsx/home/tools/containers/pytorch_pytorch:latest.sif

However, when you run Nextflow, it breaks with an error such as

Pulling Singularity image docker://nextflow_bash:latest.sif [cache /fsx/home/tools/containers/nextflow_bash-latest.sif]
Pulling Singularity image docker://pytorch_pytorch:latest.sif [cache /fsx/home/tools/containers/pytorch_pytorch-latest.sif]
ERROR ~ Error executing process > 'DebugScript'

Caused by:
  Failed to pull singularity image
    command: singularity pull  --name pytorch_pytorch-latest.sif.pulling.1727127729010 docker://pytorch_pytorch:latest.sif > /dev/null
    status : 255

As you can see from the log and error messages, the cacheDir directive is not correctly detecting the files nextflow_bash:latest.sif and pytorch_pytorch:latest.sif, and is instead looking for the files nextflow_bash-latest.sif and pytorch_pytorch-latest.sif which do not exist.

Is this a bug? This behavior is not described in the docs ( https://www.nextflow.io/docs/latest/config.html#scope-singularity ), and I have not encountered this issue in the past with filenames that lack ':'. Considering that our files are on a soon-to-be S3 backed storage system, having to constantly rename the .sif files to match this unexpected different filename is a little bothersome.  

Environment

• Nextflow version: nextflow version 24.04.4.5917

[bentsherman]

Looks like it is intentional:

nextflow/modules/nextflow/src/main/groovy/nextflow/container/SingularityCache.groovy

Lines 76 to 101 in 33fab52

	/**
	* Given a remote image URL string returns a normalised name used to store
	* the image in the local file system
	*
	* @param imageUrl
	* A Singularity remote image url. It must be prefixed with a pseudo-protocol supported by
	* by the underlying singularity tool eg. {@code docker://pditommaso/foo:latest}
	* @return
	* A file name corresponding to the image URL eg. {@code pditommaso-foo.img}
	*/
	@PackageScope
	String simpleName(String imageUrl) {
	def p = imageUrl.indexOf('://')
	def name = p != -1 ? imageUrl.substring(p+3) : imageUrl
	String extension = '.img'
	if( name.contains('.sif:') ) {
	extension = '.sif'
	name = name.replace('.sif:','-')
	}
	else if( name.endsWith('.sif') ) {
	extension = '.sif'
	name = name.substring(0,name.length()-4)
	}
	name = name.replace(':','-').replace('/','-')
	return name + extension
	}

[stevekm]

Ok. Any ideas where this convention originated? Is this a Singularity convention or something that was implemented independently on Nextflow-side?

We are building our .sif files via a scripted method, so I will have to go back and update my script to use this naming scheme instead.. and re-name all of our existing .sif files, I guess.

Not sure if there's any room for flexibility or customization for how nextflow should expect the .sif filename to look