Singularity version extraction breaks with singularity-ce

[jmfernandez]

Expected Behavior

When singularity-ce is installed following instructions at https://sylabs.io/guides/3.8/user-guide/quick_start.html#download-singularityce-from-a-release) and cwltool is ran in singularity mode, it should work.

Actual Behavior

When singularity-ce is installed and cwltool is ran in singularity mode, an unexpected exception arises. Our undergraduate student @Acivico found it when he was installing in his computer the prerequisites for WfExS-backend.

Workflow Code

As this issue is after workflow parsing but before workflow execution, when Singularity version is being detected, any minimal workflow fires it. For instance:

#!/usr/bin/env cwl-runner

cwlVersion: v1.0
class: CommandLineTool
requirements:
  DockerRequirement:
    dockerPull: 'alpine:3.9'
baseCommand: echo
inputs:
  message:
    type: string
    inputBinding:
      position: 1
outputs: []

Full Traceback

• Command line: cwltool --debug --singularity echo.cwl --message world

INFO /tmp/bb/s/bin/cwltool 3.1.20211004060744
INFO Resolved 'echo.cwl' to 'file:///tmp/bb/echo.cwl'
DEBUG Parsed job order from command line: {
    "id": "echo.cwl",
    "message": "world"
}
DEBUG [job echo.cwl] initializing from file:///tmp/bb/echo.cwl
DEBUG [job echo.cwl] {
    "message": "world"
}
DEBUG [job echo.cwl] path mappings is {}
DEBUG [job echo.cwl] command line bindings is [
    {
        "position": [
            -1000000,
            0
        ],
        "datum": "echo"
    },
    {
        "position": [
            1,
            "message"
        ],
        "datum": "world"
    }
]
DEBUG Singularity version: singularity-ce version 3.8.1.
DEBUG Singularity error
Traceback (most recent call last):
  File "/tmp/bb/s/lib/python3.8/site-packages/cwltool/job.py", line 774, in run
    self.get_from_requirements(
  File "/tmp/bb/s/lib/python3.8/site-packages/cwltool/singularity.py", line 249, in get_from_requirements
    if not self.get_image(cast(Dict[str, str], r), pull_image, force_pull):
  File "/tmp/bb/s/lib/python3.8/site-packages/cwltool/singularity.py", line 113, in get_image
    if is_version_3_or_newer():
  File "/tmp/bb/s/lib/python3.8/site-packages/cwltool/singularity.py", line 43, in is_version_3_or_newer
    return int(get_version()[0]) >= 3
ValueError: invalid literal for int() with base 10: 's'
ERROR Workflow or tool uses unsupported feature:
Singularity is required to run this tool: invalid literal for int() with base 10: 's'
Traceback (most recent call last):
  File "/tmp/bb/s/lib/python3.8/site-packages/cwltool/job.py", line 774, in run
    self.get_from_requirements(
  File "/tmp/bb/s/lib/python3.8/site-packages/cwltool/singularity.py", line 249, in get_from_requirements
    if not self.get_image(cast(Dict[str, str], r), pull_image, force_pull):
  File "/tmp/bb/s/lib/python3.8/site-packages/cwltool/singularity.py", line 113, in get_image
    if is_version_3_or_newer():
  File "/tmp/bb/s/lib/python3.8/site-packages/cwltool/singularity.py", line 43, in is_version_3_or_newer
    return int(get_version()[0]) >= 3
ValueError: invalid literal for int() with base 10: 's'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/tmp/bb/s/lib/python3.8/site-packages/cwltool/main.py", line 1247, in main
    (out, status) = real_executor(
  File "/tmp/bb/s/lib/python3.8/site-packages/cwltool/executors.py", line 60, in __call__
    return self.execute(process, job_order_object, runtime_context, logger)
  File "/tmp/bb/s/lib/python3.8/site-packages/cwltool/executors.py", line 143, in execute
    self.run_jobs(process, job_order_object, logger, runtime_context)
  File "/tmp/bb/s/lib/python3.8/site-packages/cwltool/executors.py", line 250, in run_jobs
    job.run(runtime_context)
  File "/tmp/bb/s/lib/python3.8/site-packages/cwltool/job.py", line 819, in run
    raise UnsupportedRequirement(
cwltool.errors.UnsupportedRequirement: Singularity is required to run this tool: invalid literal for int() with base 10: 's'

Your Environment

• cwltool version: 3.1.20211004060744
  Check using cwltool --version

Additional notes

The bug is at

cwltool/cwltool/singularity.py

Lines 26 to 43 in aec33fc

	def get_version() -> str:
	global _SINGULARITY_VERSION # pylint: disable=global-statement
	if _SINGULARITY_VERSION == "":
	_SINGULARITY_VERSION = check_output( # nosec
	["singularity", "--version"], universal_newlines=True
	).strip()
	if _SINGULARITY_VERSION.startswith("singularity version "):
	_SINGULARITY_VERSION = _SINGULARITY_VERSION[20:]
	_logger.debug(f"Singularity version: {_SINGULARITY_VERSION}.")
	return _SINGULARITY_VERSION
	def is_version_2_6() -> bool:
	return get_version().startswith("2.6")
	def is_version_3_or_newer() -> bool:
	return int(get_version()[0]) >= 3

, where the singularity version extraction is too rigid

[mr-c]

Thanks @jmfernandez and @Acivico for your report. Can you share the output of singularity version with singularity-ce?

[jmfernandez]

Yes, of course!

(s) jmfernandez@olympo[30]:/tmp/bb> singularity version
3.8.1
(s) jmfernandez@olympo[31]:/tmp/bb> singularity --version
singularity-ce version 3.8.1

[mr-c]

Thanks @jmfernandez! Can you confirm the fix in #1547 ?

[jmfernandez]

Yes, I can confirm you that it is now working with singularity-ce