From bea62d911e77c4febc08a4b5321be1df0683f96c Mon Sep 17 00:00:00 2001 From: vladislav doster Date: Mon, 16 Aug 2021 04:20:03 -0500 Subject: [PATCH 1/2] (docs): update README.rst Updates include the following: - correct pronoun agreement - correct spelling - correct grammar - add & remove punctuation - remove verbiage - increase readability Signed-off-by: Vladislav Doster --- README.rst | 80 ++++++++++++++++++++++++++---------------------------- 1 file changed, 39 insertions(+), 41 deletions(-) diff --git a/README.rst b/README.rst index d04cd2d94..a02afdd67 100644 --- a/README.rst +++ b/README.rst @@ -68,14 +68,14 @@ and similar Linux distribution try sudo apt-get install cwltool -If you are running MacOS X or other UNIXes and you want to use packages prepared by the conda-forge project, then +If you are running macOS X or other UNIXes and you want to use packages prepared by the conda-forge project, then please follow the install instructions for `conda-forge `_ (if you haven't already) and then .. code:: bash conda install -c conda-forge cwltool -All of the above methods of installing ``cwltool`` use packages which might contain bugs already fixed in newer versions, or be missing features that you desire. +All of the above methods of installing ``cwltool`` use packages that might contain bugs already fixed in newer versions or be missing desired features. If the packaged version of ``cwltool`` available to you is too old, then we recommend installing using ``pip`` and ``venv`` .. code:: bash @@ -117,7 +117,7 @@ Or you can skip the direct ``pip`` commands above and install the latest develop pip install .[deps] # Installs ``cwltool`` from source cwltool --version # Check if the installation works correctly -Remember, if co-installing multiple CWL implementations then you need to +Remember, if co-installing multiple CWL implementations, then you need to maintain which implementation ``cwl-runner`` points to via a symbolic file system link or `another facility `_. @@ -149,7 +149,7 @@ these will be inserted at the beginning of the command line:: Use with boot2docker on macOS ----------------------------- -boot2docker runs Docker inside a virtual machine and it only mounts ``Users`` +boot2docker runs Docker inside a virtual machine, and it only mounts ``Users`` on it. The default behavior of CWL is to create temporary directories under e.g. ``/Var`` which is not accessible to Docker containers. @@ -162,14 +162,14 @@ Using uDocker ------------- Some shared computing environments don't support Docker software containers for technical or policy reasons. -As a work around, the CWL reference runner supports using alternative ``docker`` implementations on Linux +As a workaround, the CWL reference runner supports using alternative ``docker`` implementations on Linux with the ``--user-space-docker-cmd`` option. -One such "user space" friendly docker replacement is ``udocker`` https://github.com/indigo-dc/udocker +One such "user space" friendly docker replacement is ``udocker`` https://github.com/indigo-dc/udocker. udocker installation: https://github.com/indigo-dc/udocker/blob/master/doc/installation_manual.md#22-install-from-udockertools-tarball -Run `cwltool` just as you normally would, but with the new option, e.g. from the conformance tests +Run `cwltool` just as you usually would, but with the new option, e.g., from the conformance tests .. code:: bash @@ -197,8 +197,8 @@ Running a tool or workflow from remote or local locations systems via its support for HTTP[S] URLs. Input job files and Workflow steps (via the `run` directive) can reference CWL -documents using absolute or relative local filesytem paths. If a relative path -is referenced and that document isn't found in the current directory then the +documents using absolute or relative local filesystem paths. If a relative path +is referenced and that document isn't found in the current directory, then the following locations will be searched: http://www.commonwl.org/v1.0/CommandLineTool.html#Discovering_CWL_documents_on_a_local_filesystem @@ -227,7 +227,7 @@ workflow step, or command line tool) to the process requirements that should be Overrides can be specified either on the command line, or as part of the job input document. Workflow steps are identified using the name of the workflow file followed by the step name as a document fragment identifier "#id". -Override identifiers are relative to the toplevel workflow document. +Override identifiers are relative to the top-level workflow document. .. code:: bash @@ -258,7 +258,7 @@ referenced by a workflow and builds a new CWL document with all Process objects (CommandLineTool and Workflow) in a list in the ``$graph`` field. Cross references (such as ``run:`` and ``source:`` fields) are updated to internal references within the new packed -document. The top level workflow is named ``#main``. +document. The top-level workflow is named ``#main``. .. code:: bash @@ -270,7 +270,7 @@ Running only part of a workflow You can run a partial workflow with the ``--target`` (``-t``) option. This takes the name of an output parameter, workflow step, or input -parameter in the top level workflow. You may provide multiple +parameter in the top-level workflow. You may provide multiple targets. .. code:: bash @@ -280,11 +280,11 @@ targets. If a target is an output parameter, it will only run only the steps that contribute to that output. If a target is a workflow step, it will run the workflow starting from that step. If a target is an -input parameter, it will only run only the steps that are connected to +input parameter, it will only run the steps connected to that input. Use ``--print-targets`` to get a listing of the targets of a workflow. -To see exactly which steps will run, use ``--print-subgraph`` with +To see which steps will run, use ``--print-subgraph`` with ``--target`` to get a printout of the workflow subgraph for the selected targets. @@ -318,7 +318,7 @@ Environment Variables in cwltool -------------------------------- This reference implementation supports several ways of setting -enviroment variables for tools, in addition to the standard +environment variables for tools, in addition to the standard ``EnvVarRequirement``. The sequence of steps applied to create the enviroment is: @@ -413,7 +413,7 @@ example does require an existing module setup so it is impossible to test this e "out of the box" with cwltool. For a more isolated test that demonstrates all the same concepts - the resolver plugin type ``galaxy_packages`` can be used. -"Galaxy packages" are a lighter weight alternative to Environment Modules that are +"Galaxy packages" are a lighter-weight alternative to Environment Modules that are really just defined by a way to lay out directories into packages and versions to find little scripts that are sourced to modify the environment. They have been used for years in Galaxy community to adapt Galaxy tools to cluster @@ -463,12 +463,12 @@ The resolvers configuration file in the above example was simply: It is possible that the ``SoftwareRequirement`` s in a given CWL tool will not match the module names for a given cluster. Such requirements can be re-mapped -to specific deployed packages and/or versions using another file specified using +to specific deployed packages or versions using another file specified using the resolver plugin parameter `mapping_files`. We will -demonstrate this using `galaxy_packages` but the concepts apply equally well -to Environment Modules or Conda packages (described below) for instance. +demonstrate this using `galaxy_packages,` but the concepts apply equally well +to Environment Modules or Conda packages (described below), for instance. -So consider the resolvers configuration file +So consider the resolvers configuration file. (`tests/test_deps_env_resolvers_conf_rewrite.yml`): .. code:: yaml @@ -505,18 +505,17 @@ support for Homebrew/Linuxbrew plugins is available, the most developed such plugin is for the `Conda `__ package manager. Conda has the nice properties of allowing multiple versions of a package to be installed simultaneously, not requiring evaluated permissions to install Conda itself or packages using -Conda, and being cross platform. For these reasons, cwltool may run as a normal +Conda, and being cross-platform. For these reasons, cwltool may run as a normal user, install its own Conda environment and manage multiple versions of Conda packages -on both Linux and Mac OS X. +on Linux and Mac OS X. The Conda plugin can be endlessly configured, but a sensible set of defaults that has proven a powerful stack for dependency management within the Galaxy tool development ecosystem can be enabled by simply passing cwltool the ``--beta-conda-dependencies`` flag. -With this we can use the seqtk example above without Docker and without -any externally managed services - cwltool should install everything it needs -and create an environment for the tool. Try it out with the follwing command:: +With this, we can use the seqtk example above without Docker or any externally managed services - cwltool should install everything it needs +and create an environment for the tool. Try it out with the following command:: cwltool --beta-conda-dependencies tests/seqtk_seq.cwl tests/seqtk_seq_job.json @@ -543,7 +542,7 @@ The example can be executed using the command:: cwltool --beta-conda-dependencies tests/seqtk_seq_wrong_name.cwl tests/seqtk_seq_job.json -The plugin framework for managing resolution of these software requirements +The plugin framework for managing the resolution of these software requirements as maintained as part of `galaxy-tool-util `__ - a small, portable subset of the Galaxy project. More information on configuration and implementation can be found at the following links: @@ -603,9 +602,9 @@ in an integer). For example:: Interaction with containers: the MPIRequirement currently prepends its commands to the front of the command line that is constructed. If you -wish to run a containerised application in parallel, for simple use -cases this does work with Singularity, depending upon the platform -setup. However this combination should be considered "alpha" -- please +wish to run a containerized application in parallel, for simple use +cases, this does work with Singularity, depending upon the platform +setup. However, this combination should be considered "alpha" -- please do report any issues you have! This does not work with Docker at the moment. (More precisely, you get `n` copies of the same single process image run at the same time that cannot communicate with each other.) @@ -633,8 +632,8 @@ given in the following table; all are optional. | | | | variables that should be | | | | | passed from the host | | | | | environment through to the | -| | | | tool (e.g. giving the | -| | | | nodelist as set by your | +| | | | tool (e.g., giving the | +| | | | node list as set by your | | | | | scheduler). | +----------------+------------------+----------+------------------------------+ | env_pass_regex | List[str] | [] | A list of python regular | @@ -667,8 +666,8 @@ To run the basic tests after installing `cwltool` execute the following: pip install -rtest-requirements.txt pytest -To run various tests in all supported Python environments we use `tox `_. To run the test suite in all supported Python environments -first downloading the complete code repository (see the ``git clone`` instructions above) and then run +To run various tests in all supported Python environments, we use `tox `_. To run the test suite in all supported Python environments +first clone the complete code repository (see the ``git clone`` instructions above) and then run the following in the terminal: ``pip install tox; tox -p`` @@ -685,7 +684,7 @@ The GitHub repository for the CWL specifications contains a script that tests a implementation against a wide array of valid CWL files using the `cwltest `_ program -Instructions for running these tests can be found in the Common Workflow Language Specification repository at https://github.com/common-workflow-language/common-workflow-language/blob/main/CONFORMANCE_TESTS.md +Instructions for running these tests can be found in the Common Workflow Language Specification repository at https://github.com/common-workflow-language/common-workflow-language/blob/main/CONFORMANCE_TESTS.md . Import as a module ------------------ @@ -721,7 +720,7 @@ Technical outline of how cwltool works internally, for maintainers. #. Fetches the document from file or URL #. Applies preprocessing (syntax/identifier expansion and normalization) #. Validates the document based on cwlVersion - #. If necessary, updates the document to latest spec + #. If necessary, updates the document to the latest spec #. Constructs a Process object using ``make_tool()``` callback. This yields a CommandLineTool, Workflow, or ExpressionTool. For workflows, this recursively constructs each workflow step. @@ -736,7 +735,7 @@ Technical outline of how cwltool works internally, for maintainers. there is currently no work ready to run) or end of iteration (indicating the process is complete.) #. Invoke the runnable item by calling ``run()``. This runs the tool and gets output. - #. Output of a process is reported by an output callback. + #. An output callback reports the output of a process. #. ``job()`` may be iterated over multiple times. It will yield all the work that is currently ready to run and then yield None. @@ -746,12 +745,11 @@ Technical outline of how cwltool works internally, for maintainers. inputs the step are ready. #. When a step is ready, it constructs an input object for that step and iterates on the ``job()`` method of the workflow job step. - #. Each runnable item is yielded back up to top level run loop + #. Each runnable item is yielded back up to top-level run loop #. When a step job completes and receives an output callback, the job outputs are assigned to the output of the workflow step. #. When all steps are complete, the intermediate files are moved to a final - workflow output, intermediate directories are deleted, and the output - callback for the workflow is called. + workflow output, intermediate directories are deleted, and the workflow's output callback is called. #. ``CommandLineTool`` job() objects yield a single runnable object. @@ -784,7 +782,7 @@ executor executor(tool, job_order_object, runtimeContext, logger) (Process, Dict[Text, Any], RuntimeContext) -> Tuple[Dict[Text, Any], Text] - An implementation of the toplevel workflow execution loop, should + An implementation of the top-level workflow execution loop should synchronously run a process object to completion and return the output object. @@ -821,7 +819,7 @@ resolver resolver(document_loader, document) (Loader, Union[Text, dict[Text, Any]]) -> Text - Resolve a relative document identifier to an absolute one which can be fetched. + Resolve a relative document identifier to an absolute one that can be fetched. The following functions can be set in RuntimeContext to override or augment the listed behaviors. From 8c960b618cce06957bf0914eff1de2c950e7058d Mon Sep 17 00:00:00 2001 From: "Michael R. Crusoe" <1330696+mr-c@users.noreply.github.com> Date: Sun, 29 Aug 2021 11:46:12 +0200 Subject: [PATCH 2/2] fix table formatting --- README.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.rst b/README.rst index 7cd0a4f45..1973cf38e 100644 --- a/README.rst +++ b/README.rst @@ -637,8 +637,8 @@ given in the following table; all are optional. | | | | variables that should be | | | | | passed from the host | | | | | environment through to the | -| | | | tool (e.g., giving the | -| | | | node list as set by your | +| | | | tool (e.g., giving the | +| | | | node list as set by your | | | | | scheduler). | +----------------+------------------+----------+------------------------------+ | env_pass_regex | List[str] | [] | A list of python regular |