Spec conflict?: `LoadListingRequirement` and `Directory.listing`

[tom-tan]

The spec of LoadListingEnum used in LoadListingRequirement says:

Specify the desired behavior for loading the listing field of a Directory object for use by expressions.

On the other hand, the spec of Directory says that:

If listing is not provided, the implementation must have some way of fetching the Directory listing at runtime based on the location field.

I am not sure I can say it a "conflict" but at least CWL v1.0, that has the same description of listing, implicitly requires the platforms loading the Directory listing recursively.

It would be nice if the section of Directory mentions loadListing and LoadListingRequirement.

[mr-c]

Thank you @tom-tan for spotting this. I would positively view a PR to https://github.com/common-workflow-language/cwl-v1.2/blob/main/Process.yml#L426 to do as you suggest.

[tom-tan]

OK, I will send a request later.

Can I clarify the spec about the listing to do that?
The spec says it is optional and its type is array<File | Directory>. Also, the type field of listing in the schema is (of course it is the same as the website of the spec):

    - name: listing
      type:
        - "null"
        - type: array
          items: [File, Directory]

I expect the listing field with null value in the case of no_listing but the conformance test #243 requires it should be undefined.
Does it have different interpretation from null or can we fix the tests by replacing such undefined with null?

[mr-c]

@tom-tan That's a good point:

What does each of the following mean for a class: Directory?

1. listing: null
2. listing: []
3. and no listing at all

The test at

cwl-v1.2/tests/listing_none2.cwl

Line 14 in f3ec4d1

outputEval: $(inputs.d.listing === undefined)

tests for the 3rd case (as that is how to detect a missing property in Javacsript) ; that is to say, it tests for an unspecified directory listing.

To me, case 1 and case 2 both mean that this Directory is empty. You are asking if case 1 (listing is present, but the value is an explicit null) should have the same meaning as case 3 (no listing at all): that the Directory is not necessarily empty, it is just unspecified.

[tom-tan]

Can I clarify the term "unspecified"?
Which is what you are intended in the comment?

1. listing property returns a platform-dependent value,
2. it behaves as listing property does not provided, or
3. it behaves as listing property has undefined value.

[tom-tan]

By the way,

To me, case 1 and case 2 both mean that this Directory is empty.

It makes me confused. For example, an input object param1: null is distinct from param1: [] in CWL spec (even if the resulted command line options are same). I prefer that listing: null and listing: [] have distinct interpretations because it makes easier to implement platforms especially with statically typed languages.

[mr-c]

Earlier I said

The test at

cwl-v1.2/tests/listing_none2.cwl

Line 14 in f3ec4d1

outputEval: $(inputs.d.listing === undefined)

tests for the 3rd case (as that is how to detect a missing property in Javacsript) ; that is to say, it tests for an unspecified directory listing.

@tom-tan asks

Can I clarify the term "unspecified"?
Which is what you are intended in the comment?

The CWL conformance test referenced above checks that the Directory object lacks (does not have) a property named "listing"
as loadListing: no_listing is written under LoadListingRequirement.

So we should clarify https://www.commonwl.org/v1.2/Workflow.html#LoadListingEnum to say

symbol	description
no_listing	Do not include a "listing" key in "Directory" objects.
shallow_listing	Include a "listing" key in "Directory" objects, but only include one level of contents; do not recurse into subdirectories.
deep_listing	Include a "listing" key in "Directory" objects and recursively load all subdirectories as well.

[mr-c]

By the way,

To me, case 1 and case 2 both mean that this Directory is empty.

It makes me confused. For example, an input object param1: null is distinct from param1: [] in CWL spec (even if the resulted command line options are same). I prefer that listing: null and listing: [] have distinct interpretations because it makes easier to implement platforms especially with statically typed languages.

We need explicit meanings for all three. I think we have good meanings for Case 2 (listing: [], this Directory is empty) and Case 3 (no listing attribute due to loadListing: no_listing, the contents of this Directory object are not available to CWL parameter references nor to CWL expressions).

What meaning would you ascribe to Case 1 (listing: null, the listing property is present on the Directory object and its value is a literal JSON/YAML null)?

I can't think of another useful meaning for Case 1. We can't forbid it in CWL v1.2, as null is explicitly allowed:

cwl-v1.2/Process.yml

Line 422 in e65a2de

- "null"

[mr-c]

On the other hand, the spec of Directory says that:

If listing is not provided, the implementation must have some way of fetching the Directory listing at runtime based on the location field.

I am not sure I can say it a "conflict" but at least CWL v1.0, that has the same description of listing, implicitly requires the platforms loading the Directory listing recursively.

It would be nice if the section of Directory mentions loadListing and LoadListingRequirement.

Agreed, there should be a mention in

cwl-v1.2/Process.yml

Line 425 in e65a2de

doc: |

about loadListing, the meaning of a null value versus leaving out the listing field entirely.

For example, if when loadListing: no_listing and a CWL expression adds a listing to a received class: Directory object and returns that modified object, then a new class: Directory needs to be made (perhaps at a new location) with the new listing contents, instead of the previous contents which were hidden.

This would make a good conformance test, I think.

[tom-tan]

What meaning would you ascribe to Case 1 (listing: null, the listing property is present on the Directory object and its value is a literal JSON/YAML null)?

I interpreted the following sentences in the spec

Specify the desired behavior for loading the listing field of a Directory object for use by expressions.

and

Do not load the directory listing.

as: listing field is loaded but its contents are not loaded (Important note: for statically typed programmer, it is hard to guess omitting the field itself unless it is clarified).

Also I misunderstood that the expression $(inputs.d.listing === undefined) represents "inputs.d.listing has undefined value" because I am not familiar with the detail of JavaScript.
IMO it is hard for non-JavaScript programmers to guess this behavior because it seems to compare the value rather than to check the existence of the field.

---

I can't think of another useful meaning for Case 1. We can't forbid it in CWL v1.2, as null is explicitly allowed:

Unfortunately it is hard to fix it, I guess.
The SALAD (and CWL spec that depends on SALAD) uses the null type in the following distinct meanings:

• the field has null value, and
• the field itself is omitted (it is the intention of listing that enables users to use Directory object without listing field in the input object)

As a result of that, schema-salad-tool passes the following CWL document:

#!/usr/bin/env cwl-runner
class: CommandLineTool
cwlVersion: v1.2
requirements:
  LoadListingRequirement:
    loadListing: no_listing
  InlineJavascriptRequirement: {}
inputs:
  d: Directory
outputs:
  out:
    type: boolean
    outputBinding:
      outputEval: $(inputs.d.listing === undefined)
baseCommand: "true"
hints: null   # !!!!????

$ schema-salad-tool CommonWorkflowLanguage.yml tests/listing_none2.cwl 
/usr/local/bin/schema-salad-tool Current version: 7.0.20201119201711
Document `tests/listing_none2.cwl` is valid

I'm not sure it is a limitation of SALAD itself or it comes from JSON-LD or Avro.

[tom-tan]

IMO treating listing: null as "listing field is missing" is reasonable in the current situation because it needs less implementation changes (SALAD already do that as explained above) and is consistent with other parts of the spec of CWL such as omitting rules for input and output parameters.

[mr-c]

@tetron Do you think that listing: null is equivalent to "listing field is not present"?

[tetron]

I agree. We've been trying to avoid the distinction between "null" and "undefined" in the data model, so things that are "undefined" are considered "null". That's also a much better policy for code generation where the fields already exist but can have a "null" value.