use nothing instead of Nullable

[KristofferC]

Not having to unpack every value you want to get at is quite convenient. I think this is worth it.

[KristofferC]

This is maybe too cute...

[ararslan]

I thought we got rid of the ability to do that on master? I think DataArrays calls this union Data{T} internally as a stopgap until we get T? in Base.

[ararslan]

If you use Void as I suggested, HTTP has this definition:

const Option{T} = Union{T, Void}

then defines internal fields and whatnot in terms of Option.

[ararslan]

Since null propagates through computations, it may be safer in this case to use Union{T, Void}.

[jrevels]

Since null propagates through computations, it may be safer in this case to use Union{T, Void}.

I had thought the Union{T, Null} approach was the blessed way forward for representing missing data (in the "this was supposed to return a value and didn't" sense)? It seems that if we have null, then using nothing for this purpose is a poor pun, since (AFAIK) nothing actually means "the return value of a function that intentionally doesn't return anything", and doesn't imply anything about unintentional missingness.

Apologies for my confusion - the whole Null story can be tough to follow if you're on the sidelines, so I've probably missed something along the way.

[ararslan]

The idea is that Union{T, Null} (or T? in 1.0) is the "data scientist's null," i.e. it behaves like NaN but for any type. This means that null is poisonous in that it propagates through computations. This is the behavior that someone working with statistically missing data in would expect. It's akin to NULL in SQL, NA in R, . in SAS, etc.

The other use case for missing data is the "software engineer's null," which does not propagate and is intended to stop computations by throwing a MethodError or similar to force the user to explicitly wrap and unwrap. This is useful for things like tryparse where a value may or may not have been produced. Currently we have Nullable for that, but it's likely that this will be written as Union{Some{T}, Void} in 1.0. This means that nothing is also used as a sentinel value of sorts. The type is akin to Option in Rust, std::option in C++17, nullable in C#, etc.

Admittedly the situation is quite confusing. I think in this case it would be good to follow the precedent set in HTTP, which uses nothing.

[jrevels]

Ah, that makes sense, thanks. You're right, the "software engineer's null" is definitely want we want here.

It seems like the best path forward, then, would be to avoid the nothing pun by having something like struct MissingReturnValue end (on which no methods are defined) and then use Union{T,MissingReturnValue}.

However, if the Null intelligentsia have decided that the nothing pun is the convention for this situation, than I humbly genuflect to their wisdom and will conform to the standard.

[KristofferC]

Updated this to use nothing instead of Null and rebased on #102.

Any thoughts about the choice between Union{T, Nothing}, Some{T, Nothing}, Union{T, GitHubNull}? @araslan, @jrevels?

[ararslan]

I don't think this is allowed on 0.7. IIRC Carlo Baldassi submitted a PR that removes the parsing of ? as an identifier.

[ararslan]

If you want a shorthand for Union{T, Nothing}, I recommend something like Maybe{T}.

[KristofferC]

Yeah, I'll change this.

[ararslan]

I would advise against adding another kind of null type, i.e. GitHubNull. Better to just translate GitHub's responses to existing Julia types. In terms of Union{T, Nothing} versus Union{Some{T}, Nothing}, the latter is useful to distinguish between the absence of a value and the presence of a nothing value. If a request returns something that does not have a particular field at all, that would be nothing, but if you know it returns some kind of null-like value, that would be better represented as Some(nothing). So I guess it depends a bit on what exactly the GitHub API provides.

[Keno]

Done on master by @staticfloat