-
Couldn't load subscription status.
- Fork 1k
Description
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Some code requires "logical nullability" which is distinct from the "physical" nullability (what is defined by an Arrow arrays NullBuffer, if it has one)
To this end, arrow includes Array::logical_nulls which will compute logical nulls.
The issue is that it also potentially copies the NullBuffer for arrays that have it pre-computed, such as PrimitiveArray and StringArray. While the copy is perhaps not that bad (it is several Arcs) I would like to avoid it entirely if possible
Describe the solution you'd like
I would like to change logical_nulls to return a Cow to avoid the copy, as I propose in coralogix/arrow-datafusion#221
Describe alternatives you've considered
We can not make any changes
Additional context
Thus came up in DataFusion PR apache/datafusion#8511
Logical nulls was added in #4691