Skip to content

Avoid forced copy in Array::logical_nulls #5208

@alamb

Description

@alamb

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Some code requires "logical nullability" which is distinct from the "physical" nullability (what is defined by an Arrow arrays NullBuffer, if it has one)

To this end, arrow includes Array::logical_nulls which will compute logical nulls.

The issue is that it also potentially copies the NullBuffer for arrays that have it pre-computed, such as PrimitiveArray and StringArray. While the copy is perhaps not that bad (it is several Arcs) I would like to avoid it entirely if possible

Describe the solution you'd like
I would like to change logical_nulls to return a Cow to avoid the copy, as I propose in coralogix/arrow-datafusion#221

Describe alternatives you've considered
We can not make any changes

Additional context
Thus came up in DataFusion PR apache/datafusion#8511

Logical nulls was added in #4691

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementAny new improvement worthy of a entry in the changelog

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions