Skip to content

Deprecate values/keys/etc in variable length arrays like list/map/binary in favor of 2 functions actual_values and underlying_values or something #7992

@rluvaton

Description

@rluvaton

Slicing on a variable length arrays like list/map/binary, does not slice the underlying values but instead only slice the offsets due to performance.

However when wanting to get the underlying values for a list/map/binary, calling values (on list for example) give you the child values even the ones that are not referred to by the current slice

I've lost count on the number of times I saw or had bugs with getting the values instead of the the actual values that the list point to.

this is an example where it's counter intuitive

let list = ListArray::from_iter_primitive::<Int32Type, _, _>(vec![
    Some(vec![Some(1), Some(2)]),
    None,
    Some(vec![Some(3), None, Some(5)]),
]);

let list = list.slice(1, 2);

// [null, [3, null, 5]]
println!("{:?}", list);

// [1, 2, 3, null, 5]
println!("{:?}", list.values());

we added comments on list values that mention that The list array may not refer to all values in the values array ... but this is still not enough IMO.

also, creating 2 functions that explicitly state what is returned will force the developer to think there is a cost (for example, casting list to string, you should only cast the sliced underlying values and not the entire underlying values and then slice)


References to related changes/bugs:

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions