Skip to content

Parquet: coerce_int96 does not work for int96 in nested types #15763

@mbutrovich

Description

@mbutrovich

Describe the bug

The logic that coerces timestamps to a different resolution iterates through fields and uses their key in the Parquet schema as a key to match against the Arrow schema.

pub fn coerce_int96_to_resolution(

However, this is insufficient (consider a schema with structs each with an id field) to disambiguate nested fields. I think we might need a combination of Parquet ColumnDesc's path and a normalized Arrow schema to fix the mapping.

To Reproduce

CometFuzzTestSuite for INT96 reproduces the issue immediately. I will work on including an slt test in the fix PR.

Expected behavior

No response

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions