-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Closed
Closed
Copy link
Labels
enhancementNew feature or requestNew feature or request
Description
Is your feature request related to a problem or challenge?
I consolidated the content of our previous tickets about better statistics #10806 and #10806 into a new Epic and cleaned up the subtasks
Describe the solution you'd like
Subtasks:
- DataFusion ignores "column order" parquet statistics specification #10586
- Incorrect statistics read for struct array in parquet #10609
- Reduce test duplication in tests for data page stattistics #11000
- Improve performance of DataPage statistics extraction using StringBuilder #11281
- Incorrect statistics extracted from parquet data pages when all values are null #11280
- Update ListingTable to use
StatisticsConverter#10923 -
StatisticsConverter::row_group_null_countsincorrect for missing column #10926 - Support extracting
Int8,Int16,Int32statistics from Parquet Data Pages #10928 - Support extracting UInt{8, 16, 32, 64} statistics from Parquet Data Pages #10952
- Support
String/LargeStringandBinary/LargeBinaryParquet Data Page Statistics #11026 - Support
FixedSizedBinaryArrayParquet Data Page Statistics #11184 - Support
DictionaryArrayParquet Data Page Statistics #11185 - Support
BooleanParquet Data Page Statistics #11027 - Support
DecimalandDecimal256Parquet Data Page Statistics #11111 - Support
TimestampParquet Data Page Statistics #11112 - Support
DateParquet Data Page Statistics #11113 - Support
TimeParquet Data Page Statistics #11114 - Change
StatisticsConverter::row_group_countsto returnNonefor non existent columns in parquet files #10965 - Support extracting Float{16, 32, 64} statistics from Parquet Data Pages #10951
- Add a benchmark for extracting parquet data page statistics #10934
- Update the parquet code
prune_pages_in_one_row_groupto use theStatisticsExtractor#11480 - remove rendant tests in parquet statistics
- Port code / tests upstream: Add function that converts from parquet statistics
ParquetStatisticsto arrow arraysArrayRefarrow-rs#4328
Describe alternatives you've considered
No response
Additional context
No response
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request