-
Couldn't load subscription status.
- Fork 1.7k
Closed
Labels
bugSomething isn't workingSomething isn't working
Description
Describe the bug
Attempting to create a literal value which is an array of structs causes a panic.
To Reproduce
DataFusion CLI v34.0.0
❯ set datafusion.execution.parquet.allow_single_file_parallelism=false;
0 rows in set. Query took 0.001 seconds.
❯ copy (values (make_array(struct('foo',1)))) to '/tmp/test.parquet';
thread 'main' panicked at /home/dev/.cargo/registry/src/index.crates.io-6f17d22bba15001f/datafusion-common-34.0.0/src/scalar.rs:524:74:
called `Result::unwrap()` on an `Err` value: Internal("Unsupported data type in hasher: Struct([Field { name: \"c0\", data_type: Utf8, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }, Field { name: \"c1\", data_type: Int64, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }])")
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Expected behavior
Should not result in panic. If there is some issue with how I have attempted to construct the array of struct, we could add a more descriptive error message.
Additional context
I came across this while writing tests related to #8551, but this issue does not appear related to parallel parquet writing specifically.
The line of code generating the error implies an internal constraint is violated if we reach it.
jayzhan211
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working