Is your feature request related to a problem or challenge?
#16351 added modular encryption reading and writing. We should enable multi threaded encrypted writing.
Describe the solution you'd like
output_single_parquet_file_parallelized
in ParquetSink
should use encryption properties passed with WriterProperties
.
Describe alternatives you've considered
Current arrow-rs
API does not provide public API for passing FileEncryptor
(for security reasons) so no good alternatives were identified.
Additional context
This will require an upstream change in arrow-rs
that is currently in draft apache/arrow-rs#7818.