Add an example of embedding indexes inside a parquet file (#16395)

* Add an example of embedding indexes inside a parquet file

* Add page image

* Add prune file example

* Fix clippy

* polish code

* Fmt

* address comments

* Add debug

* Add new example, but it will fail with page index

* add debug

* add debug

* polish

* debug

* Using low level API to support

* polish

* fix

* merge

* fix

* complte solution

* polish comments

* adjust image

* add comments part 1

* pin to new arrow-rs

* pin to new arrow-rs

* add comments part 2

* merge upstream

* merge upstream

* polish code

* Rename example and add it to the list

* Work on comments

* More documentation

* Documentation obession, encapsulate example

* Update datafusion-examples/examples/parquet_embedded_index.rs

Co-authored-by: Sherin Jacob <[email protected]>

---------

Co-authored-by: Andrew Lamb <[email protected]>
Co-authored-by: Sherin Jacob <[email protected]>

Author: 3 people

datafusion-examples/README.md

@@ -65,6 +65,7 @@ cargo run --example dataframe
 - [`flight_sql_server.rs`](examples/flight/flight_sql_server.rs): Run DataFusion as a standalone process and execute SQL queries from JDBC clients
 - [`function_factory.rs`](examples/function_factory.rs): Register `CREATE FUNCTION` handler to implement SQL macros
 - [`optimizer_rule.rs`](examples/optimizer_rule.rs): Use a custom OptimizerRule to replace certain predicates
+- [`parquet_embedded_index.rs`](examples/parquet_embedded_index.rs): Store a custom index inside a Parquet file and use it to speed up queries
 - [`parquet_encrypted.rs`](examples/parquet_encrypted.rs): Read and write encrypted Parquet files using DataFusion
 - [`parquet_index.rs`](examples/parquet_index.rs): Create an secondary index over several parquet files and use it to speed up queries
 - [`parquet_exec_visitor.rs`](examples/parquet_exec_visitor.rs): Extract statistics by visiting an ExecutionPlan after execution