Cannot regenerate protobuf source code using instructions in the documentation

[andygrove]

Describe the bug

I'm trying to regenerate protobuf source code by running regen.sh but it fails with an error:

$ ./regen.sh 
    Finished dev [unoptimized + debuginfo] target(s) in 0.99s
     Running `/home/andy/git/apache/arrow-datafusion/target/debug/gen`
Error: "protobuf compilation failed: protoc failed: datafusion.proto: This file contains proto3 optional fields, but --experimental_allow_proto3_optional was not set.\n"

I am using libprotoc 3.12.4 as suggested by the docs.

I see a comment in datafusion.proto ... "optional" keyword is stable in protoc 3.15, so perhaps our docs are out of date and we now require 3.15? If so, we should update the docs.

To Reproduce

No response

Expected behavior

No response

Additional context

No response

[devinjdangelo]

I hit this while working on #8605

Adding .protoc_arg("--experimental_allow_proto3_optional") to datafusion/proto/gen/src/main.rs seems to fix the issue.

I am also using libprotoc 3.12.4, which appears to be the latest release available via apt on ubuntu 22.04.