Skip to content

Unsupported functions in protobuf serializatin #8098

@alamb

Description

@alamb

Describe the bug

Inspired by https://github.com/apache/arrow-datafusion/pull/8089/files from @Syleechan I checked if there were other functions not covered by our serialization logic. It turns out that yes there are:

To Reproduce

Remove the catchall in parse_expr:

diff --git a/datafusion/proto/src/logical_plan/from_proto.rs b/datafusion/proto/src/logical_plan/from_proto.rs
index cdb0fe9bda..52a8480146 100644
--- a/datafusion/proto/src/logical_plan/from_proto.rs
+++ b/datafusion/proto/src/logical_plan/from_proto.rs
@@ -1645,9 +1645,6 @@ pub fn parse_expr(
                 )),
                 ScalarFunction::Isnan => Ok(isnan(parse_expr(&args[0], registry)?)),
                 ScalarFunction::Iszero => Ok(iszero(parse_expr(&args[0], registry)?)),
-                _ => Err(proto_error(
-                    "Protobuf deserialization error: Unsupported scalar function",
-                )),
             }
         }
         ExprType::ScalarUdfExpr(protobuf::ScalarUdfExprNode { fun_name, args }) => {

The compiler will tell you there are several more functions not covered *like ToTimestamp and ArrowTypeOf:

error[E0004]: non-exhaustive patterns: `datafusion::ScalarFunction::ToTimestamp`, `datafusion::ScalarFunction::StructFun`, `datafusion::ScalarFunction::ArrowTypeof` and 2 more not covered
    --> datafusion/proto/src/logical_plan/from_proto.rs:1303:19
     |
1303 |             match scalar_function {
     |                   ^^^^^^^^^^^^^^^ patterns `datafusion::ScalarFunction::ToTimestamp`, `datafusion::ScalarFunction::StructFun`, `datafusion::ScalarFunction::ArrowTypeof` and 2 more not covered
     |
note: `datafusion::ScalarFunction` defined here

What this means is that if a user tries to serialize a plan with to_timestamp or arrow_typeof it will error with "Protobuf deserialization error"

Expected behavior

There should be no error

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinggood first issueGood for newcomers

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions