Skip to content

Expose user defined functions in the FFI #14562

@timsaucer

Description

@timsaucer

Is your feature request related to a problem or challenge?

By adding the foreign function interface to this project, we exposed a host of features to end users. It allows for easy integration of DataFusion libraries across projects, specifically impacting datafusion-python.

I recommend that we work to expose scalar, aggregate, and window user defined functions via FFI. This is to support the issue apache/datafusion-python#1017

Describe the solution you'd like

Follow the existing pattern within the datafusion repository and expose these types of user defined functions.

  • scalar
  • aggregate
  • window
  • table

Note: The table functions are not currently in use in datafusion-python, but it would be a good addition.

Describe alternatives you've considered

For the datafusion-python project, we can work around via python calls but this would be a far more elegant approach in my opinion.

Additional context

Once we have these exposed, we could expose the function registry, which would get us very close to a FFI SessionContext. That would have impacts on the ballista and datafusion-ray projects.

Sub-issues

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions