-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Labels
enhancementNew feature or requestNew feature or request
Description
Is your feature request related to a problem or challenge?
As @goldmedal started trying to move the DynamicFileProvider so others could use it in #10745 I think it is clear that there is not a good way to add additional catalog support in the core without everything being intertwined.
Thus I think we should try and extract the different catalog providers out of datafusion core so it it easier
Describe the solution you'd like
I suggest the following final layout:
- traits like
CatalogProvider,SchemaProvider, etc in a new cratedatafusion-catalog(since these traits rely on table provider, etc I think this can't be indatafusion-commonordatafusion-expr) - The built in
Memory*providers are indatafusion-catalog - The bult in
InformationSchemaproviders are indatafusion-catalog - The newly proposed
DynamicFileCatalogindatafusion-catalog - (eventually) the LIstingTableProvider (which is by far the most complicated) moved to its own crate
datafusion-catalog-listing
Describe alternatives you've considered
No response
Additional context
No response
andygrove
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request