Skip to content

Conversation

@alamb
Copy link
Contributor

@alamb alamb commented Feb 26, 2025

Which issue does this PR close?

Rationale for this change

While working on upgrading DataFusion 46 in several projects, I found the name source super confusing as there are two similarly named traits, FileSource and DataSource but several functions / fields are named just source so it is not clear which they refer to

This was especially confusing as DataSourceExec has a DataSource which is a FileScanConfig not the FileSOurce (which is a``ParquetSource` 🤯

What changes are included in this PR?

  1. rename the fields and accessors to be explicitly data_source or file_source to avoid the confusion

The documentation on how all these traits are related to each other is somewhat sparse. I am hoping to make a PR to improve it too

Are these changes tested?

By CI

Are there any user-facing changes?

While this is an API change, the releveant APIs were introduced in #14224 and thus not yet released, so I don't think this is explicitly a breaking change.

@github-actions github-actions bot added core Core DataFusion crate substrait Changes to the substrait crate proto Related to proto crate labels Feb 26, 2025
@alamb alamb force-pushed the alamb/rename_properties branch from c773b30 to 2834d58 Compare February 26, 2025 15:29
/// Return the source object
pub fn source(&self) -> &Arc<dyn DataSource> {
&self.source
pub fn data_source(&self) -> &Arc<dyn DataSource> {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is the actual API change -- the rest of the PR is making local variable names consistent

Copy link
Contributor

@berkaysynnada berkaysynnada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @alamb, making sense

@berkaysynnada berkaysynnada merged commit 1ae06a4 into apache:main Feb 27, 2025
25 checks passed
@alamb alamb deleted the alamb/rename_properties branch February 27, 2025 15:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate proto Related to proto crate substrait Changes to the substrait crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants