-
Notifications
You must be signed in to change notification settings - Fork 28.9k
[SPARK-11504][SQL] API audit for distributeBy and localSort #9470
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
rxin
commented
Nov 4, 2015
- Renamed localSort -> sortWithinPartitions to avoid ambiguity in "local"
- distributeBy -> repartition to match the existing repartition.
|
LGTM pending jenkins. |
|
test this please |
|
Test build #45030 has finished for PR 9470 at commit
|
|
Merging to master. |
|
Test build #45031 has finished for PR 9470 at commit
|
1. Renamed localSort -> sortWithinPartitions to avoid ambiguity in "local" 2. distributeBy -> repartition to match the existing repartition. Author: Reynold Xin <[email protected]> Closes apache#9470 from rxin/SPARK-11504.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@rxin This causes an infinite loop, which isn't caught by the unit tests since DataFrameSuite only tests the Column* overload.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ankurdave can you create a jira?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@yhuai SPARK-12298 and #10271