Skip to content

Conversation

@NarineK
Copy link
Contributor

@NarineK NarineK commented Aug 21, 2016

What changes were proposed in this pull request?

Currently, once we create RelationalGroupedDataset, we cannot access the grouping columns from its instance.
Analog to Dataset we can have a public method which returns the list of grouping columns.
https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala#L457

This can be useful for instance in SparkR when we want to have certain logic associated with the grouping columns, accessible from RelationalGroupedDataset.


Similar to Dataset.columns I've added RelationalGroupedDataset.columns method which makes grouping column names accessible.

How was this patch tested?

Unit tests in DataFrameAggregateSuite

@SparkQA
Copy link

SparkQA commented Aug 21, 2016

Test build #64169 has finished for PR 14742 at commit 02d6bab.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@NarineK
Copy link
Contributor Author

NarineK commented Aug 22, 2016

cc: @shivaram, @liancheng

@NarineK
Copy link
Contributor Author

NarineK commented Sep 6, 2016

@liancheng, @rxin, Do you think adding columns to RelationalGroupedDataset is reasonable or should we find a workaround on R side ?

*
* @since 2.1.0
*/
def columns: Array[String] = {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might not be a column (a named expression). Thus, this might not be useful to end users.

@gatorsmile
Copy link
Member

Could we please close this PR now?

@SparkQA
Copy link

SparkQA commented Jun 14, 2017

Test build #78035 has started for PR 14742 at commit 02d6bab.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78035/
Test FAILed.

@NarineK
Copy link
Contributor Author

NarineK commented Jun 19, 2017

Hi @gatorsmile, #14431 depends on this. Is there a way I can access the grouping columns from RelationalGroupedDataset ?

@gatorsmile
Copy link
Member

Let us close this one. We can discuss how to resolve the issue in #14431

@NarineK
Copy link
Contributor Author

NarineK commented Jun 19, 2017

yes, we can close this, but it would be great if you could help us to find a way to access the grouping columns from SparkR in #14431

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants