Skip to content

[SPARK-12393] [SparkR] Add read.text and write.text for SparkR #10348

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 4 commits into from
Closed

[SPARK-12393] [SparkR] Add read.text and write.text for SparkR #10348

wants to merge 4 commits into from

Conversation

@yanboliang
Copy link
Contributor

Add read.text and write.text for SparkR.
cc @sun-rui @felixcheung @shivaram

@yanboliang yanboliang changed the title Add read.text and write.text for SparkR [SPARK-12393] [SparkR] Add read.text and write.text for SparkR Dec 17, 2015
@SparkQA
Copy link

SparkQA commented Dec 17, 2015

Test build #47904 has finished for PR 10348 at commit 9e2942d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

felixcheung
felixcheung reviewed Dec 17, 2015
View reviewed changes
R/pkg/R/generics.R
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't read.text be added too?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found all functions in SQLContext.R did not use setGeneric, is this on purpose or a bug?
If it's a bug, I can fix it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is historical legacy:) (you can find same cases in context.R.) I guess that because we did not implement SparkContext and SQLContext as S4 classes. If there is no strong reason, we can keep it as is.
cc @shivaram.

@SparkQA
Copy link

SparkQA commented Dec 18, 2015

Test build #48000 has finished for PR 10348 at commit c9f9bfe.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

felixcheung
felixcheung reviewed Dec 19, 2015
View reviewed changes
R/pkg/R/DataFrame.R Outdated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this a bit odd as an example? it's loading a DF as json and writing it out as text (which requires one string column?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will update it with read.text.

@SparkQA
Copy link

SparkQA commented Dec 22, 2015

Test build #48185 has finished for PR 10348 at commit 85c3c4a.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@yanboliang
Copy link
Contributor Author

ping @shivaram @sun-rui @felixcheung

@sun-rui
Copy link
Contributor

sun-rui commented Jan 5, 2016

LGTM

felixcheung
felixcheung reviewed Jan 5, 2016
View reviewed changes
R/pkg/R/DataFrame.R
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might want to add the DataFrame must have a column with the name "value" I recall there is a similar doc clarification in Scala recently.

@SparkQA
Copy link

SparkQA commented Jan 6, 2016

Test build #48813 has finished for PR 10348 at commit 5063f5f.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@felixcheung
Copy link
Member

looks good

@shivaram
Copy link
Contributor

shivaram commented Jan 6, 2016

LGTM. Thanks @yanboliang for the patch an @sun-rui @felixcheung for reviewing. Merging this to master and branch-1.6

asfgit pushed a commit that referenced this pull request Jan 6, 2016
@yanboliang @shivaram
[SPARK-12393][SPARKR] Add read.text and write.text for SparkR
c3135d0 
Add ```read.text``` and ```write.text``` for SparkR.
cc sun-rui felixcheung shivaram

Author: Yanbo Liang <[email protected]>

Closes #10348 from yanboliang/spark-12393.

(cherry picked from commit d1fea41)
Signed-off-by: Shivaram Venkataraman <[email protected]>
@asfgit asfgit closed this in d1fea41 Jan 6, 2016
@yanboliang yanboliang deleted the spark-12393 branch January 6, 2016 08:08
@HyukjinKwon HyukjinKwon mentioned this pull request Mar 2, 2016
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@yanboliang @SparkQA @sun-rui @felixcheung @shivaram