Add example that reads a local file, writes to a DFS path provided by th...

[rnowling]

...e user, reads the file back from the DFS, and compares word counts on the local and DFS versions. Useful for verifying DFS correctness.

[SparkQA]

Test build #23561 has started for PR 3347 at commit 0c89558.

• This patch merges cleanly.

[SparkQA]

Test build #23561 has finished for PR 3347 at commit 0c89558.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23561/
Test PASSed.

[andrewor14]

import order should be java > scala > 3rd party packages > spark

[rnowling]

@andrewor14 , thanks for the comments! I believe I fixed everything except for the changing the name of the example. I wanted some more feedback.

I wrote the example to test reading and writing to a DFS. It does so by comparing the result of word count on a local file to word count on the file after copying to the DFS. I'm using it to make sure that DFSs are configured properly and accessible by all nodes.

Do you still want me to drop the test suffix? Thanks!

[SparkQA]

Test build #23672 has started for PR 3347 at commit b0ef9ea.

• This patch merges cleanly.

[SparkQA]

Test build #23672 has finished for PR 3347 at commit b0ef9ea.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23672/
Test PASSed.

[SparkQA]

Test build #23675 has started for PR 3347 at commit af8ccb7.

• This patch merges cleanly.

[SparkQA]

Test build #23675 has finished for PR 3347 at commit af8ccb7.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23675/
Test PASSed.

[rnowling]

@andrewor14 Could you take a second look when you get a chance? Thanks!

[andrewor14]

I think the latest changes look fine. @pwendell any thoughts?

[rnowling]

@andrewor14 @pwendell Have you had a chance to look at this? Thanks!

[rnowling]

@andrewor14 @pwendell Happy New Year! Ping on this PR? Thanks!

[SparkQA]

Test build #26532 has started for PR 3347 at commit af8ccb7.

• This patch merges cleanly.

[SparkQA]

Test build #26532 has finished for PR 3347 at commit af8ccb7.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26532/
Test PASSed.

[srowen]

@andrewor14 if you like the change I think you can merge it; no other comments here. Otherwise i think this has timed out.

[andrewor14]

style: space after if, here and everywhere.

[andrewor14]

@rnowling sorry for slipping on this. I'll merge this into master after addressing the latest comments myself. Thanks for your patience.

[rnowling]

Thanks @andrewor14 !