Skip to content

[SPARK-5760][SPARK-5761] Fix standalone rest protocol corner cases + revamp tests #4557

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 7 commits into from
Closed

[SPARK-5760][SPARK-5761] Fix standalone rest protocol corner cases + revamp tests #4557

wants to merge 7 commits into from

Conversation

@andrewor14
Copy link
Contributor

@andrewor14 andrewor14 commented Feb 12, 2015

The changes are summarized in the commit message. Test or test-related code accounts for 90% of the lines changed.

Andrew Or added 2 commits February 11, 2015 23:43
Revamp tests + fix a few corner cases
00999a8 
Major changes include:

(1) The server used to always send the JSON to the servlet's
    output stream. However, if the response code is not 200,
    the client must actually read from the error stream. Now
    the server writes to the correct stream depending on the
    response code.

(2) If the server throws an internal exception, both the
    output stream and the error stream on the client side
    is null (not empty). This was not previously accounted
    for and could lead to NPEs.

(3) The default error handling servlet did not match the
    URL cases correctly, because there used to always be
    an empty string in the list. This is now filtered out.
Merge branch 'master' of github.com:apache/spark into rest-tests
ea48f65 
Conflicts:
	core/src/main/scala/org/apache/spark/deploy/rest/StandaloneRestServer.scala
@SparkQA
Copy link

SparkQA commented Feb 12, 2015

Test build #27331 has started for PR 4557 at commit ea48f65.

  • This patch merges cleanly.

andrewor14
andrewor14 reviewed Feb 12, 2015
View reviewed changes
core/src/main/scala/org/apache/spark/deploy/rest/StandaloneRestClient.scala
Copy link
Contributor Author

@andrewor14 andrewor14 Feb 12, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This fixes (2) on SPARK-5760

@SparkQA
Copy link

SparkQA commented Feb 12, 2015

Test build #27335 has started for PR 4557 at commit d82d971.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Feb 12, 2015

Test build #27331 has finished for PR 4557 at commit ea48f65.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

AmplabJenkins commented Feb 12, 2015

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27331/
Test PASSed.

@SparkQA
Copy link

SparkQA commented Feb 12, 2015

Test build #27335 has finished for PR 4557 at commit d82d971.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • trait DataFrame extends RDDApi[Row] with Serializable

@AmplabJenkins
Copy link

AmplabJenkins commented Feb 12, 2015

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27335/
Test PASSed.

@SparkQA
Copy link

SparkQA commented Feb 12, 2015

Test build #27363 has started for PR 4557 at commit b4dc980.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Feb 12, 2015

Test build #27363 has finished for PR 4557 at commit b4dc980.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

AmplabJenkins commented Feb 12, 2015

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27363/
Test PASSed.

@andrewor14
Copy link
Contributor Author

andrewor14 commented Feb 12, 2015

Alright merging into master and 1.3.

asfgit pushed a commit that referenced this pull request Feb 12, 2015
[SPARK-5760][SPARK-5761] Fix standalone rest protocol corner cases + …
11d1080 
…revamp tests

The changes are summarized in the commit message. Test or test-related code accounts for 90% of the lines changed.

Author: Andrew Or <[email protected]>

Closes #4557 from andrewor14/rest-tests and squashes the following commits:

b4dc980 [Andrew Or] Merge branch 'master' of github.com:apache/spark into rest-tests
b55e40f [Andrew Or] Add test for unknown fields
cc96993 [Andrew Or] private[spark] -> private[rest]
578cf45 [Andrew Or] Clean up test code a little
d82d971 [Andrew Or] v1 -> serverVersion
ea48f65 [Andrew Or] Merge branch 'master' of github.com:apache/spark into rest-tests
00999a8 [Andrew Or] Revamp tests + fix a few corner cases

(cherry picked from commit 1d5663e)
Signed-off-by: Andrew Or <[email protected]>
@asfgit asfgit closed this in 1d5663e Feb 12, 2015
@andrewor14 andrewor14 deleted the rest-tests branch February 12, 2015 23:12
@davies davies mentioned this pull request Feb 13, 2015
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@andrewor14 @SparkQA @AmplabJenkins @squito