Skip to content

Conversation

@JoshRosen
Copy link
Contributor

This patch fixes a flaky "test jdbc cancel" test in HiveThriftBinaryServerSuite. This test is prone to a race-condition which causes it to block indefinitely with while waiting for an extremely slow query to complete, which caused many Jenkins builds to time out.

For more background, see my comments on #6207 (the PR which introduced this test).

@JoshRosen JoshRosen changed the title [SPARK-11823] Fix flaky JDBC cancellation test in HiveThriftBinaryServerSuite [SPARK-11823][SQL] Fix flaky JDBC cancellation test in HiveThriftBinaryServerSuite Dec 22, 2015
@JoshRosen
Copy link
Contributor Author

/cc @marmbrus @shaneknapp

@SparkQA
Copy link

SparkQA commented Dec 22, 2015

Test build #48145 has finished for PR 10425 at commit ad2776e.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need this log?

@rxin
Copy link
Contributor

rxin commented Dec 22, 2015

LGTM.

@JoshRosen
Copy link
Contributor Author

I'm going to merge this into master and branch-1.6. We can consider a more principled fix, such as using some sort of timed-sleep UDF, in case this turns out to be flaky. My main motivation in this change was to fix the process leaks that were caused by the hour+ timeouts that this test used to experience.

asfgit pushed a commit that referenced this pull request Dec 22, 2015
…ryServerSuite

This patch fixes a flaky "test jdbc cancel" test in HiveThriftBinaryServerSuite. This test is prone to a race-condition which causes it to block indefinitely with while waiting for an extremely slow query to complete, which caused many Jenkins builds to time out.

For more background, see my comments on #6207 (the PR which introduced this test).

Author: Josh Rosen <[email protected]>

Closes #10425 from JoshRosen/SPARK-11823.

(cherry picked from commit 2235cd4)
Signed-off-by: Josh Rosen <[email protected]>
@asfgit asfgit closed this in 2235cd4 Dec 22, 2015
@JoshRosen JoshRosen deleted the SPARK-11823 branch December 22, 2015 07:14
@JoshRosen
Copy link
Contributor Author

Looks like this test is still leading to hangs in Jenkins. Given that I thought this patch would change those hangs into test timeouts/failures, I wonder whether the cancel() statement itself might be hanging. I'll see if I can manage to pull a jstack from both the driver and thriftserver processes to see if I can figure this out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants