Skip to content

Conversation

@andrewor14
Copy link
Contributor

This patch fixes a build break in maven caused by #6441.

Note that this patch reverts the changes in flume-sink because
this module does not currently depend on Spark core, but the
tests require it. There is not an easy way to make this work
because mvn test dependencies are not transitive (MNG-1378).

For now, we will leave the one test suite in flume-sink out
until we figure out a better solution. This patch is mainly
intended to unbreak the maven build.

Note that this patch reverts the changes in flume-sink because
this module does not currently depend on Spark core, but the
tests require it. There is not an easy way to make this work
because mvn test dependencies are not transitive (MNG-1378).

For now, we will leave the one test suite in flume-sink out
until we figure out a better solution. This patch is mainly
intended to unbreak the maven build.
@andrewor14
Copy link
Contributor Author

I tested this locally and I'm merging this into master because the build is currently broken.

@andrewor14 andrewor14 changed the title [HOT FIX #6441] Fix maven build failures [HOT FIX] [BUILD] Fix maven build failures May 30, 2015
@asfgit asfgit closed this in a4f2412 May 30, 2015
@andrewor14 andrewor14 deleted the fix-build-mvn branch May 30, 2015 01:36
@SparkQA
Copy link

SparkQA commented May 30, 2015

Test build #33784 has finished for PR 6511 at commit 3d53643.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • class SparkSinkSuite extends FunSuite

@JoshRosen
Copy link
Contributor

I'm running into Maven build breaks due to missing test-jar dependencies in the sql, mllib and flume-sink projects. Why did you remove the test dependency? I can understand that it wouldn't transitively inherit the test deps. of a project that it depends on, but what's the harm in explicitly adding a test dep? Also, why do you think that the Jenkins build didn't catch the local Maven test compilation issues that I saw when running make-distribution.sh? Do you think that this is something that's Maven version specific?

@JoshRosen
Copy link
Contributor

False alarm: I was building off a slightly-out-of-date branch.

Just saw the comment on your other commit; I see now that the issue is the fact that Spark Core's own dependencies aren't pulled in via the test JAR, so the test class itself is missing the deps. it needs. Makes sense.

@andrewor14
Copy link
Contributor Author

Yes, these were pushed as hot fixes later. Unfortunately maven has stricter requirements than SBT for some reason.

jeanlyn pushed a commit to jeanlyn/spark that referenced this pull request Jun 12, 2015
This patch fixes a build break in maven caused by apache#6441.

Note that this patch reverts the changes in flume-sink because
this module does not currently depend on Spark core, but the
tests require it. There is not an easy way to make this work
because mvn test dependencies are not transitive (MNG-1378).

For now, we will leave the one test suite in flume-sink out
until we figure out a better solution. This patch is mainly
intended to unbreak the maven build.

Author: Andrew Or <[email protected]>

Closes apache#6511 from andrewor14/fix-build-mvn and squashes the following commits:

3d53643 [Andrew Or] [HOT FIX apache#6441] Fix maven build failures
nemccarthy pushed a commit to nemccarthy/spark that referenced this pull request Jun 19, 2015
This patch fixes a build break in maven caused by apache#6441.

Note that this patch reverts the changes in flume-sink because
this module does not currently depend on Spark core, but the
tests require it. There is not an easy way to make this work
because mvn test dependencies are not transitive (MNG-1378).

For now, we will leave the one test suite in flume-sink out
until we figure out a better solution. This patch is mainly
intended to unbreak the maven build.

Author: Andrew Or <[email protected]>

Closes apache#6511 from andrewor14/fix-build-mvn and squashes the following commits:

3d53643 [Andrew Or] [HOT FIX apache#6441] Fix maven build failures
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants