SPARK-1297 Upgrade HBase dependency to 0.98

[tedyu]

@pwendell @rxin
Please take a look

[AmplabJenkins]

Can one of the admins verify this patch?

[vanzin]

nit: these lines are not aligned with the other dependencies.

[vanzin]

What effect will this have on people who want/need to compile against other versions of hbase? I know hbase has been a little fast and loose about changing the structure of their dependencies, so maybe the answer is "tough luck", and that's probably fine.

[tedyu]

hbase 0.98 has been declared stable release.

Since hbase 0.94 is not modularized, compilation against 0.94 or earlier releases wouldn't be supported.

[AmplabJenkins]

Can one of the admins verify this patch?

[pwendell]

@vanzin this only affects the Spark examples project, which is isolated from our normal build. So I think for normal spark users it's a no-op one way or the other.

[pwendell]

Jenkins, test this please.

[SparkQA]

Test build #23083 has started for PR 3115 at commit 2b079c8.

• This patch merges cleanly.

[SparkQA]

Test build #23083 has finished for PR 3115 at commit 2b079c8.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23083/
Test PASSed.

[cloud-fan]

The pom.xml in example should be a good example to demonstrate how to declare hbase dependency when writing spark application for spark users. However, the pom.xml here is little verbose. For example, hbase-client has already include hbase-common and hbase-protocol, and the exclude rule looks very complicated here, is there a better and clear way to do it?

[srowen]

@cloud-fan I think it depends on whether the code uses hbase-common directly as well, for example. It might. Then it's more correct to depend directly on both. Excludes are the same either way. The big exclusion rule is for hbase-server here, and I think ideally it would not be a dependency. Most people wouldn't depend on the server. IIRC there was some reason it was required by the Spark example code. Hence it is included but with a lot of exclusions. If you have a solution that would sever this dependency I think it would be a clear improvement.

[tedyu]

Thanks Sean for chiming in.

bq. Most people wouldn't depend on the server
Mapreduce related classes, e.g. TableMapReduceUtil, are in hbase-server