[SPARK-20033][SQL] support hive permanent function

[cenyuhai]

What changes were proposed in this pull request?

support hive permanent function

How was this patch tested?

[SparkQA]

Test build #74885 has finished for PR 17362 at commit 703d23e.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #74889 has finished for PR 17362 at commit a668bb5.

• This patch fails to build.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #74890 has finished for PR 17362 at commit 9325a7c.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #74912 has finished for PR 17362 at commit 92c077d.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[gatorsmile]

I think we already support Hive UDF. See the test case for permanent Hive UDF.

[gatorsmile]

Are you trying to create a Hive UDF via Spark, then call it through Hive?

[cenyuhai]

no, I just use sessionState.add_resource to download the resouces

[cenyuhai]

@gatorsmile hi，spark just suport HIVE UDF which resources is in the local disk, or spark-sql --jars xxx.jar or something else. But I think spark don't support the hive permanent function which resources are in hdfs.
CREATE FUNCTION hdfs_udf AS 'xxx.udf' USING JAR 'hdfs:///user/udf/.jar';
My pr just download the hdfs resources and add to the classpath.

[wangyum]

@weiqingy is doing Allow adding jars from hdfs.

[cenyuhai]

Ok, I think @weiqingy 's pr will resolve this problem

[gatorsmile]

@wangyum Thank you!

@cenyuhai Maybe you can close this PR now. Please also try the PR #17342 and see whether it resolves your issue. Thanks!