[K8S][HELM] Add configuration files support to helm chart

[dnskr]

Why are the changes needed?

The changes allow to:

1. set Kyuubi configuration files:
   • kyuubi-env.sh
   • kyuubi-defaults.conf
   • log4j2.xml
2. restart (recreate) Kyuubi server pods if configuration files have changed
   • this probably should be revisited after [KYUUBI #3982] [FEATURE] introduce refreshing user defaults configs #3983

How was this patch tested?

• Add some test cases that check the changes thoroughly including negative and positive cases if possible

• Add screenshots for manual tests if appropriate

• Run test locally before make a pull request

[dnskr]

I'm not sure about default values for kyuubi-defaults.conf:

1. I can't find kyuubi.kubernetes.namespace config both documentation and source code, so I deleted it.
2. I found that kyuubi.frontend.bind.host and kyuubi.frontend.bind.port marked as deprecated in the doc, but I can't find what should be used instead.

[codecov-commenter]

Codecov Report

Merging #4004 (8367825) into master (c577dc7) will decrease coverage by 0.02%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #4004      +/-   ##
============================================
- Coverage     52.11%   52.08%   -0.03%     
  Complexity       13       13              
============================================
  Files           541      541              
  Lines         29468    29468              
  Branches       3938     3938              
============================================
- Hits          15357    15349       -8     
- Misses        12709    12712       +3     
- Partials       1402     1407       +5     

Impacted Files	Coverage Δ
...rg/apache/kyuubi/ctl/cmd/log/LogBatchCommand.scala	60.00% <0.00%> (-7.70%)	⬇️
...ache/kyuubi/operation/KyuubiOperationManager.scala	80.00% <0.00%> (-2.67%)	⬇️
...he/kyuubi/ha/client/etcd/EtcdDiscoveryClient.scala	68.50% <0.00%> (-0.56%)	⬇️
...in/scala/org/apache/kyuubi/config/KyuubiConf.scala	97.45% <0.00%> (-0.07%)	⬇️
...org/apache/kyuubi/operation/ExecuteStatement.scala	78.48% <0.00%> (ø)
.../org/apache/kyuubi/session/KyuubiSessionImpl.scala	79.56% <0.00%> (+0.72%)	⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[pan3793]

Thanks @dnskr

1. I can't find kyuubi.kubernetes.namespace config both documentation and source code, so I deleted it.

You can find it in the master branch doc(unreleased), the latest doc represents the latest stable release doc.

2. I found that kyuubi.frontend.bind.host and kyuubi.frontend.bind.port marked as deprecated in the doc, but I can't find what should be used instead.

Sorry for confusing, I opened #4005 to explain and fix it.

[pan3793]

@hddong @dnskr some thoughts about the schema of values.yaml (maybe in another PR)

server:
  bind:
    host: 0.0.0.0
    port: 10009
  confDir:
  conf:
    kyuubiEnv:
    kyuubiDefaults:
    log4j2:

How about change server to kyuubi? And we may need a similar part of spark and hadoop

For "bind" part, the port is not sufficient since Kyuubi supports multi protocols, not only thrift binary now.

[dnskr]

How about change server to kyuubi? And we may need a similar part of spark and hadoop

For "bind" part, the port is not sufficient since Kyuubi supports multi protocols, not only thrift binary now.

Agree with both suggestions. I left it as is just because I'm quite new in Kyuubi and don't fully understand all moving pieces. But I have enthusiasm to continue exploring it and creating PRs.

[pan3793]

Merged to master, thanks @dnskr for your contribution and passion. The team is suffering from COVID-19, and may not be able to reply in time.

[bowenliang123]

@dnskr Hi, #3982 is closed, while I am afraid that it won't help in your case since it was refactored to refresh users' default configs only as dissucssed in PR 3983.

But you can still consider follow the similar pattern to achieve it as the following as 3982 does,

• add REST API to apply config changes and implement related method in server
• add command or config type for refreshing configs in kyuubi-admin