[SPARK-40997][K8S] K8s resource name prefix should start w/ alphanumeric

[pan3793]

What changes were proposed in this pull request?

Ensure string generated by KubernetesConf#getResourceNamePrefix always start w/ alphanumeric

Why are the changes needed?

According to https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#dns-subdomain-names, the most resource names should start with an alphanumeric character.

Currently, the KubernetesConf#getResourceNamePrefix does not ensure the return value always start w/ alphanumeric, for example, if user set spark.app.name=测试, will cause the following error.

2022-11-02 19:46:05 [ERROR] [kubernetes-executor-snapshots-subscribers-1] org.apache.spark.scheduler.cluster.k8s.ExecutorPodsSnapshotsStoreImpl#98 - Going to stop due to IllegalArgumentException
java.lang.IllegalArgumentException: '-9ea12a8438298333' in spark.kubernetes.executor.podNamePrefix is invalid. must conform https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#dns-subdomain-names and the value length <= 237
	at org.apache.spark.internal.config.TypedConfigBuilder.$anonfun$checkValue$1(ConfigBuilder.scala:108) ~[spark-core_2.12-3.3.1.1.jar:3.3.1.1]
	at org.apache.spark.internal.config.TypedConfigBuilder.$anonfun$transform$1(ConfigBuilder.scala:101) ~[spark-core_2.12-3.3.1.1.jar:3.3.1.1]
	at scala.Option.map(Option.scala:230) ~[scala-library-2.12.15.jar:?]
	at org.apache.spark.internal.config.OptionalConfigEntry.readFrom(ConfigEntry.scala:239) ~[spark-core_2.12-3.3.1.1.jar:3.3.1.1]
	at org.apache.spark.internal.config.OptionalConfigEntry.readFrom(ConfigEntry.scala:214) ~[spark-core_2.12-3.3.1.1.jar:3.3.1.1]
	at org.apache.spark.SparkConf.get(SparkConf.scala:261) ~[spark-core_2.12-3.3.1.1.jar:3.3.1.1]
	at org.apache.spark.deploy.k8s.KubernetesConf.get(KubernetesConf.scala:70) ~[spark-kubernetes_2.12-3.3.1.1.jar:3.3.1.1]
	at org.apache.spark.deploy.k8s.KubernetesExecutorConf.<init>(KubernetesConf.scala:156) ~[spark-kubernetes_2.12-3.3.1.1.jar:3.3.1.1]
	at org.apache.spark.deploy.k8s.KubernetesConf$.createExecutorConf(KubernetesConf.scala:246) ~[spark-kubernetes_2.12-3.3.1.1.jar:3.3.1.1]
	at org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator.$anonfun$requestNewExecutors$1(ExecutorPodsAllocator.scala:394) ~[spark-kubernetes_2.12-3.3.1.1.jar:3.3.1.1]
	at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:158) ~[scala-library-2.12.15.jar:?]
	at org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator.requestNewExecutors(ExecutorPodsAllocator.scala:387) ~[spark-kubernetes_2.12-3.3.1.1.jar:3.3.1.1]
	at org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator.$anonfun$onNewSnapshots$35(ExecutorPodsAllocator.scala:351) ~[spark-kubernetes_2.12-3.3.1.1.jar:3.3.1.1]
	at org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator.$anonfun$onNewSnapshots$35$adapted(ExecutorPodsAllocator.scala:344) ~[spark-kubernetes_2.12-3.3.1.1.jar:3.3.1.1]
	at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62) ~[scala-library-2.12.15.jar:?]
	at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55) ~[scala-library-2.12.15.jar:?]
	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49) ~[scala-library-2.12.15.jar:?]
	at org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator.onNewSnapshots(ExecutorPodsAllocator.scala:344) ~[spark-kubernetes_2.12-3.3.1.1.jar:3.3.1.1]
	at org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator.$anonfun$start$3(ExecutorPodsAllocator.scala:122) ~[spark-kubernetes_2.12-3.3.1.1.jar:3.3.1.1]
	at org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator.$anonfun$start$3$adapted(ExecutorPodsAllocator.scala:122) ~[spark-kubernetes_2.12-3.3.1.1.jar:3.3.1.1]
	at org.apache.spark.scheduler.cluster.k8s.ExecutorPodsSnapshotsStoreImpl$SnapshotsSubscriber.org$apache$spark$scheduler$cluster$k8s$ExecutorPodsSnapshotsStoreImpl$SnapshotsSubscriber$$processSnapshotsInternal(ExecutorPodsSnapshotsStoreImpl.scala:138) ~[spark-kubernetes_2.12-3.3.1.1.jar:3.3.1.1]
	at org.apache.spark.scheduler.cluster.k8s.ExecutorPodsSnapshotsStoreImpl$SnapshotsSubscriber.processSnapshots(ExecutorPodsSnapshotsStoreImpl.scala:126) ~[spark-kubernetes_2.12-3.3.1.1.jar:3.3.1.1]
	at org.apache.spark.scheduler.cluster.k8s.ExecutorPodsSnapshotsStoreImpl$$anon$1.run(ExecutorPodsSnapshotsStoreImpl.scala:90) ~[spark-kubernetes_2.12-3.3.1.1.jar:3.3.1.1]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_345]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_345]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_345]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.0_345]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_345]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_345]
	at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_345]

Does this PR introduce any user-facing change?

No.

How was this patch tested?

New UT added.

[AmplabJenkins]

Can one of the admins verify this patch?

[pan3793]

cc @dongjoon-hyun @Yikun, would you please take a look?

[Yikun]

Looks fine for me, and FYI: #38331

[pan3793]

Emm... I didn't notice there was a PR to fix this issue, either one get merged is fine to me.

[dongjoon-hyun]

Ya, I merged #38331 because it came first.
BTW, thank you, @pan3793 and @Yikun . Could you close this PR, @pan3793 ?

[pan3793]

Close as duplicated, thanks all.