Skip to content

Commit 2c9c862

Browse files
HyukjinKwonjerryshao
authored andcommitted
[MINOR][YARN] Add YARN-specific credential providers in debug logging message
This PR adds a debugging log for YARN-specific credential providers which is loaded by service loader mechanism. It took me a while to debug if it's actually loaded or not. I had to explicitly set the deprecated configuration and check if that's actually being loaded. The change scope is manually tested. Logs are like: ``` Using the following builtin delegation token providers: hadoopfs, hive, hbase. Using the following YARN-specific credential providers: yarn-test. ``` Author: hyukjinkwon <[email protected]> Closes #21466 from HyukjinKwon/minor-log. Change-Id: I18e2fb8eeb3289b148f24c47bb3130a560a881cf
1 parent 21e1fc7 commit 2c9c862

File tree

2 files changed

+6
-2
lines changed

2 files changed

+6
-2
lines changed

core/src/main/scala/org/apache/spark/deploy/security/HadoopDelegationTokenManager.scala

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ import org.apache.spark.internal.Logging
3232
*
3333
* Also, each HadoopDelegationTokenProvider is controlled by
3434
* spark.security.credentials.{service}.enabled, and will not be loaded if this config is set to
35-
* false. For example, Hive's delegation token provider [[HiveDelegationTokenProvider]] can be
35+
* false. For example, Hive's delegation token provider [[HiveDelegationTokenProvider]] can be
3636
* enabled/disabled by the configuration spark.security.credentials.hive.enabled.
3737
*
3838
* @param sparkConf Spark configuration
@@ -52,7 +52,7 @@ private[spark] class HadoopDelegationTokenManager(
5252

5353
// Maintain all the registered delegation token providers
5454
private val delegationTokenProviders = getDelegationTokenProviders
55-
logDebug(s"Using the following delegation token providers: " +
55+
logDebug("Using the following builtin delegation token providers: " +
5656
s"${delegationTokenProviders.keys.mkString(", ")}.")
5757

5858
/** Construct a [[HadoopDelegationTokenManager]] for the default Hadoop filesystem */

resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/security/YARNHadoopDelegationTokenManager.scala

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,10 @@ private[yarn] class YARNHadoopDelegationTokenManager(
4444

4545
// public for testing
4646
val credentialProviders = getCredentialProviders
47+
if (credentialProviders.nonEmpty) {
48+
logDebug("Using the following YARN-specific credential providers: " +
49+
s"${credentialProviders.keys.mkString(", ")}.")
50+
}
4751

4852
/**
4953
* Writes delegation tokens to creds. Delegation tokens are fetched from all registered

0 commit comments

Comments
 (0)