Differentiate between URI and SSL settings for in-cluster vs. submission #281
Conversation
There was a problem hiding this comment.
"Strictly a path" I think implies no scheme.
There was a problem hiding this comment.
"Strictly be a path" denotes no scheme.
There was a problem hiding this comment.
this .orElse isn't aligned with the others
There was a problem hiding this comment.
can you not use the requireNand from elsewhere here?
There was a problem hiding this comment.
was this not being used before? I'm surprised checkstyle or something didn't alert that there was an unused identifier
There was a problem hiding this comment.
Yeah unfortunately this slipped under the radar somehow. Are the SBT tests running now? Maybe SBT would have caught it and Maven doesn't?
There was a problem hiding this comment.
they're not running right now, but I can temporarily re-enable, you push a commit, and we watch them run to see if they would've caught this
There was a problem hiding this comment.
No need specifically but I suppose it's all the more reason to look into getting our SBT builds online.
There was a problem hiding this comment.
I think this was an original idea I had but abandoned it later, forgot to remove the class.
mccheah
force-pushed
the
allow-client-cert-pem
branch
from
ddc98c1 to
505ce93
Compare
|
Requires rebasing onto allow-client-cert-pem |
mccheah
force-pushed
the
external-vs-internal-uri-tls
branch
from
39e4394 to
377ef96
Compare
|
Finished rebasing here for now. |
mccheah
force-pushed
the
external-vs-internal-uri-tls
branch
from
377ef96 to
b5733f7
Compare
mccheah
changed the base branch from
allow-client-cert-pem
to
branch-2.1-kubernetes
| val resolvedPath = Option(uri.getScheme).getOrElse("file") match { | ||
| case "file" => s"$secretsVolumeMountPath/$secretKey" | ||
| case "local" => uri.getPath | ||
| case invalid => throw new SparkException(s"$uriType has invalid scheme $invalid must be" + |
There was a problem hiding this comment.
maybe in a fixup commit after this merges can add "has invalid scheme for secret" to give end user just a bit more context
Allows users to provide separate settings for contacting the resource staging server from within the init-container vs. from the submission client. One example use case for this is that the resource staging server might be running inside the cluster, and exposed to the submission client through an Ingress or some other front door URI. In this case, the resource staging server should be accessed from the submission client via the Ingress URI, but the init-containers that are running inside the cluster should access the resource staging server through the pod IP or service IP. The certificate that has to be presented when contacting over the external URI may also differ from the certificate that needs to be used when contacting through the internal URI.