kube-up.sh: drop unnecessary legacy mirror config, enable injecting registry mirror #126448
Conversation
k8s-ci-robot
added
do-not-merge/work-in-progress
k8s-ci-robot
added
release-note-none
|
/sig testing scalability |
k8s-ci-robot
added
sig/testing
BenTheElder
changed the title
k8s-ci-robot
added
area/provider/gcp
BenTheElder
force-pushed
the
5k-mirror
branch
3 times, most recently
from
511e332 to
9c9c0d6
Compare
cluster/gce/gci/configure-helper.sh
Outdated
| EOF | ||
|
|
||
| # DO NOT MERGE -- Testing | ||
| KUBERNETES_REGISTRY_PULL_THROUGH_HOST='https://us-central1-docker.pkg.dev/v2/k8s-infra-e2e-scale-5k-project/k8s-5k-scale-cache/' |
There was a problem hiding this comment.
I wiped the images in this just before the last run but it doesn't look like we populated any, so something is not quite right here (or the timing was off, but I don't think so).
Need to investigate this further before proceeding.
There was a problem hiding this comment.
after fixing #126448 (comment), I see many many images pulled through
k8s-ci-robot
added
size/M
cluster/gce/gci/configure-helper.sh
Outdated
| # NOTE: we need literal double quotes around some of these values | ||
| echo 'server="'"${KUBERNETES_REGISTRY_PULL_THROUGH_HOST}"'"' | ||
| echo '' | ||
| echo '[hosts."'"${KUBERNETES_REGISTRY_PULL_THROUGH_HOST}"'"]' |
There was a problem hiding this comment.
this should be host.
|
kubernetes/test-infra#33158 will allow manually vetting this fully |
|
/test pull-kubernetes-e2e-gce-pull-through-cache |
This comment was marked as resolved.
This comment was marked as resolved.
|
/test pull-kubernetes-e2e-gce-pull-through-cache |
1 similar comment
|
/test pull-kubernetes-e2e-gce-pull-through-cache |
k8s-ci-robot
added
size/M
|
/test pull-kubernetes-e2e-gce-pull-through-cache |
|
/test pull-kubernetes-e2e-gce-pull-through-cache |
…egistry.k8s.io mirror
|
/test pull-kubernetes-e2e-gce-pull-through-cache |
|
This is working as intended now. I can see the images being pulled through the cache in the test job and peeked into the e2e cluster nodes to confirm the generated config is as intended. |
|
/hold cancel |
k8s-ci-robot
removed
the
do-not-merge/hold
| EOF | ||
| if [[ -n "${KUBERNETES_REGISTRY_PULL_THROUGH_BASIC_AUTH_TOKEN_PATH:-}" ]]; then | ||
| cat >>"$file" <<EOF | ||
| KUBERNETES_REGISTRY_PULL_THROUGH_BASIC_AUTH_TOKEN: $(yaml-quote "$(cat "${KUBERNETES_REGISTRY_PULL_THROUGH_BASIC_AUTH_TOKEN_PATH}")") |
There was a problem hiding this comment.
We have to plumb this through to the nodes by placing it in the big env file we copy up here.
It does not appear to be stored in uploaded ARTIFACTS or logged in logs.
|
And now https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/126448/pull-kubernetes-e2e-gce-pull-through-cache/1818444296806731776 has passed with these enabled, and I can see many images populated into the cache: 
This should let us isolate high demand from scale testing in #126366 We would want end-users to do something similar, and we have docs for this in https://registry.k8s.io |
|
/approve thanks @BenTheElder |
k8s-ci-robot
added
the
lgtm
|
LGTM label has been added. Git tree hash: 3c2f39546b5a42d40b15120f84e403aff49f0e12
|
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: BenTheElder, dims The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
3 participants
What type of PR is this?
/kind failing-test
What this PR does / why we need it:
Enables using a pull-through cache for images in kube-up.sh
Part of resolving 5k node scale test issues #126366
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?
Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.: