Multi node network problem with driver kvm2 #10044
Description
Start minikube:
❯ cat .minikube/config/config.json
{
"WantUpdateNotification": false,
"cache": {
"kindest/kindnetd:0.5.4": null,
"nginx": null,
"praqma/network-multitool": null
},
"driver": "kvm2",
"kubernetes-version": "v1.19.3"
}
❯ minikube-v1.16.0 version
minikube version: v1.16.0
commit: 9f1e482427589ff8451c4723b6ba53bb9742fbb1
❯ minikube-v1.16.0 start -n 2Test yaml comes from #9838
❯ kgp -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
my-nginx-5b56ccd65f-mqsjt 1/1 Running 0 45s 10.244.0.4 minikube <none> <none>
my-nginx-5b56ccd65f-p6zwd 1/1 Running 0 45s 10.244.1.2 minikube-m02 <none> <none>
my-nginx-5b56ccd65f-t6bpn 1/1 Running 0 45s 10.244.1.3 minikube-m02 <none> <none>
my-nginx-5b56ccd65f-tnbcq 1/1 Running 0 45s 10.244.0.2 minikube <none> <none>
net-test-c4f9cfdd4-9dp2r 1/1 Running 0 45s 10.244.0.3 minikube <none> <none>
net-test-c4f9cfdd4-cx87r 1/1 Running 0 45s 10.244.1.5 minikube-m02 <none> <none>
net-test-c4f9cfdd4-qd5mv 1/1 Running 0 45s 10.244.1.4 minikube-m02 <none> <none>
net-test-c4f9cfdd4-zgmz7 1/1 Running 0 45s 10.244.0.5 minikube <none> <none>Then test inside pod:
❯ kx net-test-c4f9cfdd4-9dp2r -- bash # pod in node minikube
bash-5.0# curl baidu.com
<html>
<meta http-equiv="refresh" content="0;url=http://www.baidu.com/">
</html>
bash-5.0# curl my-nginx --connect-timeout 10
<!DOCTYPE html>
<html>
.....
</html>
bash-5.0# curl my-nginx --connect-timeout 10
curl: (28) Connection timed out after 10001 milliseconds
bash-5.0# curl 10.244.1.2 --connect-timeout 10 # try ip
curl: (28) Connection timed out after 10001 milliseconds
bash-5.0# curl 10.244.0.2
<!DOCTYPE html>
<html>
...
</html>❯ kx net-test-c4f9cfdd4-cx87r -- bash # pod in node minikube-m02
bash-5.0# curl baidu.com --connect-timeout 10
curl: (6) Could not resolve host: baidu.com
bash-5.0# curl my-nginx --connect-timeout 10
curl: (6) Could not resolve host: my-nginx
bash-5.0# curl ^C
bash-5.0# curl 10.244.1.2 --connect-timeout 10
<!DOCTYPE html>
<html>
...
</html>
bash-5.0# curl 10.244.0.2 --connect-timeout 10
curl: (7) Failed to connect to 10.244.0.2 port 80: Host is unreachable
Test in host machine:
❯ minikube-v1.16.0 ssh -n m02 # host m02
$ curl 10.244.0.4 # cannot connect to m01, includes coredns
curl: (7) Failed to connect to 10.244.0.4 port 80: No route to host❯ minikube-v1.16.0 ssh
$ curl 10.244.1.2 # but m01 can access m02
<!DOCTYPE html>
<html>
...
</html>