Fedora VM Drivers on minikube 1.33 cannot pull images (resolved.conf)

[llegolas]

What Happened?

minikube fails to start network cni (resolved.conf)

Attach the log file

It seem as a problem with systemd-resolved.service
running

resolvectl query quay.io

times out with the following in the logs:

Apr 20 21:41:50 minikube systemd-resolved[202]: DNSSEC validation failed for question quay.io IN DS: failed-auxiliary
Apr 20 21:41:50 minikube systemd-resolved[202]: DNSSEC validation failed for question quay.io IN SOA: failed-auxiliary
Apr 20 21:41:50 minikube systemd-resolved[202]: DNSSEC validation failed for question quay.io IN A: failed-auxiliary
Apr 20 21:41:50 minikube systemd-resolved[202]: DNSSEC validation failed for question quay.io IN AAAA: failed-auxiliary
Apr 20 21:43:46 minikube systemd-resolved[202]: DNSSEC validation failed for question quay.io IN DS: failed-auxiliary
Apr 20 21:43:46 minikube systemd-resolved[202]: DNSSEC validation failed for question quay.io IN SOA: failed-auxiliary
Apr 20 21:43:46 minikube systemd-resolved[202]: DNSSEC validation failed for question quay.io IN AAAA: failed-auxiliary
Apr 20 21:43:46 minikube systemd-resolved[202]: DNSSEC validation failed for question quay.io IN A: failed-auxiliary
Apr 20 21:45:41 minikube systemd-resolved[202]: DNSSEC validation failed for question quay.io IN DS: failed-auxiliary
Apr 20 21:45:41 minikube systemd-resolved[202]: DNSSEC validation failed for question quay.io IN SOA: failed-auxiliary
Apr 20 21:45:41 minikube systemd-resolved[202]: DNSSEC validation failed for question quay.io IN AAAA: failed-auxiliary
Apr 20 21:45:41 minikube systemd-resolved[202]: DNSSEC validation failed for question quay.io IN A: failed-auxiliary

if I run

resolvectl query --validate=no quay.io

all is good

Operating System

Redhat/Fedora

Driver

KVM2

[cirix]

I confirm the same on my MacBook M2 2023. Same behavior. The minikube addons fails. ingress and ingress-dns clearly also stating not supported(compiled image missing). I have submitted a fix for ingress-dns but they decided ignore.

[llegolas]

not sure how /etc/systemd/resolved.conf with DENSSEC=yes crept in but it is glaring difference with the previous version of 1.32, and I'm unaware of any major distro shipping with DNSSEC on by default as this is still marked as experimental in systemd upstream AFAIK.

[medyagh]

@llegolas can you paste the full logs
you can attach the file to this issue
minikube logs --file=logs.txt

, does that only affect when you enable a specific addon or only on quay.io images?
does the minikube start works on default case?

and also can you try with a different container runtime to see if that solves it ?

minikube delete --all
minikube start --container-runtime=containerd

[medyagh]

btw I just tired this myself and it seems good for me for Qemu Driver on M1 (arm64)

$ minikube ssh
$ resolvectl query quay.io
quay.io: 2600:1f18:483:cf00:3b4b:fac4:4515:599b -- link: eth0
         2600:1f18:483:cf00:e91a:d8cb:1b23:812b -- link: eth0
         2600:1f18:483:cf02:3f6c:1ed8:799c:1a49 -- link: eth0
         2600:1f18:483:cf01:455b:c2ec:5fad:7f15 -- link: eth0
         2600:1f18:483:cf02:3b2:c8d9:2c85:b678 -- link: eth0
         2600:1f18:483:cf01:cd0c:370b:1c0c:7317 -- link: eth0
         3.231.43.138                          -- link: eth0
         3.230.69.80                           -- link: eth0
         54.243.3.120                          -- link: eth0
         3.228.243.121                         -- link: eth0
         3.220.0.108                           -- link: eth0
         44.212.212.54                         -- link: eth0

-- Information acquired via protocol DNS in 26.2ms.
-- Data is authenticated: no; Data was acquired via local or encrypted transport: no
-- Data from: network
$ resolvectl query docker.io
docker.io: 2600:1f18:2148:bc00:a81:7e44:4669:3426 -- link: eth0
           2600:1f18:2148:bc01:1983:8fd2:2dfc:a04c -- link: eth0
           2600:1f18:2148:bc02:a090:6a5b:b2ff:3152 -- link: eth0
           54.156.140.159                      -- link: eth0
           52.44.227.212                       -- link: eth0
           44.221.37.199                       -- link: eth0

-- Information acquired via protocol DNS in 25.6ms.
-- Data is authenticated: no; Data was acquired via local or encrypted transport: no
-- Data from: network

[medyagh]

btw I do not see DENSSEC=yes in my resolved.conf

$ 
$ 
$ cat  /etc/systemd/resolved.conf
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it under the
#  terms of the GNU Lesser General Public License as published by the Free
#  Software Foundation; either version 2.1 of the License, or (at your option)
#  any later version.
#
# Entries in this file show the compile time defaults. Local configuration
# should be created by either modifying this file, or by creating "drop-ins" in
# the resolved.conf.d/ subdirectory. The latter is generally recommended.
# Defaults can be restored by simply deleting this file and all drop-ins.
#
# Use 'systemd-analyze cat-config systemd/resolved.conf' to display the full config.
#
# See resolved.conf(5) for details.

[Resolve]
# Some examples of DNS servers which may be used for DNS= and FallbackDNS=:
# Cloudflare: 1.1.1.1#cloudflare-dns.com 1.0.0.1#cloudflare-dns.com 2606:4700:4700::1111#cloudflare-dns.com 2606:4700:4700::1001#cloudflare-dns.com
# Google:     8.8.8.8#dns.google 8.8.4.4#dns.google 2001:4860:4860::8888#dns.google 2001:4860:4860::8844#dns.google
# Quad9:      9.9.9.9#dns.quad9.net 149.112.112.112#dns.quad9.net 2620:fe::fe#dns.quad9.net 2620:fe::9#dns.quad9.net
#DNS=
#FallbackDNS=1.1.1.1#cloudflare-dns.com 8.8.8.8#dns.google 1.0.0.1#cloudflare-dns.com 8.8.4.4#dns.google 2606:4700:4700::1111#cloudflare-dns.com 2001:4860:4860::8888#dns.google 2606:4700:4700::1001#cloudflare-dns.com 2001:4860:4860::8844#dns.google
#Domains=
#DNSSEC=allow-downgrade
#DNSOverTLS=opportunistic
#MulticastDNS=yes
#LLMNR=yes
#Cache=yes
#CacheFromLocalhost=no
#DNSStubListener=yes
#DNSStubListenerExtra=
#ReadEtcHosts=yes
#ResolveUnicastSingleLabel=no

cat /version.json
{"iso_version": "v1.33.0", "kicbase_version": "v0.0.43-1713236840-18649", "minikube_version": "v1.33.0", "commit": "4bd203f0c710e7fdd30539846cf2bc6624a2556d"}

llegolas @cirix
do you mind running this command to verify the ISO version you are using.
"cat /version.json"

[llegolas]

#DNSSEC=allow-downgrade you see there is the assumed default meaning it is on but "should"downgrade if no signature is provided. Well for some reason it is not downgrading.
I've tested both with containerd and crio runtimes. And it was the cilium CNI that was failing to start because of the DNSSEC problem. I've reverted back to 1.32 as I had something urgent to test. Later tonight I'll provide the logs when I upgrade to 1.33 again.

[cirix]

I had to work on something and downgraded. In my case in the image you will find the bootstrap on my macosx and the logs
@medyagh here you go. You have also a pending action with my fix on the upgrade of ingress-dns...being waiting for more than 2 months.

logs.txt

[llegolas]

here is the whole session:

$ minikube delete
🔥  Deleting "minikube" in kvm2 ...
💀  Removed all traces of the "minikube" cluster.
$ sudo dnf remove minikube
$ sudo dnf install ~/Downloads/minikube-1.33.0-0.x86_64.rpm
$ minikube start --cni=cilium  --container-runtime=containerd --cpus=4 --memory=16g --addons=ingress-dns,ingress --driver=kvm2
😄  minikube v1.33.0 on Fedora 39
✨  Using the kvm2 driver based on user configuration
💿  Downloading VM boot image ...
    > minikube-v1.33.0-amd64.iso....:  65 B / 65 B [---------] 100.00% ? p/s 0s
    > minikube-v1.33.0-amd64.iso:  314.16 MiB / 314.16 MiB  100.00% 7.02 MiB p/
👍  Starting "minikube" primary control-plane node in "minikube" cluster
💾  Downloading Kubernetes v1.30.0 preload ...
    > preloaded-images-k8s-v18-v1...:  375.69 MiB / 375.69 MiB  100.00% 5.97 Mi
🔥  Creating kvm2 VM (CPUs=4, Memory=16384MB, Disk=20000MB) ...
❗  This VM is having trouble accessing https://registry.k8s.io
💡  To pull new external images, you may need to configure a proxy: https://minikube.sigs.k8s.io/docs/reference/networking/proxy/
📦  Preparing Kubernetes v1.30.0 on containerd 1.7.15 ...
    ▪ Generating certificates and keys ...
    ▪ Booting up control plane ...
    ▪ Configuring RBAC rules ...
🔗  Configuring Cilium (Container Networking Interface) ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/minikube-ingress-dns:0.0.2
    ▪ Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.0
    ▪ Using image registry.k8s.io/ingress-nginx/controller:v1.10.0
    ▪ Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.0
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🔎  Verifying ingress addon...
❗  Enabling 'ingress' returned an error: running callbacks: [waiting for app.kubernetes.io/name=ingress-nginx pods: context deadline exceeded]
🌟  Enabled addons: ingress-dns, storage-provisioner, default-storageclass
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
$ k get pods -A
NAMESPACE       NAME                                       READY   STATUS                  RESTARTS   AGE
ingress-nginx   ingress-nginx-admission-create-cp9td       0/1     Pending                 0          7m4s
ingress-nginx   ingress-nginx-admission-patch-4fcb8        0/1     Pending                 0          7m4s
ingress-nginx   ingress-nginx-controller-84df5799c-2mzqm   0/1     Pending                 0          7m4s
kube-system     cilium-operator-86f4c5579c-xxsrn           0/1     ImagePullBackOff        0          7m4s
kube-system     cilium-s7nlr                               0/1     Init:ImagePullBackOff   0          7m4s
kube-system     coredns-7db6d8ff4d-jql96                   0/1     Pending                 0          7m4s
kube-system     etcd-minikube                              1/1     Running                 0          7m17s
kube-system     kube-apiserver-minikube                    1/1     Running                 0          7m19s
kube-system     kube-controller-manager-minikube           1/1     Running                 0          7m17s
kube-system     kube-ingress-dns-minikube                  0/1     Pending                 0          7m14s
kube-system     kube-proxy-m2l8p                           1/1     Running                 0          7m4s
kube-system     kube-scheduler-minikube                    1/1     Running                 0          7m17s
kube-system     storage-provisioner                        0/1     Pending                 0          7m14s
$ minikube ssh
                         _             _            
            _         _ ( )           ( )           
  ___ ___  (_)  ___  (_)| |/')  _   _ | |_      __  
/' _ ` _ `\| |/' _ `\| || , <  ( ) ( )| '_`\  /'__`\
| ( ) ( ) || || ( ) || || |\`\ | (_) || |_) )(  ___/
(_) (_) (_)(_)(_) (_)(_)(_) (_)`\___/'(_,__/'`\____)

$ sudo -i
# cat /version.json 
{"iso_version": "v1.33.0", "kicbase_version": "v0.0.43-1713236840-18649", "minikube_version": "v1.33.0", "commit": "4bd203f0c710e7fdd30539846cf2bc6624a2556d"}
# systemctl status systemd-resolved.service
● systemd-resolved.service - Network Name Resolution
     Loaded: loaded (/usr/lib/systemd/system/systemd-resolved.service; enabled; preset: enable
     Active: active (running) since Mon 2024-04-22 19:27:01 UTC; 5min ago
       Docs: man:systemd-resolved.service(8)
             man:org.freedesktop.resolve1(5)
             https://www.freedesktop.org/wiki/Software/systemd/writing-network-configuration-m
             https://www.freedesktop.org/wiki/Software/systemd/writing-resolver-clients
   Main PID: 191 (systemd-resolve)
     Status: "Processing requests..."
      Tasks: 1 (limit: 18837)
     Memory: 1.4M
        CPU: 295ms
● systemd-resolved.service - Network Name Resolution
     Loaded: loaded (/usr/lib/systemd/system/systemd-resolved.service; enabled; preset: enabled)
     Active: active (running) since Mon 2024-04-22 19:27:01 UTC; 9min ago
       Docs: man:systemd-resolved.service(8)
             man:org.freedesktop.resolve1(5)
             https://www.freedesktop.org/wiki/Software/systemd/writing-network-configuration-managers
             https://www.freedesktop.org/wiki/Software/systemd/writing-resolver-clients
   Main PID: 191 (systemd-resolve)
     Status: "Processing requests..."
      Tasks: 1 (limit: 18837)
     Memory: 1.4M
        CPU: 327ms
     CGroup: /system.slice/systemd-resolved.service
             └─191 /usr/lib/systemd/systemd-resolved

Apr 22 19:34:07 minikube systemd-resolved[191]: DNSSEC validation failed for question io IN SOA: failed-auxiliary
● systemd-resolved.service - Network Name Resolution
     Loaded: loaded (/usr/lib/systemd/system/systemd-resolved.service; enabled; preset: enabled)
     Active: active (running) since Mon 2024-04-22 19:27:01 UTC; 11min ago
       Docs: man:systemd-resolved.service(8)
             man:org.freedesktop.resolve1(5)
             https://www.freedesktop.org/wiki/Software/systemd/writing-network-configuration-managers
             https://www.freedesktop.org/wiki/Software/systemd/writing-resolver-clients
   Main PID: 191 (systemd-resolve)
     Status: "Processing requests..."
      Tasks: 1 (limit: 18837)
     Memory: 1.4M
        CPU: 341ms
     CGroup: /system.slice/systemd-resolved.service
             └─191 /usr/lib/systemd/systemd-resolved

Apr 22 19:34:07 minikube systemd-resolved[191]: DNSSEC validation failed for question io IN SOA: failed-auxiliary
Apr 22 19:34:07 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN DS: failed-auxiliary
Apr 22 19:34:07 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN SOA: failed-auxiliary
● systemd-resolved.service - Network Name Resolution
     Loaded: loaded (/usr/lib/systemd/system/systemd-resolved.service; enabled; preset: enabled)
     Active: active (running) since Mon 2024-04-22 19:27:01 UTC; 11min ago
       Docs: man:systemd-resolved.service(8)
             man:org.freedesktop.resolve1(5)
             https://www.freedesktop.org/wiki/Software/systemd/writing-network-configuration-managers
             https://www.freedesktop.org/wiki/Software/systemd/writing-resolver-clients
   Main PID: 191 (systemd-resolve)
     Status: "Processing requests..."
      Tasks: 1 (limit: 18837)
     Memory: 1.4M
        CPU: 344ms
     CGroup: /system.slice/systemd-resolved.service
             └─191 /usr/lib/systemd/systemd-resolved

Apr 22 19:34:07 minikube systemd-resolved[191]: DNSSEC validation failed for question io IN SOA: failed-auxiliary
Apr 22 19:34:07 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN DS: failed-auxiliary
Apr 22 19:34:07 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN SOA: failed-auxiliary
Apr 22 19:34:07 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN A: failed-auxiliary
Apr 22 19:34:07 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN AAAA: failed-auxiliary
Apr 22 19:36:06 minikube systemd-resolved[191]: DNSSEC validation failed for question io IN SOA: failed-auxiliary
Apr 22 19:36:06 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN DS: failed-auxiliary
Apr 22 19:36:06 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN SOA: failed-auxiliary
Apr 22 19:36:06 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN AAAA: failed-auxiliary
Apr 22 19:36:06 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN A: failed-auxiliary

~
[logs.txt](https://github.com/kubernetes/minikube/files/15067876/logs.txt)

logs are attached

[llegolas]

the full systemd-resolved.service logs:

# journalctl -u  systemd-resolved.service|tee
Apr 22 19:27:01 minikube systemd[1]: Starting Network Name Resolution...
Apr 22 19:27:01 minikube systemd-resolved[191]: Positive Trust Anchors:
Apr 22 19:27:01 minikube systemd-resolved[191]: . IN DS 20326 8 2 e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d
Apr 22 19:27:01 minikube systemd-resolved[191]: Negative trust anchors: home.arpa 10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa 19.172.in-addr.arpa 20.172.in-addr.arpa 21.172.in-addr.arpa 22.172.in-addr.arpa 23.172.in-addr.arpa 24.172.in-addr.arpa 25.172.in-addr.arpa 26.172.in-addr.arpa 27.172.in-addr.arpa 28.172.in-addr.arpa 29.172.in-addr.arpa 30.172.in-addr.arpa 31.172.in-addr.arpa 168.192.in-addr.arpa d.f.ip6.arpa corp home internal intranet lan local private test
Apr 22 19:27:01 minikube systemd-resolved[191]: Using system hostname 'minikube'.
Apr 22 19:27:01 minikube systemd[1]: Started Network Name Resolution.
Apr 22 19:27:04 minikube systemd-resolved[191]: Switching to fallback DNS server 1.1.1.1#cloudflare-dns.com.
Apr 22 19:27:04 minikube systemd-resolved[191]: Using degraded feature set UDP+EDNS0+DO instead of TLS+EDNS0+DO for DNS server 192.168.122.1.
Apr 22 19:27:21 minikube systemd-resolved[191]: Failed to send hostname reply: Transport endpoint is not connected
Apr 22 19:30:12 minikube systemd-resolved[191]: DNSSEC validation failed for question io IN SOA: failed-auxiliary
Apr 22 19:30:12 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN DS: failed-auxiliary
Apr 22 19:30:12 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN SOA: failed-auxiliary
Apr 22 19:30:12 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN AAAA: failed-auxiliary
Apr 22 19:30:12 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN A: failed-auxiliary
Apr 22 19:32:08 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN DS: failed-auxiliary
Apr 22 19:32:08 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN SOA: failed-auxiliary
Apr 22 19:32:08 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN A: failed-auxiliary
Apr 22 19:32:08 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN AAAA: failed-auxiliary
Apr 22 19:34:07 minikube systemd-resolved[191]: DNSSEC validation failed for question io IN SOA: failed-auxiliary
Apr 22 19:34:07 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN DS: failed-auxiliary
Apr 22 19:34:07 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN SOA: failed-auxiliary
Apr 22 19:34:07 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN A: failed-auxiliary
Apr 22 19:34:07 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN AAAA: failed-auxiliary
Apr 22 19:36:06 minikube systemd-resolved[191]: DNSSEC validation failed for question io IN SOA: failed-auxiliary
Apr 22 19:36:06 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN DS: failed-auxiliary
Apr 22 19:36:06 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN SOA: failed-auxiliary
Apr 22 19:36:06 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN AAAA: failed-auxiliary
Apr 22 19:36:06 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN A: failed-auxiliary
Apr 22 19:38:35 minikube systemd-resolved[191]: DNSSEC validation failed for question io IN SOA: failed-auxiliary
Apr 22 19:38:35 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN DS: failed-auxiliary
Apr 22 19:38:35 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN SOA: failed-auxiliary
Apr 22 19:38:35 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN A: failed-auxiliary
Apr 22 19:38:35 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN AAAA: failed-auxiliary
Apr 22 19:42:15 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN DS: failed-auxiliary
Apr 22 19:42:15 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN SOA: failed-auxiliary
Apr 22 19:42:15 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN A: failed-auxiliary
Apr 22 19:42:15 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN AAAA: failed-auxiliary
Apr 22 19:46:31 minikube systemd-resolved[191]: Grace period over, resuming full feature set (TLS+EDNS0+DO) for DNS server 192.168.122.1.
Apr 22 19:46:31 minikube systemd-resolved[191]: Using degraded feature set UDP+EDNS0+DO instead of TLS+EDNS0+DO for DNS server 192.168.122.1.
Apr 22 19:48:30 minikube systemd-resolved[191]: DNSSEC validation failed for question io IN SOA: failed-auxiliary
Apr 22 19:48:30 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN DS: failed-auxiliary
Apr 22 19:48:30 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN SOA: failed-auxiliary
Apr 22 19:48:30 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN AAAA: failed-auxiliary
Apr 22 19:48:30 minikube systemd-resolved[191]: DNSSEC validation failed for question quay.io IN A: failed-auxiliary
# 

[medyagh]

@llegolas and @cirix
for both of you I see "minikube start" logs detected an abnormality in your network access to registry and warns that
`` This VM is having trouble accessing https://registry.k8s.io```

**Question1 :**When you downgrade to 1.32 does minikube start still give you the same warning ?

Question2:
Can you plz share, the output of bellow command (in the same terminal as minikube)

curl -sS -m 2 https://registry.k8s.io/

(this runs the same check that minikube does to check access to registry.k8s.io)

@llegolas and @cirix are you both on a corp network with a custom DNS? or do you need to set a Proxy to access registries ? (if yes then you would need to set the HTTP_PROXY envs) in the same terminal
#15021

in our website we have a known issue for Builtin Network (not socket_vm network)
our site suggests https://minikube.sigs.k8s.io/docs/drivers/qemu/:

If possible, reorder your /etc/resolv.conf to have a general nameserver entry first (eg. 8.8.8.8) and reboot your machine.
Use --network=socket_vmnet

does that help you for socket vmnet too ?

btw for reference,
For me on qemu on macos M1 I do not get any error from "systemd-resolved"

$ journalctl -u  systemd-resolved.service|tee
Apr 22 18:53:08 minikube systemd[1]: Starting Network Name Resolution...
Apr 22 18:53:08 minikube systemd-resolved[170]: Positive Trust Anchors:
Apr 22 18:53:08 minikube systemd-resolved[170]: . IN DS 20326 8 2 e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d
Apr 22 18:53:08 minikube systemd-resolved[170]: Negative trust anchors: home.arpa 10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa 19.172.in-addr.arpa 20.172.in-addr.arpa 21.172.in-addr.arpa 22.172.in-addr.arpa 23.172.in-addr.arpa 24.172.in-addr.arpa 25.172.in-addr.arpa 26.172.in-addr.arpa 27.172.in-addr.arpa 28.172.in-addr.arpa 29.172.in-addr.arpa 30.172.in-addr.arpa 31.172.in-addr.arpa 168.192.in-addr.arpa d.f.ip6.arpa corp home internal intranet lan local private test
Apr 22 18:53:12 minikube systemd-resolved[170]: Using system hostname 'minikube'.
Apr 22 18:53:12 minikube systemd[1]: Started Network Name Resolution.
Apr 22 18:53:18 minikube systemd-resolved[170]: Using degraded feature set UDP+EDNS0+DO instead of TLS+EDNS0+DO for DNS server 192.168.105.1.
Apr 22 18:53:18 minikube systemd-resolved[170]: Using degraded feature set UDP+EDNS0 instead of UDP+EDNS0+DO for DNS server 192.168.105.1.
Apr 22 18:53:18 minikube systemd-resolved[170]: Server 192.168.105.1 does not support DNSSEC, downgrading to non-DNSSEC mode.
Apr 22 18:53:48 minikube systemd-resolved[170]: Clock change detected. Flushing caches.
Apr 22 19:01:12 minikube systemd-resolved[170]: Grace period over, resuming full feature set (TLS+EDNS0+DO) for DNS server 192.168.105.1.
Apr 22 19:01:12 minikube systemd-resolved[170]: Using degraded feature set UDP+EDNS0+DO instead of TLS+EDNS0+DO for DNS server 192.168.105.1.
Apr 22 19:01:12 minikube systemd-resolved[170]: Using degraded feature set UDP+EDNS0 instead of UDP+EDNS0+DO for DNS server 192.168.105.1.
$ journalctl -u  systemd-resolved.service|tee | grep fail
$ 

[medyagh]

@cirix @llegolas

can you the try ISO before this PR ? https://github.com/kubernetes/minikube/pull/18277/files
you can try using the different iso using iso-url:

minikube delete --all
minikube start --iso-url=https://storage.googleapis.com/minikube-builds/iso/17991/minikube-v1.32.1-1710520390-17991-arm64.iso

Other PRS to try:
we could try the ISO PRs one by one and see which one fixes your issue
#18277
#17991
#18213
#17206