Colima hangs a short while after starting

[blame-git]

Description

Shortly after starting a colima instance it seem to hang, and I’m unable to ssh into the session

% colima -v start -t vz -c 8 -m 8 --mount /Volumes/Void:w --mount-type virtiofs --mount /Volumes/Stash:w --mount-type virtiofs --mount /Volumes/Vacuum:w --mount-type virtiofs

% colima list
PROFILE    STATUS     ARCH       CPUS    MEMORY    DISK     RUNTIME    ADDRESS
default    Running    aarch64    8       8GiB      60GiB    docker     

# Roughly 5~10 minutes later
% colima ssh
FATA[0006] exit status 255

From ~/.lima/colima/ha.stderr.log I see:

{"level":"error","msg":"write unixgram -\u003e: write: no buffer space available","time":"2022-12-30T11:24:42-08:00"}
{"level":"error","msg":"cannot receive packets from , disconnecting: cannot read size from socket: read unixgram -\u003e: use of closed network connection","time":"2022-12-30T11:24:42-08:00"}{"level":"error","msg":"virtual network error: \"cannot read size from socket: read unixgram -\u003e: use of closed network connection\"","time":"2022-12-30T11:24:42-08:00"}

• [ ]

Version

Colima Version:

colima version HEAD-88390f5
git commit: 88390f54bceb72e248044aa3b452b64c676d99d1

Lima Version: limactl version 0.14.2
Qemu Version: qemu-img version 7.2.0
macOS Version: 13.1 22C65

Operating System

• macOS Intel <= 12 (Monterrey)
• macOS Intel >= 13 (Ventura)
• macOS M1 <= 12 (Monterrey)
• macOS M1 >= 13 (Ventura)
• Linux

Output of colima status

% colima status
FATA[0003] error retrieving current runtime: empty value

vm-type: vz
mount type: virtiofs

Reproduction Steps

1. Start Colima
2. 3 Docker containers start
3. Wait about 5-10 minutes

Expected behaviour

No response

Additional context

No response

[abiosoft]

Thanks for reporting this. I have experienced same with the VZ vm type and still troubleshooting.

If you do not need the faster filesystem access, QEMU is more stable at the moment.

[balajiv113]

@blame-git
If you could mention about the kind of network activity happening around that time (even your running containers info if possible) it would help debug/fix this issue better.

I have been trying to reproduce this but couldn't find a way

[abiosoft]

@balajiv113 I am not sure what fixed it precisely but I'm suspecting its lima-vm/lima#1261.

I am still monitoring it but it seems to have stabilised now and no longer freezing.

[Sangeppato]

I'm still experiencing this issue from time to time, but I haven't found a way to systematically reproduce it

[blame-git]

I’m still able to reproduce, but it is less frequent now. What seems to do it for me, is using a container that downloads files from the internet at high speed, and writing them out to volumes (mounted via virtiofs)

[blame-git]

Oops, didn’t meant to close

[balajiv113]

@blame-git

What seems to do it for me, is using a container that downloads files from the internet at high speed, and writing them out to volumes

Is this steps makes colima hang ?? If yes, can you share some command to redo this steps??

[mcelen35]

Bumping, I'm have the exact same issue.

[developerrespig]

Also having this issue and it drives me nuts :(

[balajiv113]

If its consistently reproducible please do share the steps.

Having hard time in reproducing this. It is happening but not always.

[mcelen35]

@balajiv113, there's no weird steps actually.

Just start colima, start couple of containers, it hangs after 5 min.

I have mongodb,nats,mysql,redis (from bitnami) containers only, within a custom network.

[balajiv113]

I have mongodb,nats,mysql,redis (from bitnami) containers only, within a custom network.

If you could share the docker-compose for this setup it would be great.