ERR error installing backend - failed to download layer 0: unexpected EOF

[Hello-World-Traveler]

LocalAI version:
v3.2.2 (47ccfcc)

Environment, CPU architecture, OS, and Version:
Docker

Describe the bug
9:48PM ERR error installing backend localai@cuda12-llama-cpp error="failed to download backend "quay.io/go-skynet/local-ai-backends:latest-gpu-nvidia-cuda-12-llama-cpp": failed to download image tar: failed to download layer 0: unexpected EOF"

To Reproduce
Download backend

Expected behavior
Download and install

Logs
9:48PM ERR error installing backend localai@cuda12-llama-cpp error="failed to download backend "quay.io/go-skynet/local-ai-backends:latest-gpu-nvidia-cuda-12-llama-cpp": failed to download image tar: failed to download layer 0: unexpected EOF"

Additional context
Why can't the install continue like downloading models? if downloading backends fail, the install must start again.

[Hello-World-Traveler]

Could be a docker issue. They are downloaded as images.
https://quay.io/repository/go-skynet/local-ai-backends

[mudler]

I can't really reproduce this. Note that LocalAI does not use docker to pull images. It might be rather a network issue here. Did you tried from master? it has support for mirrors and fallbacks to images from dockerhub

[Hello-World-Traveler]

Models download fine, backends don't for me. Is there another way to download them? manual?
If a bakend fails, it has to start over again, anyway to change this?

The only master image I have is for 3.0, should I pull the master image and try that?

[mudler]

Models download fine, backends don't for me. Is there another way to download them? manual? If a bakend fails, it has to start over again, anyway to change this?

There isn't really a way to workaround that, however you can still download these with other tools (but would defeat the purpose of having these handled by LocalAI) like docker, save the image locally and install it with local-ai backends install ocifile://<path_to_tar>

The only master image I have is for 3.0, should I pull the master image and try that?

I'd suggest that yes, in master now there is an added functionality that will try to another mirror if downloading fails.

Do you have issues pulling images from quay? or is it only happening when downloading backends in LocalAI?

[Hello-World-Traveler]

I'd suggest that yes, in master now there is an added functionality that will try to another mirror if downloading fails.

I'll pull the master and give it a try. Pulling images with docker pull command has no issue, just inside Local-ai

Yesterday downloaded robobrain 2.0, no issue

 Downloading 1/1 quay.io/go-skynet/local-ai-backends:latest-gpu-nvidia-cuda-12-bark: 723.2 MiB/2.9 GiB (24.43%) ETA: 26m35.885744746s

 ERR error installing backend localai@cuda12-bark error="failed to download backend \"quay.io/go-skynet/local-ai-backends:latest-gpu-nvidia-cuda-12-bark\": failed to download image tar: failed to download layer 0: unexpected EOF"

I was able to download nvidia-cuda-12-llama-cpp today and that fixed models not using GPU. Maybe error message on start up? 'UNABLE TO FIND CUDA BACKENDS FOR SOME MODELS SUCH AS_______ _____ _______" if it's set to use cuda. I know it does check what backends are installed.

I'll grab a coffee and wait to see if the master image makes any difference on downloading the images.

While you're here, quick question.
How do we know when a back-end or LLM has/needs to be updated? (not often LLM are updated) I would suggest to check the eprint data but unless it was saved on install, there is nothing to compare it too.

 title=
      year={2025},
      eprint={2503},
      archivePrefix=,
      primaryClass=,
      url=

eprint={2503}, 2025 03 (third month of the year)

[Hello-World-Traveler]

Backend Installation
Error failed to download backend "quay.io/go-skynet/local-ai-backends:latest-gpu-nvidia-cuda-12-transformers": failed to download image tar: failed to download layer 0: unexpected EOF

No difference on master image. When I next get a chance, I'll try local-ai backends install ocifile://<path_to_tar> after using the docker pull command to grab the image.

[Hello-World-Traveler]

@mudler 3.3.0 download newest model and back-end

Installation
Error failed to download backend "quay.io/go-skynet/local-ai-backends:latest-gpu-nvidia-cuda-12-rfdetr": failed to download image tar: failed to download layer 0: unexpected EOF

Backend Installation
Error failed to download backend "quay.io/go-skynet/local-ai-backends:latest-gpu-nvidia-cuda-12-exllama2": failed to download image tar: failed to download layer 0: unexpected EOF

[Hello-World-Traveler]

Pulling quay.io/go-skynet/local-ai-backends:latest-gpu-nvidia-cuda-12-diffusers I had notice it's only one layer, docker has no issue getting it however I also notice it want from 200/MB/1.99GB to 120MB/1.2GB down to 600MB while downloading but never stopped. The issue is the Web-UI and the way it is downloading.

Logging into the container and using /local-ai backends install ocifile://backends/tmp/latest-gpu-nvidia-cuda-12-diffusers.tar

7:33AM DBG Setting logging to debug
WARNING: failed to read int from file: open /sys/class/drm/card0/device/numa_node: no such file or directory
WARNING: error parsing the pci address "simple-framebuffer.0"
7:33AM DBG GPU vendor gpuVendor=intel
7:33AM INF Installing backend from OCI image backend=ocifile://backends/tmp/latest-gpu-nvidia-cuda-12-diffusers.tar name=latest-gpu-nvidia-cuda-12-diffusers

7:33AM INF Installing backend from OCI image backend=ocifile://backends/tmp/latest-gpu-nvidia-cuda-12-diffusers.tar name=latest-gpu-nvidia-cuda-12-diffusers
downloading backend ocifile://backends/tmp/latest-gpu-nvidia-cuda-12-diffusers.tar  96% |██████████████████████████████████████  |  [1m2s:1s]  

Restart
Not installed
can see all the files in the folder.

[Hello-World-Traveler]

@mudler any chance this set to be fixed on next update?

[mudler]

@mudler any chance this set to be fixed on next update?

I don't have still a clear idea on what is causing this to happen.

I can't really reproduce it here, I don't think it's really the WebUI, to confirm this - can you try installing the backend with the CLI, for instance by running local-ai backends install llama-cpp?

[mudler]

also, do you have issues downloading models? could it be an mtu issue here?

[Hello-World-Traveler]

As I said before, I can download from docker without error and can download models.

10:11PM DBG Setting logging to debug
WARNING: failed to read int from file: open /sys/class/drm/card0/device/numa_node: no such file or directory
WARNING: error parsing the pci address "simple-framebuffer.0"
10:11PM DBG GPU vendor gpuVendor=intel
10:11PM DBG Installing backend from gallery galleries=[{"name":"localai","url":"github:mudler/LocalAI/backend/index.yaml@master"}] name=llama-cpp
10:11PM DBG Backend is a meta backend name=llama-cpp systemState={"GPUVendor":"intel"}
10:11PM INF Using forced capability from environment variable (LOCALAI_FORCE_META_BACKEND_CAPABILITY) capability=nvidia
10:11PM DBG Using reported capability capMap={"amd":"rocm-llama-cpp","darwin-x86":"darwin-x86-llama-cpp","default":"cpu-llama-cpp","intel":"intel-sycl-f16-llama-cpp","metal":"metal-llama-cpp","nvidia":"cuda12-llama-cpp","nvidia-l4t":"nvidia-l4t-arm64-llama-cpp","vulkan":"vulkan-llama-cpp"} reportedCapability=nvidia
10:11PM INF Using forced capability from environment variable (LOCALAI_FORCE_META_BACKEND_CAPABILITY) capability=nvidia
10:11PM DBG Using reported capability capMap={"amd":"rocm-llama-cpp","darwin-x86":"darwin-x86-llama-cpp","default":"cpu-llama-cpp","intel":"intel-sycl-f16-llama-cpp","metal":"metal-llama-cpp","nvidia":"cuda12-llama-cpp","nvidia-l4t":"nvidia-l4t-arm64-llama-cpp","vulkan":"vulkan-llama-cpp"} reportedCapability=nvidia
10:11PM DBG Found backend for reported capability backend=llama-cpp reportedCapability=nvidia
10:11PM DBG Installing backend from meta backend bestBackend=cuda12-llama-cpp name=llama-cpp
downloading backend llama-cpp   3% |█                                       |  [31s:13m38s]

This was already installed, no errors.

Downloading transformers

/local-ai backends install cuda12-transformers
9:42PM DBG Setting logging to debug
WARNING: failed to read int from file: open /sys/class/drm/card0/device/numa_node: no such file or directory
WARNING: error parsing the pci address "simple-framebuffer.0"
9:42PM DBG GPU vendor gpuVendor=intel
9:42PM DBG Installing backend from gallery galleries=[{"name":"localai","url":"github:mudler/LocalAI/backend/index.yaml@master"}] name=cuda12-transformers
downloading backend cuda12-transformers  16% |██████                                  |  [8m22s:40m37s]

FTL Error running the application error="failed to download backend \"quay.io/go-skynet/local-ai-backends:latest-gpu-nvidia-cuda-12-transformers\": failed to download image tar: failed to download layer 0: unexpected EOF\nerror installing backend cuda12-transformers"

Downloaded diffusers and inside container: /local-ai backends install ocifile://backends/tmp/latest-gpu-nvidia-cuda-12-diffusers.tar didn't end up installing for some reason, files are in the folder. Some how it never got linked to being "installed"

[Hello-World-Traveler]

@mudler Thank you for the latest update. Back-ends now show if they are located in the back-ends folder however, if the install failed it will create a folder and list the back-end as installed.

Maybe after it detects a new installed, it tests it before registering it as installed?