Not compatible with docker without GLIBC_2.33

[fish-li]

I encountered an error while deploying with docker:

System.TypeInitializationException: The type initializer for 'LLama.Native.NativeApi' threw an exception.
 ---> LLama.Exceptions.RuntimeError: The native library cannot be correctly loaded. It could be one of the following reasons: 
1. No LLamaSharp backend was installed. Please search LLamaSharp.Backend and install one of them. 
2. You are using a device with only CPU but installed cuda backend. Please install cpu backend instead. 
3. One of the dependency of the native library is missed. Please use `ldd` on linux, `dumpbin` on windows and `otool`to check if all the dependency of the native library is satisfied. Generally you could find the libraries under your output folder.
4. Try to compile llama.cpp yourself to generate a libllama library, then use `LLama.Native.NativeLibraryConfig.WithLibrary` to specify it at the very beginning of your code. For more informations about compilation, please refer to LLamaSharp repo on github.

   at LLama.Native.NativeApi..cctor()
   --- End of inner exception stack trace ---
   at LLama.Native.NativeApi.llama_log_set(LLamaLogCallback logCallback)
   at AiModelService.Utils.LLamaFactory.InitAllModels() in D:\WorkCode\YkfAi\src\AiModelService\Utils\LLamaFactory.cs:line 32
   at AiModelService.AppInitializer.Init() in D:\WorkCode\YkfAi\src\AiModelService\AppInitializer.cs:line 14

enter the docker container:

root@8e362e937e24:/app/runtimes/linux-x64/native# ldd libllama.so 
./libllama.so: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by ./libllama.so)
./libllama.so: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by ./libllama.so)
./libllama.so: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by ./libllama.so)
	linux-vdso.so.1 (0x00007ffcc97d9000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f475b0ea000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f475afa6000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f475af8c000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f475adb8000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f475b455000)

my base image is: mcr.microsoft.com/dotnet/aspnet:7.0.13

by searching documents, i know : mcr.microsoft.com/dotnet/aspnet:7.0.13 is based on Debian11,
Debian11 not support GLIBC_2.33,
Debian12 can support GLIBC_2.33,

Refer to: https://hub.docker.com/_/microsoft-dotnet-aspnet/?tab=description
upgrade to mcr.microsoft.com/dotnet/aspnet:7.0.13-bookworm-slim-amd64
And then docker run it again,

docker exit (132)
No error output or log !

Project PackageReference:

<PackageReference Include="LLamaSharp" Version="0.8.0" />
<PackageReference Include="LLamaSharp.Backend.Cpu" Version="0.8.0" />

[SanftMonster]

The result of ldd shows that your container does not have GLIBC_2.33 and GLIBC_2.34, which means that maybe your image is a previous linux release such as ubuntu 20.04 LTS.

For a quick fix you can compile the llama.cpp yourself in the container and specify using your own libllama.so with NativeLibraryConfig.Default.WithLibrary({PATH}).

@martindevans In fact I also met this problem last weekend on my WSL2. It's most likely because the linux libraries were all compiled with ubuntu 22.04 (ubuntu-latest in github workflow). Maybe we could optimize it with one of the three ways:

1. Just find a way to compile all the libraries under ubuntu-20.04 (I think its resultm wll also work on 22.04)
2. Split linux backend package into two kinds.
3. Add a method to download correct version of llama.cpp and using the compiled libllama. Therefore user's can focus on their code, instead of cpp code. (Also benifit Windows and MAC)

[martindevans]

I think we should try option #1 first, just because it's the simplest. We can easily bump the runner down to an older version of Ubuntu and that should cover most use cases.

[SanftMonster]

I think we should try option #1 first, just because it's the simplest. We can easily bump the runner down to an older version of Ubuntu and that should cover most use cases.

It makes sense to me :)

[martindevans]

I've just submitted #297 which moves the compile job from ubuntu-latest to ubuntu-20.04.

I've started a test run of that job here. When that's finished @fish-li can you please download the new binaries and confirm if they load correctly for you.

Note that when testing it will probably crash in LLamaSharp because it's an incompatible version of llama.cpp. All we need to check is that it successfully loads into the process!

[fish-li]

I've just submitted #297 which moves the compile job from ubuntu-latest to ubuntu-20.04.

I've started a test run of that job here. When that's finished @fish-li can you please download the new binaries and confirm if they load correctly for you.

Note that when testing it will probably crash in LLamaSharp because it's an incompatible version of llama.cpp. All we need to check is that it successfully loads into the process!

Great! now that problem doesn't exist, thank you! @martindevans

[fish-li]

By the way, upgrade to .net8 also solves this problem.

I encountered docker exit (132) earlier because my dummy CPU does not support the AVX instruction set

[martindevans]

I'll close this issue now since it looks like this has been resolved by #297 and the latest release (0.9.1) should include those fixes. Please do not hesitate to re-open the issue if it's still a problem!