feat: update the llama backends.

[SanftMonster]

No description provided.

[martindevans]

It looks like something has gone wrong merging the changes in FixedSizeQueue?

[SanftMonster]

It looks like something has gone wrong merging the changes in FixedSizeQueue?

Yes and I've fixed them.

It's quite strange that I could run the test on my local linux device successfully. Besides when I list the objects of LLama.Unittest/bin/Debug/net6.0 in ci, all the libraries are there. However the ci log tells me that the library file could not be found. Any idea about that?

[martindevans]

It's quite strange that I could run the test on my local linux device successfully. Besides when I list the objects of LLama.Unittest/bin/Debug/net6.0 in ci, all the libraries are there. However the ci log tells me that the library file could not be found. Any idea about that?

I've been investigating this but I can't see any problems with the setup.

My best guess is that a dependency of libllama.so is missing, although I'm not sure what. perhaps try adding ldd -r libllama.so to the CI, to print out all dependencies?

Edit: I've added this now.

[martindevans]

@AsakusaRinne I think I may have discovered the problem, but I'm not certain.

I ran ldd -r libllama.so locally and got this result:

$ldd -r libllama.so
        linux-vdso.so.1 (0x00007ffec9a8f000)
        libopenblas.so.0 => not found
        libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f74d2391000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f74d2242000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f74d2227000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f74d2204000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f74d2012000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f74d26c9000)
undefined symbol: cblas_sgemm   (./libllama.so)

i.e. the binary is trying to dynamically link against libopenblas which is not installed for me. I am guessing that it's not installed on the Github CI runners either.

From looking at the readme (here) I assume you have set LLAMA_OPENBLAS=1 when building?

I suggest building without LLAMA_OPENBLAS for now. If that fixes it we can maybe re-add it in the future with something like #65 which checks for libopenblas at runtime and uses it if it's available.

[SanftMonster]

@AsakusaRinne I think I may have discovered the problem, but I'm not certain.

I ran ldd -r libllama.so locally and got this result:

$ldd -r libllama.so
        linux-vdso.so.1 (0x00007ffec9a8f000)
        libopenblas.so.0 => not found
        libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f74d2391000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f74d2242000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f74d2227000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f74d2204000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f74d2012000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f74d26c9000)
undefined symbol: cblas_sgemm   (./libllama.so)

i.e. the binary is trying to dynamically link against libopenblas which is not installed for me. I am guessing that it's not installed on the Github CI runners either.

From looking at the readme (here) I assume you have set LLAMA_OPENBLAS=1 when building?

I suggest building without LLAMA_OPENBLAS for now. If that fixes it we can maybe re-add it in the future with something like #65 which checks for libopenblas at runtime and uses it if it's available.

Yes, it's better to remove openblas and I'll re-compile them. Thank you for the catch!

[SanftMonster]

As shown in the ci log, the MACOS in the ci uses intel cpu instead of M1/M2 cpu. Maybe we should remove the ci for MACOS?

[martindevans]

That's unfortunate, but I don't see any other option. Let's hope GitHub add ARM based runners soon!