Nuget Universal Props File

[martindevans]

Moved the .props file to build directory, instead of just netstandard2.0 directory. This should mean it executes for all platforms.

I haven't tested this, I'm not sure how to best test a nuget change. Any assistance there would be appreciated!

[SanftMonster]

On windows I take the following steps to test nuget package.

1. Get the packages with nuget pack xxx.nuspec -version xxx. Please rename the version in LLamaSharp.csproj to a unique one such as v0.11.2-test-0408 so that it won't conflict with the nuget cache.
2. Gather the backend packages to a folder.
3. Open Visual Studio nuget package manager and add the local path as package source (on the top right).

I haven't tried this way in Linux or MAC because VS only supports Windows. Maybe dotnet cli has some instructions to use local path.

[SignalRT]

I used the same approach with MacOS and Rider.

[martindevans]

Looks like this still doesn't fix it :(

The issue only seems to happen when publishing as "self contained".

[martindevans]

@AsakusaRinne @SignalRT thanks for the tips. I've been testing with a local folder as suggested and I think this now works. I'm not very confident though, I've never done much with nuget before!

I'd really appreciate it if you guys could build packages and test them yourselves.

The issue I'm trying to fix is:

1. Create a project
2. Add LLamaSharp + LLamaSharp.Backend.whatever
3. Create a new publish profile (to folder, self contained)
4. Publish it. You should get an error about duplicate files (it's trying to placed every llama.dll in the same folder).

This change should fix that last issue.

[martindevans]

I guess the tests failing tell me this did not, in fact, fix the problem :(

[SanftMonster]

There's no rush for this PR but it will be conflicted with #688 . So just a reminder here, you could complete this PR before #688 and I'll resolve the conflicts. :)

[martindevans]

So just a reminder here, you could complete this PR before #688 and I'll resolve the conflicts.

I haven't had any more time to look at it since last time, but it's been stalled because I didn't understand what caused CI to fail last time. It's obviously some kind of path resolution issue, but it's very painful to debug!

I'll close this one for now since it doesn't work at the moment and we can go ahead with your more developed PRs.

This is a really important issue though - I think it's probably the most important bug to fix in LLamaSharp!

[SanftMonster]

What about opening an issue for it so that we won't forget it?

[martindevans]

I think #382 covers it.

[SanftMonster]

@martindevans IIRC you have once tried flatten the libraries with different names but it somehow failed. May I ask about the reason of the failure? I couldn't find the related history. :)

I'm looking into this issue and am wondering if it's a possible solution:

1. rename all the libraries but we could choose to keep the current directory structure, like below.
2. After dynamic selection, copy the selected file to a certain folder (runtimes/**/llama.dll is available because we have renamed the library file). Note that this change exactly matches the new interfaces in feat: optimize the native library loading. #688, that a library path should be returned from the INativeLibrary.Prepare. So it sounds reasonable to do the copying in this method, no matter for us or for users who want to implement it.

-- runtimes
    -- native
        -- win-x64
            -- avx2
               * win-x64-avx2-llama.dll
            -- cuda11
               * win-x64-cuda11-llama.dll
        -- osx-arm64
            ...
        -- linux-x64
            ...

# If we select windows cuda 11 and avx2 backend in proper order on Windows, the file structure will become what's shown below

-- runtimes
    -- native
        -- win-x64
            -- avx2
               * win-x64-avx2-llama.dll
               * llama.dll  # file is copied.
            -- cuda11
               * win-x64-cuda11-llama.dll
               * llama.dll  # file is copied.
        -- osx-arm64
            ...
        -- linux-x64
            ...

TBH I suspected path could not be found correctly after publishing if we take the way below. Therefore I'm also wondering the following alternative.

-- llama
    * win-x64-avx2-llama.dll
    * win-x64-cuda11-llama.dll
    * linux-x64-cuda12-llama.so
    ...

# If we select windows cuda 11 and avx2 backend in proper order on Windows, the file structure will become what's shown below. In this way, the file structure will be resumed to what it is now! 

-- llama
    * win-x64-avx2-llama.dll
    * win-x64-cuda11-llama.dll
    * linux-x64-cuda12-llama.so
    ...
-- runtimes
    -- win-x64
        -- native
            -- avx2
               * llama.dll  # file is copied.
            -- cuda11
               * llama.dll  # file is copied.

Thus, things partially went back to your previous work. So your answer will really help a lot. :)

[martindevans]

IIRC you have once tried flatten the libraries with different names but it somehow failed. May I ask about the reason of the failure? I couldn't find the related history. :)

The issue was that I could rename the DLLs that we load directly (e.g. llama.dll). However, some DLLs we do not load ourselves (e.g. clblast.dll) and they cannot be renamed, so we could still need to handle them somehow. The folder hierarchy solves it by having the dependencies next to the ones we load directly.

Copying

Ideally I'd like a solution which doesn't need dynamically writing DLLs to folders. Most software should be installed in a readonly directory, and copying at runtime would be incompatible with that.

If we can work out how to get the folder hierarchy distributed properly it's all fine. I think this PR was 90% of the way there, and actually was just breaking in CI.

[SanftMonster]

Thank you for the clarification! Glad to hear that it is going to be resolved. I'll head to another issues then