Skip to content

Conversation

@evolcano
Copy link
Contributor

This commit is originally made by lcarrere in #180 .

I have confirmed this modification is OK in my windows 11 laptop, MAUI could load GGUF model and infer successfully.

I make this commit according require of AsakusaRinne.

This commit is originally made by lcarrere in SciSharp#180 .

I have confirmed this modification is OK in my windows 11 laptop, add make this commit according require of AsakusaRinne.
@martindevans
Copy link
Member

This needs to be skipped on non-Windows platforms, or else it'll fail because the imported method does not exist. e.g. check System.Runtime.InteropServices.RuntimeInformation.IsOSPlatform(OSPlatform.Windows); and skipping the call should probably work.

@evolcano
Copy link
Contributor Author

RuntimeInformation.IsOSPlatform(OSPlatform.Windows) is already exist, the imported method AddDllDirectory will only run in windows platform:

        private static int GetCudaMajorVersion()
        {
            string? cudaPath;
            string version = "";
            if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
            {
                cudaPath = Environment.GetEnvironmentVariable("CUDA_PATH");
                if (cudaPath is null)
                {
                    return -1;
                }

                //Ensuring cuda bin path is reachable. Especially for MAUI environment.
                string cudaBinPath = Path.Combine(cudaPath, "bin");

                if (Directory.Exists(cudaBinPath))
                {
                    AddDllDirectory(cudaBinPath);
                }

                version = GetCudaVersionFromPath(cudaPath);
            }
            else if (RuntimeInformation.IsOSPlatform(OSPlatform.Linux))
            {
                // Try the default first
                cudaPath = "/usr/local/bin/cuda";
                version = GetCudaVersionFromPath(cudaPath);
                if (string.IsNullOrEmpty(version))
                {
                    cudaPath = Environment.GetEnvironmentVariable("LD_LIBRARY_PATH");
                    if (cudaPath is null)
                    {
                        return -1;
                    }
                    foreach (var path in cudaPath.Split(':'))
                    {
                        version = GetCudaVersionFromPath(Path.Combine(path, ".."));
                        if (string.IsNullOrEmpty(version))
                        {
                            break;
                        }
                    }
                }
            }

            if (string.IsNullOrEmpty(version))
                return -1;

            version = version.Split('.')[0];
            if (int.TryParse(version, out var majorVersion))
                return majorVersion;

            return -1;
        }

@martindevans, did I missed something?

@martindevans
Copy link
Member

Ah sorry, I didn't zoom out far enough in the diff to see that! In that case it all looks fine to me.

Copy link
Collaborator

@SanftMonster SanftMonster left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you a lot for this contribution!

@SanftMonster SanftMonster added the bug Something isn't working label Mar 31, 2024
@SanftMonster SanftMonster merged commit f1cb0e3 into SciSharp:master Mar 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants