Godot game engine example

[adammikulis]

I have put together a plugin for the Godot game engine that uses LLamaSharp to load a model and interact directly within the editor. I'd like to submit it as a documentation example, but it could probably use some polishing first.

Currently I'm having an issue figuring out how to automatically add the NuGet packages like the Unity example does (it looks like Unity has NuGetForUnity but there is no Godot equivalent). If anyone knows how I can do this so that the user doesn't have to open Visual Studio and manage the solution that would be great. Please let me know what other changes/improvements I should make before it could be listed in the README, thanks.

[SanftMonster]

@eublefar @Xsanf Maybe you'll be interested in this issue. I hope this mention didn't disturb you. :)

[eublefar]

I am not familiar with Godot, but maybe you could just download all NuGet dependencies and add them to the plugin manually?
You can just click Download package on nuget page, rename it's extension to zip and extract the dlls from there.
I am not sure if it will cause assembly conflicts tho.

[adammikulis]

Thanks @eublefar, I was wondering the best way to extract those files (aside from going through an existing build). I've added them to the plugin folder and will try to manually reference them in the project. I'm wondering if PR #603 (SetDllImportResolver based loading) will help with this, as the end-user of the plugin may not always have an Nvidia gpu. It seems like either way the user will need a .csproj file, making this more suitable as a template but unable to be added to the Godot AssetLib as a typical plugin.

[eublefar]

end-user of the plugin may not always have an Nvidia gpu

You can add CPU backend by default and add some comments where to get CUDA backend.

[SanftMonster]

end-user of the plugin may not always have an Nvidia gpu

@adammikulis Maybe you could include both cpu and cuda native libraries and use LLama.Native.NativeLibraryConfig to select the backend dynamically at the beginning of the program. Note that installing both cpu and cuda backends is supported in master branch but not v0.10.0. If you want to use v0.10.0, please keep those files manually.

[adammikulis]

Thank you for the help! I've made some significant updates to my Godot add-on that bring it closer to being a useful example project. At this point I'm just trying to resolve the issue of properly referring to my backends in my .csproj.

I was able to successfully refer to a locally downloaded LLamaSharp.dll by adding a path reference to my .csproj (within an ItemGroup) but I am unable to get the Assemblies to recognize the Cuda backend I've manually added. Am I using the correct name for it? I also tried LLama and LLama.Native.

<Reference Include="llama"> <HintPath>addons\mind_game\libs\runtimes\win-x64\native\cuda12\llama.dll</HintPath> <Private>True</Private> </Reference>

I am calling NativeLibraryConfig.Instance.WithCuda(); as soon as the project scene loads, so I don't think that's the problem.

[SanftMonster]

runtimes\win-x64\native\cuda12\llama.dll is the correct path, but I'm not sure whether its prefix is correct. Since you've already had an specific path of native library, you can also use NativeLibraryConfig.Instance.WithLibrary.

BTW, it might be a stupid question but may I ask whether .NET runtime higher than coreapp 3.1 could be used in godot?

[adammikulis]

For some reason I cannot get Godot to use my GPU if the CPU backend is installed, so I'll omit that backend for now and am following #670. As I am distributing Mind Game as an engine add-on, many people who use it will be well-versed with Godot but possibly not so much with NuGet. The ability to dynamically download the correct backend will go a long way towards its usability both as a technical tool and as part of a finished game.

Godot 4.3 dev6 is .NET8-compatible, but they're still working on getting mobile exports to work. Web export will take additional time, but I should be able to get LLamaSharp working on basically every device via this addon.

I have recently updated the addon to save and autoload model/inference user configurations, so with some review it may be ready to be an example project.

[SanftMonster]

@adammikulis Hi, are you using .NET standard2.0?

[adammikulis]

Hi @AsakusaRinne, I am using .NET 8.0 for this project, should I use a lower version? I'm looking to have AOT compile so this runs well on mobile devices, and I don't think .NET standard 2.0 supports that. I think I'll just try out the experimental backend downloading when it's available, as I'm also looking to integrate downloading of the models themselves into the add-on (it seems like that's another feature you all are working on). This honestly isn't an issue yet since I don't think anyone has used this add-on, or if they did they must have had an Nvidia GPU since nobody has reported an issue.

[SanftMonster]

should I use a lower version?

Nope, .NET 8 is better. I don't know much about Godot so I thought it's the same as unity, which only supports .NET standard2.0. :) Currently all the releases of LLamaSharp only supports dynamic native library loading with .NET6 or higher. The support for .NET standard2.0 will be added in #738.

---

I'll merge #688 in 2 days (with some documentation added), and then move on in #692.

I'd like to hear from you about the security concern of the backend-downloading feature. As mentioned in this comment, in some cases using this feature can lead to a security problem. After having #688 merged, it's possible to put the DLLs in your own remote resource and define the downloading process yourself. However I'm not sure whether this reduces the risk to the point where it can be used for game release.

[adammikulis]

Unity is known for its C# support, but ever since their runtime fee debacle there has been a lot of development with Godot to support C#/.NET better. Mobile export support should be available in the next few months, but HTML5 doesn't have a timeline.

I can't really argue with the security concern mentioned, despite the convenience that the auto-download would be. I think with LLamaSharp geared towards enterprise environments (or at least moreso than Python libraries), @dluc's point about the possible backdoor makes me wonder about the pushback such a feature could receive.

For my project, either I'm doing something wrong when trying to use NativeLibraryConfig.Instance.WithCuda(true); or there is some sort of backend loading that happens in Godot before the config is run (I have it as the first line executed in the first script that is autoloaded when the engine starts, and I'm not using any LLamaSharp methods/vars before calling that line). Let me know if you're able to test the current release (v0.2), or even the current dev branch if you're interested in playing around with Godot.

[SanftMonster]

I think it's a LLamaSharp issue rather than your fault. For example, if the godot output folder has a different directory structure with common .NET apps, it may lead to such an error.

Would you like to give an out-of-the -box example to reproduce this error? Since I know nothing about godot, it will save me a lot of time if I have such an example, without the need to configure the project on my own. I'll install godot tonight to take a look into it.