Skip to content

LLava_shared.dll in LLamaSharp.Backend.Cuda12 is for CPU only #639

@IntptrMax

Description

@IntptrMax

The llava_shared.dll in LLamaSharp.Backend.Cuda12 is only 850KB, the file size is much smaller than llava_shared.dll with cuda. It will take about 126000+ ms to embding an image. Take it to llava_shared.dll from llama.cpp release 2214, image embding time will be no more than 1000 ms. Will this can be updated in next LLamaSharp release?

[Current]
image

[Dll replaced]
image

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions