Skip to content

Allow local providers to fetch model list #294

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

khasinski
Copy link

@khasinski khasinski commented Jul 23, 2025

What this does

This prevents model capabilities inaccurate warning from happening while keeping local model registry check in place.

# This outputs a warning, which can be silenced, but silencing it disables local model registry check
RubyLLM.chat(provider: :ollama, model: "llama3.2:latest").ask("hello?")
# W, [2025-07-23T21:22:09.067904 #50236]  WARN -- RubyLLM: Assuming model 'llama3.2:latest' exists for provider 'RubyLLM::Providers::Ollama'. Capabilities may not be accurately reflected.

# Fetching model list puts them into a local in-memory registry
RubyLLM.models.refresh!
RubyLLM.chat(provider: :ollama, model: "llama3.2:latest").ask("hello?")
# No warning

Basically an alternative to silencing log_assume_model_exists, which still validates the model's presence.

Includes model list fetching for Ollama. Since both GPUStack and Ollama can now fetch model lists running RubyLLM.models.refresh! is all that's needed to verify's model availability.

Type of change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation
  • Performance improvement

Scope check

  • I read the Contributing Guide
  • This aligns with RubyLLM's focus on LLM communication
  • This isn't application-specific logic that belongs in user code
  • This benefits most users, not just my specific use case (mostly for local model users)

Quality check

  • I ran overcommit --install and all hooks pass
  • I tested my changes thoroughly
  • I updated documentation if needed (should I update Ollama and GPUStack manuals?)
  • I didn't modify auto-generated files manually (models.json, aliases.json)

API changes

  • Breaking change
  • New public methods/classes
  • Changed method signatures
  • No API changes

Related issues

@khasinski khasinski force-pushed the fetch-models-from-ollama branch 4 times, most recently from 94f4e4e to b7ab08c Compare July 25, 2025 07:55
This prevents assume local model warning from happening while keeping local model registry.

Both GPUStack and Ollama can now fetch model lists, so in theory running RubyLLM.models.refresh! is all that's needed to verify's model availability.
@khasinski khasinski force-pushed the fetch-models-from-ollama branch from b7ab08c to 173bc25 Compare July 28, 2025 14:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant