docs!: adjust external provider docs #3484
Conversation
cdoern
requested review from
ashwinb,
yanxi0830,
hardikjshah,
raghotham,
ehhuang,
terrytangyuan,
leseb,
bbrowning,
reluctantfuturist,
mattf and
slekkala1
as code owners
meta-cla
bot
added
the
CLA Signed
cdoern
force-pushed
the
external-docs
branch
from
8cf765d to
53e74d1
Compare
| - aiohttp | ||
| config_class: llama_stack_ollama_provider.config.OllamaImplConfig | ||
| module: llama_stack_ollama_provider | ||
| adapter_type: custom_ollama |
There was a problem hiding this comment.
Can we actually update the doc to refer to the usage of the module field instead of the custom YAML files?
There was a problem hiding this comment.
removed everything around the external_providers_dir method. However, I think we should keep the yaml examples as they are useful for folks when writing their get_provider_spec functions since that now uses the RemoteProviderspec and InlineProviderSpec constructors.
cdoern
force-pushed
the
external-docs
branch
from
53e74d1 to
5e43981
Compare
| ### Example using `external_providers_dir`: Custom Ollama Provider | ||
|
|
||
| Here's a complete example of creating and using a custom Ollama provider: | ||
|
|
||
| 1. First, create the provider package: | ||
|
|
||
| ```bash | ||
| mkdir -p llama-stack-provider-ollama | ||
| cd llama-stack-provider-ollama | ||
| git init | ||
| uv init | ||
| ``` | ||
|
|
||
| 2. Edit `pyproject.toml`: | ||
|
|
||
| ```toml | ||
| [project] | ||
| name = "llama-stack-provider-ollama" | ||
| version = "0.1.0" | ||
| description = "Ollama provider for Llama Stack" | ||
| requires-python = ">=3.12" | ||
| dependencies = ["llama-stack", "pydantic", "ollama", "aiohttp"] | ||
| ``` | ||
|
|
||
| 3. Create the provider specification: | ||
|
|
||
| ```yaml | ||
| # ~/.llama/providers.d/remote/inference/custom_ollama.yaml | ||
| adapter: | ||
| adapter_type: custom_ollama | ||
| pip_packages: ["ollama", "aiohttp"] | ||
| config_class: llama_stack_provider_ollama.config.OllamaImplConfig | ||
| module: llama_stack_provider_ollama | ||
| api_dependencies: [] | ||
| optional_api_dependencies: [] | ||
| ``` | ||
|
|
||
| 4. Install the provider: | ||
|
|
||
| ```bash | ||
| uv pip install -e . | ||
| ``` |
There was a problem hiding this comment.
I believe this part is still relevant?
There was a problem hiding this comment.
let me add parts of this back.
I removed due to the ~/.llama/providers.d/remote/inference/custom_ollama.yaml part. the rest needs to be added back, sorry thanks for catching this.
There was a problem hiding this comment.
I added back specifically 4. Install the provider: if this is what you mean.
cdoern
force-pushed
the
external-docs
branch
from
5e43981 to
a97d41e
Compare
|
@leseb let me know how this looks! |
https://github.com/llamastack/llama-stack/pull/3484/files#r2378203883 |
cdoern
force-pushed
the
external-docs
branch
2 times, most recently
from
6548af6 to
f261705
Compare
|
@cdoern one more rebase :) |
now that we consolidated the providerspec types and got rid of `AdapterSpec`, adjust external.md and external-provider-guide.md external_providers_guide.md specifically needed some updates around the proper fields required in a ProviderSpec beyond the changes in this PR. BREAKING CHANGE: external providers must update their `get_provider_spec` function to use `RemoteProviderSpec` properly Signed-off-by: Charlie Doern <[email protected]>
|
@leseb that should do it! |
What does this PR do?
now that we consolidated the providerspec types and got rid of
AdapterSpec, adjust external.mdBREAKING CHANGE: external providers must update their
get_provider_specfunction to useRemoteProviderSpecproperly