No context length setting in advanced configuration (unlike said in the documentation)

[Fade78]

App Version

3.11.12

API Provider

Ollama

Model Used

qwq:32b, qwen2.5-coder:32b, and many more.

Actual vs. Expected Behavior

I setup my context to be right to the VRAM capacity of my GPUs. For a 32b model, that means around 24k. In Roocode context length displays, it says my model has a 128k context length.

In the documentation, there is an optional point :

• (Optional) Configure Model context size in Advanced settings, so Roo Code knows how to manage its sliding window.

Well I didn't find this at all.

Detailed Steps to Reproduce

Follow the doc of setup for an ollama connection.

Relevant API Request Output

Additional Context

I want to change the context length in case it changes something to the bug I have about a file_count problem.

[dosubot]

Hello @Fade78! I'm here to help you with your issues, answer questions, and guide you as you navigate through your contributions. While you wait for a human maintainer, let’s get started on resolving your concerns!

The issue you're experiencing seems to be related to the discrepancy between the documentation and the actual settings available in the Roo Code application. According to the documentation, there should be an option to configure the model context size in the advanced settings for the Ollama API provider ^[1]. However, it appears that this option is not visible or available in your current setup.

Given that you're using version 3.11.12, it's possible that there have been changes or updates that are not reflected in the documentation. The release notes for version 3.7.12 mention enhancements in context handling, including the expansion of max tokens for thinking models to 128k ^[2]. This might explain why the context length displays as 128k in your setup.

If the advanced configuration settings are not available as expected, it might be worth checking if there are any recent updates or patches that address this issue. Additionally, you could verify if there are any configuration files or settings that might allow manual adjustment of the context length. If the problem persists, consider reporting it as a bug to ensure it gets addressed in future updates.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[Fade78]

@dosu

Since you're an efficient agent, please use your superior documentation and code crawling skills to apply what you said in the last part of your previous answer. Notify me of your conclusion with @Fade78. Thanks!

[dosubot]

To manually adjust the context length in Roo Code when the advanced configuration settings for the Ollama API provider are not available, you can follow these steps:

1. Load the Model:
   Run the model using the command:

   ollama run qwen2.5-coder:32b

2. Change Context Size Parameter:
   Set the context size parameter to your desired value (e.g., 32768 tokens) using the command:

   /set parameter num_ctx 32768

3. Save the Model with a New Name:
   Save the configured model with a new name:

   /save your_model_name

4. Configure Roo Code:

   • Open the Roo Code sidebar.
   • Click the settings gear icon.
   • Select "ollama" as the API Provider.
   • Enter the Model name from the previous step (e.g., your_model_name).

By following these steps, you can manually adjust the context length without relying on the advanced configuration settings in Roo Code ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[Fade78]

@dosu

You missunderstood the problem. I know how to setup a context length for my model. What I fail to do is to make roocode taking into account this context length. Roocode uses always 128k length.

[dosubot]

Roo Code seems to default to a 128k context length, and it appears there might be a discrepancy between the documentation and the actual functionality available in the application. If the advanced settings to adjust the context length are not visible or functional, it might be a limitation or a bug in the current version of Roo Code.

Unfortunately, I couldn't find specific information in the codebase or documentation about overriding this default setting directly within Roo Code. It might be worth checking for any recent updates or patches that address this issue, or consider reporting it as a bug to ensure it gets addressed in future updates.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[Fade78]

Thanks, so now we wait for a dev to notice :-)

[greinacker]

Can confirm, having the same issue here (I'm using LM Studio, but that shouldn't be relevant). I have a 24KB context length in the model, but Roo thinks it's 128KB.

[mrubens]

Yeah we should fix this

[R-omk]

After reviewing the source code, I discovered that Ollama uses an OpenAI-compatible API:

https://github.com/RooVetGit/Roo-Code/blob/78b208346713b77d7fd1d05a33604e335686e21a/src/api/providers/ollama.ts#L14-L17

However, the standard does not imply specifying num_ctx:
ollama/ollama#5356 (comment)

Currently, I have to manually create a model to fit into VRAM:

ollama create <mymodelnmae> --file <tunedmodelfil>

So I see only one option: don't use OpenAI, but use the native Ollama API.

Alternative: create models on the fly using the base model, but this still requires using the native Ollama API.

I would also like to note that it would be beneficial to have the ability to specify any options supported by Ollama, for example:

• num_keep is a very useful option considering the limited context window