Using Chunk Mode with CodeCompanion Tool

[Davidyz]

Since 0.6.12, the VectorCode tool for CodeCompanion supports chunk-only mode. This allows the tool to query for chunks, not full documents, from the database.

From my personal experience, this can offer improved retrieval accuracy, especially if many of your documents are chunked into pieces. Since the chunks are usually of a similar size (constrained by chunk_size), this'll also make the number of tokens in the chat more predictable: in the full-document mode, a crazy-long file can easily saturate the context window, but the chunk mode doesn't have this problem because the chunk sizes are capped.

I'd recommend that you give it a try and see if it makes things better for your coding experience. You can refer to the wiki and look for chunk_mode to see how to enable it. If your collection was created before 0.6.12, it's recommended (but not necessary) to re-vectorise the files (no need to drop; a vectorcode update will suffice) because 0.6.12 also contains a fix that improves the metadata so that the LLM can provide more accurate citations.

Credit: @guill for the initial version of the chunk-mode in the codecompanion tool.