[BUG] Continue uses large amount of persisted storage

[dancard32]

Before submitting your bug report

• I believe this is a bug. I'll try to join the Continue Discord for questions
• I'm not able to find an open issue that reports the same bug
• I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS: macOS
- Continue version: 0.8.68
- IDE version: VSCode
- Model: Llama 3.1

Description

I was notified that I was running low on storage from my OS. I did a search at my root directory and to my surprise the .continue directory contains nearly 30G of space on my drive. I believe that this is a bug since this is a decent amount of storage being used and it is not listed anywhere in the docs other than the install size of roughly 300Mb. The biggest culprits of the size inflation are (for me) the following:

• $HOME/.continue/index/index.sqlite- 21G
• $HOME/.continue/index/lancedb/* - 7.9G

To reproduce

These should be found in the .continue folder found at the home directory

Log output

N/A

[sestinj]

Out of curiosity, how large is the codebase that you work in, and for how long have you used Continue? Deleting these folders would of course temporarily help solve the problem, but I'm interested in figuring out whether we might be duplicating information

[dancard32]

@sestinj so the code base I've been using Continue with is on the smaller side. Simple React front-end with maybe 50-100 files and totaling 30k lines of logic, and a Python Flask api/backend that are both single file main.py with a total of 1k lines each.

Overall a fairly lite code base, I'm inclined to think it may have something to do with how Continue indexes node_modules

[razorree]

my .continue folder takes 20GB
i opened some bigger C projects (QMK_FIRMWARE) for example, i don't work personally on big projects in VSCODE.
if i look deeper into catalogs
index/lancedb/<qmk_firmware-prject> takes 6.5GB

is it possible to exclude some projects? or maybe opt-out by default? and ask for indexing

[karmin-secureage]

hi there, i'm facing this issue on my windows machine as well. for context, we use continue on a development server and use low power clients and vscode to access the server. i've tried reinstalling to no avail. i'm honestly also not sure what's happening? but it's taking up about 51gb of space

PS C:\Users\[user]> ((gci -force C:\Users\[user]\.continue -Recurse -ErrorAction SilentlyContinue| measure Length -s).sum / 1Gb)
51.4161714185029

specifically, there are a huge number of versions under C:\Users\[user]\.continue\index\lancedb\vscode-remotessg-remote....[long string of stuff]\_versions\. these files take up ~36.1GB

the other large file is C:\Users\[user]\.continue\index\lancedb\index.sqlite at about 13GB

i apologize if i'm not providing enough information; please let me know if more info is needed. also, would appreciate any updates or ideas as to how to workaround this issue or fix it!

thanks :)

[bd82]

My issue is related to this topic, so I'm replying instead of creating a new issue.

Context

We are using continue in a container based environment with very limited disk space (~10GB).

Questions

1. Is there anyway to limit the maximum disk space assigned to continue persisted storage?

2. Is there a garbage collection process for contents in the database? - for example: Data related to files which no longer exist.

   • If not does this mean the disk usage of continue is a monotonously increasing function?
   • For example if I checkout an old branch for a quick patch in a large repo, would continue keep the new data related to this old branch for years, even if I never checkout this branch again?

[github-actions]

This issue hasn't been updated in 90 days and will be closed after an additional 10 days without activity. If it's still important, please leave a comment and share any new information that would help us address the issue.

[github-actions]

This issue was closed because it wasn't updated for 10 days after being marked stale. If it's still important, please reopen + comment and we'll gladly take another look!