partners/qdrant: DRAFT QdrandVectorStore.aadd_texts POC becnmark

[mkhludnev]

Here I want to boost QdrantVectorStore.aadd_texts performance with truly async implementation

It's just a test in a form of a benchmark. It also has LocalAIEmbeddings implementation allowing bulk request #22666.

• Lint and test:

needs openai>=1.3 to run it #22399.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment

Name	Status	Preview	Comments	Updated (UTC)
langchain	⬜️ Ignored (Inspect)	Visit Preview		Sep 24, 2024 7:21pm

[mkhludnev]

Heya!!
I repeat the same exercise with remote Qdrant & LocalAI. And got significant gain (honestly I deliberately sized test data to give advantage for async code spawning parallel request in the beginning):

The interesting thing: the old Qdrant.aadd_texts using async generator is slower than truly async code with create_tasks().

[mkhludnev]

@Anush008 wdyt?

[Anush008]

This also includes time for generating embeddings as well. Correct?

[mkhludnev]

This also includes time for generating embeddings as well. Correct?

Right.

[mkhludnev]

@Anush008 using create_task() gives more parallelism, so it let to put higher load reducing wall clock time. However, I'm not sure if it's a purpose of async VectorStore.a* method.
It's not clear eg in scope of #11141. @baskaryan could you comment what's the purpose of VectorStore async methods?

[efriis]

hey team!

while this is in draft, could you open it against your own fork, and reopen the PR against the main project when it's ready for our team's review?

[mkhludnev]

@efriis thanks for your reaction.
Could you please share your vision: is it worth to contribute QdrantVectorStore.aadd_texts() processing batches in parallel reducing wall clock time with higher cpu utilization?
cc @Anush008

[Anush008]

If we remove the embeddings generation, is the difference in performance worth the code duplication?

[mkhludnev]

If we remove the embeddings generation, is the difference in performance worth the code duplication?

That's actually what I'm asking. Embeddings is one of the pipeline stages, even if we avoid it, aadd_texts can send a few batches in parallel. Whether it's worth it, I don't know. Here's example of parallel execution from another project (if you ever heard of ;) UPD although turns out that code doesn't proceed batches concurrently.