Error on Pinecone .upsert()

[lerzegov]

I have been testing the DocumentQnAExamples.IfYouNeedMoreControl example, using this revised code with some debugging; the pdf file path is hardcoded

import dev.ai4j.document.Document;
import dev.ai4j.document.loader.PdfFileLoader;
import dev.ai4j.document.splitter.DocumentSplitter;
import dev.ai4j.document.splitter.OverlappingDocumentSplitter;
import dev.ai4j.flows.DocumentQnAFlow;
import dev.ai4j.model.completion.OpenAiCompletionModel;
import dev.ai4j.model.embedding.Embedding;
import dev.ai4j.model.embedding.OpenAiEmbeddingModel;
import dev.ai4j.model.embedding.PineconeDatabase;
import dev.ai4j.prompt.Prompt;
import java.util.Collection;
import java.util.List;
import static dev.ai4j.model.openai.OpenAiModelName.GPT_3_5_TURBO;
import static dev.ai4j.model.openai.OpenAiModelName.TEXT_EMBEDDING_ADA_002;
import static java.util.stream.Collectors.joining;

public class DocumentQnAExamples {
    static class IfYouNeedSimplicity {
        public static void main(String[] args) {
            DocumentQnAFlow flow = DocumentQnAFlow.builder()
                    .documentLoader(new PdfFileLoader("/Users/luca/Documents/ai_pdf/large-language-models.pdf"))
                    .embeddingModel(OpenAiEmbeddingModel.builder()
                            // .apiKey(System.getenv("OPENAI_API_KEY")) // https://platform.openai.com/account/api-keys
                            .apiKey(Tokens.openAiToken)
                            .build())
                    .vectorDatabase(PineconeDatabase.builder()
                            .apiKey(Tokens.PINECONE_API_KEY) // https://app.pinecone.io/organizations/xxx/projects/yyy:zzz/keys
                            .environment(Tokens.PINECONE_ENV)
                            .projectName(Tokens.PINECONE_PROJECT) // sta minkia nella url
                            .index("langchain-retrieval-augmentation") // make sure the dimensions of the Pinecone index match the dimensions of the embedding model (1536 for text-embedding-ada-002)
                            .build())
                    .completionModel(OpenAiCompletionModel.builder()
                            //.apiKey(System.getenv("OPENAI_API_KEY"))
                            .apiKey(Tokens.openAiToken)
                            .build())
                    .build();

            String answer = flow.ask("How many parameters does GPT-3 have?");

            System.out.println(answer);
        }
    }

    static class IfYouNeedMoreControl {
        public static void main(String[] args) {
            // Load file with information you want to "chat" with LLM about.
            // Currently, text and PDF files are supported.
            String absolutePathToPdfFile = "/Users/luca/Documents/ai_pdf/large-language-models.pdf";
            PdfFileLoader pdfFileLoader = new PdfFileLoader(absolutePathToPdfFile);
            Document fullPdfDocument = pdfFileLoader.load();
System.out.println("pdf loaded");
            // Split the file into small chunks of 200 characters each with an overlap of 40 characters
            DocumentSplitter splitter = new OverlappingDocumentSplitter(200, 40);
            List<Document> pdfDocumentChunks = splitter.split(fullPdfDocument);

System.out.println("pdf split");
            // Convert chunks into embeddings (semantic vectors)
            OpenAiEmbeddingModel openAiEmbeddings = OpenAiEmbeddingModel.builder()
                    //.apiKey(System.getenv("OPENAI_API_KEY")) // https://platform.openai.com/account/api-keys
                    .apiKey(Tokens.openAiToken)
                    .modelName(TEXT_EMBEDDING_ADA_002)
                    .build();
            Collection<Embedding> embeddings = openAiEmbeddings.embed(pdfDocumentChunks);

System.out.println("created embeddings");
            // Store embeddings into vector DB for further search / retrieval

            PineconeDatabase pineconeDatabase = PineconeDatabase.builder()
                    .apiKey(Tokens.PINECONE_API_KEY) // https://app.pinecone.io/organizations/xxx/projects/yyy:zzz/keys
                    .environment(Tokens.PINECONE_ENV)
                    .projectName(Tokens.PINECONE_PROJECT)
                    .index("langchain-retrieval-augmentation")  // make sure the dimensions of the Pinecone index match the dimensions of the embedding model (1536 for text-embedding-ada-002)
                    .build();
System.out.println("Pinecone build");
System.out.println(pineconeDatabase.connection.toString());
            try {
                pineconeDatabase.persist(embeddings); // qui ERROR: io.grpc.StatusRuntimeException: UNKNOWN
            } catch (Exception e) {
                e.printStackTrace(System.out);
                return;
            }
System.out.println("Pinecone persist");

            // Define the question you want to ask LLM
            String question = "How many parameters does GPT-3 have?";
            // Find relevant embeddings in vector DB by semantic similarity
            Embedding embeddingForQuestion = openAiEmbeddings.embed(question);
System.out.println("Question embedded");
            Collection<Embedding> relatedEmbeddings = pineconeDatabase.findRelated(embeddingForQuestion);
System.out.println("Pinecone findRelated");

            // Create a prompt for LLM that includes original question and relevant embeddings
            Prompt prompt = Prompt.from("Using only the information enclosed in triple angle brackets, answer this question: {{question}} <<<{{embeddings}}>>>")
                    .with("question", question)
                    .with("embeddings", relatedEmbeddings.stream().map(Embedding::getContent).collect(joining("\n\n")))
                    .build();
System.out.println("Prompt created");

            // Send prompt to LLM

            OpenAiCompletionModel openAiLanguageModel = OpenAiCompletionModel.builder()
                    //.apiKey(System.getenv("OPENAI_API_KEY")) // https://platform.openai.com/account/api-keys
                    .apiKey(Tokens.openAiToken)
                    .modelName(GPT_3_5_TURBO)
                    .build();
System.out.println("Completion built");
            String answer = openAiLanguageModel.complete(prompt);
System.out.println("Answer returned built");

            // See an answer from LLM

            System.out.println(answer);
        }
    }
    public class Tokens {
	    // hard coded perché il metodo con dotenv non lo vede
	    static public String openAiToken = "sk-XXXX";
	    static public String PINECONE_API_KEY = "2e8b6b11-992a-41cc-83a9-23e2d6e57c08";
	    static public String PINECONE_ENV = "us-west4-gcp-free";
	    static public String PINECONE_PROJECT="82bbcd4";
	}
}

I get the following log

pdf loaded
pdf split
created embeddings
mag 23, 2023 9:16:55 AM io.grpc.internal.ManagedChannelOrphanWrapper$ManagedChannelReference cleanQueue
GRAVE: ~* Previous channel ManagedChannelImpl{logId=14, target=langchain-retrieval-augmentation-82bbcd4.svc.us-west4-gcp-free.pinecone.io} was not shutdown properly!!! *~
Make sure to call shutdown()/shutdownNow() and wait until awaitTermination() returns true.
java.lang.RuntimeException: ManagedChannel allocation site
at io.grpc.internal.ManagedChannelOrphanWrapper$ManagedChannelReference.(ManagedChannelOrphanWrapper.java:100)
at io.grpc.internal.ManagedChannelOrphanWrapper.(ManagedChannelOrphanWrapper.java:60)
at io.grpc.internal.ManagedChannelOrphanWrapper.(ManagedChannelOrphanWrapper.java:51)
at io.grpc.internal.ManagedChannelImplBuilder.build(ManagedChannelImplBuilder.java:631)
at io.grpc.internal.AbstractManagedChannelImplBuilder.build(AbstractManagedChannelImplBuilder.java:297)
at io.pinecone.PineconeConnection.buildChannel(PineconeConnection.java:99)
at io.pinecone.PineconeConnection.(PineconeConnection.java:44)
at io.pinecone.PineconeClient.connect(PineconeClient.java:37)
at dev.ai4j.model.embedding.PineconeDatabase.(PineconeDatabase.java:47)
at dev.ai4j.model.embedding.PineconeDatabase$PineconeDatabaseBuilder.build(PineconeDatabase.java:34)
at dev.ai4j.model.embedding.PineconeDatabase$PineconeDatabaseBuilder$build$3.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:130)
at DocumentQnAExamples$IfYouNeedMoreControl.main(script:100)
at DocumentQnAExamples$IfYouNeedMoreControl$main.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:139)
at script.perform(script:29)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.quantrix.scripting.core.internal.ScriptedAction$4.perform(ScriptedAction.java:213)
at com.subx.general.core.internal.context.ThreadContext.performWithContext(ThreadContext.java:77)
at com.quantrix.scripting.core.internal.ScriptedAction.invokeScriptMethod(ScriptedAction.java:221)
at com.quantrix.scripting.core.internal.ScriptedAction.access$14(ScriptedAction.java:196)
at com.quantrix.scripting.core.internal.ScriptedAction$5.performWithException(ScriptedAction.java:279)
at com.quantrix.scripting.core.internal.ScriptedAction$5.performWithException(ScriptedAction.java:1)
at com.subx.general.core.api.operation.ExceptionThrowingOperation.perform(ExceptionThrowingOperation.java:47)
at com.subx.general.core.iapi.util.ThreadSpecificOutput.perform(ThreadSpecificOutput.java:43)
at com.quantrix.scripting.core.internal.ScriptManager.captureOutput(ScriptManager.java:685)
at com.quantrix.scripting.core.internal.ScriptedAction.lambda$2(ScriptedAction.java:290)
at com.quantrix.scripting.core.iapi.ScriptingProgressRunner.lambda$0(ScriptingProgressRunner.java:115)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)

Pinecone build
io.pinecone.PineconeConnection@5c4bf932
io.grpc.StatusRuntimeException: UNKNOWN
at io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:271)
at io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:252)
at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:165)
at io.pinecone.proto.VectorServiceGrpc$VectorServiceBlockingStub.upsert(VectorServiceGrpc.java:505)
at dev.ai4j.model.embedding.PineconeDatabase.persist(PineconeDatabase.java:81)
at dev.ai4j.model.embedding.VectorDatabase$persist.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:139)
at DocumentQnAExamples$IfYouNeedMoreControl.main(script:109)
at DocumentQnAExamples$IfYouNeedMoreControl$main.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:139)
at script.perform(script:29)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.quantrix.scripting.core.internal.ScriptedAction$4.perform(ScriptedAction.java:213)
at com.subx.general.core.internal.context.ThreadContext.performWithContext(ThreadContext.java:77)
at com.quantrix.scripting.core.internal.ScriptedAction.invokeScriptMethod(ScriptedAction.java:221)
at com.quantrix.scripting.core.internal.ScriptedAction.access$14(ScriptedAction.java:196)
at com.quantrix.scripting.core.internal.ScriptedAction$5.performWithException(ScriptedAction.java:279)
at com.quantrix.scripting.core.internal.ScriptedAction$5.performWithException(ScriptedAction.java:1)
at com.subx.general.core.api.operation.ExceptionThrowingOperation.perform(ExceptionThrowingOperation.java:47)
at com.subx.general.core.iapi.util.ThreadSpecificOutput.perform(ThreadSpecificOutput.java:43)
at com.quantrix.scripting.core.internal.ScriptManager.captureOutput(ScriptManager.java:685)
at com.quantrix.scripting.core.internal.ScriptedAction.lambda$2(ScriptedAction.java:290)
at com.quantrix.scripting.core.iapi.ScriptingProgressRunner.lambda$0(ScriptingProgressRunner.java:115)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.nio.channels.UnsupportedAddressTypeException
at sun.nio.ch.Net.checkAddress(Net.java:101)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:621)
at io.netty.util.internal.SocketUtils$3.run(SocketUtils.java:91)
at io.netty.util.internal.SocketUtils$3.run(SocketUtils.java:88)
at java.security.AccessController.doPrivileged(Native Method)
at io.netty.util.internal.SocketUtils.connect(SocketUtils.java:88)
at io.netty.channel.socket.nio.NioSocketChannel.doConnect(NioSocketChannel.java:322)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.connect(AbstractNioChannel.java:248)
at io.netty.channel.DefaultChannelPipeline$HeadContext.connect(DefaultChannelPipeline.java:1342)
at io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:548)
at io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:533)
at io.netty.channel.ChannelDuplexHandler.connect(ChannelDuplexHandler.java:54)
at io.grpc.netty.WriteBufferingAndExceptionHandler.connect(WriteBufferingAndExceptionHandler.java:157)
at io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:548)
at io.netty.channel.AbstractChannelHandlerContext.access$1000(AbstractChannelHandlerContext.java:61)
at io.netty.channel.AbstractChannelHandlerContext$9.run(AbstractChannelHandlerContext.java:538)
at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:503)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 more

Is there something wrong in my configuration or am I missing something else?
Thanks

[ai-for-java]

@lerzegov thank you for reporting this issue, I will check it in more details in the evening.
For now, I would play with pinecone configurations and see if something is wrong there.
How many embeddings are you trying to persist?
You can also try persisting to pinecone directly via their SDK: https://github.com/pinecone-io/pinecone-java-client and see if the same error occurs.
Will get back with more details a bit later.

[lerzegov]

I replicated the very same example using your single page doc large-language-models.pdf

[ai-for-java]

@lerzegov are you getting the same issue with intact example? I suspect there is something wrong with pinecone configuration. Take a look at mine, hope this will help!

[lerzegov]

Thanks for the feedback. My configuration corresponds to yours except for Pod Type (I use a free account). I will do additional tests tomorrow

[ai-for-java]

Hi @lerzegov, did you manage to make it work?

[lerzegov]

Hi, thanks for asking. The issue arises from the non standard environment I use, a groovy scripting engine embedded in a Java spreadsheet application (Quantrix Modeler). Accesso to pinecone via pinecone-java-client works, but via ai4j it doesn't. I don't have the IT skills to sort out the issue, but I guess that it has to do with packaged jars calling each other in network communication.
Other ai4j examples work in Groovy, apart from the StructuredOutputExamples.java where I get an error on classes created automatically in Groovy at runtime:
Field $staticClassInfo is not annotated with @description
I will try other solutions, or change my setup, moving AI functionality to a server accessed by Quantrix via an API.
I will follow ai4j progress and maybe try it again in a future time.