Add Knowledgebase #1795

birdayz · 2025-06-04T10:05:27Z

Add KB proto
Add KB UI

github-actions · 2025-06-04T10:05:48Z

The latest Buf updates on your PR. Results from workflow Buf CI / lint-and-breaking (pull_request).

Build	Format	Lint	Breaking	Updated (UTC)
`✅ passed`	`✅ passed`	`✅ passed`	`✅ passed`	Jun 6, 2025, 7:51 AM

weeco · 2025-06-04T10:12:32Z

proto/redpanda/api/dataplane/v1/knowledge_base.proto

+  // KnowledgeBase ID.
+  string id = 1 [
+    (google.api.field_behavior) = REQUIRED,
+    (buf.validate.field).required = true,
+    (buf.validate.field).string.pattern = "^[A-Za-z0-9-_/]+$"
+  ];


We should autogenerate the id and not have it user-specified I think.

Also this id will likely go into a Kafka topic name (e.g. __redpanda_cloud__rag_<kb-id>_source_confluence-with-id) so we need to limit the character set. Slashes wouldn't be allowed.

oh yeah, absolutely. my mistake. should be xid

this is the resource. so it's required, because it's never an input

malinskibeniamin · 2025-06-04T10:30:30Z

proto/redpanda/api/dataplane/v1/knowledge_base.proto

+    }
+
+    oneof vector_database {
+      Postgres postgres = 1;


Should we differentiate as db type, since there are many flavours of postgres based DBs?

what different fields would we support? in the end, it's a conn via PG protocol. and, i guess, it needs pgvector, we could also call it PGVector

+1 call it PGVector - we shouldn't be specific and will want to support different extensions I'm sure

tbh, i did not go for PGVector. In the end, this is just postgres: dsn, table, ...
not suuper important for me though

rockwotj · 2025-06-04T19:13:37Z

proto/redpanda/api/dataplane/v1/knowledge_base.proto

+    }
+
+    oneof vector_database {
+      Postgres postgres = 1;


+1 call it PGVector - we shouldn't be specific and will want to support different extensions I'm sure

proto/redpanda/api/dataplane/v1/knowledge_base.proto

rockwotj · 2025-06-04T19:24:05Z

proto/redpanda/api/dataplane/v1/knowledge_base.proto

+  EmbeddingGenerator embedding_generator = 6 [
+    (google.api.field_behavior) = REQUIRED,
+    (buf.validate.field).required = true
+  ];
+
+  // API url for retrieval API, that allows lookup of documents.
+  string retrieval_api_url = 4 [(google.api.field_behavior) = OUTPUT_ONLY];


funky field number ordering?

yeah i will get it ordered before merging, still iterating a bit

github-actions · 2025-06-26T16:21:39Z

The latest Buf updates on your PR. Results from workflow Buf CI / validate (pull_request).

Build	Format	Lint	Breaking	Updated (UTC)
`✅ passed`	`✅ passed`	`✅ passed`	`✅ passed`	Jul 17, 2025, 3:44 PM

malinskibeniamin · 2025-07-15T15:37:50Z

frontend/src/utils/protobuf-reflection.ts

Very interesting piece of code, could be a beginning of a way to integrate it with auto form component to use grpc for validating the form directly without any intermediate measures.

malinskibeniamin · 2025-07-15T15:39:41Z

frontend/src/config.ts

@@ -188,6 +194,8 @@ const setConfig = ({ fetch, urlOverride, jwt, isServerless, featureFlags, ...arg
    transformsClient: transformClient,
    rpcnSecretsClient: secretGrpcClient,
    clusterStatusClient: clusterStatusGrpcClient,
+    knowledgebaseClient: knowledgebaseGrpcClient,
+    userClient: userGrpcClient,


What do we use userClient for?

i added it for the "auto create users", but had to stop implementing it, it was a bit too tricky.
i can remove the client - i did keep it because it's reasonable to have this client available

malinskibeniamin · 2025-07-15T15:41:22Z

frontend/src/components/pages/knowledgebase/KnowledgeBase.List.tsx

+};
+
+@observer
+class KnowledgeBaseList extends PageComponent<{}> {


This is an outdated pattern as well.

We should use functional React components (not class-based), and try to do something similar to frontend/src/components/pages/agents/create/create-agent-page.tsx where you do not rely on any MobX state at all.

done, could you check the result is what you expected?

birdayz · 2025-07-16T13:35:24Z

migrated to v1alpha3

weeco · 2025-07-16T14:03:40Z

buf.yaml

@@ -27,3 +27,4 @@ breaking:
  ignore:
    - redpanda/api/console/v1alpha1
    - redpanda/api/dataplane/v1alpha1
+    - proto/redpanda/api/dataplane/v1alpha1


I guess this can be removed now?

birdayz requested review from rockwotj, malinskibeniamin and weeco June 4, 2025 10:05

weeco reviewed Jun 4, 2025

View reviewed changes

weeco self-requested a review June 4, 2025 10:24

malinskibeniamin reviewed Jun 4, 2025

View reviewed changes

rockwotj reviewed Jun 4, 2025

View reviewed changes

birdayz force-pushed the jb/knowledgebase branch 8 times, most recently from c34d0b2 to caedc8c Compare June 6, 2025 07:51

birdayz force-pushed the jb/knowledgebase branch 6 times, most recently from 8b0116d to 96c0ea1 Compare June 27, 2025 09:39

birdayz marked this pull request as draft June 27, 2025 09:40

birdayz force-pushed the jb/knowledgebase branch from 96c0ea1 to 9553bf1 Compare June 27, 2025 12:17

birdayz force-pushed the jb/knowledgebase branch 2 times, most recently from 8f6c7b3 to cbcdcfd Compare July 7, 2025 17:41

birdayz marked this pull request as ready for review July 7, 2025 17:42

birdayz force-pushed the jb/knowledgebase branch 2 times, most recently from 247c9a1 to f58a32d Compare July 14, 2025 08:51

malinskibeniamin reviewed Jul 15, 2025

View reviewed changes

remove debug printlns

e2b00b3

birdayz force-pushed the jb/knowledgebase branch from 84ef6f2 to e2b00b3 Compare July 15, 2025 20:16

birdayz added 4 commits July 15, 2025 22:29

kb: use functional react components

2e338ad

code review comments

c0cd73b

kb: reusable hooks

6327790

add kb to v1alpha3

1cbacde

birdayz force-pushed the jb/knowledgebase branch from 1e53e73 to 1cbacde Compare July 16, 2025 12:59

weeco reviewed Jul 16, 2025

View reviewed changes

birdayz force-pushed the jb/knowledgebase branch 3 times, most recently from c36056a to d4e982e Compare July 17, 2025 08:50

Fix CI issues: use v1 openapi generator and fix frontend imports

4766383

birdayz force-pushed the jb/knowledgebase branch from d4e982e to 4766383 Compare July 17, 2025 09:39

use correct checkbox

d454f96

birdayz force-pushed the jb/knowledgebase branch from 191aa88 to d454f96 Compare July 17, 2025 10:52

make SASL configurable

22c8b41

birdayz force-pushed the jb/knowledgebase branch 3 times, most recently from 36b2e82 to 91c9283 Compare July 17, 2025 12:29

list users for dropdown

d10bef4

birdayz force-pushed the jb/knowledgebase branch 3 times, most recently from 62e08a0 to e5ae477 Compare July 17, 2025 13:10

small visual improvements

efbfcf0

birdayz force-pushed the jb/knowledgebase branch from e5ae477 to efbfcf0 Compare July 17, 2025 13:47

more ui improvements

d6e6cf4

Add Knowledgebase #1795

Are you sure you want to change the base?

Add Knowledgebase #1795

Uh oh!

Conversation

birdayz commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

birdayz Jul 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jun 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

birdayz commented Jul 16, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

birdayz commented Jun 4, 2025 •

edited

Loading

github-actions bot commented Jun 4, 2025 •

edited

Loading

birdayz Jul 16, 2025 •

edited

Loading

github-actions bot commented Jun 26, 2025 •

edited

Loading