refactor: create llm response #5499

francismiko · 2025-08-20T04:05:21Z

I abstracted the code in toolChoice, promptCall and chat into some common functions and put these reused logic in global/core/ai/request.ts

…f stream-based and complete responses

…oice logic

…in chat and toolChoice modules

…esponse handling

…ools into chat and tool choice logic

cla-assistant · 2025-08-20T04:05:38Z

All committers have signed the CLA.

cla-assistant · 2025-08-20T04:05:38Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

github-actions · 2025-08-20T04:07:46Z

Preview mcp_server Image:

registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-pr:fatsgpt_mcp_server_c659dad2f7b21401b751511c6c0b6c2ce2c481b9

github-actions · 2025-08-20T04:07:56Z

Preview sandbox Image:

registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-pr:fatsgpt_sandbox_c659dad2f7b21401b751511c6c0b6c2ce2c481b9

package.json

packages/global/core/ai/request.ts

packages/service/core/workflow/dispatch/ai/agent/promptCall.ts

packages/global/core/ai/request.ts

… token counts

…structure and removing redundant checks

… and streamlining event callbacks

github-actions · 2025-08-22T04:09:27Z

Preview fastgpt Image:

registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-pr:fatsgpt_c659dad2f7b21401b751511c6c0b6c2ce2c481b9

gru-agent · 2025-08-22T04:12:43Z

TestGru Assignment

Summary

Link	CommitId	Status	Reason
Detail	`5ab92ab`	🚫 Skipped	No files need to be tested {".husky/pre-commit":"File path does not match include patterns.","packages/global/core/ai/request.ts":"File path does not match include patterns.","packages/global/core/ai/type.d.ts":"File path does not match include patterns.","packages/service/core/workflow/dispatch/ai/agent/promptCall.ts":"File path does not match include patterns.","packages/service/core/workflow/dispatch/ai/agent/toolChoice.ts":"File path does not match include patterns.","packages/service/core/workflow/dispatch/ai/chat.ts":"File path does not match include patterns."}

History Assignment

Tip

You can @gru-agent and leave your feedback. TestGru will make adjustments based on your input

c121914yu · 2025-08-23T05:22:35Z

@cursor review

cursor · 2025-08-23T05:25:39Z

packages/global/core/ai/request.ts

+          }
+        }
+      });
+    }


Bug: Streamed Tool Calls Reset Across Chunks

The callingTool variable is declared inside the if (responseChoice?.tool_calls?.length) block, causing it to reset with each stream chunk. This prevents accumulating tool call data across chunks, leading to lost or incomplete tool calls when tool information spans multiple parts of the stream.

cursor · 2025-08-23T05:25:40Z

packages/service/core/workflow/dispatch/ai/agent/promptCall.ts

-  /* Run llm */
-  const {
-    response: aiResponse,
+  const write = res ? responseWriteController({ res, readStream: stream }) : undefined;


Bug: Incorrect Parameter Type Causes Streaming Failure

It looks like the responseWriteController is receiving a boolean stream flag for its readStream parameter. This parameter expects an actual stream object, so this might prevent streaming responses from working correctly.

Additional Locations (2)

packages/service/core/workflow/dispatch/ai/chat.ts#L201-L202

packages/service/core/workflow/dispatch/ai/agent/toolChoice.ts#L300-L301

* feat: add LLM response processing functions, including the creation of stream-based and complete responses * feat: add volta configuration for node and pnpm versions * refactor: update LLM response handling and event structure in tool choice logic * feat: update LLM response structure and integrate with tool choice logic * refactor: clean up imports and remove unused streamResponse function in chat and toolChoice modules * refactor: rename answer variable to answerBuffer for clarity in LLM response handling * feat: enhance LLM response handling with tool options and integrate tools into chat and tool choice logic * refactor: remove volta configuration from package.json * refactor: reorganize LLM response types and ensure default values for token counts * refactor: streamline LLM response handling by consolidating response structure and removing redundant checks * refactor: enhance LLM response handling by consolidating tool options and streamlining event callbacks * fix: build error * refactor: update tool type definitions for consistency in tool handling

* feat: favorite apps & quick apps with their own configuration (#5515) * chore: extract chat history and drawer; fix model selector * feat: display favourite apps and make it configurable * feat: favorite apps & quick apps with their own configuration * fix: fix tab title and add loading state for searching * fix: cascade delete favorite app and quick app while deleting relative app * chore: make improvements * fix: favourite apps ui * fix: add permission for quick apps * chore: fix permission & clear redundant code * perf: chat home page code * chatbox ui * fix: 4.12.2-dev (#5520) * fix: add empty placeholder; fix app quick status; fix tag and layout * chore: add tab query for the setting tabs * chore: use `useConfirm` hook instead of `MyModal` * remove log * fix: fix modal padding (#5521) * perf: manage app * feat: enhance model provider handling and update icon references (#5493) * perf: model provider * sdk package * refactor: create llm response (#5499) * feat: add LLM response processing functions, including the creation of stream-based and complete responses * feat: add volta configuration for node and pnpm versions * refactor: update LLM response handling and event structure in tool choice logic * feat: update LLM response structure and integrate with tool choice logic * refactor: clean up imports and remove unused streamResponse function in chat and toolChoice modules * refactor: rename answer variable to answerBuffer for clarity in LLM response handling * feat: enhance LLM response handling with tool options and integrate tools into chat and tool choice logic * refactor: remove volta configuration from package.json * refactor: reorganize LLM response types and ensure default values for token counts * refactor: streamline LLM response handling by consolidating response structure and removing redundant checks * refactor: enhance LLM response handling by consolidating tool options and streamlining event callbacks * fix: build error * refactor: update tool type definitions for consistency in tool handling * feat: llm request function * fix: ts * fix: ts * fix: ahook ts * fix: variable name * update lock * ts version * doc * remove log * fix: translation type * perf: workflow status check * fix: ts * fix: prompt tool call * fix: fix missing plugin interact window & make tag draggable (#5527) * fix: incorrect select quick apps state; filter apps type (#5528) * fix: usesafe translation * perf: add quickapp modal --------- Co-authored-by: 伍闲犬 <[email protected]> Co-authored-by: Ctrlz <[email protected]> Co-authored-by: francis <[email protected]>

francismiko added 7 commits August 15, 2025 15:21

feat: add LLM response processing functions, including the creation o…

67d7d19

…f stream-based and complete responses

feat: add volta configuration for node and pnpm versions

dafebbf

refactor: update LLM response handling and event structure in tool ch…

89d1717

…oice logic

feat: update LLM response structure and integrate with tool choice logic

41b6b58

refactor: clean up imports and remove unused streamResponse function …

ffa1cbf

…in chat and toolChoice modules

refactor: rename answer variable to answerBuffer for clarity in LLM r…

afd0fd0

…esponse handling

feat: enhance LLM response handling with tool options and integrate t…

32b41a7

…ools into chat and tool choice logic

pull-request-size bot added the size/XXL label Aug 20, 2025

c121914yu reviewed Aug 21, 2025

View reviewed changes

packages/global/core/ai/request.ts Outdated Show resolved Hide resolved

francismiko added 5 commits August 21, 2025 14:20

refactor: remove volta configuration from package.json

1bb6f04

refactor: reorganize LLM response types and ensure default values for…

f404809

… token counts

refactor: streamline LLM response handling by consolidating response …

5477658

…structure and removing redundant checks

refactor: enhance LLM response handling by consolidating tool options…

ebe54d6

… and streamlining event callbacks

fix: build error

5ab92ab

francismiko marked this pull request as ready for review August 22, 2025 04:12

refactor: update tool type definitions for consistency in tool handling

c659dad

cursor bot reviewed Aug 23, 2025

View reviewed changes

c121914yu changed the base branch from main to v4.12.2-dev August 23, 2025 05:27

c121914yu merged commit 8b47446 into labring:v4.12.2-dev Aug 23, 2025
5 checks passed

francismiko deleted the francis/invoke-llm branch August 26, 2025 12:01

refactor: create llm response #5499

refactor: create llm response #5499

Uh oh!

Conversation

francismiko commented Aug 20, 2025

Uh oh!

cla-assistant bot commented Aug 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cla-assistant bot commented Aug 20, 2025

Uh oh!

github-actions bot commented Aug 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Preview mcp_server Image:

Uh oh!

github-actions bot commented Aug 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Preview sandbox Image:

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Preview fastgpt Image:

Uh oh!

gru-agent bot commented Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TestGru Assignment

Summary

Uh oh!

c121914yu commented Aug 23, 2025

Uh oh!

cursor bot Aug 23, 2025

Choose a reason for hiding this comment

Bug: Streamed Tool Calls Reset Across Chunks

Uh oh!

cursor bot Aug 23, 2025

Choose a reason for hiding this comment

Bug: Incorrect Parameter Type Causes Streaming Failure

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

cla-assistant bot commented Aug 20, 2025 •

edited

Loading

github-actions bot commented Aug 20, 2025 •

edited

Loading

github-actions bot commented Aug 20, 2025 •

edited

Loading

github-actions bot commented Aug 22, 2025 •

edited

Loading

gru-agent bot commented Aug 22, 2025 •

edited

Loading