Skip to content

Conversation

@DouweM
Copy link
Collaborator

@DouweM DouweM commented Jul 24, 2025

Ollama + Qwen3 will emit <think>\n</think>\n\n ahead of tool calls, which we don't want to end up treating as a final result when using run_stream (as that would stop the run before even handling the tool calls).

(See example code below for context on this output)

Before, the TextPart(content='\n\n') is treated as the final result:

PartStartEvent(index=0, part=ThinkingPart(content='')),
PartDeltaEvent(index=0, delta=ThinkingPartDelta(content_delta='\n')),
PartStartEvent(index=1, part=TextPart(content='\n\n')),
FinalResultEvent(tool_name=None, tool_call_id=None),
PartStartEvent(index=2, part=ToolCallPart(tool_name='roll_dice', args='{}', tool_call_id='call_q2vb1z22')),    
...

After, only the actual text after the tool call is treated as the final result: (If it was a final_result call, that call would be treated as the final result)

PartStartEvent(index=0, part=ThinkingPart(content=''))
PartDeltaEvent(index=0, delta=ThinkingPartDelta(content_delta='\n'))
PartStartEvent(index=1, part=ToolCallPart(tool_name='roll_dice', args='{}', tool_call_id='call_frxbo7tq'))
...
PartStartEvent(index=1, part=TextPart(content='You'))
FinalResultEvent(tool_name=None, tool_call_id=None)
PartDeltaEvent(index=1, delta=TextPartDelta(content_delta=' rolled'))
PartDeltaEvent(index=1, delta=TextPartDelta(content_delta=' a'))
PartDeltaEvent(index=1, delta=TextPartDelta(content_delta=' '))
PartDeltaEvent(index=1, delta=TextPartDelta(content_delta='5'))
PartDeltaEvent(index=1, delta=TextPartDelta(content_delta='.'))

Code

import asyncio
import random

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIModel
from pydantic_ai.providers.openai import OpenAIProvider


def roll_dice() -> str:
    """Roll a six-sided die and return the result."""
    return str(random.randint(1, 6))


# --- Agent and Model Configuration ---


ollama_model = OpenAIModel(
    model_name='qwen3:1.7b',
    provider=OpenAIProvider(base_url='http://localhost:11434/v1'),
    settings={'temperature': 0.2},
)

default_system_prompt = """
You are a helpful assistant.
Do not ask any follow-up questions.
"""

model_settings = {
    'temperature': 0.2,
}

tools = [roll_dice]

agent = Agent(model=ollama_model, tools=tools, system_prompt=default_system_prompt, output_type=str)


async def main():
    prompt = 'roll the dice'

    async with agent.iter(prompt) as agent_run:
        async for node in agent_run:
            if Agent.is_model_request_node(node) or Agent.is_call_tools_node(node):
                async with node.stream(agent_run.ctx) as request_stream:
                    async for event in request_stream:
                        print(event, flush=True)


# ---

if __name__ == '__main__':
    asyncio.run(main())

@DouweM DouweM self-assigned this Jul 24, 2025
@github-actions
Copy link

github-actions bot commented Jul 24, 2025

Docs Preview

commit: d145396
Preview URL: https://91e4334f-pydantic-ai-previews.pydantic.workers.dev

@DouweM DouweM marked this pull request as ready for review July 24, 2025 19:20
Base automatically changed from streaming-think-tags to main July 24, 2025 19:23
@DouweM DouweM closed this Jul 24, 2025
@DouweM DouweM force-pushed the streaming-ignore-leading-newlines branch from 55d6c93 to 7eb4491 Compare July 24, 2025 19:24
DouweM added 2 commits July 24, 2025 19:25
# Conflicts:
#	pydantic_ai_slim/pydantic_ai/models/huggingface.py
@DouweM DouweM reopened this Jul 24, 2025
@DouweM DouweM enabled auto-merge (squash) July 24, 2025 20:58
@DouweM DouweM merged commit 41dd069 into main Jul 24, 2025
16 checks passed
@DouweM DouweM deleted the streaming-ignore-leading-newlines branch July 24, 2025 21:16
KRRT7 pushed a commit to aseembits93/pydantic-ai that referenced this pull request Jul 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants