Suppress FunctionInvoking span when no tools are defined

[verdie-g]

We use a common chat builder for all our applications that creates a FunctionInvokingChatClient. Many of these applications don't use tools but still get a FunctionInvokingChatClient.GetResponseAsync.

extensions/src/Libraries/Microsoft.Extensions.AI/ChatCompletion/FunctionInvokingChatClient.cs

Lines 233 to 235 in da94b60

	// A single request into this GetResponseAsync may result in multiple requests to the inner client.
	// Create an activity to group them together for better observability.
	using Activity? activity = _activitySource?.StartActivity($"{nameof(FunctionInvokingChatClient)}.{nameof(GetResponseAsync)}");

Would it make sense to suppress that span if options.Tools and AdditionalTools are empty?

[stephentoub]

I'm not sure if that's desirable. If no tool calls are requested, you'd get a span if tools were available but not if tools weren't? That seems arbitrary to me.

[verdie-g]

For my users, it looks a little confusing "Why is it mentioning functions when I'm not even using them". Routing the requests to a client without this middleware when no tools are defined sounds a little overkill.

[stephentoub]

What if the service still requests a tool call? The FunctionInvokingChatClient will turn around and reissue the request with a response saying that tool doesn't exist. We would still want those encapsulated in an overall span.

[verdie-g]

I'm not sure I understand that case but anyway I could always do a RoutingFunctionInvokingChatClient that bypasses FunctionInvokingChatClient if no tools are defined.

[stephentoub]

I'm not sure I understand that case

For example if you were talking to a service that was pre-told about certain functions existing, like you can do with OpenAI Assistants, or if the model hallucinates the existence of a function you didn't actually provide.

[verdie-g]

Aah I see. That's convincing thank you! I'll keep my span and an option for users to disable FunctionInvokingChatClient completely if they don't like that extra span.

[verdie-g]

Can I maybe suggest a more OTELy name for that span?

[stephentoub]

Can I maybe suggest a more OTELy name for that span?

Suggestions?

[verdie-g]

• gen_ai.function.handle
• gen_ai.function.resolve
• gen_ai.function.dispatch

Also, I don't think we should make the distinction between streaming and non-streaming in that span.