Skip to content

Conversation

@ekzhu
Copy link
Contributor

@ekzhu ekzhu commented Oct 28, 2025

Update GAIA module:

  1. Fix bug due to recent huggingface-hub package breaking changes, now using parquet file loader instead of JSON loader.
  2. Refactored sample to include OpenAI agent and add more customization.
  • The code builds clean without any errors or warnings
  • The PR follows the Contribution Guidelines
  • All unit tests pass, and I have added new tests where possible
  • Is this a breaking change? If yes, add "[BREAKING]" prefix to the title of the PR.

Copilot AI review requested due to automatic review settings October 28, 2025 23:46
@markwallace-microsoft markwallace-microsoft added documentation Improvements or additions to documentation python lab Agent Framework Lab labels Oct 28, 2025
@github-actions github-actions bot changed the title Lab: Updates to GAIA module Python: Lab: Updates to GAIA module Oct 28, 2025
@markwallace-microsoft
Copy link
Member

markwallace-microsoft commented Oct 28, 2025

Python Test Coverage

Python Test Coverage Report •
FileStmtsMissCoverMissing
TOTAL11947184884% 
report-only-changed-files is enabled. No files were changed during this commit :)

Python Unit Test Overview

Tests Skipped Failures Errors Time
1440 98 💤 0 ❌ 0 🔥 29.025s ⏱️

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR enhances the GAIA benchmark sample implementation by adding support for multiple agent providers (Azure AI and OpenAI), improved telemetry configuration, and support for the new Parquet-based GAIA dataset format.

Key Changes:

  • Added modular agent factories for Azure AI and OpenAI providers
  • Enhanced telemetry with configurable OTLP endpoint and file export options
  • Added support for Parquet format dataset files with fallback to JSONL
  • Improved command-line interface with argparse for configurable benchmark runs

Reviewed Changes

Copilot reviewed 6 out of 7 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
python/packages/lab/pyproject.toml Added pyarrow dependency for Parquet support and additional linting rule exclusions
python/packages/lab/gaia/samples/openai_agent.py New factory module for creating OpenAI-based agents with comprehensive documentation
python/packages/lab/gaia/samples/gaia_sample.py Refactored to support multiple providers, CLI args, and improved telemetry configuration
python/packages/lab/gaia/samples/azure_ai_agent.py New factory module for creating Azure AI agents with clear environment variable documentation
python/packages/lab/gaia/agent_framework_lab_gaia/gaia.py Enhanced to support Parquet format and conditional telemetry setup
python/packages/lab/gaia/README.md Removed outdated execution instructions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation lab Agent Framework Lab python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

.NET: Python - Lab Tests execute for any .NET update.

3 participants