Skip to content

TencentCloudADP/youtu-graphrag

Repository files navigation

Youtu-agent Logo Youtu-GraphRAG:
Vertically Unified Agents for Graph Retrieval-Augmented Complex Reasoning

License: MIT Paper WeChat Community Discord Community

πŸš€ Revolutionary framework moving Pareto Frontier with 33.6% lower token cost and 16.62% higher accuracy over SOTA baselines

πŸ”– δΈ­ζ–‡η‰ˆ β€’ ⭐ Contributions β€’ πŸ“Š Benchmarks β€’ πŸš€ Getting Started

🎯 Brief Introduction

Youtu-GraphRAG is a vertically unified agentic paradigm that jointly connects the entire framework as an intricate integration based on graph schema. We allow seamless domain transfer with minimal intervention on the graph schema, providing insights of the next evolutionary GraphRAG paradigm for real-world applications with remarkable adaptability.

Youtu-GrapHRAG Logo

🎨 When and Why to use Youtu-GraphRAG

πŸ”— Multi-hop Reasoning/Summarization/Conclusion: Complex questions requiring multi-step reasoning
πŸ“š Knowledge-Intensive Tasks: Questions dependent on large amounts of structured/private/domain knowledge
🌐 Domain Scalability: Easily support encyclopedias, academic papers, commercial/private knowledge base and other domains with minimal intervention on the schema

πŸ—οΈ Framework Architecture

Youtu-GraphRAG Framework Architecture
A sketched overview of our proposed framework Youtu-GraphRAG.

πŸ“² Interactive interface

Graph Construction Retrieval

πŸš€ Contributions and Novelty

Based on our unified agentic paradigm for Graph Retrieval-Augmented Generation (GraphRAG), Youtu-GraphRAG introduces several key innovations that jointly connect the entire framework as an intricate integration:

πŸ—οΈ 1. Schema-Guided Hierarchical Knowledge Tree Construction

  • 🌱 Seed Graph Schema: Introduces targeted entity types, relations, and attribute types to bound automatic extraction agents
  • πŸ“ˆ Scalable Schema Expansion: Continuously expands schemas for adaptability over unseen domains
  • 🏒 Four-Level Architecture:
    • Level 1 (Attributes): Entity property information
    • Level 2 (Relations): Entity relationship triples
    • Level 3 (Keywords): Keyword indexing
    • Level 4 (Communities): Hierarchical community structure
  • ⚑ Quick Adaptation to industrial applications: We allow seamless domain transfer with minimal intervention on the schema

🌳 2. Dually-Perceived Community Detection

  • πŸ”¬ Novel Community Detection Algorithm: Fuses structural topology with subgraph semantics for comprehensive knowledge organization
  • πŸ“Š Hierarchical Knowledge Tree: Naturally yields a structure supporting both top-down filtering and bottom-up reasoning that performs better than traditional Leiden and Louvain algorithms
  • πŸ“ Community Summaries: LLM-enhanced community summarization for higher-level knowledge abstraction
Youtu-GraphRAG Community Detection

πŸ€– 3. Agentic Retrieval

  • 🎯 Schema-Aware Decomposition: Interprets the same graph schema to transform complex queries into tractable and parallel sub-queries
  • πŸ”„ Iterative Reflection: Performs reflection for more advanced reasoning through IRCoT (Iterative Retrieval Chain of Thought)
Youtu-GraphRAG Agentic Decomposer

🧠 4. Advanced Construction and Reasoning Capabilities for real-world deployment

  • 🎯 Performance Enhancement: Less token costs and higher accuracy with optimized prompting, indexing and retrieval strategies
  • πŸ€Ήβ€β™€οΈ User friendly visualization: In output/graphs/, the four-level knowledge tree supports visualization with neo4j import,making reasoning paths and knowledge organization vividly visable to users
  • ⚑ Parallel Sub-question Processing: Concurrent handling of decomposed questions for efficiency and complex scenarios
  • πŸ€” Iterative Reasoning: Step-by-step answer construction with reasoning traces
  • πŸ“Š Domain Scalability: Designed for enterprise-scale deployment with minimal manual intervention for new domains

πŸ“ˆ 5. Fair Anonymous Dataset 'AnonyRAG'

  • Link: Hugging Face AnonyRAG
  • Against knowledeg leakage in LLM/embedding model pretraining
  • In-depth test on real retrieval performance of GraphRAG
  • Multi-lingual with Chinese and English versions

βš™οΈ 6. Unified Configuration Management

  • πŸŽ›οΈ Centralized Parameter Management: All components configured through a single YAML file
  • πŸ”§ Runtime Parameter Override: Dynamic configuration adjustment during execution
  • 🌍 Multi-Environment Support: Seamless domain transfer with minimal intervention on schema
  • πŸ”„ Backward Compatibility: Ensures existing code continues to function

πŸ“Š Performance Comparisons

Extensive experiments across six challenging benchmarks, including GraphRAG-Bench, HotpotQA and MuSiQue, demonstrate the robustness of Youtu-GraphRAG, remarkably moving the Pareto frontier with 33.6% lower token cost compared to the sota methods and 16.62% higher accuracy over state-of-the-art baselines. The results indicate our adaptability, allowing seamless domain transfer with minimal intervention on schema.

Cost/acc performance Moving Pareto Frontier radar comparison

πŸ“ Project Structure

youtu-graphrag/
β”œβ”€β”€ πŸ“ config/                     # Configuration System
β”‚   β”œβ”€β”€ base_config.yaml           # Main configuration file
β”‚   β”œβ”€β”€ config_loader.py           # Configuration loader
β”‚   └── __init__.py                # Configuration module interface
β”‚
β”œβ”€β”€ πŸ“ data/                       # Data Directory
β”‚
β”œβ”€β”€ πŸ“ models/                     # Core Models
β”‚   β”œβ”€β”€ πŸ“ constructor/            # Knowledge Graph Construction
β”‚   β”‚   └── kt_gen.py              # KTBuilder - Hierarchical graph builder
β”‚   β”œβ”€β”€ πŸ“ retriever/              # Retrieval Module
β”‚   β”‚   β”œβ”€β”€ enhanced_kt_retriever.py  # KTRetriever - Main retriever
β”‚   β”‚   β”œβ”€β”€ agentic_decomposer.py     # Query decomposer
β”‚   └── └── faiss_filter.py           # DualFAISSRetriever - FAISS retrieval
β”‚
β”œβ”€β”€ πŸ“ utils/                      # Utility Modules
β”‚   β”œβ”€β”€ tree_comm.py              # community detection algorithm
β”‚   β”œβ”€β”€ call_llm_api.py           # LLM API calling
β”‚   β”œβ”€β”€ eval.py                   # Evaluation tools
β”‚   └── graph_processor.py        # Graph processing tools
β”‚
β”œβ”€β”€ πŸ“ schemas/                   # Dataset Schemas
β”œβ”€β”€ πŸ“ assets/                    # Assets (images, figures)
β”‚
β”œβ”€β”€ πŸ“ output/                    # Output Directory
β”‚   β”œβ”€β”€ graphs/                   # Constructed knowledge graphs
β”‚   β”œβ”€β”€ chunks/                   # Text chunk information
β”‚   └── logs/                     # Runtime logs
β”‚
β”œβ”€β”€ πŸ“ retriever/                 # Retrieval Cache
β”‚
β”œβ”€β”€ main.py                       # 🎯 Main program entry
β”œβ”€β”€ requirements.txt              # Dependencies list
β”œβ”€β”€ setup_env.sh                  # install web dependency
β”œβ”€β”€ start.sh                      # start web service
└── README.md                     # Project documentation

πŸš€ Quick Start

We provide two approaches to run and experience the demo service. Considering the differences in the underlying environment, we recommend using Docker as the preferred deployment method.

πŸ’» Start with Dockerfile

This approach relies on the Docker environment, which could be installed according to official documentation.

# 1. Clone Youtu-GraphRAG project
git clone https://github.com/TencentCloudADP/youtu-graphrag

# 2. Create .env according to .env.example
cd youtu-graphrag && cp .env.example .env
# Config your LLM api in .env as OpenAI API format
# LLM_MODEL=deepseek-chat
# LLM_BASE_URL=https://api.deepseek.com
# LLM_API_KEY=sk-xxxxxx

# 3. Build with dockerfile 
docker build -t youtu_graphrag:v1 .

# 4. Docker run
docker run -d -p 8000:8000 youtu_graphrag:v1

# 5. Visit http://localhost:8000
curl -v http://localhost:8000

πŸ’» Web UI Experience

This approach relies on Python 3.10 and the corresponding pip environment, you can install it according to the official documentation.

# 1. Clone Youtu-GraphRAG project
git clone https://github.com/TencentCloudADP/youtu-graphrag

# 2. Create .env according to .env.example
cd youtu-graphrag && cp .env.example .env
# Config your LLM api in .env as OpenAI API format
# LLM_MODEL=deepseek-chat
# LLM_BASE_URL=https://api.deepseek.com
# LLM_API_KEY=sk-xxxxxx

# 3. Setup environment
./setup_env.sh

# 4. Launch the web
./start.sh

# 5. Visit http://localhost:8000
curl -v http://localhost:8000

πŸ“– Full Usage Guide

For advanced config and usageοΌšπŸš€ FullGuide

⭐ Start using Youtu-GraphRAG now and experience the intelligent question answering! πŸš€

🀝 Contributing

We welcome contributions from the community! Here's how you can help:

πŸ’» Code Contribution

  1. 🍴 Fork the project
  2. 🌿 Create a feature branch (git checkout -b feature/AmazingFeature)
  3. πŸ’Ύ Commit your changes (git commit -m 'Add some AmazingFeature')
  4. πŸ“€ Push to the branch (git push origin feature/AmazingFeature)
  5. πŸ”„ Create a Pull Request

πŸ”§ Extension Guide

  • 🌱 New Seed Schemas: Add high-quality seed schema and data processing
  • πŸ“Š Custom Datasets: Integrate new datasets with minimal schema intervention
  • 🎯 Domain-Specific Applications: Extend framework for specialized use cases with 'Best Practice'

πŸ“ž Contact

Hanson Dong - [email protected] Siyu An - [email protected]


πŸŽ‰ Citation

@misc{dong2025youtugraphrag,
      title={Youtu-GraphRAG: Vertically Unified Agents for Graph Retrieval-Augmented Complex Reasoning}, 
      author={Junnan Dong and Siyu An and Yifei Yu and Qian-Wen Zhang and Linhao Luo and Xiao Huang and Yunsheng Wu and Di Yin and Xing Sun},
      year={2025},
      eprint={2508.19855},
      archivePrefix={arXiv},
      url={https://arxiv.org/abs/2508.19855}, 
}

About

Official repository of Youtu-GraphRAG: Vertically Unified Agents for Graph Retrieval-Augmented Complex Reasoning

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages