Pinned Loading

1. vllm-project/aibrix vllm-project/aibrix Public

   Cost-efficient and pluggable Infrastructure components for GenAI inference

   Go 4.2k 454

2. ai-dynamo/dynamo ai-dynamo/dynamo Public

   A Datacenter Scale Distributed Inference Serving Framework

   Rust 5k 586

3. vllm vllm Public

   Forked from vllm-project/vllm

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python

4. sgl-project/sglang sgl-project/sglang Public

   SGLang is a fast serving framework for large language models and vision language models.

   Python 17.8k 2.9k

5. tokasaurus tokasaurus Public

   Forked from ScalingIntelligence/tokasaurus

   Python

6. LMCache LMCache Public

   Forked from LMCache/LMCache

   Supercharge Your LLM with the Fastest KV Cache Layer

   Python