Pinned Loading

1. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 56.4k 9.7k

2. sgl-project/sglang sgl-project/sglang Public

   SGLang is a fast serving framework for large language models and vision language models.

   Python 17.3k 2.7k

3. deepseek-ai/DeepGEMM deepseek-ai/DeepGEMM Public

   DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling

   Cuda 5.6k 681

4. GeeeekExplorer/nano-vllm GeeeekExplorer/nano-vllm Public

   Nano vLLM

   Python 6k 734

5. LMCache/LMCache LMCache/LMCache Public

   Supercharge Your LLM with the Fastest KV Cache Layer

   Python 5k 542

6. llm-scratch-pytorch llm-scratch-pytorch Public

   lm-scratch-pytorch - The code is designed to be beginner-friendly, with a focus on understanding the fundamentals of PyTorch and implementing LLMs from scratch,step by step.

   Jupyter Notebook 101 4