vllm

vllm

49618
7972

A high-throughput and memory-efficient inference and serving engine for LLMs

Community

GitHub

Languages

Python
85.4%
Cuda
9.4%
C++
3.7%
Shell
0.7%
C
0.5%
CMake
0.3%

README.md

Good First Issues

Ideal for first-time contributors

All Issues

Browse every open issue in this repository