Name		Name	Last commit message	Last commit date
parent directory ..
custom_kernels		custom_kernels
exllama_kernels		exllama_kernels
exllamav2_kernels		exllamav2_kernels
tests		tests
text_generation_server		text_generation_server
.gitignore		.gitignore
Makefile		Makefile
Makefile-awq		Makefile-awq
Makefile-eetq		Makefile-eetq
Makefile-exllamav2		Makefile-exllamav2
Makefile-flash-att		Makefile-flash-att
Makefile-flash-att-v2		Makefile-flash-att-v2
Makefile-flashinfer		Makefile-flashinfer
Makefile-lorax-punica		Makefile-lorax-punica
Makefile-selective-scan		Makefile-selective-scan
Makefile-vllm		Makefile-vllm
README.md		README.md
bounds-from-nix.py		bounds-from-nix.py
kernels.lock		kernels.lock
pyproject.toml		pyproject.toml
req.txt		req.txt
requirements_cuda.txt		requirements_cuda.txt
requirements_gen.txt		requirements_gen.txt
requirements_intel.txt		requirements_intel.txt
requirements_rocm.txt		requirements_rocm.txt
uv.lock		uv.lock

README.md

Text Generation Inference Python gRPC Server

A Python gRPC server for Text Generation Inference

Install

make install

Run

make run-dev