Native Shell
Native Shell: AI-generated UI, native in your app, on every platform. Apps on iOS, Android, macOS, Windows and Linux, GPUI apps and web pages load signed, sandboxed TSX modules at run time and draw th
RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
Translation pending
Developer toolsPython · CudaGitHub @alibabaSelf-installOwner unconfirmed
Paste into Claude or ChatGPT for installation instructions suited to your environment.
Have you tried it? One tap is enough. No sign-in needed.
The maker's introduction consists of the line above. Read the repository README for more details. Repository README ↗
Loading the full README.
Read the full README ↗| Documentation | Contact Us |
RTP-LLM is a Large Language Model (LLM) inference acceleration engine developed by Alibaba's Foundation Model Inference Team. It is widely used within Alibaba Group, supporting LLM service across multiple business units including Taobao, Tmall, Idlefish, Cainiao, Amap, Ele.me, AE, and Lazada.
RTP-LLM is a sub-project of the havenask project.
Trusted and deployed across numerous LLM scenarios:
Learn more about RTP-LLM's performance in our benchmark reports:
Our project is mainly based on FasterTransformer, and on this basis, we have integrated some kernel implementations from TensorRT-LLM. We also draw inspiration from vllm, transformers, llava, and qwen-vl. We thank these projects for their inspiration and help.
If you find RTP-LLM useful in your research or project, please consider citing:
@Misc{rtp-llm,
author = {Alibaba},
title = {RTP-LLM: A High-Performance LLM Inference Engine},
howpublished = {\url{https://github.com/alibaba/rtp-llm}},
year = {2025},
}