PARALLEL DATA LAB 

PDL Abstract

Towards Efficient Generative Large Language Model Serving:A Survey from Algorithms to Systems

ACM Comput. Surv., Vol. 58, No. 1, Article 15. Publication date: September 2025.

Xupeng Miao, Gabriele Oliaro, Zhihao Zhang, Xinhao Cheng, Hongyi Jin, Tianqi Chen, Zhihao Jia

Carnegie Mellon University

http://www.pdl.cmu.edu/

In the rapidly evolving landscape of artificial intelligence (AI), generative large language models (LLMs)stand at the forefront, revolutionizing how we interact with our data. However, the computational intensityand memory consumption of deploying these models present substantial challenges in terms of servingefficiency, particularly in scenarios demanding low latency and high throughput. This survey addresses theimperative need for efficient LLM serving methodologies from a machine learning system (MLSys) researchperspective, standing at the crux of advanced AI innovations and practical system optimizations. We providein-depth analysis, covering a spectrum of solutions, ranging from cutting-edge algorithmic modifications togroundbreaking changes in system designs. The survey aims to provide a comprehensive understanding ofthe current state and future directions in efficient LLM serving, offering valuable insights for researchers andpractitioners in overcoming the barriers of effective LLM deployment, thereby reshaping the future of AI.

FULL TR: pdf