#cuda

Technical publications and student engineering notes tagged with #cuda.

5 min read

Inside LLM Inference Engines: KV Caching, PagedAttention, and Continuous Batching

A deep architectural investigation into how modern LLM inference runtimes like vLLM, TensorRT-LLM, and SGLang achieve high throughput—from memory-bound decoding and KV cache fragmentation to virtual memory paging and continuous iteration-level scheduling.

ACM BIT Mesra AI & Systems WingAI Wing · ACM BIT Mesra

#Machine Learning #LLM Serving #Systems Architecture

Oct 18, 2026Read Article