Why do we need KV caching?
A visual explanation of why LLM inference caches key and value vectors, and why the trick saves compute during token-by-token generation.
9 min read
home
duahaulaplanh
@nguyen le
build · break · understand
about
- I'm interested in LLM inference. Especially kernel engineering (CUDA, Metal, Triton, etc.) and some LLM serving systems such as vLLM or SGLang.
- Beside LLM, I'm also interested in mathematics and some theoretical computer science stuffs.
-
I also love to spend time with
my family 
my friends
work
VinMotion, Vietnam
Robotics Engineer
Met great people here (my peers and seniors).
December 2025 - June 2026
VinBigdata, Vietnam
AI Engineer Intern
Made some good friends along the way.
July 2025 - December 2025
education
VNUHCM - University of Science, Vietnam
B.Sc. in Artificial Intelligence
Best investment in myself I ever made
Oct 2021 - Oct 2025
