Why do we need KV caching?
A visual explanation of why LLM inference caches key and value vectors, and why the trick saves compute during token-by-token generation.
9 min read
tags
en
6 entries
Why I want to learn to code again?
8 min read
Fantastic Directions and Where to Find Them: Dissecting the Lazy Mechanism Inside RMU
18 min read
A Little Bit About "Numbers" and "Mathematics"
16 min read
A note on chapter 3 of Sutton & Barto: Finite Markov Decision Process (MDP)
20 min read
Residual Stream is Key to Transformer Interpretability
18 min read
