Skip to content

Knowledge, ready when you need it.

Concepts, technologies, patterns and quick references for ML interviews and system design. Each topic gives you the short version first, goes deeper when you want it, and links to the Lessons and Problems that use it.

All topics

Show all topics
  • Concept

    The key-value cache: what it stores, what it costs and what it limits

    What a model's key-value cache holds during generation, how its memory grows with model shape and context length, and how that limits concurrent requests. Separates it from prompt (prefix) caching and from application-level response caching.

    Topics: kv-cacheinferencememory