KV Cache Size Calculator

Select LLM Model: Select data type: float16 bfloat16 float32 int8 Enter Number of Tokens: Calculate KV Cache Size Contribute new models on GitHub