Storage in the Serving Path: Giorgio Regni and Paul Speciale on Shared KV Cache and Disaggregated Inference.
Q1. Giorgio, you describe (*) the KV cache on ADI as “fast enough to sit in the serving path.” Traditionally, storage has been too slow for this role, which is why the KV cache has...