VQV Signal

ROBOTS & HARDWARE WATCH TECHNICAL

Mooncake team says vLLM, Dynamo and Mooncake share a rack-scale KV cache on Vera Rubin

According to a post on X by @KVCache_AI, the Mooncake team says vLLM, NVIDIA Dynamo and Mooncake together share a rack-scale KV cache across prefill instances on Vera Rubin, pooling spare GPU memory as an RDMA-reachable cache tier. The team reports a 96.1% prefix-cache hit rate and a median time to first token cut from 29.8 s to 4.2 s. The post also cites a vLLM blog dated Nov 12 that reports DeepSeek-V4.1-Flash at 2.52M tokens/s per GPU at peak on one Vera Rubin NVL72 rack. The source snippet is truncated, so further details are not available here.

Source: Scobleizer News Pulse (X) · x.com Found via Robert Scoble’s Scobleizer News Pulse · a post on X, not independently verified Published 2026-11-18T05:45:00+00:00 Detected 2026-11-18T20:03:00+00:00
View original source

According to a post on X by @KVCache_AI, the Mooncake team says vLLM, NVIDIA Dynamo and Mooncake together share a rack-scale KV cache across prefill instances on Vera Rubin, pooling spare GPU memory as an RDMA-reachable cache tier. The team reports a 96.1% prefix-cache hit rate and a median time to first token cut from 29.8 s to 4.2 s. The post also cites a vLLM blog dated Nov 12 that reports DeepSeek-V4.1-Flash at 2.52M tokens/s per GPU at peak on one Vera Rubin NVL72 rack. The source snippet is truncated, so further details are not available here.

AI-assisted summary based on the listed source.

The Mooncake team says vLLM, NVIDIA Dynamo and Mooncake together share a rack-scale KV cache across prefill instances on Vera Rubin and pool spare GPU memory as an RDMA-reachable cache tier. They report a 96.1% prefix-cache hit rate and median time to first token cut from 29.8 s to 4.2 s; vLLM's Nov 12 blog...

Teams running LLM inference on Vera Rubin hardware could see lower time to first token and better cache reuse if these reported results hold up. The figures come from the project team and a vLLM blog, and VQV has not checked them. Operators and builders should watch for published benchmarks, configuration details, and whether vLLM, Dynamo and Mooncake release the shared cache setup.

Hardware and robotics watchers may want to track whether this becomes a product, benchmark, or deployment signal.

Signal Strength 95% Technical label WATCH Public Interest 50 Category ROBOTS & HARDWARE Reader Depth TECHNICAL

Signal Strength reflects source quality, relevance, freshness and evidence. Public Interest helps organize discovery; it is not proof of truth.

Public Interest components
Recognizable Entity Score 82 Practical Impact Score 0 Novelty Interest Score 94 Consequence Score 18 Curiosity Score 16 Shareability Score 63

VQV surfaced this signal because it is recent, relevant to LLM Inference, connected to Scobleizer News Pulse (X).