Importance 2 of 5Covered by 1 source
User benchmarks 1Cat-vLLM expert caching against Strata on Tesla V100 rig
A user compared inference speeds of 1Cat-vLLM, a Tesla V100 focused fork of vLLM that recently added expert caching, with Strata. The test used a Qwen3.8-Flash-Next IQ3_XXS quant on four 16 GB V100 cards with an Epyc CPU and DDR4 memory.
Why it matters
Expert caching on older GPUs could make large mixture-of-experts models cheaper to run locally on used hardware.