Importance 2 of 5Covered by 1 source
Basalt 0.4.0 inference engine adds Qwen 3.8 27B, reaching up to 776 tok/s on an RTX 5090
The Basalt local inference engine version 0.4.0 adds its first dense model, Qwen 3.8 27B, with up to 776 tokens per second on structured output on an RTX 5090. Recent updates also added multi-model and router support, DeepSeek V4.1 Flash, video support and files in chats.
Why it matters
Faster local inference makes running capable open models on a single workstation more practical.