AI Radar

Open source

Free AI models and tools that anyone can download. Newest first.

Importance 2 of 5r/LocalLLaMA

Developer shows abliteration can be applied as a small runtime LoRA instead of full models

A post explains that most abliteration techniques are rank-1 ablations that fit in a tiny LoRA applied at runtime in llama.cpp. This avoids downloading separate abliterated models and can be toggled per request.

Why it matters: Refusal removal is cheap and easy to apply to open models, which matters for safety controls on self-hosted deployments.

Importance 2 of 5Simon Willison

Simon Willison releases datasette-maplibre 0.1a0, built mostly with one Claude Fable 5.2 prompt

Simon Willison released datasette-maplibre 0.1a0, a Datasette geospatial plugin combining MapLibre GL with the free OpenFreeMap vector map server. He says most of it was built from a single prompt to Claude Fable 5.2.

Why it matters: It is another practical example of frontier coding models producing usable software from a short prompt.

Importance 2 of 5r/LocalLLaMA

audio.cpp 0.9.5 adds real-time local avatar voice chat combining three open models

audio.cpp 0.9.5 adds an avatar voice chat app in its WebUI that combines SoulX-FlashHead for talking head video, OWSM v4 for speech recognition and LFM3-Audio for responses and TTS. Video generation still needs an RTX 4090 or better with cuDNN for real-time use.

Why it matters: It shows interactive voice avatars can run fully locally, though hardware demands remain high.

Only want what affects you?
Get a weekly digest filtered to your company. Free.
Filter my radar
Importance 2 of 5r/MachineLearning

Training-free method edits one weight row so reasoning models stop thinking early

A developer showed that folding a linear probe into a single row of a reasoning model's output layer lets it end its thinking phase once it has reached an answer. The edited model needs no training or extra inference code.

Why it matters: Cutting redundant reasoning tokens could lower inference cost and latency for teams running open reasoning models.

Importance 2 of 5r/LocalLLaMA

User benchmarks 1Cat-vLLM expert caching against Strata on Tesla V100 rig

A user compared inference speeds of 1Cat-vLLM, a Tesla V100 focused fork of vLLM that recently added expert caching, with Strata. The test used a Qwen3.8-Flash-Next IQ3_XXS quant on four 16 GB V100 cards with an Epyc CPU and DDR4 memory.

Why it matters: Expert caching on older GPUs could make large mixture-of-experts models cheaper to run locally on used hardware.

Importance 2 of 5r/LocalLLaMA

VillanovaAI releases Villanova2 open model family for five European languages

VillanovaAI released the Villanova2 family for English, Italian, German, Spanish and French under Apache 2.0. It includes a 15.5B-total, 3.9B-active mixture-of-experts model and a dense 2.4B model, each in base, instruct and vision-language versions, with GGUFs available.

Why it matters: European-built open models give EU companies more options for multilingual work under a permissive license.

Importance 2 of 5r/LocalLLaMA

Open-source tool tunes cudaMemcpy to speed MoE CPU offload prefill by about 35%

A developer released a tool that tunes CUDA memory copies used when offloading mixture-of-experts layers to the CPU. On an RTX 3090 with Qwen3.6 35B-A3B in llama.cpp, prefill rose from 345 to 466 tokens per second.

Why it matters: Cheap software tuning can noticeably improve local inference on consumer GPUs for models too large for VRAM.

Importance 2 of 5r/LocalLLaMA

Basalt 0.4.0 inference engine adds Qwen 3.8 27B, reaching up to 776 tok/s on an RTX 5090

The Basalt local inference engine version 0.4.0 adds its first dense model, Qwen 3.8 27B, with up to 776 tokens per second on structured output on an RTX 5090. Recent updates also added multi-model and router support, DeepSeek V4.1 Flash, video support and files in chats.

Why it matters: Faster local inference makes running capable open models on a single workstation more practical.

Importance 2 of 5r/MachineLearning

Marimba released as open lightweight speech decision model built on Qwen3-ASR-0.6B

A developer released Marimba, a fast, lightweight model that returns typed decisions about speech such as yes/no answers, choices or scores. It runs on a frozen Qwen3-ASR-0.6B, using zero-shot decoder scores for word and language questions and small readouts for voice activity, turn endings, speaker changes and speaker count. Weights are open under Apache-2.0 and a paper is due on arXiv.

Why it matters: Teams building voice agents could use small open components like this for turn detection and speaker handling without large models.

Importance 2 of 5r/LocalLLaMA

Axiomic Labs releases X3Guard-150M, a small prompt guard model for edge devices

Axiomic Labs released X3Guard-150M, a prompt guard model that flags whether a prompt is harmful and whether it is adversarial in a single pass. It is built on the company's GPT-X3 base model and runs in the browser with response times in the tens of milliseconds on iPhone.

Why it matters: Small guard models let teams screen prompts cheaply on device before they reach larger models.

Importance 2 of 5Hacker News (AI, 100+ points)

Essay argues all AI-generated code should be open source

A widely discussed blog post argues that code generated by AI should be released as open source. It gathered nearly 900 points on Hacker News.

Why it matters: The debate touches on licensing and ownership questions companies face when shipping AI-written code.

Importance 2 of 5r/LocalLLaMA

Meituan releases open-source LongCat-Video-Avatar 1.6 for audio-driven video

Meituan released LongCat-Video-Avatar 1.6 on Hugging Face, an upgrade to its open-source framework for audio-driven human video generation. It improves lip sync, facial and body motion, and long-form consistency, and adds inference optimizations.

Why it matters: Open avatar video models lower the cost of building synthetic presenters and video content in-house.

Importance 4 of 5The Decoder

Meituan releases open weights for LongCat-2.5, a 1.6T-parameter agent model trained on Chinese chips

Meituan released the weights for LongCat-2.5, a 1.6-trillion-parameter model built for AI agents and trained entirely on Chinese hardware. It scores 88.4 on Terminal Bench, just behind Claude Opus 5. The sample deployment runs on 194 Huawei Ascend chips with no mention of Nvidia GPUs.

Why it matters: A near-frontier open agent model trained without Nvidia hardware shows Chinese labs can work around export controls, widening open-weight options for enterprises.