llama.cpp b10069: Adreno OpenCL broadcast support
llama.cpp build b10069 adds OpenCL broadcast support for Adreno MUL_MAT operations and fixes view offset handling for Adreno Q8_0 MUL_MAT in llama-server multi-stream mode.
NInfer: 542 tok/s on Qwen3.6-35B-A3B, single request, one RTX 5090
A solo dev open-sourced NInfer, a from-scratch C++/CUDA engine that sustains 542 tok/s on Qwen3.6-35B-A3B over a full 65K-token decode, single request, on one RTX 5090.
Running three Qwen3.5 122B sessions on one Mac Studio with an on-disk KV cache
A Reddit user reportedly ran three concurrent Qwen3.5 122B sessions on a single Mac Studio, serving over 93% of prompt tokens from an on-disk KV cache instead of recomputing on GPU.
vLLM v0.25.0 makes Model Runner V2 the default and drops PagedAttention
vLLM v0.25.0 ships MRv2 as the default execution path for all dense models and permanently removes the legacy PagedAttention implementation.
Colibrì runs GLM-5.2 (744B MoE) on 25GB of RAM by streaming experts from NVMe
Colibrì is a pure-C, zero-dependency inference engine that runs GLM-5.2, a 744B MoE model, on a consumer machine with 25GB of RAM, by keeping dense weights in RAM and streaming routed experts from disk.