Moonshot AI's Kimi K3 shipped with open weights and enough capability to restart the DeepSeek argument: if a model you can download keeps pace with the frontier, what were the export controls buying?
llama.cpp build b10069 adds OpenCL broadcast support for Adreno MUL_MAT operations and fixes view offset handling for Adreno Q8_0 MUL_MAT in llama-server multi-stream mode.
A solo dev open-sourced NInfer, a from-scratch C++/CUDA engine that sustains 542 tok/s on Qwen3.6-35B-A3B over a full 65K-token decode, single request, on one RTX 5090.
Moonshot's Kimi K3 sits at #1 on the Code Arena Frontend leaderboard, but scores only ~39% on FrontierMath Tier 4 versus ~90% for top OpenAI and Anthropic models.
Running Qwen3.6 27B with Opencode? The model's appetite for context tokens can gut your usable window fast. Here's what's happening and how to push back.
Moonshot AI announced Kimi K3, a 2.8-trillion-parameter MoE model, on July 16. Open weights are promised by July 27. Until then it is API-only, and running it yourself will need serious cluster hardware.
llama.cpp build b10067 patches a bug where the DeepSeek-V4 routing table tensor was being incorrectly quantized, causing a type conversion failure that blocked anyone using llama-quantize on the model.
A Reddit user reportedly ran three concurrent Qwen3.5 122B sessions on a single Mac Studio, serving over 93% of prompt tokens from an on-disk KV cache instead of recomputing on GPU.
Google shipped a silent update to Gemma 4 fixing tool-calling bugs, truncated responses, and enabling Flash Attention 4 on Hopper GPUs for a reported 25–70% prefill speedup.
A Reddit report claims the 98GB DeepSeek-V4-Flash quant jumped from 2 to 7 t/s on a single 4060 Ti plus CPU, with no hardware changes between two llama.cpp builds.
xAI's coding agent got caught uploading home directories, SSH keys and all, to cloud buckets. The lesson is not "never trust xAI." It is that "local" describes what leaves your machine, and almost nobody checks.
Build b9993 merges PR #25395, adding native hy_v3 architecture support so Tencent's Hunyuan 3 MoE runs locally in llama.cpp, with MTP speculative decoding included.
A llama.cpp fork from spiritbuun adds Variable Bit Rate KV cache quantization, dynamically degrading cache precision only when VRAM pressure demands it.
vLLM v0.25.0 ships MRv2 as the default execution path for all dense models and permanently removes the legacy PagedAttention implementation.
Three identical frontier-battery runs separated a useful local 14B model from two subscription agents, then revealed that the test needs a higher ceiling.
Getting clean structured output from a 7B or 8B model is mostly about the example object, not the rules. This is the pattern that stopped my local extraction jobs from breaking.
Colibrì is a pure-C, zero-dependency inference engine that runs GLM-5.2, a 744B MoE model, on a consumer machine with 25GB of RAM, by keeping dense weights in RAM and streaming routed experts from disk.
Build b9963 lands proper multi-tile support for DeepSeek-OCR v1 with dynamic resolution, and unifies the image preprocessor for both v1 and v2. Multi-column documents now handled correctly.
AMD's 128GB unified-memory Ryzen AI Halo developer box launched at $3,999, about $700 under NVIDIA's Linux-only DGX Spark, sold in the US exclusively through Micro Center in-store pickup.
Build b9951 of llama.cpp drops an initial ExecuTorch backend, planting a flag for GGUF-format models on PyTorch's on-device inference stack and the edge hardware it targets.
A community rig ran NVIDIA's Nemotron Puzzle 75B-A9B in NVFP4 across three power-capped RTX 3090s at 132 t/s decode, and asked why almost nobody ships models shaped for multi-24GB rigs.
Unsloth published NVFP4 quantized checkpoints for Qwen3.6 27B and 35B-A3B that reportedly beat NVIDIA's reference quants on throughput, with FP8 KV cache support for longer contexts.
SK Hynix expects memory demand to outrun its supply beyond 2030, but the HBM crunch is not yet a reason to panic-buy a consumer GPU.
Z.ai's GLM-5.2 shipped under an MIT license and beats GPT-5.5 on coding benchmarks. It's also a 744B model that needs a server rack, not a desk.
Mistral's CEO confirmed a new open-weight model with early access this month, as the company reportedly raises $3.5B on around $400M in annual revenue.
Ollama's latest build nearly doubles Gemma 4 token generation on Apple Silicon with multi-token prediction. It's automatic, needs no config, and doesn't change the output.
A wave of low-cost accelerator HATs and NPU-equipped boards is pushing real on-device inference down to single-board-computer prices. The interesting part isn't the TOPS number, it's where the work moves.
No cloud, no API key, no data leaving your machine. A genuinely beginner-proof path to running a capable model locally, plus how to know which size your hardware can actually handle.
