Unsloth NVFP4 quants for Qwen3.6 claim faster throughput than NVIDIA's own builds
Unsloth published NVFP4 quantized checkpoints for Qwen3.6 27B and 35B-A3B that reportedly beat NVIDIA's reference quants on throughput, with FP8 KV cache support for longer contexts.
Unsloth has published NVFP4 quantized checkpoints for both Qwen3.6-27B and Qwen3.6-35B-A3B on Hugging Face, and the stated throughput numbers are worth paying attention to. According to the model cards, the 35B-A3B version runs at 1.56x the throughput of other NVFP4 quants, with a “Fast” variant pushing 1.79x. The Reddit post that flagged this also cited a 2.5x figure for the 27B, but that specific number isn’t confirmed on the primary Hugging Face model cards I could find, so treat it as unverified until you check the full Unsloth blog post directly.
The mechanism is W4A4: both weights and activations are quantized to 4-bit, which means matmuls actually run on the GPU’s 4-bit tensor cores rather than doing W4A16 (4-bit weights, 16-bit activations) the way NVIDIA’s reference quants do. That’s a real architectural difference, not a calibration tweak. Accuracy is validated on MMLU-Pro, AIME 2025, and GPQA against BF16 and FP8 baselines, and the Unsloth model cards claim the faster quants score similarly across all three. The 35B-A3B card also notes it works on a 32GB VRAM GPU, and vLLM is the recommended serving path for both.
FP8 KV cache is included and calibrated, which reportedly allows roughly 2x longer usable context on the same hardware. Both checkpoints include the MTP module for speculative decoding, so you can pass --speculative-config in vLLM and stack that on top of the NVFP4 gains. If you’re serving either of these models and you’ve been running NVIDIA’s reference NVFP4 build, it’s worth a benchmark.