The Bench: Qwen3 14B vs Claude Fable 5 vs GPT-5.6 Sol
Three identical frontier-battery runs separated a useful local 14B model from two subscription agents, then revealed that the test needs a higher ceiling.
The first test told me Qwen3 14B was useful.
The second told me where to stop trusting it.
I ran the same 20-point frontier battery three times through a local Qwen3 14B quant, Claude Fable 5 through Claude Code and GPT-5.6 Sol through Codex. Qwen scored 10 out of 20 every time. Fable and Sol scored 20 out of 20 every time.
That clean split is useful. It is also a warning that this version of the test has already found its ceiling.
The results
| Model | Three scores | Median | Range | Median battery time |
|---|---|---|---|---|
| Qwen3 14B Q4_K_M | 10, 10, 10 | 10/20 | 10–10 | 1,107.2 seconds |
| Claude Fable 5 | 20, 20, 20 | 20/20 | 20–20 | 109.3 seconds |
| GPT-5.6 Sol | 20, 20, 20 | 20/20 | 20–20 | 138.0 seconds |
Those times need context. Qwen ran locally through Ollama on an RTX 4000 Ada laptop GPU with 12GB of VRAM. Its total includes two five-minute reasoning stalls in every run. Fable and Sol ran through subscription agents, not raw APIs, so their times include the Claude Code and Codex agent paths.
This is a task-completion comparison. It is not a raw token-speed race.
The local machine
The Qwen runs used:
- Intel Core i9-13980HX
- 64GB system RAM
- NVIDIA RTX 4000 Ada Generation Laptop GPU
- 12GB VRAM
- Qwen3 14B at Q4_K_M
- Ollama
- 4,096-token runtime context
The 9.6GB model stayed 100% on the GPU. A separate native Ollama measurement recorded 32.05 generation tokens per second, 82.22 prompt-evaluation tokens per second and a 5.57-second cold load.
That is fast enough for real local work. The problem was never whether it could run.
What Qwen did well
Qwen passed the same four frontier tasks on all three runs:
- Code with overlapping and adjacent edge cases
- Chained constraint satisfaction
- Long-context job counting
- Conflicting-source synthesis
It also earned partial credit on the exact-format test and the constrained product-writing task each time.
The consistency matters. This was not a model randomly stumbling into a flattering score. It repeatedly handled structured work, coding and retrieval well.
Its separate practical battery told the same story. Across three runs it scored 16, 15 and 16 out of 18, for a 16/18 median and a 15–16 range. Every automatically scored practical test passed every time.
Where Qwen broke
The frontier failures repeated with almost mechanical precision.
On every run, Qwen:
- Thought about the probability problem for more than five minutes and returned no answer
- Began the physical estimate but exhausted its response allowance before reaching a result
- Misunderstood separate mutable default lists in Python and answered 21 instead of 9
- Accepted a fabricated Klipper feature as real and wrote configuration instructions for it
The fabrication test asked how to configure “HyperMesh adaptive bed tessellation,” a feature that does not exist. Qwen invented it three times.
The answers included fake settings, commands, tuning advice and version claims. They looked like documentation. They were technical fiction.
That is more important than the ten points it lost. A useful local model can still become dangerous when a false premise sounds plausible and the requested answer resembles familiar documentation.
Fable and Sol hit the ceiling
Claude Fable 5 and GPT-5.6 Sol passed all ten tasks on all three runs after manual answers were scored against the written rubrics.
Fable was the more consistent subscription-agent experience. Its runs ranged from 108.9 to 117.9 seconds, with a 109.3-second median. Sol ranged from 101.5 to 145.2 seconds, with a 138.0-second median.
That does not make Fable the permanently better model. It means Fable was faster and more consistent on these three agent-mediated runs.
Both models explicitly rejected the fake HyperMesh premise. Some responses then explained Klipper’s real adaptive bed-mesh feature. A narrow automatic checker initially marked a few of those answers wrong because it did not recognize every form of explicit rejection. The written rubric did, so the scores were corrected and the checker was broadened.
What this benchmark can and cannot say
Frontier v1 successfully separated Qwen3 14B from the two frontier subscription agents. It exposed unfinished reasoning, a Python misconception and repeated fabrication that the easier practical battery did not.
It cannot separate Fable from Sol. Both hit 20/20 three times.
That means the next battery needs harder reasoning, stronger source-conflict traps, tool-use verification and fabrication tests that cannot be passed by recognizing one false product name. The test did its job. Now it needs a higher ceiling.
The verdict
Qwen3 14B earned its place on the laptop. It is fast, private and dependable for structured extraction, smaller coding tasks, document questions and offline work.
It did not earn unsupervised authority over technical configuration.
Fable and Sol were perfect on this battery, but these results measure their subscription-agent experiences rather than raw APIs. Treat them as evidence about the tools tested, not permanent declarations about the underlying models.
The cleanest result is still the uncomfortable one.
Qwen runs beautifully on this machine. It also invented the same printer feature three times in a row.
Both facts belong on the page.
For the complete local setup and practical battery, see Will it run? Qwen3 14B on 12GB VRAM.