llama.cpp adds an initial ExecuTorch backend for edge and on-device deployment

Build b9951 of llama.cpp, tagged July 10, adds an initial ExecuTorch backend via PR #24179. ExecuTorch is PyTorch’s on-device inference runtime: it targets mobile, embedded, and edge hardware, with backends for Qualcomm QNN, Apple CoreML/MPS, XNNPACK, and others. This PR is the first foothold for running GGUF-format models through that stack.

The commit list is substantial for an “initial” drop. The backend ships with a working set of kernels covering the operations you actually need for a transformer: MUL_MAT, ROPE, RMS_NORM, GLU, SOFT_MAX, GET_ROWS, and several more. There’s FlashAttention support, Q4_0 and Q4_K GET_ROWS, kernel fusion (RMS_NORM + MUL), and parallelized softmax and matmul. The kernels are built as part of ggml and embedded with a filesystem fallback. The release example in the source shows a Qwen3-0.6B-Q8_0.gguf running through the backend with the ET-SoC-1 emulator, which gives you a sense of where the hardware ambitions point.

This is not a finished integration. The PR title says “initial,” and the commit log includes notes like “update limitations” and disabled offloading for debug builds. What model families are fully runnable through the ET backend in production is not yet clear from the release notes alone. Think of this as the foundational PR that makes a future llama.cpp-on-edge-silicon guide possible, not the guide itself. Worth watching PR #24179 directly if you have a deployment target that isn’t a desktop GPU.