Auto-tuning & speed
TurboLLM auto-tunes to your hardware on load: it benchmarks your exact GPU, derives fast launch defaults, and shows a VRAM-fit verdict before you load — so there's no flag guessing. It also reports real, measured tokens/sec, never a faked number.
What it tunes
On load, TurboLLM derives fast defaults for the flags that actually move the needle, so you don't have to hand-tune them per model:
Context & offload
Context length (-c), GPU offload (-ngl), and MoE CPU-offload (--n-cpu-moe).
Cache & throughput
KV-cache quant type, parallel slots, and CPU threads.
Speed features
Flash attention and speculative decoding — NextN self-speculative for models that carry a draft head.
Fast by default
TurboLLM turns on the speed wins automatically:
- Flash attention — on by default.
- NextN self-speculative decoding — on by default for any model that ships a draft head.
For sampling, auto-tune uses the recommended settings from Hugging Face: it checks
a repo's structured params / generation_config.json sidecar first, so defaults match
what the model author intended.
VRAM-fit verdict
Before you commit to a load, TurboLLM shows whether the configuration fits your VRAM — so you learn a model won't fit before waiting on a load, not after it fails.
VRAM headroom slider
Under Settings → Engine, the headroom slider (300 MB–2 GB, default 1 GB) tells auto-tune how much VRAM to keep free for other GPU workloads — ComfyUI, browser tabs, and anything else sharing the card.
Auto-fit GPU layers / MoE CPU offload
Two toggles let llama.cpp decide the GPU/CPU split for you at load time instead of using a fixed number — handy when a large context or model doesn't fit a fixed default. Both are off by default.
Real measured tokens/sec
The model list shows genuine throughput, not a marketing figure:
- Live tokens/sec while a model is generating.
- Last-session tokens/sec when the model is idle.
Profiles are saved per model, per engine, and you can run a model multi-GPU — splitting it across cards.
Speed comparison
RTX 5070 Ti 16 GB · Qwen3.6-35B-A3B at 200K context. Every number below is measured generation, not estimated:
| Build | TurboLLM | LM Studio | Speed-up |
|---|---|---|---|
| official llama.cpp — q4_0 | 74.7 t/s | 61.0 t/s | 1.2× |
| official llama.cpp — q8_0 | 72.3 t/s | ~66 t/s | 1.1× |
| TurboQuant fork — turbo4 | 24.6 t/s | 11.4 t/s | 2.2× |