Comprehensive evaluation of quantized LLM variants for local tool-calling, agent autonomy, and Finnish localization on NVIDIA GB10 hardware.
3 core models benchmarked, alongside DSPARK speculative decoding impact analysis.
| Variant | Short (15) | Tool-Call (69) | Finnish (10) | Notes |
|---|---|---|---|---|
| Q1_0 (1-bit, 3.6 GB) | 87 | 81 | 85 | Fastest, but Parameter Precision (67%) & Context (65%) weakest |
| Ternary Q2_0 (7.2 GB) | 93 | 85 | 90 | Best overall โ highest quality, clean full run โญ |
| Q2_g64 (7.6 GB) | โ | โ | โ | Corrupted GGUF โ upstream converter bug |
How DSPARK affects throughput vs single-turn latency on GB10 hardware.
| DSPARK Variant | Size | Impact on Quality | Impact on Speed / Latency |
|---|---|---|---|
| Ternary Q2_0 (Native, No DSpark) โญ | 7.2 GB | 83 / 100 (full suite) | 1.9s median turn โ Recommended on DGX Spark. PARALLEL=1, no drafter overhead. Best quality + speed on bandwidth-starved GPUs. |
| Ternary + DSpark (Corrected K=4) | 9.1 GB | 83 / 100 (+6 pts from old 77) | 4.9s median turn (3.3ร faster than old 16.2s) โ FIXED: llama.cpp backend, `--spec-draft-n-max 4`, `PARALLEL=1`. DSpark hurts on Spark โ use native. |
| Ternary + dspark (Old, Contaminated) | 9.1 GB | 77 / 100 | 16.2s median turn โ vllm backend, unverified dspark config, auto-parallel=4. See corrected run above. |
| Q1_0 + dspark | 5.3 GB | Lossless โ Short (15) score 87 | 14.2s median turn (vs 5.1s base) โ Uncapped draft depth adds latency per request. |
Visual breakdown of full-suite scores, category performance, and speed-vs-quality tradeoffs.
15-category radar comparison across key models with full category data.
Throughput gains vs latency penalties across workload types.
Deep-dive into each benchmarked variant with category breakdowns and verdicts.
Observed weaknesses across all benchmarked models.
Practical recommendations for SMB and entrepreneur use cases based on hard benchmark data.
Complete record of all 16 benchmark runs with run IDs and report files.
| Run ID | Model | Suite | Score |
|---|