community pool / model answer

Qwen3.8-27B-GGUF

chat · unsloth/Qwen3.8-27B-GGUF

44 tok/s · model median · 519 runs in the model record.

Answer

101.6tok/s

measuredRTX 5090 32GB4-bit · 13 runs · decodebest cell in the pool

  1. RTX 5090 32GB101.6 tok/smeasured13 runs · peak 21.3 GB · fits 32 GB rig memory · 5 cells
  2. RTX PRO 6000 Blackwell 96GB100.7 tok/smeasured3 runs · peak VRAM no data yet · 96 GB rig memory · 7 cells
  3. B200 192GB86.8 tok/sreported1 run · peak 17.3 GB · fits 192 GB rig memory
  4. RTX 3090 24GB72.6 tok/smeasured6 runs · peak 18.4 GB · fits 24 GB rig memory · 2 cells
  5. RTX 4070 12GB65.7 tok/sreported1 run · peak 9.3 GB · fits 12 GB rig memory
  6. RTX 4060 Ti 16GB ×253.6 tok/sreported1 run · peak VRAM no data yet · 32 GB rig memory
  7. RTX 3090 24GB ×251.6 tok/smeasured5 runs · peak VRAM no data yet · 48 GB rig memory · 2 cells
  8. CMP 170HX 64GB49.7 tok/smeasured174 runs · peak VRAM no data yet · 64 GB rig memory · 3 cells
  9. M4 Pro 24GB · unified memory47.2 tok/sreported1 run · peak VRAM no data yet
  10. RTX 5070 12GB46.3 tok/smeasured5 runs · peak 9.8 GB · fits 12 GB rig memory
  11. RTX 3080 12GB44.4 tok/smeasured3 runs · peak 9.7 GB · fits 12 GB rig memory
  12. RX 7900 XTX 24GB42.3 tok/smeasured8 runs · peak 22.9 GB · fits 24 GB rig memory
  13. RX 7800 XT 16GB42 tok/sreported1 run · peak VRAM no data yet · 16 GB rig memory
  14. Radeon AI Pro R9700 32GB41.2 tok/smeasured3 runs · peak 7.3 GB · fits 32 GB rig memory · 3 cells
  15. A100 40GB41.1 tok/sreported1 run · peak 19 GB · fits 40 GB rig memory · 2 cells
  16. Multi-GPU 56GB ×236.7 tok/sreported1 run · peak 45.5 GB · fits 112 GB rig memory
  17. Arc Pro B70 32GB31.2 tok/sreported1 run · peak VRAM no data yet · 32 GB rig memory · 3 cells
  18. Ryzen AI Max 395 128GB · unified memory26.7 tok/smeasured5 runs · peak 23.9 GB · 4 cells
  19. M5 Max 128GB · unified memory24.3 tok/sreported1 run · peak VRAM no data yet
  20. Ryzen AI Max 395 64GB · unified memory24.1 tok/smeasured8 runs · peak 18 GB
  21. Arc Pro B60 24GB21.2 tok/sreported1 run · peak VRAM no data yet · 24 GB rig memory · 2 cells
  22. RTX 3060 12GB ×218.7 tok/smeasured5 runs · peak 19.4 GB · fits 24 GB rig memory · 2 cells
  23. RTX 3060 12GB18.3 tok/smeasured5 runs · peak VRAM no data yet · 12 GB rig memory
Reference rigQuantizationMedian resultBasisTTFT (ms)Peak VRAMMax contextEngines
A100 40GB4-bit41.1 tok/sreported10,66719 GB65,536llama.cpp
A100 40GB8-bit33.7 tok/sreported10,19430 GB65,536llama.cpp
Arc Pro B60 24GB4-bit21.2 tok/sreported147- GB2,048llama.cpp
Arc Pro B60 24GB6-bit16 tok/sreported796- GB2,048llama.cpp
Arc Pro B70 32GB4-bit31.2 tok/sreported2,995- GB2,048llama.cpp
Arc Pro B70 32GB6-bit24.1 tok/sreported354- GB2,048llama.cpp
Arc Pro B70 32GB8-bit16.1 tok/sreported572- GB2,048llama.cpp
B200 192GB4-bit86.8 tok/sreported63217.3 GB32,768llama.cpp
CMP 170HX 64GB4-bit47 tok/smeasured29,673- GB132,352llama.cpp
CMP 170HX 64GB8-bit49.7 tok/smeasured27,843- GB132,352llama.cpp
CMP 170HX 64GB16-bit23.2 tok/smeasured13,004- GB67,549llama.cpp
M4 Pro 24GB4-bit47.2 tok/sreported373- GB2,048llama.cpp
M5 Max 128GB4-bit24.3 tok/sreported85- GB2,048ollama
Multi-GPU 56GB ×28-bit36.7 tok/sreported3,08245.5 GB2,048llama.cpp
Radeon AI Pro R9700 32GB1-bit41.2 tok/smeasured4557.3 GB32,768llama.cpp
Radeon AI Pro R9700 32GB4-bit39.5 tok/smeasured61521.6 GB262,144llama.cpp, lucebox
Radeon AI Pro R9700 32GB6-bit21.6 tok/smeasured64125.6 GB32,768llama.cpp
RTX 3060 12GB2-bit18.3 tok/smeasured150- GB32,768llama.cpp
RTX 3060 12GB ×23-bit15.5 tok/sreported-- GB32,768llama.cpp
RTX 3060 12GB ×24-bit18.7 tok/smeasured1,50619.4 GB131,072llama.cpp
RTX 3080 12GB2-bit44.4 tok/smeasured2,5269.7 GB65,536llama.cpp
RTX 3090 24GB4-bit72.6 tok/smeasured64718.4 GB32,768llama.cpp, ollama
RTX 3090 24GB6-bit51.5 tok/sreported206- GB2,048llama.cpp
RTX 3090 24GB ×24-bit73 tok/sreported1,175- GB2,048bunn-llama
RTX 3090 24GB ×28-bit51.6 tok/smeasured230- GB262,144llama.cpp
RTX 4060 Ti 16GB ×24-bit53.6 tok/sreported-- GB2,048llama.cpp
RTX 4070 12GB2-bit65.7 tok/sreported2279.3 GB2,048llama.cpp
RTX 5070 12GB2-bit46.3 tok/smeasured5729.8 GB32,768llama.cpp
RTX 5090 32GB2-bit88.1 tok/sreported82- GB2,048lmstudio
RTX 5090 32GB3-bit79.4 tok/smeasured38217.3 GB131,072llama.cpp
RTX 5090 32GB4-bit101.6 tok/smeasured13021.3 GB262,144llama.cpp, lmstudio
RTX 5090 32GB5-bit60.2 tok/smeasured42423.2 GB131,072llama.cpp
RTX 5090 32GB6-bit79.3 tok/smeasured42330.3 GB262,144llama.cpp, lmstudio
RTX PRO 6000 Blackwell 96GB2-bit100.7 tok/smeasured-- GB131,072llama.cpp
RTX PRO 6000 Blackwell 96GB3-bit88.9 tok/smeasured-- GB131,072llama.cpp
RTX PRO 6000 Blackwell 96GB4-bit79.3 tok/smeasured-- GB131,072llama.cpp
RTX PRO 6000 Blackwell 96GB5-bit67.8 tok/smeasured-- GB131,072llama.cpp
RTX PRO 6000 Blackwell 96GB6-bit57.5 tok/sreported-- GB131,072llama.cpp
RTX PRO 6000 Blackwell 96GB8-bit50.2 tok/smeasured42,652- GB131,072llama.cpp
RTX PRO 6000 Blackwell 96GB16-bit61 tok/smeasured40,786- GB131,072llama.cpp
RX 7800 XT 16GB4-bit42 tok/sreported-- GB121,000llama.cpp
RX 7900 XTX 24GB4-bit42.3 tok/smeasured29822.9 GB180,224hipfire, llama.cpp
Ryzen AI Max 395 128GB4-bit26.7 tok/smeasured60323.9 GB131,072llama.cpp, lucebox
Ryzen AI Max 395 128GB5-bit13.4 tok/sreported64223.1 GB4,096llama.cpp
Ryzen AI Max 395 128GB6-bit13.3 tok/sreported73025.7 GB4,096llama.cpp
Ryzen AI Max 395 128GB8-bit11.2 tok/sreported74631.1 GB4,096llama.cpp
Ryzen AI Max 395 64GB4-bit24.1 tok/smeasured4,33218 GB8,192llama.cpp

Context tested

The table above is the tested context for each exact rig and quantization cell. Empty fields remain empty rather than being filled with an estimate.