community pool / model answer

Qwen3.8-27B

chat · Qwen/Qwen3.8-27B

50.9 tok/s · model median · 105 runs in the model record.

Reference rig	Quantization	Median result	Basis	TTFT (ms)	Peak VRAM	Max context	Engines
Arc Pro B70 32GB	4-bit	44 tok/s	measured	141	- GB	131,072	llama.cpp, vllm
CMP 170HX 64GB	16-bit	24.4 tok/s	measured	5,898	- GB	65,799	vllm
GB10 Grace Blackwell 256GB	16-bit	8.2 tok/s	reported	283	- GB	2,048	vllm
H100 NVL 94GB ×2	8-bit	293.3 tok/s	reported	69	- GB	2,048	sglang
Radeon AI Pro R9700 32GB	4-bit	285.5 tok/s	measured	115	19 GB	98,304	hipfire
Radeon AI Pro R9700 32GB	6-bit	38.7 tok/s	reported	77,056	29.9 GB	131,072	llama.cpp
Radeon AI Pro R9700 32GB ×2	8-bit	46.8 tok/s	reported	-	- GB	262,144	llama.cpp
Radeon AI Pro R9700 32GB ×3	4-bit	28.4 tok/s	measured	919	- GB	829	llama.cpp
Radeon AI Pro R9700 32GB ×3	8-bit	52.9 tok/s	measured	808	48.6 GB	262,144	llama.cpp, lucebox
RTX 3090 24GB	4-bit	113.2 tok/s	reported	91	22.4 GB	65,536	vllm
RTX 3090 24GB	6-bit	53 tok/s	reported	208	- GB	2,048	llama.cpp
RTX 3090 24GB ×2	4-bit	72.8 tok/s	measured	949	- GB	262,144	vllm
RTX 5060 Ti 16GB ×2	4-bit	63.2 tok/s	measured	190	- GB	131,072	llama.cpp, vllm
RTX 5090 32GB	4-bit	157.4 tok/s	measured	279	24 GB	170,000	llama.cpp, lucebox, vllm
RTX PRO 6000 Blackwell 96GB	4-bit	81.8 tok/s	measured	215	- GB	2,048	llama.cpp, ollama
RTX PRO 6000 Blackwell 96GB	8-bit	84.5 tok/s	reported	379	- GB	2,048	llama.cpp
RTX PRO 6000 Blackwell 96GB	16-bit	64.7 tok/s	reported	92	- GB	262,144	vllm
RX 7800 XT 16GB	4-bit	28 tok/s	reported	-	- GB	2,048	llama.cpp
RX 7900 XTX 24GB	4-bit	90.7 tok/s	measured	646	17.5 GB	32,768	hipfire, llama.cpp
RX 7900 XTX 24GB ×2	8-bit	35.2 tok/s	measured	671	31.7 GB	16,384	llama.cpp
Ryzen AI Max 395 128GB	4-bit	65 tok/s	measured	2,528	18 GB	4,096	hipfire, llama.cpp
Ryzen AI Max 395 128GB	8-bit	14.2 tok/s	measured	2,090	- GB	829	llama.cpp
Ryzen AI Max 395 64GB	4-bit	18.5 tok/s	measured	4,379	17.4 GB	8,192	llama.cpp
Ryzen AI Max 395 64GB	6-bit	17.3 tok/s	measured	4,772	25.7 GB	8,192	llama.cpp
Tesla V100 16GB ×4	4-bit	391 tok/s	reported	349	61 GB	32,768	vllm

Context tested

The table above is the tested context for each exact rig and quantization cell. Empty fields remain empty rather than being filled with an estimate.

back to The wall ->capture / correct ->