community pool / model answer

gemma-4-31B-it-GGUF

chat · unsloth/gemma-4-31B-it-GGUF

15.5 tok/s · model median · 26 runs in the model record.

Reference rig	Quantization	Median result	Basis	TTFT (ms)	Peak VRAM	Max context	Engines
M5 Max 128GB	4-bit	24.1 tok/s	reported	565	- GB	2,048	llama.cpp
M5 Max 128GB	6-bit	18.1 tok/s	reported	678	- GB	2,048	llama.cpp
M5 Max 128GB	8-bit	15.3 tok/s	reported	660	- GB	2,048	llama.cpp
M5 Max 128GB	16-bit	8.9 tok/s	reported	858	- GB	2,048	llama.cpp
Minisforum UM790 Pro 64GB	4-bit	4 tok/s	reported	-	- GB	2,048	llama.cpp
Radeon AI Pro R9700 32GB	4-bit	25.4 tok/s	reported	555	- GB	2,048	llama.cpp
RTX 3060 12GB ×2	4-bit	15.2 tok/s	measured	10,911	21.5 GB	98,304	llama.cpp
RTX 3090 24GB	4-bit	38.8 tok/s	reported	-	- GB	2,048	llama.cpp
RTX 3090 24GB ×2	6-bit	14.4 tok/s	reported	2,263	20.7 GB	32,768	llama.cpp
RTX 5090 32GB	6-bit	51.7 tok/s	reported	155	- GB	2,048	llama.cpp

Context tested

The table above is the tested context for each exact rig and quantization cell. Empty fields remain empty rather than being filled with an estimate.