community pool / model answer

Baichuan-M3-235B-GGUF

chat · mradermacher/Baichuan-M3-235B-GGUF

31.7 tok/s · model median · 4 runs in the model record.

Answer

31.7tok/s

measuredCMP 170HX 64GB ×23-bit · 4 runs · decodebest cell in the pool

  1. CMP 170HX 64GB ×231.7 tok/smeasured4 runs · peak 107.7 GB · fits 128 GB rig memory
Reference rigQuantizationMedian resultBasisTTFT (ms)Peak VRAMMax contextEngines
CMP 170HX 64GB ×23-bit31.7 tok/smeasured-107.7 GB32,768llama.cpp

Context tested

The table above is the tested context for each exact rig and quantization cell. Empty fields remain empty rather than being filled with an estimate.