community pool / model answer

MiniCPM5-2B

chat · openbmb/MiniCPM5-2B

66.8 tok/s · model median · 4 runs in the model record.

Answer

66.7tok/s

measuredRX 570 4GB4-bit · 3 runs · decodebest cell in the pool

  1. RTX 5070 12GB118.2 tok/sreported1 run · peak VRAM no data yet · 12 GB rig memory
  2. RX 570 4GB66.7 tok/smeasured3 runs · peak VRAM no data yet · 4 GB rig memory
Reference rigQuantizationMedian resultBasisTTFT (ms)Peak VRAMMax contextEngines
RTX 5070 12GB16-bit118.2 tok/sreported23- GB32,768vllm
RX 570 4GB4-bit66.7 tok/smeasured17,878- GB2,048llama.cpp

Context tested

The table above is the tested context for each exact rig and quantization cell. Empty fields remain empty rather than being filled with an estimate.