community pool / model answer

NVIDIA-Nemotron-3.5-Lightning-30B-A3B-W4A16

chat · useful-quants/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-W4A16

226.4 tok/s · model median · 38 runs in the model record.

Answer

226.4tok/s

measuredCMP 170HX 64GB4-bit · 38 runs · decodebest cell in the pool

  1. CMP 170HX 64GB226.4 tok/smeasured38 runs · peak VRAM no data yet · 64 GB rig memory
Reference rigQuantizationMedian resultBasisTTFT (ms)Peak VRAMMax contextEngines
CMP 170HX 64GB4-bit226.4 tok/smeasured1,697- GB67,592vllm

Context tested

The table above is the tested context for each exact rig and quantization cell. Empty fields remain empty rather than being filled with an estimate.