Grouped by reported model identity. Exact weights, quantizations and software can differ. How results are reviewed
Documented results
24 selected tests| View ReportGeneric LLM | Llama-3.1-8B-InstructConflicting evidence | LoRA trainingDataset not reported512 configured context | 1.2864 lossTraining loss · Loss · Mean · Lower is better | — | — | 1 × R9700Single GPU · 32 GB VRAM/GPUConfiguration
|
|---|---|---|---|---|---|---|
| View ReportGeneric LLM | Llama-3.1-8B-Instruct | Token speed | — | 1,044.41 tok/sPrefill · mean · unknown | 60.37 tok/sGeneration · mean · unknown | GPU count unknownUnknown · 24 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama-3.1-8B-Instruct | Token speed | — | 1,068.59 tok/sPrefill · mean · unknown | 7.71 tok/sGeneration · mean · unknown | GPU count unknownUnknown · 24 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama-3.1-8B-Instruct | Token speed | — | — | 92.84 tok/sGeneration · per request | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View ReportServing benchmark | Llama-3.1-8B-InstructConflicting evidence | Token speed | 29.1 tok/sThroughput · per request | — | — | 1 × R9700Single GPU · 32 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama-3.1-8B-Instruct | Token speed | 61.8 tok/sThroughput · per request | — | — | 4 × R9700Tensor parallel · 32 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama-3.1-8B-Instruct | Token speed | — | — | 39.87 tok/sGeneration · per request | 1 × W7700Single GPU · 16 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama-3.1-8B-Instruct | Procyon | — | — | 53.17 tok/sGeneration · per request | 1 × RX 9060 XTSingle GPU · 16 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama-3.1-8B-Instruct | Procyon | — | — | 65.51 tok/sGeneration · per request | 1 × RX 9070 GRESingle GPU · 12 GB VRAM/GPUConfiguration
|
| View Reportllama-bench | Llama-3.1-8B-Instruct | Token speed | 92.08 tok/sInput + output · mean · per request | — | — | 1 × Ryzen AI Max+ 395Single GPU · Unified memory · 128 GB RAMConfiguration
|
| View ReportGeneric LLM | Llama-3.1-8B-Instruct | Token speed | 138 tok/sThroughput · mean · unknown | — | — | 2 × R9700Tensor parallel · 32 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama-3.1-8B-Instruct | Token speed | 77 tok/sThroughput · unknown | — | — | 1 × R9700Single GPU · 32 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama-3.1-8B-Instruct | Procyon1.0.191 | — | — | 89.79 tok/sGeneration · mean · per request | 1 × R9700Single GPU · 32 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama-3.1-8B-Instruct | Procyon | 1,787 pointsBenchmark score · Per request | — | — | 1 × RX 9070Single GPU · 16 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama-3.1-8B-Instruct | Relative performance | 1.79 x vs RTX 4500 BlackwellRelative performance per dollar · Whole system | — | — | 1 × R9700Single GPU · 32 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama-3.1-8B-InstructConflicting evidence | LoRA trainingDataset not reported512 configured context | ~1.28 lossTraining loss · Loss · Mean · Lower is better | — | — | 1 × R9700Single GPU · 32 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama-3.1-8B-Instruct | Token speed | — | 853.54 tok/sPrefill · mean · unknown | 35.27 tok/sGeneration · mean · unknown | GPU count unknownUnknown · 24 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama-3.1-8B-Instruct | Token speed | — | 819.42 tok/sPrefill · mean · unknown | 2.14 tok/sGeneration · mean · unknown | GPU count unknownUnknown · 24 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama-3.1-8B-Instruct | Token speed | — | — | 93.34 tok/sGeneration · per request | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View ReportServing benchmark | Llama-3.1-8B-InstructConflicting evidence | Token speed | 102 tok/sThroughput · aggregate | — | — | 1 × R9700Single GPU · 32 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama-3.1-8B-Instruct | Token speed | — | — | 64.3 tok/sGeneration · per request | 4 × R9700Tensor parallel · 32 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama-3.1-8B-Instruct | Power & energy | 52 WPower draw · Whole system | — | — | 1 × W7700Single GPU · 16 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama-3.1-8B-Instruct | Procyon | — | — | 87.59 tok/sGeneration · per request | 1 × RX 9070Single GPU · 16 GB VRAM/GPUConfiguration
|
| View Reportllama-bench | Llama-3.1-8B-Instruct | Token speed | — | 11.53 tok/sPrefill · mean · per request | — | 1 × Ryzen AI Max+ 395Single GPU · Unified memory · 128 GB RAMConfiguration
|
Quantizations and software
Quantizations: BF16, FP8, FP16, Q4_K_M, Q4_K_S
Engines: llama.cpp, llama.cpp / llama-cpp-python, llamafile, ONNX Runtime, Procyon AI Text Generation / ONNXRuntime-DirectML, PyTorch, vLLM, VTE
This is a selection across sources, not a ranking or a matched experiment. Reading benchmark reports
GPU benchmark pages: W7900 · W7700 · R9700 · RX 9070 XT · RX 9070 · RX 9070 GRE · RX 9060 XT · RX 7900 XTX · Ryzen AI Max+ 395