Llama 3.1 70B (checkpoint unspecified) on AMD GPUs
20 results · 2 sources · 20 measurements
Grouped by reported model identity. Exact weights, quantizations and software can differ. How results are reviewed
Documented results
20 selected tests| View Reportllama-bench | Llama 3.1 70B (checkpoint unspecified) | Token speed | — | 79.66 tok/sPrefill · mean · per request | — | 1 × Ryzen AI Max+ 395Single GPU · Unified memory · 128 GB RAMConfiguration
|
|---|---|---|---|---|---|---|
| View Reportllama-bench | Llama 3.1 70B (checkpoint unspecified) | Token speed | 35.74 tok/sThroughput · aggregate | — | — | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View Reportllama-bench | Llama 3.1 70B (checkpoint unspecified) | Token speed | — | — | 5.06 tok/sGeneration · mean · per request | 1 × Ryzen AI Max+ 395Single GPU · Unified memory · 128 GB RAMConfiguration
|
| View Reportllama-bench | Llama 3.1 70B (checkpoint unspecified) | Token speed | 38.49 tok/sThroughput · aggregate | — | — | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View Reportllama-bench | Llama 3.1 70B (checkpoint unspecified) | Token speed | — | 81.63 tok/sPrefill · mean · per request | — | 1 × Ryzen AI Max+ 395Single GPU · Unified memory · 128 GB RAMConfiguration
|
| View Reportllama-bench | Llama 3.1 70B (checkpoint unspecified) | Token speed | 37.63 tok/sThroughput · aggregate | — | — | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View Reportllama-bench | Llama 3.1 70B (checkpoint unspecified) | Token speed | — | — | 5.1 tok/sGeneration · mean · per request | 1 × Ryzen AI Max+ 395Single GPU · Unified memory · 128 GB RAMConfiguration
|
| View Reportllama-bench | Llama 3.1 70B (checkpoint unspecified) | Token speed | 40.28 tok/sThroughput · aggregate | — | — | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View Reportllama-bench | Llama 3.1 70B (checkpoint unspecified) | Token speed | 45.67 tok/sThroughput · aggregate | — | — | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama 3.1 70B (checkpoint unspecified) | Token speed | — | 96.2 tok/sPrefill · per request | — | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama 3.1 70B (checkpoint unspecified) | Token speed | — | — | 25.92 tok/sGeneration · per request | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View Reportllama-bench | Llama 3.1 70B (checkpoint unspecified) | Token speed | — | — | 15.47 tok/sGeneration · per request | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View Reportllama-bench | Llama 3.1 70B (checkpoint unspecified) | Token speed | — | — | 15.37 tok/sGeneration · per request | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View Reportllama-bench | Llama 3.1 70B (checkpoint unspecified) | Token speed | — | — | 14.98 tok/sGeneration · per request | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View Reportllama-bench | Llama 3.1 70B (checkpoint unspecified) | Token speed | — | 101.75 tok/sPrefill · per request | — | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View Reportllama-bench | Llama 3.1 70B (checkpoint unspecified) | Token speed | — | 162.83 tok/sPrefill · per request | — | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View Reportllama-bench | Llama 3.1 70B (checkpoint unspecified) | Token speed | — | 292.35 tok/sPrefill · per request | — | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View Reportllama-bench | Llama 3.1 70B (checkpoint unspecified) | Token speed | — | 315.7 tok/sPrefill · per request | — | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama 3.1 70B (checkpoint unspecified) | Token speed | — | — | 15.29 tok/sGeneration · per request | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama 3.1 70B (checkpoint unspecified) | Latency16 input · 517 output | 19,943 msLatency · Per request | — | — | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
Quantizations and software
Quantizations: IQ4_XS, Q4_0, Q4_K_M
Engines: llama.cpp, llama.cpp / llama-batched-bench, llama.cpp / llama-bench, llama.cpp / llama-cli, llama.cpp / llama-speculative
This is a selection across sources, not a ranking or a matched experiment. Reading benchmark reports
GPU benchmark pages: W7900 · Ryzen AI Max+ 395