W7700 LLM benchmarks
9 results · 12 measurements · 2 sources
Device specifications
| Variant | Memory | Memory type | Bus | Bandwidth | FP16 matrix | INT8 matrix |
|---|---|---|---|---|---|---|
| W7700 ↗ | 16 GB | GDDR6 | 256-bit | 576 GB/s | 56.6 TFLOPS | 56.6 TOPS |
Manufacturer specifications. Shared system RAM is not dedicated VRAM. Unreported values remain —; measured LLM results retain their reported configuration.
Selected reported tests
9 of 9 resultsSource-reported results, selected across reports by documentation quality. Match configurations before comparing values.
| View ReportGeneric LLM | Llama-3.2-3B-Instruct | Token speed | — | — | 56.14 tok/sGeneration · per request | 1 × W7700Single GPU · 16 GB VRAM/GPUConfiguration
|
|---|---|---|---|---|---|---|
| View ReportGeneric LLM | Llama-2-13b | Token speed | — | — | 8.41 tok/sGeneration · per request | 1 × W7700Single GPU · 16 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama-3.1-8B-Instruct | Token speed | — | — | 39.87 tok/sGeneration · per request | 1 × W7700Single GPU · 16 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama-3.2-3B-Instruct | Token speed | — | — | 89.31 tok/sGeneration · per request | 1 × W7700Single GPU · 16 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama-2-13b | Token speed | — | — | 4.38 tok/sGeneration · per request | 1 × W7700Single GPU · 16 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama-3.1-8B-Instruct | Token speed | — | — | 56.92 tok/sGeneration · per request | 1 × W7700Single GPU · 16 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama-3.2-3B-Instruct | Power & energy | 145 WPower draw · Whole system | — | — | 1 × W7700Single GPU · 16 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama-3.1-8B-Instruct | Power & energy | 52 WPower draw · Whole system | — | — | 1 × W7700Single GPU · 16 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama-2-13b | Power & energy | 108 WPower draw · Whole system | — | — | 1 × W7700Single GPU · 16 GB VRAM/GPUConfiguration
|
Models and software in this collection
Model families: Llama 2, Llama 3.1, Llama 3.2
Engines: llama.cpp
Methodology and evidence standards · Download the research dataset