W7900 LLM benchmarks
175 results · 246 measurements · 20 sources
Device specifications
| Variant | Memory | Memory type | Bus | Bandwidth | FP16 matrix | INT8 matrix |
|---|---|---|---|---|---|---|
| W7900 Dual Slot ↗ | 48 GB | GDDR6 | 384-bit | 864 GB/s | 123 TFLOPS | 123 TOPS |
| W7900 ↗ | 48 GB | GDDR6 | 384-bit | 864 GB/s | 123 TFLOPS | 123 TOPS |
Manufacturer specifications. Shared system RAM is not dedicated VRAM. Unreported values remain —; measured LLM results retain their reported configuration.
Selected reported tests
24 of 175 resultsSource-reported results, selected across reports by documentation quality. Match configurations before comparing values.
| View ReportGeneric LLM | Llama-3-8B-Instruct | LoRA trainingyahma/alpaca-cleaned; packed; revision unreported | 1.9062 lossFinal-step training loss · Cross-entropy on the final training step; not held-out evaluation · Lower is better 920.24 tok/sTraining · per GPU | — | — | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
|---|---|---|---|---|---|---|
| View ReportGeneric LLM | MinerU2.5-Pro-2605-1.2B | OmniDocBenchFull1651 pages · v1.68K configured context | 95.56 %Overall · n = 100 · Official page-level aggregates | — | — | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | HunyuanOCR | OmniDocBenchFull1651 pages · v1.6 | 94.05 %Overall · n = 100 · Official page-level aggregates | — | — | 3 × W7900Independent replicas · 48 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | HunyuanOCR | OmniDocBenchFull1651 pages · v1.61 concurrent request | 93.64 %Overall · n = 100 · Official page-level aggregates | — | — | GPU count unknownUnknown · 48 GB VRAM/GPUConfiguration
|
| View ReportCustom evaluation | gte-Qwen2-1.5B-instruct | SciFact300 queries | 78.57 %Score · n = 1 · nDCG@10 | — | — | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Qwen3.6-35B-A3B-PARO-packed | Token speed | — | 2,879.42 tok/sPrefill · median · per request | 113.26 tok/sGeneration · median · per request | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Qwen3.8-27B | Token speed | — | 868.62 tok/sPrefill · median · per request | 27.89 tok/sGeneration · median · per request | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama-3.1-8B-Instruct | Token speed | — | — | 92.84 tok/sGeneration · per request | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama-3-8B | Token speedbenchmark_throughput.py | 1,380.87 tok/sThroughput · aggregate | — | — | 1 × W7900Single GPUConfiguration
|
| View ReportGeneric LLM | rwkv7 2.9B (checkpoint unspecified) | Token speed | — | 4,315.67 tok/sPrefill · mean · per request | — | 1 × W7900Single GPUConfiguration
|
| View ReportGeneric LLM | rwkv7-g1-2.9b-20250519-ctx4096Conflicting evidence | Token speed | — | 738.94 tok/sPrefill · per request | — | 1 × W7900Single GPUConfiguration
|
| View ReportGeneric LLM | MLPerf Client 1.0 LLM suite (selected models unreported) | MLPerf Client1.0 | — | — | 111.3 tok/sGeneration · unknown | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View Reportllama-bench | Qwen2.5-Coder-32B-Instruct | Token speed | — | 663.44 tok/sPrefill · mean · per request | — | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View Reportllama-bench | Llama 3.1 70B (checkpoint unspecified) | Token speed | 35.74 tok/sThroughput · aggregate | — | — | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View Reportllama-bench | Llama 3 70B (checkpoint unspecified) | Token speed | — | 263.18 tok/sPrefill · per request | — | 2 × W7900Layer split · 48 GB VRAM/GPUConfiguration
|
| View Reportllama-bench | Llama-2-7B | Token speed | — | 3,213.17 tok/sPrefill · mean · per request | — | GPU count unknownUnknown · 48 GB VRAM/GPUConfiguration
|
| View Reportllama-bench | shisa-v2.1-unphi4-14b | Token speed | — | 1,721 tok/sPrefill · per request | — | GPU count unknownUnknown · 48 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | DeepSeek-R1-Distill-Llama-8B | Token speed | 2,634.1 tok/sTotal throughput · mean · aggregate | — | — | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Llama3 70b GPTQ (chart: LLaMA3 70B-Q4) (checkpoint unspecified)Conflicting evidence | Relative performance | 1.38 x vs NVIDIA RTX6000 AdaRelative performance per dollar · Whole system | — | — | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Wendi-2B | LatencySnake short request cache comparison1 concurrent request | 105.7 msLatency · Per request | — | — | GPU count unknownUnknownConfiguration
|
| View ReportGeneric LLM | Llama-3-8B-Instruct | LoRA trainingyahma/alpaca-cleaned; packed; revision unreported | 1.911 lossFinal-step training loss · Cross-entropy on the final training step; not held-out evaluation · Lower is better 769.14 tok/sTraining · per GPU | — | — | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | MinerU2.5-Pro-2605-1.2B | OmniDocBenchFull1651 pages · v1.68K configured context | 0.04 edit distanceText edit distance · Official page-level aggregates · Lower is better | — | — | 1 × W7900Single GPU · 48 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | HunyuanOCR | OmniDocBenchFull1651 pages · v1.6 | 0.04 edit distanceText edit distance · Official page-level aggregates · Lower is better | — | — | 3 × W7900Independent replicas · 48 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | HunyuanOCR | OmniDocBenchFull1651 pages · v1.61 concurrent request | 0.05 edit distanceText edit distance · Official page-level aggregates · Lower is better | — | — | GPU count unknownUnknown · 48 GB VRAM/GPUConfiguration
|
Models and software in this collection
Model families: BGE, DeepSeek MoE, DeepSeek R1, GTE Qwen2, Gemma 3, Granite 4.1, Llama 2, Llama 3, Llama 3.1, MS MARCO MiniLM, Qwen2.5, Qwen3.6, Qwen3.8, RWKV 7, Wendi
Engines: ExLlamaV2, MLC LLM, MLPerf Client, Transformers, VTE, hipEngine, llama.cpp, llama.cpp (hjc4869 fork), llama.cpp / llama-batched-bench, llama.cpp / llama-bench, llama.cpp / llama-cli, llama.cpp / llama-cpp-python, llama.cpp / llama-speculative, llama.cpp via semif-go, torchtune, vLLM, vLLM (AMD fork), web-rwkv
Methodology and evidence standards · Download the research dataset