LAB-Bench results on AMD GPUs

24 results · 2 sources · 24 measurements

Scores from different versions, task subsets, harnesses and scoring rules are not directly comparable. How results are reviewed

Documented results

24 selected tests


View ReportCustom evaluation	Devstral-24BINT4 AWQWeights ↗	LAB-BenchLitQA2:25 sampled questions1 concurrent request	28 %Accuracy · 7 / 25 · Exact-choice parser	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS EndeavourOS (Arch Linux) Engine SGLang Backend ROCm Harness scripts/eval/eval_and_chart.py Prompt / output ? / ? tokens Context 32768 Concurrency 1 Date 2026-04-17 Full record →
View ReportCustom evaluation	Qwen3.8-27BFP8 block (official; mixed BF16 modules)Weights ↗	LAB-BenchLitQA2:25 sampled questions1 concurrent request	56 %Accuracy · 14 / 25 · Exact-choice parser	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine SGLang Backend ROCm Harness scripts/eval/eval_and_chart.py Prompt / output ? / ? tokens Context 262144 Concurrency 1 Date 2026-08-30 Full record →
View ReportCustom evaluation	Devstral-24BINT4 AWQWeights ↗	LAB-BenchDbQA:25 sampled questions1 concurrent request	12 %Accuracy · 3 / 25 · Exact-choice parser	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS EndeavourOS (Arch Linux) Engine SGLang Backend ROCm Harness scripts/eval/eval_and_chart.py Prompt / output ? / ? tokens Context 32768 Concurrency 1 Date 2026-04-17 Full record →
View ReportCustom evaluation	Qwen3.8-27BFP8 block (official; mixed BF16 modules)Weights ↗	LAB-BenchDbQA:25 sampled questions1 concurrent request	44 %Accuracy · 11 / 25 · Exact-choice parser	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine SGLang Backend ROCm Harness scripts/eval/eval_and_chart.py Prompt / output ? / ? tokens Context 262144 Concurrency 1 Date 2026-08-30 Full record →
View ReportCustom evaluation	Devstral-24BINT4 AWQWeights ↗	LAB-BenchSuppQA:25 sampled questions1 concurrent request	44 %Accuracy · 11 / 25 · Exact-choice parser	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS EndeavourOS (Arch Linux) Engine SGLang Backend ROCm Harness scripts/eval/eval_and_chart.py Prompt / output ? / ? tokens Context 32768 Concurrency 1 Date 2026-04-17 Full record →
View ReportCustom evaluation	Qwen3.8-27BFP8 block (official; mixed BF16 modules)Weights ↗	LAB-BenchSuppQA:25 sampled questions1 concurrent request	36 %Accuracy · 9 / 25 · Exact-choice parser	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine SGLang Backend ROCm Harness scripts/eval/eval_and_chart.py Prompt / output ? / ? tokens Context 262144 Concurrency 1 Date 2026-08-30 Full record →
View ReportCustom evaluation	Devstral-24BINT4 AWQWeights ↗	LAB-BenchTableQA:25 sampled questions1 concurrent request	32 %Accuracy · 8 / 25 · Exact-choice parser	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS EndeavourOS (Arch Linux) Engine SGLang Backend ROCm Harness scripts/eval/eval_and_chart.py Prompt / output ? / ? tokens Context 32768 Concurrency 1 Date 2026-04-17 Full record →
View ReportCustom evaluation	Qwen3.8-27BFP8 block (official; mixed BF16 modules)Weights ↗	LAB-BenchTableQA:25 sampled questions1 concurrent request	40 %Accuracy · 10 / 25 · Exact-choice parser	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine SGLang Backend ROCm Harness scripts/eval/eval_and_chart.py Prompt / output ? / ? tokens Context 262144 Concurrency 1 Date 2026-08-30 Full record →
View ReportCustom evaluation	Devstral-24BINT4 AWQWeights ↗	LAB-BenchProtocolQA:25 sampled questions1 concurrent request	32 %Accuracy · 8 / 25 · Exact-choice parser	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS EndeavourOS (Arch Linux) Engine SGLang Backend ROCm Harness scripts/eval/eval_and_chart.py Prompt / output ? / ? tokens Context 32768 Concurrency 1 Date 2026-04-17 Full record →
View ReportCustom evaluation	Qwen3.8-27BFP8 block (official; mixed BF16 modules)Weights ↗	LAB-BenchProtocolQA:25 sampled questions1 concurrent request	32 %Accuracy · 8 / 25 · Exact-choice parser	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine SGLang Backend ROCm Harness scripts/eval/eval_and_chart.py Prompt / output ? / ? tokens Context 262144 Concurrency 1 Date 2026-08-30 Full record →
View ReportCustom evaluation	Devstral-24BINT4 AWQWeights ↗	LAB-BenchSeqQA:25 sampled questions1 concurrent request	24 %Accuracy · 6 / 25 · Exact-choice parser	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS EndeavourOS (Arch Linux) Engine SGLang Backend ROCm Harness scripts/eval/eval_and_chart.py Prompt / output ? / ? tokens Context 32768 Concurrency 1 Date 2026-04-17 Full record →
View ReportCustom evaluation	Qwen3.8-27BFP8 block (official; mixed BF16 modules)Weights ↗	LAB-BenchSeqQA:25 sampled questions1 concurrent request	60 %Accuracy · 15 / 25 · Exact-choice parser	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine SGLang Backend ROCm Harness scripts/eval/eval_and_chart.py Prompt / output ? / ? tokens Context 262144 Concurrency 1 Date 2026-08-30 Full record →
View ReportCustom evaluation	Devstral-24BINT4 AWQWeights ↗	LAB-BenchCloningScenarios:25 sampled questions1 concurrent request	8 %Accuracy · 2 / 25 · Exact-choice parser	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS EndeavourOS (Arch Linux) Engine SGLang Backend ROCm Harness scripts/eval/eval_and_chart.py Prompt / output ? / ? tokens Context 32768 Concurrency 1 Date 2026-04-17 Full record →
View ReportCustom evaluation	Qwen3.8-27BFP8 block (official; mixed BF16 modules)Weights ↗	LAB-BenchCloningScenarios:25 sampled questions1 concurrent request	28 %Accuracy · 7 / 25 · Exact-choice parser	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine SGLang Backend ROCm Harness scripts/eval/eval_and_chart.py Prompt / output ? / ? tokens Context 262144 Concurrency 1 Date 2026-08-30 Full record →
View ReportCustom evaluation	Devstral-24BINT4 AWQWeights ↗	LAB-Bench25 sampled questions from each of7 text-only categories (175 total)1 concurrent request	25.71 %Accuracy · 45 / 175 · Exact-choice parser	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS EndeavourOS (Arch Linux) Engine SGLang Backend ROCm Harness scripts/eval/eval_and_chart.py Prompt / output ? / ? tokens Context 32768 Concurrency 1 Date 2026-04-17 Full record →
View ReportCustom evaluation	Qwen3.8-27BFP8 block (official; mixed BF16 modules)Weights ↗	LAB-Bench25 sampled questions from each of7 text-only categories (175 total)1 concurrent request	42.29 %Accuracy · 74 / 175 · Exact-choice parser	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine SGLang Backend ROCm Harness scripts/eval/eval_and_chart.py Prompt / output ? / ? tokens Context 262144 Concurrency 1 Date 2026-08-30 Full record →
View ReportCustom evaluation	Qwen3-Coder-30B-A3BINT4 AWQWeights ↗	LAB-BenchLitQA2:25 sampled questions1 concurrent request	40 %Accuracy · 10 / 25 · Exact-choice parser	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS EndeavourOS (Arch Linux) Engine SGLang Backend ROCm Harness scripts/eval/eval_and_chart.py Prompt / output ? / ? tokens Context 32768 Concurrency 1 Date 2026-04-17 Full record →
View ReportCustom evaluation	Qwen3-Coder-30B-A3BINT4 AWQWeights ↗	LAB-BenchDbQA:25 sampled questions1 concurrent request	20 %Accuracy · 5 / 25 · Exact-choice parser	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS EndeavourOS (Arch Linux) Engine SGLang Backend ROCm Harness scripts/eval/eval_and_chart.py Prompt / output ? / ? tokens Context 32768 Concurrency 1 Date 2026-04-17 Full record →
View ReportCustom evaluation	Qwen3-Coder-30B-A3BINT4 AWQWeights ↗	LAB-BenchSuppQA:25 sampled questions1 concurrent request	32 %Accuracy · 8 / 25 · Exact-choice parser	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS EndeavourOS (Arch Linux) Engine SGLang Backend ROCm Harness scripts/eval/eval_and_chart.py Prompt / output ? / ? tokens Context 32768 Concurrency 1 Date 2026-04-17 Full record →
View ReportCustom evaluation	Qwen3-Coder-30B-A3BINT4 AWQWeights ↗	LAB-BenchTableQA:25 sampled questions1 concurrent request	48 %Accuracy · 12 / 25 · Exact-choice parser	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS EndeavourOS (Arch Linux) Engine SGLang Backend ROCm Harness scripts/eval/eval_and_chart.py Prompt / output ? / ? tokens Context 32768 Concurrency 1 Date 2026-04-17 Full record →
View ReportCustom evaluation	Qwen3-Coder-30B-A3BINT4 AWQWeights ↗	LAB-BenchProtocolQA:25 sampled questions1 concurrent request	40 %Accuracy · 10 / 25 · Exact-choice parser	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS EndeavourOS (Arch Linux) Engine SGLang Backend ROCm Harness scripts/eval/eval_and_chart.py Prompt / output ? / ? tokens Context 32768 Concurrency 1 Date 2026-04-17 Full record →
View ReportCustom evaluation	Qwen3-Coder-30B-A3BINT4 AWQWeights ↗	LAB-BenchSeqQA:25 sampled questions1 concurrent request	64 %Accuracy · 16 / 25 · Exact-choice parser	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS EndeavourOS (Arch Linux) Engine SGLang Backend ROCm Harness scripts/eval/eval_and_chart.py Prompt / output ? / ? tokens Context 32768 Concurrency 1 Date 2026-04-17 Full record →
View ReportCustom evaluation	Qwen3-Coder-30B-A3BINT4 AWQWeights ↗	LAB-BenchCloningScenarios:25 sampled questions1 concurrent request	24 %Accuracy · 6 / 25 · Exact-choice parser	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS EndeavourOS (Arch Linux) Engine SGLang Backend ROCm Harness scripts/eval/eval_and_chart.py Prompt / output ? / ? tokens Context 32768 Concurrency 1 Date 2026-04-17 Full record →
View ReportCustom evaluation	Qwen3-Coder-30B-A3BINT4 AWQWeights ↗	LAB-Bench25 sampled questions from each of7 text-only categories (175 total)1 concurrent request	38.29 %Accuracy · 67 / 175 · Exact-choice parser	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS EndeavourOS (Arch Linux) Engine SGLang Backend ROCm Harness scripts/eval/eval_and_chart.py Prompt / output ? / ? tokens Context 32768 Concurrency 1 Date 2026-04-17 Full record →

Versions and harnesses

Versions: Not reported
Harnesses: scripts/eval/eval_and_chart.py

Explore all 24 results →

This is a selection across sources, not a ranking or a matched experiment. Reading benchmark reports

GPU benchmark pages: R9700