GSM8K results on AMD GPUs

37 results · 8 sources · 45 measurements

Scores from different versions, task subsets, harnesses and scoring rules are not directly comparable. How results are reviewed

Documented results

24 selected tests


View ReportCustom evaluation	Qwen3.8-Flash-Next-GSQ-RCOIQ3_XXSWeights ↗	GSM8K200 questions; seed42	95 %Accuracy · 190 / 200 · Reported correctness; exact extraction/scorer not published 50.68 tok/sThroughput · aggregate	—	—	1 × W7800Single GPU · 48 GB VRAM/GPU Configuration Memory 48 GB VRAM/GPU OS Linux Engine Strata Backend ROCm / HIP Harness GSM8K evaluation script (not named) Prompt / output ? / ? tokens Context Not reported Concurrency 4 Date 2026-10-02 Full record →
View ReportGeneric LLM	Qwen3.8-27BCustom mixed 2.72 BPW (pareto-bf16)Model ref. ↗	GSM8K30 problems with199.7K resident context1 concurrent request	96.7 %Reported accuracy · n = 100	—	—	1 × RX 9070 XTSingle GPU · 16 GB VRAM/GPU Configuration Memory 16 GB VRAM/GPU OS Windows Engine llama.cpp Backend Vulkan Harness Custom evaluation runner Prompt / output ? / ? tokens Context 262144 Concurrency 1 Date Not reported Full record →
View ReportGeneric LLM	Qwen3.8-27BINT4 GPTQ v3Model ref. ↗	GSM8KSize not reported128K configured context	83.7 %Accuracy · Flexible extraction · Mean	—	—	1 × W7800Single GPU · 48 GB VRAM/GPU Configuration Memory 48 GB VRAM/GPU OS Ubuntu 24.04 Engine SGLang Backend ROCm 7.2.4 / Triton attention Harness Not reported Prompt / output ? / ? tokens Context 131072 Concurrency Not reported Date 2026-09-15 Full record →
View ReportCustom evaluation	Qwen3.8-27B-NVFP4NVFP4 converted to MXFP4 W4A8Model ref. ↗	GSM8K500 questions8 concurrent requests	97.6 %Accuracy · 488 / 500 · Reported correctness	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Linux (inferred; host distribution unreported) (inferred) Engine vLLM Radiance Backend ROCm / HIP Harness GSM8K evaluation (client unreported) Prompt / output ? / ? tokens Context Not reported Concurrency 8 Date 2026-09-15 Full record →
View ReportCustom evaluation	gemma-4-12B-itFP8Weights ↗	GSM8KFull1319questions;5shot8K configured context	94.54 %Accuracy · n = 100 · Original GSM8K tagged answer extraction and strict string comparison	—	—	1 × R9700Single GPU · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine vLLM Backend ROCm / HIP Harness vLLM GSM8K with original tagged-answer matcher Prompt / output ? / ? tokens Context 8192 Concurrency Not reported Date Not reported Full record →
View ReportGeneric LLM	Gemma4-26B-CIRU-AMD-STRIX-vLLMW4A16 INT4 group64Weights ↗	GSM8KSize not reported	95.45 %Accuracy · n = 100	—	—	GPU count unknownUnknown · Unified memory · 128 GB RAM Configuration Memory Unified memory · 128 GB RAM OS Not reported Engine vLLM (Ciru modified) Backend ROCm / AITER Harness Ciru native deterministic runner Prompt / output ? / ? tokens Context Not reported Concurrency Not reported Date Not reported Full record →
View ReportCustom evaluation	Ornith-1.5-35B-A3B-MXFP4-Quark-RDNA4MXFP4 weight-only (Quark; BF16 activations)Weights ↗	GSM8K50 prompts per seed; flexible extraction	90 %Accuracy · flexible extraction · Median	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine vLLM (Capicua25x RDNA4 port) Backend ROCm / HIP Harness lm-eval Prompt / output ? / ? tokens Context Not reported Concurrency Not reported Date Not reported Full record →
View ReportGeneric LLM	Qwen3.8-Flash-Next uncensored fine-tune (exact checkpoint unreported)Quant not reportedModel ref. ↗	GSM8K200-question subset	92.5 %Reported accuracy · n = 200	—	—	3 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine Radiance standalone Backend HIP Harness Not reported Prompt / output ? / ? tokens Context Not reported Concurrency Not reported Date Not reported Full record →
View ReportGeneric LLM	Qwen3.8-27BINT4 GPTQ v3Model ref. ↗	GSM8KSize not reported128K configured context	84.12 %Accuracy · Strict extraction · Mean	—	—	1 × W7800Single GPU · 48 GB VRAM/GPU Configuration Memory 48 GB VRAM/GPU OS Ubuntu 24.04 Engine SGLang Backend ROCm 7.2.4 / Triton attention Harness Not reported Prompt / output ? / ? tokens Context 131072 Concurrency Not reported Date 2026-09-15 Full record →
View ReportCustom evaluation	Qwen3.8-27BMXFP6 E2M3 W6A8Model ref. ↗	GSM8KFirst 500 test questions; two few-shot seeds256K configured context	97.5 %Accuracy · n = 100 · Flexible exact match · Mean	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine vLLM Radiance Backend ROCm / HIP Harness lm-eval Prompt / output ? / ? tokens Context 262144 Concurrency Not reported Date 2026-09-17 Full record →
View ReportCustom evaluation	gemma-4-12B-itFP8Weights ↗	GSM8KFull1319questions;5shot8K configured context	94.62 %Accuracy · n = 100 · Original GSM8K tagged answer extraction and strict string comparison	—	—	1 × R9700Single GPU · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine vLLM Backend ROCm / HIP Harness vLLM GSM8K with original tagged-answer matcher Prompt / output ? / ? tokens Context 8192 Concurrency Not reported Date Not reported Full record →
View ReportGeneric LLM	Gemma4-26B-CIRU-AMD-STRIX-vLLMW4A16 INT4 group64Weights ↗	GSM8KSize not reported	96.5 %Accuracy · n = 100	—	—	GPU count unknownUnknown · Unified memory · 128 GB RAM Configuration Memory Unified memory · 128 GB RAM OS Not reported Engine vLLM (Ciru modified) Backend ROCm / AITER Harness EvalScope1.12.0 Prompt / output ? / ? tokens Context Not reported Concurrency Not reported Date Not reported Full record →
View ReportCustom evaluation	Ornith-1.5-35B-A3B-MXFP4-Quark-RDNA4MXFP4 weight-only (Quark; BF16 activations)Weights ↗	GSM8K50 prompts per seed; flexible extraction	88 %Accuracy · flexible extraction · Median	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine vLLM (Capicua25x RDNA4 port) Backend ROCm / HIP Harness lm-eval Prompt / output ? / ? tokens Context Not reported Concurrency Not reported Date Not reported Full record →
View ReportGeneric LLM	Qwen3.8-Flash-Next uncensored fine-tune (exact checkpoint unreported)Quant not reportedModel ref. ↗	GSM8K200-question subset	91.5 %Reported accuracy · n = 200	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine Radiance standalone Backend HIP Harness Not reported Prompt / output ? / ? tokens Context Not reported Concurrency Not reported Date Not reported Full record →
View ReportCustom evaluation	Qwen3.8-27BMXFP6 E2M3 W6A8Model ref. ↗	GSM8KFirst 500 test questions; two few-shot seeds256K configured context	96.6 %Accuracy · n = 100 · Strict exact match · Mean	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine vLLM Radiance Backend ROCm / HIP Harness lm-eval Prompt / output ? / ? tokens Context 262144 Concurrency Not reported Date 2026-09-17 Full record →
View ReportCustom evaluation	gemma-4-26B-A4B-itFP8Weights ↗	GSM8KFull1319questions;5shot8K configured context	95.83 %Accuracy · n = 100 · Original GSM8K tagged answer extraction and strict string comparison	—	—	1 × R9700Single GPU · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine vLLM Backend ROCm / HIP Harness vLLM GSM8K with original tagged-answer matcher Prompt / output ? / ? tokens Context 8192 Concurrency Not reported Date Not reported Full record →
View ReportCustom evaluation	Ornith-1.5-35B-A3B-MXFP4-Quark-RDNA4MXFP4 weight-only (Quark; BF16 activations)Weights ↗	GSM8K50 prompts per seed; strict extraction	84 %Accuracy · strict extraction · Median	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine vLLM (Capicua25x RDNA4 port) Backend ROCm / HIP Harness lm-eval Prompt / output ? / ? tokens Context Not reported Concurrency Not reported Date Not reported Full record →
View ReportGeneric LLM	Qwen3.8-Flash-Next uncensored fine-tune (exact checkpoint unreported)Quant not reportedModel ref. ↗	GSM8K200-question subset	93.5 %Reported accuracy · n = 200	—	—	3 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine Radiance standalone Backend HIP Harness Not reported Prompt / output ? / ? tokens Context Not reported Concurrency Not reported Date Not reported Full record →
View ReportCustom evaluation	Qwen3.8-27BINT5 ParoQuant W5A8Model ref. ↗	GSM8K500 questions	97.2 %Accuracy · n = 100 · Reported correctness	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Linux (inferred; host distribution unreported) (inferred) Engine vLLM Radiance Backend ROCm / HIP Harness GSM8K evaluation (client unreported) Prompt / output ? / ? tokens Context Not reported Concurrency Not reported Date 2026-09-11 Full record →
View ReportCustom evaluation	gemma-4-26B-A4B-itFP8Weights ↗	GSM8KFull1319questions;5shot8K configured context	95.91 %Accuracy · n = 100 · Original GSM8K tagged answer extraction and strict string comparison	—	—	1 × R9700Single GPU · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine vLLM Backend ROCm / HIP Harness vLLM GSM8K with original tagged-answer matcher Prompt / output ? / ? tokens Context 8192 Concurrency Not reported Date Not reported Full record →
View ReportCustom evaluation	Ornith-1.5-35B-A3B-MXFP4-Quark-RDNA4MXFP4 weight-only (Quark; BF16 activations)Weights ↗	GSM8K50 prompts per seed; strict extraction	84 %Accuracy · strict extraction · Median	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine vLLM (Capicua25x RDNA4 port) Backend ROCm / HIP Harness lm-eval Prompt / output ? / ? tokens Context Not reported Concurrency Not reported Date Not reported Full record →
View ReportCustom evaluation	Qwen3.8-27BINT5 ParoQuant W5A8Model ref. ↗	GSM8K500 questions	96.8 %Accuracy · n = 100 · Reported correctness	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Linux (inferred; host distribution unreported) (inferred) Engine vLLM Radiance Backend ROCm / HIP Harness GSM8K evaluation (client unreported) Prompt / output ? / ? tokens Context Not reported Concurrency Not reported Date 2026-09-12 Full record →
View ReportCustom evaluation	Ornith-1.5-35B-A3BFP8Weights ↗	GSM8K50 prompts; flexible extraction	90 %Accuracy · flexible extraction	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine vLLM (Capicua25x RDNA4 port) Backend ROCm / HIP Harness lm-eval Prompt / output ? / ? tokens Context Not reported Concurrency Not reported Date Not reported Full record →
View ReportCustom evaluation	Qwen3.8-27BINT5 ParoQuant W5A8Model ref. ↗	GSM8K500 questions	97.8 %Accuracy · n = 100 · Reported correctness	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Linux (inferred; host distribution unreported) (inferred) Engine vLLM Radiance Backend ROCm / HIP Harness GSM8K evaluation (client unreported) Prompt / output ? / ? tokens Context Not reported Concurrency Not reported Date 2026-09-12 Full record →

Versions and harnesses

Versions: Not reported
Harnesses: Ciru native deterministic runner, Custom evaluation runner, EvalScope1.12.0, GSM8K evaluation (client unreported), GSM8K evaluation script (not named), lm-eval, vLLM GSM8K with original tagged-answer matcher

Explore all 37 results →

This is a selection across sources, not a ranking or a matched experiment. Reading benchmark reports

GPU benchmark pages: W7800 · R9700 · RX 9070 XT · Ryzen AI Max+ 395