IFEval results on AMD GPUs

15 results · 5 sources · 31 measurements

Scores from different versions, task subsets, harnesses and scoring rules are not directly comparable. How results are reviewed

Documented results

15 selected tests


View ReportCustom evaluation	Qwen3.8-Flash-NextUD-Q3_K_XLModel ref. ↗	IFEvalLLST protocol 1.0 subset · 20 pinned LLST v1.0 questions; not the full IFEval benchmark32K configured context	95 %Score · n = 100 · Source aggregate; exact strict/loose and prompt/instruction submetric unspecified	—	—	2 × RX 7900 XTXMulti-GPU · split unknown · 24 GB VRAM/GPU Configuration Memory 24 GB VRAM/GPU OS Ubuntu 24.04 LTS Engine llama.cpp Backend ROCm / HIP Harness Local LLM Standard Test (LLST) 1.0 / EvalScope Prompt / output ? / ? tokens Context 32768 Concurrency Not reported Date 2026-09-17 Full record →
View ReportGeneric LLM	gemma-4-26B-A4B-it-MXFP4-Quark-RDNA4MXFP4 (Quark; mixed BF16)Weights ↗	IFEval80 prompts; instruction-strict	92.97 %Score · instruction-strict	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine vLLM (Capicua25x RDNA4 port) Backend ROCm / HIP Harness Not reported Prompt / output ? / ? tokens Context Not reported Concurrency Not reported Date 2026-09-03 Full record →
View ReportCustom evaluation	Muse-Glimmer-30B-MXFP4-Quark-RDNA4MXFP4 (Quark; mixed BF16)Weights ↗	IFEval80 prompts per seed; instruction-strict1024K configured context	90.62 %Score · instruction-strict · Median	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine vLLM (Capicua25x RDNA4 port) Backend ROCm / HIP Harness lm-eval Prompt / output ? / ? tokens Context 1048576 Concurrency Not reported Date Not reported Full record →
View ReportGeneric LLM	Gemma4-26B-CIRU-AMD-STRIX-vLLMW4A16 INT4 group64Weights ↗	IFEval100 tasks	87.62 %Score · n = 100 · Prompt-level strict	—	—	GPU count unknownUnknown · Unified memory · 128 GB RAM Configuration Memory Unified memory · 128 GB RAM OS Not reported Engine vLLM (Ciru modified) Backend ROCm / AITER Harness Ciru native deterministic runner Prompt / output ? / ? tokens Context Not reported Concurrency Not reported Date Not reported Full record →
View ReportCustom evaluation	Ornith-1.5-35B-A3B-MXFP4-Quark-RDNA4MXFP4 weight-only (Quark; BF16 activations)Weights ↗	IFEval80 prompts per seed; instruction-strict	85.9 %Score · instruction-strict · Median	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine vLLM (Capicua25x RDNA4 port) Backend ROCm / HIP Harness lm-eval Prompt / output ? / ? tokens Context Not reported Concurrency Not reported Date Not reported Full record →
View ReportGeneric LLM	gemma-4-26B-A4B-it-MXFP4-Quark-RDNA4MXFP4 (Quark; mixed BF16)Weights ↗	IFEval80 prompts; prompt-strict	88.75 %Score · prompt-strict	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine vLLM (Capicua25x RDNA4 port) Backend ROCm / HIP Harness Not reported Prompt / output ? / ? tokens Context Not reported Concurrency Not reported Date 2026-09-03 Full record →
View ReportCustom evaluation	Muse-Glimmer-30B-MXFP4-Quark-RDNA4MXFP4 (Quark; mixed BF16)Weights ↗	IFEval80 prompts per seed; instruction-strict128K configured context	92.97 %Score · instruction-strict · Median	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine vLLM (Capicua25x RDNA4 port) Backend ROCm / HIP Harness lm-eval Prompt / output ? / ? tokens Context 131072 Concurrency Not reported Date Not reported Full record →
View ReportGeneric LLM	Gemma4-26B-CIRU-AMD-STRIX-vLLMW4A16 INT4 group64Weights ↗	IFEval100 tasks	87.99 %Score · n = 100 · Prompt-level strict	—	—	GPU count unknownUnknown · Unified memory · 128 GB RAM Configuration Memory Unified memory · 128 GB RAM OS Not reported Engine vLLM (Ciru modified) Backend ROCm / AITER Harness EvalScope1.12.0 Prompt / output ? / ? tokens Context Not reported Concurrency Not reported Date Not reported Full record →
View ReportCustom evaluation	Ornith-1.5-35B-A3B-MXFP4-Quark-RDNA4MXFP4 weight-only (Quark; BF16 activations)Weights ↗	IFEval80 prompts per seed; instruction-strict	87.5 %Score · instruction-strict · Median	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine vLLM (Capicua25x RDNA4 port) Backend ROCm / HIP Harness lm-eval Prompt / output ? / ? tokens Context Not reported Concurrency Not reported Date Not reported Full record →
View ReportCustom evaluation	Muse-Glimmer-30B-MXFP4-Quark-RDNA4MXFP4 (Quark; mixed BF16)Weights ↗	IFEval80 prompts per seed; prompt-strict1024K configured context	86.25 %Score · prompt-strict · Median	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine vLLM (Capicua25x RDNA4 port) Backend ROCm / HIP Harness lm-eval Prompt / output ? / ? tokens Context 1048576 Concurrency Not reported Date Not reported Full record →
View ReportCustom evaluation	Ornith-1.5-35B-A3B-MXFP4-Quark-RDNA4MXFP4 weight-only (Quark; BF16 activations)Weights ↗	IFEval80 prompts per seed; prompt-strict	80 %Score · prompt-strict · Median	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine vLLM (Capicua25x RDNA4 port) Backend ROCm / HIP Harness lm-eval Prompt / output ? / ? tokens Context Not reported Concurrency Not reported Date Not reported Full record →
View ReportCustom evaluation	Muse-Glimmer-30B-MXFP4-Quark-RDNA4MXFP4 (Quark; mixed BF16)Weights ↗	IFEval80 prompts per seed; prompt-strict128K configured context	90 %Score · prompt-strict · Median	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine vLLM (Capicua25x RDNA4 port) Backend ROCm / HIP Harness lm-eval Prompt / output ? / ? tokens Context 131072 Concurrency Not reported Date Not reported Full record →
View ReportCustom evaluation	Ornith-1.5-35B-A3B-MXFP4-Quark-RDNA4MXFP4 weight-only (Quark; BF16 activations)Weights ↗	IFEval80 prompts per seed; prompt-strict	80 %Score · prompt-strict · Median	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine vLLM (Capicua25x RDNA4 port) Backend ROCm / HIP Harness lm-eval Prompt / output ? / ? tokens Context Not reported Concurrency Not reported Date Not reported Full record →
View ReportCustom evaluation	Ornith-1.5-35B-A3BFP8Weights ↗	IFEval80 prompts; instruction-strict	86.7 %Score · instruction-strict	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine vLLM (Capicua25x RDNA4 port) Backend ROCm / HIP Harness lm-eval Prompt / output ? / ? tokens Context Not reported Concurrency Not reported Date Not reported Full record →
View ReportCustom evaluation	Ornith-1.5-35B-A3BFP8Weights ↗	IFEval80 prompts; prompt-strict	81.3 %Score · prompt-strict	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Not reported Engine vLLM (Capicua25x RDNA4 port) Backend ROCm / HIP Harness lm-eval Prompt / output ? / ? tokens Context Not reported Concurrency Not reported Date Not reported Full record →

Versions and harnesses

Versions: LLST protocol 1.0 subset
Harnesses: Ciru native deterministic runner, EvalScope1.12.0, lm-eval, Local LLM Standard Test (LLST) 1.0 / EvalScope

Explore all 15 results →

This is a selection across sources, not a ranking or a matched experiment. Reading benchmark reports

GPU benchmark pages: R9700 · RX 7900 XTX · Ryzen AI Max+ 395