Scores from different versions, task subsets, harnesses and scoring rules are not directly comparable. How results are reviewed
Documented results
24 selected tests| View ReportGeneric LLM | Qwen3.6-35B-A3B | HumanEvalFull 164 problems | 59.15 %pass@1 · 97 / 164 · Executable tests 38 tok/sThroughput · aggregate | — | — | 1 × RX 9070Single GPU · 16 GB VRAM/GPUConfiguration
|
|---|---|---|---|---|---|---|
| View ReportGeneric LLM | Qwen3.6-27B | HumanEvalFull164problems | 78.05 %pass@1 · 128 / 164 · Executable tests 16.8 tok/sThroughput · aggregate | — | — | 1 × RX 9070Single GPU · 16 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Qwen3.8-27BConflicting evidence | HumanEval100 problems | 98 %Reported pass@1 · n = 100 | — | 42.1 tok/sGeneration · median · per request | 1 × W7800Single GPU · 48 GB VRAM/GPUConfiguration
|
| View ReportCustom evaluation | Devstral-2-123B-Instruct-2512Conflicting evidence | HumanEval100 problems | 88 %Reported pass@1 · n = 100 | — | 7.05 tok/sGeneration · mean · per request | 2 × W7800Layer split · 48 GB VRAM/GPUConfiguration
|
| View ReportCustom evaluation | Muse-Glimmer-30BConflicting evidence | HumanEval100 problems | 97 %Reported pass@1 · n = 100 | — | 27.51 tok/sGeneration · mean · per request | 1 × W7800Single GPU · 48 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Qwen3.5-9B | HumanEvalFull 164 problems | 67.07 %pass@1 · n = 164 · Executable tests | — | 37.98 tok/sGeneration · mean · per request | 1 × RX 9070Single GPU · 16 GB VRAM/GPUConfiguration
|
| View ReportCustom evaluation | Qwen3.8-27B | HumanEval40 problems;every fourthHumanEval index · HumanEvalPlus; revision not reported1 concurrent request | 90 %pass@1 · 36 / 40 · Executable tests | — | — | 1 × Ryzen AI Max+ 395Single GPU · Unified memory · 128 GB RAMConfiguration
|
| View ReportCustom evaluation | Devstral-24B | HumanEvalopenai/openai_humaneval test: first30 tasks1 concurrent request | 73.33 %pass@1 · 22 / 30 · Executable tests | — | — | 2 × R9700Tensor parallel · 32 GB VRAM/GPUConfiguration
|
| View ReportCustom evaluation | Qwen3.8-27B | HumanEvalopenai/openai_humaneval test: first30 tasks1 concurrent request | 93.33 %pass@1 · 28 / 30 · Executable tests | — | — | 2 × R9700Tensor parallel · 32 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Gemma4-26B-CIRU-AMD-STRIX-vLLM | HumanEval100 tasks | 95.73 %pass@1 · n = 100 · Accuracy | — | — | GPU count unknownUnknown · Unified memory · 128 GB RAMConfiguration
|
| View ReportGeneric LLM | Qwen3.8-Flash-Next uncensored fine-tune (exact checkpoint unreported) | HumanEval164 problems | 87.8 %Reported pass@1 · n = 164 | — | — | 3 × R9700Tensor parallel · 32 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Qwen3.8-27BConflicting evidence | HumanEval100 problems | 99 %Reported pass@1 · n = 100 | — | 25.5 tok/sGeneration · median · per request | 1 × W7800Single GPU · 48 GB VRAM/GPUConfiguration
|
| View ReportCustom evaluation | Devstral-Small-2-24B-Instruct-2512Conflicting evidence | HumanEval100 problems | 88 %Reported pass@1 · n = 100 | — | 31.57 tok/sGeneration · mean · per request | 2 × W7800Layer split · 48 GB VRAM/GPUConfiguration
|
| View ReportCustom evaluation | Qwen3-Coder-30B-A3B-InstructConflicting evidence | HumanEval100 problems | 99 %Reported pass@1 · n = 100 | — | 96.97 tok/sGeneration · mean · per request | 1 × W7800Single GPU · 48 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Qwen3.5-9B | HumanEvalFull 164 problems | 67.07 %pass@1 · n = 164 · Executable tests | — | 39.8 tok/sGeneration · mean · per request | 1 × RX 9070Single GPU · 16 GB VRAM/GPUConfiguration
|
| View ReportCustom evaluation | Qwen3.8-27B | HumanEval40 problems;every fourthHumanEval index · HumanEvalPlus; revision not reported1 concurrent request | 87.5 %pass@1 · 35 / 40 · Executable tests | — | — | 1 × Ryzen AI Max+ 395Single GPU · Unified memory · 128 GB RAMConfiguration
|
| View ReportCustom evaluation | Qwen3-Coder-30B-A3B | HumanEvalopenai/openai_humaneval test: first30 tasks1 concurrent request | 96.67 %pass@1 · 29 / 30 · Executable tests | — | — | 2 × R9700Tensor parallel · 32 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Qwen3.8-Flash-Next uncensored fine-tune (exact checkpoint unreported) | HumanEval164 problems | 88.4 %Reported pass@1 · n = 164 | — | — | 2 × R9700Tensor parallel · 32 GB VRAM/GPUConfiguration
|
| View ReportCustom evaluation | gemma-4-26B-A4B-itConflicting evidence | HumanEval100 problems | 96 %Reported pass@1 · n = 100 | — | 71.18 tok/sGeneration · mean · per request | 2 × W7800Layer split · 48 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Qwen3.5-9B | HumanEvalFull 164 problems | 66.46 %pass@1 · n = 164 · Executable tests | — | 38.76 tok/sGeneration · mean · per request | 1 × RX 9070Single GPU · 16 GB VRAM/GPUConfiguration
|
| View ReportCustom evaluation | Qwen3.8-27B | HumanEval40 problems;every fourthHumanEval index · HumanEvalPlus; revision not reported1 concurrent request | 92.5 %pass@1 · 37 / 40 · Executable tests | — | — | 1 × Ryzen AI Max+ 395Single GPU · Unified memory · 128 GB RAMConfiguration
|
| View ReportCustom evaluation | Qwen3.5-27B | HumanEvalopenai/openai_humaneval test: first30 tasks1 concurrent request | 70 %pass@1 · 21 / 30 · Executable tests | — | — | 2 × R9700Tensor parallel · 32 GB VRAM/GPUConfiguration
|
| View ReportGeneric LLM | Qwen3.8-Flash-Next uncensored fine-tune (exact checkpoint unreported) | HumanEval164 problems | 84.8 %Reported pass@1 · n = 164 | — | — | 3 × R9700Tensor parallel · 32 GB VRAM/GPUConfiguration
|
| View ReportCustom evaluation | GLM-4.5-AirConflicting evidence | HumanEval100 problems | 63 %Reported pass@1 · n = 100 | — | 37.33 tok/sGeneration · mean · per request | 2 × W7800Layer split · 48 GB VRAM/GPUConfiguration
|
Versions and harnesses
Versions: Not reported
Harnesses: BoJl unified evaluation harness (2026-08), Ciru native deterministic runner, custom OpenAI API HumanEval runner, custom OpenAI API HumanEval runner + human-eval, harness/codeeval.py + HumanEvalPlus dataset, run_humaneval_openai_api_v2.py + human_eval, scripts/eval/eval_and_chart.py
This is a selection across sources, not a ranking or a matched experiment. Reading benchmark reports
GPU benchmark pages: W7800 · R9700 · RX 9070 · Ryzen AI Max+ 395