Mistral-Small-24B-Instruct-2501 on R9700

4 results · 2 sources · 16 measurements

Grouped by reported model identity. Exact weights, quantizations and software can differ. How results are reviewed

Documented results

4 selected tests


View ReportLocalMaxxing	Mistral-Small-24B-Instruct-2501Q4_K_MModel ref. ↗	Token speed	—	551.68 tok/sPrefill · median · unknown	38.39 tok/sGeneration · median · unknown	1 × R9700Single GPU · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS linux Engine llama.cpp Backend rocm Harness LocalMaxxing community speed test; exact client where reported Prompt / output 41 / 86 tokens Context 16384 Concurrency Not reported Date 2026-09-21 Full record →
View ReportGeneric LLM	Mistral-Small-24B-Instruct-2501BF16Weights ↗	Token speedNative Windows offline latency / mistral-bf16	~10.46 tok/sThroughput · per request	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Windows 11 Engine vLLM_for_AMD Backend ROCm / HIP Harness Not reported Prompt / output 32 / 16 tokens Context 512 Concurrency Not reported Date Not reported Full record →
View ReportLocalMaxxing	Mistral-Small-24B-Instruct-2501Q4_K_MModel ref. ↗	Token speed	—	544.15 tok/sPrefill · median · unknown	38.63 tok/sGeneration · median · unknown	1 × R9700Single GPU · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS linux Engine llama.cpp Backend rocm Harness LocalMaxxing community speed test; exact client where reported Prompt / output 41 / 79 tokens Context 16384 Concurrency Not reported Date 2026-09-28 Full record →
View ReportGeneric LLM	Mistral-Small-24B-Instruct-2501BF16Weights ↗	Token speedNative Windows offline latency / mistral-bf16	~6.17 tok/sThroughput · per request	—	—	2 × R9700Tensor parallel · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Windows 11 Engine vLLM_for_AMD Backend ROCm / HIP Harness Not reported Prompt / output 32 / 16 tokens Context 512 Concurrency Not reported Date Not reported Full record →

Quantizations and software

Quantizations: BF16, Q4_K_M
Engines: llama.cpp, vLLM_for_AMD

Explore all 4 results →

This is a selection across sources, not a ranking or a matched experiment. Reading benchmark reports

GPU benchmark pages: R9700