Mistral-Small-3.2-24B-Instruct-2506 on AMD GPUs

42 results · 2 sources · 42 measurements

Grouped by reported model identity. Exact weights, quantizations and software can differ. How results are reviewed

Documented results

24 selected tests


View Reportllama-bench	Mistral-Small-3.2-24B-Instruct-2506Q8_0Model ref. ↗	Token speed	—	1,423.41 tok/sPrefill · per request	—	1 × R9700Single GPU · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Arch Linux Engine llama.cpp Backend ROCm Harness llama-batched-bench Prompt / output 1024 / 512 tokens Context 32798 Concurrency 1 Date Not reported Full record →
View ReportGeneric LLM	Mistral-Small-3.2-24B-Instruct-2506Q4_0_SWeights ↗	Token speed	43.3 tok/sThroughput · aggregate	—	—	2 × RX 7900 XTXLayer split · 24 GB VRAM/GPU Configuration Memory 24 GB VRAM/GPU OS Not reported Engine llama.cpp Backend ROCm 6.4.4 Harness Custom serving benchmark (not named) Prompt / output ? / ? tokens Context 131072 Concurrency 1 Date Not reported Full record →
View Reportllama-bench	Mistral-Small-3.2-24B-Instruct-2506Q8_0Model ref. ↗	Token speed	—	—	23.39 tok/sGeneration · per request	1 × R9700Single GPU · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Arch Linux Engine llama.cpp Backend ROCm Harness llama-batched-bench Prompt / output 1024 / 512 tokens Context 32798 Concurrency 1 Date Not reported Full record →
View ReportGeneric LLM	Mistral-Small-3.2-24B-Instruct-2506Q4_0_SWeights ↗	Token speed	32.7 tok/sThroughput · mean · per request	—	—	2 × RX 7900 XTXLayer split · 24 GB VRAM/GPU Configuration Memory 24 GB VRAM/GPU OS Not reported Engine llama.cpp Backend ROCm 6.4.4 Harness Custom serving benchmark (not named) Prompt / output ? / ? tokens Context 131072 Concurrency 1 Date Not reported Full record →
View Reportllama-bench	Mistral-Small-3.2-24B-Instruct-2506Q8_0Model ref. ↗	Token speed	67.93 tok/sInput + output · per request	—	—	1 × R9700Single GPU · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Arch Linux Engine llama.cpp Backend ROCm Harness llama-batched-bench Prompt / output 1024 / 512 tokens Context 32798 Concurrency 1 Date Not reported Full record →
View ReportGeneric LLM	Mistral-Small-3.2-24B-Instruct-2506Q4_0_SWeights ↗	Token speed	165.6 tok/sThroughput · aggregate	—	—	2 × RX 7900 XTXLayer split · 24 GB VRAM/GPU Configuration Memory 24 GB VRAM/GPU OS Not reported Engine llama.cpp Backend ROCm 6.4.4 Harness Custom serving benchmark (not named) Prompt / output ? / ? tokens Context 196608 Concurrency 8 Date Not reported Full record →
View Reportllama-bench	Mistral-Small-3.2-24B-Instruct-2506Q8_0Model ref. ↗	Token speed	—	1,516.62 tok/sPrefill · aggregate	—	1 × R9700Single GPU · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Arch Linux Engine llama.cpp Backend ROCm Harness llama-batched-bench Prompt / output 1024 / 512 tokens Context 32798 Concurrency 2 Date Not reported Full record →
View ReportGeneric LLM	Mistral-Small-3.2-24B-Instruct-2506Q4_0_SWeights ↗	Token speed	18.1 tok/sThroughput · mean · per request	—	—	2 × RX 7900 XTXLayer split · 24 GB VRAM/GPU Configuration Memory 24 GB VRAM/GPU OS Not reported Engine llama.cpp Backend ROCm 6.4.4 Harness Custom serving benchmark (not named) Prompt / output ? / ? tokens Context 196608 Concurrency 8 Date Not reported Full record →
View Reportllama-bench	Mistral-Small-3.2-24B-Instruct-2506Q8_0Model ref. ↗	Token speed	42.33 tok/sThroughput · aggregate	—	—	1 × R9700Single GPU · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Arch Linux Engine llama.cpp Backend ROCm Harness llama-batched-bench Prompt / output 1024 / 512 tokens Context 32798 Concurrency 2 Date Not reported Full record →
View ReportGeneric LLM	Mistral-Small-3.2-24B-Instruct-2506Q4_0_MWeights ↗	Token speed	38.7 tok/sThroughput · aggregate	—	—	2 × RX 7900 XTXLayer split · 24 GB VRAM/GPU Configuration Memory 24 GB VRAM/GPU OS Not reported Engine llama.cpp Backend ROCm 6.4.4 Harness Custom serving benchmark (not named) Prompt / output ? / ? tokens Context 131072 Concurrency 1 Date Not reported Full record →
View Reportllama-bench	Mistral-Small-3.2-24B-Instruct-2506Q8_0Model ref. ↗	Token speed	120.27 tok/sInput + output · aggregate	—	—	1 × R9700Single GPU · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Arch Linux Engine llama.cpp Backend ROCm Harness llama-batched-bench Prompt / output 1024 / 512 tokens Context 32798 Concurrency 2 Date Not reported Full record →
View ReportGeneric LLM	Mistral-Small-3.2-24B-Instruct-2506Q4_0_MWeights ↗	Token speed	31.2 tok/sThroughput · mean · per request	—	—	2 × RX 7900 XTXLayer split · 24 GB VRAM/GPU Configuration Memory 24 GB VRAM/GPU OS Not reported Engine llama.cpp Backend ROCm 6.4.4 Harness Custom serving benchmark (not named) Prompt / output ? / ? tokens Context 131072 Concurrency 1 Date Not reported Full record →
View Reportllama-bench	Mistral-Small-3.2-24B-Instruct-2506Q8_0Model ref. ↗	Token speed	—	1,501.73 tok/sPrefill · aggregate	—	1 × R9700Single GPU · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Arch Linux Engine llama.cpp Backend ROCm Harness llama-batched-bench Prompt / output 1024 / 512 tokens Context 32798 Concurrency 4 Date Not reported Full record →
View ReportGeneric LLM	Mistral-Small-3.2-24B-Instruct-2506Q4_0_MWeights ↗	Token speed	152.3 tok/sThroughput · aggregate	—	—	2 × RX 7900 XTXLayer split · 24 GB VRAM/GPU Configuration Memory 24 GB VRAM/GPU OS Not reported Engine llama.cpp Backend ROCm 6.4.4 Harness Custom serving benchmark (not named) Prompt / output ? / ? tokens Context 196608 Concurrency 8 Date Not reported Full record →
View Reportllama-bench	Mistral-Small-3.2-24B-Instruct-2506Q8_0Model ref. ↗	Token speed	81.47 tok/sThroughput · aggregate	—	—	1 × R9700Single GPU · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Arch Linux Engine llama.cpp Backend ROCm Harness llama-batched-bench Prompt / output 1024 / 512 tokens Context 32798 Concurrency 4 Date Not reported Full record →
View ReportGeneric LLM	Mistral-Small-3.2-24B-Instruct-2506Q4_0_MWeights ↗	Token speed	15.5 tok/sThroughput · mean · per request	—	—	2 × RX 7900 XTXLayer split · 24 GB VRAM/GPU Configuration Memory 24 GB VRAM/GPU OS Not reported Engine llama.cpp Backend ROCm 6.4.4 Harness Custom serving benchmark (not named) Prompt / output ? / ? tokens Context 196608 Concurrency 8 Date Not reported Full record →
View Reportllama-bench	Mistral-Small-3.2-24B-Instruct-2506Q8_0Model ref. ↗	Token speed	220.48 tok/sInput + output · aggregate	—	—	1 × R9700Single GPU · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Arch Linux Engine llama.cpp Backend ROCm Harness llama-batched-bench Prompt / output 1024 / 512 tokens Context 32798 Concurrency 4 Date Not reported Full record →
View ReportGeneric LLM	Mistral-Small-3.2-24B-Instruct-2506Q4_0_LWeights ↗	Token speed	29.8 tok/sThroughput · aggregate	—	—	2 × RX 7900 XTXLayer split · 24 GB VRAM/GPU Configuration Memory 24 GB VRAM/GPU OS Not reported Engine llama.cpp Backend ROCm 6.4.4 Harness Custom serving benchmark (not named) Prompt / output ? / ? tokens Context 131072 Concurrency 1 Date Not reported Full record →
View Reportllama-bench	Mistral-Small-3.2-24B-Instruct-2506Q8_0Model ref. ↗	Token speed	—	1,498.09 tok/sPrefill · aggregate	—	1 × R9700Single GPU · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Arch Linux Engine llama.cpp Backend ROCm Harness llama-batched-bench Prompt / output 1024 / 512 tokens Context 32798 Concurrency 8 Date Not reported Full record →
View ReportGeneric LLM	Mistral-Small-3.2-24B-Instruct-2506Q4_0_LWeights ↗	Token speed	25.7 tok/sThroughput · mean · per request	—	—	2 × RX 7900 XTXLayer split · 24 GB VRAM/GPU Configuration Memory 24 GB VRAM/GPU OS Not reported Engine llama.cpp Backend ROCm 6.4.4 Harness Custom serving benchmark (not named) Prompt / output ? / ? tokens Context 131072 Concurrency 1 Date Not reported Full record →
View Reportllama-bench	Mistral-Small-3.2-24B-Instruct-2506Q8_0Model ref. ↗	Token speed	121.92 tok/sThroughput · aggregate	—	—	1 × R9700Single GPU · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Arch Linux Engine llama.cpp Backend ROCm Harness llama-batched-bench Prompt / output 1024 / 512 tokens Context 32798 Concurrency 8 Date Not reported Full record →
View ReportGeneric LLM	Mistral-Small-3.2-24B-Instruct-2506Q4_0_LWeights ↗	Token speed	152.9 tok/sThroughput · aggregate	—	—	2 × RX 7900 XTXLayer split · 24 GB VRAM/GPU Configuration Memory 24 GB VRAM/GPU OS Not reported Engine llama.cpp Backend ROCm 6.4.4 Harness Custom serving benchmark (not named) Prompt / output ? / ? tokens Context 180000 Concurrency 8 Date Not reported Full record →
View Reportllama-bench	Mistral-Small-3.2-24B-Instruct-2506Q8_0Model ref. ↗	Token speed	314.57 tok/sInput + output · aggregate	—	—	1 × R9700Single GPU · 32 GB VRAM/GPU Configuration Memory 32 GB VRAM/GPU OS Arch Linux Engine llama.cpp Backend ROCm Harness llama-batched-bench Prompt / output 1024 / 512 tokens Context 32798 Concurrency 8 Date Not reported Full record →
View ReportGeneric LLM	Mistral-Small-3.2-24B-Instruct-2506Q4_0_LWeights ↗	Token speed	17.2 tok/sThroughput · mean · per request	—	—	2 × RX 7900 XTXLayer split · 24 GB VRAM/GPU Configuration Memory 24 GB VRAM/GPU OS Not reported Engine llama.cpp Backend ROCm 6.4.4 Harness Custom serving benchmark (not named) Prompt / output ? / ? tokens Context 180000 Concurrency 8 Date Not reported Full record →

Quantizations and software

Quantizations: Q4_0_L, Q4_0_M, Q4_0_S, Q8_0
Engines: llama.cpp

Explore all 42 results →

This is a selection across sources, not a ranking or a matched experiment. Reading benchmark reports

GPU benchmark pages: R9700 · RX 7900 XTX