Workload scope covers LLM inference, serving and language-model training, plus image and video generation, image editing and upscaling. Model-linked latency, throughput, memory, power and documented evaluation metrics remain distinct. Gaming, conventional rendering and standalone compute kernels are excluded. The current snapshot dates and counts are shown on each dataset page.
Hardware scope: desktop RDNA 3 and RDNA 4 GPUs with at least 12 GB dedicated VRAM per GPU, plus the explicitly selected Ryzen AI Max+ 395 APU. Mobile discrete GPUs and other APUs are excluded. The RX9060XT requires confirmation of its 16GB variant; a pair of 8GB cards does not meet the per-GPU minimum. W7900 slot and W7800 memory variants remain distinct. Exact announcement and shipment dates stay qualified when unavailable. Mixed systems preserve the complete participating inventory without attributing total speed to one card alone.
Coverage and limits
The header counts documented browser result rows and distinct known Card / GPU-count combinations across the entire catalog. Different GPU counts are separate combinations; unknown participation is excluded from the combination count, and explicit mixed inventories remain distinct. These are catalog totals, not independent replications or numbers restricted to the current filter.
This is a broad research collection, not proof that every internet report has been found. The search included GitHub and Codeberg repositories, raw benchmark files, vendor documentation, independent reviews, Phoronix charts, LLM benchmark registries, Reddit, forums, and video descriptions. Known unparsed, inaccessible, duplicate, and nonnumeric sources are explicitly retained in coverage. Failed experiments are not represented as zero performance. Source numbers were checked against available evidence; hardware tests were not rerun.
Data model
The primary filters are Card, GPU count, and cascading model Family, Version and Quantization. Models includes the exclusive LLM / Video / Image selector. Host uses OS, Engine, Backend and Parallelization in every mode and unlocks after a model version is selected. model_series identifies a release line such as Qwen3.8; model_version identifies its checkpoint or variant, such as 27B or Flash-Next. Developer choices are ranked by distinct family/version combinations matching the other active filters, rather than measurement count. model_developer and model_name are browsing labels derived from the existing reviewed repository mappings. Upstream developers are distinguished from quantization publishers, and documented fine-tunes retain their own developer and model names. Precision/packaging suffixes are separated from model names. Family-only links and conflicting identities retain explicit uncertainty; browsing labels do not upgrade their evidence status. The source-reported model and quantization fields remain unchanged in details, the advanced Reported model / file filter, and exports. Different three-bit formats are separate values. Model family alone is insufficient. Unresolved aliases remain explicitly unresolved; they are never silently promoted to a known checkpoint.
One database measurement row represents one numeric LLM metric. The browser groups explicitly associated metrics into result rows using the separate tests table and test_id. Unknown test associations remain separate. A single run may have several rows: prefill, decode, TTFT, power, etc. A source campaign can contain many runs and can share a repository or author with other campaigns. Row counts are not counts of independent replications.
sources describes documentation strength, independence, author, dates, access, URLs and methodology. benchmarks preserves exact reported workload/model names, quantization, participating GPU count, parallelization, OS, engine/build, backend, CPU/RAM, context capacity, actual prompt/output lengths where established, concurrency, statistic, measurement scope, units, extraction evidence and source locator. Extra configurations remain in settings. SQLite also provides cards, flags, coverage and metadata; full source and record payloads are retained as JSON.
Missing values remain null unless reviewed evidence supports a qualified value. OS inferences from platform-specific run artifacts or campaign context are explicitly labeled and stored separately from source-stated OS metadata through os_evidence. An inferred Linux execution environment does not establish a particular distribution, kernel or physical-host OS. metadata_evidence retains each inference's rationale, evidence URL and original value; SQLite exposes this as metadata_evidence_json. Custom filters can select Source stated, Inferred or Unknown OS evidence. Reported values are never overwritten by these gap-filling reviews. An engine default, a current setup guide, a model-card example or a physical device inventory is not automatically evidence for a historical run. Configured context, actual prompt length, maximum active sequences and observed concurrency remain separate concepts.
Harness means the identified benchmark client, runner or evaluation harness. It is distinct from the inference engine: for example, vLLM's benchmark_serving.py can measure a llama.cpp server. A recorded llama-server timing log does not identify the application that sent its requests, so an unnamed harness can legitimately remain unknown. The host-metadata pass recovers named tools from literal recorded invocations and reviewed campaign scripts; it does not assign a harness from the backend alone.
Native llama-bench -pg tests combine prompt processing and generation. Reviewed JSON cells with both n_prompt and n_gen are labeled combined prompt + generation, not prefill-only. Their values and units are unchanged; metric_original and field-level evidence preserve any corrected extraction label. End-to-end client output throughput is likewise distinct from decode-only speed, and aggregate rates are never divided into invented per-stream results.
Source runs, repository links and multiple selections
Every row includes a source_url, a run_id, a portable source_run_url and a source_artifact_url when explicit raw evidence is available. Safe absolute raw/attachment URLs are retained; relative repository paths become permalinks only with a full commit. A reviewed settings.report_url can correct the per-record report destination without changing campaign membership or existing run IDs. A run uses that URL only when all member report URLs agree. Clicking View run opens all metrics in that group and clears other filters; the action is omitted when already at that complete scope. Open report opens the external evidence. Broader report navigation appears only when it would change the scope. Custom filters support combining reports and runs.
The runs table groups rows within a source by an explicit reported run ID, then a manifest/result file and revision, then a reported campaign label. Initial posts and explicit follow-up campaign labels remain separate. When those boundaries are unreported, a visibly labeled whole-source fallback is used. A result file or run can include a sweep across models, GPU allocations, prompts and concurrency settings. This grouping is a navigation aid, not proof of identical experimental conditions or independent replication. Current group counts and grouping evidence are retained in the dataset and report views.
model_references stores repository evidence, verification dates, availability, link type and remaining uncertainty. Rows reference it through model_link_id, and include model_repository_url and model_link_status directly. Tested weights means the source connects a repository, tag, protocol or release-pinned alias to the reported configuration; the detailed reason distinguishes this from a hash-verified historical artifact. Upstream model reference and Model family reference do not claim to identify the tested quantized files. Gated repositories are marked; a current repository or mutable tag does not prove historical bytes. A GGUF filename, a UD prefix or another commenter's download link alone does not establish a quantizer repository.
Every active measurement has a reviewed model-reference decision. Tested weights, upstream references, family references and unresolved identities remain separate; the current counts are in Data quality and the downloadable audit. Unnamed tests, contradictory identifiers and missing quantized repositories remain unresolved instead of being guessed. The compiler rejects unreviewed or conflicting mappings.
All categorical selectors allow multiple choices: OR within a filter, AND across filters. Facet counts count distinct displayed test rows, except Developer which counts distinct model variants. Choices and counts honor every other active filter, including search, card, source, ranges and dates. A facet ignores its own choices so multiple selections remain possible. Unselected zero-match options are hidden; selected zero-match values remain visible so they can be removed. Clearing constraints restores the available choices. Repeated URL parameters preserve punctuation in model names and round-trip multiple choices; legacy single-selection links still work. Individual chips remove one choice, and browser back/forward restores prior selections. Filters match all active conditions against a single measurement, then keep its associated test metrics together for context. Filtered CSV/JSON exports include all measurements in the matching tests, with matches_filters identifying the matching subset. They retain repository evidence and absolute source-run navigation links; full database exports remain portable.
Evaluation scores and result views
The explorer has exclusive LLM, Video and Image modes and starts without results until a filter, search or range is selected. Each row represents an established test/configuration or cohort. Result counts distinguish displayed rows from source reports; the count tooltip retains the underlying measurement count. Reports can share authors and are not independent replications.
LLM results keep Score/result, Prefill and Generation separate. First-token latency is an optional column under Custom filters → Columns; first-content latency and stream-update rates are not substituted for token metrics. The workload line uses reported input/output token counts and concurrency. Configured context is labeled separately and is never substituted for actual input length. Central estimates and ranges keep their identities; full detail views preserve all measurements.
Image results prioritize reported time when available, otherwise the original rate with its named unit. Seconds per image require an explicit per-image unit or an established single-image workflow. Batch timings are labeled separately. Image-steps/s is not images/s, and an unreported step count is not guessed. Procyon points are performance indices, not image-quality scores. Video results lead with readable generation time and clip output. Approximate clip duration is derived from documented generated frames and playback FPS, with known interpolated counts excluded. Average seconds/frame is optional and never represents first-frame latency. Cold/warm states and timing boundaries remain visible; missing timing details are summarized with exact omissions in the record.
Devices and Search share the top row; Models follows, then Host. Benchmark protocols, detailed host fields, workload filters and optional columns are in Custom filters. Numeric sorting displays group labels where measurement definitions differ. Best-first ordering respects higher-is-better rates and scores versus lower-is-better latency, perplexity and loss. A group label establishes a common metric definition, not an identical workload. The default browse order remains a discovery order, not a leaderboard.
The Compare workloads action is available from each interactive result. LLM and media candidates require matching reported model, quantization, GPU count and measurement definition; media also requires the same category, task and timer scope. Candidates are ordered by documented configuration similarity, never by their result value. Comparisons expose differing and unreported input/output lengths, concurrency, context, resolution, steps, batch size, frame count, cold/warm state, software and relevant settings. Unknown settings do not count as a match. Model-family references and shared upstream links alone do not establish identical tested weights. Full source evidence remains authoritative.
result_kind separates answer-quality/agent evaluations and diagnostics from performance measurements. Procyon performance points remain in the performance category. benchmark_* fields describe the reported benchmark, test set, harness/agent, scoring method, level, sample count or maximum, direction and comparison protocol. Unknown details stay null. Original numeric values, units and exact models are preserved. Fractions and explicit task counts can be shown as percentages; rounded fractions never fabricate integer pass counts. Rubric points retain their original maximum. Perplexity and loss identify lower as better. Result ranges operate on the displayed numeric scale (including percentages), while exports preserve original values. Named benchmark suites are grouped independently of metric type; generic families such as Token speed or Latency are used for reports without a named suite. Benchmark versions remain unknown when only an engine build, repository commit or receipt schema is available.
Evaluation charts require matching benchmarks, subsets, harnesses, scoring methods and scales; model/quantization comparisons still require checking settings and evidence. Numeric sorting groups different protocols rather than treating them as a single leaderboard. Overall aggregates and their task/category/depth subtests are not independent benchmark replications.
The new pi-bench campaigns keep override-capable LLM-judge approval separate from recorded FAIL_TO_PASS exit-zero counts. Test collection errors stay in the denominator and those test aggregates are marked partial. These are community subset results, not official full SWE-bench resolved rates. Terminal-Bench-Local Core19 is distinct from the complete Terminal-Bench suite; its two persistent timeout failures remain in the 19-task denominator and the reported retry policy is retained. Custom MMLU/HumanEval/LAB-Bench subsets and synthetic tool-use probes retain their actual sample definitions and scoring rules.
Contaminated SWE-bench runs, rollout-completion counts without solved-task scores, budget-invalid cells and DeepSWE claims without eligible-GPU attribution are recorded as gaps/exclusions rather than imported as clean evaluations. Full source reviews are in research/evaluation-*-quality.md.
GPU topology and measurement scope
Participating GPU chip count, physical board count, card identity/variant and split type are independent fields. A W6800X Duo board has two GPU chips; a source that only reports enumerated devices does not establish physical board count. Tensor parallel, pipeline parallel, data parallel, combined TP/PP, llama.cpp layer split, row split, and independent replicas are distinct. Unknown or conflicting participation is flagged. TP=1 on one established device may be represented as single GPU, with exact command/settings preserved.
Per-request decode is not aggregate serving throughput. Prefill and decode are not combined. Wall-time output throughput, aggregate decode throughput, combined input-plus-output throughput, and per-request distributions retain their metric identities and source definitions. No per-GPU division or aggregate-to-user conversion is inferred. Mean, median, percentiles, ranges, cold starts and warm runs stay separate. Relative performance and performance per dollar are not absolute tok/s.
Evidence strength
- High: original raw data or detailed original methods with traceable measured figures.
- Medium: original measured reports with useful details but material omissions.
- Low: incomplete, ambiguous, indirectly accessible or weakly supported reports.
These are documentation ratings, not probabilities of truth or hardware endorsements. Vendor reports and sponsorship are disclosed independently. An open repository can be well documented and still contain a correctness problem. Source-specific flags should always accompany interpretation.
Quality and history
Compilation validates required fields, unique IDs, source foreign keys, finite numeric values, integer counts, ISO date format, topology consistency, common metric units, exact duplicates and conflicting values at identical detailed locators. SQLite integrity and foreign keys are checked. Automatic completeness measures metadata presence, not correctness.
A separate review revisits source tables, footnotes, follow-up comments and correction notes. Rechecked examples and unresolved conflicts are saved under the current research/llm-*-quality.md and research/expansion-*-quality.md reports and included in audit.json. Old campaigns are retained. An explicitly withdrawn/corrected result may be marked superseded; a different engine version or newer model does not itself invalidate an older measurement. Immutable repository commits and raw file identifiers are retained when available. The record notes describe mismatched model labels, quantization changes, unproven GPU assignment, corrected tokenizers, and invalidated optimizations.
The UI never infers a speedup from heterogeneous results. Record details preserve conditions and missing fields; legacy numeric sorts remain grouped by protocol or metric units. Charts show individual measurements, not pooled averages across unrelated tests.
Reproduce the compilation
From the site directory:
``text
python scripts/build_database.py
node --test tests/*.test.mjs
python -m unittest discover -s tests -p "test_*.py"
npm run dev
``
The build uses only the Python standard library and no network. Add reviewed input files following research/SCHEMA.md; inspect and resolve build errors; then rebuild. The build writes database/database.json, database/benchmarks.sqlite, the compressed JSON/SQLite downloads, public/data/index.json with small verified data segments, audit.json, and this methodology. SHA-256 fingerprints of research inputs and review notes are embedded in the outputs. The SQLite payload_json columns retain fields beyond the indexed columns.
Schema 3 adds runs and model_references tables, source-run and repository foreign keys on benchmarks, and indexes for source-run/repository filtering. See research/LINKS_SCHEMA.md for mapping selectors and precedence. Schema 3.2 retains benchmark-family/version metadata and evidence URLs, and adds tests plus a test_id foreign key on measurements. Group membership is checked for completeness and uniqueness; raw values remain unchanged. The source-specific association review is in research/evaluation-test-groups-quality.md.
The benchmark explorer uses plain HTML/CSS/JavaScript served by a Sites Worker. It presents a fixed, versioned research snapshot, not a live benchmark feed. Anonymous visitor submissions are stored separately in private D1 storage and parsed into provisional suggestions for owner review. On-demand source fetching is limited to submitted URLs and supported public sources; there is no scheduled crawling. Hourly abuse-limit hashes expire after 24 hours. See SUBMISSIONS.md for the intake and curation workflow.
Unified-memory exception
Ryzen AI Max+ 395 results require evidence of that exact SKU and GPU-accelerated or explicitly hybrid execution. Radeon 8060S alone does not distinguish 395 from other APUs. System RAM, configured GPU memory allocation and measured GPU memory use are separate quantities. Shared memory is not stored as dedicated VRAM, and the manufacturer maximum of 128 GB system RAM is not substituted for the tested configuration. CPU-only and NPU-only comparisons are not added as GPU measurements.
Image and video generation
The Image and Video modes contain generation workloads on the same supported GPU catalog. It has a separate, source-reviewed database and independent model/workflow filters. Standalone audio measurements are not claimed unless separately documented. Each result represents one configuration and related measurements; configuration counts are not independent replications.
Record exact model and tested components, quantization, resolution, frame count, playback FPS, sampling steps, batch size, software versions, participating GPUs and parallelization. A detected-device inventory does not establish participating GPU count. Missing settings remain unknown. Base-model and family repository links do not establish the exact tested weight files.
Timing scopes distinguish complete reported workflows, sampling only, isolated model components and unspecified boundaries. Whole-workflow boundaries may exclude loading, compilation or file writing; preserve the source's timer definition. Cold and warm results remain separate, as do different adapters, caches, offload configurations and source revisions. Batch elapsed time is not single-image latency. Reported geometric-mean rates, sampler steps/s, batch-weighted image-steps/s and performance-index scores retain their own units and protocols.
Playback FPS is not generation throughput. Average seconds per frame is derived only from a documented whole-workflow time, a positive generated frame count and batch size one, with no known interpolated-frame count. It is not time to the first frame or a claim of sequential frame delivery. Failed or out-of-memory attempts are retained in research coverage where relevant, never presented as successful output timings. Related scores and speed metrics stay on one row. Rate sorting separates metrics/units; time sorting separates timer scopes. Match workflow, model, output settings and software before interpreting relative performance.
Research covers repositories, original community posts, vendor measurements, independent reviews and public benchmark exports. Evidence receipts retain URLs, locators, quoted values, reporting dates and review notes. Detailed coverage lists record inaccessible sources, uncertain attribution and remaining hardware gaps. Source documentation strength is not independent reproduction or a guarantee of correctness. The internet cannot be certified exhaustively collected; newly found and corrected reports require another source review.
Site analytics
Google Analytics uses cookies to measure visits and standard usage events on public pages. Search text, filter values, record fragments, private review pages and submission contents are excluded from this integration. Advertising signals are disabled. Browser Do Not Track and Global Privacy Control preferences are respected. A separate first-party counter stores daily aggregate browser-tab visits without visitor identifiers. Data-request abuse protection uses short-lived salted network identifiers; the application does not store raw IP addresses for these limits. Full database archives are private, while source-linked benchmark pages remain public. Dataset release history preserves snapshot citations and checksums.