Key Findings
The most interesting takeaways from the community benchmarks
The 36GB memory tier punches far above its weight. Machines with 36GB unified memory average 228.4 tok/s, nearly double the 128.4 tok/s seen at 96GB and more than three times the 77.2 tok/s at 64GB. The M3 Max 36GB in particular dominates the top-15 run list, suggesting this configuration hits a sweet spot where model fits fully in fast on-die memory without the overhead of larger unified pools.
A 550B-class MoE tops the throughput chart, almost certainly API-routed. nvidia/nemotron-3-ultra-550b-a55b logs 231.4 tok/s (247.1 tok/s through its 262-token reasoning phase) on an M5 Max 128GB via the “Command Code” backend. But 550B of weights won’t fit in 128GB of unified memory even heavily quantized, so this is most likely a cloud model proxied through a local endpoint: an interesting datapoint about the model, not a measurement of the Mac.
M5 Pro efficiency claims need revision. With 97 runs now logged, the M5 Pro median sits at 67.1 tok/s with an average of 0.63 W/tok, far above the single-run figure of 0.16 W/tok cited last month. At 0.63 W/tok it trails the M5 (0.22 W/tok) and the A18 Pro (0.22 W/tok) on efficiency, though its 690ms median TTFT remains competitive.
Prefill speeds at the top are now measured in tens of thousands of tokens per second. The M5 Max reaches 49,857 tok/s prefill on gpt-oss-120b MXFP4, and the M5 Pro hits 36,219 tok/s on liquid/lfm2-24b. Even a cloud-routed gemini-flash-lite scores 38,010 tok/s prefill on the M5 Max, suggesting the bottleneck for prompt-heavy workloads has shifted firmly away from ingestion to generation throughput.
GGUF outruns MLX by a wide margin in average throughput. Across 464 runs each, GGUF averages 95.5 tok/s versus MLX’s 73.7 tok/s, a 30% gap that contradicts the common assumption that native MLX is always the faster path on Apple Silicon. Backend choice and quantization format together appear to matter as much as the chip generation itself.
Findings last updated: 2026-08-22 · Data above is live; narrative is human-curated and refreshed on a monthly cadence.
Every finding above is a claim about this dataset, so each one links to the
pivot that produced it. These open the
data explorer with the
grouping, aggregation and filter already applied, over all
1,700+ runs. Change anything and the
address bar updates, so your own view is a link too.
New: Repeated-Run Benchmarksv3.5+
Submissions from app v3.5+ can now bundle N consecutive runs into a single benchmark group, with mean and 95% bootstrap confidence intervals computed across reps.
Why it matters. A single benchmark run captures one snapshot of the system — thermal state, scheduler choices, sampler seed, background load all bleed into the result. An N-rep group runs the same configuration multiple times and reports the mean plus a 95% CI, so you can see the actual spread, not just one point. That CI is what tells you whether a 5% gap between two configs is real or noise.
How to spot one. On the
leaderboard, group reps now show a small “±X.XX · N reps” line under the tok/s value, and the expand panel reveals a Group Context section with the full mean ± CI for tok/s, TTFT, and J/Tok. The
explorer exposes the group aggregates as sortable columns.
How to run them yourself. Open Anubis OSS v3.5 or later, set the Repetitions stepper in the benchmark toolbar to anything from 2–20, and pick a seed strategy — Random (default) captures both hardware and sampler variance, Fixed isolates hardware-only variance for tight reproducibility comparisons. Each rep streams normally; the group aggregate is computed at completion.
Reasoning Models — Three Throughput Metricsv3.1+
For DeepSeek-R1, Qwen3-thinking, and other reasoning models, output tok/s now excludes thinking time. Prefill (input) and reasoning rates are tracked separately so you can see where each model spends its time.
Prefill Speed Leaderboardv3.1+
Top input tokens/sec rates — the prefill phase often dominates total latency on smaller prompts. Computed from prompt_tokens / prompt_eval_duration on submissions from app v3.1+.
| # | Model | Chip | RAM | prefill tok/s | output tok/s | Backend | User |
|---|
Throughput by Apple Silicon Chip
Average, median, and max tokens/sec across all tested models per chip
Memory Bandwidth vs Throughput
Each dot is a benchmark run — bandwidth is the primary driver of LLM inference speed
Power Efficiency
Tokens per Watt of system power by chip
Time to First Token
Median TTFT by chip (lower is better)
Backend Showdown
Average throughput by inference backend (backends with fewer than 2 runs excluded)
Leaderboard: Top 15 Fastest Runs
Highest output tokens/sec across all submissions
| # | Model | Chip | RAM | tok/s | TTFT | W/tok | Backend | User |
|---|
Big Model Club: 100B+ Parameters
Running frontier-class models locally on a Mac
| Model | Chip | RAM | tok/s | Quant | Backend |
|---|
Model Format Distribution
Submissions by model format (GGUF vs MLX)
Memory Tier Performance
Average tokens/sec grouped by unified memory size
Community Contributors
Top contributors by number of benchmark submissions