Baseline (Phase A, do not re-run - briefing reference data)

Gaps and skips

• ROCm comparison (Part 2): SKIPPED - ROCm not installed (checked 2026-07-04); entry 1 in DEFERRED-ROOT.md.
• Tier 3 (Qwen3-235B, DeepSeek-V3): not present on disk, not downloaded per briefing.
• Tier 2 #8 (Qwen3-8B): already benched in Phase A - reused above, not re-run.

Survey entries (appended per model)

Phi-4 (dense, 14.7B params, 14.7B active)

• Quant: Q4_K_M
• File: /opt/models/staging/phi-4-Q4_K_M.gguf (unsloth/phi-4-GGUF, rev 5110b777, manifest VERIFIED)
• File size: 8.89 GB (8.28 GiB per llama-bench)
• Memory usage: ~8.3 GiB weights, full Vulkan offload
• Flags: none (defaults)
• pp512: 611.88 t/s
• tg128: 24.40 t/s
• tg128 @ 4K context: 22.36 t/s
• tg128 @ 8K context: 20.62 t/s
• Corridor prediction: 220/8.89 x ~1.0 (dense) = 24.7 t/s
• Corridor ratio (actual/predicted): 0.99
• Notes: registered band 23.5-25.5 HIT. Runs on the llama graph (llama-bench reports "llama 13B", 14.66B params). Fifth dense point at ~99-100% of naive - the dense constant is now family-diverse (Qwen x3, Llama, Phi). Consulting answer for "modest hardware floor": 24 t/s = comfortable reading speed; Phi-4-mini (Tier 2) will probe the true floor. Declared condition: Scout + Tier-1 downloads active (3c rule); pp@8192 CV 4.4%, within gates.

Mistral-Small-3.1-24B-Instruct-2503 (dense, 23.6B params, 23.6B active)

• Quant: Q4_K_M
• File: /opt/models/staging/Mistral-Small-3.1-24B-Instruct-2503-Q4_K_M.gguf (unsloth, rev d63ca941, manifest VERIFIED)
• File size: 14.33 GB (13.34 GiB per llama-bench)
• Memory usage: ~13.4 GiB weights, full Vulkan offload
• Flags: none (defaults)
• pp512: 333.97 t/s
• tg128: 15.07 t/s
• tg128 @ 4K context: 14.42 t/s
• tg128 @ 8K context: 13.81 t/s
• Corridor prediction: 220/14.33 x ~1.0 (dense) = 15.4 t/s
• Corridor ratio (actual/predicted): 0.98
• Notes: registered band 14.5-16 HIT. Dense-at-ceiling n=6, family #4 (Mistral). THE narrative number for the EU-sovereignty conversation: the strong European dense model reads at 15 t/s where the same-quant MoE workhorse does 92 - the dense penalty in one example. pp 334 is the true compute cost of dense (all 23.6B params per token; contrast Phi-4's 612 at 14.7B, workhorse's 1141 at 3.3B active). Mild depth slopes (-8.4% tg by 8K). Declared condition: downloads active; pp CVs 3.5-4.8%, within gates. Quant ladder (Part 4) will reuse this artefact as its Q4_K_M rung.

Gemma-3-27B-it (dense, 27.0B params, 27.0B active)

• Quant: Q4_K_M
• File: /opt/models/staging/gemma-3-27b-it-Q4_K_M.gguf (unsloth, rev 7cd0121f, manifest VERIFIED)
• File size: 16.55 GB (15.40 GiB per llama-bench)
• Memory usage: ~15.4 GiB weights, full Vulkan offload
• Flags: none (defaults)
• pp512: 248.43 t/s
• tg128: 12.56 t/s
• tg128 @ 4K context: 11.91 t/s
• tg128 @ 8K context: 11.65 t/s
• Corridor prediction: 220/16.55 x ~1.0 (dense) = 13.3 t/s
• Corridor ratio (actual/predicted): 0.94
• Notes: registered band 12.5-13.8 HIT at the floor edge. Native gemma3 graph. Ratio 0.94 is the low edge of the dense cluster (others 0.98-1.00) - noted, not attributed. SHALLOWEST dense depth slope (-7.2% tg by 8K): the 5:1 local:global SWA keeps KV reads light, as predicted at pin time. pp 248 = the true 27B-dense compute bill. Consulting answer: Gemma works fine on AMD, but at 27B dense it is a 12 t/s reader, not a chat engine. Declared condition: downloads active; pp d0 CV 5.3%.

Qwen3-32B (dense, 32.8B params, 32.8B active) - THE dense-vs-MoE comparison

• Quant: Q4_K_M
• File: /opt/models/staging/Qwen3-32B-Q4_K_M.gguf (unsloth, rev 931c8406, manifest VERIFIED)
• File size: 19.76 GB (18.40 GiB per llama-bench)
• Memory usage: ~18.4 GiB weights, full Vulkan offload
• Flags: none (defaults)
• pp512: 198.20 t/s
• tg128: 10.89 t/s
• tg128 @ 4K context: 10.30 t/s
• tg128 @ 8K context: 9.76 t/s
• Corridor prediction: 220/19.76 x ~1.0 (dense) = 11.1 t/s
• Corridor ratio (actual/predicted): 0.98
• Notes: registered band 10-12 HIT. The survey's centrepiece result - same family, same generation, same quant, near-same file size as the MoE workhorse: Qwen3-30B-A3B does 92.28 tg / 1141 pp; Qwen3-32B dense does 10.89 tg / 198 pp. MoE advantage at matched scale: 8.5x generation, 5.8x prefill. Phase A's 3.8x (vs 14B dense) understated the matched-size case. Physics: dense reads all 18.4GB per token, the MoE ~2GB. Consulting answer to "MoE or dense?": MoE, by nearly an order of magnitude, and it is not close. Also the matrix's slowest prefill: an 8K-context document costs ~95s of prefill (86 t/s pp @8K) before the first token - dense 32B is a batch tool on this hardware, not interactive. Declared condition: R1-Distill download active.

Llama-4-Scout-17B-16E (MoE, 107.8B params, 17B active)

• Quant: Q4_K_M (2 shards)
• File: /opt/models/staging/Llama-4-Scout-...-0000{1,2}-of-00002.gguf (unsloth, rev 72a6853f, manifest VERIFIED; briefing's bartowski URL was stale - substitution logged)
• File size: 65.36 GB (60.86 GiB per llama-bench)
• Memory usage: ~61 GiB weights, FULL Vulkan offload - fits with ~10 GiB device-local headroom; F6 split invocations used, no memory events across three legs
• Flags: none (defaults; split -d invocations per F6)
• pp512: 163.17 t/s
• tg128: 18.54 t/s
• tg128 @ 4K context: 16.64 t/s
• tg128 @ 8K context: 17.89 t/s
• Corridor prediction: 220/(0.604 B/param x 17B active) x 0.84 (pure MoE) = ~18 t/s
• Corridor ratio (actual/predicted vs naive 21.4): 0.87 - pure-MoE class, llama4's chunked attention carries no extra arch tax at d0
• Notes: registered band 15-20 HIT. Consulting answer to "Can I run Llama locally?": YES on a 128GB Strix Halo box - Meta's 109B Scout runs wholly on-GPU at 18.5 t/s reading speed (and NO on 32/64GB machines - it needs ~61GiB for weights alone). FINDING-shaped wrinkle: depth behaviour is NON-MONOTONIC - d8192 (17.89 tg / 165 pp) beats d4096 (16.64 / 146), with 8K pp matching d0. Consistent with llama4 chunked attention (8192-token chunks): d8192 sits at a chunk boundary where the local window resets. Candidate mechanism, not attributed - a finer depth sweep would map the sawtooth. Quality caveat for client conversations: Scout's mixed reception vs Qwen is a quality question; this row only settles that it RUNS, comfortably.

DeepSeek-R1-Distill-Qwen-32B (dense, 32.8B params, 32.8B active)

• Quant: Q4_K_M
• File: /opt/models/staging/DeepSeek-R1-Distill-Qwen-32B-Q4_K_M.gguf (unsloth, rev 1938d05c, manifest VERIFIED)
• File size: 19.85 GB (18.48 GiB per llama-bench)
• Memory usage: ~18.5 GiB weights, full Vulkan offload
• Flags: none (defaults)
• pp512: 223.96 t/s
• tg128: 11.05 t/s
• tg128 @ 4K context: 10.50 t/s
• tg128 @ 8K context: 10.00 t/s
• Corridor prediction: 220/19.85 x ~1.0 (dense) = 11.1 t/s
• Corridor ratio (actual/predicted): 1.00
• Notes: registered band 10-12 HIT, at ceiling. Confirms Qwen3-32B within 1.5% (qwen2 vs qwen3 graph - generation-independent). Reasoning-model reality check: 11 t/s x 1000+-token thinking traces = MINUTES per answer on this box. "Thinking" models at dense-32B scale are batch analysts here, not chat partners - the accessible reasoning option is a fast MoE with thinking mode (Qwen3-30B-A3B at 92 t/s), not the distill. Declared condition: no downloads active (last bench of the chain).

Phi-4-mini (dense, 3.8B params, 3.8B active) - Tier 2, the floor

• Quant: Q4_K_M
• File: /opt/models/staging/Phi-4-mini-instruct-Q4_K_M.gguf (unsloth, rev 78eb92a4, manifest VERIFIED)
• File size: 2.49 GB (2.31 GiB per llama-bench)
• Memory usage: ~2.3 GiB weights, full Vulkan offload
• Flags: none (defaults)
• pp512: 2149.96 t/s (highest prefill in the matrix)
• tg128: 77.44 t/s
• tg128 @ 4K context: 63.13 t/s
• tg128 @ 8K context: 54.87 t/s
• Corridor prediction: 220/2.49 x ~0.9-1.0 = band 79-90
• Corridor ratio (actual/naive): 0.88
• Notes: registered band MISSED low by 2% (77.44 vs 79 floor) - and the miss refines a constant: 0.877 of naive matches Qwen3-4B's 0.887, so SMALL-DENSE OVERHEAD ~0.88 is now n=2 and looks systematic below ~5GB artefacts (vs ~0.99 for 9-20GB dense). Floor answer: the smallest useful model clears the ~70 instant line fresh (badge dies ~2K depth, same pattern as the 4B), reads 8K prompts at 917 t/s pp. Declared condition: Coder-32B download active.

Qwen2.5-Coder-32B-Instruct (dense, 32.8B params, 32.8B active) - Tier 2

• Quant: Q4_K_M
• File: /opt/models/staging/Qwen2.5-Coder-32B-Instruct-Q4_K_M.gguf (unsloth, rev 638ed913, manifest VERIFIED)
• File size: 19.85 GB (18.48 GiB per llama-bench)
• Memory usage: ~18.5 GiB weights, full Vulkan offload
• Flags: none (defaults)
• pp512: 223.16 t/s
• tg128: 11.09 t/s
• tg128 @ 4K context: 10.53 t/s
• tg128 @ 8K context: 10.03 t/s
• Corridor prediction: 220/19.85 x ~1.0 (dense) = 11.1 t/s
• Corridor ratio (actual/predicted): 1.00
• Notes: registered band 10-12 HIT at ceiling. Statistically IDENTICAL to R1-Distill (11.05/10.50/10.00) - same Qwen2.5-32B body, third confirmation that fine-tune content never moves throughput. Coding story on this box: dense-32B coder = 11 t/s batch code reviewer; the interactive coding seat belongs to Qwen3-Coder-30B MoE at 93 t/s. Declared condition: Mixtral download active.

Mixtral-8x7B-Instruct-v0.1 (MoE, 46.7B params, ~12.9B active) - Tier 2

• Quant: Q4_K_M (mradermacher re-conversion)
• File: /opt/models/staging/Mixtral-8x7B-Instruct-v0.1.Q4_K_M.gguf (rev 92bb790b, manifest VERIFIED)
• File size: 28.45 GB (26.49 GiB per llama-bench)
• Memory usage: ~26.5 GiB weights, full Vulkan offload
• Flags: none (defaults)
• pp512: 216.13 t/s
• tg128: 26.45 t/s
• tg128 @ 4K context: 24.60 t/s
• tg128 @ 8K context: 23.46 t/s
• Corridor prediction: 220/(0.609 x 12.9B) x 0.84 = ~23.5; band 21-27
• Corridor ratio (actual/naive ~28): 0.94
• Notes: registered band HIT near ceiling. TWO findings: (1) the original TheBloke Dec-2023 GGUF DOES NOT LOAD at 067de937 (MoE tensor layout changed) - "GGUF archives age, re-conversion required" is a real operational fact for anyone keeping model archives; artefact kept as evidence. (2) The classic big-expert top-2-of-8 MoE runs at ~0.94 of naive - ABOVE the modern fine-grained MoE cluster (0.84-0.86); simpler routing appears cheaper per byte on this stack. Still obsolete on merit: the workhorse is 3.5x faster on 40% less memory. Declared condition: ladder downloads active.

PART 4 COMPLETE - Mistral-Small-24B quantization ladder (2026-07-04)

• Generation speed is EXACTLY inverse to file size - every rung lands at 0.98-1.03 of its corridor prediction. The consulting soundbite: "quantization's speed cost IS the size ratio; the only open question is quality, and that is what evals are for." Near-lossless Q8 costs 41% of Q4's speed on the same model.
• pp wrinkle (recorded, not attributed): prefill does NOT follow the same line (Q4 334 > Q8 283 > Q6 249 > Q5 236) - dequant kernel paths differ per quant in compute-bound prefill.
• All three new rungs verified against upstream oids before use (hashed directly; the chain-level hash prints only at invocation end).

TIER 2 COMPLETE (Qwen3-8B reused from Phase A) - summary above plus:

Phi-4-mini 77.44 (floor, small-dense 0.88), Coder-32B 11.09 (= R1 body), Mixtral 26.45 (vintage-GGUF finding + classic-MoE efficiency).

TIER 1 COMPLETE - summary (2026-07-04)

Six unseen models, six registered-band hits, corridor ratios 0.87-1.00. The pricing model (F10) predicts throughput of models it has never seen from file size + architecture class alone. Headline: Qwen3-32B dense vs Qwen3-30B-A3B MoE = 8.5x generation gap at matched family/scale/quant. Meta's 109B Scout runs wholly on-box at reading speed. Every dense model 24B+ is a batch tool, not a chat engine, on this hardware.