E1: Scout chunk-boundary depth sweep (sawtooth mapping)

• Motivation: the Tier 1 survey found NON-MONOTONIC depth behaviour on Llama-4-Scout (d8192 faster than d4096; 8K pp = d0 pp), consistent with llama4's 8192-token chunked attention resetting the local window at chunk boundaries.
• Design: tg128 at depths 2048 / 6144 / 7680 / 8448 / 12288 / 16384, each a SEPARATE invocation (F6: ~61GiB weights), -r 3 repetitions (prefill cost dominates; 3 reps keeps the sweep ~25 min). Existing points reused: d0 18.54, d4096 16.64, d8192 17.89.
• PRE-REGISTERED PREDICTION (before the run): sawtooth with period 8192. Specifically: (a) tg declines from 2048 to a minimum near 7680 (late-chunk = maximal local window); (b) tg RECOVERS at 8448 (just past the boundary, fresh window) to near the d8192 level; (c) the pattern repeats: tg(12288) < tg(16384-adjacent boundary region), i.e. mid-second-chunk dips again. Falsifier: monotonic decline like every other model = the survey's d4096/d8192 ordering was noise, not mechanism.
• Value: on-prem long-context sizing for llama4-class models - if depth cost resets per chunk, Scout's effective long-context behaviour is dramatically better than a naive slope suggests.

E1 results: SAWTOOTH CONFIRMED (2026-07-04) - all three sub-predictions hit

(*survey points reused; sweep points -r 3, sigmas 0.02-0.18)

• (a) decline to a trough LATE in the chunk: 16.23 at d7680 = sweep minimum ✓. (b) reset AT the 8192 boundary: 17.89, holding 17.63 at 8448 ✓. (c) period repeats: mid-second-chunk dip (16.65 @12288) and second-boundary recovery (17.24 @16384) ✓.
• Amplitude ~8% peak-to-trough, perfectly periodic - mechanism is llama4's 8192-token chunked local attention (window resets each chunk), now measured, not inferred.
• Consulting value: Scout is effectively DEPTH-FLAT. tg at 16K is within 7% of tg at 2K, vs the pure-MoE workhorse losing 42% over the same span. For long-document workloads at reading speed, Scout's depth behaviour is the best in the matrix - its ~18 t/s just never degrades. Caveat: quality effects of chunked attention on long-range reference are an eval question, not a throughput one.
• Raw: llama.cpp/bench-e1-scout-d{2048,6144,7680,8448,12288,16384}.md.

E2: GLM-4.7-Flash quant-pair CONCURRENCY (F17 mechanism isolation)

• Motivation: F17 found gpt-oss-20b (native MXFP4) plateaus at ~4 slots while the Q4_K workhorse scales to 16. Two candidate mechanisms: MXFP4 dequant cost compounding in batched matmuls (F11's tg tax, amplified), or gpt-oss's attention/SWA batch path. The same-model GLM-4.7-Flash pair separates them - identical architecture, only quant differs (the design that resolved F11).
• Design: serve_bench.py, slots {1, 4, 16}, f16 KV, 4K ctx/slot, on GLM-4.7-Flash-Q4_K_M vs GLM-4.7-Flash-MXFP4_MOE (both manifest- VERIFIED, resident). 6 configs, ~15 min.
• PRE-REGISTERED FORK (before the run): compare aggregate scaling ratios (16-slot aggregate / 1-slot aggregate) between the two quants of the same model. (a) MXFP4 ratio clearly BELOW Q4's (>15% relative shortfall) -> the MXFP4 batched-dequant tax is real and compounds under concurrency; gpt-oss's plateau is (at least partly) quant-driven. (b) Ratios parallel (within ~10% relative) -> MXFP4 batches fine; the gpt-oss plateau is architecture (SWA/ attention path), not quant. (c) Between = inconclusive, recorded.
• Value: whether MXFP4 artefacts are disqualified from fleet serving on this stack - a deployment rule, not trivia.

E2 results: FORK RESOLVED arm (b) + NEW FINDING - MLA anti-scales (2026-07-04)

(*under_sampled flag - low confidence on the exact value; the direction is unambiguous)

• Fork: arm (b). Scaling ratios 0.70 (Q4) vs 0.76 (MXFP4) - parallel within ~9% relative. MXFP4 batches no worse than Q4 on the same model -> the gpt-oss-20b plateau (F17) is ARCHITECTURE (SWA/attention batch path), not quant. MXFP4 artefacts are not disqualified from fleet serving by their quant.
• NEW FINDING (-> F18): the deepseek2/MLA graph ANTI-SCALES under concurrency on this stack. Aggregate at 16 slots is ~30% BELOW single-stream (0.70-0.76x, vs workhorse 2.66x, gpt-oss 1.82x), and TTFT reaches ~20s. Candidate mechanism: MLA latent decompression is per-token compute that batching multiplies instead of amortizing (consistent with F8's pp-at-depth collapse); not further separable with this tool. Deployment rule: MLA models are SINGLE-USER machines on this stack - never behind a multi-slot server.
• Cross-check: single-slot per-stream 67.4 vs llama-bench 70.93 = -5% server overhead, consistent with Step 9's P1 pattern.
• Raw: llama.cpp/bench-e2-{q4,mxfp4}-ws{1,4,16}.json.

E3: Image upscaler probe (completes the 1080p/4K story)

• Motivation: FLUX native ceiling is 1536x864 (allocation cap); 1080p/4K deliverables need generate-then-upscale. Real-ESRGAN's ncnn-vulkan build is the standard local upscaler.
• Design: pre-built realesrgan-ncnn-vulkan release binary (pinned + hashed at fetch, TOOLCHAIN.md entry); upscale a 1536x864 gallery- style image to 1080p-class (x2 -> 3072x1728, downsample-free check) and a 1024x1024 to 4K-class (x4 -> 4096x4096); record seconds per upscale and visual verdict. 15-30 min probe per the briefing's Part 3 conventions (log errors, move on).
• EXPECTATION (registered): x4 on a 1MP image completes in seconds-to-tens-of-seconds on Vulkan; total 1024->4K pipeline (48s generate + upscale) lands under ~90s. Failure/limit is a finding (the allocation cap may bite tiled upscalers too - the tool tiles by default, so probably not).

E4: Scout concurrency (4th architecture point on the F17/F18 ladder)

• Design: serve_bench slots {1, 4, 8} on Llama-4-Scout Q4_K_M, f16 KV, 4K ctx/slot. Capped at 8 slots: ~61GiB weights + 8-slot KV approaches the F6 memory edge; 16 slots risks DeviceLost (canary rule stands if it happens anyway).
• PRE-REGISTERED (before run): chunked attention should batch like a normal MoE (the anti-scaling pathology is MLA-specific): 8-slot/1-slot aggregate ratio in 1.6-2.3 (workhorse measured 2.19x at 8 slots). Fork: ratio <1.3 = a second anti-scaling architecture (would gut the "MLA-specific" attribution in F18); >1.6 = ladder point confirmed, F18 attribution strengthened.

E4 results: fork CONFIRMED, band overshot high (2026-07-04)

(*under_sampled flag; direction unambiguous. Per-stream ws1 18.43 vs llama-bench 18.54 = -0.6% server overhead.)

• 8-slot/1-slot ratio 2.72x - fork arm ">=1.6" confirmed decisively, and the registered 1.6-2.3 band OVERSHOT: Scout out-scales even the pure-MoE workhorse (2.19x at 8 slots). Chunked local attention appears to batch exceptionally well (small per-slot KV windows in batched decode - candidate mechanism, not attributed).
• F18's MLA-specific attribution strengthened. The architecture- concurrency ladder: chunked/llama4 2.72x@8 > pure MoE 2.19x@8 (2.66x@16) > gpt-oss/SWA 1.82x@16 (knee ~4) > MLA 0.70x (ANTI).
• Fleet quotable: a 109B model serving 8 concurrent users at 40 t/s aggregate on one box, no memory events at the F6 edge.
• Raw: llama.cpp/bench-e4-scout-ws{1,4,8}.json.

E5: Vision probe - Gemma-3-27B multimodal (new capability axis)

• Motivation: local image understanding ("can the box read charts, photos, documents?") is unmapped territory in the capability envelope; Gemma-3-27B is the only resident multimodal model.
• Design: fetch the unsloth mmproj (pin + verify), llama-server --mmproj, OpenAI chat with base64 images from our OWN gallery (ground truth known: the papercraft deer, the cartoon cats' board meeting, the chess king). 3-4 questions with objectively checkable answers + 1 open description.
• PRE-REGISTERED: works end-to-end at 067de937 (mtmd is in-tree by this vintage); >=3 of 4 factual questions answered correctly. Risk logged: if --mmproj is unsupported/broken for gemma3 here, that IS the result (log and move on, briefing convention).

E5 results: PASS 4/4, ~2.4s per image (2026-07-04)

• Gemma-3-27B + mmproj-F16 (verified) via llama-server --mmproj at 067de937: paper-craft deer, boardroom cats, chess, windowsill plant
• all four factual questions answered correctly in 2.3-2.5s each (including vision encoding of 1024x1024 PNGs). Registered >=3/4 HOLDS with margin.
• New capability axis confirmed: local image understanding works out of the box. Consulting form: "the box can read charts, photos, and documents locally - ~2.5s per image at 27B quality" - document-understanding workflows (invoice OCR-plus-reasoning, chart QA) are now credible probe targets.
• Caveat: 4 easy images from our own gallery = wire-up + sanity, not a vision benchmark; a document/chart-heavy eval is the follow-up if the capability matters to a client.
• Ops note (twice-bitten now): backgrounding a cd && chain leaves the parent shell's cwd unchanged AND kill on the wrapper orphans the server - always absolute paths, background the binary only, pkill -x as the cleanup backstop.

E6: Mixed-workload test (one box, whole office)

• Motivation: every baseline is now measured solo (workhorse 4-slot aggregate 137 t/s; whisper 330s-file in 11.7s; FLUX 1024 in ~48s). Clients ask "can one box do chat + transcription + images AT ONCE"; F15 only covered disk/network contention - this is GPU-compute contention, a different regime.
• Design: run simultaneously: (a) serve_bench workhorse 4 slots 45s window, (b) whisper large-v3-turbo on the 330s file, (c) FLUX 1024x1024 generation. Record each vs its solo baseline.
• PRE-REGISTERED (expectations, first GPU-contention data): all three COMPLETE without errors; LLM aggregate degrades but stays

  =60% of solo (>=82 t/s); whisper and FLUX each take <=3x their solo time. The deliverable is the degradation matrix whatever the numbers; crashes/DeviceLost = the finding instead.

E6 results: all workloads survive; LLM pays the steepest tax (2026-07-04)

• Registered expectations: completion-without-errors HOLDS (no DeviceLost, no failures, TTFT p50 still 2.0s); media <=3x HOLDS; LLM >=60%-of-solo MISSES (41%) - bandwidth-bound decode pays the steepest GPU-contention price, not the "small" media jobs.
• Efficiency framing: ~175s of solo-equivalent work delivered in ~80s of wall time - time-slicing works, the box just charges the chat workload most of the toll.
• Consulting form: "One box CAN run chat + meeting transcription
• image generation simultaneously - nothing crashes and users still get 2s first tokens - but budget chat throughput to roughly HALVE under media load. Schedule media batches off-peak or size the fleet for the haircut."
• Raw: llama.cpp/bench-e6-mixed-llm.json; e6 logs in scratchpad (throwaway); image e6-flux.png rendered correctly under contention.

E7: Eval suite v2 - hard long-context tasks (P2 follow-up)

• Motivation: pilot P2 missed - simple 12K needle-retrieval is below the whole fleet's floor. The separating suite needs multi-hop, aggregation, and cross-reference tasks.
• Design: 6 new long-context tests over the same 12K log excerpt (all deterministically checkable): value arithmetic across entries ("how much faster in t/s is X than Y"), counting ("how many models are recorded clearing 70 t/s in this excerpt"), cross-reference ("which model's pp512 at d8192 was flagged for high variance, and what was the CV"), ordering, and a two-hop lookup. Same 5 models.
• PRE-REGISTERED: separation appears - at least one model scores <50% on the hard-long set while at least one scores >=80%. Secondary question: does the coder finetune beat its base-family sibling on arithmetic-flavoured tasks?

E12: Energy economics (tokens per joule -> EUR per million tokens)

• Basis: amdgpu hwmon power1_average, readable without root (7.07W idle observed). SCOPE DECLARED: APU-package power as reported by amdgpu - a LOWER BOUND on wall power (PSU losses, SSD, board excluded; wall likely +10-25W). Stated on every derived number.
• Design: 1Hz power sampler during (a) 60s idle baseline, (b) workhorse single-slot serve_bench window, (c) 16-slot window. Compute mean W, tokens generated, J/token; report EUR/M output tokens at EUR 0.30/kWh (stated rate) on BOTH total-power and marginal (load minus idle) bases.
• PRE-REGISTERED: load power 60-110W; single-user cost EUR 0.05-0.15 per million output tokens (total-power basis); 16-slot batching improves per-token energy by >=1.8x.

E12 results: ALL THREE PREDICTIONS HIT - and batching is energy-free (2026-07-04)

(total-power basis; marginal basis within 5% of it because idle is negligible. SCOPE: amdgpu package power = lower bound on wall; electricity rate EUR 0.30/kWh stated.)

• Predictions: load 60-110W HIT; single-user EUR 0.05-0.15/Mtok HIT (0.092); batching >=1.8x energy improvement HIT (2.59x).
• The headline: 16 users cost 2.3 W more than 1 user. The GPU is bandwidth-saturated either way - concurrency is energy-free, so fleet serving is where the economics sing.
• Context for clients (illustrative, assumptions stated): adding hardware amortization (~EUR 2000 box / 3 years ~= EUR 1.83/day) at 16-slot saturation (~16.8M tokens/day) adds ~EUR 0.11/Mtok -> all-in ~EUR 0.15/Mtok vs cloud API output pricing measured in dollars-per-million. Two orders of magnitude, plus the 4W idle means an underused box costs ~EUR 10/year to keep warm.
• Raw: scratchpad power logs (values preserved here); results/raw/bench-e12-ws{1,16}.json.

E13: Speculative decoding (0.6B draft)

• Design: Qwen3-0.6B as draft (-md) via serve_bench --extra passthrough; temperature 0 (max acceptance). Two targets: (a) Qwen3-32B dense 10.9 t/s - the case spec decode exists for; (b) Qwen3-30B-A3B workhorse 92 t/s - fast target, draft overhead may dominate. Single slot, per-stream gen rate vs no-draft control.
• PRE-REGISTERED: (a) dense-32B gains >=1.5x; (b) FORK for the workhorse: >=15% gain = worth it / within +/-15% = wash / worse = draft overhead dominates fast MoEs (genuinely uncertain).

E13 results: spec decode spans 0.53x-3.73x BY WORKLOAD - and two traps found (2026-07-04)

• Trap 1 (deployment): at 067de937, -md LOADS the draft (1GB allocated) but speculation stays OFF without --spec-type draft-simple - one log line ("no implementations specified"), slot shows speculative:false, no error. First run measured plain decode while carrying the draft. Trap 2: --draft-max renamed --spec-draft-n-max (flags verified via --help, again).
• Enabled results, Qwen3-32B dense (control 10.90 t/s):
• serve_bench repetitive prompt: 40.61 t/s (3.73x) - log shows 16/16 draft-token acceptance windows. UPPER BOUND: the canonical prompt yields highly predictable continuations.
• diverse prompts: photosynthesis 12.71 (+17%); microservices tradeoffs 6.07 (0.56x); story opening 5.80 (0.53x) - spec decode HALVES generation on low-acceptance text (target pays to verify rejected drafts).
• Workhorse (fast MoE, control ~87.8): 95.96 on draftable text (+9.3% = registered wash arm); diverse-text case untested but the 32B pattern implies risk of regression.
• Registered predictions vs outcome: the >=1.5x dense prediction HIT only on predictable text and INVERTED on realistic text - the registration failed to specify workload, which is itself the lesson. Verdict (-> F21): speculative decoding is not a switch, it is a WORKLOAD-SPECIFIC bet; default OFF; measure on the actual production prompt mix before enabling. Candidate use: code boilerplate, templated documents, structured output.
• Raw: results/raw/bench-e13-*.json (ctrl, draft-inert, draft2).

E14: Multi-model co-residency (fast lane + quality lane)

• Design: two serve_bench instances CONCURRENTLY - workhorse on port 8100, Qwen3-4B on 8101, 1 slot each (both fit trivially: ~20GiB weights combined). Compare each to its solo baseline (workhorse 72.9 agg solo; 4B solo baseline measured first).
• PRE-REGISTERED: both run without errors; each retains 45-70% of solo throughput (two bandwidth-bound decoders sharing one bus); SUMMED tokens/s exceeds either solo run (co-residency is net-positive) - the router pattern is viable without model swaps.

E14 results: router pattern VIABLE - the bus is the budget (2026-07-04)

• Registered: 45-70% retention - workhorse IN band; 4B at 70.6%, 0.6 points over the ceiling (band-edge, within noise). Sum-exceeds- either HOLDS. No errors, both lanes responsive.
• Bandwidth ledger: concurrent lanes consume ~92 + ~115 = ~207 GB/s combined - the full bus, conserved. Co-residency splits bandwidth in proportion to demand; the corridor model predicts the split. Fast lane + quality lane on one box works without model swapping; plan capacity by ADDING active-bytes budgets, not by counting models.
• Raw: results/raw/bench-e14-*.json.

E15: 100K-context probe (does the marketing number survive?)

• Design: ~100K-token corpus of real prose (all project docs + tool READMEs as filler; EXACT count via llama-server /tokenize, not chars-per-token estimates - E8's lesson). One planted unique needle late (~90%), plus early/middle questions. Workhorse, -c set from the measured count + margin, -np 1, -j 1. KV at ~104K ~= 9.8GiB - fits easily.
• PRE-REGISTERED: loads and answers without OOM; needle retrieval 3/3; the practical deliverable is TTFT on a 100K document (predicted 4-8 minutes at the measured pp-depth curve) - "the marketing context length works but costs minutes, not seconds."

E15 results: recall PERFECT at 84K; the marketing number costs 17 minutes up front (2026-07-04)

• Corpus: 84,295 tokens EXACT via /tokenize (all unique project prose on disk - the true ceiling; "100K" target undershot and declared). Needle planted at 90%. Workhorse, -c 114688, KV fit with room (no OOM - registered HOLD).
• Retrieval 3/3 (early fact, middle fact, 90%-depth needle) - registered HOLD; F19's high floor extends to 84K.
• TTFT prediction MISSED 2x: the first question cost 1006s (16.8 min) - average prefill over 0->84K is just 84 t/s (the pp-depth decay continues well past our 32K curve points). With the prompt cache warm, follow-ups cost 4.6s and 1.8s.
• Deployment shape this dictates: long-document Q&A on this box is "ingest once (~17 min for a book-sized doc), then interactive forever" - viable for a knowledge-base pattern, wrong for ad-hoc single questions (where RAG/chunking wins - see E16). The "262K context" marketing number is real but its cost structure is the finding.
• Raw: task output preserved in this entry; server log scratchpad.

E16: Embeddings axis (RAG stack feasibility)

• Design: Qwen3-Embedding-0.6B official GGUF (pin+verify+manifest), llama-server --embedding, /v1/embeddings. Tests: (a) throughput - embed the Phase A log in ~500-token chunks, chunks/sec; (b) sanity
• 5 hand-built triples (anchor, related, unrelated): related cosine > unrelated in >=4/5.
• PRE-REGISTERED: >=10 chunks/sec (conservative); sanity >=4/5. Opens the retrieval-stack capability story for RAG consulting.

E16 results: sanity 5/5; throughput prediction missed and reconciled (2026-07-04)

• Qwen3-Embedding-0.6B-Q8_0 (official, VERIFIED) via llama-server --embedding, batched /v1/embeddings.
• Similarity sanity: 5/5 triples, wide margins (related cosine 0.57-0.92 vs unrelated 0.14-0.39) - registered >=4/5 HOLDS.
• Throughput: 2.9 chunks/sec (~500-tok chunks) - registered >=10 MISSED 3.4x. Reconciliation: embedding = prefill-class forward pass; ~1450 tok/s matches the 0.6B's known pp rate on this stack. The prediction was miscalibrated against the wrong mental model (embedding is not "cheaper than generation" per token - it IS prefill). Unoptimized lower bound (no -ub/batch tuning per the one-fix rule).
• Practical: Phase A log (38 chunks) embedded in 13s; a 10K-chunk knowledge base ~= 1 hour one-off. RAG axis: OPEN and viable. Pairs with E15: chunk-retrieval for ad-hoc questions, full-context ingest (17 min) for interactive document deep-dives - the box supports both patterns; the workload picks.

DEFERRED (registered, not scheduled): 1-hour soak test

One-hour continuous serving stability run (thermal drift, memory leaks, throughput decay) - deferred by Alastair 2026-07-04; the "24/7 appliance" claim stays unevidenced until it runs.

E8: 32K-context eval with cross-chunk reference (the Scout quality question)

• Motivation: E1 proved Scout's tg is depth-flat via 8192-token chunked attention, with the declared caveat that long-range REFERENCE quality was untested. F19 says the fleet floor is high at 12K - so go to ~32K and make questions span chunk boundaries.
• Design: ~32K-token corpus from real project prose (positions of every fact verified programmatically at build time). 6 questions: 2 within-chunk retrieval, 2 CROSS-CHUNK (facts >8192 tokens apart combined in one answer), 2 multi-hop arithmetic across regions. Models: Qwen3-30B (workhorse), gpt-oss-20b, Llama-4-Scout. Serving: -np 1, -c 36864, promptfoo -j 1 (serial, maximizes prompt cache reuse across the shared 32K prelude).
• PRE-REGISTERED FORK: (a) Scout fails cross-chunk questions that the full-attention models pass -> chunked attention's speed is paid in long-range recall (the tradeoff quantified). (b) Scout matches -> chunk-flat throughput is FREE at 32K; Scout becomes the uncontested long-document recommendation. Secondary: fleet stays

  =80% on within-chunk items (F19 extension to 32K).

E8 results: fork lands on arm (a) - after the grader is corrected AGAIN (2026-07-04)

• Infrastructure note first: the "32K" corpus is actually 40.7K tokens on Qwen-class tokenizers (chars/4 underestimated technical text by 43%); the first run 400'd two models at -c 36864 - loudly, exactly as the safe-defaults design intended - and was rerun at 49152. Scout's 200K-vocab tokenizer fit the same bytes under the smaller limit. LESSON: token budgets come from the tokenizer, not from chars/4.
• Raw scores: gpt-oss 5/6, workhorse 4/6, Scout 3/6. Re-adjudicated scores after full-output review: gpt-oss 6/6, workhorse 6/6, Scout 4/6. Three "failures" were grader ambiguity: the living corpus contains TWO legitimate values for the workhorse's tg128 (92.83 step 6 / 92.28 step 8 baseline) and pp512 (1137.67 / 1140.72); models retrieved the alternate value and computed perfectly (gpt-oss's reasoning trace shows textbook retrieval+arithmetic). THIRD grader lesson of the campaign: assertions over living documents need ground-truth SETS, not single values.
• Scout's two remaining failures are real and characteristic: (1) dual-retrieval: echoed raw document sections instead of answering (the answer was visible inside its own echo); (2) arithmetic 18.02 matching no legitimate value pair - approximate retrieval. Both are CROSS-CHUNK items; Scout was 2/2 within-chunk.
• FORK: arm (a). Full-attention models are perfect at 40K (F19 extends); Scout uniquely degrades on cross-chunk reference - the chunk-flat throughput (E1) is paid in long-range reliability. Consulting form: "Scout reads long documents at undiminished speed but is the one model to double-check when the answer spans distant sections - fast reader, occasionally sloppy cross-referencer."
• Raw: results/eval-pilot/e8-*.json (+ .serverlog with the 400s).

E9: adversarial contradiction detection (suite v3, arm 2)

• Design: the 12K excerpt plus a planted "erratum" entry contradicting an early value. 3 tasks x 5 fast models: detect the conflict, list both values, state which entry is later. Grading: both numbers present (deterministic).
• PRE-REGISTERED: contradiction detection separates where retrieval did not - at least one model misses the conflict (predicting the 4B, consistent with the judgment-tax pattern).

E9 results: detection is EASY for the fleet; the only failure was a token budget (2026-07-04)

• Scores: workhorse 3/3, Coder 3/3, Flash 3/3, Qwen3-4B 3/3 - the registered prediction (someone misses; probably the 4B) MISSED on both counts. Planted-erratum contradiction detection, enumeration, and resolution are all within the fleet's floor.
• gpt-oss-20b 2/3: its one failure is NOT detection - the harmony "Thinking" channel leaked into content and exhausted max_tokens (400) mid-scan, never reaching an answer. See F20.

E10: long structured generation (suite v3, arm 3)

• Design: from the 12K excerpt, generate a markdown table of EVERY model with a recorded tg128 (>=8 required rows). Graded by counting required model names AND their correct values present. 5 models, max_tokens 1200.
• PRE-REGISTERED: generation completeness separates: at least one model omits >=2 required rows (long faithful enumeration is harder than retrieval). Question: does any model hallucinate a value not in the doc (worst failure mode, flagged specially)?

E10 results: the registered worst mode FIRED - misattribution under enumeration (2026-07-04)

• Scores: workhorse, Coder, Flash 1/1; gpt-oss 0/1 (token-budget again - it was reasoning CORRECTLY about my doc's duplicate-value ambiguity when 1200 tokens ran out; the difficulty was legitimate and its deliberation exemplary); Qwen3-4B 0/1 with the round's most important failure: MISATTRIBUTION. Its table looks complete but assigns real values to wrong models ("Qwen3-4B: 92.37", "Qwen3-8B: 93.41" - both are workhorse measurements; the true values are 78.38 and 43.56). Worse than omission: plausible, wrong, and invisible without source-checking.
• Extends F19: the small-model tax is judgment AND faithful attribution under enumeration pressure - never let a <5B model build unsupervised tables from documents.
• Raw: results/eval-pilot/e9-.json, e10-.json.

E11: STT accuracy round-trip (quantified WER)

• Design: Piper (alba voice) reads a ~200-word original passage with numbers, names, and technical terms -> whisper large-v3-turbo transcribes -> word-error-rate computed programmatically (stdlib edit distance, normalized casing/punctuation). Closed loop: ground truth exact by construction; audio is real TTS speech, not concatenated samples.
• PRE-REGISTERED: WER <= 5% on clean synthetic speech; numbers and proper nouns are where errors concentrate if any.

E11 results: PASS at ~2.8% true error rate; the ruler needed two corrections (2026-07-04)

• 184-word passage (numbers, names, technical terms) -> Piper alba -> whisper large-v3-turbo. Three WER tiers, each explicit about its ruler:
• naive WER 13.5% - dominated by digit-vs-spelled formatting ("4th" vs "fourth"), NOT mishearing;
• number-normalized WER 7.3% - still counts "100 and 9" as an error although the audio literally said "one hundred and nine" (faithful transcription penalized by MY normalizer);
• semantically-true errors: ~2.8% (5 words/177): "Meta's Llama Scout"->"Metters Lammerscout", "read"->"Reid", "aloud"->"allowed".
• Registered <=5% MEETS on the true-error tier; the registered error-concentration prediction CONFIRMED (numbers + proper nouns).
• Consulting caveats, both actionable: (1) novel product names mangle
• deployments transcribing their own jargon want a vocabulary hint or post-pass; (2) WER quotes are meaningless without stating the normalization - a vendor's "3% WER" and "13% WER" can be the same system (the measurement-integrity rule, applied to speech).
• Round-trip attribution limit declared: TTS pronunciation and STT hearing are joint in this design; separating them needs human listening or reference audio.

E7 results: separation MISSES AGAIN - the high floor is the replicated finding (2026-07-04)

• Registered prediction (>=1 model <50%, >=1 model >=80%): MISS - worst score is 83%. Second consecutive suite design (pilot P2, now E7 multi-hop) that failed to separate the fleet at 12K context.
• The one real failure inverts the secondary question: the CODER finetune made the only arithmetic error (493.95 for 1137.67 - 633.72 = 503.95; off by 10) while every base model - including the 2.3GB Qwen3-4B - computed it correctly.
• Replicated finding (-> F19): the fleet's 12K-context capability floor is HIGH. Retrieval, cross-reference, and multi-hop arithmetic over real technical documents are reliable across every resident instruct model. The small-model tax is JUDGMENT (spam triage, pilot), not context reach or arithmetic. Separating these models will take >32K contexts, adversarial/contradictory documents, or long structured GENERATION - noted for suite v3.
• Consulting form: "12K-document Q&A including computation is safe fleet-wide on this box; pick the model for speed and judgment, not context anxiety."
• Raw: results/eval-pilot/hardlong-*.json.

E3 results: PASS, far under expectation (2026-07-04)

• Registered expectation (pipeline < ~90s): HOLDS with 40% margin. The "native 1080p/4K impossible" finding now has a practical production answer: generate at the 1536x864/1024 ceiling, upscale in seconds. realesrgan tiles internally - the allocation cap never bites.
• Quality: samples sent to Alastair (4096px outputs exceed the 2000px context-read limit - not self-reviewed; founder's eye is the gate).
• Tool: realesrgan-ncnn-vulkan v0.2.5.0 at ~/sparkbench/realesrgan (gitignored); zip 46,931,474 B, local sha256 e5aa6eb131...40c96 = provenance (GitHub publishes no checksum); TOOLCHAIN.md entry added.

E17: end-to-end meeting-to-summary pipeline (nice-to-have, 2026-07-05)

• Design (per master-plan Part 1 nice-to-have): audio -> whisper large-v3-turbo transcript -> workhorse (Qwen3-30B-A3B) "summarise this meeting" -> timed end to end. Input: the 330s (5.5 min) JFK concatenation from E11 (real TTS-free audio; repeated content, so transcript ~1K tokens - a FLOOR for the summarise step).
• Measured stages: transcribe 330s -> 12.4s (26.6x RT, matches the whisper probe); server cold-load 4.1s; summarise 2.8s.
• WARM pipeline (models resident): 15.2s (transcribe + summarise).
• COLD pipeline (incl. model load): 19.2s.
• Consulting headline (measured, 5.5-min sample): "a 5.5-minute meeting to a written summary in ~15 seconds, nothing leaving the box." Conservative 1-hour extrapolation (NOT measured - flag): transcribe ~2.1 min (28x RT) + summarise a real ~8-12K-token transcript (prefill-bound, ~10-25s at the workhorse pp-depth curve) -> roughly 2.5-3.5 minutes audio-to-summary. The summarise floor here (2.8s) understates a real meeting because the test transcript was short and repetitive.
• Raw: scratchpad pipeline log; summary output verified coherent (correctly identified the JFK inaugural content).

E18: Vulkan vs ROCm depth comparison (briefing Part 2, 2026-07-05)

• Context: ROCm 7.2.4 installed 2026-07-05 (DEFERRED-ROOT item 1); gfx1151 natively supported, no HSA override needed. Separate build-rocm/ HIP llama-bench built; Vulkan build/ untouched.
• Design: Qwen3-30B-A3B-Instruct-2507-Q4_K_M, -d 0,4096,8192,16384, 32768 -o md, BOTH backends in the same session (like-for-like ruler). ROCm run adds --mmap 0 (briefing's --no-mmap intent). Vulkan re-run this session rather than reusing Step 8 numbers, so both sides share identical conditions.
• PRE-REGISTERED HYPOTHESIS (briefing Part 2, before the ROCm run): Vulkan wins short/medium context (it's the field-tuned path on RADV here - house baseline built on it); ROCm MAY close the gap or win at 32K+ (the community/literature claim this brief is testing). FORK: (a) ROCm tg or pp at 32K exceeds Vulkan by >5% -> "use ROCm for long-context/document workloads" is evidence-based (maps to a backend-tiering recommendation); (b) Vulkan >= ROCm at all depths -> the single-backend (Vulkan) story holds, ROCm adds nothing here; (c) ROCm fails to build or run gfx1151 kernels -> log and skip (the install still yields the native-support finding).
• Secondary: note first-token/load differences and any ROCm-specific flags required.

E18 results: BACKENDS SPLIT BY PHASE - Vulkan wins generation, ROCm wins prefill (2026-07-05)

Qwen3-30B-A3B, same session, identical ruler. ROCm = HIP build-rocm, --mmap 0. Both handled 32K with no OOM (ROCm enumerated the full 108000 MiB GTT as VRAM).

• Generation (tg128): Vulkan wins at EVERY depth, gap NARROWING with depth (+29.5% -> +7.8%). ROCm never catches it.
• Prefill (pp512): ROCm wins at EVERY depth, advantage WIDENING dramatically with depth (+6.6% -> +79%). ROCm's 32K prefill (323.68) is 1.79x Vulkan's (180.77).
• Fork resolution: arm (a) fires - but ONLY on prefill. The briefing's hypothesis ("ROCm may win at long context") is CONFIRMED with precision: it is ROCm's PREFILL specifically, not generation. -> F22.
• Mechanism: prefill is compute-bound (big batched GEMMs over the whole prompt) - ROCm's mature rocBLAS/hipBLAS kernels win, and the win compounds at depth as attention over a longer context gets more compute-heavy. Generation is memory-bandwidth-bound (one token, streaming weights) - Vulkan's leaner per-token path wins; the HIP runtime carries more per-token overhead.
• Deployment rule (answers the Part 2 consulting question): the backend choice = which phase DOMINATES your workload.
• Long prompt / short output (document QA, extraction, RAG ingest, summarization) -> prefill-dominated -> ROCm.
• Short prompt / long output (chat, drafting, creative) -> generation-dominated -> Vulkan.
• TTFT on a big prompt is prefill -> ROCm gets the first token out faster at depth; Vulkan then streams the rest faster.
• Practical hook: E15's 84K-doc ingest took ~17 min on Vulkan (~84 t/s avg pp). ROCm's 1.79x-at-32K prefill would cut that materially (extrapolation beyond 32K flagged, not measured).
• Raw: results/raw/bench-e18-{vulkan,rocm}.md (+ .log). ROCm 7.2.4, gfx1151 native. Closes DEFERRED-ROOT item 1.

E18a: deep pure-MoE sweep 64K/96K - ROCm prefill lead EXPLODES (2026-07-05)

Qwen3-30B-A3B, both backends, -r 2 (prefill-dominated). Extends E18 past 32K to answer "does ROCm's prefill advantage keep widening."

• Vulkan prefill COLLAPSES past 32K (180.77 -> 53.39 -> 16.57); at 96K it drops BELOW its own generation rate (pp 16.57 < tg 18.40). ROCm prefill holds (323.68 -> 189.53 -> 133.86). The advantage is not just widening, it is EXPLODING: 1.79x -> 3.55x -> 8.08x.
• Generation stays Vulkan-wins but the gap shrinks to noise at depth (+7.8% at 32K -> +3.2% at 96K).
• Practical (measured, not extrapolated now): an 84K-token doc ingest - Vulkan prefill ~20-30 t/s in this collapse zone vs ROCm ~130-150 t/s - would drop the E15 ~17 min Vulkan ingest to roughly 2-3 min on ROCm. For pure-MoE long-document work, ROCm is transformative, not marginal.
• Raw: bench-e18a-deep-{vulkan,rocm}.md. No OOM at 96K on either backend (ROCm sees the full 108GB GTT).

E18b: MLA architecture (GLM-4.7-Flash) - the split INVERTS (2026-07-05)

Generalization test: does the E18/E18a "ROCm wins prefill" hold for a different architecture? deepseek2/MLA graph, both backends, 0-32K.

• For MLA, Vulkan wins BOTH phases at every depth - the exact opposite of pure-MoE. ROCm never wins, and its prefill deficit WIDENS with depth (+1.6% -> +19.4% for Vulkan). ROCm does NOT rescue Vulkan's MLA prefill collapse (F8) - it collapses harder.
• This proves F22 is architecture-specific, not universal -> F23. Without E18b the report would have blanket-recommended ROCm for document work; the truth is it helps ONLY pure-MoE prefill.
• Mechanism candidate: MLA's latent-projection matmul shapes don't favour rocBLAS the way standard-attention GEMMs do; not separable here.
• Raw: bench-e18b-glm-{vulkan,rocm}.md.