What One Box Can Do: A Measured Capability Map of a 128GB AI Workstation

Draft v1 - 4 July 2026 - Alastair McDermott / HumanSpark

1. Executive summary

We spent two days measuring what a single desktop-class machine - a GMKtec EVO-X2 with AMD's Ryzen AI Max+ 395 and 128GB of unified memory, roughly EUR 2,000 of hardware - can actually do as an AI platform. Not what the spec sheet implies. What it does, measured, with every prediction written down before the test ran.

The short answers, each expanded later in this report:

• Chat and assistant workloads run at genuinely usable speeds. The best all-round model we tested (a 30B mixture-of-experts) generates 92 tokens per second fresh, and still holds 67 at 8,000 tokens of conversation depth.
• Throughput is predictable before you download a model. A one-line pricing rule driven by memory bandwidth and architecture predicted the speed of six models we had never run, all within their pre-registered bands. You can size hardware from arithmetic, not vibes.
• Mixture-of-experts is not a preference, it is the decision. At matched size, family, and quality class, the MoE model generated 8.5 times faster than its dense sibling. Every dense model above 24B parameters is a batch tool on this hardware, not a chat engine.
• Electricity is a rounding error. Output tokens cost EUR 0.04 to EUR 0.09 per million in power. Serving 16 users draws 2.3 watts more than serving one. The box idles at 4 watts.
• The box is a media workstation too. It transcribes speech at 28x real time, speaks naturally at 26x real time using only the CPU, produces a finished 1024px image in 48 seconds, and reads images (charts, photos) in about 2.5 seconds each.
• The caveats are as measurable as the headlines. We can tell you exactly where "instant" chat speed dies as context grows, which architecture melts down under concurrent users, what quantization costs, and why a vendor's benchmark number needs its measurement conditions attached before you believe it.

Everything here was produced by a repeatable method: pin the artefact, register the prediction, run, log the deviation. The method transfers to any hardware you are considering. That, more than any single number, is the asset.

2. The machine and the method

Hardware. GMKtec EVO-X2: AMD Ryzen AI Max+ 395 ("Strix Halo"), Radeon 8060S integrated GPU, 128GB LPDDR5x unified memory (~220 GB/s effective), 2TB NVMe. The GPU and CPU share one memory pool - that single fact drives most of the results below.

Software. llama.cpp (pinned commit, Vulkan RADV backend, Mesa 25.2.8), whisper.cpp, stable-diffusion.cpp, Piper, Real-ESRGAN. Everything open source, everything local, no cloud dependency at any point.

Method - why you can trust these numbers.

1. Every model artefact is pinned to an exact upstream revision and verified byte-for-byte against the publisher's checksum before first use. Twenty-seven artefacts, zero exceptions.
2. Every experiment registers its prediction in writing before it runs. When results miss the band, we publish the miss and what it taught us. Several of the most useful findings in this report are published prediction failures.
3. Measurement rulers get audited like the measurements. Three separate "failures" during this campaign turned out to be the measuring instrument (a grading assertion, a text normalizer, a token estimator). Each is documented, because a benchmark that does not state its ruler is not comparable to anything.

3. Text generation: the throughput physics

3.1 The corridor rule

Generation speed on this class of hardware is memory-bandwidth arithmetic:

tokens/sec ~= (220 GB/s / bytes read per token) x architecture factor x quantization factor

The architecture factors we measured: dense models ~1.0 (they use effectively all of the theoretical ceiling), classic mixture-of- experts ~0.84-0.94, and two exotic architectures that pay real taxes (a latent-attention design at 0.66-0.80 and a hybrid recurrent design at 0.55). Post-hoc MXFP4 quantization costs ~19% on generation only - prefill is unaffected.

We validated the rule prospectively: six models we had never run (Meta's Llama-4-Scout, Mistral Small 3.1, Gemma 3 27B, Phi-4, Qwen3-32B, DeepSeek-R1-Distill-32B) were priced before download. All six landed inside their pre-registered bands, with ratios of actual-to-predicted between 0.87 and 1.00. The rule prices unseen models to within a few percent from a file size and an architecture label.

3.2 The headline comparison: MoE versus dense

Same family (Qwen3), same generation, same quantization, files within 6% of each other in size:

The MoE advantage at matched scale is 8.5x on generation and 5.8x on prefill. The physics: the dense model reads all 18GB of its weights for every single token; the MoE reads about 2GB. On bandwidth-limited hardware - which is all unified-memory hardware - that ratio is destiny. The corollary we measured across five dense families: a dense model's speed is its file size divided into the bandwidth, almost exactly. Nothing about brand or training changes it. Fine-tuning does not either: the coding variant of the workhorse matched its sibling to three significant figures.

3.3 The full survey (single user, fresh context)

3.4 Quantization: the cost is exactly the size

One model (Mistral Small 24B) at four quantization levels: speed scaled inversely with file size at every rung, within 3% of prediction. Near-lossless Q8 runs at 59% of Q4's speed. Practical consequence: choose quantization on quality grounds only - the speed cost is known before you download, because it is the size ratio.

4. Context depth: where "instant" goes to die

Every model that clears an instant-feeling ~70 t/s when fresh loses that badge as conversation or document context grows. We measured the death point per model (the depth at which generation drops below 70 t/s):

(*run-sensitive: its curve is nearly flat at the threshold)

Two structural findings sit in this table. First, ranking is depth-dependent: the workhorse and the hybrid swap places at ~16,000 tokens - by 32K the "slower" hybrid is 25% faster. Model selection needs the workload's context profile, not a single benchmark number. Second, Llama-4-Scout is depth-flat: its chunked attention resets every 8,192 tokens, producing a measured sawtooth rather than a decline. Its 18.5 t/s at fresh context is still ~17-18 t/s at 16K, the best long-context retention we measured - with a quality tradeoff we also measured (section 7.3).

Two levers help. Quantizing the KV cache costs ~4% when fresh and pays +15% at 32K while halving cache memory - free for long-context work, pointless for chat. And the 100K-class test (section 7.4) shows recall stays perfect far deeper than the speed stays pleasant.

5. Serving more than one person

5.1 Concurrency is architecture-dependent - dramatically

Aggregate throughput scaling from 1 to 16 concurrent users:

The last row deserves emphasis: the latent-attention architecture anti-scales. Sixteen users get less total throughput than one, and each waits 20 seconds for a first token. We confirmed with a same-model quantization pair that this is the architecture, not the file format. Deployment rule: MLA models are single-user machines on this stack. And note the ladder spans 3.8x - the single-user speed ranking and the fleet ranking are different orderings. You cannot size one from the other.

5.2 The router pattern works

Two models served simultaneously (the 30B workhorse plus a fast 4B) split the memory bus predictably - each lane kept 50-70% of its solo speed, summed throughput beat either alone, and the bandwidth ledger balanced to the full ~207 GB/s. Capacity planning on unified memory is additive in bytes-per-token, not in model count. A fast-lane / quality-lane router on one box is viable with no model swapping.

5.3 Everything at once

We ran chat serving (4 users), meeting transcription (three 5-minute files), and image generation simultaneously. Nothing crashed, first tokens stayed around 2 seconds, and the toll distribution surprised us: chat throughput halved while the media jobs only slowed 1.3-2.2x. One box does run the whole office - budget the chat haircut, or schedule media batches off-peak.

6. What it costs to run

Measured at the package-power sensor (a lower bound on wall power - add roughly 10-25W for the full box), at a stated EUR 0.30/kWh:

Serving 16 people costs 2.3 watts more than serving one, because the memory bus is saturated either way. Batching is energy-free, so the economics improve exactly as fast as you can find concurrent demand. Adding hardware amortization (EUR 2,000 over three years at 16-user saturation) brings the all-in figure to roughly EUR 0.15 per million output tokens. Cloud API output pricing is measured in dollars per million. The gap is two orders of magnitude, and the sovereignty is thrown in for free.

7. Beyond chat: the rest of the capability map

7.1 Speech to text

whisper.cpp with the best-quality turbo model transcribes at 28x real time - an hour of meeting audio in about two minutes. The finding that changes the deployment decision: the small model runs at the same speed as the large one on this GPU, so quality is free - always run the best model. Accuracy on a closed-loop test (our TTS reading a known passage back through whisper) was ~2.8% true word errors, concentrated exactly where predicted: novel product names. Transcribing your own jargon wants a vocabulary hint. And a caution for buyers: the same audio scored 13.5% or 2.8% depending only on the error-counting rules - insist any vendor states theirs.

7.2 Text to speech

Piper generates natural British-accented speech at 26x real time using only the CPU, leaving the GPU free for other work. A 10-minute narration renders in about 23 seconds. Founder's ear test: passed.

7.3 Images - generation, understanding, and the resolution truth

FLUX.1-schnell produces a finished image in 9.5 seconds at 512px and 48 seconds at 1024px. A 50-image style gallery (photography, flat illustration, watercolor, oil, cartoon, 3D, sketch) ran overnight at 100% success with cost completely independent of style. Native resolution tops out at 1536x864 - a GPU allocation limit, not a memory-size limit - but a 3-second upscale pass delivers 4K-class output, so the practical pipeline is generate-then-upscale at ~51 seconds total.

The box also reads images: Gemma 3 with its vision head answered factual questions about our own generated images 4/4 at ~2.5 seconds per image. Document and chart understanding workflows are credible and queued for a deeper probe.

7.4 Long documents and retrieval

At 84,000 tokens of real project prose (a book-sized context), recall was perfect - including a needle planted at 90% depth - but the first answer cost 17 minutes of processing, after which follow-up questions cost 2-5 seconds. So the long-context marketing numbers are real, and their cost structure dictates the pattern: ingest once, then converse. For ad-hoc single questions, chunked retrieval wins: the embedding model indexes ~1,450 tokens per second (a 10,000-chunk knowledge base in about an hour, once) with semantically sound similarity (5/5 on our sanity set). Both patterns work; the workload picks.

7.5 Speculative decoding: a bet, not a switch

Pairing the slow dense 32B with a small draft model tripled its speed on predictable text (3.7x) and halved it on creative prompts (0.53x). The gain is a function of how guessable the output is. Default it off; measure on your production prompt mix; expect wins on boilerplate, templates, and structured output. (Also: on current llama.cpp the draft model loads but silently does nothing without an extra flag - one of several deployment traps this report documents so you do not rediscover them.)

8. Quality: what the evals said

Throughput was the easy half. We built a deterministic evaluation suite (email triage, extraction, summarization, coding; short and long context variants; graded by string and code assertions at temperature zero) and ran it across the fleet, then iterated it three times trying to make the models fail.

What we learned:

• The capability floor is high. Every model of 20B and up passed everything we threw at it - including multi-hop arithmetic across a 40,000-token document and planted-contradiction detection. Two independently designed suites failed to separate the fleet below 12K context. For everyday document Q&A, model choice on this box is about speed and serving behaviour, not anxiety about correctness.
• The small-model tax is judgment, not context. The 2.3GB model retrieved and computed perfectly at 12K tokens - and misclassified both spam emails as urgent, and, when asked to build a summary table, attached real values to the wrong models. Plausible, wrong, and invisible without source-checking. Small models need supervision on judgment calls and unsupervised enumeration; they do not need babysitting on retrieval.
• Scout's speed has a price. The one model with degraded long-range behaviour was the depth-flat Scout: at 40K tokens it answered within-section questions perfectly but fumbled two questions whose facts sat in different attention chunks. Fast reader, occasionally sloppy cross-referencer - double-check it when answers span distant sections.
• Eval scores depend on serving configuration. A reasoning-style model failed tasks it demonstrably can do because its thinking stream consumed the response budget. Any published eval score that does not state its token limits and template handling is not comparable to anything.

9. Operational lessons (the trap list)

These cost us time so they do not have to cost you any:

1. GGUF archives age. A December 2023 Mixtral file no longer loads on current llama.cpp; the re-converted 2025 file works. Model archives need re-conversion plans.
2. Token budgets come from the tokenizer. The same 114KB document is 40,700 tokens to one model family and under 36,800 to another. Estimating from character counts overflowed two context windows.
3. Ground truth is a set, not a value. Living documents carry multiple legitimate values for the same quantity (re-measurements, errata). Three of our own grader "failures" were models retrieving a value we forgot we had recorded.
4. State the ruler, always. WER normalization (13.5% vs 2.8%), eval token budgets, spec-decode workloads, power measurement scope - in every case the stated conditions moved the number more than the system under test did.
5. Big-model memory edges are real but manageable: above ~60GiB of weights, benchmark invocations need splitting; image generation above ~1.4 megapixels needs tiled decoding. Both have one-flag fixes once you know.

10. The deployment playbook

If we were provisioning this box for a small organisation today:

• Default chat/assistant: Qwen3-30B-A3B. 92 t/s solo, scales to 16 users at 194 t/s aggregate, EUR 0.04/M tokens served.
• Interactive coding: Qwen3-Coder-30B (same speed, code-tuned).
• Long documents, fast answers: embeddings + retrieval (index once, ~1,450 tok/s). Deep interactive analysis of one document: ingest into the workhorse's full context (minutes up front, seconds thereafter). Very long single passes at steady speed: Scout, with cross-reference spot-checks.
• Quality-critical single-user drafting: GLM-4.7-Flash - never behind a multi-user server.
• Meetings: whisper turbo, always the big model; add a vocabulary pass for product names. Narration/accessibility: Piper on CPU. Blog/social imagery: FLUX at 1024 or 1536x864 + 3s upscale.
• Do not deploy dense 24B+ for interactive use, MLA models for fleets, speculative decoding by default, or sub-5B models for unsupervised judgment or table-building.

11. Limits of this report, and what is next

Numbers are from one machine, one driver stack, one llama.cpp commit; a two-tier regression trigger re-validates them on any stack change, because these are maintained measurements, not folklore. Package power is a lower bound on wall power. Quality evals are deterministic and single-shot - a statistical, human-judged eval tier is the natural next investment. Still queued: the ROCm backend comparison (needs installation), a 1-hour soak test, document/chart vision evals, and a 235B stress probe that wants physical presence.

Twenty-one durable findings, every raw output, every prediction and every miss are version-controlled in the sparkbench repository. The strongest claim this report makes is not any single number - it is that every number here was predicted, then measured, then kept accountable in public view. That is the standard we would bring to measuring your hardware, too.

Method, logs, and findings register: sparkbench repository (docs/FINDINGS.md F1-F21, results/, docs/PHASE-A-LOG.md).