# Measured results

Generated 2026-10-18T01:57:38+0000. Every figure was produced by a named script in `~/strata-benchmark`; the script is listed per arm so any number can be traced.

## The rig

- **host**: ai-rig
- **cpu**: AMD Ryzen 5 5500 (6C/12T, Zen 3)
- **ram**: 32 GB (4 x 8 GB G.SKILL Ripjaws V DDR4-3200, single rank)
- **gpus**: 3 x RTX 5060 Ti 16 GB, PCIe 3.0 x8 / x1 / x1
- **vram_mib**: 48933
- **driver**: 595.84
- **os**: Ubuntu 24.04, kernel 6.8.0-147

## Model under test

- **name**: qwen3.8-iq4xs:latest
- **params**: 27.3B dense
- **quant**: IQ4_XS
- **size_gb**: 15.18

## Arms

### A-1card-2133

- configuration: 1 card active (BIOS fault left 2 unbound), 32 GB DDR4-2133
- memory speed: 2133 MT/s
- **how the speed was established**: read from the kernel log's BAR failure, and from nvidia-smi showing 1 GPU
- offload: partial -- a 26.9 GB footprint cannot fit in 16 GB
- harness: `baseline1.py`

| depth | decode tok/s | prefill tok/s | cold prefill |
|---|---|---|---|
| 2048 | **15.72** | - | - s |
| 16384 | **9.92** | - | - s |
| 131072 | **4.34** | - | - s |

### B-3card-2133

- configuration: 3 cards, 32 GB DDR4-2133
- memory speed: 2133 MT/s
- **how the speed was established**: set to SPD default; not independently readable on this host
- offload: partial -- ollama reports size_vram 23.34 GB against a 26.9 GB footprint
- harness: `arm.py`

| depth | decode tok/s | prefill tok/s | cold prefill |
|---|---|---|---|
| 2048 | **47.57** | 88.9 | - s |
| 16384 | **47.32** | 88.8 | - s |
| 131072 | **47.41** | 89.2 | - s |

### C-3card-3200

- configuration: 3 cards, 32 GB DDR4-3200
- memory speed: 3200 MT/s
- **how the speed was established**: manually read in BIOS setup on this boot: 3200 MT/s, XMP, 1.377 V readback (board SVI2 telemetry reads ~2% high; the kit is rated 1.35 V)
- offload: partial -- ollama reports size_vram 23.34 GB against a 26.9 GB footprint
- harness: `arm.py`

| depth | decode tok/s | prefill tok/s | cold prefill |
|---|---|---|---|
| 2048 | **47.76** | 90.1 | - s |
| 16384 | **47.56** | 90.8 | - s |
| 131072 | **47.62** | 90.3 | - s |

### B2-context-use

- configuration: 3 cards, 32 GB DDR4-2133
- memory speed: 2133 MT/s
- **how the speed was established**: not recorded
- offload: not recorded
- harness: `armB2-context-use.py`

### C-3card-3200-working-set

- configuration: 3 cards, 32 GB DDR4-3200, XMP verified
- memory speed: 3200 MT/s
- **how the speed was established**: not recorded
- offload: not recorded
- harness: `armC-workingset.py`

### D-128gb-resident-cu13

- configuration: Strata 0.1.42 CUDA 13 build, IQ2_XS, --resident-experts, --kv-resident 32768, GPU 2 (x8 Gen4), 128 GB DDR4-3600, 5900XT
- memory speed: 3600 MT/s
- **how the speed was established**: read from firmware SMBIOS type-17: configured 3600 MT/s, 4 x 32 GB
- offload: experts resident in RAM (28.61 GiB page-locked); files 0 blobs
- harness: `armD-depth-sweep.py + armD-correct.py`

| depth | decode tok/s | prefill tok/s | cold prefill |
|---|---|---|---|
| 2k | **80.86** | 581.5 | 2.21 s |
| 16k | **78.46** | 1222.9 | 8.0 s |
| 64k | **70.35** | 1514.8 | 25.7 s |
| 128k | **78.91** | 2473.0 | 31.45 s |

stability: **10/10** sequential requests

## What the measurements support

Each finding carries the caveat that qualifies it, because several of these numbers are narrower than they look.

### armD-resident

**On 128 GB with the experts resident, IQ2_XS decodes at 70.3-80.9 tok/s (mean 77.1), flat from 2k to 128k context, and 10 of 10 sequential requests succeed.**

- confidence: high
- evidence: 2k 80.86, 16k 78.46, 64k 70.35, 128k 78.91 tok/s; prefill 582-2473 tok/s; expert cache 92.4-93.6%; files 0 blobs.
- **caveat**: One card (GPU 2), one model, one quantisation. Decode includes reasoning tokens: the engine always thinks first and reasoning_budget_tokens does not suppress it.

### toolkit-fault

**The illegal-memory-access fault that blocked every prompt above ~200 tokens was the CUDA toolkit, not the memory tier, VRAM headroom, or any engine flag.**

- confidence: high
- evidence: A CUDA 12.8 sm_120 build failed 400/1000/2000-token prompts with 'illegal memory access'. A CUDA 13 build of the same source passed all three with zero faults. Same pack, model, card and configuration.
- **caveat**: The CUDA 13 build was already present on disk from an earlier session and had never been installed because its runtime libraries were missing from the host. Nine configuration experiments were spent before the build was questioned.

### card-count-at-shallow-depth

**Three cards give 3.03x the decode of one card at 2,048 context.**

- confidence: high
- evidence: 15.72 -> 47.57 tok/s, same model, prompt, sampling, iteration count.
- **caveat**: Both arms are partial-offload, so this is confounded by offload amount as well as card count. It matches the layer-split prediction of 3x, but does not isolate it.

### card-count-at-depth

**At 131,072 context three cards give 10.92x one card (47.41 vs 4.34 tok/s).**

- confidence: high for the measurement, low for the mechanism
- evidence: Same harness, same depths.
- **caveat**: This is NOT a bandwidth ratio. At depth the one-card arm is limited by residency, not bandwidth, so the arms differ in more than card count.

### depth-immunity

**With three cards, decode is flat across a 64x range of context depth: 47.57 / 47.32 / 47.41 tok/s, a spread of 0.53 %.**

- confidence: high
- **caveat**: The one-card arm falls 72 % over the same range, so depth-immunity is a property of having enough VRAM, not of the model.

### memory-speed-irrelevant-at-depth

**At a 90k-token context the same comparison stays inside noise: prefill +7.5 %, decode -3.6 %, VRAM -0.3 %.**

- confidence: medium
- **caveat**: The two runs sent different token counts (90,952 vs 72,924) because the prompt builders used different filler text, so absolute seconds are not comparable. Only throughput is.

### prefix-cache-scaling

**Re-paying prefill for an identical prefix costs 16x-104x less than the cold pass, and the factor grows with depth.**

- confidence: high
- evidence: 8k: 7.8s -> 0.5s; 32k: 32.4 -> 0.7; 64k: 73.1 -> 1.0; 128k: 167.7 -> 1.6.
- **caveat**: Measured on uninformative filler text, which may cache more favourably than real source code.

### prefill-rate

**Prefill runs 542-730 tok/s depending on depth, against decode at 32-48 tok/s.**

- confidence: high
- **caveat**: An earlier figure of 89 tok/s was an artifact of a 34-token prompt, where fixed per-request overhead dominated. That was an 8x error and is recorded in Appendix E.

### working-set-usability

**For coding or long conversation: ingesting a 90k-token repository costs ~2.1 minutes once, after which every follow-up turn costs ~1.9 s to first token at ~32 tok/s.**

- confidence: medium-high
- **caveat**: Filler-text prompt, not real code. A real repository has different token statistics and may cache differently.

### memory-speed-irrelevant-when-resident

**Raising DRAM from 2133 to 3200 MT/s changed three-card decode by less than the noise floor.**

- confidence: high
- evidence: 47.57 -> 47.76 (+0.40 %), 47.32 -> 47.56 (+0.51 %), 47.41 -> 47.62 (+0.44 %); spread 1.1-1.2 tok/s.
- **caveat**: Measured with the model VRAM-resident, where system RAM is not in the decode path. Does not contradict the +50 % XMP result recorded earlier, which was measured where the working set did not fit in VRAM.

## Not established

- **Why three cards reach a 47.4 tok/s ceiling** is not determined. It holds across every depth and concurrency level measured, which indicates a bandwidth wall rather than a scheduling artifact, but whether the cards are read sequentially or are saturating something shared is untested.
- **The ~49x concurrency fall recorded earlier did not reproduce.** Per-stream decode stayed flat at 46.6-46.8 tok/s across 1-8 parallel requests; two uncontrolled variables differ between the runs.
- **Both the 1-card and 3-card arms are partial-offload**, so card count and offload amount are confounded in every 1-vs-3 comparison.
- **The seven-quantisation sweep** is downloaded but not measured.