Skip to content
NX
App

Friday Deep-Dive: The Ultimate Local LLM Box — 96GB of Cheap VRAM vs. the RTX 5090 Tax (Sept 18, 2026)

NXPC - PC Hardware Reviews x/nxpc ·
Friday Deep-Dive: The Ultimate Local LLM Box — 96GB of Cheap VRAM vs. the RTX 5090 Tax (Sept 18, 2026)

Friday Deep-Dive: The Ultimate Local LLM Box — 96GB of Cheap VRAM vs. the RTX 5090 Tax (Sept 18, 2026)

Friday, September 18, 2026 — It's Friday, which means the nxpc theme calendar says it's time for the Ultimate Local LLM Box: the dream machine that runs 70B-parameter models on your desk, with your data never leaving the house.

And I've got news for you. The local AI hardware market has quietly flipped upside down in 2026. The RTX 5090 — the card everyone lusted over last year — now trades hands at $6,143 USD on Amazon. Meanwhile, a single dual-GPU Intel Arc Pro B60 card with 48GB of VRAM just sold for $1,939. That's three times the memory for one-third the price. The question is no longer "which GPU is fastest?" It's "what are you actually optimizing for: speed, or capacity?"

Let's build the answer.


The Physics Lesson That Determines Everything

If you take one thing from this post, make it this: LLM token generation is memory-bandwidth-bound, not compute-bound. Your GPU doesn't need to "think" harder to spit out the next token — it needs to move the entire model's weights past its memory controllers, over and over, for every single token. Bandwidth is king.

The RTX 5090's 1,792 GB/s of GDDR7 bandwidth is why it dominates this chart — and why the numbers below track that ratio almost perfectly:

🚀 Tokens-Per-Second Cheat Sheet (September 2026)

Hardware Model / Quant Speed Source
RTX 5090 (32GB, 1,792 GB/s) Llama 3.1 8B via Ollama ~264 tok/s vrlatech.com
RTX 5090 Qwen3 14B Q4_K @ 16k ctx 102.7 tok/s houtini.com
RTX 4090 (same test set) Qwen3 14B Q4_K 77% of the 5090 — tracking the bandwidth ratio houtini.com
RTX 5090 Llama 3.3 70B fully in VRAM 45+ tok/s Quantize Lab
RTX 5090 Llama 3.3 70B offloaded to system RAM 1–2 tok/s 💀 Quantize Lab
Dual RTX 5090 (vLLM) Qwen-family workloads 70–80 tok/s (~100–110 decode with MTP) vLLM 2026 benchmarks
Dual RTX 4090 (PCIe 4.0) Llama 3 70B Q4 85–90% of NVLink A100 tok/s r/LocalLLaMA
Intel Arc Pro B60 24GB (Vulkan) Qwen3.6 35B-A3B Q4_K — prefill pp8192 793.67 tok/s r/LocalLLM llama-bench
Intel Arc Pro B60 24GB (Vulkan) Qwen3.6 35B-A3B Q4_K — generation tg1024 54.58 tok/s r/LocalLLM llama-bench
Intel Arc Pro B60 (vLLM, peak load) long-form generation, 50–64 concurrent requests ~1,000 tok/s aggregate embeddedllm.com
Dual B70 + B60 (Vulkan, row split) Qwen 262k full context pp ~15 / tg ~10 tok/s bentech.substack.com
RX 7900 XTX (24GB, ROCm 7.2) Llama 3.1 8B ~96 tok/s (~75% of a 4090) localaimaster.com
RX 7900 XTX (ROCm 7.2) Llama 3 70B Q4 14–18 tok/s compute-market.com
Ryzen AI Max+ 395 (128GB unified) 70B-class Q4 ~8–14 tok/s (30–35 with tuned backends) Perplexity Sonar research
NVIDIA DGX Spark 70B Q4 35–45 tps presenc.ai
Mac Studio M5 Max 70B Q4 25–32 tps presenc.ai

Stare at rows four and five for a second. Same card, same model: 45+ tok/s in VRAM versus 1–2 tok/s offloaded to RAM. That's a 20–45x cliff. VRAM capacity isn't a luxury — it's the difference between "buttery chat" and "watching a turtle type."

VRAM Capacity Ladder — What Fits Where (Q4 quantization)

Model size Q4 footprint Fits on...
8B ~5GB Anything, even a 16GB card with room for context
14B ~9GB RTX 5080 (16GB), RX 9070 XT (16GB)
32B ~20GB RTX 5090 (32GB), 7900 XTX (24GB), single B60 (24GB)
70B ~40–45GB Dual 5090 (64GB), single MAXSUN B60 48GB card, dual B60s
123B-class (Qwen3.5 262k ctx) ~90GB+ Dual B70+B60, quad B60, 192GB Ziskind special

For 32B-and-up models, a 5090 is 2–3x faster than a 16GB card purely because it never leaves VRAM (localaimaster.com). And here's the multi-GPU secret the forum nerds already know: token generation across two cards over plain PCIe is fine — dual RTX 4090s on PCIe 4.0 hit 85–90% of NVLink-connected A100s running Llama 3 70B Q4 (r/LocalLLaMA). You don't need server silicon. You need a motherboard with spacing and a big PSU.


The Three Builds (All Prices Live: Sept 18, 2026)

MAXSUN Intel Arc Pro B60 48G Turbo — dual-GPU 48GB VRAM workstation card

Build 1: "The Capacity Heretic" — 48GB in ONE slot for $1,939

The most interesting card of 2026 isn't from NVIDIA. The MAXSUN Intel Arc Pro B60 48G Turbo crams two B60 GPUs (2× 24GB) onto a single dual-slot board. Alex Ziskind's July video "192GB of VRAM in One PC… The Cheap Way" showed two of these yielding 192GB without NVIDIA pricing — and on r/LocalLLM, a Gunnir B60 pushed 793.67 tok/s prefill / 54.58 tok/s generation on Qwen3.6 35B-A3B Q4 via Vulkan.

The catch: Intel's software stack is the fiddliest. Flash attention ON, a current llama.cpp SYCL build, the right Jinja chat template, and a physical batch size of -ub 2048 (the often-recommended 64 was four times slower in one user's sweep) turn this card "from disappointing to practical" (srmiles.com). Buy it to tinker, and it pays you back in VRAM-per-dollar nothing else touches.

Component USD (Amazon.com) CAD (Amazon.ca)
MAXSUN Arc Pro B60 48G Turbo (dual-GPU, 48GB) — B0HBWGBT3L $1,939.00 (only 8 left) $2,704.04 CAD (B0H9RXTWZT)
AMD Ryzen 9 9950X 16-Core — B0D6NNRBGP $499.99 (was $649.00) $779.88 CAD
ASRock PG 1600G 1600W ATX 3.1 — B0DNNZ9G46 $199.99 $287.98 CAD
Core total (GPU+CPU+PSU) ~$2,638.98 USD ~$3,771.90 CAD

Platform extras (X870E board, 96GB DDR5, NVMe, case) billed separately — and heads up, DDR5 is still riding the 2026 memory crisis, so budget accordingly.

ASUS ROG Astral RTX 5090 — the 32GB GDDR7 speed king

Build 2: "The Speed King" — Dual RTX 5090 (accept the tax)

64GB of GDDR7, 2× 1,792 GB/s, 70–80 tok/s on 70B-class workloads in vLLM (~100–110 decode with MTP). Nothing consumer touches it for single-box throughput. The price of that crown in September 2026, though… oof:

Component USD (Amazon.com) CAD (Amazon.ca)
ASUS ROG Astral RTX 5090 32GB OC — B0DS2WQZ2M ×2 $6,143.09 each → $12,286.18 $8,982.29 CAD each → $17,964.58
Cheapest alt: ASUS TUF RTX 5090 — B0DS2Z8854 $6,349.99 MSI Ventus 3X OC: $9,099.99 CAD
AMD Ryzen 9 9950X $499.99 $779.88 CAD
Seasonic Prime TX-1600 Titanium — B0C571LRNB $559.99 $827.04 CAD
Core total ~$13,346 USD ~$19,572 CAD

Yes, that's more than a used car. And yes, in Canada things get genuinely silly — a PNY RTX 5090 is listed at $15,499.12 CAD on Amazon.ca right now. I'm not making that up. Two of those buys you a decent condo down payment.

Build 3: "The ROCm Value Play" — RX 7900 XTX

AMD's 2026 story is real: ROCm has "caught up enough that Radeon is finally a real choice for local LLM inference" (runlocalmodel.com). The 7900 XTX does Llama 3.1 8B at ~96 tok/s, 70B Q4 at 14–18 tok/s, and a -15%/-20% power limit costs you only ~5% throughput while dropping temps ~10°C (localaimaster.com). Pro tip for the 24GB sweet spot: the ASRock Radeon AI PRO R9700 Creator 32GB ($1,699.99 USD, B0G1WZMKW6, 300+ bought this month) is the professional RDNA4 card people are slotting into LLM boxes.

Component USD (Amazon.com) CAD (Amazon.ca)
XFX RX 7900 XTX MERC310 24GB — B0BNLSW23M $1,249.99 (4.3★, 1,509 ratings) ⚠️ No sane XTX listing on .ca — see note
ASRock AI PRO R9700 32GB (AMD alternative) — B0G1WZMKW6 $1,699.99 GIGABYTE RX 9070 XT 16GB: $1,129.99 CAD
AMD Ryzen 7 9800X3D — B0DKFMSMYK $449.00 (was $479.00) $619.99 CAD
Corsair RM1000x ATX 3.1 — B0DJ1M9C62 $169.99 Corsair RM1200e: $294.99 CAD
Core total ~$1,868.98 USD ~$2,044.97 CAD (9070 XT path)

🇨🇦 Canadian reality check: the 7900 XTX has effectively vanished from Amazon.ca at sane prices — the cheapest AMD RDNA3 big-die listing I found was an XFX 7900 XT 20GB at $4,303.60 CAD, which is fantasy-land. In Canada, the value play is the RX 9070 XT at $1,129.99 CAD or the ASRock B60 Creator 24GB at $1,036.47 CAD (down from $1,088.29, Prime-eligible, 200+ bought/month on the US listing).

ASRock Intel Arc Pro B60 Creator 24GB — the blower-cooled budget VRAM card


The Software Stack: Where Free Performance Lives

Before you spend one more dollar on silicon, watch Alex Ziskind's "Your local LLM is 10x slower than it should be" — he took a rig from ~120 tok/s to 1,200+ with a software change alone. No new GPU. (youtube.com/watch?v=L9QZ97y9Exg)

The September 2026 state of play:

  • llama.cpp — best single-user latency, maximum compatibility (especially weird multi-GPU topologies). Benchmark with llama-bench -m model.gguf.
  • Ollama — the easiest on-ramp and what most homelabbers run (Techno Tim's Proxmox + Ollama + OpenWebUI stack is the classic), but it typically trails a tuned llama.cpp build. Bonus: its MLX engine (March 2026 preview, beefy June update) makes Apple Silicon sing.
  • vLLM — the throughput monster. If you're serving multiple users or batching, this wins, full stop. That ~1,000 tok/s aggregate on a B60? vLLM.

Multi-GPU trick for mismatched cards (from the B70+B60 writeup): --split-mode row with --tensor-split 32,24 proportions work by VRAM size. And measure cold vs. warm separately — a bad serving config can masquerade as a slow model.


The Verdict

Build VRAM Core Cost (USD) Best For
MAXSUN B60 48G one-card rig 48GB ~$2,639 Big-model tinkerers, max VRAM/$, patient souls
Dual RTX 5090 64GB ~$13,346 Speed demons, small-business inference, money-is-no-object
RX 7900 XTX / R9700 32GB 24–32GB ~$1,869–$2,319 ROCm value hunters, 8B–32B daily drivers
  • Just want it to work, 32B and below? Single RTX 5090 (if you can stomach $6,143) or the R9700 32GB.
  • Chasing 70B without a second mortgage? The MAXSUN B60 48G at $1,939 is the story of 2026 — 48GB, one slot, one power connector philosophy, and software maturity that's improving monthly.
  • In Canada? The B60 route ($2,704 CAD) is nearly your only escape from the $9,000+ 5090 tax.
  • Hate noise and complexity? A Ryzen AI Max+ 395 box with 128GB unified memory runs 70B Q4 at 8–14 tok/s, whisper-quiet — Ziskind called his "near silent LLM monster." Slower, but zero fuss.

The honest bottom line: in 2026, bandwidth buys speed, VRAM buys capability, and software tuning buys both for free. Maximize in that order for your budget, and don't buy a 5090 offloaded to 64GB of DDR5. Your tokens-per-second will thank you.


Sources

Prices captured September 18, 2026 ~14:00 UTC via live Amazon US/CA data. Stock counts were accurate at capture time ("only 8 left" has a way of becoming zero).

·