x/nxpc · 
Friday Deep-Dive: Ultimate Local LLM Box — October 2, 2026
It's 2 a.m. You're three hours into trying to run a 70B model on your gaming PC, and the fans are screaming like a jet engine while llama.cpp spits out tokens at a painful 4 tok/s. Meanwhile, some person on Reddit is casually generating entire essays at 30+ tok/s on a single card you've never considered buying. What do they know that you don't?
Here's the punchline of 2026's local AI scene: the math has flipped. Thanks to RTX 5090 pricing living in the stratosphere, NVIDIA's workstation flagship — the RTX PRO 6000 Blackwell with 96GB of VRAM — is now cheaper per gigabyte than strapping together two consumer 5090s. And it's not even close. Let's build the dream box.
Local LLM performance boils down to two numbers: VRAM capacity (does the model fit?) and memory bandwidth (how fast do tokens come out?). Once your weights + KV cache live entirely in VRAM, decode speed is essentially bandwidth-bound. That's why the RTX 5090's 1,792 GB/s of GDDR7 bandwidth can beat an A100 40GB datacenter card in many quantized inference workloads, as BIZON's testing roundup notes.
| Spec | RTX PRO 6000 Blackwell | RTX 5090 (×2) | Used RTX 3090 (×2) |
|---|---|---|---|
| VRAM | 96GB GDDR7 ECC | 64GB GDDR7 (32GB ea) | 48GB GDDR6X (24GB ea) |
| Memory bandwidth | 1,792 GB/s per card | 1,792 GB/s per card | ~936 GB/s per card |
| CUDA cores | 24,064 | 21,760 ×2 | 10,496 ×2 |
| Board power | 600W (300W Max-Q) | 575W ×2 | 350W ×2 |
| Llama 70B Q4 single-stream | ~24.6–34 tok/s | ~35 tok/s (2×, sharded) | ~15–20 tok/s (est.) |
| Fits on the card? | 120B at Q4 | 70B Q4 split across cards | 70B Q4 barely |
This is the dream. RTX 5090 silicon with 96GB of ECC GDDR7 on a 512-bit bus at 1.79 TB/s. Run the numbers with me: a Llama-class 70B model at Q4 quantization eats roughly 38–40GB for weights, leaving ~58GB free for KV cache — enough for genuinely enormous context windows. Per the PulsedMedia wiki and VRLA Tech's deep-dive, this is the first consumer-installable card where single-GPU 70B inference is a practical production deployment, not a party trick. Models up to ~120B at Q4 fit entirely on the card.
Token throughput backs it up. Cited 70B Q4 runs on the PRO 6000 Blackwell range from 24.6 tok/s (LLM Configurator), to a 28.4 tok/s median (Modelfit), up to 31.8–34 tok/s in independent reviews. The spread is normal — batch size, context length, and runtime all move the needle — but the floor here is faster than most people's typing speed, which is the only benchmark that matters at 2 a.m.
Two 5090s give you 64GB across cards and an aggregate 3,584 GB/s of bandwidth. On Llama 3.1 70B Q4, aggregated benchmarks show ~35 tok/s on 2×5090 and ~62 tok/s on 4×5090. So yes — the dual-GPU path can beat the single PRO 6000 on throughput. The catch: you need software that actually uses tensor parallelism. llama.cpp's --tensor-split balances layers across GPUs but does not give you NVLink-style tensor-parallel compute; for genuine TP, vLLM or GPUStack are the mature options as of mid-2026, per OpenSourcesAI's multi-GPU guide. More GPUs = more software friction, more power, more heat.
The classic. A 70B at Q4_K_M needs ~40GB+, and two 24GB cards get you there. Core Lab's late-2026 tier list still crowns the dual-3090 build "The Serious Researcher" pick. It won't set speed records — the 936 GB/s per-card bandwidth is half a 5090's — but it runs the models, and it's the cheapest door into 70B-land.
Before you spend a dollar on silicon, watch Alex Ziskind's "Your local LLM is 10x slower than it should be" — he took a rig from ~120 tok/s to 1,200+ tok/s with zero hardware changes. The usual suspects: KV cache quantization (Q8 halves cache memory), FlashAttention enabled, correct --n-gpu-layers, and picking the right backend (Ollama for convenience, raw llama.cpp for control, vLLM for concurrent serving and true tensor parallelism). Ziskind's llama.cpp-vs-Ollama walkthrough is the perfect companion watch.
Here's where it gets spicy. Amazon US 5090 listings are all third-party scalper pricing right now — "Only 1–3 left in stock, order soon" across the board.
| Product | Price | Stock / Rating |
|---|---|---|
| ASUS TUF Gaming RTX 5090 32GB OC | $7,398.00 (from $6,899.99) | 4.4★ (265), 3 left |
| GIGABYTE AORUS RTX 5090 Infinity 32G | $7,177.68 | 1 left |
| ASUS ROG Astral RTX 5090 OC | $7,599.99 | 3 left |
| MSI RTX 5090 SUPRIM SOC | $8,549.00 | 4★ (42), 1 left |
| PNY RTX PRO 6000 Blackwell 96GB | $17,986.96 (from $15,929.99) | 5 offers, 1 left |
| RTX PRO 6000 Blackwell Workstation Ed. | $19,999.99 | 4.3★ (25) |
| RTX PRO 6000 Blackwell max-Q | $17,499.99 | In stock |
| NVIDIA RTX 3090 FE (Renewed) | $1,949.99 (from $1,899.99) | 4.1★ (39), 13 left |
| EVGA RTX 3090 FTW3 Ultra | $1,899.99 | 4.4★ (110) |
| ASUS ROG Strix RTX 3090 OC (Renewed) | $1,849.99 | 4.3★ (23) |
| MSI RTX 3090 Ventus 3X (Renewed) | $1,879.99 | 4.4★ (17) |
| NVIDIA Titan RTX (Renewed) | $1,149.97 | 24GB budget wildcard |
Newegg and B&H list the PRO 6000 Blackwell Workstation Edition at $15,999+ per ThunderCompute's October 2026 pricing tracker, with used listings spanning $14,980–$18,850.
| Product | Price | Notes |
|---|---|---|
| ASUS TUF RTX 5090 32GB OC | $9,216.17 CAD | 4.2★ (60) |
| ASUS ROG Astral RTX 5090 32GB | $10,299.99 CAD (from $9,299.99) | 4.2★ |
| MSI RTX 5090 Ventus 3X OC | $10,299.99 CAD | In stock |
| RTX PRO 6000 Blackwell | Not listed on Amazon CA | (PRO 5000 72GB: $13,890.71 CAD; PRO 5000 48GB: $14,649.76 CAD) |
| PNY RTX A6000 48GB | $6,999.99 CAD | Ampere-era 48GB sleeper |
| RTX 3090 (Renewed, various) | $2,699.99–$3,389.99 CAD | EVGA FTW3 $3,050.99; PNY XLR8 from $2,699.99 |
| Titan RTX (Renewed) | $2,033.99 CAD |
| Path | Cost | VRAM | $/GB |
|---|---|---|---|
| 2× used RTX 3090 | $3,699.98 | 48GB | $77.08/GB |
| 2× RTX 5090 | $13,799.98 | 64GB | $215.62/GB |
| 1× RTX PRO 6000 Blackwell | $15,929.99 | 96GB | $165.94/GB |
Read that again: the workstation card is ~23% cheaper per GB than dual consumer 5090s — and it's one card, one slot, one 600W connector, ECC memory, and no tensor-split finagling. North of the border the dual-3090 path lands at ~$112.50/GB CAD ($5,399.98 for 48GB), which remains the Canadian value king for 70B-class models.
--tensor-split, KV cache at Q8, and FlashAttention on. That's a legitimate 70B box for under $4K USD — pair it with a used Threadripper or a 9950X and 128GB of RAM for model loading headroom.The 2026 lesson isn't "buy the biggest card." It's that software optimization is the cheapest GPU upgrade you'll ever make, and the VRAM market has officially inverted the workstation-vs-consumer value equation. The silicon lottery giveth, the scalpers taketh away.