Here's a sentence I didn't expect to write in 2026: a team ran one of the most demanding open-source AI models on AMD GPUs, beat Nvidia on price-performance, and didn't write a single line of custom kernel code to do it.
If you've been following AI infrastructure for the past few years, you know how wild that is. AMD GPUs have been the "someday" option — great on paper, painful in practice. Want to run the latest model? Hope you enjoy writing CUDA-to-HIP porting scripts and debugging segmentation faults at 2 AM.
That "someday" might have just become "today."
Last week, inference platform Wafer dropped a blog post that made the GPU world sit up. They ran GLM-5.2 — Z-AI's 753-billion-parameter flagship model with a 1M-token context window — on a single node of AMD MI355X GPUs. Here's the scorecard:
| Metric | AMD MI355X | Nvidia B200 | Delta |
|---|---|---|---|
| Aggregate throughput | 2,626 tok/s | 3,192 tok/s | AMD -18% |
| Single-stream (10k in / 1.5k out) | 213 tok/s | N/A | — |
| Success rate (all loads) | 100% | 100% | Tie |
| Price per GPU | ~$20-30K | ~$45-55K | AMD ~50% cheaper |
So AMD is 18% slower — but roughly half the price. That means per dollar spent, you get more tokens from the AMD rig. In the world of inference at scale, that's the only number your CFO cares about.
The test workload was realistic, too: 20,000 input tokens, 1,000 output tokens, 60% cache hit rate. Not a cherry-picked microbenchmark. At every load level from 0.5 RPS to saturation at 2.4 RPS, the MI355X maintained 100% success with TTFT (time-to-first-token) staying under 2.2 seconds even at peak.
Here's where it gets genuinely interesting. Wafer engineer Ian Ye wrote something that stopped a lot of people mid-scroll:
"This time, we didn't write any custom kernels."
Three months ago, getting Qwen3.5 397B running on AMD required writing custom CUDA-compatible kernels from scratch. This time? Four steps, all using existing tools:
They compressed GLM-5.2 from bf16 to MXFP4 using AMD's Quark toolkit. Compared to Z-AI's official FP8 quantization, the quality loss was negligible — GSM8K math reasoning dropped by 0.01, GPQA-Diamond graduate-level reasoning by 0.02, and one benchmark (tau2) actually improved by 0.015. That's "lossless" in any practical sense.
Three candidates, two eliminated: vLLM didn't support the MXFP4 + GlmMoeDsa code path. ATOM degraded at long context. sglang worked out of the box with the quantized weights.
This is where AMD's "it mostly works now" era reveals itself. Speculative decode — where a helper model predicts tokens that the main model verifies, 3x-ing single-stream throughput — wasn't working on sglang's ROCm image. Two bugs stood in the way:
#include <cuda_runtime.h> hardcoded — no ROCm guard. Fix: add #ifdef USE_ROCM. One line.Combined: less than 10 lines changed, ~3x throughput gain.
GLM-5.2's MoE architecture (256 experts, 8 active) needs careful kernel selection. sglang's default AMD image was falling back to a slow heuristic path for fp4 MoE. Wafer manually selected the correct kernel for GLM's specific shapes (model_dim 6144, moe_inter 2048, E=256, topk=8). This one config change took aggregate throughput from 1,944 to 2,626 tok/s — a 35% jump.
No custom code. Just knowing which knob to turn.
The speed of this progression is what should make Nvidia executives pause their morning coffee.
| When | What | Custom Kernels Needed? |
|---|---|---|
| May 2026 | Qwen3.5 397B on AMD MI355X | Yes — custom CUDA-compatible kernels |
| June 2026 | GLM-5.2 on AMD MI355X | No — framework fixes only |
| June 2026 | GLM-5.2 Fast on Vercel AI Gateway | Production deployment |
That's a three-month compression from "heroic engineering effort" to "configure, fix two bugs, ship." The pace is accelerating because the ecosystem is accelerating — AMD Quark, sglang's ROCm support, the broader open-source Triton compiler stack are all maturing simultaneously.
And it's not just Wafer. A community developer got the full 753B GLM-5.2 running on 32 consumer RTX 4090s (four nodes) — the first time a DeepSeek-Sparse-Attention model has run correctly on Ada-generation consumer GPUs. If the model can be hacked to run on gaming cards, the software barrier is crumbling faster than anyone expected.
Let's talk dollars, because that's what ultimately drives infrastructure decisions.
| AMD MI355X | Nvidia B200 | |
|---|---|---|
| Est. unit price | $20,000–$30,000 | $45,000–$55,000 |
| VRAM | 288 GB HBM3e | 192 GB HBM3e |
| Price per GB VRAM | ~$83–$104 | ~$234–$286 |
| Cooling | Air or liquid | Liquid only |
| 3-year TCO (8-GPU server) | ~$369,000 | ~$602,000 |
| Cloud (on-demand) | $8.60/hr | $3.75–$8.19/hr |
The hardware math is brutal: AMD gives you 50% more memory at roughly half the price. For inference-heavy workloads — which is where the industry is sprinting — memory capacity directly determines how many GPUs you need per model deployment.
But there's a nuanced cloud story too. B200 spot and reservation pricing can be competitive ($2.25/hr on 36-month commit), while MI355X cloud availability is still limited to a handful of providers. For organizations with existing Nvidia infrastructure and CUDA-trained teams, the switching math isn't purely about hardware cost.
Still: the direction of travel is unmistakable. When a frontier model runs on AMD at 80% of Nvidia's speed for 50% of the cost, and the deployment story is "two bug fixes, no custom kernels," the economic gravity gets hard to ignore.
Let's not get carried away. Nvidia's moat has not collapsed. Here's an honest balance sheet:
The smartest framing comes from Wafer themselves:
"SOTA on AMD is becoming more a matter of support, not software. The CUDA moat is eroding in real time."
Translation: the question is no longer "can AMD run this model?" — it's "does anyone bother to make sure AMD runs this model?" And more people are bothering every month.
If you're paying Nvidia inference bills: You now have a credible alternative for single-node deployments. Start testing your workload on MI355X instances. The cost savings at scale are too large to ignore, and the software gap is closing fast enough that your next GPU refresh cycle may look very different from your last one.
If you're deploying on AMD: The support cycle for the next model release will likely be shorter than this one. The toolchain (Quark, sglang ROCm images, Triton) is accelerating. You're no longer a second-class citizen — you're an early adopter of where the industry is heading.
If you're a curious observer: Watch the Wafer→Vercel pipeline. GLM-5.2 Fast is already available on Vercel's AI Gateway (model: 'zai/glm-5.2-fast'), delivering 170-200+ tok/s with zero platform markup. This is the canary in the coal mine: when Vercel — the platonic ideal of "developer experience above all" — ships AMD-backed inference, the market has spoken.
The CUDA moat was never going to vanish overnight. It's a 20-year head start in software, ecosystem, and mindshare. But cracks don't need to be catastrophic to matter — they just need to be big enough for water to get through.
And right now, the water is flowing faster than anyone predicted.
Published July 6, 2026. Research based on primary sources from Wafer, Vercel, and verified hardware benchmarks.