Every time you read AI news, you hear that “some company just bought tens of thousands more GPUs.” Why GPUs, and not the more impressive-sounding CPU?
This piece explains the GPU in plain words: what it is, why large-scale AI usually runs on it, who leads the market, and how the generations from H100 to Vera Rubin differ. It’s the beginner version of Gate 1, “AI chips,” in The AI Hardware Supply Chain, End to End.
What Is a GPU?
GPU stands for Graphics Processing Unit. It was originally designed for games and 3D graphics. Its defining trait is a huge number of small, simple cores that can handle thousands of similar calculations at once.
Here’s an analogy: a CPU is like a math professor who focuses on one hard problem at a time; a GPU is like a thousand schoolchildren who only know addition and subtraction. Give each child a small piece, and together they finish much faster. What AI needs is exactly that: hundreds of millions of simple problems solved at once.
Core-Data Snapshot
Market-share figures are estimates from research or policy bodies, so read them for order of magnitude.
| Topic | Figure | Timing / Nature |
|---|---|---|
| NVIDIA share of AI GPUs | About 80%-plus (cloud in-house ASICs counted separately; varies by measure) | 2025-2026, research / policy-body estimate |
| NVIDIA data-center revenue (full year) | About $193.7B for FY2026 | NVIDIA official earnings |
| NVIDIA data-center revenue (quarter) | $89B in Q2 FY2027 (through 2026-07-26), up 117% year over year | NVIDIA official earnings |
How a GPU Differs from a CPU
The difference is in the number of cores and how the work is split. A CPU has few cores, but each one is strong and versatile. It’s good at step-by-step work that needs decisions along the way, such as operating systems and program logic. A GPU packs in many smaller cores and aims for total throughput, doing many things at once. That suits matrix math, image processing, and deep learning.
Inside an AI server, the two work as partners: the CPU moves data and manages the program and the network, while the GPU runs the model’s heaviest math. For a fuller comparison, read GPU vs CPU.
Why Large-Scale AI Usually Uses GPUs
Training and running AI models comes down to massive matrix math happening at the same time. Two kinds of cores in a GPU matter most. CUDA Cores are many small general-purpose units that handle ordinary parallel work. Tensor Cores are built to speed up matrix multiplication, with support for low-precision formats like FP16, FP8, and FP4 (fewer bits in exchange for faster math).
The GPU also needs fast HBM (High Bandwidth Memory) next to it to feed model weights and data quickly enough, so the cores don’t sit idle waiting. For more on this memory, see the HBM gate.
The AI GPU Market: Who Supplies It
This gate is highly concentrated in NVIDIA. Its moat isn’t only the chip but also the CUDA software ecosystem: the code developers have written runs on it, and moving everything to another vendor is costly.
AMD is the main challenger. Its Instinct MI350 series is already shipping in the cloud, and in July 2026 it launched the MI455X (432GB of HBM4 per GPU) and Helios, a full rack of 72 GPUs. Cloud giants like Google, AWS, and Microsoft build their own ASICs to cut costs and stand out inside their own clouds. For that thread, see the ASIC gate.
Keep in mind that market share and revenue change with new products, earnings reports, and how things are measured. This section describes the industry landscape, not a view on any single stock.
A Tour of the Mainstream AI GPU Generations
| Generation / Product | Positioning | Memory |
|---|---|---|
| H100 (Hopper) | Earlier training / inference workhorse | 80-94GB HBM |
| H200 (Hopper) | Hopper memory upgrade | About 141GB HBM3e |
| B200 (Blackwell) | Blackwell-generation workhorse | About 180-192GB HBM3e (by SKU) |
| B300 (Blackwell Ultra) | Even more memory | About 288GB HBM3e |
| GB300 NVL72 | Full-rack system: Grace CPUs + 72 Blackwell Ultra GPUs | About 20TB per rack |
| Vera Rubin | The generation after Blackwell | 288GB HBM4 per GPU; 20.7TB per NVL72 rack |
As of August 2026, Blackwell still makes up the vast majority of NVIDIA’s revenue; Vera Rubin started volume shipments in August 2026.
Two trends matter. First, memory keeps getting bigger and bandwidth keeps rising. Second, the focus is moving from single chips to full-rack systems that link dozens of GPUs with high-speed interconnects into one supercomputer. For details on the flagship generation, read the Blackwell gate.
Taiwan’s Role in This Gate
NVIDIA designs its GPUs in the United States, but making them relies heavily on Taiwan. Blackwell is built on TSMC’s custom 4NP process. Advanced packaging such as CoWoS then joins the GPU and HBM together, and Taiwanese firms such as Foxconn, Quanta, and Wistron assemble them into full-rack AI servers. The Vera Rubin production partners NVIDIA announced in May 2026 also include these Taiwanese firms. From silicon wafer to working system, most of a GPU’s journey happens in Taiwan.
Penchan’s summary
A GPU’s strength is parallel computing: many small cores working at once. That fits AI’s need to run a massive number of simple calculations together, which is why large-scale AI usually uses GPUs.
NVIDIA leads this gate, with AMD and cloud in-house ASICs chasing from two directions. Behind every GPU sit TSMC’s processes, advanced packaging, and Taiwan’s server assembly. Understand the GPU, and you have your ticket to understanding the whole AI hardware supply chain.