Hardware & Compute

GPUs

The parallel processing chips that made modern AI possible

What it is

GPUs (Graphics Processing Units) are processors designed to perform thousands of simple mathematical operations in parallel. Originally built for rendering graphics, they were repurposed for AI training when researchers discovered their massive parallelism was ideal for transformer computation.

NVIDIA dominates the data center GPU market with their A100 and H100 chips, which cost $20,000–$80,000 each. NVIDIA's sustained dominance is partly due to CUDA, their programming framework, which has been optimized for AI workloads for over a decade and has massive developer adoption.

A modern AI training cluster uses tens of thousands of these GPUs interconnected with specialized high-bandwidth networking (NVLink, InfiniBand).

Why it matters

GPU availability is literally the bottleneck for AI development. The reason there are only a few organizations capable of training frontier models is that you need access to thousands of H100s, costing billions of dollars. This hardware constraint shapes every aspect of the industry, investment patterns, regulatory policy (chip export controls), and competitive strategy.

Related concepts

TPUs Training Costs Why Accelerators Matter

Resources

Why GPUs Are Great for AI

blogs.nvidia.com· Straight from the source, parallel processing, CUDA, history of GPUs in AI

10 min

CPU vs. GPU for Machine Learning

ibm.com· Clear comparison with practical tradeoffs (cost, energy, real-world ML applications)

8 min

Nvidia CUDA in 100 Seconds

youtube.com· Fast-paced explainer covering CUDA, why GPUs suit AI, CPU vs GPU core differences

3 min

How do Graphics Cards Work? Exploring GPU Architecture

youtube.com· Stunning 3D-animated deep dive into GPU hardware, CUDA cores, tensor cores, SIMD architecture, parallel processing

28 min