GPU Architecture Explained: Cores, Shaders, and Clusters

GPU architecture is the structural and operational design that defines how a graphics processor organizes its cores, memory, and execution units to render graphics and run parallel computation. The architecture covers the streaming multiprocessors or compute units that group the shader cores, the specialized RT and Tensor cores, the memory hierarchy of registers, caches, and VRAM, and the process node the chip is built on. Nvidia ships architectures named Ada Lovelace and Blackwell, while AMD ships the RDNA family, each revising core counts, cache, and efficiency over the prior generation.

The arrangement of these blocks determines performance per watt, ray-tracing capability, and machine-learning throughput. This article defines GPU architecture, describes streaming multiprocessors and compute units, contrasts shader cores with RT and Tensor cores, explains the memory hierarchy, reviews the recent architecture generations and process nodes, and shows how architecture affects efficiency. A table lists the core types by function so a reader can see how each unit contributes to rendering and compute.

What Is GPU Architecture?

GPU architecture is the design that defines how a graphics processor arranges its cores, memory, and execution units. The architecture specifies the number and grouping of shader cores, the specialized hardware for ray tracing and machine learning, the cache and memory layout, and the way work is scheduled across the chip. The design determines how the GPU runs the parallel workloads described in the overview of how GPUs work.

Nvidia, AMD, and Intel each maintain their own architecture families, and each new generation revises the design to raise performance per watt. A GPU architecture has two layers: the high-level organization of cores and memory, and the low-level circuit design built on a specific manufacturing process node. The architecture sets the ceiling for what the video memory subsystem and shader cores achieve together, so two cards with similar core counts can perform differently when built on different architectures.

What Are Streaming Multiprocessors and Compute Units?

Streaming multiprocessors and compute units are the building blocks that group shader cores and shared resources inside a GPU. Nvidia calls its block a streaming multiprocessor, abbreviated SM, and AMD calls its block a compute unit, abbreviated CU. Each block contains a set of shader cores, a scheduler, a register file, and a slice of L1 cache, and the GPU contains many such blocks.

A streaming multiprocessor on a recent Nvidia card holds 128 CUDA cores plus RT and Tensor cores, while an AMD compute unit holds 64 Stream Processors plus ray accelerators. The GPU scheduler distributes work across all the blocks so thousands of cores run in parallel, the mechanism the parallel rendering pipeline depends on. Scaling a GPU up or down across a product line mostly means adding or removing these blocks, so a flagship card contains many more streaming multiprocessors than a budget card on the same architecture.

What Is the Difference Between Shader, RT, and Tensor Cores?

Shader cores, RT cores, and Tensor cores differ in the type of calculation each unit accelerates. Shader cores handle the general floating-point and integer math of rendering, RT cores accelerate the ray-intersection math of ray tracing, and Tensor cores accelerate the low-precision matrix math of machine learning.

Each core type exists because dedicated hardware runs its specific workload far faster than general shader cores. The table below lists the core types by function.

The shader cores carry the bulk of rasterized rendering, while the RT cores activate when ray tracing is enabled and the Tensor cores power DLSS and AI tasks. The balance of these core types defines a card’s strengths, which is why the Nvidia versus AMD comparison weighs ray-tracing and upscaling hardware separately from raster shader count. Each new architecture generation refines all three core types.

How Does the GPU Memory Hierarchy Work?

The GPU memory hierarchy works by placing small fast memories near the cores and large slow memory farther away. The hierarchy moves from registers inside each core, through L1 cache in each streaming multiprocessor, to a shared L2 cache, and finally to the VRAM on the card.

Each level trades capacity for speed, keeping frequently used data close to the shader cores to reduce stalls. The levels of the GPU memory hierarchy are listed below:

• Registers are the fastest storage, private to each shader core, holding the operands of in-flight calculations.
• L1 cache and shared memory sit inside each streaming multiprocessor, holding data shared among the cores in that block.
• L2 cache is shared across the whole GPU, reducing trips to VRAM; AMD’s Infinity Cache and Nvidia’s large L2 expand this level.
• VRAM is the large GDDR6 or GDDR6X memory on the card, holding textures, frame buffers, and geometry.

Recent architectures enlarge the on-chip caches to cut VRAM traffic, because cache bandwidth far exceeds memory bandwidth. AMD’s Infinity Cache and Nvidia’s expanded L2 reduce how often the GPU reaches the slower video memory, which raises effective bandwidth and efficiency without a wider memory bus.

What Are the Recent GPU Architecture Generations?

The recent GPU architecture generations are Nvidia Ada Lovelace and Blackwell, and AMD RDNA 3 and RDNA 4. Each generation revises core counts, cache sizes, ray-tracing throughput, and manufacturing process to raise performance per watt over the prior design. Nvidia’s Ada Lovelace architecture powers the GeForce RTX 40 series with third-generation RT cores and fourth-generation Tensor cores, and Blackwell follows with further ray-tracing and AI gains.

AMD’s RDNA 3 architecture powers the Radeon RX 7000 series with a chiplet design that separates the graphics compute die from memory cache dies, and RDNA 4 refines ray tracing and efficiency. Each generation also adopts a newer process for the shader cores and revises the upscaling hardware. A buyer comparing cards weighs the architecture generation alongside core count, because a newer architecture often outperforms an older one at the same core count, a factor the graphics card selection guide addresses.

How Does the Process Node Affect a GPU?

The process node affects a GPU by setting how small and dense the transistors are, which governs efficiency and the number of cores that fit on a die. The process node, measured in nanometers such as 5 nm or 4 nm, describes the manufacturing technology from a foundry such as TSMC. A smaller node packs more transistors into the same area, so a GPU gains more shader cores and lower power per operation.

Nvidia’s Ada Lovelace uses a custom TSMC 4 nm node, while AMD’s RDNA 3 uses a 5 nm graphics die paired with 6 nm cache dies. A denser node lowers the energy each transistor switch consumes, which raises performance per watt and reduces the heat the GPU cooling system must remove. Process improvements account for a large share of the gains between generations, alongside the architectural redesign of the cores and cache.

How Does Architecture Affect Efficiency?

Architecture affects efficiency by determining how much performance the GPU delivers per watt of power. Performance per watt depends on the process node, the cache design, the clock-and-voltage curve, and how effectively the architecture keeps the shader cores busy. A larger on-chip cache, such as AMD Infinity Cache or Nvidia’s expanded L2, cuts power-hungry trips to VRAM, raising efficiency without a wider memory bus.

A more efficient architecture reaches the same frame rate at lower power, which lowers heat and the cooling demand of the card and enables quieter or smaller designs. Efficiency also shapes laptop and integrated graphics designs, where a fixed power budget rewards the most efficient architecture. Each generation improves efficiency through the combined effect of a newer process node, larger caches, and a refined core design, which is why a new mid-range card can match an older high-end card while drawing less power.

How Does GPU Architecture Differ From CPU Architecture?

GPU architecture differs from CPU architecture in how the chip allocates transistors between control logic and arithmetic cores. A CPU architecture devotes a large share of its transistors to control logic, branch prediction, and large caches that finish a single sequential task quickly with low latency. A GPU architecture devotes most of its transistors to thousands of simple arithmetic cores grouped into streaming multiprocessors, maximizing throughput on parallel work rather than the latency of one task.

The cores and threads of a CPU number in the dozens, while the shader cores of a GPU number in the thousands, a contrast the comparison of GPU and CPU design details. The GPU architecture also uses a wider memory bus and higher-bandwidth video memory to feed the many cores, while the CPU uses lower-bandwidth but lower-latency system memory.

The architectural divergence explains why the two processors handle different workloads: the CPU runs branch-heavy serial logic, and the GPU runs data-parallel math. Both architectures share the concept of a cache hierarchy and a clock-and-voltage curve, but the balance of resources differs sharply.

How Do GPU Generations Improve Over Time?

GPU generations improve over time through the combined effect of a denser process node, a redesigned core layout, larger caches, and refined specialized hardware. Each new architecture generation raises performance per watt rather than only raw performance, so a new mid-range card can match an older high-end card at lower power. The improvements arrive from several directions, listed below:

• Process node shrinks pack more transistors into the same die area, raising core counts and lowering power per operation.
• Core redesigns raise the work each shader core completes per clock and improve how the scheduler keeps the cores busy.
• Larger on-chip caches, such as AMD Infinity Cache and Nvidia’s expanded L2, cut power-hungry trips to VRAM.
• Improved RT and Tensor cores raise ray-tracing and machine-learning throughput, widening the gap in those workloads each generation.
• Higher memory bandwidth from faster GDDR6X and wider buses keeps the growing core count supplied with data.

The cumulative effect means a buyer weighs the architecture generation alongside core count and video memory, because a newer architecture often delivers more frames at lower power than an older design with similar specifications. The Nvidia versus AMD comparison shows how each brand’s latest generation advances ray tracing, upscaling, and efficiency together.

Key Takeaways

• GPU architecture defines how the chip arranges cores, memory, and execution units for graphics and parallel compute.
• Streaming multiprocessors and compute units group shader cores with schedulers, registers, and L1 cache as the GPU’s building blocks.
• Shader, RT, and Tensor cores accelerate general math, ray tracing, and machine learning respectively.
• The memory hierarchy runs from registers through L1 and L2 cache to VRAM, keeping data close to the cores.
• Architecture generations and process nodes, such as Ada Lovelace, Blackwell, and RDNA on 4 nm and 5 nm, drive efficiency gains.

Last Thoughts on GPU Architecture

GPU architecture defines a graphics processor at two layers: the organization of streaming multiprocessors, shader cores, RT cores, and Tensor cores, and the circuit design built on a specific process node. The memory hierarchy of registers, L1 and L2 cache, and VRAM keeps data close to the cores, while larger caches and smaller nodes raise performance per watt each generation.

Nvidia Ada Lovelace and Blackwell and AMD RDNA show how architecture, not core count alone, sets ray-tracing, upscaling, and efficiency. Readers can continue with the explanation of how GPUs work, the guide to video memory, or the Nvidia versus AMD comparison, and the computer hardware guide places the architecture within the full system.