Explain the differences between the CPU and the GPU

Design Philosophy and usage scenarios

Design Philosophy

CPU

• Emphasizes flexibility and generalizability — efficiently processes a wide variety of instructions/data types. Generalized.
• Designed for low latency — prioritize getting things done quickly, even one task at a time.
• Smaller number of cores, each very powerful with larger caches and complex logic units.
• Excels at branch prediction (分支预测): predicts the next needed instruction and fetches it in advance, minimizing wasted time.
  • 因为 CPU 极快，从内存取指令相对很慢。If CPU waited for every if to finish before fetching next, it would waste many cycles. So it guesses path A or B and preloads the predicted instructions.
• Instruction versatility (指令多样性/通用性): built to execute large instruction sets → ideal for general-purpose software (browsers, office, etc.).
  • 用途通用，可用指令概括范围广。

GPU

• Primary philosophy: high throughput (吞吐量). Optimized for tasks separable into smaller, independent units.
• Significantly more cores, each less powerful, designed for smaller tasks → massive parallel data processing.
• Optimized for SIMD operations — same instruction applied to many data elements at once.
  • SIMD = Single Instruction, Multiple Data.
    • 一条指令被”广播”给许多数据元素。
    • 图像、矩阵、向量任务本就是”一次处理很多一样的计算”。
    • GPU 同时让大量数据执行同一种运算 → 并行计算特别快。
• Designed to move data efficiently between cores and memory — prioritizes high bandwidth over complex per-core logic.
  • GPU 不追求每个 core 的复杂逻辑处理能力，而是依靠高带宽和海量并行单元来实现超高吞吐量 → 处理大量统一重复的数据计算时远远快于 CPU。

Usage Scenarios

CPU

• Running OS and managing system resources
• Executing general-purpose software tasks
• Decoding and handling user input
• Multitasking between applications

GPU

• Processing graphics; rendering images/videos for gaming and video editing
• Accelerating scientific algorithms and ML model training
• Encoding/decoding video streams
• Cryptocurrency mining

Core architecture, processing power, memory access, power efficiency

Core architecture

CPU — ISA (Instruction Set Architecture)

• Defines the fundamental operations a CPU can perform.
• Each ISA instruction specifies: arithmetic, data movement, logical ops, control flow changes, or system interactions.
• Unique to CPU: system management and complex branching instructions/predictions.

GPU — ISA

• Designed for extensive arithmetic and data movement; less emphasis on complex logic/control flow.
• GPU-unique instructions optimized for graphics/parallel data:
  • SIMD: one operation applied simultaneously across many data points.
  • Texture mapping/manipulation: handles pixel interpolation (像素差值, computing in-between colors) and texture fetching (从纹理图片里拿出某个位置的像素).

Processing Power

The ability of a device to perform computational tasks — how much work a CPU or GPU can do in a given time, directly impacting software performance.

Factors influencing processing power:

• Number of cores: more cores → more work in given time
• Clock speed: cycles per second; faster clock → faster single-core processing
• Thermal management: better cooling → sustained peak performance
• Power delivery: stronger supply → higher frequency + all-core operation → max performance

CPU

• Fewer but more powerful cores.
• High clock speeds + high IPC + branch prediction and out-of-order execution optimize sequential processing.
• Multithreading for efficient handling of multiple tasks and complex instructions.

GPU

• Hundreds to thousands of cores optimized for large-scale parallel processing.
• High memory bandwidth and specialized cores (e.g., tensor cores) for fast large-block processing.
• SIMD maximizes throughput.
• Individual cores lower-clocked/simpler but specialized for parallel execution → tremendous parallel power.

Memory Access

How processors retrieve and manipulate data in computer memory.

CPU

• Uses a memory hierarchy (L1, L2, sometimes L3 caches — SRAM) to manage data access.
• Designed to minimize memory latency (delay between request and data).
  • Advantage of SRAM (cache): frequently-used data/instructions stay close to CPU.
• Multi-core CPUs need cache coherence protocols (缓存一致性协议) for a consistent memory view across cores.

GPU

• Often uses unified memory architecture (内存统一架构) — CPU and GPU access a large shared memory pool.
• Equipped with high-bandwidth memory for massive parallel processing demands.
• Prioritizes memory throughput (高吞吐量) over low latency — must feed thousands of parallel cores simultaneously.

Power Efficiency

CPU

• Performance per watt benchmarks CPU efficiency — higher = more power-efficient.
• Modern CPUs use advanced power management adjusting to workload, e.g., Dynamic Voltage and Frequency Scaling (DVFS, 频率调整) reduces power when full power isn’t needed.
• Thermal Design Power (TDP) 热设计功耗: max heat a CPU generates under normal conditions that cooling is designed to dissipate. Efficient CPUs deliver more performance within lower TDP.

GPU

• High-performance computing/gaming GPUs emphasize power efficiency due to potentially high energy consumption.
• Improves efficiency by distributing tasks across many cores — reduces power per task vs. serial processing. 由于总输入功率 watt 不变, 分配到多个 cores 通过 parallel computing 完成更多 tasks → power efficiency 增加。
• Has energy-saving features like lowering clock speeds or powering down idle cores.
• Generally more power-efficient than CPUs for parallel tasks.

CPUs and GPUs working together: Task division, data sharing, coordinating execution

Task Division

• CPU: sequential and control-intensive tasks — system management, logic, decisions (OS, network, I/O).
• GPU: parallelizable, data-intensive tasks — matrix multiplications (ML), pixel processing (graphics), data analysis (scientific).

Data Sharing

• Challenge: Data must be shared. Initially in CPU memory → transferred to GPU via PCIe (Peripheral Component Interconnect Express, 外设互联快线) — potential bottleneck.
• Solution: Unified memory (内存统一) lets CPU/GPU access the same physical memory → simplifies sharing, reduces transfer overhead.

Coordinating Execution

Coordination uses programming models like CUDA and OpenCL to handle task division, memory management, and synchronization. Modern systems dynamically choose CPU or GPU based on workload for optimal performance/efficiency.