The HPC landscape

From room-filling machines to accelerated exascale systems

Lars Pastewka

Learning objectives

  • Recognize that high-performance computing has always meant machines that fill rooms
  • Name today’s German flagship systems and their key hardware ingredients
  • Locate yourself in the European HPC tier structure (and know where you will compute)
  • Explain why modern flagships are accelerator-based – and what that implies for programming

Machines have always filled rooms

MANIAC I (Mathematical Analyzer, Numerical Integrator, And Computer), 1952–1958. Image source: ed-thelen.org

  • MANIAC I and UNIVAC: the high-performance computers of the 1950s
  • High-performance computing is not new – only the hardware inside the room changes

HPC 2020–2025: two flagship philosophies

JUWELS booster (Jülich) and Hawk (HLRS Stuttgart). Source: https://www.top500.org

  • JUWELS booster (Jülich, 2020, Top500 #12): ~940 nodes, 2× AMD EPYC 7402 (24 cores), 512 GB RAM, 4× NVIDIA A100, 4× InfiniBand HDR
  • Hawk (HLRS Stuttgart, 2020, #30): ~5632 nodes, 2× AMD EPYC 7742 (64 cores), 256 GB RAM, CPU-only
  • Same era, two philosophies: GPU-accelerated vs CPU-only

HPC 2025–2030: accelerators win

JUPITER booster (Jülich) and Hunter (HLRS Stuttgart). Source: https://www.top500.org

  • JUPITER booster (2025, #6): 5884 nodes, each 4× NVIDIA Grace-Hopper GH200, NVLink 4, 4× InfiniBand NDR200 – ~500 M€
  • Hunter (HLRS, 2025, #61): 188 APU nodes with 4× AMD Instinct MI300A + 256 CPU nodes, HPE Slingshot
  • Herder (2027): ~4500 nodes, ~200 M€

Takeaway: the flagship era is accelerated

  • Modern flagship machines are accelerator-based – CPU-only flagships are history
  • The price tags are of order 10⁸ € per machine
  • Whoever wants to use these machines must program GPUs

Where is Tier I in Germany?

The three German Tier-I (national) centers: Jülich, HLRS Stuttgart, LRZ Munich. Background map © 1999 maps.com

  • Jülich Supercomputing Centre (JUWELS, JUPITER)
  • HLRS Stuttgart (Hawk, Hunter, Herder)
  • LRZ Munich (SuperMUC)

The tier pyramid: where will you compute?

European/German HPC tier structure. Center and project logos: GCS, Jülich, HLRS, LRZ, KIT, bwHPC-C5

  • Tier 0: European machines (PRACE)
  • Tier I: national centers (GCS: Jülich, HLRS, LRZ)
  • Tier II: regional/topical systems (bwHPC, HoreKa at KIT)
  • Tier III: local clusters – bwUniCluster, NEMO (Freiburg) ← this is where you will compute

Classic homogeneous HPC: IBM Blue Gene/Q

Blue Gene/Q build-up: chip → module → compute card → node card → midplane → rack → system (20 PF/s). Image source: IBM

  • Machines are built hierarchically: many identical units, replicated at every level
  • JUQUEEN (Jülich): 28 racks, 458,752 cores
  • Sequoia (LLNL): 96 racks, 1,572,864 cores

The accelerated present: NVIDIA Grace-Hopper

GH200 Grace-Hopper Superchip. Source: https://resources.nvidia.com/en-us-grace-cpu/grace-hopper-superchip

  • Grace CPU: LPDDR5X, up to 480 GB, ~500 GB/s
  • Hopper GPU: HBM3, 96–144 GB, ~4000–4900 GB/s
  • NVLink-C2C between them: 900 GB/s; NVLink 4 to other GPUs; PCIe-5 for IO

Remember this number: 10×

  • GPU memory bandwidth (~4000–4900 GB/s) is roughly 10× the CPU’s (~500 GB/s)
  • This gap is the reason GPUs dominate memory-bound scientific computing
  • We will meet this number again in the memory performance lecture

The vendor landscape

Current accelerator architectures: NVIDIA Grace-Hopper, AMD Instinct, NEC Aurora. Vendor sources on slide

  • NVIDIA Grace-Hopper → programmed with CUDA
  • AMD Instinct MI300A → programmed with HIP/ROCm
  • NEC Aurora vector engines → yet another model
  • Many vendors, many programming models → we need hardware abstraction (next lecture: Kokkos)

Summary

  • HPC has always existed – machines fill rooms, from MANIAC I (1952) to JUPITER (2025)
  • Modern flagships are accelerator-based; budgets are of order 10⁸ €
  • The tier pyramid organizes access: Tier 0 (EU) → Tier I (GCS) → Tier II (bwHPC) → Tier III (bwUniCluster, NEMO) – you compute on Tier III
  • Machines are built hierarchically; today each node carries several GPUs
  • GPU memory bandwidth exceeds CPU bandwidth by ~10×
  • Many vendors and programming models → motivation for hardware abstraction

Reading