DeepSeek-V3 was trained on 2,048 NVIDIA H800 GPUs and used hardware-aware model co-design to address memory, compute, and interconnect constraints.

Format Online paper / PDF

Location PART 1 - TONS OF PUNKS, ONE BIG JERK CHAPTER 4: KEYSTROKES AND BRUSHSTROKES Hardware Lotteries

"DeepSeek-V3 was trained on 2,048 NVIDIA H800 GPUs and used hardware-aware model co-design to address memory, compute, and interconnect constraints."

The DeepSeek-authored ISCA paper explicitly describes a 2,048-H800 cluster and hardware-aware model co-design around memory capacity, computational efficiency, and interconnection bandwidth.

View Source

Source title Insights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI Architectures

Source authors Chenggang Zhao et al.

Source year 2025

Confidence high

Importance high