Musk recently stated that as Tesla prepares to launch Robotaxi in October, the AI team will "double down" on the Dojo supercomputer. Dojo is the cornerstone of Tesla's AI, specifically built for training FSD neural networks.
Musk recently visited Tesla's supercomputer cluster at the Texas Gigafactory. He described it as a system with about 100,000 H100/H200 GPUs and large-scale storage for video training of Full Self-Driving (FSD) and Optimus robots. In addition to NVIDIA GPUs, this supercomputer cluster is equipped with Tesla HW4, AI5, and Dojo systems, powered by a large 500-megawatt system for power and cooling.
Musk revealed that Tesla uses not only NVIDIA GPUs in its AI training systems but also its own AI computer Tesla HW4 AI (renamed AI4), in a ratio of about 1:2. This means there are approximately 90,000 H100 equivalents, plus about 40,000 AI4 computers. By the end of this year, Dojo 1 will have about 8,000 H100-equivalent computing power.
The total computing power of Dojo is expected to reach 100 exaflops by October 2024. Assuming one D1 chip can achieve 362 teraflops, Tesla would need over 276,000 D1 chips or over 320,000 NVIDIA A100 GPUs to reach 100 exaflops.
The D1 chip was first unveiled at Tesla AI Day in 2021, featuring 50 billion transistors and only palm-sized. In May this year, D1 chip production began, using TSMC's 7nm process node. To achieve higher bandwidth and computing power, Tesla's AI team fused 25 D1 chips into one tile, operating as a unified computer system. Each tile has 9 petaflops of computing power and 36 TB/s of bandwidth.
Six tiles form a rack, and two racks form a cabinet. Ten cabinets form an ExaPOD. Dojo will scale by deploying multiple ExaPODs, all of which together constitute the supercomputer.
Wafer-scale processors (like Tesla's Dojo and Cerebras' Wafer Scale Engine WSE) are much more performance-efficient than multi-processors. However, putting 25 chips together also poses significant challenges for voltage and cooling systems. Tesla is building a massive cooling system in Texas.
Additionally, Tesla is developing the next generation of Dojo chips.