Nvidia CEO Jensen Huang unveiled a new kind of ethernet switch, the Spectrum-X, at the Computex computer technology conference in Taipei, Taiwan. The Spectrum-X is purpose-built to provide lossless packet transmission for artificial intelligence (AI) workloads. Huang described it as “the world’s first high-performance ethernet for AI.” The Spectrum-4 is the first iteration of the Spectrum-X family, which is built to address the bifurcation of data centers into two forms: AI factories and AI cloud. The switch has the potential to change the ethernet networking market, which is currently dominated by chip maker Broadcom. The Spectrum-X family is designed to turn every data center into a generative AI data center.
The Spectrum-X family of ethernet is the first in the market that brings the capabilities of high-performance computing into the ethernet market. The Spectrum-4 chip, the first iteration of Spectrum-X, is the world’s first 51Tb/sec Ethernet switch built specifically for AI networks. The chip works in conjunction with Nvidia’s BlueField data processing unit (DPU) chips that handle data fetching and queueing, and Nvidia fiber-optic transceivers. The switch can route 128 ports of 400-gigabit ethernet, or 64 800-gig ports, from end to end, the company said. Huang held up the silver Spectrum-4 ethernet switch chip on stage, noting that it’s “gigantic,” consisting of a hundred billion transistors on a 90-millimeter by 90-millimeter die built with Taiwan Semiconductor Manufacturing’s “4N” process technology. The part runs at 500 watts, said Huang.
Nvidia’s Spectrum-X family is built to address the bifurcation of data centers into two forms: AI factories and AI cloud. AI factories are facilities that cost hundreds of millions of dollars for the most powerful graphics processing units (GPUs), based on Nvidia’s NVLink and Infiniband, used for AI training, serving a small number of very large workloads. AI cloud, on the other hand, is multi-tenant, based on ethernet, and handles hundreds and hundreds of workloads for customers simultaneously, and which is focused on things such as serving up the predictions to consumers of AI, which will be served by the Spectrum-X. The Spectrum-X is able to “spread traffic across the network in the best way,” said VP Shainer, using “a new mechanism for congestion control,” that averts a pile-up of packets that can happen in the memory buffer of network routers. “We use advanced telemetry to understand latencies across the network to identify hotspots before they cause anything, to keep it congestion-free.”
The Spectrum-X family is designed to turn every data center into a generative AI data center. The chip has the potential to change the ethernet networking market, which is currently dominated by Broadcom. The vast majority of switch silicon is supplied by Broadcom, which sells the switches to networking equipment makers such as Cisco Systems, Arista Networks, Extreme Networks, Juniper Networks, and others. These companies have been expanding their equipment to better handle AI traffic. The Spectrum-X family is built to address the bifurcation of data centers into two forms: AI factories and AI cloud. The Spectrum-X is able to “spread traffic across the network in the best way,” using “a new mechanism for congestion control,” that averts a pile-up of packets that can happen in the memory buffer of network routers.
In addition to the Spectrum-X, Huang’s keynote featured a new model in the company’s “DGX” series of computers for AI, the DGX GH200. The GH200 is the first system to ship with what the company calls its “superchip,” the Grace Hopper board, which contains on a single circuit board a Hopper GPU, and the Grace CPU, a CPU based on ARM instruction set that is meant to compete with x86 CPUs from Intel and Advanced Micro Devices. The first iteration of Grace Hopper, the GH200, is “in full production,” said Huang. The DGX GH200 combines 256 of the superchips, said Nvidia, to achieve a combined 1 exaflops — ten to the power of 18, or, one billion, billion floating point operations per second — utilizing 144 terabytes of shared memory. The computer is 500 times as fast as the original DGX A100 machine released in 2020, according to Nvidia.
Nvidia also unveiled MGX, a reference architecture for system makers to quickly and cost-effectively build 100+ server variations. The first partners to use the spec are ASRock Rack, ASUS, GIGABYTE, Pegatron, QCT and Supermicro, with QCT and Supermicro to be first to market with systems, in August, said Nvidia. The entire keynote can be seen as a replay from the Nvidia Web site. MGX is a reference architecture for computer system makers to quickly and cost-effectively build over 100 server variations using Nvidia chips. QCT and Supermicro will be the first to market with systems, in August, said Nvidia.
Nvidia’s new products, including the Spectrum-X, the DGX GH200, and MGX, are designed to help the company stay ahead of the competition in AI computing. The company’s new ethernet technology is purpose-built to provide lossless packet transmission for AI workloads, which is essential for the future of AI. The DGX GH200 is a new class of large-memory AI supercomputer for giant generative AI models. The MGX is a reference architecture for system makers to quickly and cost-effectively build 100+ server variations using Nvidia chips. With these new products, Nvidia is well-positioned to continue leading the way in AI computing.
