[ad_1]
On Monday, Nvidia introduced the HGX H200 Tensor Core GPU, which makes use of the Hopper structure to speed up AI purposes. It is a follow-up of the H100 GPU, launched final yr and beforehand Nvidia’s strongest AI GPU chip. If extensively deployed, it may result in much more highly effective AI fashions—and sooner response instances for current ones like ChatGPT—within the close to future.
In keeping with specialists, lack of computing energy (typically referred to as “compute”) has been a main bottleneck of AI progress this previous yr, hindering deployments of current AI fashions and slowing the event of latest ones. Shortages of highly effective GPUs that speed up AI fashions are largely guilty. One strategy to alleviate the compute bottleneck is to make extra chips, however you may as well make AI chips extra highly effective. That second method could make the H200 a sexy product for cloud suppliers.
What is the H200 good for? Regardless of the “G” within the “GPU” title, knowledge middle GPUs like this sometimes aren’t for graphics. GPUs are perfect for AI purposes as a result of they carry out huge numbers of parallel matrix multiplications, that are essential for neural networks to perform. They’re important within the coaching portion of constructing an AI mannequin and the “inference” portion, the place individuals feed inputs into an AI mannequin and it returns outcomes.
“To create intelligence with generative AI and HPC purposes, huge quantities of knowledge have to be effectively processed at excessive velocity utilizing massive, quick GPU reminiscence,” mentioned Ian Buck, vp of hyperscale and HPC at Nvidia in a information launch. “With Nvidia H200, the trade’s main end-to-end AI supercomputing platform simply received sooner to resolve a number of the world’s most essential challenges.”
For instance, OpenAI has repeatedly mentioned it is low on GPU sources, and that causes slowdowns with ChatGPT. The corporate should depend on charge limiting to supply any service in any respect. Hypothetically, utilizing the H200 would possibly give the prevailing AI language fashions that run ChatGPT extra respiratory room to serve extra clients.
4.8 terabytes/second of bandwidth
In keeping with Nvidia, the H200 is the primary GPU to supply HBM3e reminiscence. Due to HBM3e, the H200 provides 141GB of reminiscence and 4.8 terabytes per second bandwidth, which Nvidia says is 2.4 instances the reminiscence bandwidth of the Nvidia A100 launched in 2020. (Regardless of the A100’s age, it is nonetheless in excessive demand resulting from shortages of extra highly effective chips.)
Nvidia will make the H200 obtainable in a number of kind components. This consists of Nvidia HGX H200 server boards in four- and eight-way configurations, suitable with each {hardware} and software program of HGX H100 techniques. It would even be obtainable within the Nvidia GH200 Grace Hopper Superchip, which mixes a CPU and GPU into one bundle for much more AI oomph (that is a technical time period).
Amazon Internet Providers, Google Cloud, Microsoft Azure, and Oracle Cloud Infrastructure would be the first cloud service suppliers to deploy H200-based situations beginning subsequent yr, and Nvidia says the H200 can be obtainable “from international system producers and cloud service suppliers” beginning in Q2 2024.
In the meantime, Nvidia has been taking part in a cat-and-mouse sport with the US authorities over export restrictions for its highly effective GPUs that restrict gross sales to China. Final yr, the US Division of Commerce introduced restrictions meant to “maintain superior applied sciences out of the unsuitable palms” like China and Russia. Nvidia responded by creating new chips to get round these limitations, however the US just lately banned these, too.
Final week, Reuters reported that Nvidia is at it once more, introducing three new scaled-back AI chips (the HGX H20, L20 PCIe, and L2 PCIe) for the Chinese language market, which represents 1 / 4 of Nvidia’s knowledge middle chip income. Two of the chips fall under US restrictions, and a 3rd is in a “grey zone” that is likely to be permissible with a license. Anticipate to see extra back-and-forth strikes between the US and Nvidia within the months forward.
[ad_2]
Source_link