Jensen Huang, president of Nvidia, holds the Grace hopper superchip CPU used for next generation AI at Supermicro’s keynote presentation during Computex 2023.
Walid Berrazeg | White clouds | Getty Images
Nvidia on Monday unveiled the H200, a graphics processing unit designed for training and deploying the types of artificial intelligence models that power the AI generation boost.
The new GPU is an upgrade from the H100, the OpenAI chip used to train the most advanced large language model, GPT-4. Big companies, start-ups and government agencies are all vying for a limited supply of chips.
H100 chips cost between $25,000 and $40,000, according to an estimate from Raymond James, and thousands of them are needed to work together to create the largest models in a process called “training.”
Enthusiasm about Nvidia’s AI GPUs has boosted the company’s stock, which is up more than 230% year-to-date in 2023. Nvidia expects about $16 billion in revenue for its fiscal third quarter, up 170% from a year ago.
The main development with the H200 is that it includes 141GB of next-generation “HBM3” memory that helps the chip make “decision,” or use a large module after training to generate text, images or predictions.
Nvidia said the H200 will generate output almost twice as fast as the H100. That is based on a usage test Meta at Lama 2 LLM.
The H200, which is expected to launch in the second quarter of 2024, will compete with AMD’s MI300X GPU. AMD’s a chip, like the H200, has additional memory over its predecessors, which helps to fit large models on the hardware to run a decision.
Nvidia H200 chips in an eight-GPU Nvidia HGX system.
Nvidia
Nvidia said that the H200 will be compatible with the H100, meaning that AI companies that are already training with the previous model do not need to change their server systems or software to use the new version.
Nvidia says it will be available in a four-GPU or eight-GPU server configuration on the company’s HGX complete systems, as well as in a chip called the GH200, which pairs the H200 GPU with a fixed processor on Army.
However, the H200 may not hold the crown of the fastest Nvidia AI chip for a long time.
While companies like Nvidia offer many different configurations of their chips, new semiconductors often take a big step forward about every two years, when manufacturers move to architectures between -different that unlocks more significant performance gains than adding memory or other smaller optimizations. Both the H100 and H200 are based on Nvidia’s Hopper architecture.
In October, Nvidia told investors that it would move from a two-year architecture release to a one-year release pattern due to high demand for its GPUs. The company showed a slide suggesting that it will announce and release its B100 chip, based on the upcoming Blackwell architecture, in 2024.
Watch: We are big believers in the AI movement going into next year
Don’t miss these stories from CNBC PRO:
