NVIDIA H200 Tensor Core GPU Supercharging AI and HPC workloads
NVIDIA H200 Tensor Core GPU Supercharging AI and HPC workloads
If you have any questions, you are always welcome to contact us. We'll get back to you as soon as possible, within 24 hours on weekdays.
-
Shipping Information
Use this text to answer questions in as much detail as possible for your customers.
-
Customer Support
Use this text to answer questions in as much detail as possible for your customers.
-
FAQ’s
Use this text to answer questions in as much detail as possible for your customers.
-
Contact Us
Use this text to answer questions in as much detail as possible for your customers.
The NVIDIA H200 Tensor Core GPU supercharges generative AI and high-performance computing (HPC) workloads. As the first GPU to feature HBM3E, it delivers game-changing performance with larger and faster memory, accelerating large language models (LLMs) and advancing scientific computing.
Key Features at a Glance
- First GPU to incorporate HBM3E memory for enhanced performance.
- Optimized for generative AI and high-performance computing tasks.
- Boosts LLM inference speed by up to 2X compared to H100 GPUs for models like Llama2.
- Achieves up to 110X faster time to results for HPC applications compared to CPUs.
- Offers unparalleled performance within the same power profile as the H100.
- H200 NVL variant supports up to four GPUs with NVIDIA NVLink™ for enterprise servers.
Enhanced LLM Inference
The H200 Tensor Core GPU is engineered to deliver high-performance LLM inference, boosting speeds by up to 2X compared to H100 GPUs for models like Llama2. Its larger and faster HBM3E memory fuels the acceleration of large language models, crucial for businesses relying on efficient AI inference.
Accelerated High-Performance Computing
For memory-intensive HPC applications, the H200's higher memory bandwidth ensures efficient data access and manipulation. This leads to significantly faster time to results, up to 110X faster compared to CPUs, advancing scientific research, simulations, and other complex computational tasks.
Optimized Energy Efficiency and TCO
The H200 introduces new levels of energy efficiency and Total Cost of Ownership (TCO), offering unparalleled performance while maintaining the same power profile as the H100. This design supports faster and more eco-friendly AI factories and supercomputing systems.
Enterprise-Ready with H200 NVL
The NVIDIA H200 NVL is designed for mainstream enterprise servers, offering lower-power, air-cooled rack designs with flexible configurations. It provides acceleration for all AI and HPC workloads, featuring up to four GPUs connected by NVIDIA NVLink™ and a 1.5x memory increase, leading to up to 1.7x faster LLM inference and 1.3x more performance for HPC applications compared to H100 NVL.
Technical Specifications
| H200 NVL | |
| FP64 | 30 TFLOPS |
| FP64 Tensor Core | 60 TFLOPS |
| FP32 | 60 TFLOPS |
| TF32 Tensor Core | 835 TFLOPS |
| BFLOAT16 Tensor Core | 1,671 TFLOPS |
| FP16 Tensor Core² | 1,671 TFLOPS |
| FP8 Tensor Core | 3,341 TFLOPS |
| INT8 Tensor Core | 3,341 TFLOPS |
| GPU Memory | 141GB |
| GPU Memory Bandwidth | 4.8TB/s |
| Decoders | 7 NVDEC 7 JPEG |
| Confidential Computing | Supported |
| Max Thermal Design Power (TDP) | Up to 600W (configurable) |
| Multi-Instance GPUs | Up to 7 MIGs @16.5GB each |
| Form Factor | PCIe Dual-slot air-cooled |
| Interconnect | 2- or 4-way NVIDIA NVLink bridge: 900GB/s per GPU PCIe Gen5: 128GB/s |
| Server Options | NVIDIA MGX H200 NVL partner and NVIDIA-Certified Systems with up to 8 GPUs |
| NVIDIA AI Enterprise | Included |
Frequently Asked Questions
Q: What is the NVIDIA H200 GPU designed for?
A: The NVIDIA H200 Tensor Core GPU is designed to supercharge generative AI and high-performance computing (HPC) workloads.
Q: What is a key memory feature of the H200 GPU?
A: The H200 is the first GPU to incorporate HBM3E memory, offering larger and faster memory capabilities.
Q: How does the H200 NVL enhance enterprise applications?
A: The H200 NVL is ideal for enterprise rack designs, supporting up to four GPUs with NVIDIA NVLink™ and offering increased memory for accelerating both AI and HPC workloads.
