Nvidia Introduces Custom High-Bandwidth Memory for AI Accelerators
Nvidia has launched NVHBM, a custom high-bandwidth memory solution designed to boost AI accelerator performance and energy efficiency. Amazon's Annapurna Labs is the first partner.
Nvidia has unveiled NVHBM, a new custom high-bandwidth memory (HBM) designed specifically for its AI accelerator partners. This new memory component aims to enhance both performance and power efficiency for artificial intelligence workloads.
The company claims NVHBM offers up to 30% higher bandwidth and 15% lower power consumption compared to standard HBM4e memory. This improvement is crucial for memory-bandwidth-bound AI tasks, potentially increasing throughput in applications like AI inference. Additionally, NVHBM's design reduces the silicon footprint on the main accelerator chip, freeing up space for an estimated 30% more compute resources.
NVHBM is integrated into Nvidia's NVLink Fusion project, which enables the creation of unified computing systems by connecting multiple processors. Nvidia states that NVHBM has been co-designed and validated with leading memory manufacturers, aiming to accelerate product development for partners creating custom chips.
Amazon's Annapurna Labs has been announced as the first partner to adopt NVHBM. The company intends to utilize this technology in its future AWS infrastructure AI chips, including the upcoming Trainium 4 chip.