CEO Andy Jassy gives long update on Amazon's hardware that can 'cool down' Nvidia GPUs; Elon Musk responds

Amazon CEO Andy Jassy provided a detailed update on the company's hardware solution for managing the intense heat generated by AI chips. This system is particularly useful in data centres that house powerful GPUs, such as Nvidia GPUs. In a recent post on the social media platform X (formerly Twitter), Jassy explained that every cloud provider faces the challenge of positioning chips close together for speed while simultaneously meeting high cooling demands. Following Jassy's detailed post on the invention, Elon Musk replied: “Interesting.” The Tesla CEO’s reply highlights his ongoing interest in scalable AI hardware amid xAI's own supercomputer builds.

He stated that since waiting for new, specialised liquid-cooled facilities was not an option, Amazon "invented our own solution." In July, the company announced that it designed and delivered its In-Row Heat Exchanger (IRHX), which uses a direct-to-chip "cold plate" approach. This system allows Amazon to effectively land both liquid and air-cooled racks in the same existing facilities.


Jassy also noted that the IRHX solution offers benefits, including using 9% less water than fully air-cooled sites and providing a 20% improvement in power efficiency compared to readily available, off-the-shelf solutions. Amazon projects its liquid-cooled capacity will grow to over 20% of its Machine Learning capacity by 2026.

What CEO Andy Jassy said about Amazon’s in-house AI chip cooling system

In his long X post, Jassy wrote: "Every cloud provider faces the same AI infrastructure challenge: chips need to be positioned close together to exchange data quickly, but they generate intense heat, creating unprecedented cooling demands.

We needed a strategic solution that allowed us to use our existing air-cooled data centers to do liquid cooling without waiting for new construction. And it needed to be rapidly deployed so we could bring customers these powerful AI capabilities while we transition towards facility-level liquid cooling. Think of a home where only one sunny room needs AC, while the rest stays naturally cool – that’s what we wanted to achieve, allowing us to efficiently land both liquid and air-cooled racks in the same facilities with complete flexibility.The available options weren't great. Either we could wait to build specialized liquid-cooled facilities or adopt off-the-shelf solutions that didn't scale or meet our unique needs. Neither worked for our customers, so we did what we often do at Amazon… we invented our own solution.Our teams designed and delivered our In-Row Heat Exchanger (IRHX), which uses a direct-to-chip approach with a "cold plate" on the chips. The liquid runs through this sealed plate in a closed loop, continuously removing heat without increasing water use. This enables us to support traditional workloads and demanding AI applications in the same facilities. By 2026, our liquid-cooled capacity will grow to over 20% of our ML capacity, which is at multi-gigawatt scale today.While liquid cooling technology itself isn't unique, our approach was. Creating something this effective that could be deployed across our 120 Availability Zones in 38 Regions was significant. Because this solution didn't exist in the market, we developed a system that enables greater liquid cooling capacity with a smaller physical footprint, while maintaining flexibility and efficiency.Our IRHX can support a wide range of racks requiring liquid cooling, uses 9% less water than fully-air cooled sites, and offers a 20% improvement in power efficiency compared to off-the-shelf solutions. And because we invented it in-house, we can deploy it within months in any of our data centers, creating a flexible foundation to serve our customers for decades to come.Reimagining and innovating at scale has been something Amazon has done for a long time and one of the reasons we’ve been the leader in technology infrastructure and data center invention, sustainability, and resilience. We're not done… there's still so much more to invent for customers."

Meanwhile, Google recently rolled out its most powerful custom AI chip, Ironwood, taking aim at Nvidia. According to the company, the seventh generation of its Tensor Processing Unit is over four times faster than its predecessor.