Data Centers and Commercial Refrigeration, Part 2: Meeting the Challenge

Data Centers and Commercial Refrigeration, Part 2: Meeting the Challenge

Major challenges faced by AI and data centers, such as high energy use, heat generation, and pressure on the power grid, are being addressed through new innovations in infrastructure, hardware, and operations.
These include advanced cooling systems, alternative energy sources, and AI-based software that improves efficiency.

Tech giants are leading the shift towards liquid-cooled data centers to support AI

High-density AI chips, which are need to power large language models (LLMs) generate significantly more heat than traditional servers, rendering older air-cooling methods obsolete. New, more effective liquid-based solutions are becoming the industry standard:

• Direct-to-chip cooling: This method uses liquid coolant that flows through a cold plate placed directly on top of the most powerful and hottest components, like GPUs, allowing for faster heat removal than air cooling.
• Liquid immersion cooling: This involves fully submerging servers in a non-conductive dielectric fluid.
  This method efficiently absorbs all heat, making it especially useful for high-density server racks.
• Rear-door heat exchangers: These devices are installed on the back of server racks and use liquid coolant to circulate and remove heat from exhaust air.

According to Rcrwireless.com, ‘Direct-to-chip is currently the most common in large AI data centers because it balances efficiency and cost’, and while immersion cooling is highly effective, it requires more specialized equipment and as a result, is less widely accepted.

Companies like Google, Microsoft, Meta, Amazon, and Alibaba are heavily investing in liquid-cooled data centers to support their AI services and drive innovation. This shift is driven by the need for better performance and energy efficiency to support the next generation of AI workloads.

Intel and NVIDIA engineer high-performance liquid-cooled chips

Intel and NVIDIA are leading the charge, designing and optimizing cutting-edge chips for liquid-cooled environments. Their innovations in chip architecture and cooling technology enable unprecedented computing density and performance, which is essential for scaling AI infrastructure.

Colocation and cloud providers launch liquid-cooling solutions

Colocation providers and cloud service companies are now providing liquid-cooling-ready racks.
These specialized solutions are designed to efficiently handle the high computing demands of AI and HPC customers, supporting a powerful and sustainable foundation for business operations.

AI itself will play a role in resource management

Interestingly, AI itself is a key solution to many of the infrastructure challenges it generates. AI and machine learning are used to optimize data center operations for greater efficiency.

AI contributes to better resource management through enhanced data center security, real-time anomaly detection, proactive threat identification, data protection, predictive maintenance, and dynamic workload management—helping to improve service delivery and reduce downtime.

Thse AI algorithms analyze network traffic and usage patterns to automatically allocate compute, storage, and networking resources, ensuring optimal performance and preventing bottlenecks.

Artificial Intelligence can also predict temperature changes and adjust cooling systems in real time, significantly reducing energy use.
For instance, Google has used AI to cut its cooling costs by 40%.

In terms of maintenance, AI combined with IoT sensors and predictive analytics can identify potential hardware failures before they occur, reducing unplanned downtime and streamlining maintenance, which improves overall reliability.

Hardware and networking

Efficient hardware and networking are essential to support the speed and bandwidth needed for AI workloads.

Although GPUs continue to be the primary hardware used for AI, data centers are also integrating more specialized accelerators, such as Google’s Tensor Processing Units (TPUs) and Neural Processing Units (NPUs), which are custom-built for specific AI tasks, offering higher throughput and lower latency.

What are Google’s Tensor Processing Units? Google Cloud TPUs are AI accelerators, designed and optimized for training of and inference of AI models. Currently Google is using TPUs to power Gemini and all of its AI powered apps.

Cloud TPUs are optimized for training large and complex deep learning models that feature many matrix calculations, for instance building large language models (LLMs)

Memory advancements

Memory bottlenecks are an issue for AI, most of the solutions involve increasing memory bandwith and capacity:

High-Bandwidth Memory (HBM): This advanced memory architecture is being adopted for top-tier AI training, offering superior bandwidth, speed, and energy efficiency compared to traditional DRAM.

Compute Express Link (CXL): This standard allows for flexible memory expansion and more efficient resource utilization, improving performance for AI workloads.

How tradional cooling methods and new cooling methods meet

For AI data centers, the future of cooling lies in a hybrid approach that strategically combines traditional air cooling with more efficient liquid cooling technologies.

Hybrid cooling combines air and liquid cooling to achieve better performance, efficiency, and sustainability than using either method alone. This approach is essential for managing the heat produced by high-density workloads seen in AI and high-performance computing. Hybrid systems offer flexibility, letting operators adapt by using liquid cooling for high-density servers and traditional air cooling for less intensive equipment or to manage internal heat transfer within the facility.

• Hybrid cooling strategy: Facilities often combine traditional air circulation and aisle containment with liquid cooling solutions to create a multi-layered system. In this model, liquid cooling targets the hottest components, and air cooling manages the remaining ambient heat.
• Rear-door heat exchangers (RDHX): These are commonly used liquid-cooled heat exchangers placed at the back of server racks. They remove heat as it exits the server, preventing it from entering the data center air. These can be used alongside existing air-cooling systems, allowing higher densities without major infrastructure changes.
• In-row cooling: These units are placed between server racks and provide more targeted cooling than traditional CRAC units. They help manage heat from lower-density equipment that doesn’t require liquid cooling.
• Partial liquid cooling: In this setup, a liquid-cooled loop is used specifically to cool the hottest components, while air cooling is used to remove the remaining heat from the rest of the system.

The future is up for grabs

These various techniques are being used in different combinations, and each situation is unique.

The success or failure of these combined systems will help determine the path forward. Currently, there is no single solution that works for all scenarios.

About International Coolers®

International Coolers® combines premium-quality walk in coolers and walk in freezers with industry expertise. We’re proud to offer custom walk in coolers, walk in freezers, combos, convenience store coolers and freezers that are built to last, designed for efficiency, and that can be shipped nationwide as well as internationally. Contact us and get a quote.

We can provide walk in freezers, walk in coolers, convenience store coolers and freezers and drive in coolers. International Coolers specializes in comprehensive project management, seamlessly handling a diverse range of commercial refrigeration endeavors like food distribution centers, pharmaceutical cold rooms, and processing facilities. We have and expertise that spans from initial design and meticulous planning, including architectural blueprints, to the precise engineering drawings necessary for permits, culminating in flawless installation.

International Coolers Mexico – Walk in Coolers and Walk in Freezers