Article by Gordon Johnson, Subzero Senior CFD Manager
The recent rise in AI has led to vast amounts of computational power and data storage being required. This has led to a significant increase in the need for high-performance servers capable of handling complex tasks, pushing data centers to adopt a more robust infrastructure. However, promises to invest in AI products are clashing directly with assurances to reduce CO2 emissions.
Even tech giants such as Google and Microsoft admit their net-zero targets are in jeopardy. In fact, Google recently revealed that its greenhouse gas emissions have risen 48% over the past five years and that electricity consumption by data centers, alongside supply chain emissions, were the primary cause of the increase, endangering its goal of being carbon-neutral by 2030.
The International Energy Agency has estimated that data centers’ total electricity consumption could double from 2022 levels to 1,000TWh (terawatt hours) in 2026, with AI resulting in data centers using 4.5% of global energy generation by 2030, according to calculations by research firm SemiAnalysis.
As power and heat loads continue to rise, efficient cooling in the data center is becoming more and more challenging – cooling is considered responsible for up to 40% of a data center’s total energy consumption. In addition, data center thermal management is often a reactive process, meaning we tend to wait for server alarms or increased server inlet temperatures before we react.
What happens if critical components within the data center, such as cooling units, fail unexpectedly? Even worse, what happens if all cooling simultaneously fails due to a power outage or other significant event such as a failure at the chilled water plant? The answer? Modeling and simulation with Computational Fluid Dynamics (CFD), which includes software specific to the data center environment.
Additional Key Benefits
CFD software creates a 3D model or digital twin of the physical attributes of the data center, including the performance characteristics of the cooling units, ITE, and other significant equipment, described by some as an MRI of what is really happening inside the data center. CFD provides a detailed understanding of the complex interactions between airflow, heat removal from the ITE, and cooling systems, enabling data center operators to make informed decisions to reduce energy consumption and improve overall cooling efficiency.
The key benefits of using CFD include, but are not limited, to:
- Identifying potential server hot spots
- Validating the pre-construction phase of a data center
- Correctly placing additional ITE and cooling units in existing data centers
- Optimizing airflow and reducing excess and unnecessary cooling
- Simulating via transient modeling the ITE ride-out time due to power and/or cooling system failure.
CFD can aid in the design phase of data centers by simulating a variety of layout configurations. It looks at the placement of servers, racks, cooling units and airflow containment systems to ensure optimal thermal performance starting with the design phase down to the operational phase of the data center. This means that designs can be validated by avoiding and preventing costly mistakes.
However, CFD is not just for new design. Legacy data centers can also take advantage of the benefits that CFD provides, including simulating the results of proposed changes to the existing room layout such as capacity redistribution or ITE expansion. In all cases, a CFD is always beneficial because it assists data center operators in making data-based informed decisions while minimizing or eliminating critical potential equipment downtime.
A CFD also accurately simulates and provides a detailed analysis of airflow and temperature distribution at all stages of the cooling process. These patterns include temperature distribution in both cold and hot aisles, in addition to the temperature across the server inlets from the bottom to the top of the racks. CFD simulations further evaluate the performance of different cooling strategies. For example, it can simulate air cooling strategies deployed in conjunction with liquid cooling designs, such as those using air cooling to remove the supplemental ITE heat not removed via the circulating liquid in Direct-to-chip cold plate cooling.
Additionally, CFD now has the ability to accurately reduce risk from power and/or cooling system failure with transient modeling, showing via time increments what will happen to the ITE when either cooling is lost or both cooling and fan power (airflow) fail. When both cooling and fan power fail throughout the data center, cooling is not lost immediately since the fans will take some time to come to a complete stop. CFD can simulate this time period, showing how hot the ITE server inlets will get and how soon this will happen before power is restored to the data center.
The Carbon Footprint
The growing focus on sustainability has highlighted our need to work towards energy-efficient data center designs, recognizing the need to reduce our energy footprint while maintaining safe and optimal operating temperatures at the ITE. CFD assists with this important endeavor by identifying server hot spots before they occur, resulting in reduced downtime and improving overall ITE reliability.
This further aids in extending the lifespan of the ITE by ensuring it operates within ASHRAE (American Society of Heating, Refrigeration, and Air-Conditioning Engineers) recommended thermal temperature guidelines. This way, potential issues can be addressed before they lead to costly problems, improving both the overall reliability and uptime of the data center.
Using CFD is one of, if not the, most important step in determining data center energy efficiency since it provides a detailed understanding of the complex interactions between airflow, heat, and cooling systems. This enables data center operators to make informed decisions to reduce both energy consumption and the data center’s carbon footprint.
The Challenges
Implementing CFD into existing data center management processes is not without its challenges. As with any analytical tool, the accuracy of assessments and simulations depends on the quality of the data used. For accurate modeling, a baseline of how the data center is currently operating is crucial for reliable results, and this should include current rack kW, cooling unit supply and return temperatures, rack and room leakage, as well as other additional room and airflow characteristics.
Data center experts agree that all new data centers should always perform a CFD before construction while existing data centers should also perform a CFD before making any ITE and cooling changes for design validation.
What’s Next?
Real-world case studies and success stories highlight the impact of CFD in data center optimization by demonstrating significant improvements in energy efficiency, cooling effectiveness and operational performance. In the future, we expect to see CFD used extensively with the integration of machine learning, AI (Artificial Intelligence) ITE, and hybrid cooling infrastructure designs, with a continued priority on sustainability
In Summary
CFD modeling is a powerful tool for designing and validating efficient technology solutions for new and legacy data center environments by providing predictive results that bridge high-performance computer server operations with the critical mechanical system. CFD also safeguards the date center by simulating failure scenarios that enable data center managers to better understand and design the appropriate redundancy for power and cooling failure scenarios.
Armed with data from the CFD analysis, a suitably experienced technology partner, such as Subzero Engineering, can advise customers on impactful data center design changes, upgrades, or tweaks to improve operating improvements – all of which will help optimize and reduce energy usage while safeguarding the data center.