The People Keeping AI Cool — Why Datacenter Cooling Engineer Is the Career No One Saw Coming

Every time you run a prompt through ChatGPT, the request routes through a server that generates roughly the same heat as a human body at rest — continuously, at massive scale. A hyperscale datacenter running 100,000 GPUs generates as much heat as a small power plant. Getting rid of that heat is one of the most technically challenging engineering problems in the AI industry right now. Liquid cooling — circulating coolant through direct-contact cold plates on chips — is replacing air cooling as the only viable path. The engineers who design these systems are in extraordinary demand. Most parents have never heard the job title.

Why Heat Is the Hard Limit of AI

Moore’s Law — the observation that transistor density doubles roughly every two years — has been the engine of computing progress for 50 years. But there’s a parallel law that gets less attention: as chips get more powerful, they get hotter. More transistors switching faster means more power consumed per unit area, and power consumed equals heat generated.

NVIDIA’s H100 GPU — the dominant chip for training large AI models — consumes up to 700 watts per chip. A single server rack packed with H100s can consume 40–60 kilowatts. A hyperscale AI training cluster with 10,000 such chips is consuming roughly 7 megawatts — equivalent to the electrical demand of about 7,000 average American homes — and generating the same amount as waste heat.

The cooling problem isn’t just about comfort. Electronics fail when they overheat. Chips operate optimally within narrow temperature ranges. Sustained high temperatures degrade semiconductors over time. An inadequately cooled datacenter means slower AI processing, reduced hardware lifespan, and potentially catastrophic failures that can take AI services offline for hours.

Traditional air cooling — which dominated datacenter design for 40 years — is reaching its physical limits. Moving enough air through a rack consuming 40–60 kW requires enormous fans moving enormous volumes of air, which requires an enormous amount of energy itself. Google and Microsoft’s hyperscale AI datacenters have heat densities that conventional air cooling systems simply cannot address efficiently.

The Engineering of Liquid Cooling

Liquid cooling works because water and engineered coolants can carry approximately 3,500 times more heat per unit volume than air. A thin tube of water circulating past a hot chip can remove heat that would require a large air duct to manage.

Modern datacenter liquid cooling takes several forms, each with distinct engineering challenges:

Direct liquid cooling (DLC) circulates water-based coolant through cold plates in direct contact with CPUs, GPUs, and memory modules. The coolant absorbs heat from the chip surfaces, travels through pipes to a heat exchanger, and returns cooled. The engineering challenges include: designing cold plates with optimal contact area and flow dynamics, managing the fluid circuit under varying load conditions, preventing leaks that could destroy millions of dollars of equipment, and integrating with existing datacenter power and management infrastructure.

Immersion cooling submerges entire server boards in dielectric fluid — a synthetic liquid that doesn’t conduct electricity. The fluid directly contacts all components and carries heat to a heat exchanger at the edge of the tank. Single-phase immersion cooling uses fluid that remains liquid; two-phase immersion cooling uses fluid that boils at the chip surface (absorbing heat as it vaporizes) and re-condenses on a cooled surface above. Two-phase immersion achieves extraordinarily efficient heat removal but requires careful fluid selection, tank design, and vapor management.

Rear-door heat exchangers are a transitional technology that attaches to the back of existing server racks and uses water-cooled panels to capture heat from the server exhaust air. Less efficient than direct liquid cooling but retrofittable to existing infrastructure without replacing servers.

Facility-level thermal management is the system engineering that coordinates all the cooling subsystems — chilled water plants, cooling towers, heat exchangers, distribution manifolds, leak detection systems, monitoring and controls — across a facility consuming tens of megawatts. This is where mechanical engineering meets systems engineering meets software.

The Research Picture

The Uptime Institute’s annual Global Data Center Survey consistently identifies power and cooling as the top operational challenges in hyperscale facilities. Their 2023 survey found that the average datacenter power usage effectiveness (PUE) — a ratio of total facility power to IT equipment power, where 1.0 is perfect efficiency — was 1.58, meaning that for every watt of compute work done, 58 cents of additional power is spent on cooling and other overhead. AI workloads, with their extreme heat density, push this ratio higher unless advanced cooling is deployed.

Lawrence Berkeley National Laboratory’s “United States Data Center Energy Usage Report” projects that AI workloads will drive datacenter energy consumption in the U.S. to 76–100 terawatt-hours annually by 2028, up from approximately 43 TWh in 2023. Cooling is approximately 30–40% of that total energy consumption.

Market research from IDC and Gartner consistently finds that liquid cooling is transitioning from a specialty solution to a mainstream requirement. IDC projects that liquid cooling revenues in the datacenter market will grow from approximately $5.5 billion in 2023 to over $20 billion by 2028 — a 30% compound annual growth rate. That growth is driven almost entirely by AI workload expansion.

NVIDIA’s own infrastructure guidance for H100 clusters explicitly recommends liquid cooling for high-density deployments. Their next-generation Blackwell architecture, which will power AI systems through the late 2020s, has even higher power densities — making liquid cooling effectively mandatory.

Cooling Technology	Heat Removal Capacity	Water Usage	Retrofit-Friendly	AI Workload Suitability
Air Cooling (CRAC units)	Low (up to ~20 kW/rack)	None	Yes	Insufficient for high-density AI
Rear-Door Heat Exchangers	Medium (up to ~30 kW/rack)	Moderate	Yes	Limited
Direct Liquid Cooling	High (up to ~100 kW/rack)	Moderate	Partial	Excellent
Single-Phase Immersion	Very high (100+ kW/rack)	Low	No	Excellent
Two-Phase Immersion	Extremely high	Very low	No	Optimal

Sources: Uptime Institute, Green Grid, IDC Cooling Market Analysis 2023

Compensation for datacenter cooling engineers reflects both the criticality of the role and the scarcity of specialized expertise. Mid-career thermal/mechanical engineers focused on datacenter cooling earn $130,000–$170,000 at companies like Google, Microsoft Azure, Amazon AWS, Meta, or specialized cooling infrastructure firms (Vertiv, Schneider Electric, Stulz). Senior engineers with specific immersion or DLC expertise earn $175,000–$220,000. Principal engineers leading cooling architecture for hyperscale AI clusters have been reported at $230,000–$280,000 total compensation.

What This Means for Your Kid

The coolant that keeps AI running is managed by engineers whose foundational discipline is thermal sciences — a branch of mechanical engineering and physics that covers heat transfer, fluid mechanics, and thermodynamics. This isn’t glamorous in the way that AI research or chip design is glamorous. It’s deeply practical. And it is currently one of the most sought-after specializations in the infrastructure sector.

For parents thinking about their kid’s trajectory, the relevant observation is this: the AI boom is creating demand for every layer of the infrastructure stack, including the physical layer. The companies spending billions on AI chips and data centers also need to keep those chips cool. The engineers who understand how to move heat efficiently at massive scale have leverage that the broader job market doesn’t provide.

The foundational preparation looks like this:

• Ages 10–13: Basic thermodynamics intuition — understanding that temperature equalizes, that materials conduct heat differently, that phase change involves energy. This is middle school science, but kids who find it genuinely interesting (rather than just memorizable) are showing an aptitude relevant to this field. Hands-on projects with heatsinks, thermal paste, and temperature sensors are accessible at this age
• Ages 14–17: Physics with real engagement in thermodynamics and fluid mechanics sections. AP Physics covers relevant fundamentals. Engineering-oriented extracurricular programs that include heat transfer concepts are excellent
• Ages 17+: Mechanical engineering is the primary undergraduate path. Look for programs with strong thermodynamics, fluid mechanics, and heat transfer sequences. Some schools now have energy systems or thermal systems concentrations. Computer science or electrical engineering with interest in hardware systems also creates a path to the software controls side of cooling management

This connects to the broader question of what the hidden carbon cost of AI means — cooling is a major component of AI’s energy footprint, and more efficient cooling directly reduces that footprint. Engineers who build more efficient cooling systems are doing environmental work as much as they’re doing infrastructure work.

What to Watch Over the Next 3 Months

• NVIDIA Blackwell infrastructure announcements. As next-gen GPU clusters deploy, cooling requirements will be published in technical documentation. These specifications drive immediate hiring waves
• Hyperscaler data center construction announcements. Every Google, Microsoft, Amazon, or Meta data center announcement represents engineering hiring, including cooling engineering roles
• Vertiv, Schneider Electric, and Stulz earnings calls. These are the dominant cooling infrastructure vendors. Their revenue growth is a direct indicator of liquid cooling adoption
• IEEE and ASHRAE data center cooling publications. ASHRAE (American Society of Heating, Refrigerating, and Air-Conditioning Engineers) publishes the standards that govern datacenter thermal design — their technical bulletins are an excellent window into where the field is heading

The career was invisible 10 years ago. The AI boom made it one of the most critical in the entire technology stack. That kind of rapid emergence is the clearest signal that demand will continue to outpace supply for years.

FAQ

Do datacenter cooling engineers work in datacenters every day? It varies by role. Design engineers work primarily in offices and labs, designing systems on computers and validating with prototypes. Commissioning engineers travel to facilities during deployment. Operations engineers may work in or near facilities monitoring systems. The most senior roles are primarily design and architecture work.

Is this career at risk as chips become more efficient? More efficient chips consume less power and generate less heat per unit of computation. However, AI model complexity and deployment scale are growing faster than chip efficiency improves. Net heat generation from AI computation has increased every year despite efficiency improvements — and is projected to continue doing so through the 2030s.

How does this connect to the environmental impact of AI? Cooling efficiency directly affects the energy consumption and carbon footprint of AI computation. Engineers who improve cooling effectiveness reduce AI’s environmental impact. This framing is increasingly important in how tech companies hire and publicly discuss their infrastructure.

What certifications are relevant to this field? ASHRAE’s Certified Professional Engineer designation is respected. Uptime Institute’s ATSE (Accredited Tier Specialist Examiner) certification is relevant for facility operations. For the more fundamental engineering credentials, PE (Professional Engineer) licensure is increasingly expected at senior levels.

Can software engineers work in this field? Yes, on the monitoring, controls, and AI-driven optimization side. Software that predicts thermal loads, optimizes coolant flow rates, and detects anomalies is a growing component of the field. Software engineers with interest in physical systems and real-time controls are relevant.

Where are these jobs? Hyperscaler headquarters and their data center locations (Northern Virginia, Dublin, Singapore, Tokyo, Dallas, Phoenix for AWS/Azure/Google), cooling system manufacturers (Columbus OH, Düsseldorf, Stockholm), and infrastructure consultancies.

---

About the author Ricky Flores is the founder of HiWave Makers and an electrical engineer with 15+ years of experience building consumer technology at Apple, Samsung, and Texas Instruments. He writes about how kids learn to build, think, and create in a tech-saturated world. Read more at hiwavemakers.com.

---

Sources

1. Uptime Institute. Global Data Center Survey 2023. https://uptimeinstitute.com/global-data-center-survey
2. Shehabi, A. et al. “United States Data Center Energy Usage Report.” Lawrence Berkeley National Laboratory, 2024. https://eta.lbl.gov/publications/united-states-data-center-energy
3. IDC. Worldwide Data Center Liquid Cooling Market Forecast 2024–2028. https://www.idc.com
4. The Green Grid. Data Center Power Usage Effectiveness. https://www.thegreengrid.org
5. NVIDIA. H100 Tensor Core GPU Architecture. https://www.nvidia.com/en-us/data-center/h100/
6. Patterson, D. et al. “Carbon Emissions and Large Neural Network Training.” arXiv, 2021. https://arxiv.org/abs/2104.10350
7. ASHRAE. Thermal Guidelines for Data Processing Environments. https://www.ashrae.org/technical-resources/free-resources/datacom-series