The Physical Reality of AI: Data Centers, Energy, and Water Use

When your kid asks ChatGPT a question, a message travels over the internet to a data center somewhere, gets processed by a cluster of chips designed specifically for AI inference, and a response comes back. This takes a fraction of a second. It also, cumulatively, uses a lot of electricity and water.

The “AI is everywhere and intangible” framing that most people carry around is misleading. AI is one of the most physically demanding technologies humans have ever built. Understanding the infrastructure is part of understanding the technology — and it connects directly to questions about energy, climate, and what we’re willing to pay, literally and environmentally, for AI capabilities.

Key Takeaways

• Training a large AI model like GPT-3 was estimated to emit roughly 552 metric tons of CO2 equivalent (Strubell et al., 2019) — though newer estimates vary significantly based on energy mix and efficiency.
• Data centers consumed approximately 460 TWh of electricity globally in 2022 (IEA), roughly equivalent to the annual electricity consumption of France.
• AI inference (running queries, not training) now accounts for the majority of AI’s total energy use because training happens once but inference happens billions of times daily.
• Cooling systems in data centers use enormous amounts of water — one estimate puts Google’s data centers at 2.3 billion gallons consumed in 2021.
• Geographic concentration of AI infrastructure creates resilience risks and local resource pressures that communities near data centers experience directly.

What’s Actually Inside a Data Center

A data center is a building — or a campus of buildings — full of server hardware. For AI, the key hardware is GPUs (Graphics Processing Units, now repurposed for AI computation) or custom AI accelerators like NVIDIA’s H100, Google’s TPUs (Tensor Processing Units), or Cerebras’s wafer-scale chips.

When I was at Texas Instruments, I worked on chips that went into consumer electronics. The chips that run AI training are in a different category entirely. NVIDIA’s H100 GPU draws up to 700 watts of power — each chip. A single rack of H100s might hold 8 chips drawing 5,600 watts. A large-scale AI training cluster might have 10,000 H100s. That’s 70 megawatts of compute, before counting the networking, storage, and cooling overhead. Cooling typically adds 30–50% to the power load.

The numbers at the company level:

• Microsoft’s data center investment for AI: $50 billion in 2024 alone, a significant portion for AI infrastructure.
• Google’s data center power consumption: publicly reported at over 18 TWh in 2021 — before its AI expansion.
• Meta’s AI infrastructure: multiple large-scale GPU clusters, including a 24,576 H100 cluster announced in 2024.

Training vs. Inference: Where the Energy Actually Goes

There’s an important distinction between training AI models (building them) and running AI models (using them, also called inference). Both use energy; the profile is different.

Training is the initial phase where a model learns from data. It runs once (or a handful of times for major model versions). Training large models is extremely energy intensive — it runs for weeks or months on thousands of chips simultaneously. A 2019 paper from the University of Massachusetts Amherst (Strubell et al.) estimated training a single large NLP model emitted roughly 300,000 kg of CO2 equivalent — the researchers compared it to 125 round-trip flights from New York to Beijing.

Inference is running the trained model to answer queries. Each query uses far less energy than training — a fraction of a watt-hour per query. But ChatGPT processes an estimated 10–100 million queries per day. At scale, inference energy use dwarfs training energy use because training is one-time while inference is continuous.

A 2023 analysis from Goldman Sachs Research estimated that by 2030, AI data centers could add 200 TWh of additional electricity demand annually in the US alone — equivalent to roughly 5% of current US electricity consumption.

Activity	Energy Estimate	CO2 Equivalent (US grid)	Context
Training GPT-3 (2020)	~1,287 MWh	~500+ tons CO2e	One-time event
Single ChatGPT query	~0.001–0.01 kWh	~0.4–4g CO2	10–100x more than Google search
Google’s global data centers (2021)	~18,000 GWh/year	~2M tons CO2e	Before major AI expansion
US data center total (2022)	~200,000 GWh/year	~85M tons CO2e	~2% of US electricity
Global data center total (2022)	~460,000 GWh/year	~200M tons CO2e	IEA estimate

The Water Problem

Data centers generate heat. Cooling that heat requires either electricity-intensive mechanical cooling or evaporative cooling systems that consume water. Large hyperscale data centers — Google, Microsoft, Meta, Amazon — increasingly use evaporative cooling for its energy efficiency.

A 2023 study in Nature Water (Li et al.) estimated that training GPT-3 in Microsoft’s US data centers consumed approximately 700,000 liters of freshwater. Microsoft’s 2022 Environmental Report disclosed that their global data centers consumed 6.4 million cubic meters of water — roughly equivalent to 2,500 Olympic swimming pools.

This matters most in water-stressed regions. Microsoft, Google, and Meta have all built major data centers in arid regions (Arizona, Nevada, Texas) partly because of cheap land and power, while those areas face intensifying drought pressure. Local communities sometimes learn about data center water consumption only when drought restrictions affect their own supply while a nearby data center continues drawing from the same aquifer.

Is AI Energy Green? The Energy Mix Debate

Whether AI’s carbon footprint is as alarming as the raw energy numbers suggest depends almost entirely on the energy mix powering the data centers.

Google, Microsoft, Amazon, and Meta have all made various commitments to carbon-free or renewable energy. Microsoft committed to being carbon negative by 2030. Google reports “carbon free energy” (CFE) matching for its data centers, though energy matching and actual 24/7 carbon-free energy are different things.

The reality is mixed. Data centers in the Pacific Northwest (access to hydroelectric power) have much lower carbon intensity than data centers in Texas or Virginia (higher coal/natural gas mix, though changing rapidly). The AI energy buildout is also driving new demand that the grid has to meet somehow — and in some regions, utilities are considering extending the life of coal plants to meet data center demand.

A 2024 report from the IEA (“Electricity 2024”) specifically flagged data center demand as a major driver of electricity demand growth, projecting global data center consumption could double between 2022 and 2026.

Geographic Concentration and Community Impact

About 30% of US data centers are clustered in Northern Virginia — a region of Loudoun County sometimes called “Data Center Alley.” This concentration creates:

• Grid pressure: Local utilities have struggled to build transmission capacity fast enough to serve data center growth.
• Water rights competition: Local communities and agricultural users compete with data centers for groundwater.
• Land use change: The characteristic large, windowless buildings are replacing farmland and forest at scale.
• Jobs: Data centers employ relatively few people per dollar of investment (they’re highly automated), but construction phases and ancillary services provide some local employment.

Similar dynamics are playing out in rural Iowa (Google, Meta campuses), Central Texas (Tesla, Google, Oracle), and internationally in Ireland and Singapore.

How to Teach Your Kid About AI’s Physical Footprint

Ages 5–8: Where does electricity come from?

Before talking about AI, build the foundation: find your utility’s energy mix disclosure (most utilities publish this) and look at where your household electricity comes from. Then connect it forward: “When you ask a question to an AI assistant, it uses electricity — from the same kinds of places our house gets it. That electricity came from somewhere, and some of that somewhere is power plants.” This is geographic thinking, energy literacy, and systems thinking in one conversation.

Ages 9–12: Calculate a data center’s footprint

A simple math exercise: if a data center uses 100 megawatts of power, and runs 24 hours a day for a year, how many kilowatt-hours is that? (876 million kWh.) If the average US home uses about 10,500 kWh per year, how many homes does that equivalent represent? (~83,000 homes.) Kids who’ve done this once have a much more concrete sense of scale than kids who just see the word “megawatt.”

Ages 13+: Read the IEA data center report

The IEA publishes its “Data Centres and Data Transmission Networks” special report free at iea.org. It includes country-level data, efficiency trends, and projections. A teenager interested in climate, energy, or AI policy can read the executive summary (4–5 pages) and have an informed conversation about whether current AI energy growth is compatible with stated climate goals — a question with no simple answer.

The question to ask: “If every AI query uses 10 times more energy than a Google search, and there are billions of AI queries per day, how does that change how you think about when it’s worth using AI?”

What to Watch For Over the Next 3 Months

Month 1: Look up where your cloud provider’s nearest data center is (Google Data Center locations, AWS regions, Microsoft Azure regions are all public). Notice if it’s in a water-stressed area. This is the kind of supply-chain thinking that becomes more relevant as kids grow into consumers and citizens.

Month 2: When your child uses AI tools, practice the habit of asking whether the task justifies the energy. This isn’t environmental guilt — it’s decision hygiene. Using AI to summarize a long document is a reasonable trade. Using AI to generate 50 versions of a sentence for fun is a different calculation.

Month 3: The semiconductor industry publishes its own sustainability reports. TSMC (which manufactures chips used in AI accelerators) publishes an annual sustainability report with water and energy data. Connecting AI to the semiconductor supply chain is a systems literacy exercise that connects to how hardware enables AI.

Frequently Asked Questions

Is one ChatGPT query really worse for the environment than a Google search?

The estimates vary, but most analyses suggest AI inference uses roughly 3–10x more energy per query than a conventional web search. This is because language models require much more computation than search indexing and ranking. The comparison matters because people are replacing many web searches with AI queries — the aggregate energy impact is real, though not individually catastrophic for any single query.

Are data companies actually using renewable energy?

Partially. The major hyperscalers (Google, Microsoft, Amazon, Meta) purchase renewable energy credits and have renewable energy contracts that roughly match their consumption on an annual basis. What this doesn’t guarantee is that the electrons powering any specific data center at any specific hour are from renewable sources — electricity grids are not that granular. Some companies are investing in 24/7 carbon-free energy matching, which is more demanding and more honest.

Where are major AI data centers located, and can I visit one?

Most hyperscale data centers are not open to public tours. Google’s data center campus in The Dalles, Oregon, and their Mayes County, Oklahoma facility are visible on satellite imagery. Microsoft and Meta have major campuses in Iowa. The physical scale — warehouse-sized buildings, exterior cooling towers, large power substations — is visible from the road in many cases. It’s a useful exercise in making infrastructure visible.

Does AI chip design affect energy efficiency?

Significantly. Custom AI accelerators (Google TPUs, AWS Trainium, Graphcore IPUs) are designed specifically to perform the matrix multiplications that dominate deep learning workloads more efficiently than general-purpose GPUs. NVIDIA’s successive GPU generations have improved the performance-per-watt ratio substantially. But efficiency gains have been partially offset by running larger models and more queries. This is sometimes called Jevons paradox in energy economics — efficiency gains lead to increased total use.

---

About the author Ricky Flores is the founder of HiWave Makers and an electrical engineer with 15+ years of experience building consumer technology at Apple, Samsung, and Texas Instruments. He writes about how kids learn to build, think, and create in a tech-saturated world. Read more at hiwavemakers.com.

---

Sources

1. Strubell, E., Ganesh, A., & McCallum, A. (2019). “Energy and Policy Considerations for Deep Learning in NLP.” ACL 2019. https://arxiv.org/abs/1906.02629
2. International Energy Agency. (2024). “Electricity 2024: Analysis and Forecast to 2026.” https://www.iea.org/reports/electricity-2024
3. Li, P., et al. (2023). “Making AI Less ‘Thirsty’: Uncovering and Addressing the Secret Water Footprint of AI Models.” Nature Water. https://arxiv.org/abs/2304.03271
4. Goldman Sachs Research. (2023). “AI Is Poised to Drive 160% Increase in Power Demand.” https://www.goldmansachs.com/intelligence/pages/ai-poised-to-drive-160-increase-in-power-demand.html
5. Google LLC. (2022). “Google Environmental Report 2022.” https://sustainability.google/reports/
6. Microsoft. (2022). “Microsoft 2022 Environmental Sustainability Report.” https://query.prod.cms.rt.microsoft.com/cms/api/am/binary/RW15mgm
7. Patterson, D., et al. (2021). “Carbon Emissions and Large Neural Network Training.” https://arxiv.org/abs/2104.10350