The Electric Elephant in the Room

Addressing AI’s biggest bottleneck while diffusing a ticking climate bomb

Access to sufficient reliable, low-cost, low-carbon energy is the real bottleneck for AI

The International Energy Agency (IEA) projects that AI-related energy demand will surge tenfold by 2026—a staggering increase that far outpaces the current expansion of power generation - particularly renewables - and transmission infrastructure. Utilities are already struggling to keep up, with some even reconsidering new fossil-fuel-based power generation to meet new demand quickly. 

Key figures in AI are warning about the energy bottleneck. Tesla’s Elon Musk has pointed out that while AI development was previously "chip constrained," the future challenge will be power supply. Amazon’s Andy Jassy has noted that there has already been insufficient power available this year to run new generative AI services. OpenAI’s Sam Altman - who is backing companies like nuclear fusion start-up Helion and advanced solar PV developer Exowatt - has emphasized that the future of AI depends on breakthroughs in clean energy, highlighting the need for massive, scalable solutions. 

It becomes increasingly clear: Without sufficient, low-cost, clean power, AI development could not only be hampered but also become a major climate risk. Currently, AI is responsible for ~1% of global greenhouse gas (GHG) emissions. However, AI’s GHG emissions footprint could grow significantly - depending on factors such as the carbon intensity and energy efficiency of required hardware, the energy intensity of AI models, the scale of AI-model adoption/use, and the carbon intensity of the energy sources leveraged. At the same time, large parts of the global economy - such as the transportation or buildings sectors - will require more and more electricity as they decarbonize - competing for the same, still relatively scarce “green electrons” as the growing AI industry. 

Therefore, we must urgently address the "Electric Elephant in the Room"— AI’s insatiable hunger for energy. We have to find innovative ways to manage and meet this increasing demand in line with our climate goals, ensuring that AI will be part of the solution to climate change and not the problem. 

AI-related energy demand is expected to increase exponentially - but utilities are overwhelmed, and clean energy infrastructure deployment is lagging 

Data is essentially energy, and data centers are the "power plants of the digital world," transforming energy into data. These centers are among the most energy-intensive buildings being constructed today, consuming 10 to 50 times more energy per square foot than typical office buildings. Data centers have emerged as significant drivers of global energy consumption, accounting for approximately 2% of worldwide energy use (and more GHG emissions than the global aviation industry). Today, traditional data centers and cryptocurrency mining dominate the bulk of data center energy consumption, while AI currently represents a small fraction of this demand (Figure 1).

However, AI will drive growing data center energy needs. Globally, data center energy demand could double by 2026. In the US, it could rise from ~3% of national electricity consumption to 6% by 2026 and to 7.5% by 2030. Both, globally and in the US, this surge will primarily be driven by AI-related energy demand, expected to increase tenfold from ~10 terawatt hours (TWh) today to ~100 TWh by 2026. This equals the annual energy consumption of countries like Argentina, the Netherlands, and Sweden, or ~0.5% of the world's electricity use. The trend is toward more extensive use of AI across sectors, with increasingly larger, more complex models. The more complex the AI models, the higher the energy consumption to develop them: For example, training GPT-4 required over 50 gigawatt hours (GWh)—about 0.02% of California's annual electricity generation and equivalent to running an average-size nuclear power plant for 50 hours straight —and 50 times the energy that was needed to train GPT-3.

FIGURE 1 | Estimated electricity demand from traditional data centers, dedicated AI data centers and cryptocurrencies, 2022 and 2026, base case (IEA) 

This development will be supported by a significant increase in the number of power-hungry data centers worldwide. Major tech giants have announced ambitious data center expansion plans and are aggressively engaging in a "grab for green electrons" in an attempt to align their data center expansions with their decarbonization goals.

For example, Amazon plans to invest ~$150 billion in data centers over the next 15 years and also recently purchased a data center campus near a nuclear plant in Pennsylvania. Microsoft aims to triple the rate at which it adds data center capacity within the next year and also signed the world’s largest renewable energy purchase deal, securing 10.5 GW. Google is also pouring billions into its data center expansion - for example, a $3 billion investment for US data centers for its AI training and inference efforts - and also partnering with advanced geothermal company Fervo to power its data centers in Nevada. Together, Meta, Microsoft, Amazon, Google, and Apple together account for over half (~45 GW) of global corporate renewable procurement. 

However, even more power is needed, and utilities are inundated with an increasing number of requests for grid-connected power supply for new data center operations, often on a gigawatt scale rather than the usual megawatts. For example. in Virgina, home to the largest concentration of data centers in the world, the state’s largest utility, Dominion Energy, projects that data centers will be the main driver of energy demand in Virgina within the next 15 years - with data center peak energy demand potentially growing from ~3 GW in 2022 to 7 GW in 2032.

As a result, utilities - finding themselves for the first time at the onset of a demand growth phase - are increasingly overwhelmed and revising their energy resource plans, sometimes against their decarbonization commitments. For example, in the US, utilities like Dominion Energy, Georgia Power, and Duke Energy are considering the deployment of new gas-powered plants to meet the growing energy needs of data centers reliably. This is because, today, renewables do not always fit the bill. Due to deployment challenges, grid integration issues, and the intermittent nature of wind and solar power (see bottlenecks laid out in my previous article), flexible power generation, such as gas, remains necessary until alternative, zero-carbon solutions like long-duration energy storage or advanced nuclear technology (e.g., small modular reactors) become available.

All these developments and projections underscore the urgent need to manage the growing power demand from new data centers with clean energy solutions. Furthermore, ideally, we can find ways to create synergies between the massively growing energy demand from AI and the decarbonization of our energy systems.

Before exploring potential strategies, let’s take a closer look at AI’s energy footprint. 

AI's energy footprint: Driven by computation, model usage, and data center cooling needs

AI's direct energy footprint is primarily driven by two factors: (I) computation, including model development, data handling, and model usage (inference), and (II) data center operations, particularly hardware cooling. Below, these factors are explained in more detail.

Note: AI’s indirect energy demand also includes the energy consumed in producing the required hardware. While the energy intensity - and embedded carbon - of AI hardware are crucial topics that need close examination, they will be addressed in future discussions rather than in this article.

I) Computation

• Model development and training. Training AI models involves numerous calculations over massive datasets, often taking days or even weeks. Specialized, power-hungry hardware (e.g., GPUs, TPUs) handles the complex computations. Essentially, the larger (the more parameters) the model, the greater the energy required to train it. Between 2010 and 2024, compute used to train models (such as large language models) grew 4-5 times per year. Research indicates that achieving a tenfold improvement in AI model efficiency could necessitate a computational power increase of up to 10,000 times. 

• Fine tuning. Ongoing iterative fine-tuning and optimization of an existing model, involving multiple cycles of training and testing, also consumes energy. However, the demand for this activity is relatively moderate compared to developing a new model from scratch.

• Data handling. Storing the vast amounts of data needed for training and inference and transferring this data between storage and computational units also consumes significant energy. The larger the datasets, the more energy is required for storage and transfer.

• Model usage (Inference). The cumulative energy consumption over time can be substantial, depending on the model's size and usage frequency. For example, large, high-frequency models like ChatGPT consume significantly more energy for inference than for training. 

II) Data center operations 

• Most of the power consumed in data centers, aside from that used for computation, goes towards cooling processors and other hardware. Cooling requirements are influenced by several factors, including computational load, data center layout, ambient temperature, and hardware efficiency.

Determining the exact share of power demand between computation and supporting data center operations is challenging due to numerous influencing factors, as discussed above. However, on average, modern data centers allocate about 40% of their power to cooling hardware. Within the power needed for computation, model training typically accounts for approximately 20-40%, while inference could consume a lion’s share of 60-80%, depending on the model's complexity and deployment scenario (e.g., high vs. low-frequency use). For instance, a high-usage large language model like ChatGPT is expected to split its power demand with 20% for training and 80% for usage.

We should leverage AI’s growing energy footprint to accelerate renewables deployment - while reducing the energy intensity of computation and data centers  

To mitigate the climate risk arising from AI’s energy footprint, we must address both AI’s energy demand and energy supply. On the demand side, innovative strategies already exist to reduce the energy intensity of computation—covering model development, training, and usage—as well as data center operations. However, even with significant improvements in energy efficiency, the absolute energy demand from AI will rise substantially. Therefore, decarbonizing the energy supply from the outset is equally, if not more, critical. Ideally, we find ways to leverage AI’s growing energy demand to accelerate renewable energy deployment. 

In summary, solutions for addressing AI’s direct energy demand and potential negative climate impact can be structured around three pillars:

1. Leverage AI-related energy demand to drive renewables deployment 

2. Reduce the energy intensity of computation

3. Maximize the energy efficiency of data centers

Here are examples—some already implemented and some still in the research phase—linked to each of these three pillars:

1) Leverage AI-related energy demand to drive renewables deployment, e.g.:

• Create additional and/or onsite renewable energy generation. Where possible, data center operators should locate their operations in regions with abundant renewable resources, such as solar, wind, or geothermal power, and opt for dedicated onsite renewable energy generation, with the grid as a backup. Where more suitable, data center operators can engage with utilities to support the deployment and integration of new renewable energy sources into the grid. For example, Meta partnered with the utility Tennessee Valley Authority (TVA) to create a green tariff for new solar energy (on behalf of Meta, TVA signed a purchase agreement with the independent power producer (IPP) Silicon Ranch). Another example is Amazon’s investment in the transformation of a Maryland coal mine site into a solar farm. 

• Shift computation to times or geographies with clean energy supply and less grid congestion. Data centers can strategically schedule compute tasks to periods or locations where the electrical grid experiences lower demand or where renewable energy is more abundant. By doing so, data centers can help alleviate pressure during peak load times and make use of surplus renewable energy that might otherwise go unused. Google, for example, shifts compute tasks between different data centers based on regional hourly carbon-free energy availability. 

• Set up data centers for “renewable computing”. Certain computations can be processed in batches at a time - such as training AI models, mining cryptocurrencies, or running large-scale simulations (e.g., climate modeling, protein folding analysis). Batchable computing can be interrupted, meaning it is resilient to power loss. Hence, it is perfectly suited for intermittent renewable energy, and can be turned into “renewable computing”. Soluna, for example, locates data centers that run batchable processes near renewable energy sites and buys excess renewable energy that would otherwise go to waste from IPPs. This can make renewables more economically viable and push their penetration by, e.g., avoiding curtailment and providing independent power producers with additional tax credits. 

• Use existing infrastructure better: Certain bottlenecks to renewables deployment - such as interconnection queues - cannot be solved overnight. Therefore, data center operators have to find ways to use existing infrastructure more efficiently. For example, data center companies could encourage utilities and grid operators to deploy Grid-enhancing technologies (GETs) to connect to the grid more rapidly, with lower transmission and distribution upgrade costs, while tapping into renewable energy capacity. Data center operators could also partner with other tech providers to leverage existing generation capacity better. For example, Meta works with a Texas battery storage provider to better use the state’s wind and solar resources. 

2) Reduce energy intensity of computation, e.g.: 

• Build more energy-efficient models. Choosing efficient machine learning architectures can cut computation needs by a factor of ten and substantially reduce energy needs. A 2021 study by Google and the University of California, Berkeley, demonstrated that the company significantly lowered the carbon emissions of its deep learning transformer model within four years. By employing more efficient models, optimizing processors, and prioritizing low-carbon resources, data centers were able to reduce the training footprint by over 700 times. 

• Optimize model development and training. AI model developers typically test thousands of configurations to improve an AI model’s accuracy, a process known as hyperparameter optimization. MIT researchers have found that predicting a model's performance early can lead to substantial energy savings. By halting underperforming models early in the process, they achieved an 80% reduction in energy consumption for training. This innovative approach has proven effective in applications such as computer vision and natural language processing and holds significant potential for advancing the efficiency of AI training. 

• Reduce energy demand from inference. Model inference usually relies on the use of redundant hardware - on “stand by” all the time, ready to perform computations precisely when needed. By leveraging the most appropriate, energy-efficient hardware, inference energy demand can be reduced. For example, researchers at Northeastern University created an optimizer that matches a model with the most carbon- and energy-efficient hardware mix - such as high-power GPUs for computationally intense processes and low-power CPUs for less-demanding computation - resulting in a 10-12% reduction in energy used without compromising the speed and quality at which a model responds. This approach is particularly useful for cloud customers, who lease computational power from data center operators.

• Select the right models for the right use cases. If all of Google’s ~9 billion searches would leverage a large, complex model like ChatGPT instead and turn into 9 billion Chatbot interactions per day, Google would need as much power as Ireland to run its search engine. This shows how important it is to efficiently match models to specific use cases, and thereby reducing overall energy consumption and enhancing efficiency. Different machine learning models have varying computational and energy requirements based on their complexity and intended applications. For instance, simpler models can be used for tasks that do not require high computational power, such as basic classification problems. 

3) Improve the energy efficiency of data center operations, e.g.:

• Reduce data center cooling needs. Cooling is the second largest energy-demand driver in a data center next to computation. Reducing cooling needs requires a holistic approach primarily encompassing data center design, the cooling technology leveraged, and how the cooling system is managed. Regarding data center design, the data center should ideally be located in a fairly cold region and built in a way that it can also leverage passive cooling (e.g., by ensuring good insulation, using heat-absorbing roofing material, and taking advantage of shades). In terms of cooling technology, one advanced underexplored technique is liquid cooling. Liquid cooling systems transfer heat more effectively - with less energy and water requirements - than traditional air cooling by circulating a coolant in direct contact with the heat-generating components. Additionally, using sophisticated data center infrastructure management tools can further enhance efficiency by e.g., optimizing cooling systems (e.g., avoiding overcooling and hotspots). For example, leveraging an AI-based cooling solution from startup Etalytics, enabled a 9% energy efficiency improvement at a data center in Frankfurt. 

• Improve data center hardware use: Utilizing energy-efficient hardware and optimizing hardware setups are crucial for minimizing energy consumption in data centers. For instance, research at MIT showed that power capping - limiting the amount of power a GPU can draw - can reduce energy consumption by approximately 12-15%. Although this extends task completion time by about 3%, the impact is minimal compared to the significant energy savings. For instance, capping GPU power to 150 watts for training the famous BERT language model increased training time by only two hours (from 80 to 82 hours) while saving the equivalent energy of a US household's week. This method also results in cooler and more consistent GPU operating temperatures, potentially reducing cooling needs and enhancing hardware reliability and lifespan. 

What we do now determines whether AI is part of the solution to climate change - or not

We will not be able to stop the race for more powerful AI—and with it, AI’s growing energy footprint—but we can and must find solutions to align AI development and deployment with climate efforts, making AI a critical part of the solution to climate change, not the problem. Let’s not forget - AI holds immense potential to enable ambitious climate action.

However, the choices we make now— such as leveraging AI-related energy demand to boost renewable energy deployment, reducing the energy intensity of computation, and maximizing the energy efficiency of data centers while avoiding the use of AI for climate-negative activities (e.g., AI is estimated to contribute $425 billion in profits for the fossil fuel industry by 2025)  — will determine AI’s role in our climate efforts. 

To create incentives and pressure for these actions, we need AI developers to be more transparent about the direct and indirect energy demand and resulting GHG emissions of their products and services. Greater transparency can help identify areas for improvement, promote accountability, inform decisions on where and how AI is deployed, and guide policy-making—all to ensure AI development and use aligns with climate goals. Today, this transparency is not available.

However, regulators are already heightening their scrutiny. For example, earlier this year, the US Senate introduced a proposal to assess AI’s environmental impact. Lawmakers in Virginia want to mandate sustainability goals for data centers. In California, a new climate disclosure law will require businesses—including key AI players like OpenAI and Google—to disclose their climate risks and impacts in detail.

Most importantly, stakeholders need to work closer together. In particular, data center operators and utilities need to collaborate on the deployment and integration of new renewable energy capacity, optimizing the use of existing power grid infrastructure, and utilities’ energy resource planning.

The pressure to address the “Electric Elephant” in the room is increasing, but great ideas already exist. We now need the right actors to come together and implement these solutions effectively.