TY

Environ_impact_AI_2023

HAL

  • HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents.
  • The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.
  • Environmental Impact of Artificial Intelligence by Etienne Delort, Laura Riou, and Anukriti Srivastava, INRIA; CEA Leti, 2023.

Artificial Intelligence and Eco-responsibility

  • Internship Bibliographic Report by Etienne Delort, Laura Riou & Anukriti Srivastava, September 2023.
  • ENVIRONMENTAL IMPACTS OF ARTIFICIAL INTELLIGENCE

Contents

  • Context of this Bibliographic Work
  • Introduction
  • Important Definitions

Environmental Impact of ICT and AI Among It

  • According to the Intergovernmental Panel on Climate Change (IPCC), from 1870 to 2010, human activities emitted 2100 GtCO2e, leading to a global climate warming of 1.3°C within those 140 years.
  • The IPCC warns about the high risks of going above 1.5°C of global warming.
  • Keeping global warming under this limit was fixed as an objective by 195 countries through the Paris Agreement in 2015.
  • To meet this objective, global Greenhouse Gas (GHG) emissions, estimated at 49 GtCO2e in 2010, must be reduced each year by 9% and reach zero by 2050.
  • Information and Communication Technology (ICT) sector is one of them, in which AI is included.
  • ICT is an umbrella term that includes any communication device, encompassing radio, television, cell phones, computer and network hardware, satellite systems, and so on, as well as the various services and appliances with them such as video conferencing and distance learning.

Environmental Impacts of ICT

  • Different orders of environmental impacts will be shown.
  • The importance and looking at multiple criteria and categories of impact will be proven.
  • ICT’s contribution to global warming in 2020 and its contribution to the latter in the coming decades.

Orders of ICT’s Environmental Impacts

  • Hilty et al. introduce a conceptual framework that classifies the effects of ICT on the environment into three orders or levels.
  • First-order effects: Caused by the physical existence of ICT and include environmental impacts of production, use, recycling, and disposal of ICT hardware.
  • Second-order effects: The indirect environmental effects of ICT caused as a result of its power to transform processes such as transport, production, or consumption, resulting in a reduction or increase in the environmental impacts of the latter.
  • Third-order effects: Caused as a result of medium- or long-term adaptation of behavior (E.g. consumption patterns) and economic structures to the availability of and the services provided by ICT.
  • Understanding these three categories of orders helps with the recognition of the overall impact the introduction of a technology such as AI can have.
  • In the rest of the bibliography, it is primarily first- order impacts that are discussed because those are the ones being partially estimated or measured.

Multi-criteria Analysis of ICT’s Environmental Impacts

  • The earth system has physical boundaries of different natures.
  • The concept of planetary boundaries, introduced in 2009 aimed to define the environmental limits within which humanity can safely operate.
  • These 9 planetary boundaries of course include climate change.
  • A recent study from Richardson et al., re-assessed these planetary boundaries and out of the 9 of them 6 have been transgressed and one is close.
  • For each of the planetary boundaries, a choice is made of one of several control variables to capture the most important anthropogenic influence at the planetary level of the boundary in focus.
  • The introduction of this concept is important for grasping the necessity of measuring not only GHG emissions that primarily impact climate change but also other indicators that account for other environmental impacts and planetary boundary crossing.
  • Moreover, by looking at only one indicator, there is a real risk of moving the environmental impacts from one boundary to another
  • A recent ADEME-ARCEP study mandated Agence Française de l’environnement et de la maîtrise de l’énergie (ADEME) and France’s “Electronic Communications, Postal and Print Media Distribution Regulatory Authority” (ARCEP) to evaluate the environmental impact of ICT in France.
  • The study follows a rigorous methodology called Life Cycle Analysis (LCA) for evaluating multi-criteria impacts and shows that the depletion of natural abiotic resources (minerals and metals) accounts for around a quarter of ICT’s overall normalized pollution in their assessment.

ICT’s Contribution to Global Warming in 2020

  • Gaining growing importance in our societies, the GHG emissions of the ICT sector have become a subject of interest for researchers in the past few years.
  • GHG emissions produced by this sector can be divided into three phases of the life cycle of the devices it is composed of:
    • Embodied emissions which are a result of ICT object manufacture.
    • Usage or operational emissions that are caused during the use phase.
    • End-of-life emissions, that are emitted as a result of the disposal of the object after it ceases to be used.
  • Since ICT devices are electrically powered, the use phase emissions are closely linked to the Carbon Intensity (CI) of the country in which the ICT object is located.
  • The papers studied here obtain the CO2e emissions by multiplying the energy consumption by the CI.
  • In a review paper about ICT’s carbon footprint Freitag et al., the authors have systematically studied peer-reviewed papers post-2015 assessing the environmental impacts of ICT to compare the existing models and results for the same.
  • Based on these works, Freitag et al. have found predicted ICT emissions for 2020 between 1.0 and 1.7GtCO2e, which represent between 1.8% − 2.8% of worldwide GHG emissions for this year.
  • Freitag et al. have split the emissions into 3 domains or tiers: user devices, data centers, and networks.
  • This paper also demonstrates the difficulty of setting a perimeter for the study of ICT emissions.
  • The precise frontiers of this sector are discussed by the researchers.
  • They all include computers, smartphones, tablets, data centers, networks and servers.
  • IoT, blockchain, or intelligent systems in cars are not taken into account yet they enter into the definition of ICT as they rely on the same base.
  • Since these examples have high growth potential, they can also grow ICT’s emissions.

ICT’s Contribution to Global Warming Projected for the Coming Decades

  • If the ICT sector followed the necessary decarbonization trajectory identical to the rest of the economy, it would have to reduce its footprint by 42% by 2030, 72% by 2040, and 91% by 2050, then reach net zero or have an equivalent effect on other sectors.
  • Meanwhile, the sector is expected to grow according to 2 out of 3 research groups.
  • Andrae and Elder expect an exponential increase in the GHG emissions of the sector, more realistically an increase by 50% by 2030, and by 500% in their worst scenario.
  • Belkir and Elmeligi also believe that an exponential increase is more likely, causing ICT’s GHG emissions to increase 2.6-fold by 2030 and 5.1-fold by 2040.
  • On the contrary, Malmodin and Lund´en forecast a plateau for the sector because of a saturation phenomenon.
  • Freitag et al. argue against that by saying that ICT companies generally have a strong incentive to prevent saturation from happening.
  • Also, Freitag et al. prompt the readers to carefully explore little-known ICT domains.
  • They especially urge the scientific community to assess the impact of IoT since the number of IoT devices is expected to explode from 15 billion internet-connected devices in 2015 to 75 billion of such devices in 2025 according to Statista.
  • In parallel, AI is also expected to grow massively.
  • Freitag et al. argue that the world’s data is doubling every 2 years, that it was already 59 ZetaBytes in 2020, and that AI computational training cost is doubling every 3 months.
  • Closely related, AI and IoT create a risk of accelerating the environmental impacts of ICT dramatically.
  • Therefore, measuring AI is crucial.

Artificial Intelligence Among ICT

  • To our knowledge, no environmental assessments have been made for AI as a domain yet.
  • However, since AI is a part of the ICT sector, its impacts are included in the impacts of the latter.
  • Artificial Intelligence is raising environmental problems of the same nature as the rest of the ICT.
  • It relies on more or less similar hardware (with the exception of some specialized accelerators used for training large models), with multiple significant impacts during the manufacturing and end-of-life phases, and the main concern up to now is the GHG emissions due to energy consumption during the use phase.
  • Nevertheless, AI is remarkable among ICT for several reasons:
    • It is noticeably data-, hardware- and compute-intensive
    • Poorly assessed
    • Getting a lot of political attention

AI Is Compute-, Data- and Hardware-Intensive

  • Nowadays, AI solutions are becoming more and more widespread in society. With the aim of always scoring higher on performance metrics, the AI models are growing exponentially in several respects:
    • The number of operations performed during training
    • The amount of data used
    • The energy consumption
    • The monetary cost
  • Thus, deep learning outgrows classical Machine Learning regarding all those indicators.
  • And more recent techniques like continuous learning are even more costly.
Computation-intensiveness
  • AI is increasingly compute-intensive.
  • According to an article by OpenAI, the computational power and in turn, the energy required for deep learning research and development has been doubling every 3-4 months resulting in an approximately 300, 000 times increase between 2012 and 2018.
Data-intensiveness
  • Improvements in specialized hardware and algorithms have given rise to AI models that are trained on very large data sets which, despite optimizations, have a large carbon footprint due to computation, network, and storage costs.
  • Acquisition and transfer of such vast amounts of data have an added cost.
  • According to a report commissioned by the European Parliamentary group of the Greens/EFA, AI relies more on hardware and data than the rest of the ICT and this can be attributed to the above- mentioned reasons.
Hardware-intensiveness
  • AI relies on specialized hardware such as GPUs and TPUs which are responsible for many embodied emissions.
  • To demonstrate this point, we can consider the fact that AI is the main driver of growth in Meta’s data centers and Nvidia, an American multinational technology company that is a software and fabless chip-making company that designs graphics processing units, application programming interfaces for data science and high-performance computing, is experiencing huge growth this year thanks to the boom in generative AI and its underlying need of those specialized processors.

AI Is Poorly Assessed

  • Despite the efforts by a few to measure the environmental impact of AI, the state of reporting in AI research publications on carbon footprint is low, and is way lower when it comes to other environmental metrics.
  • Environmental reporting for AI is uncommon.
  • In 2019, the rise of large NLP models trained on large amounts of data caught the attention of researchers on the topic which led to a publication regarding their energetic, financial, and environmental training cost.
  • Then the introduction of the Green AI concept and tools to measure its energy consumption and related GHG emissions.
  • Henderson et al. randomly sampled 100 NeurIPS papers issued in 2019 and found that out of these:
    • 1 measured energy in some way
    • 45 measured run-time in some way
    • 46 provided the hardware used
    • 17 provided some measure of computational complexity
    • none provided carbon metrics.
R&D impact is underestimated
  • Big AI models are usually associated with a considerable R&D cost, especially in terms of energy consumption and hence carbon footprint.
  • Before the training of the final model, a lot of experimentation is led.
  • Smaller models are trained to benchmark the several model components that will be used in the final version.
  • When the final model’s structure is decided, during a phase called hyperparameter tuning, short training sessions are performed to select a definitive set of hyperparameters.
  • However, usually, only the efficiency of the final model is published.
  • This leads to an underestimation of the true impact of AI research.

Political Interest in AI

  • AI is becoming a priority in innovation policies worldwide.
  • For example, AI has been getting a lot of attention from the French government in the last few years.
  • In 2017, the French deputy and mathematician C´edric Villani led an initiative regarding the creation of a France and Europe-wide AI strategy.
  • Out of the propositions made in this report, with the aim of pioneering innovation by 2030, France has invested 1.5 billion euros in AI between 2018 and 2021, and 2 billion euros more were scheduled for 2021-2025.
  • In 2025, AI should produce an income of $90B for France, while it produced only $7B in 2020.

Paths for Taking AI’s Impact into Account

  • As said previously, many software tools for measuring the energy consumption of AI training were introduced in 2019 following the rise of awareness of its high energy demand among ML practitioners.
  • These tools are part of a measurement approach, which is indeed a lever for action to reduce environmental impacts.
  • It is important to note that the carbon assessment of AI services is the most common environmental assessment, partly because it is the easiest to carry out.
  • However, as shown in Section 1.1, merely measuring the carbon footprint of the training phase of an AI lacks completeness.
  • So, other ways of taking into account the environmental impacts of AI need to be explored.
  • Ligozat et al. add proposals for reducing the environmental footprint of AI.
  • Their paper is targeted towards the community of machine learning practitioners.
  • It suggests going beyond simply measuring the impact of training AI models.
  • Their suggestions are divided into two categories:
    • Measures to be taken as a practitioner
      • Reduce your I/O and redundant computation/data copying/storage: Start with smaller datasets to debug your model, and use shared data storage with members of your team so you don’t need to have individual copies.
      • Choose a low-carbon data center: When running models on the cloud, consult a tool like Electricity-Map to choose the least carbon-intensive data center.
      • Avoid wasted resources: by steering clear of grid search and by reusing or fine-tuning previously trained models when possible. Also, strive towards designing your training and experimentation to minimize discarded computing time and resources in case of failure.
      • Quantify and disclose your emissions: use packages like CodeCarbon, Carbon tracker, and Experiment impact tracker, which can be included in your code at runtime or online tools like Green algorithms and ML CO2 Impact that can allow you to estimate your emissions afterward. In both cases, share these figures with your community to help establish benchmarks and track progress!
    • Measures to be taken as an institution:
      • Deploy your computation in low-carbon regions when possible.
      • Provide institutional tools for tracking emissions and enable them by default on your computing infrastructure.
      • Cap computational usage at say a maximum of 72 hours per process, in order to reduce wasted resources.
      • Carry out awareness campaigns regarding the environmental impact of ML.
      • Facilitate institutional offsets for those emissions that cannot be avoided, such as commuting and building construction.
  • In addition, some researchers want to integrate environmental considerations into the design of AI projects.
  • For example, Lef`evre et al. propose an environmental assessment framework document intended for responses to calls for projects involving Artificial Intelligence (AI) methods.
  • It helps to take into account various environmental criteria, in particular, the general impacts of digital services but also the specificities of the AI field (impacts of the learning and inference phases, data collection, etc.).
  • This framework document provides an excellent summary of the aspects to be taken into account when assessing the various types of impacts of the proposal for a new AI service.
  • For example, measuring and publishing the carbon emissions of training AI models is a first step towards taking account of the environmental effects of AI.
  • Indeed, energy measurement methods and tools for estimating carbon emissions are important.
  • However, in order to be exhaustive, the impacts of AI must be considered in a multi-criteria and multi-order manner (direct, indirect, and societal impacts), but this is not the most widespread paradigm as of now.

Quantifying the Environmental Impact of AI

  • The booming evolution of AI described in Section 1.2 makes AI environmental impacts seminal to study and quantify.
  • Currently, existing methods for measuring the impact of AI mainly focus on measuring energy consumption and the carbon footprint of machine learning algorithms during the training as well as inference stages.
  • However, the carbon footprints of training and inference respectively only make up a small part of AI’s overall impact on the environment.
  • As explained for ICT in Section 1.1.2, the environmental impact of AI is not restricted to carbon emissions.
  • Metrics such as abiotic resource depletion, water consumption, as well as the impact of biodiversity, need to be studied alongside the carbon footprint even though these are harder to measure due to the lack of specific methodology and data.

Life Cycle Analysis for AI

  • The LCA methodology built by ETSI/ITU is a way to exhaustively assess the environmental impacts of ICT.
  • LCA is the most widely recognized methodology for environmental impact assessment.
  • It is a standardized method with ISO standards: 14040 and 14044.
  • This methodology covers different steps of the life cycle of the target system (product or service) and quantifies multiple environmental criteria.
  • Since a single impact category cannot result in the proper evaluation of a product, multiple impact categories need to be studied.
  • ADEME, the French national environmental agency, counts 13 environmental indicators, including impacts on air, water, earth’s resources, as well as human health, and emphasizes the importance of being exhaustive about these indicators for environmental impact assessment.
  • The ITU standard states that out of the various existing impact categories, climate change resulting from high energy consumption is an important category that is hence mandatory.
  • Nevertheless, they mention that it is important to consider other impact categories such as ozone depletion, human toxicity, ionizing radiation, eutrophication, acidification, land use, and resource depletion (water, mineral, or fossil).
  • However, there is no consensus on the criteria to take into account within the LCA community.
  • So, it is up to practitioners to decide which impact categories are relevant based on the ICT product system being studied and its purpose.
  • Ligozat et al. propose a methodology for applying LCA to AI services that relies on the specific methodology standard of International Telecommunication Union.
  • They illustrate the insufficiency of the current reporting with Figure 5.
  • A proper LCA of an AI service should include production, use, and end-of-life impacts, and those impacts should cover not only global warming potential but also water usage, human toxicity, and abiotic depletion potential.
  • In the literature, some papers reporting the carbon footprint of training AI models have been published lately.
  • Using carbon footprint as the only metric is however far from exhaustive for measuring the entire environmental impact of an AI.
  • A big mistake that is made when it comes AI is the narrative around its immateriality.
  • As Ligozat et al. show in their paper, each task of an AI service relies on dedicated hardware, and each of them should be the subject of a LCA if one wants to exhaustively assess the impacts of an AI service.
  • Nonetheless, it is hard today to find publicly available data regarding the environmental impacts of hardware used for AI such as GPUs.
  • Popular Life Cycle Inventory (LCI) databases such as NegaOctet for ICT or EcoInvent that cover a lot of general domains cost several thousands of euros.
  • Thus, Ligozat et al. suggest that the AI community could lobby companies to open more of their data so that AI practitioners can start reporting the environmental impacts of their models in an exhaustive manner.

Measuring Energy Consumption & Carbon Footprint of AI Computation

  • Energy consumption is usually what has been looked at in papers when it comes to AI’s environmental impact since the key paper from Strubell et al. raised awareness on this topic.
  • AI services can have a high environmental impact in terms of GHG emissions because of the substantial energy consumption of the computational facilities used to develop and train them.
  • Calculating and estimating the carbon footprint of AI projects systematically can help raise awareness, encourage the development of energy-efficient software, and limit the waste of resources.

Relationship Between Energy Consumption and Greenhouse Gas Emissions

  • Most of the biggest AI models are trained in data centers.
  • On the other hand, smaller AI models can be trained on personal computers.
  • This section focuses on how training an AI in a data center leads to GHG emissions.
  • The training is done on hardware specialized in parallel computation (when available) -GPUs or TPUs - benefiting from the fact that training computations are highly paralleliz- able.
  • Such types of hardware consume significant amounts of electricity.
  • For example, the NVIDIA Tesla A100 GPU power consumption can reach 300 W, which is about the power consumption of a plasma television.
  • Additionally, training requires a CPU (or several) and some Dynamic Random Access Memory (DRAM), though their consumption is often way lower than the one of GPUs.
  • Performing computation in data centers involves an energetic overhead in addition to the energy consumed for the computation itself.
  • For example, cooling the server on which the computation is performed is energy-consuming.
  • To take this into account, Power Usage Effectiveness (PUE) is generally used.
  • The PUE, which is an efficiency metric for data centers, describes how efficiently a computer data center uses energy. It compares the whole energy consumption to the energy consumption dedicated to computation.
  • Its formula is given in Equation 1: PUE = \frac{TotalFacilityEnergy}{IT EquipementEnergy}
  • According to Brady et al., the energies used in Equation 1 should be obtained by tracking the consumption of the data center over a year.
  • This is because the consumption of the total facility varies over time.
  • The closer the PUE tends to 1, the more optimized the use of the computing resources.
  • Nevertheless, as stated in Brady et al., it is important to note that a good PUE does not necessarily depict eco-friendliness.
  • For the rest, PUE on being multiplied with the energy of computation gives the energy consumption of all the devices in a data center that enable a model’s training.
  • This can be done like in Equation 2: Etraining = PUE \times (EDRAM + ECPU + EGPU)
  • Finally, the energy consumed during training is linked to GHG emissions through an indicator called carbon intensity (CI).
  • It describes how polluting the electricity powering the data center is in terms of GHG emissions with coal-based electricity being more polluting than wind energy, for example.
  • GHG emissions, energy consumption, and CI can be linked as in Equation 3. This equation was introduced in a paper by Strubell et al.
  • mGHGtraining = Etraining \times CI

Methods for Measuring the Energy Consumption of Computer Nodes

  • As described by Jay et al., the energy consumption of a running program can be measured in four different ways:
    • External devices (Wattmeters and Power Distribution Unit (PDU))
    • Intra-node devices (Baseboard Management Controller (BMC))
    • Hardware sensors and software interfaces (Running Average Power Limit (RAPL), NVIDIA System Management Interface (nvidia-smi))
    • Power and energy modeling (based on power with the Thermal Design Power (TDP) or based on usage).
  • One way to access power information is through physical power meters which are external devices that are hence not embedded in computational nodes and measure the entire power consumption of the node.
  • They are usually placed at the interface between the power socket and the power supply unit.
  • Some energy measuring devices can also be placed inside computing nodes such as a BMC, which is a small and specialized processor used for remote monitoring and management of the host system it is placed on.
  • It is placed between a computing node’s power supply and the main board. It can give information at the component level(CPU, GPU, DRAM, etc.).
  • Some CPU and GPU vendors provide tools to track energy consumption.
  • Intel and AMD for instance provide an RAPL interface each (intel RAPL, AMD RAPL).
  • This interface, among other things, enables access to the value of energy consumed since the processor was started. It gives this information for different power domains that are physically meaningful like DRAM or the entire CPU socket.
  • On the other hand, NVIDIA GPUs provide information about its energy consumption through the nvidia- smi.
  • Strubell et al. query these interfaces to obtain the average power draw from the hardware used for training the four NLP models being studied.
  • Lastly, when these previously cited measurements are imprecise or unavailable, one can resort to approximations of energy consumption.
  • This can, for example, be done with a constant average power value or a proportion of usage of the measured device.

Software Power Meters

  • The energy model used by Strubell et al. for estimating the energy needed to train large NLP models is also implemented in some open-source tools available as Python packages:
    • CodeCarbon
    • Carbon Tracker
    • Experiment Impact Tracker
    • Cumulator
    • Eco2AI.
  • These tools use mainly the RAPL and NVIDIA’s NVML library or TDP to make estimates.
  • Although the above-mentioned tools were developed especially for AI workloads (machine learning, NLP, deep learning), they work for other types of computation too.
  • Since these tools offer software-based ways to measure the energy consumed during the execution of a code snippet, they are referred to as “software power meters” in the paper by Jay et al.
  • Even though these tools have the same goal, there are significant differences between them that are highlighted in the previously presented papers.
  • These differences include:
    • The OS and hardware compatibility
    • The release date
    • The open source license
    • The carbon intensity used
    • The multiplication by a PUE factor or not, the value of PUE used and if it’s configurable or not
    • The value of carbon intensity used and if it is real-time or not
    • The way the results are presented
    • The use of usage factors
    • The frequency of measurements
    • If the code has errors that require code modification
  • The tooling used to measure CPU GPU, and Memory consumption.
  • Tracarbon and Cloud Carbon Footprint can also be cited but they aren’t documented in a scientific publication.
  • They may have specificities that have been overlooked here.
  • Other tools measure energy without supplying a CO2e measure.
  • The previous tools of this section are included in this study among others.

Predicting Energy Consumption & Carbon Footprint of AI Compute

  • In order to have some control over one’s GHG emissions, one may want to predict how much an AI task will emit before running it.
  • Several methods to do so have been developed, focusing on different targets of the AI workflow.

Prediction from partial measurement

  • Carbon Tracker simply predicts total energy consumption by averaging the energy consumption of previous epochs.

TDP approximation

  • Some simple tools available online like ML CO2 Impact and Green Algorithms fulfil this goal.
  • They use TDP as an approximation for the power consumption of hardware.
  • It is a hardware-specific value provided by the manufacturer and that represents the maximum amount of heat generated by the component under a steady workload.
  • If the amount of time required for the job is known, the total processor energy consumption can be estimated as follows:Etotal = TDP \times t
  • There are 2 issues with this equation:
    • One, in most cases one doesn’t know the execution time before running the program, so it limits the predictive capacity of those tools.
    • Two, it doesn’t grasp the effects of the executed code’s nature of power consumption.
  • Some steps of the execution might be computationally intensive while others might let the processor almost idle, so there is no guarantee on how good the TDP approximation is.

FLOPs as a metric of complexity

  • There is a need to find a better metric than time to describe the complexity of a piece of code.
  • Schwartz et al. propose instead the Floating Point Operation (FLOP)s.
  • Roughly, they are the number of operations that need to be performed by the hardware to execute a piece of code.
  • It is computed analytically by counting two base operations, ADD and MUL.
  • They have some interesting advantages.
  • They can be computed by parsing the code analytically, without executing it.
  • For example, in Equation 5, the FLOPs count is equal to 2, due to one + and one × operator.:x \leftarrow x + y \times z
  • A variation of FLOPs that can be seen in the literature is called Multiply Accumulate Operations (MACs).
  • It has the same idea of operations counting but in Equation 5, the MACs count would be 1.
  • It fits the behavior of some hardware, that is able to perform both the multiplication and the accumulation at once.
  • Also, the number of FLOPs is agnostic to the hardware on which the code is used.
  • Thus they are a metric of choice to describe the workload of a piece of code.
  • In particular, it is possible to compute the number of FLOPs necessary for an AI model inference.

Predicting Inference Consumption With FLOPs

  • Rodrigues et al. lead experiments to link FLOPs to the energy consumption of Convolutional Neural Networks (CNN) for inference.
  • They pick a set of 9 CNN models. Within this set, they run inferences on an Nvidia Jetson TX1 while measuring the energy consumption, the number of bus accesses, the number of Single Instruction Multiple Data (SIMD), and the number of MACs.
  • Afterward, they try to find relationships between those parameters.
  • They first find a linear relationship between the number of bus accesses and the energy consumption for an inference.
  • Then, they find a second linear relationship between the number of bus accesses and the number of SIMD.
  • Next, another linear relationship is found between SIMD and MACs.
  • Thanks to these linear relationships, Rodrigues et al. create a linear predictor from the 3 linear relationships, using solely the number of MACs as input, to predict the energy consumption.
  • Their linear regression shows satisfying results, with a relative error of 7.08 ± 5.05%.
  • Nonetheless, it is worth noting that the relationships and predictor depend on the hardware used: running the same model on different hardware doesn’t consume equally.
  • Moreover, they didn’t try to see if those results were generalizable to other models.
  • Provided that the results can be generalized, their conclusion is that it is possible to predict the energy consumption of an unknown CNN inference on the NVIDIA Jetson TX1 architecture.

Predicting training consumption with FLOPs

  • Mehta et al. focus on predicting the energy consumption of training a Deep Neural Network.
  • To predict the energy of training, they make the following assumption:EnergyT raining ≈ T.(EnergyF P + EnergyBP )
    • Where:
      • T is the number of epochs
      • EnergyF P is the energy required for a full forward pass
      • EnergyBP is the energy required for a full backward pass
  • They characterize the energy consumption of a forward pass and backward pass as follows:
    • Energycomp F P ≈ bs(\alpha{flop}\sum^{S-1}{s=1} n(s)n(s+1) + \alpha{act}\sum^{S}{s=1} n(s))
    • Energycomp BP ≈ bs(2\alpha{flop}\sum^{S-1}{s=1} n(s)n(s+1) + \alpha{err}\sum^{S}{s=1} n(s))
      • Where:
        • bs is the batch size used for training
        • \alpha{flop} is the energy cost of Floating Point Operations on this hardware, which can be obtained by running a micro-benchmark program
        • \alpha{act} is the energy cost of the activation functions used (ReLU, Tanh, etc.), also obtained with a micro-benchmark program
        • \alpha{err} is the energy cost of computing the propagation error in a neuron (micro-benchmark program)
        • n(s) is the number of neurons in the layer s.
  • They run small programs on the NVIDIA Tegra K1 development kit to measure the energy consumption of the different kinds of operations of training.
  • They count the number of each operation, weigh them with the associated energetic cost, and sum the energetic costs together.
  • Similar to Rodrigues et al., they propose a method to evaluate energy consumption without running the whole training.
  • A limitation of this work is that the model created to predict the consumption only works on a given hardware.

Measuring Water Consumption & Water Footprint of AI

  • Water consumption is an important environmental impact of AI which is rarely reported in peer- reviewed papers.
  • According to the United Nations World Water Development Report, nearly 6 billion people will suffer from clean water scarcity by 2050.
  • Boretti et al. suggest that this number may be higher owing to not only the drivers of water scarcity like population (economic growth and water demand), resources, and pollution but also unequal growth accessibility and needs, which are underrated.
  • Water is often used in data centers for cooling servers and for maintaining humidity levels.
  • According to Holzle, data centers that are water-cooled use approximately 10% less energy.
  • Hence, they have approximately 10% less carbon emissions as compared to many air-cooled data centers.
  • This explains in part the trend of switching from air-based to water-based cooling in data centers.
  • While methods like LCA take into account the embodied water footprint of AI models (for example, that for manufacturing hardware on which an AI model is trained), Li et al. take into account the operational water footprint of AI models associated with only training and inference.
  • The water footprint of AI models, as estimated by the latter, depends on the type of cooling system used in a data center, the water consumed in producing the electricity used to run the AI programs on the servers, the time of the day during which one runs the AI programs as well as the physical location of the data center.
  • For example, according to Li et al., training GPT-3 in Microsoft’s U.S. data centers can consume 700,000 liters of clean freshwater, while training in Microsoft’s Asian data centers means triple the water consumption.
  • Li et al. use Google’s large language model, LaMDA, to demonstrate their methodology for water footprint estimation.
  • They also underline the need to address the water footprint along with the carbon footprint for creating more sustainable AI.
  • In addition, they