AI for Power & Energy with Laurent Boinot - #683

0:03All right, everyone. Welcome to another episode of the Twimple. I am your host Sam Charrington And today I'm joined by Laurent Boineau. Laurent is Power and Utilities Lead for Americas at Microsoft. Before we get going, be sure to take a moment to hit that subscribe button wherever you're listening to today's show. Laurent, welcome to the podcast. Thank you, Sam. Thanks for having me. I'm looking forward to digging into our conversation. We'll be talking all about AI and energy. Before we dig into the topic, I'd love to have you share a little bit about who you are and what you do. So I am initially from France, but I live in Toronto, Canada.

0:40I'm Canadian as well. And my background is basically consulting for the energy sector. I have an experience as an entrepreneur as well. I had my own online learning company. And then when I filled the business, moved to Canada with my family. And I've been at Microsoft now for three years, focusing on power and utility, covering the entire continent with a mess. Energy and AI is a potentially very broad topic. How can we maybe start to frame out the role of energy and some of the levers that AI can begin to manipulate? Yeah, so I think it's important first to talk about what energy is and what it isn't.

1:26Because if you ask the average person or me before I took this role, basically, you think about energy like the price on your bill, basically, or filling the car. or if you're an economist you think about energy as a sector of the economy but energy is really our ability to change the world and I don't mean that in a big philosophical way I mean that's actually the definition of the word energy if you do anything you either release or use energy so if you change the temperature in a room that's energy if you lift a piece of paper or anything that's energy the heavier the more energy if you drop it you lift this energy so in a very real sense the energy sector is all about making things happen and in an even more real sense with the advance of the industrial revolution because the way basically the energy sector is feeding the machine if that makes sense And a lot of our lives, from driving, but also from food production, from obviously anything we wear, anything we manufacture, is dependent on machines.

2:49And the machines eat energy. And we eat the production of processes that involve machines as well. So it's very, very core. That word energy is very, very core to anything we do. And so, having laid that landscape, you can then talk about the fact that we see reliable access to energy has everything to do with economic growth and job creation. And then you can talk about the technological innovation that enable some access to energy, because things that relied on more energy before can be done now with much less energy. But also energy itself allows for reaching out to more energy. And you can talk about the kind of geopolitical implication of reaching for energy.

3:45You can talk about the resilience, the disaster preparedness of energy production. But in my role specifically in power and utility, I'm at the crux of the environmental impact conversation. Because we are going from a world where, when I say energy now, it's mainly electricity. So electricity consumption, energy consumption was stable, basically in line with population growth to now with data centers and electric cars and the electrification of a lot of processes, including steel mills, by the way. We're now in a situation where the grid, so the big meta machine or meta system that allows us to distribute this electricity around, that big machine may need to be upgraded.

4:45upgraded with actual shovels on the ground, new transmission lines, new generation, smarter distribution, but also upgraded and digitized, made smarter to enable the future we want, rather the future that we will get if we don't do anything. One thing that's key there, and I guess it's obvious to anyone who thinks about it, and we've been talking about it for a little bit, and that is that there are two sides to the impact of AI on power in particular. One is the load that AI and data centers are creating, and the other is opportunities for optimization and more effective distribution. How do you characterize the impact of of data centers and ai have you um do you follow any of those stats about the you know how the growth of ai is impacting power you know more concretely yeah yeah so uh microsoft is opening one data centers every one that i start microsoft is opening one data center every three days so that in a very and those are not those are not a server literally new data centers every three days yeah and yes is this is that rate uh projected to continue uh you know or accelerate right now yes because the big bets are security and ai at the moment.

6:25But there is a fact, to answer your question, there's a fascinating tension between the increase in data centers, therefore increase in consumption. AI also with an interesting retroactive loop allowing a lot of efficiency gains in the system. But also AI going from being statistical models to very, very power-hungry Gen AI models. And then those Gen AI models going from LLMs to SLMs. So by that, I mean that the huge power requirements of training large language models may not necessarily be replicated when you train a much smaller model. They can do the same thing as the large language model did six months before.

7:19and also there is an efficiency gains in the chips every generation so there's a lot of different graphs with different exponentials in different directions um i was just going to jump in and say even on its face like the data center every three days is surprising from the perspective of uh it surprises me um and i think some of the context that i was bringing to the conversation is like thinking about the rise of cloud computing and how that has meant a consolidation. I wonder if we think about enterprises closing data centers and companies like Microsoft opening data centers, like what the net new data centers is.

8:11Is that positive or negative? So that's a great question. And it's definitely positive. the net is positive uh you'd have to close a lot of data centers to uh uh to net out the impact of the and i also would have thought that the data centers were were getting larger um more so than opening up new ones uh yeah but there's also value in being uh well there's several things here there's value in being close to your customers there's value in having obviously redundancies So you don't want all the data centers in the same place. And there's value in being close to new generation. And also some local regulations basically force hyperscalers to have local data centers as well.

8:58It surprises me a little bit as close as I've been to this from the cloud perspective and generally from the AI perspective. That's still a surprising number in terms of new data centers opening. Yeah, but again, going back to trying to figure out where this is going, putting my old McKinsey on and trying to predict the future, you can keep going with the exponential and try to figure out how much consumption the centers will have eventually. But an interesting data point for me is looking at you today using Bing or ChatGPT on your device. And when you do that, you actually obviously don't use it on device.

9:45Your device is just sending a request and a big data center is actually on Azure in both cases. the request is processed and then the answer is sent back to your app on the device. And that was the state of the art not so long ago, not six months ago or so. But with small language models that Microsoft is working on and others, like 5.3, for instance, or Microsoft, you can have the same output on device. And so that big consumption that I just described is gone. So maybe your device will get a bit hotter. Maybe you need to charge slightly earlier tonight. But still, we're talking about a 10 times reduction in consumption or more.

10:35So the conversation is much more complex and just keep plotting a lineup. up. The answer lies more into, and that goes back to your question about closing data centers at clients as well, closing on-prem data centers. So the question of what the future will look like is a lot about what you do with this intelligence, because you have this deep artificial intelligence now? And is this just fun and games? It's just something you do on the side? Is it just a request a day just to confirm it works? Or do you deeply, strategically reinvent your business with these new capabilities? And if you do, then you rely very, very heavily on processes that today consume a lot of compute, therefore a lot of power.

11:29Microsoft has a mandate to be carbon negative in 2030. And that means removing all carbon from data centers by 2025, which is next year, at the end of next year. And there, basically, that means that for Microsoft to keep operating, you need a reliable, clean energy. And that's interesting for two reasons. Well, many reasons, but the two main reasons for the purpose of that conversation is how does that work? What needs to happen on the grid? How much solar power you need? How many roofs you need to cover? How many new power stations, new clean power stations you need to have in place to make that happen?

12:15But also from an AI standpoint, what needs to happen on the grid to prove that the electron you just got is a clean electron? Because there's a big difference between being net clean, which means that you are consuming, let's say, 100 megawatts of energy, and you've also bought 100 megawatts of clean energy. And actually knowing that every single megawatt that you have bought was actually produced cleanly. And that's a very different conversation. And that's something we're working with at Microsoft with partners like Constellation, for instance. we have a tool called 24-7 matching where we can confirm exactly that that sounds a lot like some of the work that's happening in other supply chains to ensure some degree of fairness I'm thinking like fair trade for food and similar types of governance regimes yes so I'm not an expert in those areas but one of the specificities and one of the things that makes power and utility absolutely fascinating is that when you want to eat a tomato in my world, you have to grow it the moment you want to eat it.

13:37There's very, very little ability to actually conserve the tomato somewhere, like to put it in the fridge or in the freezer or to stock it anywhere. So if the electrons that allow our conversation right now, or the electrons that allow people watching or listening, were created, like for us right now, in my case, somewhere in a nuclear power plant near Toronto, right now, not six months ago. So the whole conversation around how do you add renewables to the grid or how do you prove that your electricity is clean is really a real time conversation. You cannot go back and actually trace. You have to do it at the moment.

14:23You made a comment, if I'm remembering correctly, in one of our previous conversations that suggested that from the perspective of the power grid, having too much power is just as bad as having not enough power. Yeah, yeah. Sometimes you read in the press that the price for electricity goes negative. And there is a lot written about, you know, maybe there's something wrong with the market. But no, it's quite simple. this morning you had a shower and i'm sure you paid for the water right you were happy you knew that by getting under the shower you were expecting water to fall in your head and you're happy to pay for that water to fall in your head when you get out of a cab in a windy new york november evening you are happy to pay for an umbrella to stop the water from falling on your head it's the same molecule, it's the same head, different situation.

15:21So it's exactly the same for electricity. If you don't have enough of it, that's a brownout or a blackout, you need to cut distribution lines. For whatever reason, that's a big problem. When you have too much of it, it's also a big problem because it actually results in the same situation where you end up with fuse going off and you basically cut lines as well. And the end consumer is affected all the same. So this necessary, and again, it's not a six month by six months balance you have to find. It's a real time balance in the generation and consumption of energy. But that also means that, and it's worth digging into this, when you think about right-sizing the grid or making sure that in a specific geography, you can adapt to a growing population or a new factory or new usage.

16:19We can talk about new usage for electric cars, for instance. Or when you bring in new power through renewables. Again, think about the water on your head. Is it water that's falling where you want the water to fall or is it water that's falling where you don't want water to fall? So if you have new usage in the evening with electric cars charging in the evening, can you compensate that by bringing new solar generation during the day? No. There's storage in the middle. Otherwise, you have an imbalance. You add more to the grid when nobody's using it, or not enough people. And then you want to take electricity from the grid when you haven't added any generation at night.

17:07So that's why I said that in the whole sustainability conversation, obviously renewables are actually part of the conversations. And you have to move away from coal and a lot of things like that. But you always have to keep in mind two things when you think about adding generation to the grid. One is, am I adding something that I can turn off or on whenever I want? That's key. And two, what is the charge factor of what I'm adding on? So is what I'm adding on, does it have the ability to work for a whole month if I want to? or will it be on and off during this month? If you think about what's happening behind the meter, the reason why the whole industry is looking at smart meters and advanced metering infrastructure is because understanding what happens behind the meters not only allows you to more precisely predict what you need on the generation and distribution transmission, but you can also, by adjusting consumption, answering some of the previous conversation we were having.

18:30Not consuming is the same as bringing more power in. So it's called peak shading. So you have to size the grid for the peaks. Otherwise, you will have it browned out, basically. What are some of the ways that AI is playing a role in helping energy companies, power utilities, manage the complexity of delivering power, forecasting demand, all these things? Yeah, so forecasting demand is a great example. So the energy industry, the power in your city industry, has a crazy amount of data, not used at all to its full potential at the moment, but there's a crazy amount of data. And if you think about what used to happen in the past, still happens in most places, with the old-fashioned meters, you basically read, like literally you read the meter by sending an employee who will open the little box and confirm how much power was used in the past year, for instance.

19:36So we have no understanding, no visibility as to when and why the power is used. And the whole value add of the smart meters, Sometimes we think about smart meters enabling the distributed energy resources, like allowing solar power, allowing you to rely on your own little windmill. But it's more basic than that. It's just making sure that the provider of energy really understands where the need is. And we had a project in Canada where we were able to help our customer, so a power energy company, use data from the meters to identify areas of a city that needed new substations and new transformers.

20:37because we were able to detect the presence of electric cars because of the data usage. So this is an example of how this feedback cycle, this loop that you didn't have before with the kind of, quote-unquote, dumb infrastructure, you were not able to see that and we just get a run out, potentially. But then also, AI is a big part of allowing what's called virtual power plant or the prosumer, which is when individuals have solar roofs or their own, again, small windmill. they then can supply the grid and or in some cases be almost off the grid and only use the grid for resilience because with a small battery they're able to just never really use more than what they've been able to store the day before.

21:42And all this is managed automatically through AI systems. And I'll keep going because something that's fascinating to me is some of the conversation around electric cars. So there are different levels of view. There are the petrol heads that just will never get over the fact that a electric car does smell out of petrol. So, okay, let's move on from that. That's a different conversation. Then there's a conversation around are they really more sustainable than the old-fashioned internal combustion engine cars? Because yes, there's a lot of mining required for the battery. in my opinion that conversation is dealt with already if you never use your car if you literally buy a car just to look at it which some people do on the high end but not the average person it is possible that indeed just having a normal engine is less taxing on the environment but if you actually drive your car you need to burn oil and in that case the electric car would be far superior over even a short lifespan.

22:55So that's the second conversation. The third conversation is trying to understand the impact of EVs on the grid. And this is absolutely fascinating because you see a lot of badly plotted graphs or rough order of magnitude estimates, but that really, in my opinion, really miss the crux of the conversation, which should be how much energy do people spend driving a day? And can the grid cope with that? That's the only thing that matters. If you look at the physics, the question is, how much energy do you need in your car? And can the grid provide that for everybody or like a neighborhood? The question is not, can everybody supercharge their car at the very, very same time on the same day?

23:45If that's the question, then yes, the grid cannot cope with people driving EVs. But that's not what the question should be. And are you referring to incentive schemes and others like what you mentioned earlier in terms of distributing load and getting people to charge at different times? Yes, that's part of it. I mean, if you don't have control over everyone plugging their car in at the same time when they get home from work at six o 'clock, like how is that not the question? Exactly, but it can be a lot more basic than this. It can be just making sure people understand that they don't need their car to be full in the morning, for instance.

24:22I don't have the stats in front of me, but I think the average American drives 30 miles a day. An electric car is typically close to 300 miles of range. So if you come back home in the evening, you might not even need to charge for a week in some cases. So the idea that really the grid needs to be dimensioned for that charging every night, first of all, that's not the right question. But also the speed of the charging is very, very different. If you want the swimming pool to be filled in six months, then we'll just put the garden hose and wait. Right. Yes, there'll be some evaporation from the sun, but, you know, it will rain as well.

25:10So all in all, just wait. if you need to be filled in 10 minutes then you know you need a big big track or like a big plane to come immediately and dump a bunch of water in it it's a completely different architecture it's a different mindset and the physical world needs to look different for a different reality so like when I don't actually own a car but I typically rent electric cars when to travel. And I now understand that even with 110 in the US and in Canada, that's typically enough for the driving I do during the day. I mean, we're talking broadly about the ways that AI can be used to aid power companies and utilities.

25:54They've got access to, you know, tons of data from, you know, these smart grids that they're putting in and, you know, various sensors that are across the grid like what specifically are our companies doing to try to take advantage of that data and use ai so one example we've seen on the transmission side we haven't really talked about transmission so far so transmission it's you have generation which is where the the electricity is generated generation uh this distribution is how it gets to the customer and the transmission is in between. So it's the big power lines. And there's a physical limit to how much latency can go through a cable.

26:38It's the size of the cable, the temperature. It's also the curvature of the cable, the sag. And so we've done work with customers in estimating the ideal sag of transmission line. There's a big difference between putting a bit more tension on the cable so it sags less and that's where it can conduct more than digging a big hole and putting a pylon next to it in a forest all along. And we've also done work around inspection for fire hazards and for vegetation management. It's a big issue in Canada and in the US as well, where you have pylons crossing a forest and the forest grows. And at one point, the forest might be too close to the actual transmission lines, which can create short circuits and forest fires.

27:38You don't want that, obviously. And so there's a lot of work done with inspections, automated inspections, automated warning of inspections. and there's even work done around using a GenAR layer for the staff at the Power Intuitity company to communicate to robots. So instead of having to learn the language of the robots or drones that will then go to inspect power lines, you can ask them in natural language what they need to the robots. And then on the purely software end, there is a whole sustainability suite which is similar to an ERP so ERP typically is in the finance world you have an accounting software that takes data helps you make certain other data, create reports and Microsoft others as well obviously but Microsoft has a tool that allows a company to really understand at a deep level, as deep as you need to go basically on the ground, it's emissions and what it can do to manage the emissions and then all the reporting requirements as well.

28:56So both from a let's really understand how to manage our emissions and also let's create the reports that satisfy any regulations we might be subjected to. The power industry is very regulated and we've seen customers starting to use Gen.AI capabilities to create some of the key documents they need to operate. So I can give you two examples. So one very concrete example is rate cases. So power-in-sheetly company typically, for the regulated ones, typically earn a rate of return on their assets and when they need to increase that, when they need more money, basically, they need to put together a rate case where they explain what they're going to do with the money.

29:48It's a bit like a teenager and he did more pocket money. And so to put together the cases, they now use Gen.AI for the format to kind of automatically reuse some of the wording from prior years to automatically integrate with other systems and pull in data and give them a really strong draft. And that can be used on the other end as well by the regulator to make sense of a very complex document that fits the format of what is asked for, but can be very complex. And an even more extreme example of this, which at this stage is an idea, but we'll be looking into the nuclear sector because the nuclear sector is even more regulated, rightly so.

30:42And we often think, as normal human beings, as the need for paperwork at the beginning of a project, like you build a house, there's paperwork for building the house, and then you live in the house, and maybe there's paperwork for destroying a house at some point. I don't know, probably. But there's not too much paperwork on an ongoing basis. A nuclear power plant has very much the need for ongoing paperwork. Anytime you bring in new fuel or you produce spent fuel and the spent fuel has to go somewhere to be treated or stored, that needs paperwork as well. And so that's an area where you can imagine GNI being used as well here.

31:25How prepared are utility companies for applying both of these types of models? We don't think of utility companies as being at the cutting edge of technology. Yes, there is a slower rate of innovation in that sector than maybe you would see in other sectors. But it's such a consequential sector that there's a good reason for it. In any case, to answer your question,

31:53I'm a former consultant, so I have this 4x4 matrix in my head. I think it's not really the type of AI that makes it difficult or easier to adapt to. It's how that AI, how those models are accessed. And so the 4x4 I have in my head is really, as you said, you've got the kind of old-fashioned, quote-unquote, AI from like a decade ago or so. the ML and basically the very advanced statistics, right? And then you have Gen AI where you can actually produce content. And so that's two different types of models. Yes. But the way you access the models, so are you actually creating the model yourself or are you deploying the models in your Azure subscription and how are you having to handle all this?

32:52or are you, like you and I, when we use spam detection in our mailbox, are we struggling to use AI? Would our parents struggle to use AI in that context? They don't even know there's AI in that context, right? So the answer is no, it's easy. Gen AI is almost the same. So deploying Gen AI in your subscription, I have customers who did that. In fact, Ontario Power Generation, the power entity company based in Toronto, has a year ago deployed their own ChatGPT version. They used an open AI model on Azure to deploy ChatGPT, a model that a year ago was able to speak French and English, and then they added capabilities in some native languages as well, native to Canada.

33:47And that tool is used to discover documents internally, but also interact with customers in some cases. So that's one thing that requires, especially a year ago, that requires a lot of innovative foresight. But I would say for the Gen.E.I. piece, it is much easier when Gen.E.I. is integrated into your search engine or into Bing or into what you do with Microsoft 365. You open a Word doc and it suggests a few lines for you. Or you go into Teams and you ask for the recap of meeting. that requires less innovative muscle, but not none. Because to get the most value out of it - You have to understand the models.

34:28You have to understand how to prompt them, how to use them, that kind of thing. Yes, that's true. The prompt engineering side is definitely something you have to have in mind. But also to get the most value out of it, you need to understand what the new capability of the system changes for the reality of your business processes. and that's where we work hand in hand I mean I work directly hand in hand with our global system integrators the accenture of this world but also the strategic consulting companies of this world to understand how everything needs to change basically and then once you figure that out what is the rollout process how you make the culture evolve at the customer what change management looks like how the culture will evolve and how the day-to-day job will evolve.

35:21And I don't think it's quotable quite yet, but one of my customers in North America, after testing an off-the-shelf Gen AI, so I'm talking about the SaaS model Gen AI, you don't need to deploy it on your own server, you don't need to create yourself, you just take something that exists. but after a little pilot figured out that each of their employees were saving an average of 8 hours a month of work. So that's a full day every 20 days of work which is significant. What I'm hearing you say in response to this innovation question is that there's a couple of aspects. One is that generative AI models in their usage present a bit less complexity than the more traditional models, perhaps.

36:20And also that these models, you know, in either case, offered as a service eliminates a lot of complexity for the user and they can supplement their skills, you know, in tackling the remaining complexity by partnering with the, you know, the systems integrator and others. So in other words, the, you know, fairly or unfairly were, you know, whereas these utilities are not necessarily considered to be kind of cutting edge innovators, there's enough supporting infrastructure to enable them to, you know, apply and use these technologies and get significant advantage out of them. Yes, right. And to stay on the topic of innovation in the power and utility industry and to look into what YAI can do, but also sustainability as a topic.