DigitalPath's Ethan Higgins On Using AI to Fight Wildfires - Ep. 211

0:10Hello, and welcome to the NVIDIA AI podcast. I'm your host, Noah Kravitz. Wildfires have ravaged California over the past decade, burning millions of acres of land, destroying thousands of homes and businesses, and claiming hundreds of lives. According to Cal Fire, in 2020 alone, the state experienced five of its six largest and seven of its 20 most destructive wildfires ever. And the total dollar damage of wildfires in California between 2019 and 2021 was estimated at over$25 billion. But a new AI-powered tool promises to provide timely alerts to first responders across the Golden State every time a blaze ignites.

0:51The Alert California Initiative, a collaboration between California's wildfire fighting agency, Cal Fire, and the University of California, San Diego, uses advanced AI developed by DigitalPath to spot signs of fire in real time. With me to talk about the very real problem of wildfires and how Alert California aims to help is Ethan Higgins. Ethan is a system architect at DigitalPath and has been central to the Alert California project. Ethan, thanks so much for taking the time to join the NVIDIA AI podcast. Thank you for having me. There's a lot to get into here, so I want to turn it over to you.

1:25Can you tell us about Alert California, about DigitalPath's role, about working with UCSD from whatever the best starting place is to tell the story? Yeah, so the UCSD program has been putting cameras on mountaintops for almost 20 years now, starting with the HP Wren camera network, I believe. And then for the longest time, we've been a service provider for those cameras. A few years ago, UCSD and their partners, PG &E, SoCal Edison, utility companies, CalFire, reached out to us, and they essentially wanted more innovation out of the platform. At that point, we sort of got engaged and then just had been working from there.

2:14So DigitalPath is an ISP primarily? Yes. DigitalPath is an internet service provider. AlertWest is the subsidiary that handles the software side of things for AlertCalifornia. Okay. And so DigitalPath was maintaining the camera network. So I would assume that includes the backbone communications and all that stuff. Yeah, exactly. Got it. Okay. And so how does AI play a role? How did that take shape? Yeah, so the issue we came into after we built out a sort of management platform where we could move the cameras, anybody, user management, that kind of stuff, pretty basic functionality. The issue quickly arose is how are we possibly going to process these images?

2:56We ingest anywhere from 8 to 16 million images a day, and there's no way for a person to go through all of those on their own. I only can imagine what that's like because we've had some deep space astronomers on the show before talking about AI to process those images. So I can at least not just drop my jaw and stop the conversation right here, but that's an immense amount of data. So the network patrols how much space? I don't know if square miles or miles, how you measure it? We measure it as percent of high-risk fire areas, and we're right around the 90 % mark. Got it. 90 % high risk fire in California.

3:35Right. So huge amount of data and images coming in every day. Yeah. And we have image data, right? So the obvious choice is computer vision to solve that problem. One of the things we realized early on, though, is that it's not necessarily a problem about just detecting a fire in a picture. It's a process of making a manageable amount of data to handle. And it's subtle difference there, but it really changes your outlook on things in that our, Our goal, at some point, a human is going to be involved. It's not fully automated and it's never going to be for the foreseeable future. Humans in the loop, definitely.

4:11So really, the question was, how can we make it so that a human can review all of these images in an efficient manner? So computer vision was the first step there. And that's a big part of it, being able to actually look at the image and see if there's smoke in there. That's a huge component. But really, that's not enough because there's still a lot of images of smoke, even when there's not. I think we estimate around 100 ,000 events a day of smoke even filtered down. And most of those are true positives. Right. And how many wildfire, I don't know how you call it, but how many wildfire positive identifications?

4:48Almost all of those. Per those 100 ,000. Okay. Yeah, in the minimum of tens of thousands closer to that, 90, we're pretty high up there in the percentage-wise. So even that is still an immense amount of data. And we talked around to our partners a lot. There's been a lot of attempts to solve this problem before in the past. And the thing that we kept hearing over and over and over again was there's too much data every single time. Is the problem that you mentioned that it's not as simple as looking for fire in an image, what is it that you're looking for? And what's the problem? I assume that out of these hundreds of thousands of smoke identifications a day that are linked to fires, like there's something further that you're looking for to act on.

5:38Not exactly. So we don't take any determinations on whether a fire matters or not. Okay. We just say this is a fire. Got it. But the thing is that we talked to our partners and they were saying that they were getting 7 ,000, 10 ,000 alerts per day on fires. They were true positives. Yeah. But that works out to around one every five seconds. Right. And they're getting those on email alerts. And how many of those do they need to act on or urgently act on? Well, all of them, theoretically, they need to act on. Okay. At least to identify if it's real or not, something they need to be concerned about.

6:15But how can a human possibly interact with 7 ,000 one every five seconds events? So that was the problem we needed to solve. That was new. And really what it came down to was we needed to send out a single alert per fire. And that's it. No more, no less. Got it. And that works out to around, well, 1 % reduction. So we're going from our 10 million-ish average down to 100 ,000 events. and then we need to reduce that by 99 % again to make that a usable number. So first reduction was associating if you've got 50 ,000 images of smoke, but you realize that they're all from the same fire. Yes, exactly.

7:00And you can reduce that down to one. Okay, got it. But now you have to go another 99 % reduction. Yeah, so to make it useful. So we used actually almost like an adversarial network. like a GAN would use. We use that in real time, though. So we use various algorithms to discern if something matters or not or if it's something we've already seen or a variety of things. So we classify every single item that comes in as if it's something that needs to be reviewed, something that needs to be acted upon, and do all that. And that's actually been the hardest part. That's been much harder than computer vision.

7:36I'm not sure how familiar the listeners are with the modern... break it down as far as you want. Yeah. Because every case is always a little bit different, right? Even if you're using similar technologies. So, yeah. The fact is that computer vision in 2022, 2023 is really, really easy, actually. Almost deceptively so. Because all the state-of-the-art models are open source. You can go download them for free and train your model today. The data side is where most of the issue is at. And collecting a data set that actually matters. So we, we play, you know, we were like, oh, let's just train a model.

8:15It's super easy. Download it from the internet, run it from there. We learned very quickly that data was definitely, we had that same issue where we came across the data and that was a huge thing, but it only took us a little bit to solve. You know, it's, it's a pretty mundane process once you get it figured out. What was the, was there a breakthrough moment or sort of something that unlocked, made it easy to solve? Or was it just a matter of just looking at the problem and... Just a brute force, getting a bunch of images of smoke. Got it. And, you know, there's some efficiencies that can be gained throughout that process.

8:52But generally, looking at smoke. And we actually did, to make it a little easier on ourselves, we had access to the move logs for all the cameras because it's our automation platform. So we can look and see where CAL FIRE personnel are moving the cameras to look at stuff. And we grabbed those images. we can prioritize rather than having, you know, millions of images to review. We just look at the images after we move it. So how often do the cameras move? It's kind of a loaded question. I'm really not trying to make this hard on you, I promise. No, no. So the cameras... I'm just wondering, because you mentioned about, you know, being able to use that data point as well, right?

9:31The moving locations. And so just, I'm wondering about the complexity of everything you're trying to make sense of. Yes. So the cameras are always moving. They're never standing still. If you go to our website, it looks like the cameras are standing still. They're not. They're moving in the background. Are they on vehicles or drones? No, they're fixed cameras on top of mountaintops. But in terms of pan, tilt, zoom, so just looking around, they're always moving. They never stop moving. They're always doing something in the background. Now, when a user moves a camera, they can point it at a certain direction, and then the camera will always revisit that spot on a regular interval.

10:12So it appears on the website like the camera's not moving. But in the background, it's always moving. It's always scanning for stuff. Got it. So when, say, Cal Fire, I'm talking about Cal Fire moving the cameras, that would occur if they move it when they see a fire, generally. Right, okay. And they'll move it once or twice, and then they'll stop moving it for a few hours at least. Right, right. So that data we can pull really easily. And we have, you know, that gives us a ton of data on smoke right now. Now, the problem with that is that it's all data after the fire has already been detected.

10:49So that brings another problem in because we want data from before the fire is detected. Is that ultimately what you're looking for those conditions that just precede the breakout of a big fire? Exactly. How does it work? Yeah, okay. Yeah, exactly. So we want to target data for what we want to detect, right? So we want to detect a fire before anyone else knows about it. So we try to tailor our data set towards that, basically, the very beginning, first starts. Like if you run our model against a lot of the fires, like a campaign fire that'll be burning for weeks, a lot of times it won't have strong detections past that first hour or two of smoke burning, which is kind of strange, right?

11:34Because you would think it's fire. It's the same thing. It's just a smoke plume. But there are behavior characteristics that are different between the two different time periods, how long it's been burning. And part of that, too, is due to just how we process it. and that the model can't get full context. Like if the smoke is really big, takes up the whole frame, the model doesn't have the full context to identify that. But yeah. So the emails you were talking about before when you're getting all the positives and it was sending out the emails and the goal of reducing them to fewer, more mission-critical, let's say, emails.

12:12Who are those going to? Those are going to your partners at Alert? No, those actually go directly into operation centers around the state. Okay. So we provide the ability to self-sub to whatever alerts they want based on time levels, area, whatever. So we'll have - Is it wildfire or fighting specific? Yeah, so CAL FIRE, like 911 centers, get phone calls actually. PG &E has integration. SoCal Edison has integration. And they all have the option of email, SMS, or voice messages, and those are all automatically sent out. And we've seen a pretty high take rate, actually, on voice messages into 911 centers.

13:00So when a fire gets detected, our system automatically calls their 911 center and lets them know. I'm speaking today with Ethan Higgins. Ethan is a system architect at DigitalPath, the Northern California ISP, that has been working on something called the Alert California Project in conjunction with Cal Fire and the University of California, San Diego. The Alert California Initiative is using camera networks, AI, and other advanced technologies developed by DigitalPath to spot signs of fire in real time and send actionable alerts, as we've been talking about. Ethan, we've been talking about everything that computer vision can do and how much more accessible the tools have become over the past, you know, five to 10 years and how it's enabled folks to come up with and build initiatives like this quite quickly.

13:51But nothing's perfect. Are there any downsides to using computer vision to detect wildfires? Yeah, the biggest one that we see is that we're going to be extinguishing more fires. Now, that's a positive in the short term, but a negative in the long term, right? because extinguishing more fires means that there's going to be more buildup of fuels and the potential for larger wildfires in the long term. And our partners at UCSD have actually been tackling that problem right now because we all are in the same boat where we think that could be a big concern. In collecting LiDAR data, super high-resolution LiDAR data across the whole state, I think up to 300 points per meter, if I'm not mistaken.

14:32And that is data right now necessarily won't be processed, but it'll give us a very strong baseline moving forward of how environments change and how we need to better address different fire environments as we move forward. So LIDAR, I know the term LIDAR from cars and vehicles using LIDAR to sense autonomous, semi-autonomous, driving assist kind of features, that kind of thing. How does LiDAR compare to computer vision for this kind of work? Do you use them both in parallel? Are they for different tasks? What's the technology comparison there? Yeah, so they'll be for different tasks primarily.

15:15The big thing we see is stopping wildfires right now is computer vision, but identifying risk is going to be based on LiDAR moving forward. Okay. So one of the big things, you know, kind of the way to think about it, or the way that I think about it is that fire suppression is kind of like a dam and it's holding back all this fuel from burning and the fuel's the water. But at some point, that water's going to get over your dam, regardless, unless you let it out continuously. Right. So the big concern is if we hit, you know, say we're at a million, we can hold a million gallons. I have no idea if that's a big number.

15:53But our dam can hold a million gallons right now and adding the computer vision detection and make it so we can hold 2 million gallons. Well, we're still going to get to a point if we don't let that water out where that water is going to spill over and it's all going to come rushing out. So that LiDAR data is really critical to identify where we need to let out those fuels in the form of prescribed burns. Right. What, you kind of touched on this just now, but what does the future hold for, I'm laughing only because I'm sitting in my home office talking to you and looking out the window at lots of oak trees and wildlife here in Oakland and thinking, what does the future hold for wildfires?

16:35But here's an easier question, perhaps. What does the future hold for wildfire detection technology, wildfire suppression efforts and technological advances that pertain to that? And this particular project, which with DigitalPath, California, UCSD, what's in the near-term future for you? Yeah, I think we're at a point now where our computer vision models are good enough. We're detecting around 40 % of fires before 911 right now. And we see some small potential to grow that, but not a ton right now. It's going to come down to, I touched on this before, using adversarial networks to identify what is important.

17:16And I think you'll see that more and more across AI as a whole in that AI is not perfect, but when you couple multiple models together, it can get really close. Yeah. And you're seeing that with just GAMs, generative adversarial networks, where they work together to solve a problem or actually, I guess, fight against each other to solve a problem. Right, right. And I think you'll see more and more of that happen in production where we have models analyzing the output of other models in order to get a more cohesive view of what's going on. For us, that's where our main focus is right now, is on those adversarial networks and in generative AI, the applications there.

17:58One of the big problems that we see a constant theme recurring in the Firespace is there's a lot of really awesome simulation tools out there, but they all take a long time to process. And I think we're seeing right now, industry-wide, that's an issue, where there's these really cool simulations coming online, and they're awesome, but they take three weeks to run. Right. And I think generative AI has the potential to really accelerate that beyond anything else that's possible. Yeah. Hearing that a lot these days. When it comes to you mentioned before sort of the benchmark of benchmark is my word, not yours, but of detections before 911 calls.

18:42Right. That the Alert California Initiative, I believe you're talking about Alert California, is starting to or has been detecting smoke and fires before 911 calls. When it comes to something that could help prevent, you know, I don't know, the next big fire from becoming enormous and hugely tragic or the next small fire from becoming a big fire. Is there a milestone that you see in the distance for, you know, AI in some form or the work that you're doing in some form? Like there's some, you know, milestone out there that you're trying to achieve. Yeah, I think identifying problem areas are the most important thing right now.

19:22Okay. Meaning the geographical situation or the terrain? Yeah. Yeah. Identifying where these fires can occur. Yeah. And then detecting them before they occur or as they are occurring so we can stop them and manage that fuel load on our own time. Right. Yeah, I think this is what's the saying. All models are wrong. Some are useful. I think it's something like that. Right. I think I get the gist. Yeah. Yeah. It's something along those lines, but it is true. And I think pushing more towards more useful models to identify that risk. There's a lot of data out there that has yet to be harnessed. Sure.

20:03And I think, you know, collecting that data through UCSD is going to push that forward. But identifying those issues in advance are the real problem. And it doesn't necessarily have to be even long-term advance. You know, if we can have an hour warning that this area is going to be a major concern, we can at least stage assets there, get people on the ground and ready to fight a potential fire. Right. Yeah. For listeners who want to find out more about anything we've been talking about, including DigitalPath, where online can they go? Websites, social media feeds, where would you direct them to learn more?

20:44Yeah, so all of our data is available publicly at alertcalifornia.org. Okay. And you can learn more about DigitalPath at, I believe, alertwest.org. Ethan, I thank you for your time today, obviously, but thank you for the work that you and all of your colleagues involved with us are doing. You know, as I mentioned, as someone who lives in the state, but I mean, really, you're seeing it more and more, not just in California. And the world's changing. Wildfires are a part of it. And, you know, we're grateful for folks like you who are putting the advanced technology to work, trying to curb the problem, make the world safer.

21:18Yeah, it's my pleasure. It's all a passion project, too. The team is all, it's all a passion project here. So we love it. Fantastic. Well, all the best to you and your colleagues. And again, thanks for coming on the podcast to tell us about it. Yeah, thank you.

21:52Thank you.

22:20Thank you.