Dr. Olga Russakovsky: Shaping the Next Generation of AI Leaders

0:04Hey, everyone, and welcome to Generative Now. I am Michael Magnano. I am a partner at Lightspeed And this week on the podcast, we're talking to Dr. Olga Rysakovsky, a computer science professor at Princeton University. She is a leading researcher in the field of computer vision, having worked on many important breakthroughs in the field, including ImageNet. She's also the co-founder and board chair of the nonprofit AI for All. This was an honest and surprising conversation about who is shaping the future of AI, what today's computer science students are learning, and why the next innovation in AI may not come from where you expect it.

0:38So check out my conversation with Dr. Olga Rosakovsky. Hey, Olga, so good to see you. Hey, Mike, thanks so much for having me. Yes, thank you for doing this. I've been looking forward to this for a while. I have a bunch of questions for you. I want to learn from you in this hour that we have. Like so many people have probably in your career. I guess before we dive in, though, I think we should tell people a little bit about your career. And so, yeah, do you mind just sort of telling people about your work and your research and everything you've been working on these past couple of decades in AI?

1:17Yeah, absolutely. So I'm an associate professor in computer science at Princeton, also associate director of the Princeton AI lab. Let's see, I started my career at I would describe starting my career at math camp in high school, was a math major in college at Stanford. And then went to do a PhD in theoretical machine learning, sort of got very interested in machine learning and in AI and wanted to sort of do the more theoretical work. But over time, kind of drifted more and more into applied AI research or drifted into computer vision, which I think is just fascinating. I know there's a lot of hype and excitement about natural language processing these days and about text processing, but I still think computer vision is by far the most interesting one.

2:03Sorry, no offense to anybody, but that is very much my passion. And so I've been doing a lot of work both on building computer vision systems, but also kind of studying them, analyzing them, thinking about explainability of these systems. Recently, I've been thinking a lot about AI fairness and bias, particularly, you know, in computer vision systems and in AI systems more generally. So that's kind of, yeah, that's kind of my trajectory. Do you mind just explaining computer vision a bit? I mean, I have I have an understanding of it, but just for the audience sake, like dive into computer vision.

2:38When you say that, what does that what does that mean? Should we be thinking, you know, self-driving cars or more just sort of identification of objects within within images? Like tell us more about computer vision at a high level. Yeah, I think it's anything that's related to understanding pixels, understanding images or videos. I think you can think of autonomous cars for sure. You can think about when you upload a photo to any social media platform and sort of says, OK, here are the faces. Here is maybe who these people might be. It automatically tags your friends. You know, you upload, you create a photo album and it sort of sorts your photos by not just by date, But by sort of who is who is in the photos, what what is the scene like?

3:21So but also, I mean, you can think about a lot of applications like medical applications. You can think about sort of computer vision that's used to do to diagnose skin cancer. You take a photo of a of a lesion on your skin and it sort of is able to tell you, OK, should you see the doctor? Is this worrisome? So you can also think about applications to things like agriculture. So both in terms of building better systems that are able, you know, better robotic systems that are able to sort of operate on the ground and do some of the agriculture tasks. But you can also think about like aerial photography and trying to understand conservation efforts, trying to understand sort of how the Earth landscape is changing over time.

4:05And that's also computer vision. You're sort of analyzing these photos that are taken from space. So there's all kinds. You can also think about, you know, space exploration. You send a robot to Mars, right? It's trying to kind of navigate and move around in its environment. You're not going to have somebody steering it with a joystick, right? That's not going to operate just because of the latency, right? And so it needs to understand to be able to have a camera that looks around, understands what it's seeing, kind of help guide it to, you know, drive around safely. So, yeah, all kinds of things.

4:37Tons of applications of computer vision right now, probably more than ever before. in the field. And the other thing I think of when you start talking about computer vision is something that I don't think is the same, but I wonder if there's intersection, which is kind of the other side of images right now with an AI, which is more around the generative AI of sort of diffusion-based image models. Where does that sort of intersect with computer vision? Or are they sort of like two completely unrelated fields? Yeah. So it's very interesting because I I think initially we would think of that as a sort of computer graphics, as kind of generating, you know, kind of generating visual data.

5:17But it's very much become part of computer vision. So that was actually a very interesting transition. So I think when generative models, sort of GANs, you know, generative adversarial networks, diffusion models started coming out. So that's now very much part of computer vision. And that's been sort of interesting that it's kind of become both the images to some kind of understanding of these images, but also going back. And so, yeah, so we do a lot of work on diffusion models in my lab as well. And that's now we've kind of embraced that as part of the computer vision research. It's probably, I'm guessing, driven a lot of interest, like new interest in the field.

5:52And I wonder what that's meant for your work and your research and maybe the computer science department at Princeton. Yeah, so there's definitely, I mean, for the computer science department, we're the largest major on campus. And I think that that's that sort of says it all. A lot of undergraduate students were very interested and very interested in taking courses, very interested in doing research in this space. Then, you know, lots of applications from prospective PhD students, you know, graduate students. And I think we're trying to, you know, hire more and more faculty in this space to sort of keep up with the interest.

6:26I think the new Princeton AI lab is a good example of sort of trying to connect all of the folks doing AI research on campus and sort of grow that. I think it's when I think about AI research, I actually think of it as being broader than just computer science. I think there's a lot of I mean, like you say, it's sort of the advancement of this technology has driven a lot of interest, but it also raises a lot of more social questions. It raises a lot of questions about the impact of this. It raises a lot of questions about how people are actually interacting with these systems. It raises questions about impacts on different fields.

7:03And so I think when we think about computer science research or AI research, you know, in the past, AI research has been much more of kind of engineering or mathematical endeavor. And I think now it's becoming much more interdisciplinary. So if you need to really connect to folks who have studied things like societal shifts in other contexts. and now can help us kind of grapple with how should we think about some of these changes in AI? How should we be thinking about analyzing and understanding these models, which we no longer can analyze or understand using kind of standard methods? And so there's just a lot of opportunity to grow this field and grow the way we're thinking about it as well.

7:45Yeah, that makes a lot of sense. I mean, I feel like, you know, more and more you're seeing people within AI or tech more broadly think about, like you said, the societal impacts or, you know, get philosophical and compare and compare back to previous times or technological innovations in history. So, yeah, I've actually been thinking quite about that. Is the science, the computer science aspect of this, like over time, does that become like kind of less relevant and sort of the field of AI becomes, as you say, like a little more generalizable and sort of applicable to almost like more different like humanities related topics like it's super interesting like i maybe just to take that a bit further before i before i hand it to you but like i was a computer science major and um you know we didn't we didn't talk at all about philosophy or societal impact we just learned how to program in a bunch of different languages right and like learned about like this you know the the hardware and the with a lot of these generative models, it seems like we could get into a point where that stuff, the type of stuff that I learned with my computer science degree, actually becomes less and less relevant over time.

8:54And the stuff I think that you are talking about becomes more and more relevant. So, yeah, I mean, how do you think about the future impact on computer science as a result of AI? Yeah, so it's a very good question. I mean, so first of all, I think to your point about, you know, learning about the different, you know, coding language, you know, different programming languages and so forth. I mean, I think that is becoming somewhat less relevant by virtue of the large language model. I mean, we've seen a shift in how people code, right? We've seen a shift in sort of some of the models actually kind of assisting you in coding.

9:32And I have, you know, it raises a lot of interesting questions about how do you design assignments in computer science courses? And how do you think about, you know, the fact that historically a lot of the assignments are around, you know, write this code to solve this particular problem. But now, you know, you can ask Chad GPT and it'll write that code to solve that exact problem because it's been in computer science assignments all over the world. And now the model has learned from all of that data and it can easily, you know, solve all of these kind of classic problems that we teach students to solve.

10:02So I think it raises questions about what is the role of programming and programmers. And I think that's sort of one can of worms. I think to your point about that sort of computer science will become less relevant. So I actually don't think so, because I think building these systems and sort of the technical work around building, deploying, sort of developing levers of control and things like that, I think that's going to stay core technical, but I think a lot of it is going to be driven not just by kind of intellectual curiosity of can I do this, but driven by sort of concrete needs and applications and concrete questions that are being asked.

10:48And those applications are going to be very broad, but also the kinds of questions that we're going to ask about sort of what does it mean to understand the system or trust the systems or what kind of safeguards do we need around these systems or how are these systems actually influencing society and what do we need to sort of steer it more to influence it for the better rather than for the worse. I think a lot of those questions are going to be asked sort of by scholars who are much more interdisciplinary. And so I sort of I don't think the computer science role is necessarily shrinking. I think computer science role is kind of staying the same, but maybe the field is kind of growing to bring in more and more ideas and questions to kind of help steer the computer scientists.

11:37Yeah. How much is that actually happening right now? Like, you know, is curriculum changing even now to head more in this direction? Or is this something you see like further down the line? Yes, I think this is happening right now. I think the curriculum is changing in many ways. And I think the connections to sort of interdisciplinary work is very much, very much happening. So later today in a few hours, right, so I'm so my co-instructor and I are teaching the last lecture of the undergrad computer vision course. at Princeton for the semester. And what we're doing is we're hosting a fireside chat with Dr.

12:14Molly Crockett, who is a psychologist from the Department of Psychology. And they're going to come in and talk about one of their latest papers on sort of AI as used for scientific discovery and sort of what are some of the pitfalls of using AI in sort of scientific research or how it sort of creates an illusion of understanding, but really like, what does this mean for scientific research? And so that's sort of the conversation we're going to have in last computer vision class for the semester, because we think it's sort of important to ask some of these more fundamental questions in addition to, you know, a semester of teaching about convolutional neural networks and, you know, deep learning and different methods and different techniques for building the systems.

13:05Can you give us a little preview of what the pitfalls are of using AI to make scientific discoveries? That sounds super fascinating. I'm not going to give it justice, but there's a lot of sort of opportunities to speed up the research process using AI. My interpretation of this is that it can drive research towards sort of what is currently the most sort of promising direction. I think sort of the scientists will start using kind of similar tools to summarize the data in similar ways and will drive the field towards, you know, you're trying to sort of get a paper published. You're going to, you know, select sort of the current thing that sort of is best supported by current evidence or will kind of is the easiest to put together the data for.

13:58And you're going to analyze the data in particular ways. And it's ultimately it's kind of going to limit creativity. It's going to limit creativity and sort of some of the joys and potential of scientific discoveries that you are thinking about these things in different ways. ways you're going down the wrong path, you're failing and then you're succeeding. And some of that breadth of approaches and thinking about things, that's what's sort of leading to scientific discovery and sort of relying too much on AI models to steer that is going to sort of lead to, I think, short-term success, but not long-term success.

14:39Yeah, that makes sense. It's almost, you know it's gonna it's almost like gonna converge to the path of least resistance almost in a way whereas the more the the research is happening directly via humans like you said to use the word like creativity there's randomness random ideas that that get explored um that makes a lot of sense maybe good tie into ai for all uh this is a big part of of your work and your research. Well, I'll let you explain it. Tell us about AI for All. Yeah. So, so AI for All is, you know, what I do in my copious free time. So I'm a co-founder and now chair of the board of this nonprofit.

15:18We are working to increase diversity and inclusion in AI. So I think I personally see the big, you know, I know there's a lot of conversations about what is the biggest threat to AI? What is the sort of existential threat? There's a lot of this talk. And to me, the biggest threat of existential threat is the lack of diversity of thought in this field. You can, you know, it's harder to measure diversity of thought. It's easier to measure diversity of demographics as sort of a proxy for diversity of thought. And you can look at, you know, within AI, there's different statistics, but it's around 15 % women.

15:57It is very few black and brown folks. It's very few folks who are black, very few folks who identify as Latina or Latina. It's very few indigenous folks. And what that's doing is it's driving us towards echo chambers and decreasing the diversity of thought in the field, decreasing the creativity with which we're able to approach some of these problems. And ultimately, I think going to drive this field into the ground. I think we're not going to reach the full potential of AI if we keep going the way we're going. And, you know, just just to sort of, you know, I know the question, some of the questions about like demographic diversity get into very complicated sort of legal and ethical questions these days.

16:48And I totally get that. But one way one way that I like to think about it, which I think is a lot less controversial, is that the reality is folks who are working in AI these days come from particular schools of thought, particular types of training. particular sort of top schools that have produced these students. A lot of them read similar books as kids. A lot of them play with similar toys as kids. A lot of them run in sort of similar social circles. So they talk to folks who are like them. And then all of this kind of translates into how we think about building the technology. A lot of this translates into the kind of applications they care about.

17:30This translates into their value systems. This translates into kind of how they approach problem solving. And at the end of the day, this limits the creativity of the field as a whole. Yeah. You know, oftentimes I feel like when this topic is discussed, like, you know, what I gather and what I learn is that, but I would love for you to tell me if this is right or not, is that a lot of the bias in these models comes from the data that it's trained on. Sounds like what you're saying is it also comes from like the way that these systems like are actually designed. Can you can you like can you add a little more specificity on on like how they might be designed in such a way that it leads to to to a bias in one direction or the other?

18:17Absolutely. So I think the bias comes sort of at every stage of the pipeline. So you mentioned, you know, I mean, data is it's actually interesting because that was a controversial point a few years ago that people would sort of disagree that it comes from data. But but that's actually become sort of well accepted that. Yes. So so a lot of the bias will will there's bias in the data. There's bias in both sort of the way that the data is collected. So, for example, you know, I'll just give I mean, I know you want to talk more broadly, but let's just give one example on the data bias. So if you look at the common computer vision data sets, you can run analysis on geographic distribution of that data.

18:57So you can, you know, using a lot of this data comes with GPS tags, right? You just grab the GPS tags and you can plot sort of which countries does this come from. This comes primarily from the U.S., countries in Europe. And that's more or less it. So you can you can there's there's various papers, like including some of the work from our lab, but lots of other folks as well, where you have this map of the world and you have the U.S. highlighted, you know, bright colors and parts of Europe highlighted bright colors. And then sort of the rest of the world, in particular, sort of South America and Africa are just completely missing from from this data.

19:32And so then you get things like object recognition systems that are purported to sort of, oh, it can recognize all objects in the world. And then in practice, if you feed it a picture of a house in Africa or you feed it a picture of a plate from Africa or you feed it, you know, a picture of one of the classic examples is like bar soap. And it will refuse to recognize bar soap as soap, but it will recognize the US brands of liquid soap as soap. And so, you know, so that's sort of the data answer. But I think thinking more broadly, right, what are the big applications that people are working on now?

20:08Like thinking about sort of, I mean, we're working a lot on autonomous driving and I think there's a lot of power in autonomous driving. But if you think about, well, where is this coming from? We have a lot of folks in the Bay Area who are sitting in traffic for, you know, two hours a day, you know, each way. Right. And so, of course, we're going to work a lot on autonomous driving because this is sort of front and center on people's minds. And I want to be clear, there's lots of, you know, excess, like very important sort of accessibility needs and environmental impacts that and, you know, the 100 people that get killed, economic impact.

20:44And also, you know, I think in the US, 100 people a day die in car accidents, right, which can be remedied. But if you think about the relative lack of work on solving hunger crisis and solving sort of economic, and I think a lot of that sort of comes from who is the help, like who is driving some of this, some of this work. And if you look at, you know, if you look at startups, if you look at research projects and sort of different students undertake that there's you will kind of immediately see some of the connections where what people are passionate about, what they're choosing to work on is very much influenced by their values, by their culture, by their upbringing.

21:30Right. That's that's what they feel passionate about. That's what they're going to work on. And so if we want all of these applications to be actually sort of solved and tackled and get the attention that they deserve, then we need people who are going to be passionate about that kind of work, about sort of each of those. how does it happen though when we when as a society at least in the u.s we have the incentives that we do right where i mean built around you know the capitalistic structure that we all you know we all exist in and the motivation of you know of private private companies i mean yeah i definitely hear you on the autonomous driving thing like from a humanity standpoint there probably are like more important things but um you know uh a company or a corporation or you know an organization trying to to to maximize economic impact like you can see why it leads to things like um full self-driving or identification of the the you know the to use the example you used earlier of like to identification of the people in the photo for the social media site you know what i I mean, so like how does how yeah, how do you shift it away when the incentives like very clearly point in the other direction?

22:46So I don't think we know what will happen with more diversity of thought in the field. I very much hear you. Right. And I'm not, you know, I'm not sort of pollyish about this. Like I grew up in the Bay Area. I understand sort of economic. I get it. I don't think we know sort of the kinds of things that can be done that can be potentially sort of aligned with with the economic incentives as well. Right. I suspect there is, you know, like. I don't know, the food industry is is changing, I think, in many ways. And I think that's, you know, driven by various economic incentives. And I think, I don't know, kind of the random things that come to mind are sort of like development of of the like fake meats.

23:33Right. The like different kinds of like impossible. Right. And and that's something that, you know, would we have thought before this started happening, that this is something that would have, you know, enough economic incentive to to drive that. But yes, yet it seems to be happening. I mean, I think something I mean, there's a lot of money in the medical system. Right. So there's like economic incentives to drive some of the medical innovations. I mean, I think we're also kind of limited because we're seeing what's what's happening now. I don't think we have the like the full creativity that we need to reimagine, like what are the different kinds of applications that could be tackling and could be aligned with the with the economic incentives, but also aligned more with the full range of things that we could do with AI and that could have both sort of economic and social and sort of social good incentives.

24:21Yeah. So how do you how do you do that? How do you how do you get people building in this way? So what we're trying to do at AI for a little sort of train more students and provide pathways into the AI space and sort of concretely thinking about, you know, I mentioned in the beginning that sort of, you know, the students that are going into this space come from a small set of universities that have very good AI training programs. And, you know, I'm proud to be at one of those universities. Princeton students get a lot of training in AI, but there's lots of universities around the U.S. that don't, that don't have the AI faculty, that don't have the strong AI curriculum or that, you know, don't have the capacity to teach students, you know, how to pursue some of the, you know, their own passions in AI.

25:09Maybe they have sort of the basic courses, but not the opportunity to really do, you know, independent work or projects. And so at AFRL, we run this program called AFRL Ignite for college students, particularly targeting Black, Latinx and Indigenous women and non-binary students from around the U.S. that come into our program. It's a year long program. It combines both sort of education on AI, in particular on responsible AI. So sort of thinking about both teaching them some of the core machine learning skills, but also getting them to think about data, about evaluation of these models, about sort of all of the various decisions and values that get embedded into the systems.

25:50And then they get to work on a portfolio project guided by industry mentors. And there's, let's say there's lots of folks in industry who are very eager and willing to contribute their expertise. And maybe they don't themselves come from some of these backgrounds, but they're very passionate about volunteering with some of these students and sort of bringing in more younger, more diverse, passionate voices into this space. And so they guide the students through a, I just want to say a passion project of theirs. I mean, it's a portfolio project. They're working on something concrete, but it's driven by the student's interest of sort of what do they want to explore with this technology?

26:30And then we provide some, you know, career readiness and career prep workshops and sort of helping them put together their resume, helping them think about how they're going to apply for sort of their first paid AI internship and kind of get their foot in the door. And then from there, you know, hopefully the world is their oyster. Hopefully they've, you know, at least sort of gotten to experience some of the, you know, joys and hardships of building in this space and they can they can take it in in the direction that they want from there. So it sounds like it's about bringing in different and more diverse types of people into the field rather than, say, like deliberately changing the way the technology is built to maybe like correct for something.

27:17I'm just thinking of like this example that I'm sure you saw. I don't know. Maybe it was like six months ago with Google, Google Gemini, where, you know, Google was was was worried about like the bias of the model. And so they I don't know exactly how they did it, but they sort of overcorrected it in the other direction to the point where it was basically factually incorrect about a bunch of different things. You're not advocating for that. What you're advocating is for bringing in different perspectives, different types of people that typically are not involved in the field to solve for this.

27:50Is that right? Yes. Got it. Yes. And we've done, I mean, like a lot of the research in my lab has been about, you know, tackling bias and sort of how do you define bias? How do you think about bias in these systems? How do you measure it? How do you develop algorithmic methods to correct for it? And all of that is really hard. It's very hard. And then the root problems and the root cause is that we are not being creative enough or thoughtful enough or diverse enough in how we approach this, right? So some of the, you know, lack of geographic diversity in the data, I mean, why aren't we? I mean, we're sort of using the simplest data collection approach.