We Need Both Symbolic and Statistical AI

0:00Hey, everyone. Today's Real Vision Daily Briefing is sponsored by Engrave, maker of the coldest hardware wallet, Zero, and stainless steel, backup, graphene. Engrave brings you the highest security in a touchscreen experience to safely manage all your crypto offline. Enjoy a 10 % Real Vision discount in engrave.io shop with the code Real Vision. Now to the top analysis of today's markets.

0:35Hi there, everyone. A belated Happy New Year from me. It's my first time back on the platform in 2024. I hope you are all extremely well. I'm joining you as usual from London, a very chilly London. And we are in for a huge, huge treat today because today we have a special guest. His name is Pradi Gravikumar. He is a professor of machine learning at Carnegie Mellon University. And he's here, of course, to talk to us about AI. We've all watched this AI moment that we've been living through for the last two, three years in amazement with wide eyed wonder, essentially. I want to dive deep into that with Pradeep in this session.

1:21We'll look at where things are heading in 2024, how things are going to develop, the moves we think some of the big players, including OpenAI, will make. but we'll go deeper than that. I want to look with Pradeep at the issues that form the center of his work. I think it's fair to say that he believes that we're at a crucial threshold moment for machine intelligence and I want to dive into that and what it really means, what it means for where all this is heading and the impact it can have on the real economy and our lives. So without further ado, Pradeep, thank you so much for joining us. How are you?

2:07Yeah, I'm doing great. Also chilly in Pittsburgh, but still doing great. Thanks for having me. Thank you so much for joining us. And I hope we can both stay warm during this conversation, warm enough. So look, let's start by just learning a tiny little bit about your background and your research work right now, because that will set us up for everything that will follow. Yeah. So my research actually dialing back 20 years was in statistical machine learning. So this is actually all of AI these days involves statistical machine learning. And this is something that really picked up steam in the 90s.

2:54And during my PhD in the 2000s, That's where we laid a lot of the foundations for the statistical machine learning in terms of figuring out statistical associations between a lot of data and making predictions and so on. So all those foundations are right now being used in all the state-of-the-art AI models. And so fast forward to the present. So now I'm really looking forward to the next evolution of AI beyond statistical machine learning. And I'm sure we will talk a lot more about that today. Yes, indeed we will. And that taps into this big thematic distinction that we'll get into between statistical and symbolic approaches to machine intelligence, because I think it's fascinating and it's so crucial for everyone out there to understand if they want to think about where this is heading across the next few years.

3:58But let's start with where we're at right now. I mean, it goes without saying, for most of us out there, the AI moment we're living in now, chat GPT, mid-journey, generative AI, all of that, it came as a huge surprise. Where do you think this is heading across 2024? Because so far we've had huge excitement and a degree of wonder with the capabilities and the outputs of these models. It feels like, you know, we've done that now. And the questions are more around the questions I'm asking myself are more around. how does this have an impact on the real economy? Where does it head across 2024 in terms of jobs, economic impact, and so on?

4:50How are you looking at that? And how are you looking at the role of the big players, you know, OpenAI and DeepMind in all of that? Can ChatGPT have an impact on the real economy? Or is it just not ready to? Right. I think yes and no. If you think about chat GPT and large language models, they are very good, great even, on figuring out statistical associations between word sequences and understanding essentially just collection of words. But with the more modern multimodal foundation models, they can also understand images, even audio, and all of these things together. And so basically, an upshot of this is that if we have data that consists of images and text and audio, then we're going to have these AI models that are going to understand what we want from them.

5:52And I think this is going to be a big unlock, because if you think in terms of how, let's just say, we participate in the economy, Let's say there is a manager and he wants, he or she wants their direct reports to do something. They typically convey this using audio, using text, potentially images, right? Maybe we communicate using memes. But the interesting thing is that if you can have AI models that can understand this, then that's a potentially huge unlock. And so I think what we might see over the next year are what I would say are things both above and below these large foundation models where you can have things above where you could basically have these large language models suggest certain tools to use.

6:54and then you could have other wrappers that can use those tools and then you can also have things below where it can clean up how you interact with these foundation models and so I imagine that there would be a lot of companies in that space that can help us do that but at the same time I would say that it's not all of the things that we want to do take the form of analyzing text and images and audio, right? So I think that's going to be a realization that we are going to also see. So there's going to be these positives that I mentioned, but there also could be potential negatives or I don't want to say disillusionment, but because I think there are going to be other solutions along the lines of what I'm going to talk about later on.

7:45But people are going to realize that just the large language models are not going to work if you have data that is not just images and text. Right. So, yeah, I mean, essentially what ChatGPT and other similar tools fueled by large language models can do is understand your text inputs, your language inputs, and spew out some new text. And there are instances in the workplace where that is useful, right? Right. Do you think that that can eat jobs yet?

8:22Technology has always, so one perspective of technology is that it takes energy and transforms it into something instrumental. And by and large, if you think about, I guess, the industrial revolution and 100 years back, a lot of it was in the physical world. But But if you think about even our jobs, right? So for instance, we are communicating actually on video or the internet right now. And so a lot of our jobs don't actually involve the physical world. So again, there is some physical infrastructure for the internet. A lot of it just involves the world of ideas. A lot of our jobs involve ideas and moving ideas around.

9:09And there is an interesting question on whether you can take energy and transform it in an instrumental way to move ideas, not just atoms. And if you think about a lot of the jobs we do, including personal computers and, you know, we have Microsoft Word, PowerPoint, a lot of it are tools, technology that help us do that. But it's basically a lot of steps, very laborious. And we can think of a lot of the new wave AI models as essentially simplifying those steps. And once we do that, that's such a huge unlock for humanity that I think it'll create more jobs rather than just destroy jobs because there are many companies that just won't be feasible using basically our current set of technologies.

10:06And so, for instance, if I have a very, very enterprising entrepreneur in India, say. And they do not have enough funding to have a team of 10 people. So this is a company that's never going to exist. But with these tools, maybe it exists. So I wouldn't think of that as, so let's just say they start this company with two people instead of 10 people. I wouldn't say that you lost eight jobs. I would say you gained two jobs because that's now a company that wouldn't have existed. In fact, if you look at the downstream effects, then that would actually create even more jobs. So I think I'm pretty excited about the possibility of essentially using energy to change things in highly interesting instrumental ways.

11:00Hey, everyone, we're going to take a quick break right now to hear a word from our partners. We'll be right back with more of the day's top analysis on the Real Vision Daily Briefing. Have you ever wanted to trade Bitcoin but haven't dared try? With Plus500 Futures, you can trade crypto without the hassle of opening a wallet. With just a few clicks, you can register and start practicing with their free and unlimited demo. See a trading opportunity? You'll be able to trade it in just two clicks. Feel ready? You can move to real money with as little as$100 once your account is approved. And the great thing is that in addition to crypto, Plus500 gives you access to a wide range of instruments.

11:36S &P 500, NASDAQ, gas, and much more. Explore equity indices, energy, metals, forex, and beyond. With a simple and intuitive platform, you could trade anytime, anywhere. Experience the fast, accessible futures trading you've been waiting for with Plus500. With over 20 years of experience, Plus500 is your gateway to the markets. Visit us.plus500.com to learn more. Trading in futures involves the risk of loss and is not suitable for everyone. Not all applicants will qualify. Plus500. It's trading with a plus.

12:12Yeah, and that's such an interesting perspective because, you know, one of the big questions I'm asked and one of the refrains we constantly hear about ChatGPT right now is, oh, what about my job? Any knowledge worker, you know, many knowledge workers, what about my job? I'm concerned about my job. Is it going to eat my, you know, my task in the workplace? It doesn't feel as though ChatGPT is sort of competent enough or multimodal enough, right, to just eat an entire person's job, even if their job is essentially knowledge-based. Would you agree? Absolutely. And this will also tie into some of the discussions we're going to have later on, where I think a lot of what ChatGPT and these large language models are going to do, I think are best understood as productivity enhancers.

13:04they are not going to be end-to-end solutions because we cannot trust those as end-to-end solutions. So we, for instance, Microsoft has this GitHub Copilot where it suggests code snippets based on various specifications you may have. And it even takes your existing code and tries to complete it forward. But you cannot trust it completely. So what I find is that it enhances the productivity of a programmer by 2x, 3x, but it cannot replace the programmer. So you still need the programmer. And will we ever get to a place where we can use this end-to-end? Not in the near future. Yeah, I mean, and that's a really interesting perspective and potentially a reassuring one for many people out there.

13:59and it has to align with certainly aligns with mine and it has to align with most people's you know anecdotal experience of using for example chat gpt which is exactly that you know it feels like a sort of jet pack for the mind it feels as though it can enhance your productivity but you can't just set it you can't just set it off and and have end-to-end replacement of entire entire tasks? So actually, if you look at the word copilot, Microsoft actually gave a lot of thought to what do we call this device? And they are calling it a copilot. So it's very clear to the people using the system that this is something that is a companion, that's something that's going to help you, but it cannot be something that is end-to-end and you cannot trust it fully.

14:49But at the same time, you can kind of interact with it in a very different way and so on. So you can kind of learn how to interact with it, but it's still a co-pilot. And I think that was a very smart nomenclature that exactly captures what it is. Yeah. Yeah. I think that was a very sort of wise choice of words and an accurate one. And so you have that sense of it being these tools being a productivity enhancer, sort of letting us think further, faster, I suppose. And I also love your perspective on the way it can help create jobs, because as you say, you know, you might have a company that needs five employees, but it can't possibly afford that.

15:36Well, now it could have two employees plus a whole lot of AI and you haven't lost three jobs you've created two because that company otherwise would not have existed. So do you see already where we're at now with these large language models and tools like ChatGPT? Do you see that sort of democratizing startup culture and the ability to to do your own thing and start your own thing? That's part of the path we're on that I find interesting. Absolutely. I think this is something that we can think of as there are certain tools. Microsoft had wanted to do this with just computing itself. So this was actually Bill Gates' vision, which is that he wanted to bring the power of computing to individuals.

16:31He wanted to democratize access to computing to individuals. And of course, we have gone far from that. We now have smart devices rather than just desktop sitting on our tables. But now if we move that forward, we not only will have computing devices, but we will also have programs, ways to interact rather with these computing devices that is more democratized. So we can think of programs as one way in which we communicate with these machines, these personal computers. And this was a language like the priests of your, where they have speaking in, you know, strange languages to these strange machines.

17:22And we needed to intermediate our interactions with these machines through these priests, which are these programmers. But if you can allow normal non-programmers to also interact with these machines, wouldn't that actually accelerate or be actually a more natural next step of Bill Gates' vision of having computing devices at the hands of everybody, but also in a way where they can interact with it fully? I can ask my smartphone to do certain things that I have in my mind, but I'm presently not able to do so because I won't be able to create an app. But now if I can do that, isn't that amazing?

18:09So I think that's actually a very, very exciting frontier, I think. It is hugely exciting, isn't it? And I mean, if you're roughly our age, you obviously remember the kind of the whole, you know, the birth of the app store and back in the sort of early 2010s, I guess it was when every third friend you had was creating their own app. And the constraint on that for many people is that they couldn't code. You know, everyone had an idea for an app they wanted to make. Most people can't code, so they simply couldn't make that app. Right. My feeling is we're removing that constraint. Yes, absolutely.

18:50Or at least reducing that constraint a little bit. So it won't be as simple as our grandma being able to code an app just from scratch. So we are still not there yet, but we are definitely reducing the barriers, i would say yeah yeah so the the potential for a sort of flourishing of creativity around computing is huge because to a much greater extent it feels to me you know if you can imagine it you can have a go at creating it you know whereas that simply wasn't true before that's right everyone on on twitter or x as we're now supposed to call it is excited today about the launch of this new like r1 rabbit device you know this little device that is like an ai companion there's also the humane ai pin so we're seeing this incredible the burgeoning of this of this wave of innovation around these ai companion devices do you think that's going to become a bigger part of people's of the story in 2024 and a bigger part of people's relationship with AI across the coming year and beyond?

20:09I love the vision at the very least, but I'm a little bit skeptical, I have to say. So I love the vision of reducing friction. So Apple is very good at this, where many people completely underestimate and underrate friction that prevents us from various tasks. And if you think about Apple, they try to make these devices so that the friction they're using in those devices is reduced. And if you think in terms of natural language interfaces, that reduces friction. So if I want the computer to open a particular app, instead of me going, opening the computer and then trying to find the app, if you can just say, please open DoorDash, and it opens it, that's great.

21:01And simple things like this, I think, can be enabled. But I think if we wanted to be a full-fledged companion in terms of saying, hey, book me a flight to London at the end of this month, I'm not sure we are there yet. So I think if we dial down our expectations for these companions, I think we are definitely going to get something. But I think to have a completely – have a computing device that is going to be completely disintermediated in the sense of all interactions that we have with these, if it's completely going to be intermediated by natural language and audio, I don't think we are there.

21:54I mean, Amazon tried to do this with Alexa. where they tried to, and Alexa is pretty good, right? So voice recognition is pretty good. They tried to get people to do various things with Alexa and they failed because there's still the just amount of friction. It's not perfect. And if it's not perfect, I don't want it to book a flight, right? So I think that's essentially the problem. Yeah, yeah, exactly. And I think they're going to have to play with kind of voice and audio and screens quite a lot to find the perfect combination of the two. Because, you know, when I book a flight, I want to be able to see the options in front of me.

22:40It's no good just telling me what they are. I think that those details are crucial. It's going to be, I mean, my other concern for those startups like Humane and the little rabbit device that's going on now is aren't they going to get crushed by Apple? Like when Apple puts a large language model on its devices and can just roll that out across a billion people, don't you think? Absolutely. I think if Apple or Google, for that matter, they do have this Gemini Nano that they already plan to push on to their Pixels or Pixel Pros, that's really the next frontier in the sense of having these powerful models on our computing devices natively.

23:32so that they don't need to communicate through the internet, through some server and pay some cost and so on. Once we have that, then it is, I think, a worthwhile question to say, do I need a pin or do I need a completely different device? Probably not. I would definitely agree. But on the other hand, companies like Google and Apple are not going to roll out half-baked products, right? So they won't roll out something unless it's close to perfect, especially Apple. They love to reduce the friction, but they're not going to release something unless the friction is completely gone. And so I think there's still some time for a company like Apple to actually release something.

24:22So I think in the next two, three years, these companies have, I think, a free reign. And maybe they even get acquired by Apple for all we know. So there's still, I think, a good space to explore potentially. Because the dream with Apple is some sort of vastly upgraded Siri, and maybe they don't call it Siri, that is a true on-device virtual companion that just knows everything about you. And you can talk to it. It's like a counselor. It's an assistant. It's a friend. But do you think that Apple have not done that yet? because it's so hard to perfect. And we know they're a perfectionist company.

25:02Exactly. That's really what is stopping them right now. It is always possible to roll out a half-baked product right now, but Apple won't be the one to do it. And so they are going to wait. And just even after coming with a product, they're going to experiment with it. And from a technological standpoint, We are nowhere near the place where a company like Apple would be happy with the product. So we are just not there. Now, speaking of the next frontier for machine intelligence, let's really dive into this, because I know that fundamental to your work is this idea that to really move beyond where we are now and to really see an impact on the real economy, sort of a deep economic impact, we need to move beyond the focus on the statistical approach to machine learning that sort of governs this moment.

26:05So this can start to feel somewhat technical. It doesn't actually have to be that technical, really, and you explain it so brilliantly. So let's get into all of that, just let's start by talking about what we even mean by statistical and symbolic approaches and what it is about the current moment that makes it the statistical moment. And that means diving in a little bit to the history of AI, doesn't it? Right. Yeah, absolutely. Yeah, I think it's a great point to talk about the history of AI. So if you dial back all the way to 1950s, when we had these powerful computing devices, people thought, oh, our next thing is to just build AI.

26:52If we have powerful computers, how hard is it going to be to build AI, right, is what they thought. So the first wave of AI was using what are called symbolic approaches. We are so-called because if you think about any beliefs we may have or any statements we may have. So, for instance, let's say David is a great journalist at Real Vision. So this is a statement and I can basically have various symbols to represent the statement. There's an entity named David who's a person and who's a journalist who works at Real Vision. And so all of these are, I can have symbols to represent this knowledge.

27:39And then I can also have reasoning to go from certain propositions to certain other propositions. So, for instance, given all of these statements, I would then perhaps have another proposition that says David does great interviews. And so that's a reasoning step where I go to a completely different proposition and go from A to B. And these were the kinds of programs that people were building early on. And we can think of these as largely rule-based systems where these rules operate on symbols. And the heyday of these were actually in the 80s and including in the 90s, where one prototypical instance is called Dendrol, where what it was, was it was given compounds, let's say hydrogen peroxide, H2O2.

28:34So given various measurements, I think spectrographic measurements, maybe I want to know the composition of this compound. It's two hydrogen atoms, two oxygen atoms. And they built a system that was better than organic chemists. And the way they did this was using a bunch of rules. So they had various symbols related to the measurements. They had various symbols related to the atoms and they related the measurements to the atoms and so on. And so given the measurements, looking at these rules, they'll say, oh, I think it's hydrogen peroxide. And there were these things called expert systems that were these gigantic rule-based systems sitting on top of symbols that people were building in the 80s.

29:19that, again, the similarity to the present is quite interesting where the companies and industry were spending millions upon millions of dollars to build these large-scale systems, and people thought that this was the future of AI. And it turned out not quite. Interestingly, Deep Blue that beat Caspro in chess was another instance of such symbolic approaches, where there the symbols related both positions and particular actions that you can take. So I think there are 64,

30:01I forget how many exact moves you can make from any given position, but what you can do is that you can roll out. So I will play a particular position, then the opponent could play any of are there 64 different possible positions? And I can just completely roll this out until the game finishes and then see if I move my knight to this position, then do I win? And I can just roll this out, but this obviously will take too long. And so they had various tricks to reduce basically the search. And this is how basically Deep Blue beat Casper. It was all very kind of rule-based approaches. But like I said, this kind of fizzled out because this was too laborious because where are you going to get the symbols?

30:51Where are you going to get the rules, right? But then there was a little bit of despair in the late 80s came this new field called machine learning. Where, let's take the dendrol example. So I have certain measurements, spectrographic measurements, and then I have what the compounds are. and instead of having organic chemists provide rules and what the symbols are, what are atoms and whatnot, what if I just give a training data that says, okay, here are the measurements, here is the compound, here are the measurements, here is the compound. I just give thousands and thousands of such mappings and let the machine learn how to associate bits of the input.

31:34There are statistical associations between parts of the input and the parts of the output and it will automatically learn that. And so this was the statistical learning phase, the second wave of AI that really picked up steam in the 90s and 2000s. And if you look at 2010s, there was a even further boost in terms of these architectures called deep neural networks that together with the foundation that people built, really turbocharged the statistical revolution. So that's where we are presently in terms of where AI is. We're going to take another quick break to hear a word from our partners. We'll be right back with more of the day's top analysis on the Real Vision Daily Briefing.

32:28Yeah, thank you. And I mean, it's just such a fascinating and expertly delivered, very brief history of machine intelligence. and so you have you know the initial first wave which as you say almost grew with the birth of computing itself there was an almost naive optimism that we've created computing it can't be long until we create a human-like intelligence and the route to that is is via symbolic representation am i right in understanding that part of the reason that fizzled out is because that symbolic approach can be effective for highly structured rule-bound domains, like, for example, chess.