#250 Pedro Domingos on the Real Path to AGI

24 Apr 2025 · 1 h 8 min

Ask about this episode

Ask anything about it. ChatGPT or Claude reads this page and answers with the times it was said.

Connect VO and ask about every podcast you hear, including the moments you saved. Add to ChatGPT · Add to Claude

In short

Episode Summary: Eye On A.I. - #250 Pedro Domingos on the Real Path to AGI

Host and Guest

  • Host: Craig S. Smith
  • Guest: Pedro Domingos, Professor at the University of Washington and author of *The Master Algorithm*.

Episode Overview In this episode, Pedro Domingos explores the intricacies of Artificial General Intelligence (AGI) and discusses the missing links in achieving it. He emphasizes the need for a radical unification of the five foundational paradigms of AI: Symbolists, Connectionists, Bayesians, Evolutionaries, and Analogizers.

Key Themes

  • Unification of AI Paradigms:
  • Domingos argues that a comprehensive approach incorporating elements from each of the five paradigms is essential for advancing towards AGI.
  • Limitations of Current AI Approaches:
  • He discusses why deep learning alone is insufficient for achieving AGI and highlights the importance of reasoning by analogy and evolutionary algorithms.

Key Topics Covered

  1. Five Tribes of AI:
  2. Symbolists: Focus on logic and rules.
  3. Connectionists: Based on neural networks and deep learning.
  4. Bayesians: Emphasize probability and statistical methods.
  5. Evolutionaries: Inspired by biological evolution.
  6. Analogizers: Use reasoning by analogy for problem-solving.
  1. Challenges in Achieving AGI:
  2. Creativity is easier than reliability in AI systems. Domingos suggests that reasoning by analogy could lead to breakthroughs akin to those of Einstein.
  3. Current AI models, including transformers (e.g., GPT-4), while impressive, are incomplete and lack true understanding.
  1. Hype vs. Reality in AGI:
  2. The tech industry's hype may overshadow actual scientific progress towards AGI. Domingos highlights the discrepancy between the excitement around AI and the real challenges that remain.
  1. Master Algorithm Concept:
  2. Domingos describes the quest for a "Master Algorithm" that could integrate the various learning paradigms. He contends that the search for a single, unifying learning algorithm is ongoing and complex.
  1. Creativity vs. Reliability:
  2. Creativity is seen as less complex for machines to achieve, whereas reliability remains a significant hurdle.
  1. Role of Evolutionary Algorithms:
  2. Evolutionary algorithms mimic biological processes and could play a crucial role in developing intelligent systems.
  1. Implications of Quantum Computing:
  2. Domingos expresses skepticism about the immediate applicability of quantum computing to AI but acknowledges its potential long-term impact.

Detailed Discussion Points

  • Designing with Bit Strings:
  • Domingos explains how genetic algorithms can evolve programs and circuits through simulated evolution, which can lead to innovative solutions.
  • Case-Based Reasoning (CBR):
  • He discusses how CBR can help AI systems adapt solutions from past experiences to new problems, making them efficient.
  • Combining AI Paradigms:
  • The discussion covers how various AI paradigms have historically been siloed but are increasingly merging in modern research.
  • Future Directions in AI Research:
  • Domingos advocates for deeper collaborations among AI researchers to explore unifying approaches that could lead to AGI.

Conclusion The conversation wraps up with Domingos expressing cautious optimism about the future of AI and the potential breakthroughs that could arise from integrating various AI paradigms. He emphasizes the importance of understanding the complexities of human intelligence and the challenges that remain on the path to AGI.

Additional Resources

  • Pedro Domingos: Author of *The Master Algorithm* and an expert in machine learning.
  • AI Research Community: Constantly evolving; the discussion encourages continued exploration of unifying AI methodologies.

---

Stay Updated

  • Follow Craig Smith on X: [@craigss](https://x.com/craigss)
  • Follow Eye on A.I. on X: [@EyeOn_AI](https://x.com/EyeOn_AI)

Written by AI. May contain mistakes. Listen to the episode to check what was said.

Hear the part that matters, and keep it.Open this episode in VO. Double tap your headphones to save a moment as you listen.
Get VO free

Transcript

Automatic transcript. May contain errors.

0:00All of these five tribes are 70 years old. They were there from the beginning. It's actually remarkable. In the world of AI, where change is perpetual and ever more rapid, certain things are remarkably constant. One of which is these five paradigms. They haven't changed. They all were all invented in the 50s. All of them in one way or another. And they're still the same ones today. When I talk to people about the five schools, people who are not AI experts, the one that immediately resonates with them is reasoning by analogy. Everybody can understand because we do it all the time. Create an oasis with Thuma, a modern design company that specializes in furniture and home goods.

0:38By stripping away everything but the essential, Thuma makes elevated beds with premium materials and intentional details. I'm in the process of reorganizing my house, and I'm giving Thuma a serious look for help in renovating and redesigning. Thuma combines the perfect balance of form, craftsmanship, and functionality. With over 17 ,000 five-star reviews, the Thuma Bed Collection is proof that simplicity is the truest form of sophistication. Using the technique of Japanese joinery, pieces are crafted from solid wood and precision cut for a silent, stable foundation. With clean lines, subtle curves, and minimalist style, the Thuma bed collection is available in four signature finishes to match any design aesthetic.

1:40Headboard upgrades are available for customization as desired. To get$100 toward your first bed purchase, go to Thuma, that's T-H-U-M-A dot C-O slash IonAI. IonAI all run together, E-Y-E-O-N-A-I. So for$100 off your first purchase, go to Thuma dot C-O slash IonAI. That's T-H-U-M-A dot C-O slash IonAI to receive$100 off your first bed purchase. Hi, I'm Pedro Domingos. I'm a professor of computer science at the University of Washington and an AI researcher. A few years ago, I wrote a book called The Master Algorithm that was an introduction to what I call the five tribes of machine learning, the five main paradigms.

2:38and we've been going through them one at a time. We've covered the symbolists who traditionally dominated AI, the connectionists or deep learning who dominated today, the Bayesians who have always been there and always will be. And today we're going to do the last two, which is the evolutionaries who do AI inspired by evolution and the analogizers who do reasoning by analogy. Okay. uh well why don't why don't we start with the uh evolutionaries or or the other way around which other whichever is more current uh i mean they're both current maybe the evolutionaries are a nice segue from uh the connection it's because they have something very important in common which is they are also inspired by biology yeah the evolutionaries and the connections both believe in doing ai inspired by psychology the others don't the others think that's a silly idea because biology is a mess and suboptimal and blah blah but the connectionists are inspired by the brain the architecture of the brain if we can produce that in hardware then we're on our way uh but the evolutionary say like well wait a second that's really not the whole problem like where did that architecture come from right you're only tweaking some way it's big deal right what you want to know is like how do you create a brain in the first place and uh you know nature has an algorithm for creating brains and robots and a lot of other things, and that's evolution.

4:05And you truly can't think of evolution as an algorithm. In fact, people in the 19th century are already saying the equivalent thing. I forget who it was that said, maybe it was George Boole, right, who invented the logic that computers are based on. He said words to the effect that God does not create animals and plants. He creates the algorithm by which animals and plants are made. So what we're going to do is be a little God on the computer and mimic the evolutionary process on the computer, except that instead of evolving animals and plants, we evolve programs, circuits, robots. In fact, people have done the whole gamut.

4:40Even there's patents for, I don't know, things like radios and amplifiers based on designs that were created by some of these so-called genetic algorithms because they're inspired by genetics. And there's also this whole field of genetic program where you evolve programs. And it's interesting how your basic genetic algorithm is really a very literal translation to the computer of our basic understanding of evolution. There's a population of strings, literally bit strings. And then there's several generations. You start out with random ones. This is the amazing thing. You start out with random strings and after a while you're doing amazing things.

5:18Like for example, there's this guy, Hod Lipson, who's at Cornell or maybe somewhere else now. Now he has this lab where they literally evolve robot insects from scratch. They start by doing it in simulation. And after a while they just manufacture them like they 3D print them or something, and they start crawling and flying in the real world. So you start with these random strings and then you mutate them. You cross them over, right? That is the key thing in evolutionary computing that is not present in say, you know, a gradient descent or things like that. there's, you know, you measure the fitness of your systems or programs or whatever based on that string at the task, right?

5:57There's, as usual, a reward, an objective function of some kind. And then the best performing ones get to mate. Literally, it's like sex on a computer. And then the offspring, right, are the new generation, and those in turn will be evaluated and so on. So it's this very basic mimicking of evolution that works to a surprising degree. Now, we can go into some of the history and why they kind of diverged from the rest of machine learning, and some people are very skeptical about AI and whether or not they're truly capturing the important things about evolution and what the future of the whole thing is.

6:33But there's certainly a lot of interesting things there. Let me just ask, can you define when you say these strings, Are you talking about functions or what are you talking about when you say strings? Sorry, yeah, I lapsed into technical jargon there. A string in computer science is just a series of bits. I mean, it can be a series of characters. For example, a sentence in English for a computer scientist is a string of characters. Okay. Because it looks like a string, right? It's a long linear thing. and the genome, right? A DNA is a string of letters. That's the amazing thing is that like everything about how you and I are made is encoded in a bunch of these strings of TGA, TC, et cetera, et cetera, right?

7:19It's kind of mind boggling, right? But there it all is, right? If we know how to manipulate those strings, maybe we can get far. So it's not a function. It's about a string is just about the simplest thing you can possibly have in computer science. Yeah. And you say you start with random strings. How then do they evolve? Do you have an algorithm operating on them? Again, the idea is kind of mind-bogglingly simple, and it shouldn't work, but it does. And the proof is real-life evolution, right? So let's suppose that I want to... What's a simple example? I want, and this is a real example. I want to build a radio, right?

8:07A radio is a bunch of electronic components put together, you know, a transistor, you know, resistors, capacitors, blah, blah, blah. You need to tune it to whatever frequency, right? We know how to do that. But maybe there's better, more efficient ways to do, to make a radio. So the way you do a general algorithm is that you start out with a bunch of random strings. But what you have, and often this is a very simple algorithm, but it needn't be, is something that that string is a specification of how to build a radio in that case. Like for example, if there's a one in a certain position, it means that this transistor is connected to that resistor.

8:44So it's like, I have a pile of transistors and resistors, and my string just specifies who's connected to whom, right? If you have all the necessary components and we know what they are, and you allow possible connections, one of those strings is your radio. And another string is potentially an even better radio. So now you take a random string, you build the corresponding radio, and in practice you only have to simulate it. There's packages to do that that electrical engineers use. And then it's probably a terrible radio. Like your first generation of a thousand things, they're all terrible radios.

9:17But one of them kind of picks up a little bit of something. Just randomly, some will be better than others. And so there's another. So then you take those two strings and you randomly mutate them, right? Because some of evolution is driven by random mutations being like, you just flip some of the bits, which means, oh, you know, this transistor was connected to this resistor, but now let me connect it to that capacitor instead at random. And more powerful in principle, right? And this is really where the interesting part is. You say like, well, here are two strings that actually seem to be better than random at describing a radio.

9:50Let me do a crossover between them like you do in, you know, in evolution, which is I'm going to take half of the string from one side and half from the other, the mom and the dad, if you will. And I have a new string that's in your radio. And maybe that string is actually more garbage than the previous ones were. But if you think about it, like if this string had something good here and this string had something good here, if you pick this part and this part, the new string is actually better than either the previous ones. And if you do that for 100 generations, lo and behold, you actually have a fantastic radio.

10:23And driving the mutations and choosing the leaders or the winners out of the offspring, is that reinforcement learning? Or what is the assessment algorithm? of them. It's not reinforcement learning, but in fact, you can think of reinforcement learning as being sped up evolution. It's actually good to, and you know, some people have actually formalized this and there's interesting lessons. Reinforcement learning is what animals do, right? And it's discovering how to do something properly, which prior to there being reinforcement learning, could only be done by evolution. So you can, it's actually the other way around.

11:10You can, you can think of reinforcement learning as a more efficient way to evolve. And in fact, You think of us people having ideas as an even more efficient way to evolve and, you know, things keep speeding up. But evolution actually is, or at least this basic, you know, version of evolution, which is really a cartoon understanding of evolution, is really just this. How does evolution evaluate animals? It throws them out in the world and sees if they survive and reproduce. That's what it is, right? And, you know, there's a mathematical theory of evolution at this point, which, of course, you know, Darwin didn't have, but is, you know, part of the so-called modern synthesis.

11:44And there's this notion of a fitness function, which is really the equivalent of the reward function in reinforcement learning. Reward function is, you know, you touch the stove, you get pain, right? You eat an ice cream, you get pleasure, right? The fitness function in evolution is essentially how many offspring you have. but how many offspring you have is driven by how well adapted to the environment you are so if you're a bird then you have a better wing or you have a lighter skeleton you're able to fly better faster farther etc etc so that's your fitness function so the whole art if you will there's always somewhere in machine learning where the whole art goes and and and and in generic columns it's in defining the feature the fitness function yeah right i mean and for example in the in the case of the radio that i was talking about your fitness function can literally be this for example this thing called spice which is a software package that will simulate any electronic circuit right you you defined your your circuit as i described using a random string or at some point a not random string but then you you you create that circuit in software you put it through spice and and you test it as a radio you go okay let me try to listen to you know fm 101 one does does that work when i do that so you have this battery of tests which is you know the the the generic algorithm of like throwing out the circuit in the real world and seeing if it actually does you know what it's supposed to be fit for which is you know let you listen to the radio and this process you said you know uh you end up you know 10 000 generations down and you have something uh it's all automated or it does it require uh intervention no i mean there's all sorts of variations but the basic version does not require any intervention and it works surprisingly well and it's often not even 10 000 generations i remember a long time ago i seeing this demo something that was very famous then called the connection machine it was one of the first massively parallel computers.

13:46And it did the following thing. This was at, you know, some graphics exhibition or something. It showed you 10 random images, random images. And you pick the one that you like best, right? Why not? Right? Like, I mean, abstract art. And you do this for half a dozen times. And even after that small number of times, right, literally, while other people are standing in line, after those, let's say, 10 generations, it's actually generating really amazing images. Right? So evolution is a shockingly effective learning method. Yeah. Are there any practical applications out there that use this method?

14:26I mean, you talked about the insects. Was that at CMU, you said? The insects were at Cornell. It was this guy called Hodlipson. I think he's moved to NYU or somewhere else. But anyway. So as I mentioned, there have been, you know, real radios and amplifiers and whatnot designed in this way. The genetic algorithms folks have a whole list of things they claim are successes of genetic algorithms, like creating a, you know, robot playing soccer, a robot soccer playing team, et cetera, et cetera. This is very controversial, however, because the other people in machine learning said like, nah, those applications aren't really real.

15:11And you could do them with, you know, just greedy search. And, you know, there was actually at one point a very famous bust up, which we could go into. And the consensus in machine learning is that that stuff is useless. In fact, many of the machine learning people, when they saw my book, said like, why did you even write a chapter about that? But on the other hand, you could have said that about neural networks circa 2000. Like, why did you even write a chapter about that? That's crap. It doesn't work. So, you know, place your bets. Okay. And then the analogizers. So let's talk about the analogizers.

15:51So analogizers are, we're going to do machine learning and AI based on reasoning by analogy. So first of all, what is reasoning by analogy is I have a problem to solve. And what I do is I retrieve from my memory similar problems that I solved before. I don't, I, you know, typically, yeah, like you build up your solution one little piece at a time, which is incredibly expensive and inefficient. Right. And you do chain of thought prompting and like blah, blah, blah. Right. And the analogy is like, oh my God, that's such a headache. what you do what you and i do right is we we do this automatically every day from the smallest things to the biggest ones is like we retrieve from memory similar episodes similar problems and then we adapt the solution to that problem to the new one this is an incredibly powerful thing to do and in fact the terminologizer was coined by a famous guy douglas hofstater the writer of Gödel Escher Bach.

16:51His most recent book is called Surfaces and Essences, Analogy as the Fuel and Fire of Thinking. And it's basically 600 pages proving, according to him, that every single thing in cognition, from the simplest word use things to the highest achievements of the Einsteins and whatnot, it's all reasoning by analogy and nothing else, right? Right. So it really is, you know, he really does think that analogy is the master algorithm. Now, I think he's gone a little too far, of course, because, again, analogy solves some problems, but not all of them. But there's no denying that this argument has a lot of force.

17:31And I would even say that reasoning by analogy is the most unfairly ignored school in AI. However, right, so interesting point. When I talk to people about the five schools, like people who are not AI experts, the one that immediately resonates with them is reasoning by analogy. You know, vision, learning, that's what is that? Like Bayes' theorem, symbolic AI, all of that is like not, you know, neural networks. Yeah, the brain, but it's a pile of numbers. But because we do it all the time right so there's a there's a lot of intuitive appeal in it and moreover and very importantly in psychology and cognitive science there is a literature going back decades of thousands of papers doing this thing of showing how you do all these things by analogy often in different ways than what douglas of stutter says like you know there's a thing called structure mapping and whatnot which is you know which would uh which has gotten you know uh um a lot of play uh it was invented by by data gentner etc etc so that's one aspect but this was influential maybe in ai i don't know like decades ago right what what is more relevant is that until fairly recently until the alex net explosion the dominant paradigm in machine learning was kernel machines right everybody that everything is in kernel machines including vision, right?

19:00The state of the art of vision was a so-called support vector machine, which is a simple form of kernel machine, right? That was the state of the art. That's what people believed was the right thing to do. They don't call themselves analogizers, and people like Hofstadter, but it really, you know, like kernel machines are a primitive form of reasoning by analogy. A kernel is a similarity function. Yeah. And I just, for listeners, I did an episode on support vector machines, but can you describe, define for listeners what a kernel machine is? Yeah. So what is a kernel machine, right? First of all, what is a kernel?

19:42A kernel is just a clunky mathematical term for a function that measures the similarity of two objects. You give it two objects. You know, you give it you and me, and it gives us a score for how similar we are using some set of attributes, right? You know, like how tall are you? How smart? What is your job? Or whatever, right? Or how similar two images are based on the pixels or something more sophisticated. Again, in kernel machines, the secret sauce is how you design the kernel, and it could be learned and whatnot. But then what it does is like for every pair of examples, it spits out a number saying you're very similar or not so similar, right?

20:16And then the kernel machine, in addition to the kernel, what it has is a bunch of examples that it saw in the past from your so-called training data is all machine learning. It throws out most of them, but it saves some key ones. And those are the support vectors. Support vector machine comes from the term support vector. And the vector is just, you know, it's an example, right? It's a series of values of pixels or whatever, right? It stores those support vectors because they're the ones that are going to support the decisions that you make. And then when the new example comes along, it's actually very simple.

20:48Like I'm a doctor. I want to diagnose my new patient. I don't know. Let's say I don't know about anything about medicine. Right. And what do I have? I have a file of past patients. I have a new patient in front of me. I ask her, what are your symptoms? Tell me like I fill out this vector for her. And then I go in my file system and I look for the patient with the most similar symptoms. And I say, oh, this patient has whatever COVID. you have COVID too, which sounds incredibly dumb and simplistic, but there's a mathematical proof, right? That if you do this with enough examples, you can learn any function.

21:25In fact, nearest neighbor, which we talked about before, is the simplest analogy, you know, similarity based algorithm is nearest neighbor, right? Which is as simple as it can get. And kernel machines are really just a more sophisticated version of the nearest neighbor algorithm. yeah so how how does uh how does i mean beyond support vector machines how does uh how do the analogizers develop new architectures or algorithms to do new things let me give you an example so actually first of all let me tell you what structure mapping is structure mapping is probably the key concept in reasoning by analogy.

22:07And the idea in structure mapping is that every problem, every domain has a structure. And what I do to solve a new problem is map the structure from the problem that I've seen. So for example, famous example of structure mapping, reasoning by analogy, Niels Bohr's model of the atom. In his time, people discovered that, hey, you shoot these alpha particles at an atom, and most of them go right through. A few of them bounce right back. They knew that there were electrons with negative charge around somewhere, and now clearly the positive charge seems to be concentrated in a small thing in the middle, right?

22:44Most of it is vacuum. What does this remind you of? It reminds you of the solar system with the nucleus as the sun and electrons as the planets. And so Niels Bohr was like, aha, I'm going to do a model of the atom by analogy with the solar system, which actually historically turned out to not to be that accurate, but it was a very, it was a key step in the development of quantum mechanics. So what did he do, right? He noticed a similarity between the atom and the solar system, and then he mapped the structure of the solar system with the sun in the middle and the planets revolving around it at various distances to the atom, which in many ways is a remarkably accurate picture with the different shells, you know, which are electrons at different distances, you know, roughly speaking and so on.

23:30Right. So mapping the structure gives you such a leap in problem solving, right? You're like, you're not lost in the woods anymore. Now you're like, okay, this is how I'm going to try to solve the problem. And now let me adjust it to give a more modern and more relevant example. One form of, of, of analogy, you know, based AI is called case based reasoning, which is reasoning by cases. and has actually been very popular for decades in call centers and help centers. And they work just like this. You know, like you have, you know, you're Microsoft. You have a help desk for people who are having problems with whatever, Windows.

24:04Right. And you call up and say like, hey, you know, my printer isn't working. It's spewing out garbage. Help me. Right. And then what the system does, and again, this works really well in a large fraction of the case, like it asks you a few stock questions and it goes like, aha, what are of the problems in my database, right? Or knowledge base that have these characteristics and let me not just suggest the same solution, but now tweak it. Oh, you have a different version of Windows. So this part changes and your printer is Epson instead of SP. So this part changes, but you take that solution and you adapt it to the new customers.

24:39And the shocking amount of the time this solves the problem at, of course, much less cost than asking a human or having some big, you know, symbolic AI or like large language models spending a ton of money to get to the same conclusion. Yeah. But that to me sounds like a kind of search, not a way to solve more complex problems. Or are there... So nearest neighbor and kernel machines effectively, you could say they don't do any search there are versions of them where for example i mean like let me refine that the basic and we'll do this in a few stages the basic neural nest in this problem doesn't do any search it remembers all the examples and it just spits out the answer you have whatever breast cancer or you don't or etc right now there are versions of newest neighbor that try to cleverly select the best examples to remember so they don't have to remember all of them and it's more efficient and that's what a support vector machine is it's actually clever way to search for the best example So there's already some search going on there.

25:43Now, what is the advantage of structure mapping or case-based reasoning? You're right. There is still search going on here. But the key thing is that this search is way more efficient than trying to find your solution one step at a time. It retrieves a whole chunk. Little in psychology, they call this a chunk, right? It retrieves a whole chunk that is relevant to solving your problem that you then only have to tweak. So you're right. It's just, I mean, on an ideal day, there is no search. You found the answer, you give it to me, right? My problem is exactly the same as the one that somebody had.

26:18That actually happens a good chunk of the time. But more generally, there will be search, but the number of steps in that search might be, you know, a dozen instead of, you know, a million. Yeah. Yeah. Yeah. Okay. So I want to spend the last half hour talking about how these schools, where they stand today. And I remember talking to you a few years ago about the master algorithm in the book. and you were saying that, well, the master algorithm is probably not a single algorithm. It's a system of algorithms or a family of algorithms that work on different parts of any problem. And how much, I mean, certainly reinforcement learning is being blended with generative AI with transformer architectures.

27:23But how much blending or working together is going on these days among these different schools? or do they remain fairly siloed? No, they don't. So in fact, it used to be, so there's a very interesting history in all of this. The first point of which is that all of these five tribes are 70 years old. They were there from the beginning. It's actually remarkable. In the world of AI where change is perpetual and ever more rapid, certain things are remarkably constant. One of which is these five paradigms. They haven't changed. They all were all invented in the 50s, all of them, in one way or another.

28:08And they're still the same ones today. And another very interesting point is every decade, a different one dominates. Right? In the 60s, you know, neural networks dominated. And then in the 70s, it was symbolic AI. Yeah. It said, et cetera, like, you know, the 90s were the Bayesian decade. The 2000s were the kernel decade. And then, you know, and then neural networks came back. Now you could say, well, this time is different. Neural networks are going to dominate forever and the others are now irrelevant. A lot of people think that. Or you could say, well, extrapolating from history, some other one of these paradigms is going to make a comeback any day now.

28:51Now, it used to be that these paradigms were fairly separate and the people actually had a somewhat antagonistic relation. Circa 1990, the symbolists would say, like, like, neural networks are a bunch of garbage. And then neural networks, people would say, symbolic AI is a bunch of garbage, right? And so on. And then the Bayesians came along and said, like, no, you guys are both a bunch of garbage. I mean, I remember I used to go to ICML, that was the symbolic machine learning conference, and to NIPS, which was the neural conference. And almost nobody, there were like half a dozen people in the world that actually went to both of them, right?

29:27But then they actually mailed it, right? And this was really largely brought about by support vector machines that first took over the neural network community and then took over the symbolic community. And then people started publishing in one or another indiscriminately. At this point today, there is no difference between ICML and Europe. So at that level, things have completely merged. Now, there are still people who identifiably work in these paradigms, but there are also a lot of people, including myself, who've done a lot of work combining them. right for example there's this whole area called neuro symbolic ai whose whole agenda is is to combine neural ai and and and and symbolic ai and in fact this was popular in the 80s you know jeff hinton circa 1990 was doing what they called connectionist symbol processing yeah right and here's the thing it comes and goes but actually right now this very moment today combining symbolic and and and neural ai is the thing right not necessarily by that name but what you see in the papers is getting these neural models to reason, which is of course is what symbolic AI is for.

30:28And what people have been doing sometimes consciously, sometimes unconsciously and reinventing the wheel is bringing techniques from symbolic AI over. So for example, what is a one, right? Chat EPT is a one. It's a combination of an LLM, which is a neural system with symbolic search in ways that they haven't made public, but in one way or another, this is what's going on. So this agenda of combining the paradigms is very much alive and I would say gaining power. Although again, this waxes and wanes, right? There was a previous one a few years ago that petered out and maybe this one will peter out as well.

Read the full transcript

31:00My bet is at the end of the day, you will need to combine these and not just ideas from two of them, but from all of them, right? How long that will take to happen is an open question. Now, you mentioned that I say there is necessarily one master algorithm. What I mean by that is not that you're going to have this. A lot of people do this, like I have a symbolic subroutine that I call when I have a symbolic problem, like, you know, retrieval augmented generation. For those of you who are familiar with it, is it really a symbolic subroutine in a neural system? Right. That is not that's a very shallow combination of the two.

31:37And, you know, I spent some time in the book explaining why that is not the answer. I really think, UD, the deep unification of the two, and a good analogy to this, reasoning by analogy, is electromagnetism. Maxwell didn't say the physical world is a program that has the electricity subroutine that he calls some of the time, and other times he calls the magnetism subroutine. That's not it at all. What he actually showed brilliantly is that they are the same force. It's a unification, not a combination. And I believe this is actually what's needed. And when AI is mature, as in any mature science, this is what we'll have.

32:18Now, there's an important point here, which is that algorithm, there's no single form that it has to take. And again, a good analogy here is Turing machines. Alan Turing discovered or invented the concept of a Turing machine, which is a machine that can do anything, which at the time was a very strange idea. like a sewing machine would sew and a typewriter would type. But what's a machine that both sews and types? But a computer, this is the essence of a computer. It's not a machine that does anything. But there are hundreds or probably even thousands of different things that are what are called Turing complete or Turing equivalent.

32:53His actual machine nobody uses. We have a von Neumann computer. What you have in your cell phone and we all have are von Neumann computers, which is an architecture. So the real Turing machine is the von Neumann architecture, but conceptually they're the same. And there will continue to be new ones. So my point is that we need to arrive at the first form of the master algorithm. In some ways, it doesn't even matter what it is. Then there will be many variations that are good for different things. But in machine learning, if you will, it's inductive reasoning. We don't even have what they have for deductive reasoning, which is the concept of a Turing machine.

33:26And that's the first thing we need to get to. Yeah. And to get to these, you know, these reasoning models, whether they're unified or whether they're, you know, two systems, one calling the other, as they get more powerful, I mean, do you think that they will help solve this problem of how to unify the values? various tribes. I mean, do you have any confidence in reasoning models' ability to advance scientific research? Yeah. So there are many paths to the master algorithm, and people are following them. You could start with any two of these things, combine them, unify them, and then bring in the third.

34:20That's largely what I've done over the last 20 years. And so there are many ways to get there. The one that is most popular right now is to start with a connectionist system, with a deep learning system, and then bolt on symbolic reasoning capabilities. The way that has been done mostly so far is very shallow, and I don't think it's going to survive the test of time. But I do think, in very concrete ways, that there is a deep combination of them. But to take maybe a better example, think of transformers. A transformer is a type of neural network. A transformer is a neural network, but it's much more powerful than a multilay perceptron, which was the architecture that preceded it, in a way that there's many attempts to understand what it's doing.

35:05But I would say that at least my best understanding of what it's doing is it actually has some of the capabilities that symbolic AI has that previous neural networks didn't have. So in a way, a transformer is, in a way that we don't fully understand yet, a combination of connectionist and symbolic features. And that's what makes it so powerful. In fact, the closest thing to the mass rocket that we have today is a transformer. And if you think about it, right, a lot of people were skeptical when the book came out because it was before transformers came out. But transformers today are one algorithm, right, that does all of these things that you see in the media every day.

35:41It's remarkable. Yeah. Yeah. But on the idea of using AI to advance research, you know, basically transformer-based systems, do you see promise in that? or do you think that the AI itself won't be able to advance thought, that we're still going to need human intuition and creative thought and all of that? I mean, at some point, it seems like these models are becoming powerful enough and there's enough human knowledge encoded out there that they can ingest that there will be some creativity. So I think you have a number of questions there. So let me try this one part at a time. Creativity, right?

36:48People used to think that creativity was something that computers would never have. you know about more of x paradox which is this notion that the things that we think are easy for you are hard and vice versa yeah because the things that are easy for humans are easy for humans because evolution spent 500 million years evolving us and i have this slide that i've used in various talks that is more of x paradox easy versus hard and one of the lines that i have in there is easy creativity hard reliability and i would say to people reliability is hard creativity is easy and be like, what are you talking about?

37:26You're smoking. And even a couple of years ago, right? And these days I just rest my case. And I like, like creativity is like, well, you know, use whatever Dali or Midjourney, like generate videos, generate poems, generate music, just like, you know, whatever. Creativity is easy. Reliability, no one knows how to make an LLM reliable today. And that's the problem, right? So I don't think creativity, intuition, are there's nothing magical about them right we i mean i used to be a musician and write songs and people you know have this notion like writing a song is like some kind of magical inspiration that comes to you from the muse right it's not right in fact anybody can write an okay song writing a hit is really hard right but you know like if you spend you know whatever a year learning to play guitar piano and start playing you will write okay songs right so creativity is not magic So I don't think there's anything that humans do that AI can't.

38:20And most people in the community, this is what they believe. It might be very hard. It might take a long time. But unless you have some mystical belief about what goes on in the brain, it's a bunch of atoms. And in fact, if you believe in reductionism, then the mass problem exists because it's the one running in your head and mind right now. So I think at that level, there's no doubt that we'll get there. Now, where are we in terms of AI being able to, for example, do real scientific discovery? Could an AI, for example, come up with general relativity or solve the problem of unifying it with the standard model?

38:54And the answer to that is today we're nowhere close to that. And this is very interesting. In fact, people have remarked on this, that the application of AI across the sciences is progressing very rapidly. Right. Physics, economics, biology, et cetera. They're full of AI these days, but it's AI that does lower level stuff. It doesn't do that really creative, you know, things like people, Newton and Einstein and whatnot did. Right. And it's interesting because the LLMs have a bigger knowledge base. They've read every paper that's ever been written. So come on, like, where are the discoveries? Right.

39:29A human being with that knowledge base would be doing amazing things every day now. So clearly something is still missing. And our job, the researchers, is to discover that thing that's still missing. And I have this long running argument going on with Jan LeCun because Jan, he thinks back propagation is the master algorithm. He thinks machine learning will evolve and blah, blah, blah. But at the end of the day, the solution is still going to be gradient descent. right he's like a fundamentalist connectionist in that regard and you know and i asked him this question that he has no answer to he's like okay how did einstein come up with general relativity by gradient descent right now you're like there's no answer to that question so clearly something is missing and do you think what's missing is in one of these schools that you've defined Exactly.

40:25That is the right question. So for example, Douglas Hofstadter in his book that I mentioned, general relativity is one of these examples of something that was discovered by analogy. So clearly reasoning by analogy is important. And again, it's very interesting that Jeff Hinton, who's really the godfather of deep learning, he says, he's been saying this forever, that neural networks are better than symbolic AI because they reason by analogy. but Jeff where is the reasoning by analogy explain to me where it is now I I mean I can tell you like where I think the reasoning by analogy is happening in neural networks and and and at the end of the day we're going to have a single algorithm that in a way looks like a neural network but does reasoning by analogy and the symbolic reasoning and you know we could get into the weeds there but this is I think you know where the solution is are there are there is there active research on that in building reasoning by analogy into these systems?

41:29So, for example, there's a longstanding active area of research on what is called automated discovery. And it started in the 70s with people like Pat Langley doing theses where they show, like, look, this system rediscovers Kepler's laws or Boyle's law or simple laws like that. And actually, again, recently that's picked up and people have all this work on, you know, discovering differential equations and discovering, you know, how different systems work using AI. It's still, I think, at the level of Kepler, not at the level of Newton. The level of Newton requires, I think, some of this reasoning by analogy.

42:03And again, there are people like, you know, in psychology and cognitive science who have looked at, you know, how that might work, right? So there's a lot written on this. I don't think anyone has solved it. I also think that, disappointingly, in mainstream AI research, there's a lot of stuff going on, but not this. There aren't a lot of people going explicitly, like, how can we get a neural network to reason by analogy and therefore do scientific discovery? Which to me is a scandal, right? Like, you know, some fraction of people should be doing this instead of doing more tweaks on LLMs. yeah and also the evolutionary uh uh tribe or or school that sounds very promising I mean you don't have to start with random strings you can start with a system that's already very advanced and uh and go from there I mean are there people doing that I mean there are and and and what you make a very good point is like, you probably don't want to start with random strings.

43:07And fortunately, a lot of machine learning people have this machine learning and the connectionists and evolutionaries are both great examples of this. They have this fundamentalist machine learning attitude in that like, no, we want to learn everything from scratch. If you put in knowledge, you're cheating. The Bayjans and the symbolists don't have that problem at all. On the contrary, the Bayjans are all about priors, which is literally putting in your prior knowledge. And the symbolists are all about combining learning with knowledge-based AI, because that's what symbolic AI traditionally was.

43:37You start with the knowledge base and then you refine it, which I think is an excellent idea. So why throw that away? And you could think of what LMs are doing today is they are acquiring a knowledge base from texts. That's what they're doing in a very convoluted way. And then they're flexible about how they reason with that knowledge base. It's very opaque. And then clearly what's missing is the ability to reason on top of that text, which really is what things like Awan and so on and DeepSeek are trying to do. So there is a way to look at all of this and say, yeah, in one way or another, things are moving in the right direction and we will eventually get there.

44:14Yeah.

44:19It just seems that there is enough research out there now. and in all these different disciplines or schools that these AI reasoning models could or should be able to go across all of it and find analogies or go across all of it and find opportunities for evolutionary algorithms to advance what's already there? I mean,

45:10it's interesting because, I mean, so to take the evolutionaries to begin with, they are at this point the tribe that is most distant from the others. The other four, I mean, the support vector machine people, I mean, like all these people, they're mixing it up at this point. The evolutionaries, there's very little, but there are a few things like, you know, generative adversarial networks, as you very sharply pointed out last time, there is a flavor of co-evolution to that. So there is one path by which they could come in. There's also this whole area of multi-agent systems. So there's a type of reinforcement learning that is very close to ideas from generic algorithms.

45:51OpenAI at one point, before the whole chat GPT thing, they had these papers showing like, look, there are a lot of problems for which surprisingly, if instead of reinforcement learning, we just use a simple generic algorithm, it actually gets to the solution faster. So a lot of this is happening. Now, unfortunately, the problem is that like there's more ai research than ever before today like by an order of magnitude or two but most of it is along a very narrow front and you know one common view i would even say probably the prevailing view is that like you know we're just going to keep pounding on this and you know we're going to do so many things eventually we'll solve the problem i'm not so sure because you know the saying that you know nine women can't make a baby in one month right you don't get you You don't get, for example, to general activity by having a thousand random physicists just do what they do for a century.

46:43It doesn't work that way. One way that I often put this is like solving AI is not a sprint. It's a marathon. I really think somebody needs to go really deep. And there are people trying to do that, but I think not enough. And then once we do that, we will see how all these people are just spinning their wheels. and all of that unfortunately is going to go in the garbage can of history and like I know which part of those I want to be in you know I've been talking to people about quantum computing in the past week which is advancing and you know and the timeline is looking more not realistic I should say it's looking shorter to getting to a practical quantum computer.

47:35Do you think that will advance any of this? I mean, if you can, and not for just large problems that classical computing struggles with, but presumably with quantum computers, you'll begin to understand where quantum physics and Newtonian physics meet. Like what happens at the quantum level when a virus attaches to a protein? You know, right now it's very Newtonian the way people think about it. You know, it's shapes.

48:30But anyway, do you track what's happening in quantum and think about how that may address some of these issues? So the promise of quantum computing is that it can solve problems exponentially faster than classical computing. And if that ever comes to pass, boy, can we ever use it in AI. We will be the biggest consumers of quantum computing in the universe. No questions asked. So that is the number one promise of quantum computing. Now, having said that, there are a lot of caveats here. One of them is that if you talk to the people who are kind of like serious and knowledgeable of quantum computing as opposed to the people hyping various things, they will tell you that it's exceedingly unlikely at this point that there will ever be a general purpose quantum computer in the same way that there are general purpose classical computers right that's just not it doesn't look like it's going to happen there may be quantum computers for specific problems that are there that is the hope right and if those problems are are important enough then great and and those some of those problems are in ai so in fact there's one type of quantum computing that is about finding global optimal by tunneling out of the of the local ones right there's this company called d-wave that you know uh claims to have done this and yeah we could totally use that So that's the promise.

49:52Now, will that ever happen? And how soon will that be? We are still, you know, I roughly track, not very closely, what's happening in quantum computing, just out of curiosity more than anything else. I mean, honestly, I think it's a hard bet to lay because quantum computing is such a hard problem. And there's this whole notion like, well, there's all this computing in superposition and that's where the magic happens. But the error correction really kills you. The whole thing is so fragile. And making it robust is so expensive. You need a thousand qubits to actually have a robust one. And it goes on from there.

50:32You need super, super low temperature. Right. And and, you know, the first real, you know, quantum computer that does something useful is years away, maybe decades away. Right. And you could also make arguments, which some people do, about why this is never going to happen, that all of quantum computing is really a misunderstanding and a misconception and they're just deluding themselves. Or that it's a very nice idea at a theoretical level, which is where it began, but it'll never be practical. So, you know, as an AI person, my attitude to this is I wish them great success. I don't think we in AI, we will depend on that success.

51:08We, you know, in a way, AI is about a different path to doing exponentially faster computation. It's about being smart about your classical algorithms. And in fact, one suspicion that some people have, in fact, Demis Asavis was talking about it the other day, is that maybe, and I think this is a real possibility, we in AI, or in computer science more general, will discover algorithms that are smart enough that actually you don't need the quantum computing anymore, that exponential gain, we can already have it in other ways that, you know, just use classical computers. So we'll see. It's an interesting space.

51:42But I think, you know, for the most, and there's, I mean, there's a lot of, you know, there's papers on like quantum machine learning and blah, blah, blah. I should say there's another way intriguing one in which quantum computing might be relevant to AI, which is there are ideas. This is often how fields of research wind up having an impact is not in what they were trying to do because it failed, but because people came up with ideas that were then useful somewhere else. Right. And it could be that it could very well be that quantum computing comes up with ideas that in the end will actually be useful in machine learning.

52:16You know, there's almost nothing in this world that isn't potentially useful in machine learning. So I could see that happening with quantum computing. But so far, on either that front or the front of practical computers, I haven't really seen anything that I think people in AI need to be paying attention to. Having said that, I've heard rumors that this is what Ulysses Skiver is doing in his new company, is quantum computing for AI. So we'll see. yeah uh you know since you wrote the book uh ai has advanced dramatically and and very quickly and seems to not be slowing down there was talk like two or three months ago that everything was slowing down and you know i don't see it slowing down so um do you do you have in your head kind of a timeline for how close we're getting to a master algorithm uh great question and and you know my answer is we could be almost there or it could be very far away nobody really knows anybody who gives you a precise prediction is making it up or diluting themselves now uh here's why technology progresses in S curves, right?

53:35It's something slow progress and then fast progress, and then it slows down again and it plateaus, right? The early part of an S curve looks mathematically like it is an exponential. But then what people forget is that, you know, the slowdown is coming, right? You have an initial phase of increasing returns, the exciting one, and then a phase of decreasing returns, which is where most technologies are stuck for most of their existence, like cars and planes and TVs and whatnot. not. And now the thing that has happened in the AI in the last 10 years, clearly we've been on that upward curve, right?

54:07And every few years people go like, oh, things are slowing down now. In fact, I forget if you mentioned this already, but like I had this conversation with the illicit skiver at iClear in 2017, where he said like, oh, you know, deep learning is slowing down. Like, you know, there's no more progress. It's like, well, not so fast. Like, you know, new things will come up right and you know a month later the transformers paper came out so you know and there are people today i could actually argue both sides of this so like you could look at today and say like well but things are slowing down in particular the way you see things slowing down is that it takes exponentially more resources to produce the same amount of progress in fact the folks that you know open the end topic is like yeah yeah that's the way it is right you know like you gave me a billion dollars and now you have to give me 10 billion and tomorrow it'll be a trillion you know pony up which to me is alarming.

54:56But by that standard, things are slowing down. But again, that's normal. The question is, will there be a new idea that gives us another boost? For example, LXNet was one such idea. It was really just doing things on GPUs, but that's fair game. And transformers are another idea. And in many ways, you could say there hasn't been a big idea. And GANs maybe were such an idea. Again, it plateaued. But some people say there hasn't been any major progress in AI since Transformers came out, which is almost 10 years ago, or Attention, right? Attention is 10 years old now, right? So who knows, right? It really depends.

55:33This is not preordained, right? It's not like we're on some deterministic curve. It's like, you know, we, the researchers have to, you know, come up with ideas. And if we do, you know, the exponential will keep going until finally it saturates somewhere. The question is, is it about to saturate? Then will it saturate for a year or 10 or 100? My hope, right, uh uh is that no we we if i had to lay down my bets right i said like no we are not about to plateau we are going you know like this fast progress of the last 10 years is going to look slow compared to the next 10 right get your hand around that but this is not going to happen by magic it's really going to require uh major new ideas of which you know with all due respect to the guys doing o1 and deep seek those are not those are tweaks they're nice they're perfectly good work, but those are not the thing that's going to give us, you know, the next phase of the exponential.

56:26Yeah.

56:31Although, you know, and granted, Sam Altman is, you know, he's got a proprietary model that he's trying to protect and generate excitement around. But when you listen to him, it does feel like this reasoning is going to continue to develop. When you listen to Jeff Hinton, I mean, he already thinks these generative pre-trained models are conscious at some level. I mean, it's incredible the stuff he interpolates or sees in these models. And then you have, you know, Rich Sutton's team. I mean, Ilya's team. And all of these people are starting to jump ahead to ASI, to artificial superintelligence.

57:38intelligence, like don't even worry about general intelligence. Well, but I don't know. I'm a journalist and I've, I've seen what the field has done since I started paying attention in 2017 when I first met you. And you know, if we get to super intelligence, then basically we have the master algorithm, right? Well, once we have the master algorithm, we have AGI. I mean, by definition, we have AGI. Otherwise, it's not the master algorithm. And again, as people never tire of pointing out, once we have that, then just scaling it up, we can have 10 or 1 ,000 times or a million times the intelligence of a human being.

58:25Right. So that part is easy. The hard part is getting to that master algorithm. Right. But, you know, like you mentioned people like Sam Altman, Jeff, and et cetera. It's interesting because these are all different cases. Right. and they have different reasons to say what they do. And it's good to understand a little bit of that. Sam Altman is a very smart guy, but he's a VC. He's a great hustler. He's great at raising capital and sniffing out opportunities. He's really good at that, right? And persuading people and so on. But his technical knowledge, I think he himself would admit it, is not very deep, right?

59:02And I remember him saying like in an interview with Hoffman like years ago, Like, yeah, transformers aren't going to do it. He doesn't say that now because it would be inconvenient. But he didn't know what would, he said, like, something else is going to have to come along, but he didn't know what it was. And I think that's true, right? It's just not what you will hear him say today, right? I think it's easy for a lot of people, and I think he partly has fallen into that. You see what these algorithms are doing, and you get very excited and go like, oh, superintelligence is almost here. You have to remember that when you see, I mean, like, this is a lesson that you have to learn in machine learning.

59:36is like the algorithm always seems to be doing a lot more than it really is. You find examples where it's amazing. You're like, wow, you know, super intelligence. But then there's other examples where, you know, it does stupid things that a child wouldn't, right? And that problem is still with us. So you have to not get too carried away with that. There's also the sales pitch, but ignoring that, right? Now, Jeff Hinton, right, is a guy who is perennially over-optimistic about what neural networks can do. and you know more power to him because that's what kept him going for 40 years right whereas others gave up but he has always i think underestimate he has this notion that the like let me put it this way jeff does believe in the mass algorithm as does rich satin in fact i asked a bunch of people at the time i was writing the book and those were the two strongest yeses i got were you know jeff and rich of course they're different for them but i think jeff believes in a no, like what Jeff thinks of as the master algorithm and how the brain works is too simple.

1:00:37He actually, I think Jeff massively underestimates the true complexity of the human brain or even a simplified algorithm that would do the same. So every decade, and in fact, this is a well-known joke in the field, and he will say that himself, you know, in a self-deprecating way. He always thinks that like, yeah, I've just figured out how the brain works. We're on the verge of whatever consciousness, et cetera, et cetera. So in a way, his latest things about like chat GPT being blah, blah, are totally consistent with Jeff, but unfortunately, they're totally consistent with him seeing more than is there and underestimating the, you know, the length and the difficulty of the path to human level intelligence.

1:01:18And Rich Sutton in his own way with reinforcement learning instead of new networks, is also prone to that. I think he's, you know, at this, you know, like Jeff in some ways to oversimplify has been very successful and Rich hasn't. So in a way, Rich has learned the famous, you know, bitter lesson. And I think he's a little bit more, a little wiser now, if you will. But at the end of the day, I mean, I think, you know, let me put it this way. The founding fathers of AI were crazy, right? In the 1950s, we're saying we'll have human level intelligence in 10 years. They're crazy. They were madmen. But thank God for those madmen because they started the field, right?

1:01:59So in a way, it's good to have those madmen and those are optimistic people. But if you were an investor deciding what to invest in, I would take what people like Jeff and Rich and Sam said with a large grain of salt. Yeah. Yeah. Although, you know, you have models now that, well, and they're not single models, they're assemblies of models, but that can handle audio, you know, or speech. they can handle text, they can handle imagery, they can handle whatever other modalities are out there, and they can generate answers in all these different modalities. I mean, that's certainly more general than things were in the 90s or the early 2000s.

1:02:58So there is, I mean, AGI, it's not going to be a moment. There's a spectrum. And it seems to me we're in the early part of that spectrum where, yeah, you have these models that are multimodal and now have a certain amount of reasoning. so if you look at it that way there's a progression and we're moving along that spectrum Absolutely, so let me even make that point more strongly before making some caveats these aren't even assemblies of models you have some of these transformer based models that one model the same model does audio and video and text and speech and all of that, it's remarkable right this really is the closest to the mass rock and we've ever gotten and is a completely different place from where we were in the 90s like it just doesn't compare yeah i mean in the 90s i was doing a phd thesis on the sets with you know 500 examples trying to learn to do medical diagnosis and you know create assignment and things like that we're in a completely different place now right having said all that and you very you know you made the key point there which is agi is not a point is like AGI is, you know, human intelligence is a whole bunch of different abilities.

1:04:18In some of them, computers are already way better than humans. Right. They can add to numbers a lot faster than you like, and they can play chess better. But in other ways, they're still far behind. So there's not going to be a point at which you reach AGI or a point at which you reach super intelligence. You need to think at a final, like, in each of these dimensions, how far along are we? And in some of those dimensions, we are far along. However, and, you know, this is the big caveat, is like, where's my house bot? My hotspot is nowhere in sight. We're nowhere near AGI. You could say, oh, you've reached AGI once you beat humans at every single one of those things.

1:04:54Well, to take the twin to the spectrum, we're nowhere near there on producing Einstein's. We have these systems that know more than anybody ever could, but no Einstein. Or, you know, a MAID, right? MAID is an incredibly sophisticated system that no AI can mimic right now. making the beds, you know, you know, loading the dishwasher. Like we don't have that. We don't have that. And we do not have, I mean, I talk to robotics people, not, I'm not a boss, but like, so, you know, what's happening. I'm like, no one has a, you know, despite what you might hear in the media, you know, like we do not currently have a path to having a house bought in your house anytime in the next, whatever, five or 10 years.

1:05:35Right. Right. Okay. Okay, well, we're up to over an hour. Let's leave it here. And I really enjoyed this series, and I'm hoping listeners have learned a lot. Are you working on a new book? So I do have a couple of books that I want to write and that I'm making notes towards, right? But I did recently publish a book, 2040, A Silicon Valley Satire. The main focus of my work right now is doing research. So I do want to, you know, I have something that I think will make a big difference. And I want to get that ready and release it and then see where that goes. And then after that, I probably will be writing my next book.

1:06:18Create an oasis with Thuma, a modern design company that specializes in furniture and home goods. By stripping away everything but the essential, Thuma makes elevated beds with premium materials and intentional details. I'm in the process of reorganizing my house and I'm giving Thuma a serious look for help in renovating and redesigning. Thuma combines the perfect balance of form, craftsmanship, and functionality. With over 17 ,000 five-star reviews, the Thuma Bed Collection is proof that simplicity is the truest form of sophistication. Using the technique of Japanese joinery, pieces are crafted from solid wood and precision cut for a silent, stable foundation.

1:07:16With clean lines, subtle curves, and minimalist style, the Thuma Bed Collection is available in four signature finishes to match any design aesthetic. Headboard upgrades are available for customization as desired. To get$100 toward your first bed purchase, go to Thuma, that's T-H-U-M-A dot C-O slash IonAI. IonAI all run together, E-Y-E-O-N-A-I. So for$100 off your first purchase, go to thuma.co slash ionai. That's T-H-U-M-A dot C-O slash ionai to receive$100 off your first bed purchase.

From the publisher

This episode is sponsored by Thuma. Thuma is a modern design company that specializes in timeless home essentials that are mindfully made with premium materials and intentional details.

To get $100 towards your first bed purchase, go to http://thuma.co/eyeonai

 

Can AI Ever Reach AGI? Pedro Domingos Explains the Missing Link

In this episode of Eye on AI, renowned computer scientist and author of The Master Algorithm, Pedro Domingos, breaks down what’s still missing in our race toward Artificial General Intelligence (AGI) — and why the path forward requires a radical unification of AI's five foundational paradigms: Symbolists, Connectionists, Bayesians, Evolutionaries, and Analogizers.

 

Topics covered:

  • Why deep learning alone won’t achieve AGI

  • How reasoning by analogy could unlock true machine creativity

  • The role of evolutionary algorithms in building intelligent systems

  • Why transformers like GPT-4 are impressive—but incomplete

  • The danger of hype from tech leaders vs. the real science behind AGI

  • What the Master Algorithm truly means — and why we haven’t found it yet

Pedro argues that creativity is easy, reliability is hard, and that reasoning by analogy — not just scaling LLMs — may be the key to Einstein-level breakthroughs in AI.

 

Whether you're an AI researcher, machine learning engineer, or just curious about the future of artificial intelligence, this is one of the most important conversations on how to actually reach AGI.

 

 

📚 About Pedro Domingos: Pedro is a professor at the University of Washington and author of the bestselling book The Master Algorithm, which explores how the unification of AI's "five tribes" could produce the ultimate learning algorithm.

 

 

Stay Updated:

Craig Smith on X:https://x.com/craigss

Eye on A.I. on X: https://x.com/EyeOn_AI

 

(00:00) The Five Tribes of AI Explained

(02:23) The Origins of The Master Algorithm

(08:22) Designing with Bit Strings: Radios, Robots & More

(10:46) Fitness Functions vs Reward Functions in AI

(15:51) What Is Reasoning by Analogy in AI?

(18:38) Kernel Machines and Support Vector Machines Explained

(22:23) Case-Based Reasoning and Real-World Use Cases

(27:38) Are AI Tribes Still Siloed or Finally Collaborating?

(32:42) Why AI Needs a Deeply Unified Master Algorithm

(36:40) Creativity vs Reliability in AI

(39:14) Can AI Achieve Scientific Breakthroughs?

(41:26) Why Reasoning by Analogy Is AI’s Missing Link

(45:10) Evolutionaries: The Most Distant Tribe in AI

(48:41) Will Quantum Computing Help AI Reach AGI?

(53:15) Are We Close to the Master Algorithm?

(57:44) Tech Leaders, Hype & the Reality of AGI

(01:04:06) The AGI Spectrum: Where We Are & What’s Missing

(01:06:18) Pedro’s Research Focus

 

More from Eye On A.I.

All 266 episodes
#250 Pedro Domingos on the Real Path to AGIEye On A.I. · 1 h 8 min
Listen in VO