#248 Pedro Domingos: How Connectionism Is Reshaping the Future of Machine Learning

29:53It's like, you get this, it's much harder, you know, if you think about it, like, it's not that hard to do a bad job of synthesizing speech from text. You know what the phonemes are and you string them together. It will sound very stilted, but it's not bad. that speech recognition is a hard problem, right? There's like this noise wave coming at you, all garbled, and people eat up syllables, and you've got to turn that into text. That is really hard. So you're right, NetTalks, it was what people talked about back in 1990. But circa 2005, 10, what Jeff and his students were doing was, again, something that the speech people had to take seriously.

30:31Again, I sometimes joke that the purpose of Machine Learning Conference is to publish application papers before they're good for the application conferences. You publish speech there and vision because as tests of the machine learning algorithms, and then when they mature, they go to their natural language, whatever, conferences, which brings us back to Transformers. And it's exactly, if we go back to the point where I said, and then people said, let's do language. Let's pick up the story there. as i mentioned the main guy of that very small group that had always been interested in language but but he even he he did his phd on i think speech and then kind of didn't work on that for a while because again they got stuck for reasons that we can go into um and then came back and his group started looking at machine translation who is that i'm sorry who who was that so so the uh um attention right is the key thing in transformers right and attention it's an interesting story um it was so yoshua banjo said to his to a new postdoc of his kyun hyun cho and and and someone who was an intern here's the best part of this more people don't know this transformers were not invented at google i mean the name was and the tool architecture was but the key id in Transformers is attention.

31:56I mean, the Transformers paper is called attention is all you need. Right. Because actually the main thing they did was take other stuff out, which is a valid contribution, but attention was invented by an intern in Joshua Benji's group. He wasn't any, you know, great machine learning researcher or anything. He was an intern who didn't know anything and just started working on this and he came up with that idea. So what was the problem? What was the solution? Right. He's like to do machine translation, the way they tried it initially was like you would go through for example the translating french to english you would go through the french text with what is called a recurrent neural network that processes things sequentially over time and you would wind up with with the state of the neural network that captured the text that you had read and then you would start from this state with and this was called the encoding part right because they encoded french into the network and then there was the decoding part where you turn that internal state which is a bunch of numbers that nobody understood and still doesn't and turned it into english again sequentially by generating the text on where that time and this didn't work very well there was a bottleneck like like there was too much information lost and they tended to remember the recent text meaning the last parts of the text but not the early one and so yosha said to them like you know you know we got to do something about this right and and what is the attention idea in a way the attention id is really just doing with neural networks, what people have been doing in the statistical approach to machine translation for decades.

33:25There was always this phase called the alignment phase, which has nothing to do with what people call alignment. It's this fact that, for example, when you're translating fresh English, word order is often different. Like the adjectives will come before or after, right? And so it's not enough to say, I'm going to go through the text one at a time. When I'm generating the next word, let's say the next word is the adjective. Like, for example, I saw a big cat. Right. And then like, where does the big come from? Right. You got to go find in the previous text, the word that big is translating. Oh, Le Grand, Cha.

33:59Okay. Right. I already translated Cha, not Le Grand. Whatever. Right. You get the idea. Right. And so what they did was they did the neural network version of the same process. And this is what attention does. is like now that I'm generating the word, you know, like what is my next word is going to be? Well, what is it the translation of? Right. Oh, you know, big, you know, that's big as the translation of God. And then you can learn this using backprop. Right. So that was that was their innovation. Very important one. Right. But they were just trying to solve a machine translation problem. The paper is just about machine translation.

34:35And then the transformer guys, same thing. They, you know, they refined, you know, a bunch of stuff and basically to cut out stuff that no longer was was longer useful and and and then i mean i it's interesting because like i remember reading these papers at the time and each case thinking this is a very cool idea i bet you can do a lot of other things with it besides translation next paper and then what people found and this is partly you know the the beauty of machine learning is like they took the same architecture and started you know doing more and more things with it and keep and it kept working amazingly right and here we are today right chat gpt the t in gpt is transformed yeah yeah uh and then the guys at google there was a team uh that that as you say stripped out everything uh and made a very simple architecture around attention right um i i'm i'm i'm i'm laughing because it's anything but simple.

35:31But I mean, so it wasn't one team. There were several teams, maybe three. That's why the paper has eight authors. There are three different groups of people who, you know, for one reason or another said like, let's try to use this for whatever, right? Or, you know, they had their different concerns, but they were all started using this new mechanism of attention, right? The basic attention is just, you know, a couple of equations. And again, we can try to understand what it's doing in multiple ways, but they weren't making any headway. And then the key guy was actually Noam Shazir. He's one of the eight authors of the paper, but he's really the guy who invented transformers in that sense, if you take attention for granted.

36:11I mean, again, I've talked to several of these people. They just hacked a million things, as often the case, until something worked. There was no great insight there, at least that they can yeah maybe there is but you know but we don't know what it is yet uh probably but you know noam was the guy who who's like figured out how to make it work right and and then and then and then um it you know i was smiling because he you know the transformer so the recurrent neural networks are are one of the problems is that they work sequentially so they They also have to be trained sequentially, which makes it hard to paralyze them.

36:50And paralyzation is very important so that you can scale up to make things run faster. And initially, from the Montreal group, from Benjo's group, they were just doing attention on top of a recurrent neural network. And then what this guy said, very, very important, right? And again, there's a history in machine learning of these things happening. is like actually once you have attention you can take out the recurrence and now you can just train in parallel and now you can train on much bigger corpora this is really the key uh thing that happened in that paper right now this system that they have at the end of the day is anything have and have even more so today is is anything but simple it's a huge pile of different stuff and hacks left and right and and you know this actually slows down progress right is that it's there's nothing simple about the, I mean, things tend to become simple over time as you understand the better.

37:45And I think that's the case with transformers, but they have all these different kinds of layers and mechanisms and whatnot, which we could go into, but like a transformer, the full architecture is a, is a big pile of complication. And, you know, that, that holds us back. Yeah. How does, I don't mean to jump around, but as I mentioned, and Sap Hochrider, who had developed LSTMs much earlier. How does that relate to attention? Because it's looking back a certain number of tokens and holding them in memory. And that, to me, sounds like attention. Right. So LSTMs were very hot in machine learning for language at one point.

38:43And that was the point at which people are already focusing on language, but attention had not been invented yet. And LSTMs are a type of recurrent neural network. The problem with recurrent neural network that really, for example, made Yoshio Benio stop working on was what is called the vanishing gradients problem. Right. You think about it, when you do back propagation, the further back you go, the more diffused the signal becomes. It's like the neurons at the output layer, yeah, you are to blame. You're clearly wrong, right? Once you get to a neuron like, you know, 100 layers behind, right, it's like there's a million paths to that neuron.

39:20Like, who knows if it's to blame or not, right? It's like if you think of like a drop of ink, you know, diffusing through, you know, a glass of water, right? As time passes, it becomes spread over everything. And if this is the credit or blame that you have to assign, right? Like a recurrent neural network over time, right? Even if it's just one layer, because you use it over and over again, it becomes, you know, an infinite, you know, layer network. And the signal just diffuses through it until it's like, you know, the ink all over the water. You don't, there's no, you can't learn. And there's nothing to learn.

39:52Everybody's equally to blame or not to blame, right? And LSTMs were a solution to this problem. And the solution to this problem, the LSTM, And, you know, people tried a bunch of things at the time, neural Turing machines and blah, blah, blah. But LSDMs, again, which had been invented way before by Sepp and his advisor, Jürgen Schmidhuber, notorious in machine learning for various reasons, their solution was actually to do something that works a little bit like a computer. It has these memory cells, and you can actually write stuff into the memory cell, and then it stays there until you decide to access the cell, which is really like a von Neumann computer that we all use works.

40:32And then what they had was these gates that controlled access to the memory. There was like the forget gate, et cetera. And what it did was it allowed you or didn't allow you to, you know, write into the memory, read the memory, et cetera, et cetera. And again, the key was that there was, this wasn't digital, right? It was something that you could gradually learn, right? Which again is what attention does later. It's like, it doesn't just say like, oh, I'm going to pay attention to this. it's paying attention to all the past, you know, to what is called the context window. And this is like, well, the backprop says pay more attention to this guy.

41:05And then the backprop says pay even more attention to this guy, right? And LSTM was honestly a more complicated than hack here and more unwieldy way to accomplish this effect. Attention in many ways is a misleading term, right? Because it really doesn't have anything to do with the psychological version of that term. In fact, the irony is that Kyung Hoon and Dima Bada, now Dima is the intern who invented it, they didn't call it attention. Yoshua Ben-joo came at the end of the paper and put attention everywhere, and then they took it out again, as they tell the story, except they left it in a bunch of places, and then the rest is history.

41:46So then you've got the Transformers. Again, this is all in the Connectionist stream. And then there's some things happening off to the side. there's or maybe we're going to talk about this next in the next episode but there's generative adversarial networks and well let's talk about GANs or is that on the evolutionary algorithm end of things no very good so GANs generative adversarial networks were very popular for a few years in AI. And now it's kind of died down. And it's shrewd of you to point out their connection to evolutionary learning. Most people don't make that connection, but I think it is a very important connection.

42:48And GANs are actually how the term generative came. So the term generative from the beginnings is a term from statistical machine learning. And the idea of a generative model is that it's a model that generates the data. It's like, I want a process by which you generate samples. And then, so let me turn this the other way around. This is actually the whole Bayesian way of thinking is, I see a bunch of data. I'm going to postulate that they were generated by some process, and I'm going to try to identify what that process was. This is actually what generative learning is. So you come up with this model like a Bayesian network.

43:27This is like, well, this is how you generate patients and their symptoms. First, you decide which disease they have. And then you decide which symptoms they have, what fever are they going to have based on the fact that they have COVID, et cetera, et cetera. You generate the data. And this is why it's called the generative model in opposition to discriminative models, which is what most of machine learning is about, which is to just go say like, OK, I'm going to go forward from this data and try to decide, you know, what you suffer from. Or whether it's a cat or a dog, as opposed to like, how do I generate, you know, a cat or a dog?

43:56Right. So there's always been this thing in machine learning between generative and discriminative. discriminative always has always dominated and really still does because it just works. It's an easier problem and it works better. It's in some sense less satisfying because like, what is the deep science here? I'm just hacking to get the result. I want to know, you know, like, for example, physics, right? Is about generative models, right? Like here's how the universe is generated. So there's a certain type of person who really likes, you know, who thinks generative models should win at the end of the day.

44:29Now, you know, young Goodfellow, who's the guy who invented GANs, was another student at the same time in Yoshio Benjo's group, who had a huge group. And so it's not entirely an accident that so many people came out of it. And they were actually trying to solve a different problem, which was the problem. There's a type of neural network, which again, is a generative model, right? Not trained by backprop. And by the way, these were the ones that jeff hinton was always into jeff hinton even though he's a co-author on the paper never liked backprop he's actually on record they're saying that backprop is not the future he actually told me at one point the only reason i i publish papers on this primitive i wrote the only reason i do this community learning is in order to publish the papers because he was what was written was generative neural networks that actually generate you know stuff right but the problem with those new networks is that the inference is intractable and so they were trying to come up with a way to make it tractable right there's a long history of trying to do this in various quadrants of course including the bayans and whatnot and and and generative adversarial networks were actually a way to do this and you know without going into too much of the of the history or the details what is the idea is that the adversarial network is a game between the generative network that generates data and the discriminative network that tries to classify it.

45:53So actually have a generative model, the classifier working against each other, which is a brilliant idea because the idea is that the classifier is trying to distinguish between real data and the data that was created by the generative network. And so that forces the generative network to get better. Otherwise, in the beginning, it's basically like, ha ha ha you don't look anything like a cat falls right and then it gets better but then that forces the discriminator to get better as well so they get into this evolutionary arms race right it's co-evolution is the term in evolution right it's like the predator and the prey that generated in the discriminator and this was like a genuinely new idea in the context of neural networks in fact at the time you know yan lakun was going around saying like this is the most important idea in neural networks of the last 20 years.

46:46Then it petered out because it turned out it's very unstable. There's a lot that goes wrong. People never really, to my disappointment at least, you want to take that further into the whole evolutionary realm of things because co-evolution is very powerful. Maybe that's what we need in machine learning, but it was too hard to cut a long story short. It petered out, but that's where this whole thing of generating deep fakes began. right? Yin in his paper had this few, you know, grainy images that were generated, right? They were like, I mean, at the time, this was actually very impressive, right?

47:21But then like, if you think about applying more power to this, the images get bigger, they get, you know, final resolution, you go from images to video, and the whole, you know, deep X, the industry was born. So even though now there are the techniques like, you know, diffusion and whatnot, which we could also talk about being used to generate these things. So GANs are no longer the state of the art, but the whole, like that term generative in the neural network context really took off there. And this whole notion, which a lot of AI things gave you, but they're like, you know, how can I dazzle you with generating text or generating images really started with GANs?

47:58Yeah. Where should we go from here? We don't have a lot of time left. I wanted to, I don't know if it needs a big chunk of time on its own, but then you have Rich Sutton working on reinforcement learning, sort of coming at it from the behavioral psychologist side. And that, to me, is probably the most powerful idea. and it was certainly applied, but it seems to have become the darling of generative AI right now with this DeepSeek is training with pure RL. Can you talk about how RL fits into all of this? Absolutely. So reinforcement learning is actually different. So there's supervised learning, unsupervised learning, reinforcement learning.

49:01It's a different type of learning that, again, has been around for decades. It is inspired by animal psychology. In fact, that's where the term comes from. Like, you know, Pavlovian experiments, you reinforce behaviors by giving rewards and et cetera. And, you know, Rich Sutton has been the longstanding leader in that area. So what is the basic idea in reinforcement learning? It's important to start by understanding that. And by the way, reinforcement learning, like supervised learning, can be applied with any of the paradigms, but it has always been most popular with neural networks. Yeah. So with supervised learning, you have a teacher.

49:39The teacher says, aha, right answer. Like, you know, yes, this is a cat or no, this is a dog. Like you label things with the right answer. Yeah. Which makes learning easier, but is, you know, where does that data come from? There are many domains where you don't have that. If you look at how animals learn, right, or like how we learn, right, you touch the stove and then you learn not to. But the interesting thing about enforcement learning is that I feel the burn when I touch the stove, but I really shouldn't even have moved my hand towards it. So you need to back up that signal.

50:16So supervised learning, if it was just like at the last moment, don't touch it, that would be supervised learning. The whole idea of reinforcement learning is they have this thing called delayed rewards. You do something and you only see the result later, and then you need to propagate those results back to where you should have taken the actions. A famous example of this is AlphaGo. Yeah. And indeed, reinforcement learning goes back to the 50s, and the paper that coined it to machine learning had a precursor of reinforcement learning to learn to play checkers, and they learned to play that at human level.

50:46And the idea is this is like I play a whole game of checkers, either computer with you, the human. I don't know if my moves are good or bad. The only thing I know is at the end of the day, I won or I lost or we tied. Right. So there's a very small piece of reward, as it's called. Reward is like you won. Right. You get a you know, you get a reward. Right. You got to an enforcement learning is a set of techniques or really it's just the problem. Reinforcement learning is the problem of like, how can I take those rewards and propagate them back to where I need to make a decision? so that I make the right move that many steps further will lead me to win the game.

51:22And this combined with neural networks for vision applied to the Go board was, of course, what let DeepMind beat Lisa Dahl and et cetera. Now, reinforcement learning as done by Rich Sutton and other people for a long time was always very theoretical. They wanted to, they did these tiny little experiments that were, you know, not convincing to anybody, honestly. And they tried to prove theorems. What the deep mind folks did when they came along that was very important was they said, like, you know, to hell with all that. We're just going to engineer the heck out of this and get it to win. And they did, right?

52:03And so, again, for a while, deep reinforcement learning was very hot. There were, like, many papers about it. They actually started by doing it in video games, in Atari, and then went on to things like going chess. That, unfortunately, has petered out also because it didn't, DeepMind's agenda when they came out was deep reinforcement learning. And their idea was like, we're going to start with these games and then we're going to move to robots in simulation and finally robots in the world. And this is how we're going to solve AI. Right. Them is as obvious as we're saying, like, where are the Apollo Project AI?

52:33We're going to get there. They don't say that anymore. And so these days, reinforcement learning, as you pointed out, is, again, popular. But honestly, those of us who know this history at this point have some justified skepticism because we keep seeing this over and over again, which is people come out saying like, oh, look at this great success for reinforcement learning. And then when you look at it more carefully, there was no great success. What they were doing was effectively supervised learning or could have been done by supervised learning. In fact, there's this thing called reinforcement learning from human feedback that OpenAI popularized that they say is very important to make their chatbots work and whatnot.

53:11There was this paper out of Stanford that basically showed effectively you're just doing supervised learning. So I would say the same with DeepSeek. Until I see proof to the contrary, I will assume, and again, the evidence points in that direction, at least as far as I can tell, that there's really no reinforcement learning magic going on. The problem with reinforcement, so the appeal of reinforcement learning to the people who really want to solve AI is that like, clearly we do something like that as even a correspondence to some of the neuroscience with dopamine circuits and whatnot. The problem is that people have, disappointingly, despite so much effort, they have never really truly managed to make it work.

53:48In fact, it boils down to this. If your rewards are delayed, it's learning from sparse delayed rewards. If the rewards are not very delayed, they're not very sparse, then you can just use supervised learning, which is way easier. If the words really are delayed and sparse, then it doesn't work. So, so far we haven't really solved that problem. So when you see claim in the meantime, however, in the last few years, reinforcement learning has just become the sexy term that you can throw around and you call your thing reinforcement learning. And it sounds more impressive, even to yourself, which seems to be also what had happened with DeepSeq.

54:27Yeah. Okay. We have a couple of minutes left. Where should we go from here? Well, let me take that question the following way. Where should the neural network community go from here? And I think the high order bit is, as we alluded to earlier, what you've seen is broadly from the point of view of your app, but even from the point of view of machine learning and then deep learning, what we're seeing is more and more people doing more and more research about less and less. So I think where we really need to go from here is we need to broaden the scope. By the way, as an example, there are papers now showing that maybe LSTMs and recurrent networks aren't such a bad idea.

55:10A lot of things that seem dead in machine learning or AI have a tendency to come back, you know, neural networks being one of them. So I wouldn't be surprised at all if one of these things has dethroned transformers by next year. Or maybe it'll stagnate for 10 years or who knows, right? But I think a lot of people should be doing a lot. We really should let a thousand flowers bloom. And right now, it's basically one flower blooming or one petal of one flower blooming. And that's not very healthy. Yeah. And that phenomenon, because we've talked about it during the supervised learning vision phase, when everyone is focused on one narrow aspect and then they're forgetting about the Bayesians or something.

55:56That's a matter of funding, isn't it? Oh, funding is a big part of it. In fact, Minsky and Papert publicly admitted that the main reason they wrote that book that killed neural networks for a while was that DARPA was the big funder of AI research, so historically the biggest funder. When we finally get to AI, we will have DARPA to thank as much as anybody for having stuck with it until the industry took over. But they were getting jealous that neural networks were so sexy and they were getting all the DARPA funding and symbolic AI wasn't getting enough. So they said, we got to get rid of these guys.

56:31And so they did. So funding has always been an issue. I mean, particularly in research and industry as well, the phenomenon is a little different. DARPA for a long time after that, In fact, when I became a researcher, DARPA did not fund much machine learning research, if any, because the consensus, which in the field and among DARPA program, there is like machine learning is a waste of time. In the meantime, machine learning was taking off and DARPA was still not waking up. Finally they woke up in the 2000s and started getting into machine learning. And again, the funders themselves, whether in industry or in academia or the funding agencies, they're very prone to this hurt behavior.

57:14It's like the VCs, right? VCs are supposed to be looking out for different things, but at the end of the day, they're all looking out for the same things. So, you know, you need to inject some, you know, there's this, you know, gradient descent, right? There's like looking for the optimist, what neural networks do. But there's this method called simulated annealing where you actually inject noise, which seems like a bad thing, but the noise actually helps you not go straight to the local minimum and find a deeper one. We need more simulated than kneeling in AI research. When you need somebody, you need somebody.

57:47You want to hire people fast. The longer you wait, the more the market drifts, the more your needs drift. For example, you just realized your business needed to hire somebody yesterday. How can you find the right candidate fast? Just use Indeed. When it comes to hiring, Indeed is all you need. Stop struggling to get your job posts seen on other job sites. Indeed's sponsored jobs help you stand out and hire fast. With sponsored jobs, your job postings jump to the top of the page for your relevant candidates so you can reach the people you want faster. And it makes a huge difference. According to Indeed data, sponsored jobs posted directly on Indeed have 45 % more applications than non-sponsored jobs.

58:37One of the things I love about Indeed is that it makes hiring so fast. I've searched for candidates before, and it can take months, particularly if you're just working through word of mouth. In those days, I didn't have Indeed. Plus, with Indeed sponsored jobs, there's no monthly subscriptions, no long-term contracts, and you only pay for results. How fast is Indeed? In the minute I've been talking to you, 23 hires were made on Indeed, according to Indeed data, worldwide. So there's no need to wait any longer. Speed up your hiring right now with Indeed, and listeners on this show will get a$75 sponsored job credit to get their jobs more visibility at indeed.com slash IonAI.

59:26Just go to indeed.com slash IonAI. That's indeed, I-N-D-E-D dot com slash IonAI. All run together, E-Y-E-O-N-A-I. Go right now and support our show by saying you heard about Indeed on this podcast. Indeed.com slash IonAI. Terms and conditions apply. Hiring indeed is all you need.