Are Large Language Models a Path to AGI? with Ben Goertzel - #625

0:07All right, everyone, welcome to another episode of the TwiML AI podcast. I'm your host, Sam Charrington, and today I'm joined by Ben Gertzel. Ben is the CEO of SingularityNet. Before we get going, be sure to take a moment to hit that subscribe button wherever you're listening to today's show. Ben, welcome to the podcast. Hey, thanks. It's a pleasure to be here. It's great to have you on the show. I'm looking forward to digging into our conversation. We'll be touching on a number of topics, including your work around artificial general intelligence and top of mind for many of us, the intersection between that and large language models.

0:45Before we do, I'd love to have you share a little bit about your background and how you came to the field. Sure thing. So I have a PhD in math from the late 1980s, but I've been interested in AI and AGI and related topics since really the early 70s when I first encountered it in Star Trek in the space 1999 and so forth. We passed through 1999 and didn't get to human level AI yet, but it seems we're close now, right? After 10 years in academia, teaching math, computer science, and cognitive science, I've been in the software industry, building AI systems since the late 90s, both research toward artificial general intelligence, real thinking machines and applied AI across quite a variety of vertical markets from finance to biology to language processing and robotics.

1:37I did the software and AI behind the Sophia Humanoid Robot as we became the first robot citizens. You know, playing around with AI of various sorts for a long time, but now is by far the most exciting time to be doing it. It keeps getting better and better. It is a super exciting time to be in the field. You mentioned that we're getting close. I'd love to have you riff on that a little bit. And in particular, what does it mean to get close to AGI? How do we know when we're getting close to AGI? We talk about tools like the Turing test, but that doesn't seem to be a sufficient benchmark for measuring artificial general intelligence.

2:15How do you think about defining AGI and how close we get to it? Yeah, first of all, I think having a rigorous definition of AGI is not that important any more than biologists need a rigorous definition of life to work on synthetic biology or analyze viruses and whatnot. There is a strong mathematical theory of AGI. You can look at the book Universal AI by Marcus Hutter, who's now at Google DeepMind. But one of the lessons of this theory is humans are not that general in the scope of all possible general intelligences. If you think about full-on AGI is the ability to, let's say, achieve any computable reward function in any computable environment as one overly simplistic definition, which Marcus puts forward.

3:03I mean, we're very far from that. I'm a fairly smart human being. I cannot optimize an arbitrary computable reward function. I can barely run a maze in two dimensions, let alone 755 dimensions, right? So, I mean, what we mean in practice by AGI is having a decent amount of ability to generalize and to extrapolate and creatively leap beyond one's programming and one's training, like roughly as much ability to creatively leap beyond one's background as human beings have, right? And that's sort of pragmatically what we mean by it. And I think we're clearly not there yet with any of the AI systems that we've created so far.

3:42On the other hand, progress in that direction is very, very interesting at the moment, right? And the opposite of AGI, I think of as narrow AI, which are AIs that do sort of one specific thing that you've configured them to do, which could be drive a car, answer questions based on a certain knowledge base, play a board game, and so forth. But it's not entirely clear, even to the greatest experts in the world, which of the things that humans do can be solved really well by a narrow AI approach versus which ones are what we'd call AGI hard and really need something with human level general intelligence.

4:21I mean, that's something on which people have made the wrong judgment over and over again through the decades the AI field has been around. Do you differentiate AGI and sentience that comes up often in that context? Most recently, the whole thing around Google's Lambda, LLM. And how do you think about the distinction between those ideas? Well, I think AGI is at least a relatively well-defined quantity. You have Marcus Hutter and that whole theory of AGI is maximizing computable rewards and computable environments. And you have Weaver, David Weinbaum's theory of open-ended intelligence, where general intelligence is about a complex system maintaining its boundaries and then seeking to transcend itself and achieve new functions.

5:07I mean, at least there's not an agreement on what it is exactly, but there are formal mathematical approaches to it. I'd say sentience is meaningful, but remains a bit fuzzier. Like what's the relation between sentience and sapience and intelligence and consciousness that's really more fully in the domain of philosophy, I would say. So, I mean, on the question of machine consciousness, there's a lot of different philosophies out there. I tend to be pen psychist and, you know, believe that this ballpoint pen has its own species of consciousness, which is not as complex or dynamic or richly structured as human consciousness.

5:47And my own feeling is when you get an AGI system carrying out human-like cognition and intelligence, it's basically going to have human-like conscious experience. But then to nail down what that means and argue for it scientifically remains somewhat of a vexed question, right? Now, a concept like sentience or sapience, it's a weird blend of consciousness and intelligence, which isn't all that well defined. Now, when it was claimed that Google Lambda was sentient, what was meant wasn't really just that it was giving smart answers. What was meant was that it seemed to be responding holistically and emotionally to what was going on.

6:29Like when you asked it questions that made it nervous, when you asked it questions that were sort of maybe hitting some filter or something and it wasn't sure if it was allowed to answer, then it would display sort of textual patterns like hamming and hawing and so on that were consistent with nervousness, right? So it appeared to the guy interacting with it to be giving some indication of a holistic emotional response to the sort of tenor of the overall dialogue it was in. And you've seen similar phenomena like that in people's chats with Microsoft's like Bing chatbot with Bing on chat GPT, right?

7:05It's not just in it saying smart things, it seems to be responding emotionally as a whole system to the overall like tenor and context that it's involved in. And that's interesting. It's kind of a slippery and not terribly well-defined thing at the moment. It's not really something that drives my research at the moment, but it's interesting. Thinking about getting closer to general intelligence, to what degree does the recent progress with large language models play into that? Do you see that as evidence of getting closer or is it ancillary too? I think somewhere in between the two. I don't think that current large language models display a heck of a lot of general intelligence.

7:54On the other hand, there's certainly evidence that we've come a long way in the AI field generally. And I think they can be used in various ways to accelerate progress toward general intelligence. And I will talk a bit about that. If you look at a system like chat GPT or Google Lambda and so forth, these achieve what seems like a lot of generality relative to the human scale of understanding. But they achieve it basically by having a very, very general training data set, right? Like the training data set is the whole damn web, right? So if everything in the web is in your training data set, you achieve a certain generality just by doing fancy nearest neighbor matching against the whole web.

8:38Because almost anything that people throw at the system has some near matches somewhere on the web. So it can throw back stuff, which is minor variations and combinations once in its training data set. It's achieving what's a lot of generality relative to any one human without a heck of a lot of ability to generalize. right? That's a weird thing for us to wrap our brains around because we don't have a great intuition for what it means to have the whole web at the fingertips of your memory. I mean, just as we don't have a good intuition for the difference between a sextillion or a septillion or something, right?

9:12I mean, there's some things we can do mathematically and on the computer that bollocks our intuition. Now, on the other side, I'd say some of the results with few-shot learning on LLMs do demonstrate an intriguing ability to generalize, but yet there are severe limitations. These few-shot learning techniques on top of LLMs don't deal well with negation, let alone like nested negations in quantifier bindings or something like you find in a math group for a scientific hypothesis. There is some ability to generalize, which you can see in few-shot learning on LLMs. It is real progress. It also has fairly strict limitations, right?

9:55So I could see why some people would think these LLMs are on the path to AGI. I mean, there's a sort of naive dumbass reason for thinking that, which is, wow, they can answer a lot of questions. And then there's a subtler reason for thinking that, which is, I mean, few-shot learning, which is just learning in modulating the activation pattern in the network without updating the weights is able to make some leaps beyond what was in the training data, even if not really, really big leaps. That's an interesting phenomenon that's not fully understood, right? I don't think that transformer neural nets, in the end, represent knowledge in the sort of way that you need for human level AGI.

10:36I mean, I think even though these are deep networks in the sense of having multiple layers, I think the knowledge representation is far too shallow and surface level and will not be able to make leaps of innovation beyond training data the way that people can. So as one sort of illustrative example, you know, if you're using a model like MusicLM to generate music, you can imagine training a MusicLM model on all music up to 1900 or something and no music from after 1900. then give it prompts asking it to put western church music together with their west african polyrhythms what's it going to spit out it's not going to invent jazz right i mean which in a way comes from putting together west african polyrhythms with western church music and its chordal structure i mean it's going to at best give you mozart with a west african beat or something it's not representing these things deeply enough to make a really major creative leap like the invention of jazz or the invention of modern poetry or something or invention of quantum mechanics, right?

11:41So there's clearly a limit of creativity and generalization the human mind can do. And these models just aren't representing things in an abstract enough way to do. Now, from a practical standpoint, it might be that 95 % of human jobs don't need this sort of radical creativity. I mean, not many people invent jazz or quantum mechanics in their spare time, right? So, I mean, it could be one lesson of this sort of AI is almost all the things that people do for a living just require sort of shallow repermutation of stuff that was done already. And then you can obsolete almost all human jobs without making an AGI breakthrough just by fancy K-nearest neighbor matching against everything that's been done.

12:22And that could be huge economically and in a humanitarian sense without yet making the leap to something with human level ability, generalization and imagination. So, I mean, I think there could be multiple approaches to really get to AGI. I mean, you could simulate the human brain at a much more thorough level, including recurrent neural networks and glial cells and, you know, charge diffusion through the extracellular matrix, all sorts of stuff in the brain that are not informal neural net models. I'm more interested personally right now in hybrid systems that put together neural net symbolic logic systems and evolutionary learning systems, sort of neural symbolic evolutionary systems.

13:03And I think the symbolic aspect has a capability for abstraction we don't see in current neural nets. I think evolutionary algorithms have a capability for creativity we don't see in current neural nets. And hybridizing these together within an appropriate sort of synthetic architecture could be interesting. But if you do take that sort of approach, it could be that LLMs are a powerful accelerator, right? So one thing I'm working on now is using systems like OpenAI Codex or Salesforce's CodeGen, using LLMs trained on code to make models that translate English sentences into predicate logic and that term logic and other forms of logic.

13:43So if you can basically use LLMs to translate the whole web into structured logical statements, feed these in to a logical theorem prover in a neural symbolic AI system, then you may have something with the same scope of knowledge as an LLM, but much less susceptible to hallucination and bullshit because it can do inductive, deductive and abductive reasoning and can combine different pieces of knowledge together. So, I mean, that's one example of how LLMs could be used as a tool to accelerate a somewhat different approach to AGI. And, of course, if my team in SingularityNet and the OpenCog project is thinking in that direction, there's going to be other teams.

14:23There will be others. There's other teams also doing that in their own ways, not just thinking, how do we double, triple, quadruple down on GBT-type models, but thinking, how do we use the power that's here to help accelerate other approaches to AGI? So in that sense, they may be significant on the path to AGI, even though what they're doing now isn't exactly AGI in the sense of the ability to generalize. Yeah. So let me ask, you kind of pose this kind of limitation of LLMs with regard to creativity, in a sense, to put it, their inability to create jazz based on their training data. Have there been attempts to formalize that notion, the limitations of LLMs with regard to generalization?

15:04Well, the limitations with regard to creativity are particularly hard to formalize. Of course, there are attempts to mathematically quantify what's a creative leap. I mean, that's not refined in area of study, right? And the limitations of LLMs at generalization, I'm not aware of any really sort of knockdown rigorous study of this. there's certainly a lot of papers giving case studies and examples showing many many cases where llms fail to generalize in pretty simple ways i mean so there was a recent paper that got a lot of attention on social media and mainstream media on like large language models can do theory of mind right what theory of mind is the phenomenon in cognitive science where like i have a sense of what you know, I may know what you can see, even if it's different than what I can see, I know that right now, I see, while recording this podcast, I see your head in a little box on my laptop, and I see a robot behind the laptop, but I know that you cannot see that robot, which is behind my laptop, you just see me, right?

16:15This theory of mind originates in human babies during the first year of life at various points during their maturation. So you can give a large language model various puzzles regarding theory of mind. Like, you know, two people are in a room together playing with a toy, Bob and Jill. Then Bob walks out the door into the next room and shuts the door and scratches his head. Can Jill, who was in the room with him beforehand, can she see him scratching his head, right? And there was a paper published showing that large language models are able to solve some problems regarding theory of mind. And then when you investigate further, you see it does really well at solving problems on theory of mind that were in psychology papers before.

17:01Because there's so many write-ups about the exact solution to that problem. But it can solve some puzzles that don't seem to have been any exact psychology paper before. But if you deviate too far from the theory of mind puzzles that have been discussed in psychology literature, then it will screw up, right? It's a bit subtle. Like it's not just spitting out only the examples that you fed to it. It is abstracting to some degree, but it's not abstracting to the level of a two-year-old child who can deal with examples that are more divergent from the cases they've seen in the past. So how to rigorously measure these limitations is interesting, and I'm sure a bunch of academics are working on it.

17:44From my own standpoint as a researcher, I can see it's limited from playing with it. I'm more interested in working on building AGI systems and in rigorously documenting the shortcomings of systems with qualitatively obvious shortcomings. That's totally fair. I think where the question comes from is you hear, I hear casual dismissal of LLMs is interesting. They're blurry JPEGs. They just are a poor copy of things that already exist. But there's an obviously creative element to what they're doing. They are the poem that it created about your mom, you know, flying a dragon didn't exist anywhere on the Internet.

18:26There's a degree of creativity there. In your example, where you are kind of combining previous types of music, maybe that's an LLM that to which some randomness is injected and it's is creating more things. So how do we like formalize these ideas to have? I'm using LLMs for music generation. And it is interesting. and I haven't thought about formalizing the limitations that much. But I can say, like, I don't have a model as good as MusicLM yet, but I have some models similar to that, trained on smaller amounts of data. And, I mean, you can say, like, play me a passage of a song with a reggae beat, but with a rhythm guitar section that sounds a bit like Master of Puppets from Metallica and a female vocalist, and it will do that, right?

19:13And that's really cool. On the other hand, it sounds like how a bunch of fairly uninspired studio musicians who are very talented would do that. It doesn't sound amazing, right? But it's remarkably competent at putting the pieces together. This is my own qualitative evaluation as a musician, though. One thing that interests me there is could we improve the creativity of output from music LM type models by using like beam search and making up some mathematical measure of what's interesting. Like you use mutual information or relative entropy, use some information theoretic surprising this measure.

19:49Yeah, yeah. Specifically try to milk a continuation out of Music LM, which is a valid continuation with reasonably high probability, but also has a high level of surprising this value, right? So I would like to milk as surprising things I can out of these models, because for the band I play keyboard in, I would like to be able to play a riff, an AI model, respond and play back a riff that incorporates what I played, but with an interesting twist on it. Right now, what it will do, it can play back what you played, but it will homogenize it and make it sound kind of boring and typical for the genre, which is not what I really want as a musician trying to make interesting music.

20:30I think you'd see the same thing in every domain. Like I talked to a friend of mine is the editor of Cointelegraph, which is a crypto magazine, and I'm an online crypto zine. And they evaluated using ChatGPT and other systems to write articles in Cointelegraph. And they finally decided, like, these are just really boring articles, right? I mean, it's more like if you got middle school students to write articles for your magazine. I mean, it's competent. It starts with a summary, tells you the thing. it it gives you another summary at the end it raises a few key points so i mean at first blush like wow it's amazing that i can do this on the other hand it's also clear the articles are more boring than the average article in coin telegraph which is is a decent online journal but is not like atlantic monthly either right like that's not the absolute highest end so i would say in that case, it's not a mathematical measure, but in that case, I mean, you could take the music that comes out of a music LM type model or the blockchain articles that come out of ChatGPT.

21:36I mean, you could submit those articles to be evaluated by human magazine editors or by the folks who audition students for the Berkeley School of Music or something. And it's pretty clear what you would find is they would find these things are competent, but mediocre, right? I think that experiment is underway with a certain science fiction magazine. Yeah. I mean, CNET, various other outlets have run articles created by ChatGPT, and it makes sense, because if you're doing a product review, I mean, that's a... Pretty mechanical. It's a boring kind of article, right? I mean, no, it doesn't need that much drama to it.

22:13But then much simpler language generation algorithms have been used for a long time to summarize sports scores and weather reports. So what we're doing now is a leap beyond that, but it's not a leap to the level of the average journalist, let alone to Mark Twain or James Joyce, if you want to position it. And how to measure that in a mathematical way, as opposed to a sort of human and qualitative way. So you posit that our path to AGI is likely some hybrid of neural approaches, symbolic approaches. I think there's multiple possible paths to AGI. I'm really interested in neuroscience. I spent some of my life doing computational neuroscience.

22:54I think there's a path to AGI where you really do a nonlinear dynamics simulation of the brain. You've got amazing chaotic dynamics in the brain. You've got EEG waves that are generated by glia and neurons and weird dynamics in the water mega molecules in the brain. There's amazing stuff happening in the brain. I think we need a revolution in brain imaging before we can measure enough about the brain to really do that right. But on the other hand, people aren't trying very hard, right? Like no one has tried to take, say, an Ishekevich neuron, like the best nonlinear dynamical model of the neuron, the best models of glia and astrocytes, model all the, you know, 350 major networks known in the human brain and try to put together a real large-scale brain simulation, doing something intelligent attached to a robot's body, right?

23:47Like that's a doable thing. It could be done for no more money than, you know, Google, Microsoft, Facebook, Tedson or something spend on AI. I mean, no one is doing that. I'd hoped Henry Markham would do that with the Human Brain Project, but that wound up sort of fragmenting and going in different directions. I think that's really interesting. It's not the approach I'm now taking. I think that a sort of deep mathematical fusion of methods from probabilistic programming, neural net learning, evolutionary program learning, and probabilistic logic, I think this can probably get us there faster. so I mean the casual one look at that is you're making a hybrid system where you're taking a deep neural net for large scale pattern recognition and synthesis you're taking a symbolic logic engine for inductive, abductive, deductive reasoning you're taking an evolutionary programming engine to create new things and you're sort of glomming them together I mean in reality we're not building like a modular system with a neural evolutionary and symbolic box I mean we've actually done a whole bunch of mathematics to put neural probabilistic and evolutionary systems in a common mathematical framework using higher order probability distributions and the homotopy type theory and intuitionistic logic and the Galois connection.

25:09So a whole bunch of math voodoo that we can't go into in the time we have here. But we're really trying to find a mathematical level that shows that neural symbolic and evolutionary are all just different aspects of the same underlying methods for probabilistic search of program space, right? And so, I mean, key there are mathematical results like Curry-Howard correspondences that show there's an equivalence between logic systems, programs, and categorical abstraction. So really, when you're doing a logical theorem proof, you're also learning a program, and you're also learning an ontology that could be represented in category theory.

25:47Like these AI paradigms that historically have seemed very different, like symbolic versus neural or learning versus reasoning. Actually, if you dig into the math behind them, there's morphisms between them all. They're all kinds of different ways of looking at the same thing. On the surface, it's a hybridization of methods from different AI paradigms. We're really trying to make a common math framework that leads to all of them. And then trying to figure out how to deploy this at large scale, which is a very interesting software challenge, right? Or even small scale. How close have you gotten to a concrete demonstration of this kind of approach?

26:24Well, so we had a system called OpenCog that we launched in 2008 with some code going back to 2001. And we were doing various experiments on this sort of AI. I mean, we published a bunch of journal papers and we used this on the back end of commercial AI systems in financial prediction and functional genomics and so forth. These are more narrow AI applications using a framework that was being developed with AGI in mind. What we're doing now, we're basically rebuilding the whole infrastructure of OpenCog systems. We have a new framework called OpenCog Hyperon, and we're trying to build a framework where we can do this sort of AI at the large scale because what we found with the old version of OpenCog is to really attempt AGI-like things with the system.

27:10We needed quite large knowledge bases and our system was just too slow. The underlying architecture, we have this large knowledge graph or more properly metagraph, which is a more general graph-like object than a graph. And we have this big knowledge metagraph living in RAM across multiple machines and then distributed across multiple machines using processing technology and you need to get metagraphs with many billions of nodes in them and then your neural evolutionary and logical ai algorithms they're actually little programs represent as nodes and links in that same metagraph so you have this big like self-modifying self-reprogramming knowledge metagraph we implemented that in open cog and what we found is to try to do really serious things with it it just is like 10 000 times slower than tensorflow or don't think it's terrible so it's a very high level of abstraction but i was i was reminded how in the 90s you know in the 90s i was teaching neural nets in the university and you were doing recurrent backpropagation you had a neural net with like 40 nodes it would take three hours to train right so you could do a few things now obviously things are scaled up tremendously because we have gpus and all these libraries bottoming out right so i think with the new programming language we built in OpenCog Hyperon, which is MetaTypeTalk, and the new distributed knowledge graph that we built.

28:34Our hope is to be able to scale up all of our earlier experiments. And so it's been a bit painful to take a year out of doing AI R &D to build scalable infrastructure, looking at how much good scalable infrastructure did for deep neural nets. I mean, it seems like an interesting thing to do. And I mean, we're also working with a company called Simuli out of Florida on a dedicated AGI board, which has a chip for hypervector manipulations, a chip I designed for large scale graph pattern matching, and a deep learning chip and a CPU like wired together very tightly on the same board. If that goes well, then Simuli's AGI board can do for this sort of hybrid AI approach, basically what GPUs did for neural net approach, meaning fairly modest variations on algorithms that have been in the literature before, suddenly you deployed them at greater scale and they started doing magical seeming stuff, right?

29:32If all goes according to this plan, it's probably early next year by the time we start to see really amazing results put out of this. I mean, it's not like five or 10 years or something. That's much sooner than where I thought you were going. I thought you were going to need quantum computing to materialize or something like that. I would love to see quantum computing materialize, but I think the new chip I mentioned is three years out. We're now in the software simulation phase. We need to go to FPGA. We need to go to hardware. We already have the distributed knowledge graph running across many different machines.

30:08We have a limited version of the interpreter working, and we're working on using row calculus, which is an advanced form of process calculus, or the next step beyond pie calculus to allow it to use multiple processes on a single machine. Because we need to go beyond map reduce. Like map reduce is good for something that's matrix multiplication based. What we're doing isn't just mapping. I mean, if you're into functional programming, you need not just fold and unfold. You need like chronomorphisms and future morphisms. You need a whole bunch of ways of holistically processing data structures beyond mapping.