AI Agents That Reason and Code with Imbue Co-Founders Kanjun Qiu and Josh Albrecht

0:05Imbue is a company developing AI agents that can reason and code. Today, Ilada and I sit down with Kanjin Q and Josh Albrecht, co-founders of Imbue, to discuss training large foundation models for high-level reasoning, why agents require architectures different from large language models or language token prediction models, and how current computers are getting in the way of their users. Kanjin, Josh, welcome to KnowPriors. Thank you. Thanks. So perhaps you can start by just telling us the story of how you guys know each other and where the idea for Imbue came from. And Josh and I met at a conference and then started a big house together.

0:41It was a big house, 20 % house, and also started this first company around the same time. I've always been really interested in agency and kind of like, how do we enable humans to have more agency? And Josh has always been really interested in AI. And so it kind of made sense. We, at that time, talked about like, oh, you know, someday we're going to be able to have AI systems that give humans a lot more agency. Fast forward to 2018 or so, we were running an AI recruiting company called Sorceress. And that was actually kind of the first AI agent that we built. It was, you know, not transformer models, like more old school NLP.

1:24But it was a system that recruiters used and kind of automatically got candidates in their inbox. And we learned a lot about, oh, if you have an autonomous system like this, like what do you actually need to make it work? And around that time, some of our housemates were building GPT-3. And we were seeing like, oh, scaling works. You know, if we just keep scaling, actually, you're going to get pretty far with a lot of these language models. So our question at that time was, you know, how far can we get with language models? Does this kind of self-supervised learning, which is working so well on language, work in other modalities as well?

1:59So in early 2020, that's when we first started seeing self-supervised learning working across video and images and language. And we were like, huh, there's something really interesting here where maybe machines are learning the same kinds of representations or similar representations to what humans are learning. And maybe they can get to a point where they can actually do the types of things that humans are able to do. And that's when we first started Imbue or started talking about Imbue. You clearly know a bunch of people working at sort of large language model research labs. Well, when you looked at what they were doing, how did the focus come to be on agents in particular?

2:36And how is that different from a general language model? Yeah, I think we've always been interested in agents in not just, you know, recommender systems or classifiers or things like that, but in systems that are going to go do real work for us, right? That are going to actually be useful in the real world. Right now, you can ask some kind of chatbot something and it'll give you back a response. But the burden is sort of on you to go do something with that to verify whether it's correct or not. I think the real promise of AI is if we can get systems that can actually act on our behalf and can accomplish goals and kind of do these larger things and sort of free us up to focus on the things we're interested in.

3:10Yeah, one thing that I think we often forget because we're in it every day is our computers are actually like, they need to be micromanaged. The reason we're in front of our computers every day is because nothing really happens. Like they can't make decisions on their own. Nothing really happens unless I'm in front of it. And I'm like in front of it doing all this really detailed stuff, kind of like operating a factory machine with all of these like little knobs that are really specific. And there is a future where computers don't need to be micromanaged that I can like give a computer an instruction, whether it's natural language or something, some other kind of instruction, like it can go off and understand what I'm trying to do and help me do it.

3:46The diff between this where we are today and that is kind of like the diff between the first calculator and where computers are today. Like the first computer, very first digital computer was a room size calculator. All it did was calculate Fourier transforms and things like that. And I think that's kind of the potential for where AI can be given the current, like where the technology is going. It's very possible. What do you think is, so when I look at technologies, there's almost like three types, right? There's things that just are never going to work. And maybe some aspects of Theranos for that, right?

4:21Like there were questions whether the physics of Theranos would ever work as you miniaturize things sufficiently. There's things that can work immediately today or with a little bit of work or engineering, you can get there. And then there are things that are clearly going to happen. And they're at some point in the future. So for example, in the 90s, people to find out almost everything that cell phones would do. And then eventually it got there once you had better processors on phones and more bandwidth in terms of cellular networks, like you need to build a bunch of stuff in terms of infrastructure.

4:48And it was clear what was going to happen. What do you think is missing, if anything, technologically to start to build real worlds for the performant agents? The way we think about agent tasks is it's a spectrum of difficulty. So some agents are very possible today. Like we see a lot of them. There are these conversational bots that take over some of the customer success workflows and they'll fail over to real customer success people if the agent doesn't know how to deal with it. And those are actually what we see inside companies is that they actually have pretty complex reasoning workflows.

5:22They're somewhat hard coded. And so they're not general. They don't generalize to other company workflows. But, you know, we are already seeing agents. And then maybe there are two spectra. There's like specific to general. So today we have very specific agents. And over time, if they are better at reasoning and better at certain other things, interacting with your computer, then they become more general, be able to use the same agent, and it'll learn something new. And then there's also a spectrum from copilot to more autonomous. So today we see a lot of copilots and the human in the loop, and over time it becomes kind of incrementally more autonomous.

6:02And so I don't see it as being so binary, like, oh, there is a technology missing for agents. But rather that as capabilities improve, we're going to see more and more of these use cases be eaten up by more general, more autonomous agents. There are a few categories of things that are in the way today. So I would say where we are today, we're kind of in the era of like maybe lossy Ethernet with no error correction or like analog computers, something like that, where we have these models and they don't work reliably. And that's, you know, when we talk to founders building agents, that's really the biggest thing.

6:40It's really hard to get these systems to work reliably and output exactly what I want it to output and kind of do the right thing at every step all the time. And so the question is, okay, how do you get it to work more reliably? Well, there's a lot of why we work on reasoning. And when we say reasoning, it's kind of all of the things around getting tasks done in the world, like when does the system come back to you? How does it know it's not certain about its output? Can it kind of think through different action plans and figure out, okay, this plan is the better plan, and we should try going down this path first.

7:15reasoning is one big piece of improving reliability. And second chunk of things is like all of this error correction. And I think like chain of thought, tree of thought, these are error correction techniques. We have a lot of other techniques internally, and that also helps improve reliability. And so if we think about this problem as a reliability problem, then you can incrementally make a lot of progress on it. I loved your framework of sort of generalizability and sort of that two by two that you had. If I look at a lot of the language models today, what I'm observing a lot of people doing is they basically start off prototyping something, say on GPT-4, because it's the most advanced model.

7:55They see if it works or not. And if it works and they have any sort of scale, in some cases they move to GPT-3.5 and sometimes they've thought about fine tuning or not, but sometimes they'll move to an open source model, which works dramatically less well in some cases, but then they'll fine tune it for a high volume use case. and it's all because of cost optimization. Basically, as you know, like if you have a really big model, it costs a lot more for inference in terms of compute than like a smaller model. How do you think about that relative to generalizability? Because I guess if you make something really generalizable, my assumption, which may be incorrect, is it's more expensive, right?

8:28You'll need some forms of memory for it. You need some broader logical capabilities versus just saying, I'm just going to do the thing that's going to like order flights really well or whatever it may be in terms of agents. And so I'm sort of a little bit curious about that framework. I think that's a, that's the right way of thinking about it. I think when Kenjin was saying, you know, the spectrum from more specialized to more generalizable, I think we're talking about the ability to solve more general problems, like the ability to do these problems that you've only seen once or twice. I think even as that ability goes up, we're still going to see kind of, you know, a thing coming behind that, a force that takes each of those things.

9:00Like maybe you start out by doing your plane booking with GPT-4, but eventually you realize like, oh, actually, like this is so expensive and slow. Like I just want the thing to be really good at it. But what you can do is you can apply these agents. and this is part of the reason why we're interested in agents that code, you can apply those agents to the original general system to have it go make a more specialized version of that. So it's kind of specializing the things that you're doing a lot. And you can look at each of those things like, okay, I'm making 10 ,000 calls to this. This is super expensive.

9:25Can I just write a piece of Python code that does this? As you have more general capabilities, you actually can use those more general capabilities to kind of do that specialization. What we see with a lot of the agent builders today is that they'll use, you know, an agent workflow is complicated. It has lots of different pieces. And so they may use a specialized model for parts of it and a general model for other parts of it. And the way we think about it, it's quite pragmatic. Basically, that as capabilities increase, what we want is like minimal viable models for each capability. And so a lot of the models are much smaller and very specific and like pretty specifically trained.

10:01In the personal computer kind of revolution around that time, I think there was kind of like branching. So some people built supercomputers. computers, they're like, oh, we're going to make the more powerful computer. And then other people built personal computers. And it turns out personal computers is a much bigger market, and supercomputers, not that many people needed that much computing power. And I suspect we're going to see something similar where a lot of use cases are going to be able to be addressed by something pretty pragmatic and relatively small. We're definitely not pushing the bounds of what we can do with data today on small models.

10:31And so smaller things can work well. I want to go back to what I think is like a really deep topic of discussion at Imbue in terms of how you define reasoning and like this being an area of differentiation in terms of your research efforts. It's like, you know, we all have a bunch of friends at OpenAI and other labs working now publicly on multi-step reasoning and more process supervision, as you were describing. What makes you excited and confident that there needs to be a different approach versus just general language models in order to make the reasoning you need for agents to work, work? I think there's a different process.

11:12Like, language models are great. They're really good, you know, predicting the next word. They're good at, you know, making like a very easy classifier. They're good at all sorts of things, but there are obvious limits. We know even in theoretical senses, they cannot learn to do multiplication in the general sense because it literally doesn't fit in the context window. Multiplication, they can learn to do addition in a modular sense, and they can learn to do it actually almost perfectly if you train them in the proper way, but they're not learning the general algorithm for addition. Instead, if you want something to actually execute the general algorithm for addition, you need to have a thing that works in a different way, that has some sort of outer loop about what steps should I take next.

11:49That's just a kind of definitional thing. There has to be some other sort of wrapper. There has to be a different sort of outside process. Everyone at OpenAI and at Imbue and at Anthropoc, we all know how this works. I don't think anyone is proposing it's just shove it all in the language model. You can get really far, but I think we're interested in what is that other higher level system? How do we decide what is the right next step to take? When should I go collect more information? Am I certain about this? All of these kinds of other things, those are, I think, the questions that are much more interesting.

12:16I think there's actually a lot of work to be done there. I think we're still very early in the days of creating these systems. Natural language is not a bad medium for it. Code is also another example of a medium for it. Language is pretty compressed. And so that's helpful for dealing with these situations. Is that one of the reasons you all decided to focus on code as one of the first HEPSA agents that you have started with? Or could you explain more about the logic behind that? Yeah, I think for us, code is useful when we're thinking about reasoning. One way that we're sort of making, you know, collectively reasoning agents today is founders are just hard coding the reasoning process of like, okay, if there's a customer support complaint about this thing, then I do this.

12:58If it's like this, then I do that. And so you have this like very special case version of the thing, right? And there's a spectrum between code and language or more kind of general reasoning abilities, but it's a spectrum. It's not a binary thing, I think. And so you can have code now that we have these language models that kind of mixes the language models and the code layer, right? Where it's like, sometimes you're using the language model to decide what to do. Sometimes you're using an if statement. And so it's more about like a fusing or like melding of these two different things and being able to like be in the right place on that spectrum.

13:25And so code is actually like a really important part of this. And as you do things that you want to do more robustly and you want to do in a more repeatable way, then you want to move it more towards code, right? And so to the extent that you've never seen this task before, maybe you should be doing it in this more kind of nebulous intuitive sense. And then over time, get better at it, critique it and turn it more into code actually. way. Yeah. And when we see founders and ourselves building these agents and people shipping them into production and us shipping them internally for ourselves, like basically the agent loop can be very complex and breaks down into different chunks and we can like turn certain chunks into code.

14:02And it really feels like programming in a lot of ways. So there's something kind of interesting. Can you talk a little bit about just where you begin in terms of like how to structure the research effort? Like if there are certain tasks you work on, if you start by working on policy or reinforcement on certain tasks, or there's data you want to collect, like how do you start? Yeah. So we have this idea we call serious use, where basically we should be building agents that we want to use every day. This is actually one of the biggest blockers. Like it's really hard to get agents we want to use every day because of the reliability issues.

14:37And so a lot lot of what we work on is coding agents, but we also work on agents for other operational business processes. And that kind of helps drive, oh, okay, like these parts of the agent loop are really complicated. Like, can we simplify them? Can we make them more reliable? And in a lot of ways, it is an incremental kind of set of work that helps us get from like, you know, 60 % reliable to 70 % to 80%. And that's what forces development of new techniques. It's not like, oh, magical, We train a giant model and stick everything into it and then magically it works. It does not work. It'll get better at random parts of the agent loop, but that's not what we want.

15:16And is the premise here, like you start with a serious use, smaller task in code or something like a recruiting communication automation task? Or how do you choose? Yeah, we pick tasks kind of depending on a bunch of different factors. One, like how useful, how frequent, how possible is this going to be to do right? How generally applicable is it? How much is it going to help push the techniques that we want to push forward? Does it scale to more complex versions of the task? Yeah. So we're purposely trying to pick some with some diversity. Like we have one agent that will just go do a random to do in your code base.

15:51And so this can be super, super general. It can take a really, really long time to do this, right? And we have another, the opposite end of that spectrum is we have an agent that will look at every single pull request and run a linter against it and ask like, okay, are there any type errors? Okay, how do I fix them? all right, great, here's like a PR with me fixing the type errors for you. But very, very specific. But really, you can imagine how, you know, you can invoke the to-do agent to fix a specific type error. And you can expand the type error fixer to do unit tests and to do security flaws and to do renaming these variables.

16:21And they sort of meet in the middle as you kind of make these things both more capable. And so they're just different ways of kind of looking at the problem of how do we make a useful coding agent. And Eli, to your point of kind of the specialized versus general dichotomy, Um, one thing that is kind of interesting that we're seeing in agents is like agents can call sub agents. So our to do agent can figure out like, oh, there's already a sub agent for this thing, um, for this function you're trying to write. And like, let me call that sub agent because it seems likely to succeed. And then if I try it and it doesn't succeed, I'll do something else.

16:51And so you can kind of have this like more general reasoning layer and also a bunch of sub agents where that general reasoning layer is actually very specific. It's a specific planner. It's not that good at like browsing the web and things like that. But the system itself altogether is more general as a result. So how do you guys do evaluation for both these, like, let's say, categories of agents that you're working with today from the, I assume, more closer to production grade to do to broader coding agents? Yeah, the evaluations, I think, are actually one of the most important parts and one of the places where we spend the most time and think about it kind of the most.

17:30There's a lot of work in specifying exactly what you want from the to-do agent, for example, right? Like, how do you know, like, it gives you back some code. Okay, is that good? There's sort of a spectrum, but if it's faster, it's better. If it gives you less code, that's better. But if there's bugs, that's not good. So you really need to take it and break down, what did I really want to happen here? And I think when you start to break this down, you start to say, okay, there's some things that are kind of qualitative. Like, do I trust it? Did it come back with tests? Can I run this code immediately?

17:57Like the kind of feel of it. There's other things that are just for the code itself. There are different attributes. Is it in the same style? Does it have good variable names? Like, is it a minimal change? Or did it change all sorts of stuff that it didn't really need to change? Each of those things are actually something you can measure a little bit more easily than the overall task. So you can make another kind of metric that's like, okay, how good are the variable names? All right, how similar are they? You can break that down. You can kind of keep breaking it down until you get to a point where it's like, okay, I mean, you know, a regular language model or even just a person looking at this, like there's an objective answer.

18:24One of the reasons why we work on code is that there are objective answers to a lot of these questions. Either the test pass or they don't. Either the function is correct or it isn't. Those kind of things are much easier to evaluate. And so we're starting a lot more of our tests are in that zone as we sort of build up eventually to the ones that are a little bit more qualitative because the evaluation is so much harder there. But I think the whole the strategy of breaking these things down, like basically the strategy is we take the output or the answer and we like ask a bunch of questions about the output and then we evaluate those questions.

18:53And we also evaluate the output. And the interesting thing about that is it scales pretty well to like non-code tasks. So for like our recruiting tasks, we can also do a very similar process. I think part of why a lot of teams try to work on just math or code reasoning is because those are the easiest to evaluate and like the clearest answers. But just relying on like, is the output correct or not, that loses a lot of information in the evaluation. Yeah, I think it's likely to be a pretty rich space. I'm curious for your point of view, but we've looked at a lot of startups building, let's say, interesting AI, like development tools.

19:29Right. And one of the things that we've spent a bunch of time thinking about is like what makes for a good scalable eval loop. Right. And that could be objective and easy to test. Right. Like it doesn't compile to things that, as you said, might be richer in data. Like how easy is it to check the functionality of something? Right. Do you have to do static analysis? Is the performance better? Are there examples if you want to focus on a particular problem like, let's say, like Python 2 to 3 upgrades or something? I think one of the things that's most attractive about this domain is there are lots of ways to evaluate, even beyond the contributions to reasoning that you guys describe.

20:10And it's just going to be productive. I mean, maybe on that topic, do you guys think of yourselves as a product company? Is it important to go get this functionality in front of users or just focus on research and how do you think about that sequencing? Yeah, of course, we're a product company. We're a company. But I think looking at the history of computing, there is like a right time for technology. And today, I think what you see, what you both see is that it's pretty hard to make agents that work that like can be productionized and used all the time. And it's because like the technology is just not there yet.

20:49And we use reasoning as like a bucket term. we've described a little bit all the nuances of what we're actually trying to do to get agents to work. And then we lump that all under the term reasoning because it's easier for people to conceptualize. But the reality is like what we're trying to do is to make kind of a system, a set of tools and maybe frameworks that actually makes it so that we can build reliable agents really fast, really easily. So today, like writing agents feels like writing code in assembly. And And that really limits the types of agents we can build and also limits the number of people who can build them.

21:26And kind of what we're going toward is like programming languages that are a little bit more ergonomic, where we can build agents much more easily, where they can work much better, and where a lot more people can build them. Whatever it is that we release, that's what we hope it's going to enable. And so that's why we kind of work on different parts of the stack. We work on the underlying models because there need to be more specific underlying models that work for specific things. And that's what allows a lot of these capabilities and agents to be more reliable. We also work on other pieces of it as well.

22:04Maybe if we just project forward a little bit, what are you guys most excited about? You want to be a tools at different levels of the stack company. What are you imagining people build? Or what are you already seeing people build that you think is going to be? let's say useful a year from now and useful five years from now? Yeah, I think a year from now, we're going to start to see some of these use cases actually work that today you can write these. We have the capabilities. You can make some kind of agent to triage your email or to do scheduling or many of these workflows that we really should.

22:35Why don't we have that today? That definitely can be done. There's nothing stopping us. And I think five years from now, we're going to have something where it's not just, okay, we have a scheduling bot, we have this other thing, but we really have these more general, more robust systems where each of us can individually say like, I want a thing that does this. I want to do this particular weird research workflow. And I want it to work like this and blah, blah, blah. And just specify in language. I think one thing that... Personalized agents. Yeah. One thing that our recruiter mentioned yesterday that I thought was kind of funny is he's been describing to candidates like, we're actually sort of a software dev tooling company.