The future of AI is human-centered, with Kanjun Qiu

0:00Starting a business comes with its share of ups and downs, which is why staying true to your vision is essential. a non-negotiable for Romeo and Milka Bregali, Capital One business customers and co-owners of Ra's plant-based restaurant in New York. Romeo and Milka took a leap of faith when starting their own restaurant, gutting an empty space and building it from the ground up. Every pipe, every wall, every detail. But building from scratch came with a heavy financial burden, which is when they turned to their Capital One business card. With the flexibility of the card's no preset spending limit, they were able to spend more and earn more rewards while bringing their vision to life.

0:36Today, Raz's success is proof that with passion and the right support, it's possible to make your dreams a reality. Learn more at CapitalOne.com slash business cards.

0:50There's a lot of talk about we're going to use the AI to do the human's jobs. It's going to be so smart that it's going to tell the human what to do. It's going to tell the human, like, go here, get these materials, go here, get these materials, assemble them in this way. and now you have a nuclear reactor. And like, that's good. And I really want to like topple that story. Like that's not a good future. That's Ken Joon-Kyu, computer scientist and co-founder of the unicorn AI company Imbue. And the future that she wants to see looks a lot different than autonomous AI. A good future is one in which the human is in the driver's seat and the human is able to make good decisions because of what the AI is helping the human understand, what the AI is helping the human do and execute.

1:43And so it's kind of like, how do we use AI to create that much greater sense of agency, freedom and power over our lives? And that's the open problem I'd like to figure out. This is the kind of AI future that I want to see too. It's a future where humans are at the core of AI innovation, a future where there's broad access to AI tools so that everyone can benefit from them. I mean, imagine if everyone could create their own AI. Kan Jun, in part, is trying to achieve this human-centered future through her company, Imbue. They're building agentic AI, but these aren't your typical customer service agents.

2:25these agents are of a different caliber, with the lofty goal to empower everyone. On this episode, I'm talking with Ken Joon about how we can achieve a more democratized AI future, her vision of the next stage of the personal computer, and how AI agents can be a force for creativity. I'm Rana El-Khalyubi, and this is Pioneers of AI, a podcast taking you behind the scenes of the AI revolution.

3:04Before we get into my conversation with Kanjun, I want to talk about some big AI news that dropped last week. Of course, I'm talking about DeepSeek. If you haven't heard, DeepSeek is a Chinese company that released an open-source AI model that rivals the U.S. models from giants like OpenAI. The kicker? They claim to have trained their model for a fraction of the cost. Plus, because their model is so much smaller than, say, the default chat GPT model, it's a lot more energy efficient, too. The news rattled the tech industry as well as the stock markets. At this point, it's been over a week since this has unfolded, and I have two main takeaways from it all.

3:50First off, any AI product, no matter how cheaply made or how sustainable it is, needs to be built on trust. I've been playing around with DeepSeek, and I've got to say, it's pretty great. I like the simple UI. I like how it has visible reasoning so you can see how it arrives at its answers. And I've actually been doing a side-by-side comparison with ChatGPT, and the answers are pretty comparable. But for me personally, I just don't trust the platform enough to input any personal information. Chinese laws make it easier for the government to access this kind of data, which worries me. And I don't think I'm alone.

4:28Companies may avoid using a cheaper model if it means risking their data security. Full stop. My second takeaway is the big one. And it's not even really about DeepSeek itself. It's about how DeepSeek was made. DeepSeek trained their model using a fraction of the amount of compute and investment. And they did so by using several machine learning techniques like distillation and a mixture of experts to compress their models and reduce these hardware costs. And so my main takeaway is that AI innovation is no longer the exclusive domain of big players. Smaller companies have an opportunity to break through in AI.

5:10And the good news is, as AI becomes more efficient and more accessible, its adoption will soar, both by consumers as well as in business. Which means that smaller companies can more easily harness AI, and this will all unlock and accelerate innovation. Look, I don't think last week's news means an end to the AI race. But what I do think is that VCs and the U.S. government should be investing more into the startup and innovation ecosystem, because this is where cutting-edge technology is born. I'm going to continue following what happens around DeepSeek, and as an investor, I also have my eyes on companies innovating in this space.

5:51If you have thoughts about DeepSeek or how these global companies affect your life, I'd love to hear them. Leave us a voicemail at 601-633-2424 or email us at pioneersofai at weightwatt.com. Now let's get to my conversation with Kanjoon because her company is truly at the cutting edge, leading the charge in agentic AI. Let's hear it. Hi, Kanjoon. Thank you for joining Pioneers of AI. Thank you, Rana. So we both have the MIT connection in common. and you did your undergrad and master's degree at MIT. And you also spent time at the Media Lab, which is where I did my postdoc. How was that experience like?

6:35Yeah, MIT was such a fun experience. You know, they always say, MIT, IHTFE, you either, I have truly found paradise or I hate this f***ing place. And I definitely felt both very strongly. But the Media Lab was really such an interesting place. I think that's actually where I first got my taste of the like power and magic of computing. And, you know, Imbue as a company, what we're really trying to do is reinvent computing and reimbue that power back into computing. And so I really felt the sense, yeah, at the Media Lab that, like, I want to figure out how to make computing more accessible. And Imbue is another variation on how to do that.

7:10Yeah. So let's talk about Imbue and specifically AI agents, which is one of the focus areas for Imbue. So when I think of AI agents, I think of basically AI tools that can do stuff, can execute tasks on your behalf. On this podcast, we have had guests who are building customer service AI agents. We've had guests who are building companies that do voice AI agents to automate healthcare workflows. What is Imbue's focus? Like what's your definition of an AI agent and what are you guys building? We started Imbue in 2021. and we started as a research lab. We were very interested in how do you build general agents?

7:54We really had this feeling that if you could have a general agent, in the same way, you know, ChatGPT is based on an LLM that is more general, you know, has more general knowledge. If you could have general agents, then you could get your computer to do much more for you and that would unlock a lot of creativity. These days, you know, agents are really hot this year. Oh, yeah, everybody's doing an AI agent. Yeah, everyone's doing agents. And these days when people talk about agents, it's often talked about in a very specific way. People think of agents as this kind of personal assistant that does stuff for you.

8:26You tell it what to do. It's maybe something like Siri or a customer service bot, or it is something quite specific. And then this agent will go and do that task on your behalf. And that's kind of how people conceptualize agents. And when we first started in Vue, that's also how we thought of agents. Perhaps you could build a general thing on your computer that could help you do stuff on your computer. But over time, actually, we learned a lot about what makes agents interesting and powerful. One of the big things we learned, actually, is that delegating a task to an agent is a very difficult thing for a human to do.

9:02You know, as a founder, delegation is hard. Right. I have to figure out what to delegate and how. You have to trust that the person or the thing you're delegating to is going to get this job done at least as good as you will, right? Exactly. You've actually nailed it. The core issue is trust. What can I trust this agent to be able to do? What can I not trust it to be able to do? And what we found is that it was very hard for people to use a general agent because they were like, I don't know what I can use it for and what I can't use it for. And there was a second piece that really struck me where I'd been thinking a lot about how do we have a good future with very powerful AI systems and humans?

9:41There's been all of this discussion about taking over people's jobs and EGI killing us all and things like that. And it's something I think about a lot is how do we create a future that's humane and where technology serves people and not the other way around? And so that actually caused us to rethink what we believed agents to be. What we realized is, huh, if you really think about what an agent is, an agent is this intermediary between you and your computer. It's just a piece of software. It is a system that talks to your computer by asking it to do stuff, and it talks to you, either through language or an interface or something like that.

10:19And the most general way for it to talk to your computer is by writing code. because, you know, even if you work with a customer service agent, someone had to write the code for what that agent is doing. And so the most general way of thinking about what an agent is, the most powerful way is not as these vertical personal assistants, but rather as a system that lets you write code, arbitrary code on your computer. It is essentially a higher level programming language. That's what agents are in their most kind of imaginative view of what they can be. And once we realized that, we were like, ah, what AI allows us to do is enable every person to program at a higher level, at a much more intuitive level.

11:00Imbue is still in its research phase. They don't have any products available for commercial use yet. Coding right now is like a super, super detailed task where I'm writing out every single character. And now like AI systems can generate functions, but it's still like very low level. And so we realized like, okay, this more empowering vision of what an AI agent could be is actually as a system that democratizes coding and democratizes the ability to control your computer and get your computer to do what you want it to do. Right now, we're like customers of all of these pieces of software that other people built.

11:38But I think that in a future where building programs is super easy and cheap, we would build a lot of stuff for ourselves and we would make our digital built environment very custom in the way that our physical homes are very custom to us. Yeah. So let's bring this to life for our listeners. So say I want to write an app that pulls data from all of the different wearable sensors I wear, and also perhaps my electronic health records and maybe my latest blood biomarker data, and it's going to draw all of this data together and then give me health and nutrition and exercise recommendations. So one way to do this is to put my computer science hat on and actually code this, right?

12:18I guess with Imbue, I could like literally, what, talk to an Imbue agent and say, hey, like take my Whoop data and combine it with this data and visualize it in a beautiful graph. Is that the vision? Is that the idea? That would be awesome and magical, but would not work. Right. We're not there. We're not there. We're not there yet. Unpack that for us. Yeah, that's a great question. So, you know, right now when people think of kind of AI coding systems, they think either GitHub Copilot, I'll autocomplete your next line of code, or they think App Builder, thing I can tell instructions to and it'll make the app for me.

12:55But as programmers, as computer scientists, like, we understand that coding is more than just writing the code. It's also architecting, like, what is the data model and what kind of abstractions do I want? And also, it's about managing changes. I make a commit. Now I'm going to make a new feature. Making a commit is basically saving changes to your code. And sometimes I need to roll back this feature because it didn't work when I was testing it, and I actually need to rethink it. What we're building right now is a tool that allows people to work at a slightly higher level, but kind of more at the feature level.

13:35So I've got a, maybe I've got a code base, or maybe I'm starting from scratch, and I'm writing a feature. So maybe the first feature I would write is integrating with your aura ring or something. And you're now getting that sensor data. Okay, now you got that data in, I'm going to add a next feature, which is integrating my EHR system. EHR system, as in electronic health records. What Kanjoon is talking about here is my dream, a way to cohesively integrate my wearable biometric sensors with existing health records from my doctors. It's not as simple as dictating what kind of app you want to make to an imbue agent.

14:11But the tools they're working on can help make building an app like that so much easier. You know, earlier we talked about trust. It's this slightly more fine-grained control of the system that gives people trust. Because I've used all of the app builders and the app is not what I wanted it to be. Even if I'm interfering in the middle of it while it's thinking, it's like still not quite what I want it to be. And so would it be correct to say that your first kind of set of users are actual software engineering teams, and this is helping them be more productive and get to market faster as they're building their products?

14:48Yeah, it's a great question. So I would say our initial users are software engineers, or people who know how to read and write code. I would say I'm not a very good software engineer anymore, but I do read and write code. And so someone like that. And it should help people get things to market faster and build features faster. I think that's a really nice piece of it. We're still in the user testing stage and we're still testing it out ourselves and we'll have an alpha relatively soon. Actually, the thing I was most surprised about is how much context switching I do when I'm programming and how much this reduces my context switching and keeps me in flow.

15:28And one of the things I realized is like, oh, wow, this feels really good to be working at a slightly higher level. I don't have to be jumping around the code base so much. And maybe this is what programming can feel like more and more and more. And I was just dealing with all this context switching before, and I didn't even realize that it was kind of painful. Yeah. You know, the analogy that you've made me think of is when I first started, like one of the early classes I took when I studied computer science as an undergrad was how to code an assembly language, which is like this low-level coding language.

15:59and oh my God, it was so arduous and just, I hated it. And then, you know, and then I learned Pascal and C++ and Python. And these are all like abstractions that make this idea of coding more accessible and you're taking it even, you know, a few more steps. Yeah. Yes, I think the way we think about what we're building is we're building the next layer of abstraction on top of programming languages. And that next layer of abstraction, it is actually in some ways a programming language. itself, but it is, you know, this AI-enabled programming language. And one thing that's been interesting about it is that it's more than just writing the code.

16:37It also has to take into account the sociological process of programming. There's like this sociological process we've developed where we write a spec doc for a feature to think through what to build next. And we figure out the data model and we make commits and we do testing and all of these things that make complex software possible, those sociological processes actually are part of, you know, how we think about the next layer of abstraction. If you're not a computer scientist or can't read or write code, this is still a relevant innovation for you because it means that the barrier to writing complex code will be a lot lower.

17:17Maybe at this point your spidey senses are going off. Wouldn't AI that can expertly write code reduce the need for computer scientists? Well, it's complicated. While Kanjoon wants to see a world where everyone is empowered to write software, she thinks there will still be a need for coders. We'll get to why after a short break. Stay with us.

17:55doesn't fail because of ideas. It fails because teams can't move together. Enter Atlassian's teamwork collection. It has planning in Jira, documentation in Confluence, video updates in Loom, and now AI agents in Rovo, which connects the dots across your work so nothing gets lost. It's one AI-powered teamwork platform designed for how modern teams actually build. Learn more at atlassian.com slash teamchanger. That's A-T-L-A-S-S-I-A-N dot com slash teamchanger. All right, so we both studied computer science and spent a lot of time, both actually at the undergrad and postgrad level, learning how to code.

18:43Do you think what you're building is going to change the demand for computer science as a profession and as a degree. And I ask, my son is almost 16. He is very tech forward. He's very interested in tech, but I don't know if it would make sense for him to major in computer science. I don't know. So what's your view? I think people who understand how to build software will still have a very big advantage. The tools that we see coming out, the tools that we're building, they still require you to be somewhat technical. and one of the dreams I have is empowering everyone to be able to write software but the reality is that a lot of people are probably just going to use software that other people are making for them and that's okay but I think the power to create software like computing and the the digital built environment is going to be such a dominant thing in our future it's already pretty dominant today but the digital environment is able to do and process so much information like being able to have creative power in that environment is really powerful.

19:45And so I would say studying software engineering or studying how to program those skills and those concepts are still really important, like learning about algorithms and learning about data structures and learning how to assemble a system and learning systems thinking. These are all still going to be necessary no matter how good the programming tools get. It's like if you really wanted to be a painter, you still have to learn how to work with paints, even though someone else is now manufacturing the paint for you. Yeah, and I would imagine we are going to continue to innovate on the algorithm side, and we're still going to need people who are deeply immersed in machine learning to continue to innovate on these models and these approaches.

20:23Exactly. And for us, you know, one of the things we're excited about is enabling people to build their own agents. So, you know, if you want to build your own email bot that processes email exactly the way you do or something like that, and that's still going to take quite a bit of skill. So it's still a useful discipline. You know, I love your focus on empowering people to write their own software. And the way I'm thinking about this is, and you mentioned this earlier, like how can we harness AI to unlock human potential? This is something that I'm very passionate about. Can you talk a little bit more about how you see these AI agents empowering people as opposed to taking away jobs or taking away opportunities from humans?

21:05It's something I think about a lot. As a company, our mission is explicitly empower humans in an age where machines are becoming more and more powerful. And we really view these systems as tools for people. They should be built as tools for people. And so we actually think quite a bit about decentralization versus centralization of power. I think part of why people feel and why I feel concern around current AI systems and the increase in their capabilities and why we feel this kind of like latent fear is because those systems are becoming more and more powerful. And we don't feel like we actually have that much control over them and how they will impact our lives.

21:48It feels like something being done to us as opposed to something that we're doing. And there's kind of an interesting historical analogy here where in the 1960s, people were really excited about the supercomputer. And people thought the supercomputer is going to be the future. It's the future of business. Everyone's going to be timesharing on terminals on centralized supercomputers that are going to be really, really powerful. And that's the future of computing. And then it took a group of people in the 70s, researchers at Xerox PARC, to invent the desktop and the mouse and the GUI and files and folders and all of these primitives that make computing more understandable to us.

22:29Because they took a lot of the ideas that we already understand, those concepts we understand, and they imbued it into computing. And that is what enabled the personal computer. And when the personal computer first came out, people thought, this is a toy. No one's ever going to use this. It's not that powerful. But the power really was in how people figured out how to be creative with it and how to build with it. And I think there's actually something similar with AI where there is this default centralizing force right now. And that centralization is real. That centralization of power is real.

23:02And the default path is that we end up with these entities that are very powerful corporations that have the power of these very large models at their fingertips, and they can do stuff with these models. And I think it's incumbent on us, Imbue, and also, you know, people building technology to figure out how do we take that power and give it to people so that people can be creative in this new medium. I love this analogy because I often reference this vision of 45, 50 years ago of giving a personal computer to everyone. And the analogy that I've been using for the world we live in today is giving everybody access to a personal AI assistant.

23:44But I actually love your tweak on it, which is giving everybody access to an AI agent that allows people to express themselves and get things done that they would have otherwise not been able to. That's really powerful. I love that. Yeah. An assistant is somebody else's thing that they made. The true power is like, you know, I feel a lot of power over my home. I can add whatever objects I want. I can do construction. I can change things. And that's what makes you as a person feel like you have power over that environment. And so, yeah, I think it's really important actually to go away from the assistant analogy and go toward the creative kind of future where like, how do we enable people to create with this?

24:28Yeah. You almost need a different word than agent. I know. Like, yeah, right? Yeah, agent's the wrong term. I think in 10 years, we won't be talking about agents as much. Yeah. Okay, so we talked a little bit about trust and how important trust is to this process because there's a trade-off between trust and autonomy, right? Like how much autonomy do you give this thing on your behalf to go do stuff? How do you instill these principles into the work you do? Like, how does it translate into actual principles or frameworks? Yeah, I would say trust is at the core of our product development in a lot of ways.

25:05And the question of trust actually drives a lot of our research. So when it comes to writing code, I can trust the code if I know that it's doing what I wanted it to do. As a software engineer, usually the way I handle that is either by reading the code or by testing the code. So I'll write tests for the code. Then I can say, okay, I tested this code. It doesn't have the edge cases that I didn't want it to have. It does exactly what I expected it to do. And that kind of helps me trust the code. And so actually as a company, we do a lot of work on verification of code. How do we effectively verify code so that as a user I can trust without having to read the code in detail, which is for LLM generated code, very arduous.

25:51How do I trust that it's correct? And so that question of trust drives our research direction around verification. It also drives how we think about the user experience. So from a user experience perspective, I would say autonomy is not the goal. The goal is to have the system be able to do more useful or bigger useful things for you. We try to get away from delegation and autonomy, and we try to move toward trying to make the interface feel tactile. An autonomous agent inherently is not tactile. I inherently don't actually feel like I have that many levers to control what it's doing. And for us, the question that we always ask ourselves is like, how do we build more tactility into the interface so that I actually feel like it's like clay?