How Agentic AI is Transforming The Startup Landscape with Andrew Ng

0:05Hi, listeners. Welcome back to KnowPriors. Today, Alad and I are here with Andrew Ng. Andrew is one of the Thank you so much for being with us. Always great to see you. I'm not sure where we should begin because you have such a broad view of these topics. But I feel like we should start with the biggest question, which is, you know, if you look forward at capability growth from here, where does it come from? Does it come from more scale? Does it come from data work? Multiple vectors of progress. So I think there is probably a little bit more juice out of the scalability element to this piece. So hopefully we'll continue to make progress there.

0:53But it's getting really, really difficult. society's perception of AI has been very skewed by the PR machinery of a handful of companies with amazing PR capabilities and because that number of companies drove scales and narrative people think of scale first as a vector progress but I think you know agentic workflows the way we build multi-modal models we have a lot of work to build concrete applications and there are multiple vectors of progress as well as wild cards like brand new technologies like can diffusion models which are used to generate images for the most part will that also work for generating text.

1:25I think that's exciting. So I think there'll be multiple ways of AI to make progress. You actually came up with the term agentic AI. What did you mean then? So when I decided to start talking about agentic AI, which wasn't a thing when I started to use the term, my team was slightly annoyed at me. One of my team members that I won't name, he actually said, Andrew, the world does not need you to make up another term. But I decided to do it anyway, and for a lot of reasons, it stuck. And the reason I started to talk about agentic AI was because like a couple years ago, I saw people would spend a lot of time debating, is this an agent?

1:55Is this not an agent? What is an agent? And I felt there's a lot of good work and there was a spectrum of degrees of agency where there are highly autonomous agents that could plan, take multiple sets of reasoning, do a lot of stuff by themselves. And then things that were lower degrees of agency where we're prompting our own, we're affecting this output. And I felt like rather than debating, is this an agent or not? Let's just say the degrees of agency and say it's all agentic to spend our time actually building this. So I started to push the term agentic AI. What I did not expect was that several months later, a bunch of marketers would get a hold of this term and use it as a sticker to stick it on everything in sight.

2:33And so I think the term agentic AI really took off. I feel like the marketing hype has gone like that insanely fast. But the real business progress has also been rapidly growing, but maybe not as fast as the marketing. What do you think are the biggest obstacles right now to true agents actually being implemented as AI applications? because to your point, I think we've been talking about it for a little while now. There are certain things that were missing initially that are now in place in terms of everything from certain forms of inference time compute on through to forms of memory and other things that allow you to maintain some sort of state against what you're doing.

3:01What do you view are the things that are still missing or need to get built or will sort of foment progress on that end? I think the technology component level, there's stuff that I hope will improve. For example, computer use kind of works, often doesn't work. I think so the godrails, evals is a huge problem. how they quickly evaluate these things and drive evals. So I think the component is this room for improvement. But what I see as the single biggest barrier to getting more agentic AI workflows implemented is actually talent. So when I look at the way many teams build agents, the single biggest differentiator that I see in the market is does the team know how to drive a systematic error analysis process with evals?

3:40So you're building the agents by analyzing at any moment in time, what's working, what's not working, what you improve as opposed to less experienced teams kind of try things in a more random way that just takes a long time. And we're looking for a huge range of businesses, small and large. It feels like there's so much work that can be automated through agentic workflows, but the talent and skills and maybe the software tooling, I don't know, just isn't there to drive that disciplined engineering process to get this stuff built. How much of that engineering process could you imagine being automated with AI?

4:10You know, it turns out that a lot of this process of building agentic workflows, It requires ingesting external knowledge, which is often locked up in the heads of people. So until and unless we built AI avatars that can interview employees doing the work and that's a visual AI that can look at the computer monitor, I think maybe eventually, but I think at least right now for the next year or two, I think there's a lot of work for human engineers to do to build more agentic workflows. So that's more the kind of collection of data, feedback, et cetera, for certain loops that people are doing. Is that other things?

4:48I'm sort of curious what that translates into tangibly versus... Yeah, so maybe one example. So I see a lot of workflows like maybe a customer emails your document, you're going to convert the document to text, then maybe do a web search for some compliance reason to see you're working on a vendor you're not supposed to, and then look at the database record, see the pricing is right, save it somewhere else, and so on. It's multi-stake-way genetic workflows, kind of next-gen robotic process automation. So we implement this and it doesn't work. You know, is it a problem? If you got the invoice date wrong, is that a problem or not?

5:17Or if you routed a message to the wrong person for verification? So when you implement these things, you know, almost always it doesn't work the first time. But then to know what's important for your business process. And is it okay that, I don't know, I bothered the CEO of the company too many times? Or is the CEO of me who doesn't mind verifying some invoices? So all that external contextual knowledge, often, at least right now, I see thoughtful human product managers or human engineers having to just think through this and make these decisions. So can an AI agent do that someday? I don't know.

5:49Seems pretty difficult right now. Maybe someday. But it's not in the internet pre-training data set and it's not in a manual that we can automatically extract. I feel like for a lot of work to be done building agentic workflows, that data set is proprietary. It's not general knowledge on the internet. So figuring it out is still exciting work to do. What is the, if you just look at the spectrum of agentic AI, what's the strongest example of agency you've seen? I feel like leading edge of agentic AI, I've been really impressed by some of the AI coding agents. So I think in terms of economic value, I feel like there are two very clear, very apparent buckets.

6:28One is answering people's questions. probably, you know, OpenAI, ChaiGPD seems to be more the leader of that with real takeoff, lift off velocity. The second massive bucket of economic value is coding agents where coding agents, like my personal favorite cloud developer to right now is cloud code. Maybe it'll change at some point, but I just use it, love it. Highly autonomous in terms of planning out, you know, what to do to build a software, building a checklist, going through it one at a time. So the ability to plan a multi-step thing, execute the multiple steps of a plan is one of the most highly autonomous agents out there being used that actually works.

7:06There's other stuff that I think doesn't work, like some of the computer use stuff, like go shop for something for me and browse online. Some of those things are really nice demos, but not yet production - Do you think that's because of looser criteria in terms of what needs to be done and more variability around actions? Or do you think there's a better training set or sort of set of outputs for coding? I'm sort of curious, like, why does one work so well or almost feels magical at times and others are, you know, really struggling as use cases so far? I think, you know, engineers really good at getting all sorts of stuff to work, but the economic value of coding is just clear and apparent and massive.

7:47So I think the sheer amount of resources dedicated to this has led to a lot of smart people for whom they themselves are the user. So also good instinct on product, building really amazing coding agents. And then I think, I don't know. You don't think it's a fundamental research challenge. You just think it's like capitalism at work and then domain knowledge in a lab. Oh, I think capitalism is great at solving fundamental research problems. Yeah. At what point do you think models will effectively be bootstrapping themselves in terms of 99 % of the code of a model will be written by agentic coding agents?

8:20Or the error analysis. I feel like we're, I'm certainly suspect we're slowly getting there. So some of the leading foundational model companies are clearly, well, they've said publicly, they're using AI to write a lot of the code. One thing I find exciting is AI models using agentic workflows to generate data for the next generation of models. So I think the lawnmower research we were talking about is where older version of lawnmower would be used to think for a long time to generate puzzles. that then you train the next generation of the model to try to solve really quickly without needing to think as long.

8:53So I find that exciting too. Yeah, multiple vectors of progress. It feels like AI is not just one way to make progress. There's so many smart people pushing forward in so many different ways. I think you have rejected the term vibe coding in favor of AI-assisted coding. What's the difference? I'm assuming you do the latter. You're not vibing. Yeah. Now, vibe coding leads people to think, you know, like, I'm just going to go to vibes and accept all the changes it cursor suggests or whatever. And it's fine that sometimes you could do that and it works, but I wish it was that easy. So when I'm coding for a day or for an afternoon, I'm not like going with the vibes.

9:34It's like a deeply intellectual exercise. And I think the term vibe coding makes people think it's easier than it is. So frankly, after a day of using AI-assisted coding, like I'm exhausted mentally, right? So I think of it as rapid engineering, where AI is letting us build serious systems, build products much faster than ever before. But it is, you know, engineering just done really rapidly. Do you think that's changing the nature of startups? How many people you need, how you build things, how you approach things? Or do you think it's still the same old kind of approach, but you just have people that get more leverage because they have these tools now?

10:05So AI Fund, we built startups, and it's really exciting to see how rapid engineering, AI-assisted coding, is changing the way we build startups. So there's so many things that would have taken a team of six engineers three months to build that now, today, one of my friends or I was just building on a weekend. And the fascinating thing I'm seeing is if we think about building a startup, the core loop of what we do, right? I want to build a product that uses love. So the core iteration loop is write software, it's a software engineering work, and then the product managers maybe go do user testing, look at it, go by gut, whatever, to decide how to improve the product.

10:43So when we go look at this loop, the speed of coding is accelerating, the cost is falling. And so increasingly the bottleneck is actually product management. So the product management bottleneck is now we can build what do we want much faster, while the bottleneck is deciding what do we actually want to build. Well, previously, if it took you, say, three weeks to build a prototype, if you need a week to get user feedback, it's fine. But if you now build a prototype in a day, then boy, if you have to wait a week for user feedback, that's really painful. So I find my teams, frankly, increasingly relying on gut because we go and collect a lot of data that informs our very human mental model, our brain's mental model of what the user wants.

11:26And then we often have to have deep customer empathy. So we can just make product decisions like that really, really fast in order to drive progress. Have you seen anything that actually automates some aspects of that? I know that there have been some versions of things where people, for example, are trying to generate market research by having a series of bots kind of react in real time. And that almost forms your market or your user base as a simulated environment of users. Have you seen any tooling like that work or take off? Or do you think that's coming? Or do you think that's too hard to do?

11:54Yeah. So there's a bunch of tools to try to speed up product management. I feel like, well, the recent Figma IPO is one great example of design. AI, Heidi and I, Dylan did a great job. Then there are these tools that are trying to use AI to help interview prospective users. And as you say, we looked at some of the scientific papers on using a flock of AI agents to simulate a group of users and how to calibrate that. It all feels promising and early and hopefully wildly exciting in the future. I don't think those tools are accelerating product managers nearly as much as coding tools are accelerating software engineers.

12:29So this does create more of the bottleneck on the product management side. It does make sense to me that my partner, Mike, has this idea that I think is broadly applicable in a couple different ways of like computers can now interrogate humans at scale. And so there's companies like Listen Labs working on this for like consumer research type tasks. But you could also use it to understand tasks for training or for the data collection piece that you described. When you think about your teams that are in this iteration loop, has the founder profile that makes sense changed over time? To me, there are so many things that the world used to do in 2022 that just do not work in 2025.

13:06So in fact, often I ask myself, is there anything we're doing today that we're also doing in 2022? And if so, let's take a look and see if it still even makes sense today. Because a lot of stuff, a lot of workflows in 2022 don't make sense today. So I think today, the technology is moving so fast. Founders that are on top of J &I technology, thus tech-oriented product leaders, I think are much more likely to succeed than someone that maybe is more business-oriented, more business-savvy, but doesn't have a good feel for where AI is going. I think unless you have a good feel for what the technology can and cannot do, it's really difficult to think about strategy where to lead the company.

13:48We believe this too. Yeah, cool. Yeah, yeah, yeah. I think that's like old-school Silicon Valley even. Like if you look at Gates or Steve Jobs slash Wozniak or a lot of the really early pioneers of the semiconductor, computer, early internet era, they were all highly technical. And so I must feel like we kind of lost that for a little bit of time. And now it's very clear that you need technical leaders for technology companies. I think we used to think, oh, you know, they've had one exit before, so two exits even. So let's just back that founder again. But I think if that founder has stayed on top of AI, then that's fantastic.

14:23And I think part of it is in moments of technological disruption, AI rapidly changing, that's the real knowledge. So actually take mobile technology. You know, like everyone kind of knows what a mobile phone can and cannot do, right? What mobile app is, there's GPS, all that. Everyone kind of knows that. So you don't need to be very technical to have a gut for, can I build a mobile app for that? But AI is changing so rapidly. What can we do with a voice app? What engineering workloads do? How rapidly foundation models do? What was the reasoning model? So having that knowledge is a much bigger differentiator.

14:53Whereas, you know, knowing what a mobile app can do to build a mobile app. It's an interesting point because when I look at the biggest mobile apps, they were all started by engineers. So WhatsApp was started by an engineer. Instagram was started by an engineer. I think Travis at Uber was technical-ish. Technically adjacent. Technical adjacent. Instacart at Purva was an engineer at Amazon. Yeah, and Travis had the insight that GPS enabled a new thing. But so you had to be one of the people that saw GPS on mobile coming early to go and do that. Yeah, you have to be like really aware of the capabilities.

15:24Yeah, you have to know the technology. Yeah, it's super interesting. What other characteristics do you think are common? I mean, I know people have been talking about, for example, it almost felt like there was an era where being hardworking was kind of poo-pooed, or do you think founders have to work hard? Do you think people who succeed were, I'm just sort of curious, like aggression, hours work, like what else may correlate or not correlate in your mind? You know, I work very hard. There are periods in my life where, you know, I encourage others that once have a great career have an impact, like work hard.

15:56But even now, I feel a little bit nervous in saying that because in some parts of society, it's considered not politically correct to say, well, working hard pretty correlates to your personal success. I think it's just a reality. I know that not everyone, at every point in their life, is in a time where they work hard. When my kids were first born, that week, I did not work very hard. It was fine, right? So acknowledging that not everyone is in circumstances and work hard, just the factual reality is people that work hard accomplish a lot more. But of course, you need to respect people that aren't in a phase where they - Yeah, I'd say something maybe a little less correct, which is less politically correct, which is like, I think there was an era where people thought there was a statement that startups are for everyone.

16:39And I do not believe that's true, right? I think you're trying to do a very unreasonable thing, which is create a lot of value impacting people very quickly. And when you're trying to do an unreasonable thing, you probably have to work pretty hard, right? And so I think people, I think that got very, the sort of work ethic required to move the needle in the world very quickly disappeared. Yeah. Was that a quote? I wish I remember who said this, but was it the only people that would change the world are the ones crazy enough to think they can? I think it does take someone with the boldness, the decisiveness to go and say, you know what?

17:14Just to say the world, I'm going to take a shot at changing it. And there's only people with that conviction that I think can do this. It strikes me as being true in any endeavor. I used to work as a biologist, and I think it's true in biology. I think it's true in technology. I think it's true in almost every field that I've seen. It's the people who work really hard, do very well. And then in startups, at least the thing I tended to forget for a while was just how important competitiveness or people who really wanted to compete and win mattered. And sometimes people come across as really low key, but they still have that drive and that urge and they want to be the ones who are the winners.

17:47And so I think that matters. And similarly, that was kind of put aside for a little bit, at least from a societal perspective relative to companies. Actually, I feel like I've seen two types. One is they really want their business to win. That's fine. Some do great. Some are they really want their customers to win. And they're so obsessed with serving the customer that that works out. I used to say early days of Coursera, Yes, I knew about competition, blah, blah, blah. But I was really obsessed with learners, with the customers. And that drove a lot of my behaviors. No, that's a really good framework.

18:19And when I say competition, I don't mean necessarily with other companies. But it's almost like with whatever metric you set for yourself or whatever thing you want to win at or be the best at. One thing I found is in a startup environment, you just got to make so many decisions every day. You just have to go by gut a lot of the time, right? I feel like, you know, building a startup feels more like playing tennis than solving calculus problems. Like you just don't have time to think, just make a decision. And I feel like, so this is why people that obsess day and night with a customer, with a company, think really deeply and have that conceptual knowledge that when someone says, do I ship product feature A or feature B?

18:57Like you just got to know a lot of the time, not always. And it turns out there are so many, to use Jetbase's term, two-way doors in startups, because frankly, you have very little to lose. So just make a decision that is wrong, change it a week later is fine. But to be really decisive and move really fast, you need to obsess usually about the customer, maybe the technology to have that state of knowledge to make really rapid decisions and still be right most of the time. How do you think about that bottleneck in terms of product management that you mentioned or people who have good product instincts?

19:26Because I was talking to one of the best known sort of tech public company CEOs, and his view was that in all of Silicon Valley or in all of tech kind of globally, there's probably a few hundred at most great product people. Do you think that's true? Or do you think there's a broader swath of people who are very capable at it? And then how do you find those people? Because I think that's actually a very rare skill set in terms of the people who are, you know, just like there's a 10X engineer, there's 10X product insights, it feels. Boy, that's a great question. I feel it's got to be more than a few hundred great product people.

19:59Maybe just as I think there are way more than a few hundred great AI people. I think there are. But I think one thing I find is very difficult is that user empathy or that customer empathy. Because to form a model of the user or the customer, there's so many sources of data. You run surveys, you talk to a handful of people, you read market reports, you look at people's behavior on other parallel or competing apps or whatever. But there's so many sources of data. but to take all this data and then to get out of your own head to form a mental model for what your maybe ideal customer profile or some user you want to serve, think and act so you can very quickly make decisions, serve them better.

20:38That human empathy, one of my failures, one of the things I did not do well in an early phase of my career, for some dumb reason, I tried to make a bunch of engineers product managers. I gave them product manager training and I found that I just foolishly made a bunch of really good engineers feel bad for not being good product managers, right? But I found that one correlate for whether someone would have good product instincts is that very high human empathy where you can synthesize lots of signals to really put yourself in the person's shoes to then very rapidly make product decisions and how to serve them.

21:12You know, going back to coding assistance, it's really interesting. I think it is like reasonably well known that the cursor team, like they make their decisions actually very instinctively versus spending a lot of time talking to users. And I think that makes sense if you are the user and then like your mental model of like yourself and what you want is actually applicable to a lot of people. And similarly, like I think, you know, these things change all the time, but I don't think Cloud Code incorporates, despite, you know, scale of usage, feedback data today from like a training loop perspective.

21:51And I think that surprises people because it is really just like, what do we think the product should be at this stage? So one advantage that startups have is while you're early, you can serve kind of one user profile. Today, if you're, I don't know, like Google, right? Google serves such a diverse set of user personas. You really have to think about a lot of different user personas and that adds complexity and the product changes. but when you're a startup trying to get your initial wedge in the market you know if you pick even one human that is representative enough of a broad set of users and you just build a product for one user that you have or one ideal customer profile one hypothetical person then you should actually go quite far and I think that for some of these businesses be it Cursor or Cloud Code or something if they have internally a mental picture of a user that's close enough to very large prospective users that you guys should go really far that way.

22:46The other thing that I've observed, and I'm curious if you guys see this in some of our companies, is just like the floor is lava, right? The ground is changing in terms of capability all the time. And the competition is also very fierce in the categories that are already obviously important and have multiple players. So leaders who are really effective in companies a generation ago are not necessarily that effective when recruited to these companies as they're scaling. like because the pace of, there's a velocity of operation or the pace of change. It's interesting to see you say like, I'm looking at what I was doing in like today and in 2022 and saying like, is that still right?

23:22Versus if you're an engineering leader or a go-to-market leader and you've like built your career being really great at how that's done, that may not be applicable anymore. I think it's a challenge for a lot of people. I know many great leaders in lots of different functions still doing things the way they were in 2022. and I think it's just got to change. When new technology comes, I mean, once upon a time, there was no such thing as web search today. Would you hire anyone for any role that doesn't know how to search the web? And I think we're well past the point that for a lot of job roles, if you can't use OMS in an effective way, you're just much less effective than someone that can.