Building AI Agents for Enterprise Operations

0:00Luis Paarup and Olivia Moore and Anish Acharya. Voice was the unlock to many of the operations that are really needed to move the world if we talk about supply chain. This is not a supply chain specific problem that we are solving. It's actually an enterprise coordination problem. The bigger problem in the coming years for like Voice AI is really knowing when to talk and when not to talk. So it's understanding all these nuances in the work more than making the latency faster or making the voices more realistic, which I don't think that's a limiting factor today.

0:29Olivia Moore:I feel like Happy Robot has always been at the forefront of kind of humanness. Do you want the customers to know they're talking to an AI? Where does that go?

0:38Anish Acharya:I think it's super important that most AI demos happen in controlled environments. The real challenge begins when AI has to operate inside large organizations, where information is fragmented across systems, teams, emails, phone calls, and workflows that have evolved over years. Logistics and supply chains have become an early proving ground for these systems. Success depends not just on model intelligence, but on coordination, context, and the ability to execute work reliably in the real world. Anisha Charya and Olivia Moore speak with Pablo Palafox and Luis Parra from Happy Robot about voice AI, enterprise agents, and the challenges of deploying AI in operationally complex industries.

1:24Anish Acharya:Olivia and I are here with the two incredibly talented founders of Happy Robot, Pablo and Luis. Welcome, guys. Thank you, guys. Super excited. Very excited to have you. We're overdue to have this conversation. Well, look, we're here to kind of talk about the company and the incredible journey that you've been on. I know when we first met you, there had been a lot of buzz amongst YC founders and other folks about how you guys were sort of at the edge of the technology and then really getting a lot of pull from a go-to-market perspective. So maybe take us back in time to the little office that had four or five people on 20th Street and what the origins of the company and the product were.

1:58100%. So Luis and I met on our second day of college, just to set the scene, ever since we've been building stuff together. Our other co-founder, Javi, he happens to be my brother, so I've known him for a little while. We always wanted to build something together, right? So when we got into YC, we were looking for complex problems we could solve. Keep in mind that Lisa and I had been literally building submarines for robotics competitions to find mannequins underwater. That is the sort of problems we were looking for. So when we decided on solving for that complexity, we looked at what Javi was doing as CFO of the largest olive oil distributor in the world.

2:35He was literally moving tons of olive oil across the ocean. And that was that complexity that drew us into logistics and supply chain. He literally had to hire interns to call drivers to see where they were, to see where the shipment was, because Walmart was asking him, where the hell is my shipment of olive oil? So that was the sort of problems that we wanted to tackle. And maybe you can talk about why we actually started with voice there. I guess we took it from a very tech driven approach. Really, the limiting factor back then was having an agent that could speak on the phone realistically.

3:07Like we were in conferences, Javi was like traveling all around, like asking people, hey, if we were to create a voice agent that could pick up the phone and sell these loads and track these shipments, would you buy? And it's like, dude, of course, this is a no brainer. I just don't think you can do it. So it was more so like the idea market fit or product market fit made sense from the beginning. It was more so like, can we prove ourselves we can build this technology? And, you know, LLMs were picking up. We're talking about late 2023, probably LLMs were like decent enough. 11th was picking up with the text-to-speeds and everything was kind of working together, but we had to build something that could actually connect all the dots and actually make something work, no?

3:42That kind of shaped our company where we really had technology and innovation as the core of our company and always pushing this frontier, no? And solving problems like firsthand. So that's how we got started in the voice phase.

3:53Olivia Moore:Amazing. One of my favorite memories of working with you guys is actually when we first met outside a very crowded coffee shop and you called one of the live voice agents and it was seamless And it did an incredible job in a very non-ideal environment. I feel like a lot of people might know Happy Robot from your amazing demo videos of the voice agent. And that's definitely not all the product is, but it's an important part of it. So maybe walk us through, like, why voice to start and then what voice is maybe unlocked for you more broadly. Yeah. What Luis was saying is very important. Voice was the unlock to many of the operations that are really needed to move the world if we talk about supply chain.

4:33So when we were going to these conferences and people were like, no, you're going to build these things that talk on the phone. Negotiating rates on shipments was actually a big one. So we actually fine-tuned LLMs back then. Like we fine-tuned Mistral and Llama to actually make those voice agents faster. Because otherwise using some GPT-4 at the time was like extremely slow. And GPT-3.5 at the time was like terrible at reasoning and actually negotiating. So we had to do a lot of tricks behind the scenes, build our own agent infrastructure, if you will, but also build our own voice agent capabilities so that we could innovate faster than competition.

5:12And that actually gave us a really good edge in logistics and transportation in the early beginnings. So we started working with these freight brokers. Then we expanded to these freight forwarders, then ocean carriers, then trucking companies. And today we actually serve many of the largest companies in the space of supply chain. We were discussing before, nine of the top 10 freight brokers. in the U.S., seven of the top 10 tracking companies, like some of the largest fleets that actually move our goods everywhere in the U.S., which is crazy. Two of the largest ocean carriers, those big boats we see in the bay, that is the sort of customers that we needed to build for and where voice was the analog for many of the operations.

5:48Anish Acharya:So it sounds like it wasn't just voice. It was also voice plus negotiation. So perhaps track and trace, which is customer support and sales, which is sort of this negotiation is where we started. And I think that forced us to build a deeper set of technology than we otherwise would have built. Maybe Luis, take us on the technology journey a little bit. Yeah. So before I tackle that, I guess one of the things that we had very clear from the beginning when you're working on the frontier of technology is really what you have to reinvent versus what already exists, no? And I think people might take an approach where they just reinvent everything just for the sake of it.

6:19Some people would just wrap around anything else and be like more of a go-to-market thing. we started like tackling the limiting factor always and again back then gpt paulo mentioned like 3.5 was relatively fast and not so good so we had to fine tune the llm soon enough we realized that prompting and all these good models came out prompting was good enough scratch that let's do that and always focusing on that limiting factor then voices like the background noises like supply is extremely messy you're talking with drivers in their trucks with the radio on and background and music and noises and accents.

6:52So always focusing on those limiting factors. On the negotiation part, something we got very often was how do you prevent the bot from hallucinating a raid or like max buy? It's like, dude, I'm building this thing and it's just hallucinating max buy and it doesn't know how to negotiate. How are you guys able to do that? And I think it's because you don't need to show the AI what it doesn't need to see. And I think we're very opinionated about this from the beginning where we're building these proxy servers and actually exposing to the agent only the things they need to see. and actually max buy the max amount of money the bot can actually see or actually negotiate is not even exposed to the bot.

7:26We were not exposing that. We were doing external negotiation algorithms so that the bot would just ask for permission, literally the same way a human would like, hey, let me ask my boss. And it was really just calling a tool and asking for permission to do more. And we would inject back the rate, no? So those sort of things, instead of like just putting in the context window and having the LLM just freestyle it, we do it in a more deterministic approach. So it's always that mix of probabilistic plus deterministic where you need to let everything to the AI, no? It's building for the real world.

7:53The real world is messy. Those things are going to happen where someone tries to jailbreak the agent and get that max amount of money that they can get. But we needed to build those guardrails very early on so that we could actually go to the likes of C.H. Robinson or Uber Freight and all of these big players that would only trust us if we actually were building real technology. That was pretty clear for us. We knew that we didn't want to focus on the long tail of logistics and transportation because it's a very tricky space, but we knew we needed to serve the enterprise in transportation and supply chain and logistics.

8:27So that was very clear. That shaped the type of products that we had to build, the type of primitives that we had to build.

8:32Olivia Moore:It's so interesting because Happy Robot was very early to both voice AI and enterprise agents more broadly, which is great. And also, it's like the ground has been shifting under our feet because the models themselves are kind of changing and evolving so rapidly to your point about fine-tuning versus prompting versus kind of what to do next. Maybe we can talk through a few of the use cases you have where it's very clear that a smarter model by itself doesn't just do it and like why you need to buy a platform. I can bring up the Kuninagel use case. We recently announced our partnership with the marquee freight forwarder, great partners.

9:12I was having a personal lunch with their head of air. Shout out to our friend Imwe at Kuninagel at his house in Spain. And what I learned from their operations is that this is not a simple customer service type of create a ticket in Zendesk and you're done or you reply based on a knowledge base. Customer support for these real economy industries like logistics, transportation, freight forwarders, broader supply chain, even other industries like the telco space or the utility space. It's not as easy as just replying based off of a knowledge base again. There's a lot that happens afterwards that really has to get done to provide that update to the customer.

9:55So example, freight forwarding, Kuninagel, they are serving customers, very large customers, I cannot name who. But imagine that you are a big customer of Kuninagel and you ask, hey, where is my shipment? What happens now is an agent has to turn around and go find it. That go find it is very complex. You need that coordination. You need basically an orchestration agent that is, okay, this is an air shipment. So obviously relates to airlines. Who is the airline on this shipment? Okay, let me go to the airline's website. So we have browsing agents that go and scrape the website of the airline. Oh, bummer.

10:31It's not there. There's no update. Damn, I need to go send in an email. Okay, I'm sending an email to the airline. Two hours later, no reply. Okay, I need to reason that if they don't reply now, I'm going to miss my SLA with my customer. So now I need to call them. I'm going to keep calling them until someone picks up at the airline and tell me where the hell is my shipment. So that is the sort of coordination that we need to make happen for transportation and logistics, really. And that has shaped the type of product that we had to build at NIV. Yeah, no, I subscribe everything. I guess another example on the negotiation, which is always like how we started and all these demos went pretty viral.

11:13And I guess one point of how raw intelligence really wasn't enough is when you're negotiating, for example, loads and there's like 10 carriers or 10 like buyers calling at the same time. You cannot have all those agents like doing work independently, which is what happens to a certain degree with like humans. They're on the floor and sure, they shout to each other and they're like, hey, this is a very hot load. Please negotiate hard. I have someone interested, you know, all this information is really not in the model. so what we started doing is when you have inbound calls for the same load you can start like sharing context across them like hey i have someone they're very interested please push harder like this is a hot load please also all this information sharing is literally what you put in the context window at any point in time like general intelligence or the raw intelligence doesn't really know if someone else is calling on that load so it's all that about like what do you know about the business what do you know about the negotiation strategies maybe you know that pushing harder on this load because it's like cross border is going to be better or whatnot like that's not general intelligence that's very specific and different enterprises operate differently like you cannot just build an agent fine-tune it and have it work at any type of company all those nuances is outside of the model it's that context layer that we're trying to create no and that actually like we can talk about how actually doing the work and executing the work is what gets you that it's like learning by experience is you do something and you learn and you explore that space of the context layer so that you can keep learning, no?

12:43Anish Acharya:Really interesting. So you talked about two different things there, Pablo. You just talked about a very cross-functional workflow. Luis, you talked about the complexity of really mastering sales, you know? You guys started as sales and support. And so what are some of the other surprises that you've had having started with more complexity, I think, than some of your competitors? So one thing that we heard from one of the largest tracking companies recently was typically when we buy technology, we see where we can apply that. With you guys, we actually have a problem and we come to you guys with that problem because we know that as a platform that you've built, we can pretty much build any type of agent for our operations from sales to customer service, back office support and operations and even collections.

13:28So some of the use cases that customers came to us were, hey, we have a huge collections problem. Can you build an agent to reach out to customers via email or voice and collect money? We're like, of course. We talked about this use case with one of our largest supply chain companies and customers where we need to call customers to recover duties on parcels. And today we're running campaigns of 20 ,000 to 50 ,000 daily outreach to customers, collecting duties on parcels that otherwise they would not get if they don't pay the duty on the parcel. So that sort of surprises, if you will, we've gotten from customers.

14:11Like, yeah, I also need to recruit drivers. Can you do that? And we obviously can build an agent that not only just recruits drivers, actually connects to the operation so that now they know they can service a truck with a customer earlier because now they have a driver to move that. So there's all sorts of interesting connections between the functions. Maybe I'll give you another example. We built an agent to reach out to maintenance shops to see where a truck or when a truck was ready. You could just leave that agent in a silo and just have an agent that is proactively reaching out to those repair shops to see when the truck is ready.

14:50Well, it turns out that the sooner you know when the truck is ready, the sooner you can put it in the market to sell it as capacity for your customers to actually move things. So that was a very interesting realization of how sales in this case and maintenance were tightly connected. So that is the context that we talk about. There has to be an underlying context sharing across the different functions in a business so that the whole business optimizes for a global maxima, if you will, or a global minima, depending on what optimization problem you're trying to run versus just minimizing the problem in one function, if that makes sense.

15:28And then maybe can you talk about,

15:29Anish Acharya:like, how do you discover these workflows? Who discovers them? Who builds them? How do they get built? I mean, maybe, Luis, talk a bit about that. Yeah. So we're very forward deployed. So we very early on understood that, really, to solve the customer's pain point, we had to build software that adapts to their operations and not the other way around, which is like the old era before AI was you build something and ask people to like run their business however you think they should be run but we think it should be the other way around so from the very beginning we started like hiring and building this forward deployed motion with the FDEs for deployed engineers like everyone is talking about them now but I think it's about like really being customer obsessed and really focusing on like the value add and their problem and really sitting down with them and going to their offices and learning what they need.

16:19So sure, there's a lot of like synergies in the industry and what you learn from a customer might be relevant to another. But very soon we realize that there's not a one size fits all, even within inbound care sales, even like within this workflow in the enterprise, maybe like there's like a long tail where this might apply, but enterprises operate very, very differently. and that's why we build a platform that is flexible enough to adapt to anyone's operations. And it's because we were trying to like plug and play what we built somehow with a customer to another one. It didn't really work. Like they want something different.

16:54They want to change the procedure. They want to call these tools. They want to escalate whenever the carrier is not vetted and someone else wants to do it automated. So we really had to build up almost like horizontal technology because of the variety of all the nuances in this industry, you know? And that's how we create a platform that is not optimized for like specific tasks, but more so optimized for like doing work. So our primitives are around workflows and data and integrations and, you know, SOPs, prompts. You don't see like particular tasks being modeled because that's almost too opinionated.

17:27And customers don't want like opinion, like their vendors forming opinions on how to run their operations. Like they've been running it for a long time. They don't know more about their business than I do. I just come with the technology and I just want to come to solve their problems.

17:40Olivia Moore:I feel like the forward deployed motion has been crucial for AI application layer businesses. And also it's prompted a lot of questions about what are margins? What is like a service versus a product? Kind of where is the long term alpha and moat? Maybe walk us through how you think about productizing the work that your FDEs do, which I think is kind of a unique strength, happy robot. where does the forward deploy motion start and end? Do you do custom work for one customer? Would love to understand how that works. Maybe let's start in the beginning. Yeah. I was the first forward deploy engineer without knowing it, I guess.

18:20Yeah. Which is pretty much what any founder would do. You just go to your customers, spend a week there and just chase down the people that are actually doing the thing that you want to help them automate, right? So I did that and I would be like pinging these guys like, dude, you need to build this thing because it's going to make my life a lot easier and it's actually going to be replicable across customers because I've seen it. So please build it. And he would be like, really? Do I need to build that? So there was that good tension between kind of that forward deployed motion and the product team.

18:50So we kept going with these like separate worlds for a little bit where I would be like leading the FDE team and the deployment strategists that we realized at some point we actually needed. That was a bit of a realization, a bit of a parenthesis here. we started just with forward deployed engineers and then they're like, the customer's like, wait, you have these people building, but like who is managing? I'm like, I guess that's like the deployment strategist to some degree. So the deployment strategist is a figure that scopes the problem so that the forward deployed engineer can spend more time on building.

19:21Although now what we see is that the right FD or deployment strategist, they have to be very cross-functional, close parenthesis on the type of profile. So what ended up happening is we were too disconnected from the forward deployed world and the product team. So we realized that that needed to be part of Luis's world so that the FD team would actually be an extension of product, which is what they should always be. It's an extension of product so that we can implement product faster, we can gather the feedback faster from the customers, and hence capture that context faster than anyone else.

19:57So it's a bit of this iterative loop that Luis really realized we needed. Yeah, I'll add to that. I mean, if we go with first principles, what are we doing? We're deploying agents across different functions and channels in the enterprise. So our product is built for the deployment of an agent. We really understand the deployment lifecycle because we work very closely with our customers, and we are actually deploying these agents. And something cool that happens is that you have, to a certain degree, your user in your house, because we're building for the FDs for the most part. And it's not entirely true.

20:35Of course, some enterprises, they really appreciate having a platform. And we can talk about that later, about how interesting the mix of coming with a platform they can also use and an FD that they can trust is actually something very rare. And they mentioned that. But I guess to the point of the deployment lifecycle, we really understand what it takes to deploy an agent. There's a scoping phase. There's a building. There's testing, there's monitoring, there's like a self-learning loop. So I guess the point is every feature we're building in the product is optimized for that deployment lifecycle.

21:05And the only way to know if that works is being very close to the deployment. And FDs are doing these deployments or they're getting feedback from the other team. So actually, more than the FDs being very close to the product, I think it's more so like our product is a combination of a platform and a forward deployed motion, and it would really not exist. and there's like this conversation about like services and stuff the difference is that the forward deploy engineer are like catalysts or accelerators to value but what we're leaving in the customer are like agents running there's a platform once the fds are have done their work they leave a working thing so it's almost like you spend that time you deploy the thing but the what you're not delivering like an output that the fde has done you're literally delivering the agent working on a platform no so it's a very different distinction of pure services versus like a forward deploy implementation plus the platform running the value forever, hopefully.

22:02Awesome.

22:03Olivia Moore:I feel like another thing that has been a topic of discussion is kind of what is the value of systems of record in the AI era? Does every application company need to become one of those? And I know you at Happy Robot have a view on kind of systems of record versus maybe systems of action or systems of execution. So we'd love to talk through kind of your view on that topic. Maybe I'll start quickly. We see ourselves as that layer of execution, really. Like that's where the magic happens. You have to start doing the work to capture that context. So it's very important that we start with executing work, with getting the thing done, implementing one agent, implementing the second agent, connecting them through that context layer.