Building the "App Store" for Robots: Hugging Face's Thomas Wolf on Physical AI

0:00many, many startups just already being built on top of LoroBot. Just, you know, they want to build something. They have this idea of a manual test they can automate or they have an idea of something that could do in a physical world. And then they take LoroBot. They take already like the basic building blocks with SHIPT, which is just a very simple robotic arm. S -100 that we designed basically to be the cheapest LoroBot account to be $100. And they already like trying to start business around this. At the bottom, you know, it's the again face editors, which is you bring all this platform, all this basic belly blocks for people to be already crazy things on top.

0:34A robotrix is the same goal for us.

0:52In this episode we interviewed Thomas Wolf, co -founder and chief science officer of Hugging Face, the largest open source community for AI. Thomas has consistently predicted the future, and he's responsible for why Hugging Face invested heavily behind Transformers and Language models, enabling the LLM way a handful of years ago. Now he sees the same opportunity for robotics and physical AI, and is helping to shepherd hugging faces low robot project, which brings together policy models, data sets, and physical embodiments to help developers everywhere become roboticists across the diversity of use cases and form factors.

1:27We're excited to chat with Thomas about the robotics explosion, and the bottlenecks ahead for physical AI, the US versus China in the open model race, and a lot more. Enjoy the show. Tommas, in your role at Hugging Face, you help Hugging Face invest behind moon shots as the Chief Science Officer, one or two years ahead of the field. You mentioned to me last time we chatted that, you know, you have a spidey sense that we're at the same moment for robotics today that we were for Transformers and Language models a handful of years ago. Tell us what you're seeing. Yeah, I think honestly, I studied two years ago.

1:59I mean, we studied activities in robotics 18 months ago. And I think at this time, there was a couple of breakthroughs out of, I mean, these labs, you know, were a way to stand for them, basically, these things. We're starting to show robots that were able to tie notes, fall clothes, you know, cook, like through things in the air on the pan and grab them. Basically, all of this thing, we've in one way very little data, but also with also good perspective to be able to leverage some of the world models and things like that that we see really benefit from internet size data. So all of this kind of pointed to a short future where robotics were going to work in a way like hardware was already there and in my opinion it's been there for quite some time but the missing brick was really software that could adapt, that could be dynamic all of that.

2:48And we started to see that and that's why we started to work on the robot a little bit more than 18 months ago. And I think for us the success of the robot, like seeing like this huge community. For us, the big beds was, can you build a big community in robotics as well? There was small community of kind of hobbies or like, or people very seriously building robots for like, you know, factory lines, but it was kind of like a tiny vertical in my opinion. The question was, could you move this tiny vertical to like a full horizontal thing? Just like nowadays, every software developer is kind of an AI researcher almost.

3:22They all want to know how, you know, LLM works, how you train them, and there is this very smooth transition where the 100, 200 millions of software developers are becoming kind of AI aware. And I think there is a potential future transition where all of these people also become robotist in a way, if you give them the tools. And so that was kind of our goal. So we started with the software library and the success of that kind of brought us to also try to go in hardware, which is a big question as well. And that's why we acquired first hardware company, DC Apple and Robotics. And we shipped our, At least we opened the orders for our first robots.

3:56There was mid -Julys, that's one month ago. Really? And this went crazy. Can you tell us what LeBer robot is? Of course, yeah. So LeBer robots is our attempt of doing again the success of Transformer, reproducing the success of Transformer's library in the robotic field. So this idea is having kind of a thoughtful library that everyone would use, and that would bring in a very simple and accessible and easy way, all the latest technology, all the latest algorithms that people use to train robots, efficiently all the latest data set that they use to train this and also connect these to actuators, which is the hardware part of robotics.

4:36And that's this intersection and this mix of three aspects, the policy, the models, the data set, and the hardware which we try to combine in robots. But how does the role of hugging face change in robotics? For people building in the physical world, does hugging face play the same role or a different role than hugging face is played for people building in the digital world with LMS? I mean, our goal is to play the same role, which at a very, very high level is building communities and bringing people in society that AI can be open source and it's not something you only consume, but it's something you can tweak, you can train, you can control, you can host where you want.

5:14And actually hosting where you want is even more important in robotics because in the future where you have robots everywhere You kind of want a lot of these models to run locally because if your robot lose connection to the Wi -Fi or something and then run in the wall Or maybe I don't know run in your kids. It's gonna be much more dramatic than you know, just an LLM Elitinating so safety question in robotics. I think I have a good reason you may want to really be able to not you know depend on the distant API but have the models as possible to the hardware. So I would say our role is maybe even more important for safety and ensure for robotics than it is in 11.

5:52Could you say a word on the size of the community you have at Lerbaat? How many people are building, how many people are contributing data sets and things like that? Yeah, for sure, should have checked actually the latest number because it's exponentially growing. So it's several thousand people, I would say six to ten thousand. One event we did a couple of months ago, we did a hackathon, which was worldwide. We had a hundred locations in over six continents. So it's still not in the million of people, for sure, but it's way above several thousand people. And the main indicator for us is we can measure the number of data sets, for instance, on the hub.

6:26What we see in all of this thing, number of community members or data sets, we see kind of this exponential growth, which is, I think, very good indication that we are on the right track. And you have to keep in mind that the hardware that's available right now is still very much kind of a hobby hardware. So it's like 3D printed arms. It's like they're still wired everywhere. And that's why if starting this summer we wanted to bring much more mass market hardware I would say. So something that would interest not only the hacker and the people who are used to plug cables everywhere but also everyone in families, something that looks much more polished.

7:03What's the persona of developers in your lower bot community? And I'm curious how it's the same or difference from the people that have traditionally been building, you know, like controls, classical base systems. I would say there is three, three type of person now. I mean, one is the, the traditional robot is this, they think you want to use AI. So for a lot of them, they know how to build hardware, they know what they can use, but they've been frustrated by, you know, the limitation of the software stack, all the optimal control models. And all these are very limiting what you can do. So all of these are people who have really happily joined the band bag and we see many, we see the same effect we saw in transformers, which is many academic labs starting to use robots because it's it's it's a very nice entry point for all less students.

7:47And so and so this has been growing very strongly. The second community is is is much more interesting in my opinion. It's people who were not really into robotics, but because they they're into AI and robotics looks like a physical manifestation of AI, I kind of want to go into robotics. And so these people that covers, I would say, software developer, but even people who are just interested in robotics. So a good example of talking here is interesting, but a lot of investor have actually bought, you know, S -100 arm, just to try to understand physically, what is this robotic thing, what can it do?

8:19And because it seems so accessible, you get the arm and the software is just a Python curve. And now, you know, with a little bit of vibe coding, you can actually even tweak it or control it quite easily. We see people who maybe are not purely technical, but who want to understand what's happening in robotics and they use this entry point, which is low robots. So you can vibe code a robot? Yeah, let's read my goal. So you can already do that a little bit, but for the new robots, reach the meaning, I definitely want this to be one of the easiest way to use. I would love my kids to be able to vibe code a girl on the robots.

8:51What sort of phase of maturity do you think we're in for the robotics market writ large? You know, like, what do we have a chat GPT moment in the world of robotics? Yeah, that's the thing I'm looking for. I like sometimes called it also the iPhone moment. Maybe what would be the first use case? The first, you know, the first moment where everyone or like a large fraction of population will we think I want to robot in terms of consumer. I mean, I think the enterprise market is quite complex. There is in some place there is already a lot of robots in some kind of industry. The car manufacturer is a basic sum.

9:23Then there is this second part where there is like an entry of robots and here there is a lot of challenge around reliability. Will these robots be reliable enough to be deployed in retail store and basically be really useful? And the third part I'm much more interested in is actually entertainment, fun, demo, education where I think maybe this question around, I need a $3 ,000 robots because I need the our less pregnant and so you can take a robot that's really accessible. So which menu, for instance, is priced as $300? That's something that can definitely be like an impulsive buy. You buy it as a gift and you're not sure it is going to work on that.

10:06But for this price, what we want to find is, is there not a lot of potential in more entertainment, fun, education, learning AI through a physical interaction stuff? Instead of just coding on the chat but are coding on the screen. And I think that's something that has not been explored at all. I think there was a couple of tried, I mean, the mid media lab, since you had the G -Ball, for instance. But in the past, they were using their price to be high, I think, above 1000. And more importantly, I think the software that was there was very limited. So you would buy a robust, that would be firm, but you had maybe five or 10 behaviors.

10:48And once you've tried them all, that's it. And it's finished. And here the goal for each chimney is really to make kind of almost like a smartphone. So it comes, you have a couple of behavior, but just because you can tweak it and people can build new behavior and share them and plug all the new VLN speech models, chat model, activity to possibility, arcane, endless. It's kind of a open door on like rebuilding, you know, the up store of iPhone basically. So that's what I'm very excited. And to be honest, these last parties, they're very much a bad because nothing exists there. Like there is no, real proof.

11:21My major sense are all this exponential growth of community which make it quite plausible. So you see Richie Minnie as like the you know the re - incarnation of the robot dog dogs of the 90s was like how people can actually play and experiment and you know have like these robot companions and households. I mean to be fair this one is a big bet but something I was discussing just yesterday actually on robotics at Tech Babacu. Someone was telling me as an investor, you know, how just is so many, many startups just already being built on top of LoroBot, just, you know, people who are like, they want to build something, they have this idea of a manual test, they can automate or they have an idea of something that could do in a physical world.

12:05And then they take LoroBot, they take already like the basic building blocks with SHIPT, which is just a very simple robotic arm, S100, that we designed basically to be the cheapest robotic arm to be $100. And they already like trying to start business around this to start like to say all that something around this and Richie Minis also in a way designed for that it's a very wide simple robot kind of a white labeled thing and if you want to adapt it and if you think hey I have a business idea around this but I need a robot to interact with people I don't know in hospitals or something like that you can take this and you can actively start to build your A.

12:40And let's get at the bottom, you know, it's the again phase ethos, which is you bring all this platform, all this basic building blocks for people to build really crazy things on top. And robotics is the same for us. Really cool. I'm going to talk about data as a bottleneck. Like I think one of the big differences between language and robotics is you have trillions of tokens out in the public internet to train all amps that dynamic doesn't exist in robotics. And actually I think that's where hugging faces role and ecosystem could be much more interesting in in terms of decentralized data sets, curation, creation.

13:13Talk about what's happening on the data set side of LoroBot. Yeah, it's super interesting, I think. So, I mean, there's a couple of challenges in LoroBot X. And I mean, the general, the main challenge is in the data. There's just not enough data. There is some ways to use video on the internet as training data, but it's very limited. And in some way, we may be able to use models, but in some other if you want to automate a test, there is no way around just recording someone or the robot possibly just doing the task. I think here there is one possibility on one limitation. I mean, the main limitation is you can record a lot of tasks yourself, but usually what you will lack a lot is the diversity.

13:56So you will basically be able to train a robot to do something very well in your room when everything looks the same, but once you put it in the next door room where maybe the walls are green instead of red, the robots has a lot of troubles to general likes. So this is the main limitation. So our idea with the herb was that everyone could record data sets and if we managed to insensivize them to share the data, then we could maybe build a very multi -colored data set that would be extremely diverse. Any addition hopefully would be also very big. So I would say that's a long -term goal. We hope this can help.

14:35But another more direct thing we try to do is to work also directly with the actors of the community. So we release a couple of data sets to try to help them releasing some data set. We think in robotics, one of the nice aspects is a lot of people in the end want to sell the hardware. And so they can actually afford to even modern LLM. They can afford to share a bit of the software as open source if it brings all the field above because in the end, that's not really directly what they sell. So that's kind of what I'm trying to convince a lot of robotics company to do. And surprisingly, it's actually something a lot of them seems to be interested in.

15:13Super interesting. You tweeted out world models today. I think you and I met the same world model founder. What's happening in the world model open source and how does that help or not help? What's going to happen in robotics? And can I ask on what to do? Is there a why now in world models at the moment? because it feels like they've started to pop up recently. So what is interesting is it's, it feels like it's a couple of teams who have been actually working on that independently for, for Fumarth and they just happen to release this right now, right? Because when you talk about all of them, they are not really coping each other.

15:45I mean, I guess one thing was the, the advent of really cool, really good image generation and basically finally understanding how to fix this six finger things and basically just get a more reliable and more coherent world model for image, which naturally was transposed to video. And so we see now some really cool video models as well. And this one is just one next step. And a lot of the founders I've talked to in this field also say that they were helped by the advance of like open source video model generation and open source image generation. And basically they take this video generation model and then they fine tune them and then they train them to be able to react to some inputs, which is also what we do in robotics.

16:29There's a lot of common points between these two things. And it seems to work quite well. So you start to have this kind of very interesting in my opinion, totally new experience where you actually have a film that's controllable. that's like both photorealistic and also reacting in a very coherent way to the action you will input, which is either just moving around or just asking it to add something, you know, add a rider, a castle, a car driving, and you see this thing that just reacts very well. And you have here, I think, a lot of potential applications, you know, both obviously an entertainment, actually some form of entertainment that might be just totally new, something we've never seen, which is maybe the first time we create a new form of really unusual entertainment, but also a lot of application in business and how you can have interactive things.

17:19And one of these downstream applications is generating more data for robots. I mean, there is just two ways to generate data. One is to record it in the real world, which I think is still very interesting. And the other is to simulate it. And surprisingly, on the simulation, we have not seen a lot of, you know, I mean, There was some development, but it's not like they've been some really breakthrough recently on simulation. So maybe this is the first breakthrough I've seen on simulated generated data in quite some time. Yeah, I was very excited to see some of the, even like what DeepMind's doing with Genie to train their Mollied robots.

17:53Super exciting. Humanoid. Do you believe in humanoids as the kind of ultimate form factor? Yeah, big debates, big debates. What is sure is that I'm quite more excited about trying other form factor right now. I mean, the main problem with humanity, I think there is too many problems. The first one is it's always quite expensive, just because you need a lot of models and all the price in the robots is just the actuator. That's always like 70 % of the price tag. So when you have six actuators, that's just your bill is the year. And so it's really hard to drive humanity below the price of a car. I think the price of a car is still already quite a high requirement.

18:30If you buy something that's a price of a car, you do expect to have a lot of value out of it, right? And so that's why we're exploring smaller robots, like just one arm or just moving head and stuff. There is some possibility that we can get cheaper humanity at some point. And K scale was trying to do. You need trees they've been trying to cut the price and there's a lot of company trying to aim, but it's gonna be really hard, I think to get this under like 10k, 10k, $10 ,000. The nice thing about the humanity, of course, is once you've sold humanity, you solve like a lot of tasks at the same time.

19:04So if you solve the imaginary, you can do everything a human do, which is very exciting. And the main question is do you need to solve the human reads? So on my side, I'm more like I would like to see a galaxy of different form factors. I also think some of them are much more cute than a humanoid. I think for social adoption, I think the humanity is also asking a lot from people. It's like this, you're directing this kind of uncalee value with something that looks a lot like you move a lot like you. So I thought this would be a big limit for social adoption. Now to be honest, I've seen a lot of Unitry robots and then about you, but you kind of ignore them at some point.

19:41So I'm also much more confident on the people who just say, yeah, that's just maybe we too worried about it and can be very in robotics and maybe at some point once we start to, I've seen like a couple of robots, people we just accept them very, very easily. Okay, so we're going to see the low robot humanoid soon. I mean, the goal would be three chimini and our small robots work really well. At that point, we'll climb back to make it a humanoid form factor, which I kind of progressively as we've done, bringing the community along with us. As you imagine, the world in 10 years, like, how many robots do you think there are among us?

20:16Do you think it's like 80 % of the more humanoid than 20 % of this long tail of this diversity of hardware in use cases or like, how do you think the world plays out? Yeah, and I would love to see the second option because I think that's an option where we have much more robots in our life. What I would really not be super excited about is the future where kind of robots are kind of a elite thing because they cost 100 ,000 and so basically if you're rich, you have three robots at home and if you're not, you don't. I mean, hugging faces always always been also about the big communities that we care about that.

20:48And so for this reason, and much more excited to see a lot of form factors that are basically accessible to a lot of people, some of them are cheap, some of them are expensive, then just this single humanoid that cost a lot and if you can buy, that's nice and if you cannot do that for you. So I would say, I talking phase as the future we try to manage to first or I think also is much more fun because you in a way you're also restricting yourself, you're just like LLM if you just try to make them copy you man is one thing but if you try to think maybe they can do something that you and cannot do, it's also much more interesting in a way.

21:22Do you think we're heading towards a world of big foundation models that can kind of do everything and then be adapted quickly to any new domain with just like, you know, a few prompts or do you think that developers in your community are going to start from like a small base model and then do a lot of their own data collection, customization to adapt to their domains? Hmm. I think we'll see more and more both. I mean, I think as the field evolve, you know, we start to see really a long tail. So, for instance, if we take the downloads on Hiking Face, we see both very large state of the art models being downloaded, which are usually too large to run on a local laptop.

22:01But we see also some of the most downloaded models are actually just the right size that fits to run quickly on the laptop. So we see these two like really modality. And I think as the fill can match here, we'll start to see this more and more, which is it's not like you choose one or the other, It's just depending on what you need. You may choose locally or not. And I think GPT -5 with the router is a good example of this. Maybe the largest model are the most reasoning. The longest reasoning chain is not the answer to everything and you actually need to smartly select the one you want. So it can be behind the router.

22:36But it can also be just locally. You will run some models here. They might be extremely useful. And we know better and better how to train model that actually is extremely useful. But when you need something much more complex, when you need reflection for a long time, then you will turn to much larger models. One of the narratives that's been really popular over the last few years is this narrative of the battle between open and closed, you know, closed models versus open models who's going to win. And just in the last few weeks, OpenAI is now present on hugging face. And so I'm curious what to make of that.

23:11And sort of what it might imply about the future of open versus closed or maybe how they work together. I mean, we're super happy to welcome the back. They were there. I mean, first model, our work done. And the reason we switched from being a game company to an open source platform was GPT -1, which not a lot of people remember, but it was very funny because it was trained mostly on novels and romance novels. And so when you would put two characters in the continuation, they would always fall in love and some way on They shoot still miss a little bit this one and then and then Google took this idea and train it also on Wikipedia Which had a lot of world knowledge and then he expanded to GPT all of that But at that time there were very very pro open source and I think I think open source just like in software I think both both solution which is we just go exist and having company that open source both or that do both I mean, Google has been an example for quite some time, right?

24:09With the Gemaline and Gemini line. And some interesting moments were some time I heard that one Gemal model was actually so good that it was better than closed source models. So they had to not open source it. So the front here, what is sure is quite seen at the moment in a way. And the challenging new players mostly in China, but I think we start to see also some new foundation models team in the US. So I think we might see also some challenger in the US. The frontier we stay quite seen, I think, and both both think we'll see with a tiny difference of performance. The main reason right now, I think, is to be honest, at this exact point in time, I think we are not exactly in a kind of cost saving time of AI.