Franz Tschimben (ALLSIDES) & Michael Brehm & Ben Scheidt (Redstone): Why physical AI needs 3D data

0:00We're now setting out to build the largest ever created data set and data infrastructure to then let anyone train on top of that within our platform. You'll use some of the larger models for coding. You need other ones more for the physical world. So that's a huge trend where it becomes pretty obvious that one size fits all doesn't work. I love this thesis about converging from the bits level to the atom level, which is basically enabling AI to really do stuff in the physical world. This will leap humanity forward, but we are not there yet. For these robots to work, they basically use 3D data as their native language.

0:45The core bottleneck today is not more parameters, it's better reality-grade data. And while we get this quality that is unprecedented, at the same time, these machines crank out these assets at five minutes per object, which is unheard of. A lot of these companies who are now hyped in the VC field will have troubles keeping up the high valuations that they raised money for. And what will really count in future is still defensible USPs.

1:13Andreas Munk Holm:Welcome everyone back to the European podcast. Cast, today, I want to say that we've spent so many years obsessing over models. Who has the best one? How big is it? What benchmark it tops? And I think that that conversation is more or less over because models are becoming commodity and that's happening fast. The real question now is what do you actually do with them? And more importantly, what do they still fundamentally lack? The answer, at least for anyone building in the physical world, such as robotics, simulation, autonomous system, is we need grounding. We need to ground the models in real-world data.

1:45Andreas Munk Holm:We need models that are not just scraped from the internet, but that are captured from reality itself in 3D. That's what today is all about. Joining me for this conversation is Michael Brehm, managing partner at Redstone, one of the earliest investors in deep tech infrastructure across Europe. Ben Scheid, partner at Redstone, focused also on AI and emerging data platforms. And of course, Franz Schimben, founder of AllSites, building what might become the next 3D data layer for physical AI.

2:14Andreas Munk Holm:This show is not investment advice, and the hosts of this episode may be invested in the funds and companies featured. Michael, I want to ask you, we've been talking so much about AI being a horizontal layer. Why are we now shifting towards more verticalized systems? Well, first of all, thanks for having us on EUVC, probably one of the best places to talk about technology in Europe. And yeah, I'm very excited to talk about that. We have seen this kind of horizontal layer and as you said, this red race to the best models. And what has been very interesting is that especially a lot of the Chinese open source models have been catching up so fast with the top U.S.

2:56models, the more closed ones. So the question is, where does the real defensibility come from? But also where does the need come from? And what now becomes more and more obvious that depending on the use case, depending on the application, you have very different requirements for the type of compute, for the type of chips interference, for the type of data you need, for the type of labels. models and there's more and more specialization. Even with the larger models, you'll use some of the larger models for maybe for coding. You need other ones more for the physical world. So that's a huge trend where it becomes pretty obvious that one size fits all doesn't work.

3:43And I think that's the really big trend that we're seeing. And if you look like it started and also like at Redstone, we've been probably active for over 10 years now in the AI space. I myself co-founded an AI company 10 years ago in the voice space. And at the time also the thesis was, oh, there is one voice company that will cover everything for every use case. And then we've invested in the security stack, Xane, one of the world market leaders for cybersecurity, for AI chips, for XAI as a generalistic model that has been a tremendous success. And even there now you see new developments and the recent investment has been AMI as a world model with a very different thesis from France.

4:30So it's pretty exciting to see based on this compute explosion with more specialized hardware, augmented reality for like industrial application is getting dramatically better. Virtual reality in video games is getting better and better. But in all these spaces, you need really entirely different vertical experience and data than you needed before. You need specialized infrastructure systems that perform reliably also in a messy physical world, something very different from a purely digital world.

5:06Andreas Munk Holm:And Franz, this is exactly what you're building. So maybe this is the perfect time to bring you in. Tell everyone exactly what Allsides is, what the problem that you're set out to solve is. And then from there, we can expand back into the more intellectual conversation around AI. Absolutely. And it's a pleasure to be on EUVC. So thanks for having me. I think Allsides is a groundbreaking company. We're a deep tech company between Europe and the US. and we're building the data layer, the data infrastructure and platform for the applications of physical AI and let's say generative 3D AI. Michael, you mentioned world models, which falls under this category.

5:45Ultimately, I think these two new frontier AI models, I think they will come together over the next decade or so. But right now to get them off the ground, it's quite difficult if you compare these types of applications of AI in the real world, let's say, that has to do with physical properties mostly. It's not like we used to see from text or image or video-based model, where, of course, fantastic developments have been made since the, let's call it the chat CPT moment. But the data was just available in abundance and various different quality levels. So it was never really a question what you use to train your models on because you could just, let's say, scrape the Internet.

6:33Now this has shifted. Already we see this in text, image and videos and audio, as you mentioned, Michael, as well. But it becomes more and more obvious in the physical world where the data is just scarce, but it's also very, very difficult to create. And so this is what All Sites is covering. So we're really enabling anyone to build in this space, to build faster, to build quicker, and to build more verticalized and specialized AI models.

7:03Andreas Munk Holm:And maybe I tend to say that I really love talking to you because you're really good at translating the technical to the more concrete for someone like myself. Maybe you can talk a bit about what is it that breaks when we are trying to build robotic solutions without having data that comes from the real world, but instead is more 2D internet data, as you might call it. Yeah, so I really love what Franz is doing with all sites. So at Redstone, we think about the markets and how they develop in the future. And definitely AI is changing the world. You can feel it everywhere where you look at. But at the current point, AI is scaling a lot of logic and a lot of text-based stuff that you see in the Internet, and mostly two-dimensional and not three-dimensional.

7:55And what is interesting is that if you look at the global GDP growth, it's actually not so well at the moment, even though everyone is acknowledging, hey, AI is changing the world. So why is this the case? And here I love this thesis about converting from the bits level to the atom level, which is basically enabling AI, for example, to control robots, to really do stuff in the physical world, maybe to even work in hospitals and take care to do their own surgeries. This will leap humanity forward, but we are not there yet. And for these robots to work, they basically use 3D data as their native language.

8:48And yeah, we can dive deeper into this. Why specifically 3D data and what these robots need? But I leave it for now back to you.

9:00Andreas Munk Holm:Franz, I want to ask you to make it super concrete. When you're building these data sets, how are you doing it? Are you setting up an iPhone in lab and then it's recording a bunch of robots doing stuff? Or how are you doing it? Yeah, so maybe just to go a bit back to the origins of the company when we started a few years ago, we already had imagined that, of course, AI will leave a mark also on the physical world. But there was really limited means out there to really capture it at the high quality and at scale. So I want to stress those two points. Quality matters in any AI model, right? So the quality in means very likely quality out in terms of output of these big AI slash machine learning models.

9:47And of course, the size of the data set matters as well. I mean, you can train on two data points or you can train on two billion data points. It's quite different. And I think by now everyone has understood that. So those were our guiding principle as well. You, to your example, you could, of course, record data with an iPhone and take a bunch of images. And then there's open source software and some more dedicated software out there that reconstructs, that's the correct technical term, a 3D model based on a bunch of images and other information. So that exists today. But there's inevitably a quality bar, right?

10:22And so as we started out the company, we tried to resolve this fundamental problem that has been existing for the past 20, 30 years in various industries, right? So if you look at 3D overall, and I think we were mentioning video games before, we were mentioning industrial processes with CAD models, CAD models, let's say, and e-commerce, for instance, with their own 3D models to promote some products. 3D is not something new, has been around for quite some time already. But there was always a glass ceiling in terms of how many high quality assets you could produce automatically and at scale. It was mainly a manual process.

11:01And so we set out as a company to solve that. And we solved this by developing something that is called a fully automated 3D scanner. So we have various products of that. And you can imagine that to be a machine of two by two by two meters, lots of lights, lots of cameras, lots of other sensors that in about five minutes capture all the data you need to then derive. and there is a lot of AI technology on our end as well, under the hood, then to derive a high-quality 3D model, which in our case means a mesh plus PBR, which stands for Physically Based Rendering. So we'll be capturing the physical attributes of that object, which means besides the weight, the exact measurements, we get the collision, we get very, very detailed geometry, sub-millimeter geometry.

11:51And while we get this quality that is unprecedented, At the same time, these machines crank out these assets at five minutes per object, which is unheard of when a human or manual process takes about two days to create these types of assets. So all of a sudden, we have the quality plus the quantity scale combination. And this, of course, in this day and age, when everyone is craving this type of data to train their robots or to develop their generative 3D slash world models is, of course, a fantastic start for us. And that's what we set out to do. One more thing I will say, we're now setting out to build the largest ever created data set and data infrastructure to then let anyone train on top of that within our platform.

12:37And this is the mission of the company. Right now, we work with customers such as Meta, Amazon, Nike, Adidas. As for various applications, to the point of verticalized AI, like Michael and Ben were introducing at the beginning of the podcast, AI is getting more verticalized, especially in tough applications like dealing with robots or generating new 3D from a bunch of data inputs. We're quickly onboarding both on the US as well as on the Chinese side, the leading AI and robotics companies. And we have a very deep relationship also with NVIDIA.

13:09Andreas Munk Holm:Ben and Michael, I want to ask In venture, we always talk about the total addressable market. That's kind of the thing that any wanting to be a founder hears when they talk about venture. You've got to have a huge time. Let me hear from you. How have you thought when you looked at all sides in the beginning? How did you think about the market sizing here, the potential for a technology like what France is bringing to market? You can do either bottom-up analysis where you really look at what customers are existing and how large can these projects be. The market for the core business was already quite large.

13:48But then we thought, hey, this could also be quite a unicorn, if not even a decacorn, if they monetize the database. When we invested, the tipping point at the market wasn't reached. so we we needed also to to check a little bit how this is developing in the meantime nvidia has also created a standard for 3d assets which is called sim ready so simulation ready data and all sites is also living up to the standard and can create it it's getting bigger and bigger and also reusable for new kind of clients so that's what's making the whole thing super attractive and yeah we believe that we are building a decacon here based from south tyrol and want to bring it around the globe i mean if you're looking at at 3d what you're looking at two areas one is you want to have this immersive experience where a digital 3d experience is indistinguishable indistinguishable for a real world experience and we're not yet there we're close, super close, but we are not yet there.

15:01And what's missing is super high quality reality data. And then obviously you have all the more all this physical AI where you also really need super high fidelity 3D understanding. And the core bottleneck today is not more parameters, it's better reality grade data. And that was one of the core thesis. And we said, if you solve that, it's basically that in touch point with every human nearly every day for that type of technology. And they would build basically the backend for that. So it's one of the largest TAMs that we have ever seen in terms of addressable market. And we've looked in depth into that space.

15:42And the question was, how could you tackle that? And what Franz said is what's interesting, you can get 3D data, but the quality even of things you film with iPhone or with cameras ultimately is really bad. What was super impressive that they solved the problem from the very ground and said, what type of data do we need and built a whole machine. It's like they have huge machines that basically scan and that are like a thousand X better in terms of quality and time and cost than anything that's out in the market. And now building that, I think it came a little bit short. it was a huge announcement.

16:17You're building these huge data factories, like real factories with dozens of scanners and like automation. It's enormous. It's one of the most important projects for AI, I think globally right now for the advancement.

16:31Andreas Munk Holm:Let me ask you a question, Franz. Is this not a temporary problem? As in, we have LLMs. When we needed to build LLMs, we said, okay, we've got 28 letters in the alphabet, at least we do in Denmark. And then we've got this many numbers and we've got this patent. Once that's mapped out, you're all done. Isn't it like that, that once you've taken your big machines, you've kind of defined what a shoe looks like, what a hammer looks like. And then in two years, you're done. You've kind of mapped the world. You don't need a 3D model anymore because you've got all the data. This is obviously asked in a provocative Well, it's a fantastic question that, by the way, you're not the only one to ask.

17:16And of course, it's a recurring theme also here at the company. First of all, I think we need to lift the entire physical AI space off the ground, right? I mean, of course, what is available on the tool side and what NVIDIA is putting out there helps a lot. But ultimately, you will always have this huge, huge bottleneck in what you're able to do just because there's no data there. So we need to lift this entire trillion dollar industry up by providing the data. And that's already a humongous effort that will take the next five, six, seven years, I would say, you know, to really, really see this all come together.

17:53I compare this sector almost like to the beginning of the self-driving car industry, right? So there's lots and lots of stuff to do, although, of course, AI is at a different level and can help speed up many of these processes. So that's the first step. and along the way, I think we can build an amazing, amazing business, both from a technological level in terms of just size and on the revenue side. By doing so, we're not just a company who says, oh, we're building the data, here's the data, license the data, and then bye-bye on to the next one. There's deep integration with the big AI labs in the world, right?

18:27And so this deep integration, first of all, will teach us a lot, will hopefully teach us on the AI labs a lot too, And I believe if you look at the next five to 10 years, there will be applications built on top of it that we don't even know can exist yet. So if you compare this to maybe like the smartphone moment at the beginning, nobody knew that maybe an Airbnb, an Uber or a Instacart might be killer applications and worth, in their own regards, several dozens of billions of dollars. And I think we can be in the driver's seat, in the pole position to then ourselves either help build these applications or build them ourselves, much like to the verticalized AI topic that the Redstone gentlemen were mentioning.

19:09And so I think it's even more exciting what we can do afterwards, I would say.

19:13Andreas Munk Holm:Am I right, fans, to also say that, yes, while you might get to a point where you've mapped the world, so to say, you will also, by having been one of the core players mapping the world, be the ones that own that data, which then means that you can monetize the ownership of basically the 3D mapped world. Absolutely. That's what we plan to do from the beginning, by the way, and maybe to stress the point on if we are ever done mapping the world. I mean, there's so many products and objects that come out each year. I mean, we live in a consumer-based society, right? It's like billions and billions of different items each year, and it will be a recurring theme to map them all out.

20:02Then again, on top of it, again, as you develop the applications, even more specialized data is needed. They might not be then publicly available to everyone, but maybe just tailored to different customers. And that is an exciting prospect as well. I think what it also, and we had the same discussion, and I think it's one of the core questions about temporary. But what you're building, if you have this large database, and if you're understanding about it, You can also build more and more tools to kind of interact with them, to use them, to manipulate them, to combine them, build them. And we believe that one of the best companies in that position will be all sides because of the competence in data.

20:43And then you're becoming more a kind of 360 solution provider than just a data source.

20:52Andreas Munk Holm:Let me ask you, your core customers right now, who are they and who do you expect them to evolve to be? Yes, so right now, the customers that we can probably mention are Meta, Amazon, Nike, Adidas, Zara. We're starting to do a lot with also the large Chinese players, which I cannot mention yet. And also with the large AI labs out of the US. And I always have to mention the deep integration on the NVIDIA side of things, as they are shaping the space with their tooling. We want to shape it together with them on the data side of things. And that has been really working well for us. And those are the ones that right now either license the data or they buy the technology to produce data for the very specific applications themselves.

21:44Andreas Munk Holm:To all three of you, what does it say about the current state of the market? that your core customers today are the hyperscalers, the known entities, because one would think that there is a long list of rest of world that would also be using data like yours and wanting to help build and benefit from the 3D world's data. But I'm guessing that they're just not there yet from an adoption perspective. Maybe just to answer this question, so we foresaw that as well, Not everyone might be in a position to do this scale that the hyperscalers are needing and doing. But there's lots of other companies out there that need 3D assets for their own applications, AI and not.

22:31And for that, what we did in the go-to-market effort, we partnered with the leading content creation and 3D companies in Europe and the US to stand up so-called 3D tech centers or 3D scanning centers. where basically companies who just want to get started. They don't have the large, large volumes net or the large needs yet. They can just ship their products and get them done kind of as a service model. And this is for us an easy way to track the market, to get lots of prospective customers hooked before they then really roll it up and go into other dimensions in terms of scale of data. But the technology should be accessible to anyone.

23:09And we're working hard to make this really available across the globe. In fact, we're also continuously developing better technology, just more suitable to different use cases so that everyone can get started at all times. Ben and Michael, I would love to ask both of you, from investing in France on all sides, what has this taught you?

23:30Andreas Munk Holm:What has it taught you about how to think about AI in the physical world? You're investing both in the pure software layer, but you're also investing in hardware. I'm super curious to hear what is an investment like AllSites doing for you when it comes to thinking about how AI is evolving? I mean, when we invested, actually, it was a hardware company. And it was a quite controversial one because it was just producing hardware. And we're investing and say, like, look, we couldn't find a good solution to create the data model. And said, you first have to build the hardware to do that. And then you're able to build the data layer.

24:09And that was not an easy one because you had complex hardware problems before you could build that, but now it gives it a lot of defensibility. And it was, let's say, a little controversial, a little bit contradictory out of the box investments. If you look at the people who are involved from amazing universities, amazing places, all the big centers, but it's also based in South Terrell. So people said like, that's strange, pure hardware in South Terrell. What the heck do you want to do there? This doesn't make any sense. And now they have a lot of the biggest brand names, the largest companies in the world by any metrics as customers.

24:50It has encouraged us to think controversial and do contradictory investments, or at least on my side. I don't know about Ben, what he has taken away from so far. Yeah, I think it's also a little bit like if there's a gold rush, you should invest into the shovels. You have a lot of fast-growing AI companies now on the software side, but at the same time, most of them don't really have a USP. So you can even copy it with a team of students over a weekend, and then you have the whole process up and running. so I think that a lot of these companies who are now hyped in the VC field will have troubles keeping up the high valuations that they raised money for and what will really count in future is still defensible USPs and this you can get from this combination of hardware, software, technical knowledge real scientific knowledge, a great team behind it And this is all what we combine with all sides.