Roblox Studio Head Stef Corazza: Using AI to Empower Creators

0:00We have this unique synergy and collaboration with the community. We're basically, we told the community, hey, give us access to your data to train AI. We're going to make the best AI companion, the best AI assistant that we can. And that assistant goes back into studio and is free, right? So we're not making money of your data. We're actually helping you create more. And so we found the overwhelming majority of the creators in our community gave us permission to use their data for training and that's why I was mentioning earlier. We have not only one of the largest data sets in the world, but also the most multi -modal.

0:57Welcome to Training Data. Today we have an amazing guest in Steph Coraza. He leads Generative AI at Roblox, one of the largest gaming platforms on the planet. That's it. Roblox has 79 million daily active users, and they have a creator economy that pays out hundreds of millions to creators. Because of this, Roblox's uniquely positioned to transform how games are made and played with AI. Stuff is a founder at heart. He started Miximal, pioneering AI company for character animation, and was acquired by Adobe. Roblox brought him in to revolutionize how games are made with AI at their own platform.

1:35Under desk leadership, Roblox is pushing the boundaries of AI and gaming. From their groundbreaking AI assistant, which lets you generate games with simple natural language, all the way through their 3D Foundation model technologies, welcome staff to training data. Today we get to talk about games, in particular we're talking about AI at Roblox. Robox is one of the largest gaming universes on the planet. And I say universe instead of platform, platforms are pretty overused term and really Robox has created much more than a platform. It's a place where you can create, it's a place where you can play, where you can meet new friends, and it's all done online virtually.

2:20Now, we are technologists, but we're also investors, and I wanna spend a moment on how remarkable of a business Robox is. So obviously an amazing technology and we'll get to there, but the business is exceptional. You've got a $29 billion market cap company with over 3 .5 billion of run rate revenue. An amazing stat here is they're really building an economy. It's not just a selfish company. It's a company that actually has produced over 800 million for their creators, for the people actually building on Roblox over the course of a year. 70 billion hours of gameplay on RoLox per year, 70 billion, and they're able to deliver cash operating profit.

3:04So $600 million operating profit. That's because they've taken catered to a huge audience, 79 million daily active users. Over time, they've actually shifted up and age, and you've got 46 million of those daily active users are actually now over the age of 13. 3 .8 million daily active voice users and the numbers is continue to grow on this amazing business. And so, Steph, we're so excited to have you here today. We could not have asked for a better person and make the category of AI gaming. From machine learning to computer vision to biomedical engineering, you have a pretty impressive technical background, an amazing journey from what was initially by a medical to a generative AI efforts.

3:53And we were hoping that you could kick it off by just telling us how you got here. How did you get to becoming the head of generative AI? When you started off as an engineer in a very different field many years ago. And thank you for the great introduction about Rob Rocks. It's really like an amazing example of a compounding effect. And every day we are mesmerized ourselves to about the success. My journey started a Stanford about 20 years ago when I came from Italy, a spot on my exchange program, and I was focusing on computer vision, machine learning for the measurement of human motion. And so you could have basically two markets there.

4:31One is the biomedical that you mentioned and the other one is animation. And so we were basically at the boundary between the two and at some point I realized there was a much bigger opportunity in the animation space and so I basically work on like video based motion capture and animation creation solution that then led to the spin -off of Mixamo the company that started in 2008 they get later go acquired by Adobe in 2015 and it's still today one of the most used machine learning services in the industry to rig and animate characters and so after a few great years at Adobe. Actually, seven of them, I helped build the 3D offering there, including products in my team build, like Adobe Stager and Adobe Arrow for AR.

5:21And then we acquired a LEGO Rhythmic, and so we built the full 3D portfolio. And then after that, I was really passionate about Genie Eye. I was working with the CTO of Adobe, trying to figure out completely new ways to generate things. And that's when BISCII Roblox reached out. And I had, you know, breakfast a few times with Dave and we talked about it. And I realized that potential of the Genai head was really finding in the gaming space and specifically in Roblox, being a platform where so much data exists, you know, 15 million experiences every year are played. I found BISCII Roblox be the place where this could really blossom with a massive impact worldwide.

6:05So before we get into the AI components, can you tell us a little bit more about the platform? You're an engineer by background, and so scale is incredibly exciting to you, I would imagine. The sheer scale at Roblox is pretty mind -boggling. I think to anyone, especially users, there are obviously platforms that have more daily active, the Facebooks of the world. There are platforms that have more monthly active, et cetera, but it's very rare to have the amount of bandwidth with and compute and graphics and everything in one place. Maybe you can share a little bit about what that means to have a 15 million daily sessions and 79 million monthly active and a little bit about the stats and what that means technically.

6:49Yeah, we always like to talk about the daily active, which is the 79 million that you mentioned, but the monthly active, that I think we don't communicate to the outside world. It's even more staggering in being in the several hundreds of millions, right? So it's a massive community that is growing very healthy and pretty fast. And sometimes people ask us, you know, where are those games coming from? The people play, right? Some of those games are now like worldwide franchises. We like tens of millions of concurrent players. And then sometimes I give this number, which is also like a reminder of the scale.

7:29So every day we have roughly 90 ,000 experiences in games published on Robots. And so that gives the scale of the human creativity if you want. And how much really this is becoming more and more of a creation platform and gain the development platform that has like incredible numbers in terms of scale of creation. And also like it's an economy on its own. As you mentioned, we paid out $800 million to our creators. And so people have jobs, people buying houses, people have companies. They're now, we have game studios. They have more than 100 people. Some of them are VC funded. So it's basically creating its own creation economy and type of that.

8:18And so I think it's very humbly and at the same time as incredible potential. Because the uniqueness, I think, of Roblox is that we are one of the most vertically integrated companies on the planet. We own our own data center, we have many data centers around the world, where we own the hardware, but then we also, on the app that distributes all these games and the players and all the services like a video chat and live chat and chat translations on. But then we also have the Creation tool, which is a Robert Studio that I have the privilege of leading with my team. And then also we basically have all the services for creations that they go with it.

8:59So it's all the way from like the bare metal CPU, GPU cluster all the way to the creation tools. With the big difference that we only charge our, we only basically take some revenue ourselves when our creators make money, right? We make money when they make money. There's no upfront fee to get into the game. There's no upfront fee for the tool. The tool is free. A lot of services are free and so there's really very little friction to start using Roblox And if your game has one user or a million user You don't have to worry about anything. We scale it for you. We pay for all those like CPU GPU instances storage in the cloud and everything and it's completely opaque to us the creator So I think that's the uniqueness.

9:48And as part of the fully vertical integration, we are also able to subsidize AI. So we are one of the few companies out there with a full, a full -fledged AI assistant offering for game development, for code creation, material texture, asset, everything, and it's all free to the creators. I'd love to get into that. Maybe can you just walk us through today? What is the experience of creating a game? Like, who are the typical creators on your platform? Is it a high school student? Is it a professional game developer? And what types of games are they creating? That's a good question. So we have several million creators on a monthly basis on the platform.

10:30So big numbers there as well. And usually, I think the average age is in like the mid -20s. It's a little bit older than our player demographics, of course. And typically the majority of them are doing like word building, they're like building stuff, they're artists, they're making games and then we have about 30, 40 % actually are coding, right? And then of course there's an overlap between the two audiences, but basically roughly, this is what we're seeing. And it's used for the most different creations you can imagine from natural disaster simulations to learning, to the cloud more classic gaming experience, events, concerts, fashion, design.

11:20We are seeing new type of experience popping up on a daily basis, which is very fascinating. What's your favorite game? I've been playing Ritalitally Racing Empire quite a bit. It's like I like other games. So that's a good one that I really like. Steph has actually surprised to hear that you said 30 to 40 % of the developers are actually coding as opposed to world building. Why is that and maybe that kind of parlay is into what you've created with assistant? Yeah, so there's just a skill that it's harder to master and to get into. Right, a lot of people just like go from players and they want to create something and so they start like building the world and that is probably more intuitive than than writing code where you have to understand this like high level constructs and apply them to go into activity.

12:16And so that is one of the things we wanted to tackle with assistant. We wanted to basically remove that friction or having to learn to code every time you learn programming language in order to create interactivity. And so that was like one of the initial inspirations and that's why you know code assist was the first feature that we released. And then now this was like in March, 23 and 3, which feels like two decades ago. And now, you know, a year and a half later, we basically have a full, a full basically assistant that has gained the development capabilities they go from writing code, auto completing code, explaining code, debugging code, applying scripts to parts and objects in your scene.

13:04So that's all the coding stuff. Then we have documentation. People ask, how do I do X, Y, Z? And usually they have to browse through their forums and documents on the internet instead. Now, a system can basically summarize those information for them. And then the third aspect is creation of assets. So we wrote out a material generator. We wrote out a text to generator, which was a lot more complex, where you can basically texture any 3D object with quite good field of resolution just from a text prompt. And so all these together is what we call assistant as a within umbrella name. And it basically is allowing now to create entire like simple games from scratch just by typing natural language.

13:53In the future there will be more like multi -modal input through images. But basically right now we have like simple games that people are making also as a test where like they're only using assistant And so you can imagine you can make that that game on your phone You know with a microphone You just pick to it and then a system will just generate the world will create the forest will create The enemy the boss that you have to fight and then also add all the game mechanics To it everything automated. Are the games created with assistance? Like how good are they? If you had to give a score out of ten, the games created from coding versus what what people are doing right now with AI versus where it's going.

14:34I mean, when you if you can code, of course, you get to another level of sophistication in terms of the the complication of the gameplay and and all the nuances that make like a game fun to play. So of course, we're not there. The examples they they I see are mostly like we do game jams and you know we spend like two hours to make a game with assistant and what can come out is pretty incredible but we haven't had you know in the community I'm sure they're gonna take it to the next level the goal is not to exclusively use assistant the goal is to basically combine assistant and learn skills through assistant so maybe at the beginning the first script a system will write it for you and will attach it to a part, then you know where the script should go and then you know how to I don't know make some hobby platform move up and down and then you learn on the way.

15:30So we see a system as a companion that is showed you by doing how you make a game and then over time people really develop skills, otherwise it's hard to acquire. Stephanel, have you seen that the type of development has changed as in the assistant not only has language, like the documentation you described, it does also have code. It even has images. I saw that it's and we'll get into the technical specifications in a little bit, but as a user you have language, you have code, you have images, have you seen the behavior change for how people develop in Roblox? As in tactically, have you seen the number of people actually coding going up, I would have guessed that outside looking in because the barriers to entry of coding have gone down, but does that just mean that more people are developing in general and the ratio is actually state the same?

16:19That's a great question. So, what we have seen, we have measured the productivity of people that use assistant versus not. And so, we have found that people that use assistant create a 180 percent more code. Wow. And so, the individuals are a lot more productive. That's benchmark to people who are ready wrote code. Or does that also say just more? Yeah, wow, okay. Two cohorts of codeers, one using assistant and code assist. Code assist is the suggests code and an assistant creates from scratch, but they all kind of integrate into a similar user workflow. And then if you're looking at the same cohort comparison for material generator, creators, they use material generator, create 60 % more materials.

17:09And so also on the art front there's more productivity. And then if you look at the final goal, which is how much they publish, right, because publishing the game is the ultimate goal, the lift is about 30%. So people that use assistant publish 30 % more than people that are not using it. And remember that this is now still in beta, mostly. It's gonna get out of beta soon. I can give you the exact day, but it's gonna be relatively soon. And so we're gonna see even a broader adoption, of course, there, and impact. One more question on this, Ansonia's quality question. Like, what about usage? So they're publishing 30 % more.

17:52Do you have any KPIs that actually give you sensitive to the assistant games, the hours spent, the robots spent on them, whatever the KPIs might be, if they are also that 30 % lift. Yeah, that's a good question. So our number one, KPIs right now is around retention. And so we are seeing like a week of a week retention of people using a system that is much higher than any other features that we roll out. And also we are seeing the overtime that retention in the long run increases quite a bit. So we have people are using a system with studio. We are seeing significant lift in the overall retention of studio.

18:31And then of course we see a high retention system. So the number of daily users has been growing organically and very steady. We don't do any marketing, of course, on this. And it's free, which is, I think, the best marketing. You know, like, well, it's not cheap. Let me tell you that, right? I think Roblox is very generous on that. but we have this unique synergy and collaboration with the community. We're basically, we told the community, hey, give us access to your data to train AI. We're going to make the best AI companion, the best AI assistant that we can. And that assistant goes back into studio and is free, right?

19:13So we're not making money of your data. We're actually helping you create more. And so we found the overwhelming majority of the creators in our community gave us permission to use their data for training. And that's why I was mentioning earlier. We have not only one of the largest data sets in the world, but also the most multi -modal, if you want, because we have code, we have images, we have 3D assets, we have audio, video, all that is part of our gaming experience. And also, with the interactivity, there is the glue for all of that and with the analytics on the usage. right? So it's a very, very powerful data set that we are very, you know, treating with a lot of respect and a lot of, you know, the best practices to keep that it, of course, very secure.

20:03But at the same time, allows us to really harvest the value. And ultimately what we are doing is we are teaching AI, game development. Right? Basically, that's what we are doing, right? We're not teaching how to make an image. We're not teaching how to write code. We are teaching game development. That's the ultimate goal. And so it's going to all these tools, the beginning will feel a little bit, hey, this is a tool to make a material. This is to make the text, it's to write code. We already see that they are converging. We already started that process of converging some of those like lower level tools into larger ones where basically the AI is actually learning how to develop a game is supposed to just how to do like a small touch.

20:46I love the vision of the AI learning how to develop a game. And the question I have is if you break up game development into its component parts, which parts do you think are most likely to be taken over by AI well in the near term versus, what do you think humans will be uniquely good at for a while? That's a great question. So we don't see AI as taken over by the way. I know. I think we made with the investor, the parlour that we made in ADARDC like a couple of weeks ago, was AI is your dishwasher, right? Nobody wants to wash the dishes, right? And so we are really focusing on tasks, especially with the last release of assistant actions.

21:30We are focusing on the tasks that you don't want to do, right? Washington dishes doing the laundry. And so we have introduced amongst the assistant capabilities a new capability where assistant can basically make large scale modifications to the data model. Give an example. I have made a beautiful open -world game with a huge forest. This forest has 100 ,000 trees. Now all of a sudden I want these trees to actually follow the seasons and get more leaves more yellow because fall is coming. the amount of work that will take me to implement that will be huge. Assistant can do that with three lines of text.

22:09I can say, select all the trees or I can say, select all the pine trees. Actually, those don't become yellow, slow, but one, sorry. Select all the trees other than the pine trees and make the leaves yellow. I can just give these three liners. Assistant can go and select the 57 ,000 trees. there are no pine trees and can change the color of the leaves for me in just a few seconds. So does the kind of task that we are seeing a lot of value and honestly this was the feature that the community loved the most of all the iFeatures we released in studio in the last two years. This was by a landslide the one that got the highest scoring appreciation for the community because again it was the dishwasher.

22:54We are now replacing your talent which we believe is irreplaceable, but we actually help you know in the task that you don't want to do. I am always so impressed by Roblox's emphasis, genuine emphasis on community and teaching. You've said this in the past few minutes. You've talked about community quite a bit about teaching how to code and how to create games and really uplifting the entire community, frankly. And I just want to say that this is very much true all the way to the core of the business. I got to follow my really good friend Craig Sherman to board meetings in 2017 and 2018 at Roblox and even behind closed doors, you know, years ago, this was always the focus.

23:38It wasn't the the banality that a lot of board meetings on monetization and financials, it was community. It was uplifting community, teaching them how to code, teaching everyone how to use Roblox in a way that actually benefits themselves in their own learning. And this dishwasher analogy sounds very consistent with the ethos of your business. How impressive. How did you implement it? It sounds like a segment, anything type of algorithm? Was it a segmentation approach or this particular feature? How did you do it? That's a good question. So we basically have found a way to give a system, So, Assistant is really good at generating code.

24:21As, you know, Ella is based on LLAMs and some of them, actually, we are supporting as open -source project like StarCoder and we train with our own data. So over time, Assistant is getting better and better at generating code. Some of the code, instead of running runtime, you can actually run it at the time in Studio. Studio has a command bar where you can just execute some of that code. And so assistant actions basically is creating code that is executed directly in studio at the time. And because we integrated in studio has full awareness of the data model of your scene. And so it knows, oh, this is a three, this is a car, it knows what you did and it has full awareness of that.

25:07And so we have combined the ability to generate code with the ability to create using the The same code, Lua, commands in studio and the awareness of the data model. Those three things coming together, and basically unleashed the power of the LLM onto like data model manipulation. Is that, do you have these things labeled or is it dynamically determining that this is a car? There's a good amount of inference that just happens. Cool. Yeah. How do you imagine the creation experience evolves? Let's say 10 years from now. Good. How do you see a system evolving? So, when we met roughly two years ago and we said, okay, how is the AI going to impact creation and where should we start?

25:54The one paradigm shift that we envisioned that we thought was going to happen in the industry, not just the Roblox, was a shift between control on the fine control on the creation to capturing intent. So I spent quite a bit of time in the Photoshop land where basically there you give control to the color of the individual pixel of an image even though most people don't need it but some might. So there it's all about 100 % control, non -destructive workflows but 100 % control on the actual artifact you're generating. And then we are now moving towards a complete different generation of tools where the tool is successful and can produce good outcome only as long as it can capture the intent from the user.

26:51And so all the digital tools that we have seen in the last 30 years, it's all about surfacing more control to a user and then the user will figure out how to use that control. right? Instead we are migrating from control to capturing intent and so we're gonna see probably a quite a big change and there's gonna be you know a thousand startups trying to do in different ways and it will be new UX part I'm popping up but we can see audio is an input we can see high -level gesture not just you know with your hands but also with mouse and keyboards so things that are just providing an input into what you want to do.

27:30We are seeing multi -modal input. I can describe a world better if I can type and I can provide maybe a concept art and maybe some high -level sketching on top of it. So this is very different from having the phenomenally granular control but at the same time the velocity is like two orders of magnitude faster. The challenge in all this is that for for casual creators or people that have limited a month of time to spend on it, it probably, AI is providing, it's already good enough to share on TikTok or to make an experience on Roblox and invite your friends over, but for people that want to spend a lot more time, you still have to provide that fine control.

28:19So the challenge is how do I make, how do you make a tool that is really good at capturing all the initial intent, but at the same time allows the real pros to iterate with the same level control of the traditional tools. So that's a little bit the challenge there. I think a lot of companies like we are facing in studio, a lot of other companies are also facing. We have a really fun episode of training data with this team that created a company called Dust. And one of the founders Gabriel Hubert talks a lot about rasterization versus This is vectorization and this is definitely your language as a graphics person.

28:57They also are Stanford computer vision type folks and it seems like that transformation has happened. And the vectorization, kind of what you're describing. You can expand it to a great deal based on intent. You can shrink it to very small and almost like fractals go down into more and more detail. When do you think we will get there? What do you think we'll get to the point where you can say, hey, this is the attack. and then you can go in and a fractal level of detail, change each pixel, also based on that intent, but on a much smaller scale. Yeah, that's a million dollar question. So what we're trying to do is allow assistant first to be able to do, to perform those operations, right?

29:43So the same AI that gives me the rough, there's a lot of machine type of input. I throw in some text and image, I pull the lever, let's see what we get. We want basically a system to also be able to go in and do the fine grain change. So, hey, only the trees there are above, you know, five feet, can you just change the color of that, right? So, and then can you take the texture of the tree and then open up and I'm gonna paint over it, right? And so, we want to allow AI to already be able to go beyond the like one shot and allow for iteration with the B believe iteration is a fundamental way people create.