How Roblox Uses Generative AI to Enhance User Experiences - Ep. 227

0:10Hello, and welcome to the NVIDIA AI Podcast. I'm your host, Noah Kravitz. Since it launched in 2006, Roblox has grown into one of the biggest gaming and digital experiences platforms in the world, by basically any metric you want to look at. According to the company's fourth quarter 2023 financial reporting, Roblox boasts more than 71 million average daily users, accounting for 15.5 billion hours of engagement last year. Those numbers include both players and creators, as user-created content has always been central to the Roblox experience. Providing both cutting-edge tools for creators and quality experiences for users at scale is no easy feat.

0:52And with so many of Roblox users being under the age of 16, including my own son, who can often be found drifting virtual supercars through various of the Roblox games, keeping things civil and safe for everyone has to be a high priority. How do you run a platform like this at such massive scale? And how does AI play a part in the process? Here to take us under the hood of Roblox is Anupam Singh, Vice President of AI and Growth Engineering. Anupam, welcome, and thank you so much for joining the NVIDIA AI Podcast. Hey, Noah. Thank you very much. The son I mentioned in the intro made sure I did my homework on this one.

1:28He's very excited we're having this conversation. Maybe very briefly, because there's a lot to dig into in terms of the infrastructure and the things that are under your purview, again, from both the creator and end user side of things. But for folks who might not be familiar with Roblox, and I think there are at least a few out there, maybe give us an overview of what Roblox is all about. So at a vision level, as a founder talks about it for the last 10 years, it's about reimagining the way people come together. You and I want to hang out somewhere. Obviously, we have many, many different ways to hang out.

2:03But our vision is to bring a billion people together with optimism and civility. Now, especially the civility part is very important because that gets us to deep technical innovation to keep everything safe, everything civil. And from day one, we've invested a lot in that area. And optimism is about if you go into one of our experiences, you'll be amazed on how encouraging and interesting they could be. The latest one is something called Dusty Trip. There's another one called Pet Simulator, where you hang with your pet, but there's a lot of other things going on, like Valentine's Day, St. Patrick's Day.

2:49So it's all about connecting a billion people with optimism and civility. And it's a group experience, right? As you put it, it's as much, it seems, about hanging out with your friends as it is about this incredibly diverse array of games and experiences. Yes, absolutely. And so how do people hang out? It's text chat, voice chat. How does the interaction happen? Yeah, so it starts with you open the Roblox app or the Roblox web page, and there's hundreds of experiences that are there for you to select. Think of a movie that you have to select to watch, but after that, you're inside the movie. That's the difference.

3:32And so you press play like you would do on any other app where you consume content. But then you get into the universe. And my favorite ones are where you can actually immediately create an avatar. You know, you get feathers if you're willing to. And if you're interested, you get like a lovely feather dress. And, you know, there's a lot of audio going on. And so another favorite ones of mine are where you can dance with people. So I'm going to say ballroom dance is one of my favorites right now. Excellent. And I just keep learning dancing, something that honestly I wouldn't be able to do in my normal weekday.

4:18But I can do a ballroom dance in the evening. So let's talk about your role, what's under your purview, and how that sort of informs the experience for the millions and millions of Roblox users. One of the things, as I understand it, that you're responsible for is managing and optimizing infrastructure across the platform. And there's a lot happening now with AI and generative AI tools in particular. So maybe you can take us a little bit into that. Yeah, so two-part answer. When your son presses play, he doesn't realize that we have to connect him to a data center, which is hopefully close by, so that his experience is amazing.

4:58Remember, your son is of a generation who believes that press play does not have buffers. And I'm sure looking at you, you remember the time when, you know, you would press play and some buffering would happen. So that's the first magical experience. Oh, I remember the AOL dial-up sound. That's what I'm thinking about. There you go. There you go. But this generation of users on the internet just expect to press play. Then you have to play inside the experience. Behind the scenes, what's happening is he's connecting with one of our edge data centers. And then you have to decide who are the best people your son could play with.

5:35It has to be age appropriate. It has to be interesting. Maybe he already has friends on the platform. So I have to quickly figure out where are his friends? Are they in the same data center or they're in a different data center? Placing them in the right instance of the experience. Our biggest experience sometimes has 40 ,000 to 50 ,000 copies running worldwide on 20 plus data centers. So we have to place them in the right instance. After that, let's say he wants to chat with people. So we have to enable that chat. Let's say he wants to do a voice call with one of them. We have to enable that.

6:09let's say he actually wants to talk to somebody in India, which means he has to talk in Hindi, but assuming, let's say, her son doesn't know Hindi. We have real-time translation to facilitate chatting with somebody in India, so in Hindi. So that's the kind of stuff that we in the infrastructure team build. Now, we like to be invisible. You should never think about the data center that you're connecting to. How does machine learning play a role? Broad question. Very broad question. So we think about machine learning typically from, to start with, either for the creator or for the user. So you have to create these magical experiences.

6:52We talked about ballroom dance. We talked about dress in press. There's a lot of storytelling in there. But most of our creators have the right code to build that world. anything that facilitates their building of the world. It's their idea. We don't build our own experiences. It's millions of developers who build experiences. We need to facilitate that really, really quickly. And I'm sure we'll go into those examples. On the other side are the users. The users just join an experience. And if we want a safe environment, if we want a civil environment, a very simple example, you and I are chatting on Roblox.

7:33every letter that you type every word that you type goes through a text filter for a civility and just that portion of our of our world is i have to get this right it's it's 90.7 billion messages are translated and 2.5 billion chat messages are sent every day and every one of these have to be checked whether you're being civil to me whether i'm being civil to you And this is in real time across all the different languages, all the different data centers and geographic locations and the private chats and the public chats and just the scale. Everything. Yeah. Now, of course, people choose to sample this in other settings.

8:19But from day one, we've not sampled. Every one of these goes through a text filter, which means you need a machine learning solution to look at 2.5 billion check messages. And sometimes, as you said, in a private setting, we might be more colorful with each other. But in a public setting, we're not going to be colorful. We should not be colorful. So making all those decisions in real time could have only been possible with machine learning. How do you approach a problem sort of at that scale? You know, how do you get started? Has that been from day one? has the importance of real-time civility tools, to put it that way, has that always been baked into Roblox's mission, or is it something that kind of evolved out of necessity?

9:10Yeah, so one thing for us is it's always been baked into it. So our engineers never start building something, saying maybe we should sample, maybe we should just do it for NOAA, but not for UNFIRM, which is arbitrary, right? It's for every user. So first, your design principle should be clear that I want to do it for every user, every chat message. Once you've established that every few years, we have had to change our machine learning model. So a few years ago, there was a model that was what, you know, AI people have to have acronyms, by the way, Noah. So one of them is SOTA, state of the art.

9:48The problem is the SOTA changes every year. So we are a state-of-the-art model which would do a billion messages. So we took the model, we trained it with our own data, and then we had to make it smaller to run faster. It could be billions of parameters, and then you sort of put it on a diet, so it can run faster. And so we go from, let's say, 70 plus billion to 7 billion parameters. But what's happened in the last, I would say, 24 months, which is exciting, is the entire model architecture has changed. So it's not just about, hey, let's take one more model and that does natural language processing.

10:34You now have all these generative AI models that can do text summarization, that can do text moderation, etc. So we take those bigger models and to run it on scale, we take every technique possible. We have to distill it, we have to make it smaller. We have to fine tune it, which take our data and fix it. And then we run it in production. And honestly, once you start running things in production, you have many face palm moments where you say, wow, we could have done this. We could put it in memory. We could back the thing. So, yeah, it's our design thinking and then running it in production. Now, I know that broadly speaking, and also in Roblox, and as I understand it, even within the purview of scaling safety and civility, multimodal LLMs models are, I don't want to say the new thing, but they're becoming more and more important and talked about in all phases of technology.

11:30How do multimodal LMs play a part, multimodal models play a part in your work? And this may or may not be relevant, but it's something that popped into my mind talking about civility. When you were talking about running the chats through text filters, it made me think about slang and lingo and how do you sort of keep up with, you know, words that I might not understand, but might be extremely colorful to use that word to people in the know. So how do you just keep up with all of that? And then second question is about non-text things. So if an avatar has something questionable, or I don't know actually if you're allowed to share images and memes and that sort of thing, but yeah, we can talk about that.

12:15Yeah. So let's talk about multimodality and then we'll talk about what I would call internet wisdom. Perfect. Things that you and I might not know are offensive, but they're offensive. But multimodality, you're 100 % right that in the last two years, we just assume text models are there. Any of us can access PPP, any of us can access, you know, any model that you want to pick up, right? What's exciting about the last one year is the use cases are also changing. So more and more on our platform, people would love to use voice. So what we did around two years ago, we decided that we will think twice the same way as we have teacher tech, which means you want to check for profanity and other offensive words in speech.

13:07And so we use a very large model, and thankfully large models are available both open source and closed source, to convert firstly voice to tech. And then we use all our tech knowledge from 10 plus years to moderate voice. The next question that you ask is even more interesting, which is images. we always had image moderation in our platform. Since 10 plus years, our founder likes to tell the story of he used to do the moderation few maybe 15 years ago. He does not anymore, but he used to. We can make billions of thumbnails every day. And the exciting part is each one of these large models has the wisdom of the internet packed into them.

13:56Earlier, we might have to train it from set. Maybe five years ago, Roblox would have to train it from set. So now let's say there is a new plan or a new offensive in it. I don't get the benefit of somebody else training a large state-of-the-art model and open source in it. But now we start many of our initiatives with a large open source model that has the wisdom of the internet and is able to update to newer things. then we fine-tune it when we distill it to roblox's use case so image voice and text are all in production for us then if you're going to take multimodality further out and i know you talked about this in some of your other podcasts is 3d that's not just a picture that is you know I could make something really interesting and I don't want to just focus on civility here it's just the beauty of what you can create on the platform and how quickly can you create about it so we are very focused on 3d and 4d as the next modality and there honestly we don't see a lot of models that have already been built in open source.

15:14There is no easy wisdom of the internet as far as 3D and 4Ds. And so you're using kind of across all these things, I'm generalizing, but you're using a mix of available open source, you know, available models that you find train and otherwise, but then you're also building models in-house. Yeah. Okay. So for example, for recommendations, our model is internally built. Right. For certain safety use cases, our model is internally built. We have a coding assistant, but we have our own language. So we take an open source model, and then we combine with our language and build a large model. I'm speaking with Anupam Singh.

15:54Anupam is Vice President of AI and Growth Engineering at Roblox. Roblox is one of the leading online platforms in the world, bar none, but certainly when it comes to creating and using all kinds of digital experiences, from games to roleplay simulators, ballroom dancing, as you said, on a palm. It's just amazing, all the things that are out there and being built literally every day. You touched on this briefly, and I want to sort of flip over to the creation experience and how, you know, you mentioned Roblox has its own code, that Roblox Studio, I believe, is the platform for creating. But actually, first, I wanted to start with talking about avatars, because my experience has been, you know, as a user, as a parent of a user, but then as a user myself, I'll go on and play games.

16:44I love the driving games. They're so fun. And big paintball is one of my favorites. But the whole idea of an avatar is just incredibly essential and important. And, you know, it's vital to so many people's online identities broadly, but particularly on your platform. Every time my son will show me, like, hey, check out my new avatar's outfit, and it'll be his favorite footballer's uniform, or it'll be some just wild costume or this or that. So can you talk a little bit about the technology behind being able to create and customize avatars and how that relates to, I mean, basically anything that is important to you, but talking about civility, but then more talking about just giving users the tools to express themselves.

17:40Yeah. Autars have such a rich theme. We could spend hours about it. Sure. Let's start with a very simple application. There's a likelihood that you want the author to reflect some of your personality, right? Including you have this lovely backdrop behind you, which people can't see on the audio. a podcast of these lovely electronic devices. And so let's start with taking your photo and creating an outside audio. It would be simple if it was only 2D. 2D filters are on the phone. You can do them today. Great technology, right? By all the great phone manufacturers. But you want it to more on, you want it to come on.

18:20That's when, you know, creation of these photo of our generation of 2D and 3D are major innovations. And it is ongoing. There, our technology has to, you know, both be built from scratch, but also use some of the existing 2D to 3D, 2D to 3D innovation. Sure. Then, okay, we took your photo, we built it out, but we want to set it up. You know, we want to give it a phone, which is beyond just your maybe face. that's how thought auto set up next we want to track your upper body because your upper body movements i love to gesture a lot maybe you just said less how do we capture that and after all of that we need to put it in in a world in a cohesive way so each one of these think of these as almost separate modalities movement is a different modality from texture that is a different modality from what you're wearing between material and none of that to you or your son is interesting it is our job as an AI platform that the right models get called the right models get orchestrated of course in an ideal world and I'm looking very very into the future all of this should be answered by one model it can do photo to octar it can set up your upper body it can set up your face and body coordination.

19:50It can set up your gestures and then put you into a world where you're dancing doing press or dressing doing press. But in reality, I'll tell you, each one of these are different models being called for many specific, special things to do. Let's talk about creators. You mentioned you've got your own language that creators can code in. There's obviously, as we've alluded to during the conversation, all kinds of visual, 3D, physics engines, all kinds of things happening in these experiences. How are you incorporating generative AI into the creator tools? And what are some of the, not just concerns, but some of the exciting things that are happening, whether they're at the level of an individual creator, or if we're talking about orchestrating all of this at this massive scale?

20:39Yeah. So the creator journey is very fascinating because many of our creators are amazing storytellers. Right. And that's and forgive me for interrupting, but just to sort of emphasize, it's been like a wake up calls to sort of how, you know, this generation. And for me, it's the generation of my son, but obviously your users spanned ages. But just how important storytelling and just sort of the like personal narrative, emotional aspects of these experiences really are. I'll watch my son, you know, role play, buying a house and decorating it and just doing what to me seemed like mundane things.

21:17But he's hanging out and living this experience. And it's it's it's amazing. Oh, yes. Building a house on roadblocks. It's so much fun. We could go on. But let's go to the creator. So there are creators. One of my favorite experiences right now is being able to land on any airport in the world, which is, you know, airport simulators has been around for a long while. but combining it with Roblox and storytelling. Now, but think about the creator. What they have to do is they have to start building the effort and are always bit by bit. We want to facilitate that really, really quickly. It all starts with the most popular use case today in the industry is coding assist.

21:56Just let me code faster, right? But after I've coded the world, now I want to give it texture. Now I want to give it material, right? You're giving your creator, you're giving your users lovely aviator glasses. It would be pilots, right? Top gun reference. And these aviators should have textures, they should have shine, the material should be maybe metal. All of this takes some time for our creators. And they're in a hurry to tell a beautiful story, but this takes time. So for every step of their way, generative AI starts helping them. The coding assistant is a pure text model, but the material and texture models are much more richer and more interesting.

22:46And now every one of our creators would love to have their players worldwide. Right, of course. Now, should they spend time translating? And that's where Roblox comes in. Our translation is automated. You, if Noah, is creating a brilliant experience, but now you want to translate it everywhere. We just make it happen. And now once people are into your experience, we do all the chat translation too. So from the time of coding to the time of release, we are helping you everywhere. We're generally aware. That reminds me of something I wanted to ask you earlier. When we talk about real time in the context of say, translation or even civility filters to put it that way, Do you have a benchmark for what real-time means?

23:35So real-time for us is literally as you're typing, the translation happens. So as long as the user doesn't see the thing, you know, we talked about when you press play, you shouldn't see buffering. That's what we aspire to. So we shouldn't even exit. A great example of that is, and our CEO, our founder loves to show this example is, no one on a phone get on a one-on-one on Roblox. We have Roblox office. You know, you and I get on one. Noah says something colorful. Yeah. In real time. As I want to do. Yeah. In real time, you will get a nudge from our voice moderation system. Right. Then you say something more colorful.

24:16You will now be suspended maybe for a few minutes. Right. So that is what real time means to us. It is not a warning that will come to your inbox a day later. As we start to wrap up here, when I ask you the question we always ask at the end of these conversations, what's next? What are you either working on now or kind of looking ahead to? You mentioned, of course, way out in the future, the one model that can handle all these different modalities and things. Are there either problems that you're trying to tackle right now or something in the near horizon that you're really excited about that you might give us a glimpse into?

24:53Huge excitement is 4D. I know you've talked about 3D and 2D in many of your podcasts. The fourth dimension, and I sound science kitchen-y, but the fourth dimension is you want a car in your experience. The car doors have hinges. It has to open the right way. My favorite example is I want to create, I live in San Francisco. I want to recreate all of San Francisco in its glory. Alcatraz Iron, Fareed Building, The Painted Lady, Victorian Houses on Roblox. I can create that, but it will take me some time. But once I create a Victorian house, I want to enter it. I want to open the door. I want to hear the echo because the sound is different when I enter a Victorian house where there's a glass house.

25:38All of that takes effort on our current platform. Of course. What if you could just talk to it and then you start building the story? You've seen all these movies set in San Francisco, right? From romantic comedies to action thriller. What if you could do that storytelling much faster because the base pancrums was created so quickly for you? So that's what I'm excited about, where they don't think about models. They don't think about coding assistance, voice translation, et cetera. They just think about storytelling and the model is in the background just assisting them continuously. It's not even called an assistant because it is so much in the background.

26:15Adhapam, we could talk for hours at least about any of these topics, as you alluded to throughout, and perhaps we'll get the opportunity to do this again in the near future. I don't think Roblox is going anywhere. So it would be great to check back in as technology advances and the experiences advances. But we do have quite a few technical-minded listeners in the audience. So for folks who want more, want to dig into how all of these things, the models and the filters and the real-time translation are happening and are happening at such scale, are there places online, obviously Roblox has a website and a blog, but are there specific places that you would direct folks to learn more?

26:56Oh, yes. We have a Tech Talks podcast. Oh, fantastic. That is hosted by our CEO and founder, Dave Bazzucchi. Every one of the topics that we talked about, whether it is South Arts, growth engineering, safety, civility, we have 40 to 45 minutes of deep dive by Dave himself. Great. And that's Roblox Tech Talks? Yes. Excellent. There's a developer blog as well? Yes, there's a blog.showblocks.com where you can go and look at what we're not. Great. Well, Anupam, thank you again. This podcast is not about me, but you've made me a star in my own household, at least for a day, getting to talk to you. So I appreciate that.

27:37But more importantly, appreciate your coming on the podcast just to give people a little bit of an insight into everything that has to go on behind the scenes so that whether you're chatting with a friend, being kept safe and civil, or, you know, experiencing these amazing 3D experiences, you don't have to wait for things to buffer. They just happen. Yes. Thank you very much. This was, I'm an avid listener of your podcast. So this has been great. I appreciate that. Look forward to chatting again and all the best to you and your teams. Thank you.

28:38Thank you.