Edutainment for AI and AWS PartyRock with Mike Miller - #661

0:09All right, everyone, welcome to another episode of the TwiML AI podcast. I am your host, Sam Charrington, and today I'm joined by Mike Miller. Mike is Director of AI Product at AWS. We are, of course, coming to you live from the Future Frequency Podcast Studio here at the reInvent Conference, which I've been covering via X, aka Twitter, or FKA Twitter, and LinkedIn. Be sure to follow me on those networks for the latest reInvent and AI updates and insights. Mike, welcome to the podcast. Hey, Sam. Thank you so much for having me. It's really exciting to be here and chat with you a little bit about what we've been up to.

0:45Yeah, I'm looking forward to digging into our conversation. We'll be talking about the product that you are currently heading up here at AWS, a product called PartyRock. Absolutely. I'm kind of representing with my builder's vest on here, PartyRock. Anybody can build AI apps, so absolutely. As well as some other products that you've worked on previously. But before we dive into those, I'd love to have you share a little bit about your background and how you came to work on these edutainment AI tools. I love that word. Thanks, Sam. Absolutely. So, again, my name is Mike Miller. I've been with Amazon for about 11 years.

1:20First five of those, I worked on the Fire TV product where I helped build and launch that product. And the last six, I've been with the AI ML team at AWS, where I've led the development of a number of our kind of hands-on learning products focused around AI and ML and most recently generative AI. Awesome. You didn't mention those specific products, but they are DeepLens. That's right. DeepRacer and DeepComposer. You got it. DeepLens I have and have used. DeepRacer I have and have used. DeepComposer I don't know that I heard of. It was the third one that we did, 2017, 2018, 2019. Yeah, Deep Composer, I think we were a little bit early, actually, because Deep Composer was trying to teach transformer models through music.

2:07And, you know, of course, now these large language models, these foundation models are all based on this transformer architecture, right, where you're predicting the next tokens. And that was what we were doing with music. We had MIDI notes that we used to train the models, and the predictions were the next MIDI notes in the sequence. And so you could actually play a little melody on the little keyboard, and, like, the model would kind of predict or sort of complete the melody for you. Interesting. Yeah, but those products were a lot of fun. You know, we launched Deep Lens in 2017. That was cool.

2:37I love that. I was at like one of the launch workshops and got one of the things. Did you get a hot dog? That was right. The kind of canonical app was hot dog or not hot dog. No hot dog. We were just chatting about that last night with a couple of coworkers. That was a lot of fun. Thank you for coming to those workshops. They were very memorable. It combined like, was it Greengrass? Yep. And some like Pupsauld. Yep. We used like SNS, I think. SNS. We would send a text message if it detected a hot dog or not a hot dog. It was awesome. It was awesome, like, integration of a bunch of different services and a demonstration of capability in an accessible package.

3:15We taught you Lambda. We taught you Greengrass. I mean, we taught you, you know, object recognition models. We pushed the model down to the device, the IoT device. So, yeah, we were ahead of our time there. You know, it was definitely a fun one that gave people sort of a fun opportunity just to get hands-on with all these new technologies. We launched SageMaker that year. So it was all about, hey, how can we sort of build this story and make sort of provide some thought leadership around artificial intelligence and machine learning when it comes to developers getting hands on? We like to say we learned, we sort of observed that developers at Amazon learned a lot faster and were more effective when they had sort of hands on sort of opportunities to get into the technology.

3:56And really that sort of undercurrent has, you know, sort of fed the justification of all of those products, Deep Lens, Deep Racer, Deep Composer. and even this Party Rock product, what we've discovered is if you make learning fun and you kind of use play to motivate people, they're much more excited, much more interested in sort of engaging. Even if it is some learning and learning technology products or technology sort of concepts, people are a lot more interested in engaging and learning. Yeah, DeepRacer was also an interesting one. It was kind of like a car with built-in computer vision. That's exactly right.

4:28We took a 1 18th scale RC car and we bolted on a compute board. It was an Intel processor that we bolted on it. And then we taught you reinforcement learning, which was a kind of a branch of machine learning that allows sort of machine learning to sort of follow a very complex path and achieve an interesting end goal. And we used reinforcement learning along with the computer vision on the car to teach it to drive around a track. And so you first develop that model in a simulator. So we have a cloud-based sort of 3D virtual racing simulator. And then you train the model there. And then you download the model into the physical car.

5:01and you come to one of our events, like Reinvent or one of our other activities and you can drive the car around the track. And so, of course, we said - And it's been quite a platform. Like this, it was launched four years ago, five years ago maybe. Yeah, still going strong. Have you seen it? I've seen it here. They've got a two-story booth. We did a conference, TwiMLCon in 2019 in San Francisco and one of the evangelists brought a deep racer set up and showed us all how to use it. It's a very cool tool. It's a lot of fun. And again, if you make it playful and fun, it's just a lot more interesting for people.

5:33We, DeepRacer did really well. I think very sticky. You know, when we developed it, we said, hey, what's a car driving around a track without a little bit of competition? And I think the competition is, you know, an element that really motivates, you know, certain people. It's not for everyone. We try to make it fun and sort of non-competitive to get involved. And then for those who want to race and like get more competitive, we have this whole DeepRacer League, you know, and it's evolved. We have regions all around the world that compete against each other. When you come to the events around the world to qualify for the championships, which we hold here at reInvent.

6:04You know, the winner has like a giant trophy that they get to hoist. So it's a big thing now. It's got a huge following. Has the vehicle itself evolved? The vehicle itself, surprisingly, no. It's been like really great, solid sort of platform for deploying these reinforcement learning models. And it's got just enough performance for it to provide a pretty entertaining drive. Like I was just over there yesterday and they've got a pretty complex track and these cars are like zipping around there Like 14 seconds is the time, you know, the sort of leading time. And that is pretty fast for these cars around the track.

6:36It's really impressive to see. And they just drive themselves. So it's really neat. Well, I will say that I have a DeepRacer that is gathering dust somewhere. And if anyone is listening to this and is inspired to play around with DeepRacer, then like give me a shout out. Tell me a great story about why you want this thing. And assuming that you're not in a place that will be difficult to get it to, I will get it to you. Very generous of you, Sam. That's awesome. even a deep lens. All right. Now we're talking. So your latest product is Party Rock. I was poking you a little bit earlier because my initial foray into Party Rock was, I couldn't tell if it was meant to be a joke or if it was serious because the results that I were getting were so wacky, but you had an interesting response to that.

7:19Yeah, I mean, I think that's a great thing. Let's start by having you explain what Party Rock is and what the goals are. Sure, absolutely. So we developed PartyRock as a fun, easy to use generative AI app builder designed for anyone of any skill level. You know, no coding required, no AWS account required. Just log in with your social account and you in just under a minute can be building AI, you know, generative AI based applications. The goal here is to give folks some intuition about generative AI and how it works. We think this is going to be infusing all of our lives in the near future. And so just having a better understanding of how these things work, large language models, image models, eventually multimodal models, it's going to be useful for everybody.

8:05And we wanted to build a platform that sort of lowers the risk for experimenting with these things. And I think, Sam, as we were chatting before, like play is a great motivator. And, you know, that's kind of it's reflected in the name. It's reflected if you go visit the website, partyrock.aws. You can see it in the website. It's fun, bright colors. It's not meant to be serious at all. It's like we lean into the rock motif. You know, you've got vinyl records around, you know, the area where you manage your stuff is called the backstage. And so, yeah, we were just trying to have a lot of fun with it.

8:35And hopefully that sense kind of rubs off on the folks who use it. Yeah. Now, when you go to interact with it, it feels very much like GPTs, which folks who listen to this podcast will know. So essentially, you can kind of write and save a prompt and then process further prompts or inputs against that prompt. Is that the general idea? That's right. I think we wanted to make it a little more complex and interesting, though. So we actually try to teach you about kind of chaining prompts together, allowing model selection, allowing you to kind of manage some of the configuration like temperature or top P of some of the things.

9:12So we try to make it like... I clearly didn't get far enough. Yeah, yeah. We kind of make it easy, though. Yeah, like you can sort of like jump in and just get a little slice, right? We have this thing called the app builder, which you can actually generate an AI app just by typing a sentence. Like you can say, give me an app that generates nicknames, right? Period. And then we'll actually generate like a few different widgets for you in an app. One widget would be an input widget, like, and it'll say, okay, enter the name. And then the second widget will be a, you know, a generative AI powered, you know, text widget.

9:40And it'll default to one of the models. And the prompt will probably be something like using the, you know, and then it's got a little reference sign at name to reference the prior widget, generate five fun nicknames based on that name. And so you enter a name in the input prompt and you see some output. What's super interesting then for users is on that tech, that generative AI powered widget, there's a model selector. So you can go for all the models in Amazon Bedrock. You can kind of take a look. So we have, you know, Dropix Claude and Amazon's Titan and, you know, other models as we add them to Amazon Bedrock.

10:11so users can experiment that way. But what I think is super neat is you can also chain widgets together so users can see that the output that gets generated from the generative AI can actually be used to form the input of another widget. So there's a common pattern that we've seen folks who want to do role-playing character generation, right? Where you can have two input boxes, right? Like the race and the class. Or you can even make it even more ambiguous. You can say, what are the strengths of this character? or the emotional state of this character. And then you can write prompts that say, based on those two things, like generate sort of a name and a short description of this character.

10:49And then you can have another prompt or another generative AI widget that says, hey, based on the description that the model came up with, generate an image for what this character could look like. Oh, now based on the image and the description, tell me like a secret backstory of this character. And so you can actually have these things kind of chained together and feed one into the other. So there's a little bit of depth there for building more complex applications. Yeah, I mentioned to you that my initial foray was playing around with translation prompt. And what's a PartyRack app called? An app?

11:18We just call it an app. Okay, so the app translated into a surprisingly diverse set of languages beyond the one that I was intending to target. Exactly. Yeah, and Sam, I think we were mentioning like that sort of showcases like, hey, different models have different sort of specializations or different models are good at different kind of tasks. That was the point that I was just going to raise. My initial reaction was, oh, this thing is crazy. Is this serious or what? But the fact that I could then select another model that may be more appropriate, I think is part of the learning experience because models aren't all the same.

11:52They don't all excel at the same things. And we all know, like, there's the cliche, right? You learn from your mistakes. And in this kind of app, this kind of scenario, like I said, we want to take some of the risk out of making a quote mistake, right? Like, it should just be a fun learning experience. Like, oh, that didn't kind of quite turn out the way I wanted it to. What if I did this? Oh, wait, actually, that works a little better. And then you can kind of see that. I think the other really neat thing about PartyRock that we added is this remix functionality, which you may not have seen. But the goal is to give inspiration.

12:21So if you see an app, you know, somebody, and the cool thing is these apps are just shareable via URL. So somebody shares, posts an app on social or shares you an app via URL. If you think that app is pretty cool, but you're like, hey, wouldn't it be cool if it did this? Or if it like was tweaked a little bit, just click the remix button. And in engineering parlance, we fork it and we make it your own. So you basically get a copy of that app. Now you can remix it. You can change the prompt. You can change the input. You can sort of edit the text that it says and kind of make it your own. So we've got this built in ability to get inspired by what other people are building inside of PartyRot.

12:51Yeah, very cool. Can you talk a little bit about some of the things that you learned about the underlying technology? Well, let's maybe first talk about the underlying technologies that PartyRock is built on and then the things that you learned using those. Yeah, well, primarily the full name is Party Rock and Amazon Bedrock Playground. And so, you know, at the end of the day, we're 100 % built on top of Amazon Bedrock. So if there are foundation models that are available in Amazon Bedrock, once they're generally available, they will eventually make their way into Party Rock. So really under the covers is the various foundation models from Amazon Bedrock.

13:28And there's kind of two use cases, right? As we described, there was the app builder where you can just give it a sentence or two and it'll try to build the app for you. And then you can also go into manual mode and you can just sort of plug widgets down on your little canvas, you know, whether they're static text or input or text generation or image generation. The former case where you just enter some text and you say, hey, PartyRock, build me an app. Like that's all built today on top of Anthropics Cloud. And we have kind of a little bit of like infrastructure underneath there that tries to build some scaffolding.

13:59And then it asks the model to kind of fill in the scaffolding to actually build an app. Scaffolding in what sense? If the app's going to need some input from the user, like let's add an input widget and make sure you're prompting for the right thing. If the app needs to generate an image, like, okay, add an image widget and then make sure the prompt is correct. So that's the kind of scaffolding that I mean, like under the covers. In the manual building mode, it's really up to you as a user, like which models you want to pull in. You can kind of drag and drop in the different widgets. So we have an input widget where we can take something from a user, a static text widget.

14:32if you want to like provide instructions in your app. And then there are three generative AI powered widgets. So there's the text one that just, you give it a prompt and it generates text. The image one where you give it a prompt and it generates an image. And then there's a chat bot where you can kind of provide a prompt that guides an interactive sort of chat experience with a user. And so we've seen people use those for like, you can give it a prompt like, hey, pretend you're a sommelier at, you know, such and such fine restaurant and provide wine advice to the questions that the user asks, right?

15:02if you want to build like a menu recommender app or something like that. So you can kind of imagine like people can kind of do some fun stuff with those apps. But the tech is really based on Amazon Bedrock. And one of the patterns that we've seen in the past with like some of the SageMaker products is like there's a light version, but you can press a button and like push that into SageMaker and access the underlying code or whatever and then build on it. Is there any concept like that here or is it just kind of for play? Yeah, it's a really astute observation. We debated this a bit as we were building this product, like how much do we want to make it so that you can build something here and then sort of export it over to a notebook, for instance, or what are the bedrock API calls that I could use to sort of mimic this application?

15:46So I mean, it's almost like if you look at it, you know, you kind of turn your head and get past like the rocky motif. It's almost kind of like a mini streamlet, you know, LLM. Yeah, like a little prototyping environment almost. Yeah, and I think that's something that we've been debating all along. So I wouldn't be surprised. I can't tell you what's in the future because we're still kind of figuring it out for this product, but I wouldn't be surprised if we added some capabilities like that to make it easier so that, let's say you did build something, you prototyped something, and you're like, tested out some prompts, and you're like, oh wait, actually this is like the golden prompt.

16:17I want to use this. Click a button, export to the Bedrock API calls so that I could copy into my application or something like that. Or maybe even further down the road, you prototype something and then it exports it directly into your like code catalyst environment or something like that. I could see some of those type of things happening, but certainly you're absolutely right. At the beginning, we're very much focused on, you know, low barrier to entry, you know, no coding skills required. And so we want to sort of advance that sort of message in the app. But I think there's an opportunity to do a little more, as you're saying, down the road.

16:46Yeah, also not necessarily in line with no coding experience and low barrier of entry. One of the announcements that I was most excited about in Swami's keynotes today was the model evaluation capabilities for Bedrock. For Bedrock, yeah. And it would be really interesting to see those surface via very accessible tool that allows people to start thinking about, okay, I can throw up some toy app. How do I actually think about how it performs, you know, relative to model selection or relative to prompts, things like that. Yeah. If I could put in my plug for the next Party Rock extension. Picture request.

17:24It would be model evaluation. Yeah. Well, I think you can get a kind of a light version of that already in PartyRock, right? Because you could potentially just create three output widgets and each output widget. Interesting. Just pick a different model. And that's a good way, just like we were talking about, some models are better at certain tasks. That's a great way to kind of look and say, hey, which model is the best suited for this task that I want to sort of pursue, right? We also give you, today it's free for a limited time. And what we do is we give everybody kind of a bucket of credits and you whittle down those credits by building and using apps.

17:58And what we do in the model dropdown is we give you kind of like a relative scale of like how expensive, quote unquote, each model is going to be. Right. So if you want to sort of prototype an app or sort of explore like, hey, I want something that's like very low cost, but very fast, that excels at like a specific task, like a text summarization. Like that's something that, you know, a smaller model that might cost less that has lower latency will get to will allow you to access, you know, very cheaply. Whereas a more expensive model, it might be overkill for like just a simple text summarization task.

18:29Right. Because you don't want to provide a bunch of instructions and have it follow instructions and engage with you in a conversational tone where you'll need a much bigger, like more like instruction tuned model to kind of get that kind of output. I don't know that the target audience is necessarily going to get that, but that's potentially really deep. I have a friend that talks about the LLM supply chain and the model development process. And the idea is you start with the most powerful, most expensive model that will get you there quickly. And then you try to poke at cost management and try to scale it back and see how far you can take it.

19:06And the idea that you can play around with that idea in this toy platform is interesting. Yeah, it lets you do some exploration. you know, because I don't think I don't think everybody, you know, there's probably a different approaches, right? You could always start with like the big heavyweight ones and then scale back or start with the low ones and sort of work your way up, right? I think there's different approaches to kind of tackle that. But yeah, absolutely. And that's what we're going for with the product is, you know, kind of a quick and easy way to get hands on with some of this stuff.

19:30And maybe you discover like, hey, I need to go a little bit deeper, I need to, you know, measure latency or measure accuracy or do more sort of very, very subjective kind of tasks around measuring which Bedrock model selector feature will probably accomplish for you. And so you've kind of suggested this, but it's obviously a lot easier to evolve software products like PartyRock than it was to evolve DeepRacer or DeepSight with the hardware. Yes. Do you see PartyRock evolving over time? Oh, yeah, absolutely. I think that's one of the neat things about AWS and our philosophy is listen to the customer, see how they use your product.

20:08And I think just in terms of my history of in launching products as a product manager, I love launching v1.0 products because you almost never like get it exactly right. People are always going to use your product in like new, interesting ways that you didn't quite realize. And then the faster that you can like skate to the puck where that puck is going and like respond to how people are using it. That's really like the recipe for success for building something that's engaging and sticky and sort of useful for those people. So absolutely, we're looking at how people build these apps and use them.

20:37We've got a great little feedback mechanism on the website, you know, monitoring, you know, socials like where people are posting. And absolutely, we want to start improving this thing over time to see what, you know, match up to what people want it to do. Beyond the kind of edutainment aspect of it, are you seeing folks like creating actually useful things and sharing them? Not really. And I think that's 100 % fine with us. Exactly. We kind of don't want you to build anything like production. Yeah, maybe a utility app. Unintentionally mission critical. PartyRock app? No, please. If you're listening to this podcast, do not do that with PartyRock.

21:13I'm scraping a SQL converter and using it on my hyperscale website. Exactly. My production dashboards. No, don't do that. Yeah. No, we've seen a lot of really funny ones, some useful ones, some adults. Some parents are generating bedtime story generators, right? They'll take as an input, what's the kid's favorite character or what mood are they in? right? They'll enter that and PartyRock, they'll prompt it to like generate a story based on what the kid enjoys or what mood they're in, right? So that's kind of a utility one that's useful and it's creative. It's like a perfect sort of fit for like a PartyRock sort of application.

21:49One that's gotten a lot of traction, it's called the Sassy CDK Helper. And it was done by - The Sassy CDK Helper? Yeah. So the CDK is an Amazon toolkit, right? That you can use. And so it was one of our AWS Heroes wrote this, but he wrote the prompt such that it was like, hey, give good answers, but do it in like a really sassy sort of satirical kind of way and name drop some of these people. And so, you know, people use it not to get like real like advice on how to use the CDK, but just to see like this sort of sassy interaction. Kind of like the Pirate's Guide to EC2 or something. It's like, what do you mean use serverless?

22:23Like, that's crazy. Like, you know, everybody's going serverless now. You'd be an idiot not to, you know, for instance, or something fun like that, right? So people are having a lot of fun with it from that perspective. And I think that's where it shines and sort of gets people's juices flowing is those creative uses. And are most people doing kind of simple one-step prompt thing or taking advantage of this chaining? Yeah, no. You should see some people have like 10 or 12 widgets that they'll put together because they want to build an app that's based on like three or five or eight different inputs from the user, right?