Gaurav Misra: Building an AI-Powered Creative Studio

7 Dec 2023 · 1 h 9 min

Ask about this episode

Ask anything about it. ChatGPT or Claude reads this page and answers with the times it was said.

Connect VO and ask about every podcast you hear, including the moments you saved. Add to ChatGPT · Add to Claude

In short

Generative Now Podcast Episode Notes

Episode Title

Gaurav Misra: Building an AI-Powered Creative Studio

Podcast Overview Generative Now is a weekly series from Lightspeed that explores the stories, strategies, and insights behind AI companies shaping the future of work. The host, Michael Mignano, interviews founders and innovators using AI in groundbreaking ways.

Episode Summary In this episode, Michael Mignano interviews Gaurav Misra, the co-founder and CEO of Captions.AI, an innovative tool that transforms video creation through AI features such as AI-corrected eye contact and automatic captions in 28 languages. Gaurav shares his journey from a design engineer at Snap to launching his own AI startup. The discussion covers the evolution of Captions.AI, the challenges of starting a business, and the future of video creation.

Key Topics and Themes

  1. Early Life and Career Development
  2. Gaurav Misra was born in Boston but spent most of his childhood in New Delhi, India.
  3. He was heavily influenced by his parents to engage in various extracurricular activities, which fostered his learning and adaptability.
  4. Gaurav pursued a Computer Science degree at Boston University, where he was well-prepared in programming from an early age.
  1. Professional Journey
  2. Lattice Engines: Gaurav's first job involved applying machine learning, where he learned about matrix multiplication and scalable computing.
  3. Microsoft: Worked on an ML platform on Azure, meeting influential people and gaining experience in a large tech environment.
  4. Localytics: Gained experience in analytics and data, which would later inform his work at Captions.AI.
  1. Transition to Snapchat
  2. Gaurav joined Snapchat's elite engineering team, focusing on design engineering to keep innovation alive in a growing company.
  3. He contributed to significant feature developments, including the Spotlight feature and internal navigation improvements.
  1. Founding Captions.AI
  2. Gaurav and his co-founder Dwight discussed startup ideas and settled on video creation tools inspired by the rise of TikTok.
  3. Initially aimed to build a social network, they discovered a strong need for video captioning and editing tools.
  4. The Captions.AI app took off unexpectedly, generating significant revenue shortly after launch.
  1. Product Development and Features
  2. Gaurav discussed Captions.AI's unique features:
  3. AI-Corrected Eye Contact: Enhances the authenticity of talking videos.
  4. Text-Based Video Editing: Allows users to edit their videos by selecting words rather than using traditional methods.
  5. LipDub Feature: Converts videos into different languages while making it appear the speaker is speaking in that language.
  1. Future of Video Creation
  2. Gaurav predicts that video creation tools will evolve significantly, allowing for more intuitive editing and potentially even text-to-video generation capabilities.
  3. He emphasizes a user-focused design approach, aiming to simplify the video creation process for novices.

Key Takeaways

  • Gaurav’s journey highlights the importance of adaptability and learning in technology.
  • Captions.AI's success stemmed from identifying and solving specific user pain points in video creation.
  • The rapid advancement in AI technology has opened new possibilities in content creation and editing.

Call to Action

  • Captions.AI is hiring across various roles, including engineering, design, and marketing. Interested applicants can learn more on their [website](https://captions.ai).

Closing Thoughts This conversation demonstrates the impact of AI on creative processes and how innovative solutions can emerge from recognizing user needs. Gaurav Misra’s insights offer a glimpse into the future of video creation, emphasizing the potential for AI to transform how we communicate and express ourselves through media.

Stay Connected

  • Generative Now: [Website](http://generativenow.co)
  • Lightspeed: [Twitter](https://twitter.com/lightspeedvp), [LinkedIn](https://www.linkedin.com/company/lightspeed-venture-partners/)

---

These notes encapsulate the key discussions and insights from the episode, offering a detailed overview for readers interested in Gaurav Misra's journey and the innovations within Captions.AI.

Written by AI. May contain mistakes. Listen to the episode to check what was said.

Hear the part that matters, and keep it.Open this episode in VO. Double tap your headphones to save a moment as you listen.
Get VO free

Transcript

Automatic transcript. May contain errors.

0:05All right. Hello, everyone, and welcome to Generative Now. Now, if you're just tuning in for the first time, this is the show where we talk to the builders who are creating the world's most exciting AI products and companies. I am Michael Mignano. I am a partner at Lightspeed. And today we've got an awesome conversation for you. I sat down with Gaurav Misra, co-founder and CEO of Captions. Captions is an app that makes it really, really easy for anyone to edit their video using a bunch of really, really cool AI-powered technology. And in this conversation, Gaurav taught me a lot, including a story in which they found an app that they had decided to sunset and retire, surprise generating about$500 ,000.

0:50It's a fascinating story and I'm excited for you to hear it. So listen to this conversation with Gaurav Misra, co-founder and CEO of Captions.ai. Hey, Gaurav. Hey, how's it going? Good. We did it. We finally made it happen. Finally made it happen. I'm excited. I know. Me too. Thanks for doing this. Of course. Yeah. Anytime. So captions, obviously, you know, an amazing success story. We're going to get into it. I want to hear all about it. But I always like to start these things by sort of going back to the beginning. I think I feel like the company is in many ways about the founders. And so to understand the company, we need to understand you.

1:32So take us back. Early days, Gaurav. you know, give us, give us your life story. Yeah. I mean, where do you even begin? So all the way, all the way back. Yeah. So yeah. I mean, so I was born in, I was born in Boston actually. And when I was three, we moved back to India where we were originally. And so I grew up in India for the most part, uh, went to like, you know, school there, middle school, high school, all that good stuff. Where in India? In New Delhi, in New Delhi. Okay. Um, which, you know, it's a it's a whole place yes there's so many people yeah it's amazing but uh for college i it was just always known or sort of like a plan of like oh i got to come back for college and so yeah right when that came around started applying to a bunch of places i think i applied to like you know 20 places which apparently nobody does like most people just pick like three or four or something.

2:29I had no context, right? Like I had never seen, you know, what I went to Boston university. Like I hadn't, I had no idea what Boston university looks like. Right. Or, or for any other university for that matter. Right. Or like how they even compared to each other, what's better or worse. And what, what do you study at different places? Like I had no idea. And so I was kind of like blind coming in, never even toward the campus. Right. Um, but my parents were like, you know, Boston's really good. So you go to Boston. Yeah. And you never, you never went back you never visited no never yeah got it so basically it was brand new you know it was just like wait i've yeah i have no idea what i'm getting into and my parents just like came here they just dropped me off and they're like see ya wow that i mean that must have been that must have been really really hard maybe maybe going back a little bit what were you what were you into growing up in india what were your interests um you know what'd you do what'd you do to keep busy A lot of different things.

3:26My parents were very much believers in, you know, these extracurricular activities, as they call them. Right. And they were like, oh, you have to do as many, learn as many things as possible. So I was essentially enrolled in like almost every class and anything that you can possibly learn, every sport, every musical instrument, like just as many things as you can possibly learn. My parents were just like wildly spending on all this stuff. It's like having me learn as many things as possible. So that actually, I think, kind of helped me quite a bit because I think there's a skill to be able to learn something.

4:02Like being able to learn something is a skill in itself. And it's something that you can actually practice. And if you do more and more of it, you get good at it and you can pick up things really quickly. I did like language classes. I did all kinds of stuff, right? Right. So it was, yeah, quite interesting. And but I definitely had like a special place for just computer science, you know, because in India, the education system is a bit different. And like you actually learn computer science, like part of the curriculum from way the beginning, right? Literally from the first grade, I think. Right.

4:35And so by the time you get to like fifth grade, you're doing programming. So they start you off with like Visual Basic, which is what used to be back then an interesting language to learn. I did some Basic back then. Yeah. Yeah, it was the cool thing when you and I were young. It was a cool thing. And it is like a little bit further back in technology, too. So like a lot of like we had like DOS computers, right? Like when we would make like graphics programs and stuff, they would be written, right, like on the there was no GPU. you it was just written on the vga controller or whatever was there at that point right and so it felt like you know probably growing up like i mean we had like dot matrix printers and like yeah some weird stuff some old technology um but yeah so i was always very interested in that stuff and i always was like you know spending extra time just learning programming learning like you know how to do all these different things that weren't taught in class and stuff so it was like a special area for me and something that I always like really cared about.

5:40And so by the time you get to college, it sounds like you know how to program your programming. Did you major in computer science? I did. Yeah. Got it. So you already knew you already knew how to do it. Totally. I mean, yeah. In like 11th grade, we do C++. Right. And so we'd already covered like essentially like all the stuff that you usually do in college back in like high school. So you come out of BU and I think you had like a couple engineering jobs out of school. Is that right? Yeah, that's right. Tell us about that. Yeah. I mean, so out of BU when I graduated, I got an offer from, so I had interned at Microsoft and there was like a Microsoft Research Development Center in Boston.

6:22I think there still is. It's still there. And so I had interned there and they kind of gave me an offer and they gave me an offer for like a software engineer in test. What does that mean? So I think the role was basically, essentially, you were supposed to test software and write automation that tested software. And I didn't take that role just because, and I think it probably was the right move because they were offering like very high salaries for these roles. Because I think once you kind of get into that path, it's very difficult to escape it. and they were trying to get talented people into this role.

7:03But Microsoft actually completely eliminated that role at some point, I think maybe a few years down the line, just completely. Yeah. And so people had to like either switch and there were also layoffs and stuff. So it was probably the wrong path to have gone down. But I ended up taking a job at like a startup for like a very low salary. And honestly, I learned so much over there because that was my first actual application of machine learning in real life. So I actually got to work on the actual multiplication, like matrix multiplication happening when computing in inference, right? And doing that in a scalable way because the libraries and stuff didn't really exist at that time.

7:54So a lot of it people would just do manually. like they would just write the code for it um so that was exciting what was the startup it was called lattice engines it was like a boston startup okay and they were doing like sales and marketing like lead scoring basically which was like all the hype at that time got it okay so you're so you're there you're doing that um what was that experience like i mean what ended up happening to the company What did you learn? The experience was definitely interesting in many ways because I met a lot of people at that company, a lot of great, very talented people that actually kept in touch with it afterwards and then also had joined me in future jobs and pulled them into where I'm working in the future.

8:43The company didn't do too well, I believe. I think it did well for what it was, but it didn't have like some spectacular exit or anything like that. And yeah, I think why that might have been, it's tough to say. I was probably too junior to be able to figure it out at that point as to what was going on. Like my scope and vision and sort of like what I could see was very limited. But I did enjoy the stuff that I did. And, you know, yeah. And to sort of like complete that story a little bit more, like after I graduated from college, I was doing a PhD for like a good, yeah, it was like a little less than a year.

9:23But I started doing that PhD in machine learning. And I was kind of disillusioned with it a little bit because the academia, just the academic side seemed just a little bit slower than I wanted it to be and a little bit more incremental than I wanted it to be. And so that's why jumping to like a startup where everything is fast paced and we're just out there was like actually like super exciting you know in comparison yeah i'm sure so wait what what year was that and i mean i guess i have so many questions about this ml phd like curious to know like how different what you were studying and learning back then is from what you know what you're seeing today obviously we're going through sort of like an explosion of ai and machine learning like talk to us a little about those differences totally i mean it was a totally different time, right?

10:10I think, you know, this is a time when TensorFlow didn't exist, right? And just explain that for the listeners. Yeah. So it's like a library that, you know, a lot of people use for their machine learning training and inference and all kinds of things. It's something that's made by Google, right? And this was, you know, before that. And, you know, it was just around the time where deep learning was, you know, catching on and people knew there was something there. And there were some really promising research that had come out of it. And we could now actually train deep learning models efficiently, which wasn't the case before that.

10:49And so people knew that it's going in some interesting direction, but yeah, maybe it wasn't clear exactly where. And I don't think anyone could have imagined that we would end up here at this point. But what I was working on in my PhD was very unrelated to a lot of that stuff. It was more like towards the side of like, almost like data mining. Yeah, a little bit different or combinatorial optimization, like those types of problems. But it did prepare me for, you know, a career in this domain. And I wanted to sort of get more, just closer to the actual sort of ML applications and being able to actually build and ship stuff.

11:29And I would hear from my friends who were working at like companies being like, oh yeah, we just built this thing and we launched it and people are using it. And I'm like, oh, well, I could write a paper about it. I wrote a paper. That wouldn't actually, yeah, exactly. So yeah, that's what got me to the startup, right? And so then I got to do the actual nitty gritty and figure out doing the actual math and storing these models and retrieving these models and all that kind of stuff. So it was super fun. That's awesome. So, so you leave, you go to Lattice Engines, you're there. What ended up happening?

12:07I know you said the company didn't have a great outcome. Did you end up leaving? Did the company get acquired? What happened there? Yeah, I ended up leaving. I think eventually the company got acquired, I believe. But I think, yeah, I went to Microsoft after that. And that's kind of where. So you ended up at Microsoft after all. I did. I did end up at Microsoft, but this time in the right role and on the right team. So they had spun up a new team in Boston. And that new team was focused on building like an ML platform on Azure. And so the idea was that, oh, anybody should be able to do machine learning.

12:53Kind of the classic like drag and drop. You can drop in different models and you can connect data to them and you can train these things in a GUI type of interface. And the models at that time, these weren't deep learning models or anything. It was support vector machines and decision trees and random forests and that kind of stuff basically. You would be able to have this visual interface where you can drag and drop stuff. The idea was more people would be able to access this technology and use it. And so this was a brand new team. So it was like a good time to join because it has like the startup energy, met a bunch of great people there that I still keep in touch with.

13:34Actually, in that group, a bunch of really interesting people came out of it. Actually, my manager was at that time, I think most recently was running engineering at Cruise. and then the PM lead who was there is the CEO of Slack now. Oh, wow. Some interesting people came out of it for sure. A little bit of like a mafia, the Microsoft Azure ML mafia. That's awesome. But yeah, so ended up obviously getting a lot more exposure to ML over there. But one of the interesting products that sort of was in that orbit was this product that would guess your age. And essentially, you would upload a photo. And this is a Microsoft product, by the way.

14:29And it was a web page where you would upload your photo. And then it would just guess how old you are. I think it was called howold.net. And it went completely viral. I think I remember this. Yeah. And so it got me really excited about like those types of applications. That's how I heard about Snapchat initially. Really? Yeah. What do you mean? Because Snapchat was sort of in that domain of like face filters and things like that. Right. And so it felt like, oh, yeah, is there a company that's sort of in this domain that's working on anything like this? So that kind of stuck with me. but um yeah i ended up leaving microsoft i think after a while and then joining another startup after that so that was where i met my co-founder actually um at localytics yeah that was i feel like at a in a certain time that was like a pretty important company for startups it felt like one of the most accessible sort of analytics platforms if you're building an app or a website it was one of the first yeah yeah yeah it was one of the first i mean there was like that was there was another one that was even before like flurry probably flurry flurry i was thinking yeah yeah but localytics it's a great combination of like being powerful yet also being really accessible affordable for startups i mean i used it as at a couple startups that i that i worked on so um yeah so my company anchor was definitely the first thing we used and then i think we used it at aviary before that so it was a good product um yeah definitely a good product so what were you doing there yeah so i came in as like a an ml slash data engineer so there were a couple of different sort of big opportunities of the company at the time so they were using like a database called vertica which is like a column database basically and it's an older school sort of technology.

16:23And it was reaching the limits of its scalability in terms of like how many requests it can serve and things like that, just because of the way the database was designed. And it was costing the company a lot of money to run that thing, like a ton of money. And so we discovered this thing called Snowflake, which nobody had heard of at the time. And we were one of their first customers and actually the largest customer at that time. Yeah. No way. So part of what I did was, you know, migrate to Snowflake, basically figure that out. Got it. And then the other part was building ML models for different types of things, like best time to send, things like that.

17:05So I was part of like the hired as part of the ML team, basically. Got it. So did that. And like Dwight, my co-founder, like he didn't directly work with me. He was on a different team. He was a PM, you know, working on other things. And we overlapped for like less than six months, I think actually in total. Uh, so it was a short period of time. I have a couple of questions about that, but we should, let's cover that when we get to the, um, get to the caption story. Um, but I guess in between localytics and captions, you, you, you go from, from doing sort of analytics infrastructure to being at kind of one of the most like consumer facing companies there is right snap yeah talk to us about that journey yeah i mean i think you know i was trying to figure out what my next step is and you know i wanted to see sort of what it was like um you know at a large scale sort of consumer company and kind of you know what problems there are and like people were obsessed with distributed systems and things like that at that time and And everybody wanted to work on those types of problems.

18:13So that's what got me excited. And plus the ML applications and stuff, I knew that there was something interesting about this company. And at the end of the day, I spent a lot of time thinking about whether I want to move to New York, this place I've never been to, and all my friends are in Boston, et cetera, for this job. But just something just felt right. I think there was something about that team. The person who started it is probably one of the most amazing, like, just technologists, I guess, that I've ever met. Like someone who's really able to just create something out of nothing. And this person started the New York team.

18:58He's like an engineer by trade, I should say. and he started off the team and the interview was pretty wild and just different than anybody else. The people were just like very smart and something just felt like, wow, there's like something happening here that I can't really exactly put my finger on, but I love it. So when you say this team, that's the New York team or is there a specific like function of this team? It was just the New York engineering team. It was very small. It was like at that time, maybe like 20 people or less. it was probably like 10 to 15 people. I don't remember exactly, but in that range.

19:35And this amazing, super innovative person, who is this person? His name is Andrew Lynn. Okay. He's done a bunch of different stuff. He was at Hulu. I think he was like the CTO of Airtime. I don't know if you remember this company. Yeah, of course. So yeah, a bunch of different things, but he's like exceptionally smart person. Very different, very rare to run into these types of people. so and he'd built this whole team around like that just that culture of just innovation and just like hey let's just build something you know let's just try a bunch of things and see what happens and throw some things on the wall and just see what sticks type of thing you know which i wasn't really used to because like that wasn't you know as an ml engineer and stuff like it's just it's not what you're doing normally right you know everything is like a two-year process or something like that and you're just kind of going down a road there's less exploration maybe right And so, yeah, that was like completely new.

20:32I think it definitely taught me a lot about kind of how to build good product, how to think about exploring in the product space and how to think about actually solving user needs and solving, you know, real problems that users are having. And you were this team, or I think, you know, the role that you, I believe you ended up taking on was running design engineering. Is that right? That's right. So I actually wrote a blog post on this, which is on the, I think it still might be on the Snapchat blog. But, and there's a story of kind of how I got to design engineering. That was like maybe two years down the line from when I started on this team at Snapchat in New York.

21:14And the specific purpose of design engineering was that as companies grow, they kind of slow down a little bit, right? And like there's massive organizations for engineering and product and all kinds of stuff. And the company itself starts to operate a little bit slower, becomes a little bit more risk averse. So there's like, you know, you don't want to just like change a bunch of stuff and move super fast because, you know, there's existing sort of business to protect. Right. And so the idea was having a team that's able to iterate quickly and ship products very quickly in isolated ways, like a couple of high schools or like things like that.

21:54test the product, figure out whether it actually works or not. And only if it actually works do we move sort of the larger engineer organization to build that thing at scale and solve all the problems of scale that are needed for a company like Snapchat. So we would build tons of products very quickly and experiments and even small changes, features, whatever. And we had this system where we would be able to layer that on top of the Snapchat like main app basically. So it would be like a series of PRs, right? That would all get merged, you know, on top of without having any, you know, branches or anything actually truly checked in.

22:34It would all get kind of merged automatically on top and people will be able to work on different things and it could create different combinations. Builds were like different combinations of features and things like that. And that was distributed as an internal build inside Snapchat called Spooky. and people could get spooky to see like what experiments the company's running basically internally or what where the thought process is going basically around how the product evolves and you'll see all the new features in there and uh some of those things might make it right so then engineering will have to help sort of build those things out and actually take them out to market got it so design engineering i when i actually read it i thought when i read the title I thought, oh, this was the team that sort of like implements design, you know, whether it be like animations or, but that's not it at all.

23:22It actually, it sounds like it's more like the innovation team. And it's like the skunkworks team. It's like a product experimentation team. Yeah, exactly. It's a skunkworks team. That's exactly right. So we would get a lot of like direct feedback from Evan on like, Evan is like the CEO of Snap. And he would say like, you know, we, you know, we should try something in this space or we should do this or that. Right. And then we would come up with a lot of ideas and try to build things and present it to him, present it to other people in the company. And anybody could download Spooky so they would see these things.

23:53And it would also build momentum in the company very quickly because people would see these things and it would go viral inside the company. Yeah. Because people would talk about it, oh my God, this is a cool thing someone's trying out. Have you seen it? Have you heard about it? Right. And so a lot of things would gain momentum very quickly. so that was like the general nature of how it worked and so that team was a part of the snapchat design team so it was like part designing part engineering sort of like a mix of the two got it i've heard that snap had a very unusual product development process in which a pm or a product manager is basically also a designer and they're just sort of living in at the time i I believe it was probably Sketch, maybe it's Figma now.

24:44And you're just designing features and then just sort of like showing them to Evan. And Evan's like that one. And Evan's obviously a designer and a product person too. Is that accurate? Is that how product works at Snap? It's not inaccurate. I think I'm not sure how it works now, but at least at the time that was, you know, I think Evan cares a lot from what I can tell. I can't speak for him, But it seemed to me like he cares a lot about design and to make sure that products are built well. And I think he had some unique insights about what made Snapchat tick that really almost nobody got ever.

25:24Right. Like he knew things that nobody else knew. He understood the product and the people, the users in a way that nobody else maybe ever did. Right. And so there were many instances like so I think what you're describing is correct. Like he had this design team. He still has a design team, I guess. But like there was this design team that was, you know, directly working with Evan. He would meet with them. They would drive both sort of the product roadmap and design things as well, right? Both at the same time. And they would be kind of driving a lot of like what the company did in terms of product.

25:58Obviously guided by Evan, right? And so it's a unique structure. And like having like 10 or 12 designers running things in a, you know, three to 5 ,000 person company is definitely, it's a unique way to do it. I mean, it worked in many ways, right? Like it did. And I think one of the benefits of that approach is that it keeps everything very cohesive and doesn't let things kind of diverge into different directions. I think the design team and Evan probably cared more about like visual changes and product changes that were happening. And maybe a little bit less about like, oh, let's optimize, you know.

26:32The login conversion rate or whatever, something like that, maybe could continue to go on and driven by PMs and stuff under the covers. But new products and true product changes were always driven by the design team. Got it. And so maybe shifting away from that, your team, the design engineering team, Skunkworks team, trying new things, running experiments. I believe some fairly big features came out of those experiments. Is that right? That's right. So we actually did the first prototypes and the first builds of Spotlight, which is like Snap's like vertical scrolling, you know, video platform, basically.

27:14And it was pretty wild because we actually, you know, Snap used to be different, if you remember, right? Like Snap's always about horizontal scrolling. Snap's always been about video scrolling horizontally. Yeah, stories. Yeah, exactly. right? And so videos crawling vertically was like a whole new concept, right? And I think there was a lot of internal discussion about like, even just like the ease, the thumb, right? Like moving up is easier than moving sideways and like just the efficiency of that or something like that, right? And what impact that could have. But with Spotlight, we did sort of the, we did the initial prototypes.

27:51We built out, like we experimented with whether we should have a like button or not and we we had like different variations with like you know a lightning bolt i think it was like called boost at some point um there were many other like just like experiments that we did and we even made our own little algorithm which would rank some of these things it was very like it was a prototype right but we took a bunch of the videos rank them with this algorithm use different signals to like resort them and like people in the company would play around with it they would like swipe through and be like oh i don't like these videos like what algorithm is this one right um and so we did a lot of that initial exploration for that product another sort of big thing that we ended up doing was if you remember snapchat used to be like three screens the inbox the camera and then the discover plus stories on the right side yeah okay but there were a couple other features like the map right and the map actually was the most amazing thing it was behind a pinch right You had to pinch on the camera to get to the map.

28:52And then there was also a shows page, which was really premium content, like mini series, things like that, which was actually one additional swipe after Discover. And it was quite hidden actually because of that. And so Snapchat never had buttons for any of these things. You just had to swipe and you just had to know to pinch or to swipe or whatever at the right time. There was no labels, buttons or anything. So one of the big things that we did was actually introduce the navigation bar, having the five tabs. And that's actually something we not only sort of concepted and designed, but actually also shipped in production.

29:32So the actual production code went out from our team, which was a little bit different than how we normally operate. It's tough to do production code. It's a lot of work and you need a lot of time and a lot of just eyes to be able to make sure it works at scale. so normally we wouldn't touch that but uh and we would have like engineering teams build that out in the right way but it was a bit of a rush thing and we yeah that's one of the things that we actually like truly shipped to production as well yeah i remember this it's it was a big deal at the time um because i think what you're talking about these sort of like hidden gestures was sort of a big part of the culture of the snapchat product that you know if like you were in on it you you understood it but i i sort of remember and i might be imagining this but i kind of remember some like earnings call or something where evan was like we need to make the product easier to use like our interface is like holding us back and then i remember them shipping this this big navigational change and it was just like this huge shift for the product yeah that's really cool to know that that your team was behind that that's right yeah it's actually one engineer on our team We did the whole thing.

30:40Oh, wow. So, yeah. That's amazing. So, an amazing experience at Snap. And maybe talk to us, like, how do you go from Snap to then starting captions? Obviously, I had been in touch with Dwight. And, you know, we were sort of meeting up every couple of months. And, you know, we would discuss startup ideas and, like, what all we could build and, you know, what the opportunities are. and it did seem like you know there was a big opportunity on the video side and that things were you know getting really interesting because you know 2019-ish is like when TikTok really started to take off in the U.S.

Read the full transcript

31:20and we saw like that evolution from like dancing videos to like talking videos and then people like actually expressing themselves and sharing opinions and stories and things like that. And I think what got us excited about that was that for the first time like in the history of the internet, people were able to really express themselves on a new level and things went viral, not because of the visual or aesthetic nature of things, which was kind of the case before, but more for like people would go viral for their personality, right? And so if you We can see body language and facial expressions and just the level of information just was elevated like never before compared to text.

32:07And that was new and it definitely was exciting because of that. And so our initial take was like, hey, let's actually build, we're going to build a social network or something like that. And we would start with sort of the creation tools and then use that to bootstrap some sort of network or something like that. but that's kind of what got us excited i mean of course things were going great at snap like you know it was a amazing place to work amazing people uh but i think i just wanted to do something and it felt like the right time um and yeah it just yeah it's just a feeling i guess so you you and dwight you decided to do this together maybe talk a little bit about that because you made a point earlier of saying you worked with Dwight, but not that closely.

32:57You didn't know him that well. How do you guys end up then co-founding something, which is about as close as you can get to another person that's not like your spouse or family member? Yeah. I mean, so we actually, we did keep in touch quite a bit after we both sort of left localytics. And we both obviously ended up moving to New York and we would catch up like every couple of months and like get dinner or whatever or drinks and just like throw out ideas of what to build and what's what could be cool and what's interesting uh but it was always the case of like oh well it's not the best time for me and then oh yeah it's not the best time for you okay i get it right like and so like that kind of continued for a long time until 2021 when the when the stars aligned i guess and and and what is the idea behind the initial social network that you guys decide you want to build is it is it similar to tick tock or is it something else completely i think we wanted to focus more on the communication aspect right like just like i think we were not sure exactly what it was going to be to be honest but just that we wanted to focus on talking videos and get people like you know either talking to each other or something like that you know so that was just the general gist of the idea that's all we really had and we were experimenting with like community apps and like video communities and things like that uh that was like the first uh couple of weeks we did that um and captions actually was created in that time as well got it what is a talking video and sort of what's the what's the origin of that where does that come from that behavior yeah i mean i think so it's a video in which someone is actually talking basically right and that could be a voiceover or it could be you know someone talking with like a talking face video basically and um it's something that i think really gained popularity post tiktok i think it's actually something that originates from tiktok a little bit um because before that like videos were just like you remember how like videos used to be just like someone zooming into food, like that was a video, right?

35:08Or like someone panning across a landscape, another video, right? Or yeah, that was the nature and extent of videos before that. Like no one ever said anything in a video, right? People would just like kind of zoom and pan into things basically and maybe write some text on top of it, right? Which was a Snapchat thing, I guess. But the talking video, I think, really took off with TikTok, especially at the scale that we see. I think there were probably other companies doing similar things like Loom. It's kind of like talking videos in a way, but for different use cases, right? What do you think the TikTok use case is of talking videos?

35:48People are taking landscape videos or food, but what are they trying to do with talking video on TikTok? And why did it happen on TikTok, I guess? Yeah, I think it happened in TikTok because TikTok was the only platform where the default behavior was to wait and watch, right? Like by default, anything that comes on your screen, you're kind of like, oh, TikTok, the for you page chose this for me. It must be good. I'm going to watch a little bit at least, right? And so you give everything a chance. Whereas before that, every platform, including stories, like the default was skip. So how a lot of people consume stories is like, skip, skip, skip, skip, skip, skip, skip, skip, boop.

36:28That was interesting. Go back. All right. Skip, skip, skip, skip, skip, skip, skip, skip, right? Like that's how people consume stories, right? Whereas TikTok, it's like swipe. You're watching, you're watching, you're watching. Oh, it's not good. Okay, next. Right. And so it's a complete shift, right? It's actually like one of the most amazing innovations in many ways, right? It's almost like a platform that's just like constantly running ads and everybody is making ads. and all these ads are competing with each other, right? That's basically what it's doing. I was just going to say that. It's a great analogy because, I mean, per your point, you know, the default behavior is sort of look at the video and decide if you want to consume it.

37:06And so if you think about how that drives the creator's incentive, their incentive is to basically win you over in those first few seconds, right? Or the first second. And so how are they going to do it? I mean, I think per your point from a few minutes ago, they're going to talk to you. They're going to talk to you and they're basically going to try to win you over. Totally, yeah. It's a sales pitch. It's a sales pitch. It is, yeah. And personality and all that stuff really comes through. And the minor things really matter on platforms like TikTok, right? Because all these things are competing on a global scale of content, right?

37:35Like there's just everybody making content. They're in constant competition and TikTok's algorithm is constantly looking at the top performing in all these different metrics, finding the best of the best, surfacing it to more people, looking at their performance, finding the best of the best, surfacing it to even more people, right? And that's just a constant loop that's running and incentivizing everybody to optimize, like, how do I really best connect with the audience, have them watch, you know, whatever video they're making as much far through as possible. So that's the game, right? And so, okay, so talking videos happens.

38:13You decide to build this product, maybe a social media company at some point, or social media products, how does that lead you to the product that you have today? Like, how do you end up optimizing for creating talking videos, I guess is the question. Yeah, so I think that was the big question for us, right? Is how do we make a talking video platform is what we were thinking at that time, right? And so we thought, let's start with the tools, right? So when we thought about the tools, we were like, well, let's start with something we can do for talking videos, right? Where people can have it easier somehow, right?

38:45And so that's where the idea for captions came was, oh, let's have maybe automatic transcription, right? Because that's not something that existed at that time. I think a lot of people forget that TikTok, Instagram, like nobody had that at that time. It didn't, you know, exist in the form that captions made it, where it's like appearing on top of the video and stuff like that. And so a lot of people at that point, there was a movement around accessibility and a movement around like, you know, people would sound off and I want to understand what they're saying. And so a lot of people would manually do that.

39:23You know, they would manually put all those words and there was been an hour, like a whole hour doing that manually. Transcribing their own audio. Exactly. Yeah. So, you know, we kind of noticed that we saw that we're like, hey, perfect. this is the place to begin because it caters to talking videos is the perfect thing to start and we made it you know made the app i think it took like a couple of days to put together the first version launched it and kind of went to sleep and woke up the next morning and it was at the top of the app store so that was a surprise oh my god yep it took you three days yeah very fast yeah i mean I remember just calling Dwight and being like, oh my God, there's like, you know, 600 videos an hour being made.

40:06Like, did you do something? And he's like, nope, I didn't do anything. And yeah, it just did that like crash your servers. Like, I mean, were you ready for that? Yeah. I mean, the app was very simple at that time. Like we had no service. There was no backend. It would just call directly into Google APIs. And I mean, the bills went up and we hadn't raised money so i was paying like ten thousand dollars a month or more out of pocket but it it didn't like there were no servers so yeah nothing could crash basically um it was quite a simple app really and that was the beginning right and that's kind of where we got interested in like oh we can solve problems in video creation and use you know ml in different ways or you know design in different ways to solve them uh and we got really excited about the talking video space obviously after that we got even more excited right so that's kind of where things started off and when was this this was um 2021 like early 2021 probably yeah and was this the first thing you tried so you see you were like okay here's this problem transcription three days build it boom top of the app store yep it was the first thing we tried and actually we were doing sort of vc pitches at that time.

41:23So the timing was absolutely perfect, right? Because in the middle of the pitches where people were starting to question like, well, is this really going to work? We had something to show off like, wait, it is working. You can go check it out. Right. So that helped us a lot. Wow. What a story. I mean, it's so rare that that actually happens. And are you like, while you're pitching, are you pitching the social media side too? And he said you want to start with the tool. Yeah, we are pitching exactly that. But I think people would obviously question like as I would, if anyone told me this idea, right?

41:57It's like, okay, sure, maybe if you pull off the tool, which is pretty hard to begin with, you know, how do you even make that into a social network? That's a whole other challenge, right? Right. It's hard to actually believe that, right. And so but people are getting even stuck up on like, well, actually making a tool is pretty hard. Yeah. And so the timing was absolutely perfect, because we could then show like, hey, wait, actually, we did make a tool and it is working wow okay so you so you have this like success out of the gate you said it inspires you to now want to solve these other problems where do you go from there it's just you and dwight at this point it was yeah me and dwight we had a contractor uh who actually is still with us um nice uh who's awesome yeah he's in the canaries uh the canary islands that's awesome so where so where do you go from there i mean the thing is exploding you're raising money yeah i mean then we set on our on our path to make a social network which oh you did basically we did yeah we did and that lasted about about a year or so um we spent a year you know really trying every possible idea trying every possible angle and i mean the bottom line is it's really hard i think you know everybody knows this and i think we knew it too but we really wanted to do it right so uh hard to blame us for it but we we pivoted to doing more photo stuff at some point and that started to really work um so i think there was also a little bit of a movement at that time around like instagram was becoming more video and so people were like well where's my photos right like where do my photos go and so there was a need building that we could see on like platforms like TikTok and stuff with people talking about, you know, well, where do photos go?

43:45Right. And so we started doing photos for a while that worked fairly well. We had like a US only app that we built for photo sharing and that took off. And, you know, we hit, I think we hit like 50, 60 ,000 Dow in the US, which solid, you know, and yeah, totally. And so we ended up going and raising a series a on that actually um which was like early last year yeah and at this point is the is the talking video tool still happening it's like running off to the side while you're building the photo app it is it's there it's kind of sitting there and we actually you know we're a little bit worried because it's costing money right yeah yeah like it's not a lot like $28 ,000 a month or something, but that's a quarter million dollars a year, right?

44:40Yeah. It's not nothing on a seed company's balance sheet, right? And so we were debating whether we should just kill it, which was, we discussed for a bit and we were like, actually, we decided, you know what? Yeah, let's shut it down. so when i went to actually go shut it down i had this like random idea of like why not just put a paywall and that way if and the paywall will block the entire thing and so if no one pays then it's essentially shut down right right for all purposes and if people pay then it'll pay for itself and we don't have to shut it down so we can stop worrying about it and so i put the paywall and I shipped it just like on the weekend without telling anybody actually.

45:32I think our iOS engineer, I told her, I was like, oh, if you see PR coming in from me, don't worry about it. And yeah, and that was it. Then we forgot about it basically. And then we were working on our photo sharing app, which was doing really well. And that's all we were focusing on. And we raised a series A for that app. We hit 50, 60 ,000 DAO. And literally like millions of photos have been posted. Things are going pretty well. After the Series A raise, I went into my account, my personal Apple account, which was separate from the company one. And the personal account had the Captain's app on there.

46:15And I go in there and it's made$500 ,000 basically.

46:23And you're looking at the ARR growth. that is just like going straight up basically with zero employees, just zero people working on it, you know, zero customer support. There were like 1800 open support tickets that no one had answered for six months. Right. And it was just kind of just going. Right. And so that's kind of where we had to really think about, you know, what do we really want to do here? You know, and we got really excited about the original vision with, you know, captions and all the things we had thought about with ML and AI and all that stuff. And, you know, yeah, it kind of made clicked for a second a little bit more of like, you know, this is all the things that I've worked on in my career in one thing.

47:09And it's working. Exactly. Yeah. And it's all of my skill sets in one place. Right. That's fascinating. So it wasn't even like the social network had necessarily failed. Like you hadn't given up on it. You just have this other thing over here that is just clearly working so much better. So, oh my goodness. So how do you then make that transition? Everyone just signed up to do this social app. Like, how do you tell your investors? How do you tell your team? Yeah. So, I mean, we had to, team was, you know, hard in a way, but I think we had the right team. We only were, we were four people at this point, right?

47:47Okay. So, you know, I think people were in the mindset of like, yeah, let's just build new stuff. and we'll pivot and we'll do whatever we need to do. So it was definitely a setback though in a way because there was so much excitement. We had just raised Series A, right? Like people were actually excited about what we were doing. But I think once we had discussed it as a team and like thought about the pros and cons and like we thought about like, you know, what the business model could be for a social media company and like where that could go. And the options were getting more and more limited, right?

48:16Like a lot of companies are still struggling to figure out monetization. They're public companies, right? Right. So, and Apple is not, you know, a friend to these companies either in many ways. So that was definitely heavy on our mind at that time. Right. And so we got a lot more excited about the sort of video space because it was back to videos, the things that we really cared about using ML, like what we really cared about. So yeah, that's what got us really excited. But the team was so, you know, easier in a way. I think investors, you know, maybe we took it a lot more seriously than the investors did.

48:52You know, we went to our investors, we told them and they were like, oh, like, are you sure? Like, what are the pros and cons? Like kind of the classic questions, right? But at the end of the day, they were like, you know, do what you think is right. And if you think this is right, go do it, you know? And we were like, amazing, let's do it then. Was there any sort of hesitation from them around, hey, we signed up for a social network, social networks end up, you know, the winners end up being the biggest companies in the world. like that's what we signed up for or were they just it sounds like maybe they were just like whatever whatever you want to do i think there was i think maybe to some extent they still thought that it could still evolve into that at some point okay um at least in the beginning but and there was some like proving out to be done from our side too that this can work as a business um Um, but I think once we started working on it and the trajectory changed, like almost instantly in an almost like extreme way, I think everybody was on board very quickly.

49:55What a story. Crazy. Yeah. Yeah. It's, it is crazy. So then you, so you go all in on the video app on captions. Um, and initially you're just doing dubbing or not, not even dubbing. You're doing, you're doing captions. Translation captions. Yeah. Translations. What next? Now, how do you start thinking about other problems you can solve? Yeah. So, I mean, at this time, you know, there wasn't really a text-based like video editing app that existed. There was Descript, which was doing more audio related stuff at the time. And so we saw that like as a natural transition of like, okay, how do we make talking videos easier to edit?

50:35Right. So the first thing we did, this was the first things we literally they started working on is making editing word-based, right? Because you're talking in the whole video, you don't even have to listen to it now. You can kind of like scrub through, find the right words, you know, add an image there, you know, cut the video after this word, right? Or whatever you want to do basically. But on your phone, simple interface, very easy to use, right? Low, just low touch, almost like low complexity, few buttons, right? And it was a huge sort of exercise in like progressive disclosure and just like designing things in a way where things kind of reveal themselves to you when you need them rather than all the time you know being present like you don't want like 30 buttons on the screen and like overwhelming choice all the time especially as someone's starting out right you want like limited you know simple to use uh things come up when you need them right so a lot of it was an exercise of designing that type of interface which is very different than any other app that was in the market at that time.

51:37And that's where the real product market fit started. When we started doing the text-based editing, the video editing stuff, but you do it by word, that's where it really took off. And that's where the trajectory just completely changed. Really? Yeah. I mean, what looked like a straight line up before became a flat line after that. It was that big of a difference. Wow. Wow. Was it a specific feature or a specific release that you remember where it just changed? Yeah, it was like a July release from last year. It had all the text-based editing stuff. It had like, you can basically do all of video editing just based on words now.

52:16That's so cool. It's so funny, like, and I think we've talked about this with other guests, but sort of one of those moments where you kind of, before it, you kind of think you have product market fit, but you're probably doing a lot of like the pushing the boulder up the mountain yourself and then you get over that that hump in this case like that july release and you see it actually rolling down the other side you're like oh wait totally now this this is product market wow very true that's so cool so you so okay so you do the text picks video editing and then um what other features do you start to bake in i mean now the captions app has so many features how do you get to those totally i mean so at the end of the day like our goal was to help people make videos, right?

53:00And we've thought about it from really a first principle standpoint and just like thinking about like, you know, forgetting about how things have been done today and what's been done so far and really thinking about how do we provide the best experience for people, you know, and really focusing on our target customer, which is like not the professional, not the person who's already done this for two years, but really people starting out, which we think, you know, is a pretty big market. And so we designed the entire app, everything from like, you know, the idea generation stuff, script writing, all of our camera related features, right, editing features.

53:35And then even on the export side, right, we did a bunch of innovation there on distribution. And so all that was designed to make the process as simple as possible for anybody, right. And at the end of the day, what people come to us for is making videos, right. And so we are trying to build and have built almost every feature that you would want to have to achieve that, right? And so over time, people would ask for things of like what they thought we should have, features we should have, shouldn't have. People even have opinions on like what we shouldn't be doing, right? And so we think of our roadmap in like two different categories.

54:11There's like the public roadmap, which we think about as anything that someone would ask us for, right? If anyone asks us for even a single time, that's public roadmap for us, right? And sometimes there's obvious stuff, like operational, sort of nitty gritty, like people want HDR export or people want the ability to undo and redo or something like that. These are obvious just things in the workflow. And we must do these things. We should prioritize them as what might be most important to users and get them done in the right order. But they are must-dos and they're essentially table stakes to be able to just sit at the table.

54:52Right. And the other thing is everybody will have them. Right. We will have them. Everybody else will have them. They'll be fine. So that's the public roadmap. And then we do the secret roadmap, which is really about, you know, coming up with new things that nobody asked for directly, but we think will like change behavior in a meaningful way. Right. Like when someone tries this new way, they'll never go back to the old way again. Right. Whatever that might have been. Right. Right. So if we can look at sort of user behavior and figure out those places, you know, how can we really change user behavior, simplify things, right?

55:26Like draw new paths and new lines across, you know, places that were difficult to get through for people before. Right. That's been sort of our play of how we use AI specifically too, right? Is to cut through a lot of the noise for people. What are some of those features that you've shipped where, you know, you sort of said going in, this will change the way people do X? Yeah, I mean, eye contact is probably one of my favorite examples because that came out of like, we did a bunch of work on teleprompters first, right? And we were helping people record videos. How do you actually make it easy to record something?

56:02And it's kind of hard because you'll have to memorize the script and repeat this thing and you have to do all these retakes and like, oh, I said it wrong, start again, right? And you have to think about so many things. And on top of that, you'll think about like, okay, I got to be looking at the camera, right? because I want to build trust with my audience and it matters. And so we started doing, we introduced a teleprompter and we were trying different sort of like AI power teleprompters, or I should say just like ML power teleprompters. So they listen to what you're saying, do the pacing automatically, things like that.

56:37That's smart. Yeah. So small things like that can make a big difference. But then people were like, well, it looks like I'm reading. My eyes are moving side to side. It looks like I'm reading. And so we're like, okay, so maybe we'll move the teleprompter right close to the camera, right underneath it. But it still looks like I'm reading. And so then we're like, how can we just get rid of this reading problem? And that's kind of where the eye contact thing came from. And we got immediately super excited about that as soon as that idea came across. and we spent you know a good month or two like building it out testing it and it was like one of our most popular features like we still go viral for it today you know it's been like what a long time since we shipped it right and it still goes viral until today because people are like wow this is actually pretty cool that's really really cool yeah i mean it makes total sense it's like you could try all these different band-aids and hacks to like fix the problem or you just change the output totally yeah just let them read however they want to read right i mean that's exactly really smart um and you're right it does kind of change the way people would think about reading for their videos right totally i'm sure with that tool you can just focus on whatever you have to do to make it easy to read exactly you can put a script to the side like a lot of people like to read from paper right you can do that if you want to you can do whatever you want right and right i I think that's the kind of stuff that just, it's a workflow changer, right?

58:06It's like, and it's hard for any other company that then compete with it, right? If we're building teleprompters in the business of building teleprompters, there's competition in teleprompters. Then it's like, you know, everybody's copying each other's teleprompters all of a sudden. Well, it was something like this. It just completely changes the entire game, right? You don't even need a teleprompter anymore, right? Yeah. And recently you announced LipDub. Talk a little bit about that. Yeah, so this is something we've been working on for a while. We actually started working on the dubbing, the lip dubbing technology, right about that time when we were doing the eye contact thing.

58:39We actually started looking into what was available, what the research was so far. And it was pretty limited at that time. I think it's kind of accelerated since then quite a bit. But we started working on training sort of our initial models back then. And over time, we've had to evolve and evolve a bunch. and build these things over and over again from the ground up to be able to get to the point where we are. So to give you the context, LibDub is an app and it's a technology as well, right? Which is able to basically take a video in one language and then convert it to a totally different language and make it look like you're actually speaking that language.

59:26and you know I think what got me most excited about this is because I'm actually like I love languages and love like learning languages and things like that just from my background and stuff and so seeing something like this it's like just so amazing to like see yourself speak like a totally different language that you wouldn't expect right and the use cases that come to mind with it you know obviously with dubbing you know you could think about like creators you could think about film. You could think about all kinds of different things. So we were just really excited about this application. I think one nice thing about this application too, is that it's generally like a, it's like a net positive for the world.

1:00:04Like people just get more connected, right? And there's limited abuse potential, which is amazing for this type of technology, right? It's like an opinionated product. And so, yeah, that's, you know, that got us super excited about it. That's awesome. And what are you seeing people do with it most? Is it the creator use case? Is it, I don't know, communication, people being able to have a conversation but not speak the same language? Definitely. I mean, I think we're seeing a little bit of both, but maybe more on the communication side even than we had expected. Oh, wow. Like people sending messages to their grandparents who don't speak the same language or people sending a message to their significant other who was from a different country or grew up with a different language and like surprising them or something right uh we've seen even people like talking to each other kind of back and forth you know uh and i'm either hearing that from like you know from our users like we have intercom and stuff so people reach out to us um and then we kind of get that in reviews and all kinds of places how much of this is made possible by what's happened within the past year or so in ai right like obviously there's been this explosion of AI technology.

1:01:15It feels like, you know, the first big moment was Dali. Then of course it was ChatGPT. Like how much, how much has that actually enabled what you're able to do today? So much, so much. I mean, I think it would have been practically impossible before that, right? Because think about it. The core translation has come along so far too, right? Like we use GPT-4 to power a lot of the core translation, right? like the actual language to language, right? And, you know, it's so much better at being able to figure out like nuanced translations. I think one thing that a lot of people don't think about with translation is it's actually not as simple as Google Translate makes it seem.

1:01:59Because in a video specifically, depending on me being like using masculine or feminine pronouns or just like my own identification, right, certain languages will completely structure the sentence differently, right? The sentence will sound different depending on, will be structured differently depending on what you choose. And so that can like completely, that can be lost in a Google Translate context, right? The Google Translate will always pick like the masculine, I think, or I don't know which one. And so you can't tell Google Translate, like translate this, you know, with this specific nuance, right?

1:02:33Or there's respect levels in certain languages, both for yourself, like in Hindi, for example, you can speak a sentence respecting yourself or kind of like lowering your respect level for yourself. And you can speak to other people with three different respect levels, right? Of like sort of the everyday, sort of the middle and like the high respect level. And that kind of context can't be provided to like Google Translate, right? But it can be to GPT. And so it really changes the game for a lot of like translation oriented use cases just in a way that wasn't possible before. And it actually ends up being a lot more accurate too.

1:03:10And it's really good at different types of translations, including using like slang and using like everyday language. And, you know, we actually have a feature that makes you like, just uses all of Gen Z terms from like TikTok and makes you sound like, you know, a TikTok or something. So yeah, all that kind of stuff isn't possible, you know, without some of the more recent technologies. And then, you know, with our own models, you know, When we were training them, even with the A100s, which was a state-of-the-art a year ago or whatever, it would take three to four weeks to train these models on A100s.

1:03:45And with the H100s, we can do them in less than a week, like five days or something like that. So it is a huge leap. And the more data we use, the better models we can train. With these more recent GPUs, we can train them even faster, run the cycles even faster. because every time we find some issue and then we fix that issue, we rerun it again. That can happen a lot faster. So all these things like wouldn't have been possible. It's incredible. A few minutes ago, you mentioned how, you know, these features, each time you have one of these like really transformative features like eye contact or the text-based editing, like it just, it completely changes the way you solve a problem or a person solves one of their own problems.

1:04:31If you sort of fast forward and you look into the future and you think about all the ways that you can sort of like 10x improve these problems, like what does video editing or creation even look like five years from now? Yeah, I mean, it's going to look completely different from what it is today. I think that's for sure. I think there will still be like levels of like how deep people want to go into it. And there's going to be professionals to do it. There's going to be everyday people who are going to do it. But, you know, I think with how the technology is evolving, a lot of companies are working on text to video, you know, which definitely could be interesting.

1:05:05And as that evolves as well, we can see that becoming, you know, more and more popular. I think it's already, you know, getting quite interesting and people bring some of these text to video videos into our platform to edit sometimes. And we see that a bunch. So I think use cases like that might take off a lot in the future. And then similarly, maybe text to video editing, you know, really prompt-based editing or something like that. Maybe that could be the future. I think one thing's for sure, though, that people will be able to modify videos in almost any way you can possibly imagine post-recording, right?

1:05:42And maybe they'll be able to even generate them from scratch, right, without even ever recording at all. What are, maybe outside of video, what are some of the other AI products or applications of AI that you personally find really interesting right now? I mean, I hate to be boring, but I just really, I love GPT. You know, I just, it's one of the most amazing things. I think it's only scratched the surface of its potential, you know, where it can reach. I heard recently that they were valued at like$80 billion or something like that. And I'm like, you know, that's too low. It should be like a trillion or something.

1:06:15What do you use it for? A lot of stuff, almost everything. Like these days, yeah, it's kind of funny because like my parents discovered ChatGPT, they live in India and nobody in the neighborhood knows about ChatGPT. And so all of a sudden, my dad's sending out to the neighborhood like these well-written emails, you know, with like all these like just amazing words and everybody's just like, how are you doing this? How did you get so good at writing this? And I mean, that's, you know, it's noticeable. That's how big it is, right? So I don't know, it's just, it's powerful. Yeah, it's really, really powerful.

1:06:52Where can people learn more about captions? Are you hiring? Make all your plugs. Yeah, I mean, yes, we're hiring basically across all roles. I think, you know, we're really excited about, you know, we're solving some of the most interesting problems, I think, right at the cutting edge in technology, you know, not just the AI angle and sort of the ML training, ML operationalization, but also on the, you know, graphic side and GPUs we use there, you know, for mobile. So it's all like super exciting. I think, you know, it's one of the most interesting times to be in this space, at least from what I've seen so far in the last 10 years or so.

1:07:35There's so much happening every day, every week. And yeah, it's just super fun. So we're looking for people in New York for essentially every role. So marketing, engineering, design. So yeah, and you can check out more on captures.ai. Awesome. Gaurav, this has been an incredible conversation. I learned so much. Thank you so much for your time. Thank you. It's been awesome. Thanks for listening to my conversation with Gaurav. This has been Generative Now. If you liked what you heard, please do us a big favor and rate and review the episode on Spotify and Apple Podcasts. That actually really helps us a lot.

1:08:15If you want to learn more, You can follow me at Magnano on all the socials, or you can follow Lightspeed at LightspeedVP on all the socials. Generative Now is produced by Lightspeed in partnership with Pod People. I am Michael Magnano. We will be back next week with another conversation. Thank you so much.

From the publisher

Captions.AI is taking the world of video creation by storm. With features like AI-corrected eye contact and automatic captions in 28 languages, more creators than ever can bring their ideas to life. Captions.AI Co-Founder and CEO Gaurav Misra joined Lightspeed Partner and Host Michael Mignano for a conversation spanning from his days as a design engineer at Snap to finding a surprise $500k in his Apple account. 


Episode Chapters

(00:00) An intro to Gaurav Misra, co-founder and CEO of Captions.AI

(06:03) Gaurav got an early start in startups 

(09:24) Getting a phD in machine learning before it was cool 

(12:04) From Lattice Engines to Microsoft to Localytics

(17:26) Making the leap to Snapchat’s elite engineering team

(20:57) Keeping innovation alive at a huge consumer  company

(26:48) Being on the ground floor of Snapchat’s major feature overhauls

(30:47) Leaving Snapchat to found a top-tier AI company

(33:48) It all started as a social network

(38:10) The happy accident of the original Captions.AI tool

(44:08) They raised a series A for the social network only to find a $500k surprise

(50:00) Building a text-based video editor from the ground up

(58:25) Beyond the editor - launching the AI-powered Lipdub 

(01:04:14) What will video editing look like 5 years from now?

(01:06:52) Is Captions.AI hiring?

Stay in touch:



The content here does not constitute tax, legal, business or investment advice or an offer to provide such advice, should not be construed as advocating the purchase or sale of any security or investment or a recommendation of any company, and is not an offer, or solicitation of an offer, for the purchase or sale of any security or investment product. For more details please see lsvp.com/legal.

More from Generative Now | AI Builders on Creating the Future

All 90 episodes
Gaurav Misra: Building an AI-Powered Creative StudioGenerative Now | AI Builders on Creating the Future · 1 h 9 min
Listen in VO