OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic)

27:23And actually the way Canvas and tasks came about to be was like, it started with like one person prototyping. And creating like a spec, it's kind of like PRD, is like creating a spec of like the behavior of the model. I don't think like tasks is like extremely like ground -breaking ground -breaking feature in necessarily. What makes it like really cool is because the models are so general, model can now search, they can like write sci -fi stories, they can like search for stocks, they can like summarize the news every day because the models are so general like giving something familiar to people that like, you know, notifications is like very familiar.

28:11Like having reminders is like very familiar. So like feeling like a form factor for the people who like very familiar, Canvas like Canvas, very just like a bulldoffs is very familiar. But then you add like mychical AI moment and it becomes like very powerful. By the way, it comes usually like operationally like, yeah, size is like a prototype, like literally prompted prototype or like how you would want to like the model to behave. For like tasks, for example, like you kind of like need to design, a little bit like design, design systems design that you need like, okay, like well, if this, if the user says like, remind me to go to lunch like at 8 a .m.

28:51tomorrow. Okay, what kind of information does the model needs to extract from that prompt in order to create a reminder? And so this is how you like design like a stack for a new feature, like a tool, Canvas and tasks out all tools. So it's like, how do you like create the tool stack? And then it's like mostly like like, developing JSON schema. I was like, okay, like from this prompt, maybe the model should extract like the time that the user requested. And then you're thinking about like, what, which one right do you want the time to be? And then like, how do you want the model to like notify you?

29:30Is like basically, if the user should give instruction to the model, and then this instruction would like fire off like every day or something at the particular time. So for example, if you say like search like every day, I want to like learn, no, but the latest AI news, the models should rewrite into like, okay, like search for the latest AI news. And this will, this task will get fired at that particular time that the user requested. And then you know, your design is like tool stack. And then actually, I don't know, like if you like sometimes like, it's like through conversations, I like, I don't like people ask me to like join the, you know, like team and they're like, oh my god, we need to be searchers.

30:20We need like some support. Like we need to train the models. Also, there's like, what kind of this is like mostly like, I just pitched the idea of like, it got staffed quite immediately during the break. So, you know, like a, like dependent project. And usually it was staffing is like mostly like a product manager, model designer, actual product designer, a couple of researchers don't like by children like applied engineers, the puzzle, the complexity of projects. And then like, you know, it took for tasks, it took like, like two months or so to go from a year to one basically. A lot. For canvas, this was like four or five months, I guess, to go from zero to one.

31:05But yeah, I'm like, you know, you teach product managers how to like both evolves. And like maybe, you know, how do we not only like ship the better feature, but how do we think like, well, good term like, what kind of like cool features did you want tasks to have? Like I think it would be nice for tasks to be like, a little bit more personalized. It would be nice to have like, to create tasks via voice and on a mobile, right? Like, so you kind of need to like, this is how you get like research route now right here is like, thinking like how the feature will be developed in the future. And then from there, let's like, you like start creating data sets like with eWAS, you want to make sure that goes well.

31:49And then like, you need to have like a trade off between like what methods you want to use. And the reason why I really love like synthetic, like relying on synthetic data instead of like collecting data from humans is because it's like much more scalable. It's cheap. Let's have like, you literally sample from the model. And you teach the core behaviors of the model that will generalize on to all sorts of diverse coverage. And when you launch the better feature, you learn some my family users that you can like all your synthetic sets can be can be shifted into distribution of how the users behave and on the part behavior.

32:27And this is how you improve. And this is what happens to when you are from better to J. This episode is brought to you by LUM. LUM lets you record your screen, your camera, and your voice to share video messages easily. Record a LUM and send it out with just a link to gather feedback at context or share an update. So now you can delete that novel length email that you are writing. Instead, you can record your screen and share your message faster. LUM can help you have fewer meetings and make the meetings that you do have much more productive. Meeting start with everyone on the same page and and early.

33:05Problem solved time safe. We know that everyone isn't a one take wonder when it comes to recording videos. So LUM comes with easy editing and AI features to help you record once and get back to the work that counts. Save time, align your team, stay connected, and get more done with LUM. Now part of it last year, the makers of JIRA. Try a LUM for free today at LUM .com slash LENI. That's L -O -O -M dot com slash LENI. Something that I want to help people understand, and I don't even 100 % understand this, is what's the simplest way to understand the job of a researcher versus say a model designer and other folks involved.

33:44Like what's the simplest way to understand what researchers do at OpenA? The project that I discovered mostly like product oriented, research is mostly product research. Another part component of my team is actually more like longer term exploratory projects. And it's more about developing new methods, understanding those methods, and a variety of circumstances. So basically developing methods, you kind of like need to follow very similar kind of like recipe of like building e -balls. But it's like most of us thinking it evolves. You kind of want to have like other distribution, or like if you want to like measure generalization, if I don't need to like after thoughts.

34:26But it's basically more science in a way where you know, it's the talk about synthetic data like one of the hardest things about synthetic data is like how do you make it like more diverse diversity and synthetic data is like one of the most important questions. Right now, and it's like exploring like ways to in check like diversity as a general method that will work for all is like a one of the black research operations. Other ones is like more like the bottom making new capabilities. I feel like it's always about like, you know, like you you work on this like new method. And you have like signs of life that it's working.

35:03I think you think of like how do you make it more general or you think of like how do you make it very useful or like. And this is how like longer term projects become more like media and like time project. That makes sense. Essentially working on developing ways to make the models smarter. Or oh, four or five or six ways to like a one was a big breakthrough right the way it operates where it's not just here's your answer. It actually thinks and has right takes time to think through the process of coming up with an answer. Okay. Yeah. Very helpful. Speaking of that, I was thinking about the future where things are going.

35:38I want to spend some time on just this insight that basically you are building the cutting edge of AI like at the very bleeding edge of where AI is going and where it is. And so I'm very curious to hear just your take on how you think things are going to change in the world and how people work based on where you see things are going. And I know it's a broad question, but let's say like in the next three years, how do you see the world changing how do you see people's way of working changing. It's a very humbling experience to be in both labs. I guess like to me when they first came to and I was like, I will love from the engineering.

36:16And then like the reason why I switched to like research is because I realized at that time is like, oh my god, like cloud is getting better like for and ends. Like cloud is getting better like coding. I think cloud can like develop new apps or something. And so like it can like develop new features for the thing that I'm working. So it's like, it was kind of like this meta realization where it's like, oh my god, like they were all does actually changing and they're like, when we first like launched a hundred key context at that time. Obviously, you know, I'm thinking about like from factors that's like, yeah, like file uploads were like very natural way familiar to people.

36:55But you can imagine because just like make like infinite chats in the cloud that AI up, right. Like as if like it's like a honeykey context. But because like file uploads is like foreign follows function is like the form factor of the file uploads kind of enable people to just like literally upload anything the books of like any reports financial and like ask any task to the moral. And I remember it was like, you know, enterprise customers like like financial customers are like really interested in that is like, oh wow, like actually be. It's actually one of the very common tasks that people do in that setting is like kind of crazy to like see how some of the redundant tasks are getting like automated basically by the select smart models.

37:47And they're entering them the error where I actually don't know for example, sometimes like is a one gives me the correct answer or not because I'm not an expert in that field. And it's like, I don't even know how to verify the outputs of the models is because like on my experts, not like they can like verify this. So yes, so basically there are trends that are going on the first trend is the cost of reasoning and intelligence is drastically going down. I had a blog post about this. Maybe I should update them like latest benchmarks because at that time like MMO everybody was like doing like. So like one benchmark interview like who could saturated the benchmarks like now we need to like do the same flop but was with another like frontier evolve.

38:40But the cost of intelligence is like going down because it's becomes not much cheaper smart small models are becoming even smarter than like large models and that's because of like the distillation research. This happened with a clotty high school. I was like working on like pursuing a lot of high school and I realized it was much smarter than like clot tool which was like way you know bigger let's not add. But like the power of like small models become very intelligent and fast L cheap. We are moving towards that world that has multiple implications but the news that like people who have more access to AI and that's really good.

39:25But like builders and developers will have much better access to AI. But also it means like all diverse that has been like bottleneck by intelligence will be kind of unblocked. So anyone like I'm thinking about a healthcare right like if I have instead of going to doctor I can like ask Chai GPT or give chat GPT a list of symptoms and ask me like which like would I have like a cold flu or like something else like I can literally get the access to like doctor almost and there's been some like research studies around that. Yeah, there's a New York time story about that where they compared doctors to doctors using Chai GPT to just chat GPT and just just chat GPT was the best.

40:15Yeah, like like doctors made it worse. Yeah, yeah, that's crazy. I like like education. I think I will have friends if like I had the tool like Chai GPT and when I was like young and like would learn something like that. But it's like people can now learn almost anything from this model so they can learn new language. They can learn how to build new look ups like I write anything that you want and like I'm so like it's humbling to like have like launch canvas and like bring that thing to the people enable them to do something else that they couldn't have ever before. I think this is something like magical around this experience.

40:57So education has will have massive implications like I guess like scientific research right like I think it's like the dream of like any I research is like I've made a research. It's kind of scary I'd say which makes me think that like people management will fade you know it's like one of the hardest things to it's like emotional intelligence for the model also like create a creativity in itself is like one of the hardest things. So writers I don't think like people should be worried as much I think it's like I think it's a lot of like redundant tasks for people. This is awesome. Okay. I want to fall this start for sure and it's funny that the way you described is like you're an engineer and then drop it.

41:40And you're like okay. Claude is going to be very good at engineering. This isn't going to be a potentially career long term. So I'm going to move into research and may I is going to need me for a long time to build it to make it smarter. I would say we still have like I think canvas you have still have like a really cool like front engineers that I really like you know people who like really have all like interaction design like interactions. There's like I don't think like models are there yet like I think if but we can get the models to like this top 1 % of like front end something for sure.

42:15So I want to move on to next along these lines is just and this is just speculation but what skills do you think will be most valuable going forward for product teams in particular. So folks are listening and they're like okay this is scary. What should I be building now to help me stay ahead and not be in trouble down the road. What skills do you think are going to be most more and more important to build. Yeah I think like creative thinking like you kind of want to like come up like generate a bunch of ideas and like filter through them and not just like build the best product experience listening.

42:56You know you want to like build something that like the most general model will not replace you and oftentimes you build something and you make it really really good for like specific set of users and actually the mode is now in like your user feedback. The mode is like more and like you whether you listen to them like whether you you can like rapidly it's like the mode is like in here. I don't think like we are yet to like there are so many ideas. I think that abundance of like ideas that you look like about is like I wouldn't be worried. I feel like in fact like I just think like people in AI fields are like I wish they were like a little more creative and like connecting thoughts across like different like feel to something like that to like develop really cool new like generation.

43:50A new part times of interactions with the AI like I don't think we've cracked this problem at all. A couple of years ago I was like telling some people I was like you know you kind of want to like build for the future so it's like it doesn't necessarily matter whether the model is good or not good right now. But you can build product ideas such that like by the time the models will be really good it will work really well. I think it just like happened not really like for example like I don't know like right like the cloth artifacts and I feel like early days of canvas was like back in like 2022 like before Chichibit.

44:33Like writing ID was like I know it's just but I feel like cloth 1 .3 model itself was like not there to like made like really extreme good like high quality edits for example like coding. And I feel like I see like start up as like car sir and it's like doing super well like I must because you like iterate so fast they like events like new ways of like training models they move really fast they listen to like user is like massive distribution is like yeah it's kind of. That's really helpful actually so what I'm hearing is that soft skills essentially are going to be more and more important powerful to talk about management leading people being creative and coming up with innovative insights listening.

45:20There's a post I wrote that I'll link to right look I try to analyze what AI how AI will impact product management and we're actually very lined. And my sense was the same thing that soft skills are going to become more and more important and the things that are going to be replaced is the hard skills which is interesting because usually people value the hard skills like coding design writing really well and it's interesting that AI is actually really good at that because it's taking a bunch of data synthesizing it and writing creating a thing versus all these fuzzy things around of what influences convinces people to do things in a lining and listening like you said creativity.

45:58Anything along those lines come up as I say that. I think it's actually really really hard to just the model how to be aesthetic or like do like visual really good like visual design or like how to be extremely creative in the way they're right. I think like I still think like chagype kind of sucks of like writing and that's because it's like it's like bottlenecks by this like creative reasoning. I think like prioritization is like one of the most important like I think like for a manager I feel like I actually like AI research progress is bottlenecks by like management like research management is because you have like.

46:35The question set of compute and you need to like allocate the compute to the research path that you feel the most. Commits about was like you need to like really you need to have like a really high conviction and the research that to put the compute and like it's more like return on investment kind of situation is like okay yeah like I'm thinking a lot about like like like okay like how do I across all my projects which projects a higher priority is like prioritization and also like on the lower level is like which experiments are really important to run right now and which are not in like cut through the line so I see like prioritization.

47:12Priorization communication like management people's kids like empathy like understanding people like I know like collaboration like I think like canvas wouldn't be like an amazing launch if these wasn't like about like people and I think it's the wonderful group of people and like. I got a chance to like rub with like people like Lee Byron who's like a cook for you know like GraphQL and like some of the best like Apple designers and it's like so cool to like see and like how do you create this like collaboration between people is just like something that's still human I think. Let me just follow us are a little bit because I imagine people listening are like okay but once we have a GI or SGI it'll do all this it's you know it's like there's a world where like why isn't all this done.

48:01I think it's easy to just assume all that I'm curious this idea of creativity and listening why you think AI isn't good at it other than it's just very hard to train it to do this well is there anything there just like why this is especially difficult for AI now lambs to get good at. I think currently it's difficult. For many reason I think it's still like an act of like research area and there's something that like I think my team is like working on is like okay I like how do you teach the model to be like more creative and like the writing and actually like. I think he like doesn't your paradigm of life the models.

48:40I think more should actually lead to like better writing and itself but like when it comes down to like idea generation or like. Discommunicating of like what is the good like visual design and not I feel like it hasn't had learned like examples from like people to discommunicated very well I do think it's because like. You know there are not many people who are like actually like really like it's not like accessible to like model so learn from these people like us. So I guess like that's why it's often yeah make sense basically there's not enough of you yet researchers. Teaching it to do these things slash people that have incredible taste and creativity that can teach these things you could argue this will come but I'm not we don't need to keep going down that thread.

49:35Let me ask you a specific question in this poster road I made this argument that a lot of people disagreed with that strategy is something that AI tooling will become increasingly great at and take over there's the sense that that's a thing that people will continue to be much better at and you. Can't offer to AI basically developing your strategy telling you what to do to win my cases isn't strategy just take all the inputs all the data available. Understand the world around you and come up with a plan to win feels like I would be like an L I would be incredibly smart at this what's your take I think so to I think like again like we teach the model all sorts of like tools and like capabilities and like reasoning right and it's like when it comes down to like.

50:19I was just a for canvas right now you're very cool to like for the models just like aggregate all the feedback from users like some arrived me like the top five like most painful flow flows like music experiences and then like the model itself is like very capable of like like thinking of like knowing how it's being made. Figure out like how to like treat a data sets for itself to like train on it and I think like me a far away from that kind of like soft improvements models becoming like self improved by a like then like the part of development is basically kind of like self improving like it's kind of like it's all like organisms and something yeah like I gave like strategies like it's more like data analysis and like.

51:09Coming up was like. Like I think what matters are really good is like. Like connecting the dots I think it's like okay if you have users you back from this source but you also have an internal like dashboard with metrics and then you have you know like other kind of like you back. Or like inputs and then like it can co create like a plan for you like look at the nation's event and it is like one of the most common use cases for actually people is like coming up was like this sort of things that makes sense like essentially. A human can only comprehend so much information at once and look at so much data once to synthesize takeaways and as you said these context windows are huge now here's all the information what's the most important thing I should do.

51:59Yeah same was like scientific research is because like you like ideally the model would be both like suggest like ideas with new ideas like it already on the experimental like given the empirical results of the previous experiments like how do you. Like come up with like new ideas or like the methods yeah man. Okay so just to close the loop on this conversation this part of the threat is the skills you're suggesting people focus on building and leaning into soft skills like creativity managing influence collaboration looking for patterns. Is that generally where your mind is that yeah I'm meeting a lot of all like how do you make a relationship more effectively and I think this is more most like management I guess.

52:45It's like how do you organize like research teams like generally teams like combine. Compose teams such that they will be at the maximally succeed like at the maximum like performance of what can possibly. Like you can like literally create like the next generation of computers is just like the matter of conviction and like the way you manage to that is like scaling organizations or like scaling product research as. Yeah I think what it like you're basically building this thing and not efficiently doing it is like limiting the potential of the human species right now is. Right mismanagement within the research team and open and I and then I can some of these other models.

53:32Yeah I've kind of crazy to think about holy moly okay so speaking of a drop making opening out you've worked at both very few people have worked at both companies and seen how they operate. Care is just what you've noticed about the differences between these two how they operate how they think how they approach stuff what can you share along those lines. It's more similar than different obviously there is a lot of like. There are some differences also comes to like nuances or to culture. I really love on top of it and they have a lot of friends there and also love opening out and I still love a lot of friends.

54:05So it's like it's not about like enemy I feel like there's like in the animals all like yeah the competitors that's like enemies that's actually like one big community and like a people like doing the same thing. I would say would have learned from on top of it is this like real care and craft towards like model behavior model cost model training. And I've been thinking a lot about like what makes cloud cloud and what makes strategy and strategy and it's like I just sounds like operational processes that kind of leads to the outputs to the model is the output and model and it's like the reason why collage has so much more personality and like.

54:52It's more like a librarian I don't know what I don't I am like visualizing a cloud being like a librarian like a very like naughty or something. It's because I feel like it's the reflection of the creators who like making this model and like a lot of like details around like the character and the personality and like whether the model should follow up on this question or like not like. Was to correct like ethical behavior for the model to like in the scenarios like a lot of like craft and like truly did like the results and this is where I learned that part of like art I guess I don't know that I was in the darkness like much smaller like when I joined it was like what like 70 people when I last did those times people like obviously the culture changed so much.

55:44I really enjoyed being like early days sort of like lives and like people. Neuriches as a family but like the culture shifted I would say like under I learned from a topic that like they much better at like focusing and like prejudices and like very very hard like very high school practice. I just like to do it like that I think like opening eyes like much more in a way to and much more like risk take hardest in terms of like products or like research actually you know like. I don't really can like your full time job can be just like teaching the model how to be like creator writers and it's like this some luxury in this like research freedom that that comes to scale maybe I don't know.

56:30But it gives you it's like you'll help I feel like I have much more creative like product freedom to do almost anyone I guess within like opening I like of all especially to be into like the region that is like more like yeah probably what is off it is yeah that's how I was thinking about is feels like opening eyes more bottoms up distributed people bubble up ideas try stuff there's more law and that emerged leads to more products launching imagine more things just kind of being tried versus more of a let's just make sure everything we do is awesome great and craft and thinking deeply about every right investment that's really interesting I've never heard it described this way.

57:11Karina we've covered so much ground this is going to help a lot of people with so many ways of thinking about where the future is going before we get to a very exciting lighting around I'm curious if there's anything else they think might be helpful to share get into one of my regrets I guess when I was early days at on top of was that like I think there was like some luxury of the time this pretty China to be key to actually like. Comment was like a bunch of ideas on like prototype was almost every day and I think like we did a lot of cool ideas like cloud and slack was actually one of the first like tool use like products like a cloud could operate in like your workplace now it's like kind of cool but you like at cloud to summarize the threat so maybe you have a good entire conversation with someone and then you want to like a summary.

58:04What happened like you can see like at the class and crisis also was really fun to like even like to read on the model itself is like when you just like talk to the model in like slug forever and if you're just on the social element it's kind of what was kind of a journey and like. This discord like people learned so much about the content and like how do you work with like clouds actually one of the features that was like early tasks part of time is like you know every Monday clock which is like summarized the entire channel or like every Friday we just like summarized like bunch of channels and give like the news about the organization or something so it's and it's going to be a lot of fun to do.

58:48It's kind of like really cool like form factor I think I think about like phone factors like a really important question like in AI especially we haven't even figured out like how to be create like an awesome like products experience was like O series models is like the paradigm between like synchronous real time given answer paradigm into like more synchronous paradigm of like agents working on the background but then now the questions like the agents should be. So you know you're both trust that you're right and trust both over time which is like this humans and you know you start this collaboration which is why like a collaborative like this collaboration model was like you and the models like some important because you both trust and the model are certain preferences so that it can become more personalized and it will start predicting the next like action that you want to take on the computer or something and it's like kind of like more predictive much more.

59:48And from like personal computers like personal model basically here you know that's why is it not a thing that seems like such an obvious feature that every element should have is a slack bought version of them is that is that a thing I can have you installers that not a thing right now. I know that cloud and thought was unsaturated in like 20 or 20 years on thing but I think I think I think it was like after Chitra Pt it was mostly like the focus on like consumer just cases like enterprise cases. I think the one like I think the form fucked up like cloud and slack is like was kind of constrained.

1:00:24A little bit when you want to develop new features. I want that. I know that GIGP have like slack part to I or not like maybe it will come back. All right I would I would pay for that. Any other memories from that time of early days because that's a really special place to have been as early days and the topic any other memories or stories from that time that might be interesting to share. I think the very first launch when they fall like when clips and use again was like a hundred key context launch is like when the models could input the entire book and give you like summer you know the book or something.

1:01:03Of the entire financial or like have like multi files financial reports and then like give you an answer to the question to very specific question. I think there was something in there that kind of like oh my god this is like a really cool new capability not like model capability but more like the capability that came from the product form factor itself rather than like the model capability as much. I think like other project types that you were thinking about like you know like there's like one part of the clockwork spaces and it's like kind of the same like idea like. Claude and I would have to share workspace and that share work space like a document and he like to read in the book and I feel like sometimes the ideas like primary is log and they'll lock for like two years.

1:01:58Just like in this case it's interesting there's a milestones that kind of open up our view of what is happening and where things are going. Chatchy PT I think was the first of just like wow this is much better than I would have thought you talked about one hundred K context windows or get up a little book and ask questions have it summer is actually use that all time when I have interview guests and they wrote a book. I sometimes don't have time to read the whole book so I use it to help me understand what the most interesting parts are and then I actually dive into the book just to be clear. And then I don't know maybe like voice was another one where you could talk to say chatchy PT there are any other moments there that you're like wow this is much better than I thought it was going to be.

1:02:39Yeah I think like computer use agents like the model operating the desktop and you can essentially think of like you know new kind of like experience where the model can learn the way you browse and from that preference it can just like browse as just like you. It's kind of like simulation simulated that person up and it's actually very similar to the idea of like okay like maybe some album doesn't have a lot of like time maybe I want to like talk to like his team leader's like his simulation and ask like or like for example like yeah like I really appreciate some of the technical mentorship like yeah cool like but he doesn't have a lot of time so it's like I really want to like ask him this question is like kind of respond like simulated environments like those will be really cool.

1:03:37It's a great place to plug Lenny but I have one of those it's trained on all of my podcast the newsletters and I it's on many models that I know which one exactly they use but it's exactly that it's and it's not even me it's all the guests that have been on the podcast. I'm just like I wrote and you could just ask it how do I grow my product how do I develop a strategy and it's actually shocking we could do feel like it reflects. Yeah, it's the best part of it is you can talk to it it's built there's an 11 labs voice version that's trained on my voice on the phone spot gas and it's actually very good and people like I've told me they sit there for hours talking to it.

1:04:15Wow and somebody told it interview me like I am on Lenny's podcast ask me questions about my career and he did a half hour podcast episode with Lenny but that's so fun. It's incredible future is wild. Yeah I think content transformation is like you know like I would imagine sometime like you know but you generally decipher a story in canvas like you can like transform this into like audio book. I would go like where I have like very natural like content transformation like one media to another medium. I think like one of my early inspiration is like one of the last episodes of like Westworld where I want to explore but where Dolores comes to her work at the time and she comes to like this like new workspace and she starts like writing a story and then she like virtual reality starts like creating on the fly so I don't want to hate that.

1:05:21Yeah, critical. Wow speaking of medium I guess I was wondering if I should go in the structure but real quick Kevin Wile slash Kevin wheel I don't know exactly that her pronounce was last name the CPU of I think real wheel okay okay let's just say that okay we know he was he did a panel at the Lenny in front summit last year and he made this really fascinating point that Chad is a really interesting interface for these tools because they're just getting smarter and smarter smart and smart smarter and Chad continues to work as a paradigm to just interact with them similar to a human you could talk to Albert Einstein you could talk to someone not very smart and it's all conversation still and so it's a really flexible way to interact with increasingly good intelligence at some point it will not be so great and you're talking about all these ways that you're adding additional ways to interact but it's interesting chat proof to be a really powerful layer on top of all stuff.

1:06:21Yeah, that's really cool. I feel like Chad also has the social element which is like very humane it's like you sometimes want to like get into a group chat and like having conversations with the eyes kind of like a group chat and it's out for some massive thing. Actually this idea of like how do you both like features like this like I see tasks as like this like general kind of like feature that will scale very nicely as the models would develop like new capabilities as well as like like the model will be able to like do better like searches and like you know create new like come up with like more creative like writing on like render you know react apps and like HTML like apps and like you can have like every day and you have a new section that we ended up going down is this idea of your agents using a computer.

1:07:22I know this is actually something you are going to launch today the day recording it which will be out by the time this comes out call operator you talk about this very cool feature that people will have access to. Yeah, so I unfortunately did not work on that but I'm really really excited about like this launch. It's basically an agent that can complete the task in its own like virtual computer like in its own virtual environment. You can do any literally task like ordering me a book on Amazon and then ideally the model will either like follow up with you like which book do you want or like know you so well that it will like start recommending like oh here's the five books that I might recommend to you to buy and then like you hit like yeah help me help me buy and then the model goes off into its own like virtual little browser and like complete the task and by the book and the Amazon and then if you give the model like credentials kind of cards obviously it comes with like a lot of trust and like safety and then it will just complete the thing for you.

1:08:33It's a virtual assistant. It's interesting and this just sounds like obviously this should happen like why is this not a yet a thing which is also mind blowing that we're just assuming this should exist like just some AI doing things for you on a computer. It just has to do like it's absurd. It's actually really hard and I think like you're just still cracking this but I feel like I don't know if he was like tuple it's like a band programming.

1:09:04But I don't know if you love their programming. So if you shop or fight is this a member came up on a podcast episode. Yeah so it's a very cool product where you can just like call anyone at any time and then like share screen and the other person can like have access to this stream and like start like literally operating your computer and it's very like real time like the lead is just like very high quality and it's just like I kind of want the same is like I want to like pair program with like my model and like the model should you want to talk to me like draw like very specific like section in my code and yes go tell me like obviously to me and you can have like different modes.

1:09:48It's like right here it was like a product right here for you. I don't know people sent you most of the old dad. It sounds like a startup just got birthed. Yes from someone listening to this. You mentioned that it's very hard to do this agent controlling a computer as you and helping out what makes it so hard for whatever however much you can explain briefly. My two bit is like because right now the models operating on like pixels instead of like language or whatnot like pixels is actually really really hard for the models because like perception or visual perception. I think there's still a lot of like multiple like research that's going on.

1:10:31But I think like language scaled so much like easier compared to like multi models because of that. Another was like thing that I just like my team is working on is like how do you derive human intent very correctly. It's like sometimes like doesn't model know enough information to ask about question or like to complete the task. You kind of don't want like an agent to like go off for like 10 minutes and then come up with like an answer that you didn't even want that actually creates like much more versus experience. And this is comes with like teaching the model like like people skills is like you know like what do people like like kind of like creating like the my style model of the user and like care about the user in order to ask certain questions like actually that part is like hard to the models.

1:11:27And I'm related to what we talked about earlier this kind of the soft skill people skills pieces not were these models are strong yet. Okay. I'm going to skip the lighting ground. I want to ask just one question from the lighting round something fun. Okay, so when AI replaces your job, Karina, I'm curious what you're going to give you a stipend gives you a monthly stipend. Here's your salary for the month. What would you want to do? What do you want to spend your time on? What will you be doing? In a future world. I think thinking about this. Oh, sometimes I have I really have a lot of jobs options.

1:12:04I would love to be a writer I think I think that would be super cool. You should like write like short stories like sci fi stories. No, I really like art history. So you know, it's like conservationist to like in the museums, which is like try to preserve like art paintings, but just like painting through a lot of things. I think that would be really cool to do. Yeah, that sounds beautiful. I don't know. What I'm hearing is you need to nerf these models to not get very good at writing so that you can continue. Although at that point, you already need to do it from like you don't need people to buy you're just doing it for fun.

1:12:48So it doesn't even matter if they're incredibly good at writing or art or conservation. Oh, man, what an episode or conversation. What a wild time we're living in. Karina, thank you so much for being here. Two final questions. We're confused by new online if they want to reach out and follow up on anything. And how can listeners be useful to you? You can sign me. I'm a Twitter. I'm in. You can also shoot me a email on my website. And my team is hiring. And so like I'm looking for research engineers, research scientists, as well as like machine learning engineers, like people who come from a party.

1:13:25And engineers want to learn about model training, I'm actually hiring for like my team, my team is called the frontier part of research. And the train models be developed new methods, but for part of the oriented outcomes. What a place to work, holy moly. What's the best way for people to apply for these very lucrative roles? I think you can shoot me a DM on Twitter or. I'm yet to create a job description. Okay, this is the job description. Are you going to fly into like post training? Okay, this you're going to get a flood of DMs. I hope you're prepared. Karina, thank you so much for being here.

1:14:02This was incredible. Thank you so much, Lenny. Bye everyone. It was fun. Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or a leaving review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at Lenny's Podcast dot com. See you in the next episode.