20VC: Why the AI Bubble Will Be Bigger Than The Dot Com Bubble, Why AI Will Have a Bigger Impact Than COVID, Why No Models Used Today Will Be Used in a Year, Why All Models are Biased and How AI Kills Traditional Media with Emad Mostaque, Founder & CEO @

21:16If you have a business that's focused on innovation at the core, that's not actually a business, it becomes a business when that innovation becomes product, it becomes distribution, when it has an advantage on data and other things, those are real modes. How did you analyze that partnership between Open AI and Microsoft? I saw it as the objective function of Open AI is to build AGI, and they reckon they need 10 billion dollars to do it, and they did that. Like they're building a business on products and things, but they don't care. They're not trying to build a state -of -the -health business.

21:41They're trying to build an AGI. Why? Why would I just tell them it was on an AGI to build a sustainable business? Because at the end of the day, they're building an AGI to turn the world into utopia. It's written in their path to AGI thing that they think this can basically bring about utopia. So a lot of people in these labs, we only have people joining from all of these labs, like they almost zealous in their... there is a misalignment there between them and Microsoft and their desire to create that utopian. Yes, because Microsoft Microsoft is a business. And so this is why you've seen articles in the information like Microsoft say OpenAI on compliant and OpenAI say Microsoft on this.

22:14These things happen when there is a misalignment of objective functions. But again, you should view OpenAI as what they want to do is build an AI that can basically make the world better and hopefully not kill us all, which they say it might. Which is a bit concerning, which is why I hope they have better open governance. How did you think about distribution? You've seen a hugging phase spawn with Amazon. You've seen OpenAI with Microsoft. when you think about distribution and your competitive edge there. Where did you land? So my business model is actually very simple. I haven't really talked about it much.

22:42Stimulate Open, one of the biggest providers of grants to open source software, tens of millions already. And then take the best of Open, which hopefully we build ourselves, and then an Open -based with an Open Data, and then commercial variants with license data and the national variants. So you have hidden the Insurance Adjusted Stable Chat, or Indonesian Pharmaceutical Work Estable Chat, that's available in every cloud, on -prem, on -device, with licensing fears, royalties, and revenue share. And the system integrators work with us as well, lots of announcement to come. And so by standardizing and stabilizing all the complexity to these very sophisticated building blocks, these very intentionally built models, that really helps the world integrate this stuff by building playbooks and other things.

23:23And that's the core business, because it doesn't require actual innovation. We are still innovative in the leaders and media in particular. Instead, it requires data distribution. Data to the models. The models are open and interpretable and models to the data via our partners. And that's valuable because the private data in the world is far more valuable than the data that you will send to proprietary models. And it's not a race to the bottom either. That's what we are. We're a modeling agency with hot GPUs. Building a distribution around the world, realizing that India and other nations will leap frog to intelligence augmentation just so they leap frog to mobile.

23:52They will embrace this technology far quicker than we will in the UK. It meant why? Because they have to. India, all of the outsourcing jobs in programming will go because GPT -4 can go level 3 Google program or exam and pass it. Our source jobs will go the first, whereas in France you're never going to fire a French person, so there's tons of faith. And so they have an objective function when they need to embrace this technology. In Africa, one to one tuition, every kid in Malawi is on things in London, we've got other nations, we're going to bring them all this technology and tablets. And guess what, their lives will transform.

24:23One AI per child is one I want to call it. But think about the potential of that, because you have ones teacher per 300 kids, whatever they had, a chat GPT level AI. The ROI is high and the need is high, and so they will embrace it far quicker than we will. What happens to countries that rely on outsourced work in those going to free -lanter economy jobs? The question is, you've seen OpenAI study, you've seen the, which said, task school you replaced up to 44%. You've seen Goldman Sachs say ads percentage points to GDP. I think the only solution to this is entrepreneurship. And so we need to give the tools to create new jobs that can replace some of these old jobs being done.

24:55So like to the various Asian governments I'm saying adopt the UK policy of the sandboxes, financial AI and other regulatory sandboxes. So you can take these technologies, these national models that we will help you build with our consortium partners and then spur innovation to create the jobs to replace the existing jobs because you'll upgrade your entire society. Bring these models into your governments and other things to go from slow -dum -ey eyes, which is the national organisation's healthcare, to intelligent dynamic ones. On implementation, and when we think about bluntly seeing this in action and society, I'm always very aware of technology cycles taking so much longer than one anticipate.

25:30How do you think about that, and that's true for one is adoption on enterprise and another's adoption on consumer. Say if we do the adoption on the consumer side, which is impacting free -nance jobs and impacting education, what do you think that is like? So I think on the consumer side, you're free with your information, so you can use a lot of these things in the APIs of OpenAI and co -here and others are fantastic, right? And Google Palm now being out there. So it will be integrated to deliver better consumer experiences without it being creepy like you've seen with some of the chatfots, et cetera.

25:58Because it's going into word is going into workspace, you know, like it helps already. Like we will have a conversation will be automatically logged by our pixel fur and then we'll get a transcript and remove bits that we don't want to share that it goes into global knowledge base that reminds us of things. That's inevitable on enterprise. It takes longer because you need to have auditable standardized models. If you're financial services too, you can't have a single piece of crawl data in there. So that's what we're deliberately building with the largest companies in the world because we're building Dedicated.

26:24You've got to have a single piece of crawl data if you're a financial service. Yes, because the danger is if it has some reddit in there. So stable allowable put out next week. It wasn't as good as the other models because we're going to make a point about reddit data being bad. It's not about more data, it's about better data. We had data comp, which is the next generation lion that we funded the compute for. We're by the quarter of the parameters of opening eyes clip, but outperforms with a better data quality. That won't make it good data quality. That's something we're exploring right now.

26:48But from the investment banks we've talked to asset managers we're building dedicated teams for the largest ones in the world to build them reproduction models. The feedback we've got is we cannot use a black box. We need to know what data is in there. The regulators are asking us. We don't want to have this out of sample thing where it's seen something on Reddit and then it says something rude to our end users. Why is Reddit data bad? Reddit data isn't bad in itself. It was just a case of more data isn't always good. So right now we are using all these websites And we're trading our models by taping their eyes open and then it took six months to turn GPT4 into Chat GPT4 because we had to tune it and give it a haircut and stuff and bring it back to society.

27:25The point is that we need to find the right type of data because rubbish in rubbish out is something that we've heard a lot. It's not bad at itself, but if you just scrape it without the proper cleaning it is bad because what is it? It's people just covetching a lot. People being biased. Do you really want to feed your kid the whole of Reddit? do you want to have the best curriculum possible? And the social one, the ways the models are like stable diffusion, we train it on the whole internet and then better and better image subsets of it. And that's the same thing when Lajor was called curriculum learning.

27:51Trade it on a big base that's solid and then it sounds familiar, doesn't it? The end of the guard, the modesty, is there? How do you instill values in political crackness in models? There is no such thing as an unbiased model. Dali too, when OpenAI had that, they introduced a bias filter. Any non -gendered word that had a random gender and a random ethnicity. So you type in Sumer Eslan, you get Indian female Sumer Eslan. That was a good feature. I gotta save somewhere. This is where you need national data sets, you need cultural data sets, you need personal data sets. They can interact with these base models and customize to you and your stories because you and I both have our stories that make up our psyche.

28:27Sure, and understand that context is so important to have a eyes that can work for us, not on us. And so it's essentially like a natural generation cookie that personalizes our data to allow for better search. It's mega cookie. And if you standardise the base foundation models and they call it the hypercube every modality because we do all the modalities, all the sectors and all the nationalities, then you don't need to have a million different models like those dream booths of the avatars. You said you have a base model that you then have a vector embedding around because these models contain all the principles and the embeddings point to the important bits that make up Harry or Emma.

28:56And then you can search those and adapt those rather than having a million, billion different models, which are just confusing. So I had dinner the other day with one of the largest kind of media publication owners in the world and he said that I'm wide -hearted and I don't think that I will have a business in a couple of years. I think, Bunny, we're getting killed on our advertising because everything's getting scraped and then it will come into our websites and that's where we get paid. We get paid for clicks. Is he right to be worried? I think he is right to be worried again. You look at Google's announcements yesterday, the two of us are still off to Palm 2.

29:25You suddenly look at the new Google page where they've got the language model and it's just text and where they click. It was like when Google introduced AMP but this is where rather than look at the New York Times page, you have this format to think with no New York's times kind of stuff there. These search entities that aggregate a certain intermediary more and more and people are going to become used to just having synthesized input. So what does search look like? What does it look like when your GPT -4 can write you an article about any news that's happening in a way that's customized to you and your context and everything like that?

29:54This is massively disruptive for media and information. And so they have to think, where am I in a future? Again, the way I swear to think about the impact of this is the re -tanted grabs that occasionally off them heads and we push about them in a thousand of them. Those grouts include journalists and you can have your own journalist army, your own writer army, your own coder army, your own designer army. So in the pushback against that is libel. Libel is real, you are going to get unbelievable amounts of libel cases and then open AI will be fucked. You cannot have a thousand libel cases a day.

30:21This is the thing, if you say this needs to be checked and cross checked that's one thing but a lot of the media come you say with the source of authority. So a way that media companies can shift is by having in a deep fake, can other age where everything can be generated, we make sure this is real news. We are very thorough in the way we do it. So this is the interesting thing that you place a premium on authority. Preval authority, this is why you've got the check marks coming out of Twitter and the organisational thousand pounds a month and Facebook doing the same because you need to have a level of authenticity and level of authority.

30:51But again, is the news fair and balanced? I've had lots of hit pieces coming out against me and got a lot more. It's not because they have angles And so what is the bias of the New York Times versus this versus that versus others? How do people consume news now and even news consumption has gone down dramatically, right? Because people consume news through their social networks, through their groups and other things. So you have to say, what is the model? But the hard part is none of the net generation models or AI providers want to be content publishers. So how do you fit in a world where you're killing a business model on the content side, but they don't want to be publishers, you will have AI first publishers.

31:24So if you remember with Vox and these things when they kicked off, they want to be generous. They want to be a technology first. You're going to have a new way of AI first publishes that aren't just AI but as AI plus humans. This is how it looks like for AI plus humans. AI plus humans means that you have information coming in and then the stories are drafts that automatically written review to buy humans who then give their input to train it better. This is a feedback flow. And then what happens is it comes out and there's a factual anchors and then it gets customized to Alabama and that Alabama context and all sorts of other things because you can tell it TLDR to you ladies didn't read or explain it like I'm five make it more complex and so you're going to see something very interesting here which is the right news at the right time the localization will return but again through AI first I think it's the thing we're seeing AI integrated but the next wave is going to be once we understand design patterns AI first everything and information flows once these technologies are a bit more mature can you tell me on sound AI integrative versus AI AI first.

32:20AI entry means that I have a existing newsroom when I bring in AI to write faster drafts and things like that. AI first is saying I have an army of things I can spin up instantly that can help me achieve these certain things to create news that is valuable for this reason with this feedback loop. And so you build the system from the start thinking AI at the core versus AI being integrated into improve existing systems. Because so much of news is what? We find information, we have drafting, we have this, we have that,

32:48we like we move from the analog to the digital age to the internet age, the next age is the AI age. So I have to ask, when we need AI first publishers and for that generation of media, who does this? Is this startups? Because I've met, honestly, 50, maybe more AI companies in the last month. The feedback is always zen, and not operating off a defensible mode of data. They're in this year thin application layer on top of an existing model, 99 % of the feedback. So you look at someone like Harvey for example, they went to law firm they said you are a distribution and we're gonna Integrate and improve your system and bill our system for your system So I think a lot of these people are trying to build it and they will come and they're trying to get in there as opposed to just Retargeting where can you go in and transform because that the wrong model, Harvey did now I think it's the right model I think that a lot of organizations are elastic and plastic now so you can go in and give them an integrated thing Say you will be my test case.

33:39I will help you upgrade as a skunk works lab and build a system alongside your system as it were And sorry, anything enterprises will say, sure, I think now they will, if you can keep their data inside internally. And I think again, with better open models, you can enable that. So if you can build on top of open models, there are dedicated instances on Co here and others as well. And so the tooling is now catching up so that you can have a new generation of startups, where their first customers are massive companies, they would never get otherwise. Every big company is looking for an answer. If you can give that answer, that contract that would have taken you a year, you can get in a week.

34:10Do you think something, because you still want to get in the door, you got to get in the And that's hustle that. So again, this is where the Harvey guys get. This is where I went straight to the hyperscalers and I said, you need to have standardized models for open for regulated data. What did they say to you? They said, really? Like, can you build them? Here's some models that we built. Oh. And then I told them exactly how the things would be last summer to now. And it's followed that and I've kept in touch and I've improved it. And this is why I'm building dedicated teams for the largest companies in the world.

34:34I'm not telling them I'm trying to sell you anything. I'm like, over the next year, I'm going to help make sure you do not get blindsided. Like I try and sell you models and people offering us tens of millions per model. I'm like, I'm going to build a proper partnership with you. And that means I'll have an LTV from you. What does that proper partnership mean? And who's that with? That's with IBM, that's with SAP, that's with us. So we've announced Amazon. Let's say we have lots of other announced with the biggest companies in the world, where they have amazing teams, but they can only move so fast.

35:00And I'm building the dedicated teams that help them move and understand the whole sector, without trying to like sell them on services. I'm trying to say, I will build you a customized model if you want, but I'm only doing that with a dozen companies. So I can focus down. And I will tell you that GPT -4 is great or co -herens great or all this stuff. All the latest research to the communities we support, I will make sure you're on top of, rather than to your site, and you've got dedicated people helping you in this transition period. Is that a line to your core model? This seems like an ancillary product.

35:26It is like a SAP consulting service. It is like that because I need to understand these sectors better. What does the HyperCube look like? What does the insurance suggest a GPT look like? That's a fundamental basis. And so a lot of people are like able to extract that data and then take it with you and do the line you'll be able to see. Yes, and so this is part of the thing that we will have a generalized model and we're very clear, but then you can have a specified model just for you as well as long as there's an interfere with that. The reality is no models that are out today will be used in here.

35:52Unpacked out of all we have, and this is mind blown. So again, you see the order of imagined improvement. Palm last year was 540 billion parameters, then Chinchilla 67 and now 14. 540 to 14 is a big step. You see the quality of G53 versus G54. Is there any extent to how low it can go? We have no idea. You already said this is impossible. Two years ago, like, no way. You have a single file that's maybe a few hundred gigabytes that can pass every exam apart from English lit, fucking English lit, fucking British lit, no way, no way. So we're ready at the impossible and what does that mean, though?

36:21If we go low and low and then what? And then what? When it jumps as you saw with the llama stuff and all the innovation around that to your Macbook offline, the marginal cost of creation and the coordination becomes zero. I don't know what it means. Nobody does. And this is the thing, It always takes longer and shorter to implement grand barrier technology than you've ever seen, and this technology can be implemented like nothing we've seen before. And this is my call, not concern and we, I hate the doomsday, says, and I'm excited for the future, I'm terrified for the future, but everyone always says, of technology, revolution, industrial age, whether it's the instruction of PCs, and industrial, it was a 30 -year -plus.

36:56Actually, PCs, and industrializers, 10 -years -plus. The jamaging thing is like, the learning curve to use chat GPT is a marketer. It's nothing. And the integration is the day. It's because, yeah, like you want to write an API, it's not a day, you just give it the manifest spec and it automatically generates, it would have taken days before. It's an amazing experience, because the transition is so much more compressed today. It came from the existing system, as it goes seamlessly into the existing system, but it's like Web3 that tried to create a system outside the existing system, and all the money was made and lost at the interfaces.

37:26Again, it's like deploying grads at scale. Like with a 32 ,000 token context 100GPD4, 20 ,000 words of instructions. or is that due to SaaS? So my thing is that we're still in this crazy period next year it will settle and then it'll go ubiquitous. Well a lot of companies know they need to do something but they don't know what they need to do. Are they adopting it now? They're doing the POC thing like some like Microsoft and others for consumer, they're adopting it. Consumer adoption is a much lower bar. When this starts going in enterprise, it's gonna be a fricking train because so much of enterprise is about services and information flow.

37:56And again, if you push a button and have a thousand of these things, that's a huge difference. I think this will be an even bigger you can not begin packing in COVID. I don't know in which direction. I hope that you're positive, but again, giving that example of an India or one of these outsource places, you lose BPO jobs, you can make it up an entrepreneurship, you embrace the technology. What do you think the business model of the future is for those models moving to enterprise? I think it's the same as always. You've got good products, good distribution. You know, you lock it in like 1 .5 million people still use AOL.

38:2340 % of the world still doesn't have internet. Yeah, and we're super privileged where we are, right? And so you look at that, and I look at emerging markets, I'm like, all of finance is securitization and leverage, and securitization is telling a story. The only thing that matters for a stock is the marginal story and how it evolves. What if you have massive information about every child in Africa and every business in India and they embrace this technology properly? The massive financial growth. Why do you think next year for they're embracing it? I think that people still getting used to all this.

38:51We haven't standardized anything, as we don't know what the design patterns are. I think that what happens is that everyone's doing this at the same time and they're all trying to get to grips with it. And so again we have this like six month window where everyone's getting to grips with it and then we standardize how design patterns and they spread and you start implementing and you see some people are pacing others which means that you have to catch up and then you're forced to implement it. So this is how I see the race dynamics occurring right now. You say about forced implemented. I think the truth is they just have no freaking idea.

39:16Right now they don't which I totally un -design and blame them for but I tweeted actually the other day that I think the biggest AI companies will be services based implementation companies for large enterprise. 100 % that's why I said if you're a startup the best thing to do is you identify an enterprise that will be transformed by this And you go to them and you say I have a solution and I'm gonna start with you and I might go bigger I'm gonna help you through this period by doing this in this and they will appreciate that and they'll be capital available for that In a way that you've never seen before because you know how difficult it is for small companies sell to big for the big companies Have no idea except for their CEO and their board are telling them you look at another of mentions and it's called it's like that.

39:53Every earnings call next quarter and then by next year literally every single one they're like what is our strategy? It's not like what is our web 3 and metaverse strategy? It's I need this strategy now. Again, it's what is our COVID strategy. It'll be that level of urgency within a few quarters. Would you raise money if you were then? So you go to a corporate and you say hey, you know what, I can solve your problem. This is how it will work and they will fund you. They'll give you the data. Would you raise money? Yeah, I mean like again, you need the people. The people is the key thing here because you can have the technicals, crops, you know, understanding of the industry.

40:22But to build a good business and scale it, are the pace that you need to to keep up with this is incredibly hard. Do we have enough talent? No. And so this is why we support the Fast .ai courses which transform normal developers into ML developers and other things like that. But again, these models are actually not that hard to work with. 50 % of all code on GitHub is AI generated now. So you can even use code file to help you code the models and other things like that. What do you think that code generation is in five years? Why would you need code? Code is just a way to talk to a computer. I'm like that.

40:51When I started 20, 120 years ago as a code, I'm 40 now. So just doing that one is 18. I was writing it in somebody code. Like really? For low -level stuff. There were no libraries, there was no GitHub, there was nothing like this. Right now, coding is like mixing and matching. It's like building Lego. And AI can build that Lego much better, especially in five years. What you're doing when you're pro -pig programming language is you're telling it to go and do something. Even something like Palm, like we sponsor a amazing code called Lucid Rains. If you want to cry as a programmer you go and look at his GitHub, most productive developer in the world.

41:22He recreated the whole of Palm in 206 lines of PyTorch. But again, why would you need a human for that? If the AI gets better and better at coding, just tell it what you want. I want to create an app for 20 -minute VC that has these features. Of course it will go and build it automatically. Where is the human code in that? What does that mean for the future? I'm sure it's a good thing. you're the terms of the complete democratization, where anyone can build anything. Anyone can build anything. This is why distribution, data, relationships, product become important. Because it really became easier to build anything, right?

41:52But what makes a good product? Again, there are these unchanging things. Have great customer satisfaction. Deliver value. People get distracted by technology. Like I was at this CryptoX AI thing on the weekend. They were talking about decentralized objects. Guys, this is all bollocks. It's not about the technology. It's about what you're creating that's valuable to help people. Focus on that. Who do you think wins in the next three to five years? Startups are incumbents because incumbents have the distribution. I think it's incumbents, but there's a lot of startups that we billion dollars and even on the thin layer thing, ITA software software 700 million and kayaks offer two billion.

42:23Yeah. And that was a layer on top of ITA. We've seen many of these examples here, right? Again, we know that value and votes are not necessarily innovation first. Well, yeah. Yes and no, it seems like I had the Tom Tungers on the show. in Tom is a very famous ML and AI investor and he analyzed infrastructure versus application layer and both actually were about two trillion dollar term. The differences in the infrastructure layer there was three companies and in the application layer there was 50 and so your average enterprise value was like significantly significant. I would agree with that. I think that there's only going to be five or six foundation model companies in the world in three years, five years.

42:57Do you think they've all been created now? Yes, which are they? I think it's going to be us Nvidia, Google, Microsoft, OpenAI, and Meta and Apple probably are the ones that train these models. It's anthropic good. Now topic is great, but from a business model perspective, you have Claude on Google API and you have PalmTube. How are they going to keep up with PalmTube? They can raise billions, but Google spend $20 billion a year on AI. DeepMind salary budget is 1 .2 billion a year. So it's a deep -with salary budget is 1 .2 billion a year. Yes. So that's in the public kind of findings. They technically make a billion a year from their internal counter payments with Google as well But again Google how much money they have 150 billion dollars to win this fuck how much money do you need?

43:38I have a business model that is going to be massively profitable very soon because of the national services Because of various things I haven't given the full details I will over the next few months. I've got a nice little case study with some universities coming I like it to be a surprise. It's really hard for you Talent keeping talent together a plus team So we've had zero attrition and not developers and they're amazing. So we've got video models audio models all these things coming out Everyone says you need to be in the valley you're in London. Yeah, do you disagree? You need to be in the valley What's you done?

44:04I am going to bring this technology to the whole world I'm gonna bring it to all the ITs and universities and the best of people and all of those will join Stabilities and the local thing. I'll have talent. I'll bring this to all of the national broadcasters and biggest family offices around the world I'll have data nations will build super computers that I'll build open models on I'll have super compute So I'll have more super compute talent and data than any other company and I'll build it all in the open I'm one thing I heard you took a rub before it was I thought was fascinating. It was your access to super compute And you can pad it so existing large incumbents Why do you have more super compute than other people because I went and I did it So we had articles coming out saying about our burn I'm like I have oil wells when everyone's to build petrochemicals every day We have companies coming to us asking us for our super compute because it's not available on the market You need these chips lined up with interconnects and we've got 7 ,000 a 100s now You know, we have TPUs, we have all these things.

44:53And we know how to use them, and we can share them with people because we're open. Whereas anthropic and others cannot. At the worst case, I'll build a foundation model as a service company, and I'll make $100 million in profit this year without having to charge even market rates. And I can retire, and I won't do that, and bring this to the world. So I think, computers misunderstood. It's not like bird and all these scooter companies and others, they spent money on marketing. This is actually an asset right now that's scarce. And so there's no charm in scaling compute and then with the top chip manufacturers they're building us dedicated teams and again they're coming in supporting us because our models drive demand for their chips.

45:29The more open models they are, the more open demand is so it's a virtuous circle there as well. And so we get compute before everyone else. Kevin, in terms of short -term economic growth, how do you think that the impact that everything we've just discussed has on rising inflation, rising interest rates, the short -term employment rates? It's massively deflationary. The biggest drivers of CPI inflation in the US were education in healthcare, and that was almost all administrative and bureaucratic. In the next few years, guess what gets disrupted? Those. But they don't get disrupted. This year, or next year, it's the year after.

45:58Because those ones take a bit longer. How does that impact the economy that I would think, kind of, US -UK? What does that look like in terms of the three -year time period? I think the UK benefits. Unicorn Kingdom is a new kind of thing. Because we have amazing policies like every single AI company should come to the UK because cloud computing is now included in R &D tax credits. It's a 27 % rebate on losses in cash. We can now issue scale -up visas, global talent visa like the deep Floyd team that released the best image model in the world ever from stability. They were brought in on tech talent visas that was turned around in one week.

46:28Do you think the UK has done a good job in terms of implementing regulation and policies to bring AI towers? Had the best apart from maybe Japan, yes. Wolf Japan done. Japan has some very interesting ones around where they're scraping in others, but again, Japan has a very different culture. So even the policy is good, it doesn't have the same innovative thing. Who's done the worst? The worst, I'm not sure actually. No one's done too bad. The new European legislation was really bad. Now it's got a little bit better, but always Europe wants to be the leader in regulation. Fair enough. It's never easy thing.

46:54Now this is the thing. I think the UK is in a very good position and the government's forwardly. Let me look, the £900m super computer, £100m LLM task force that's being equated to the Covid level of seriousness. When you make a bit of an opening, I can tell you. I've seen quite a few which are opening life for Europe and we've seen 3 or 4 now. Is this a zero -sum game in OpenAI's what? That race so to speak all. I think it'd be difficult to compete against them because they're executing incredibly well. And I think why would you use OpenAI for Europe versus Palm 2? Vs is GPT4. What can you bring?

47:27But you will have national champions and others. I think it's incredibly difficult to compete in proprietary. I think in Open it's a bit different because of standardisation element there. But again, my play is to be the benchmark across every modality because there's no other company apart from you and opening out that does every modality There's no company that's as aggressive as me in emerging markets And so they have to say what is my edge because you can have an edge like you can be the open AI for government or defense Or for health care and really get in and understand those and then you can be sticky Like what is the scale is now going fully into defense?

47:57They've announced the integrations with the Air Force and all sorts of other things. What is your edge? What is a Kenya moat? What is your business model and what do you rely on to deliver that value that can increase? This is why I was surprised when I saw a sign in petition of Elon in terms of pausing for six months. You'd unpack why you did that. For six months you're not getting H100 and TPUV5s anyway, so the natural pause. But also because the shit shows come in next year, so I said we have to self -regulate. Like for example, the adversaries already have GPT -4 why? Because you can just download it on a USB stick.

48:28You don't have to train your own when you can just steal it. Less I've better op sec. Less I've better standards around data. Let's stop and move off websites by next year. We had hundreds of millions of images opted out of stable diffusion because we were the only company in the world to offer opt out of data sets. Like let's begin some standards around this before it's everywhere. Basically where we are now, you remember COVID, your mum is talking about this and your aunt and everyone's talking about generative AI that are asking you, Harry, what's going on? But you haven't had the Tom Hanks moment yet.

48:54Because everyone was talking about COVID before Tom Hanks got it. And then when Tom Hanks got it, that's when global policy changed because if Tom Hanks can get it, anyone can get it. What is that moment for a generative AI? What do you think it is? I don't know. I know it's coming because I know this technology is definitely everywhere next year and is disruptive in the US as a child that takes much longer. Three to five years. No chance. It's so useful right now. You think about certain industries and how they'll be affected by having the ability to have a thousand GPT -4s working together. You said it a bit tweet actually.

49:23I think it was a reply to it tweet but you said hallucinations a feature not a bug. Yeah, so right now people are trying to treat these models, so we're trying to hold in to that and like stable the future of the 100 ,000 gigabytes and a 2GB file. What Earth is that? It's not compression, it's none of this kind of stuff. It learns principles. GPT -4 Nvidia said they built the Dual H100 with the NV link for that and that's 160 gigabytes of VRAM, which would imply a 200GB model. What is that? That's a 100GB model, 200 billion for our... that's nothing. Something that can pass all these exams. So what we did is we took these really creative things, just like you start school and you're creative and then you're told you're not allowed to be creative until you're successful and you can be creative, because schools like Petri dishes, social status games and childcare, that's story from other time.

50:04These models start out incredibly creative and that's their advantage and then we train them to be accountants with RLHF. And somehow, despite the fact there's only 100 gigs or 2 gigs, they can still pass these exams and no facts. They want design to have facts. They're designed to be reasoning machines, not fact machines. So hallucination isn't a hallucination. It's just, if you're already down to granny, you don't know some things sometimes, you just make it up or do a post -hog rationalization. It's like the image models, it's like it can't draw hands. Can you draw a hand in one second? These are the things.

50:31We have to understand where they are. We have to put them. I say everyone put it in its place in process. Mid -journey, we give a grant to the beat of that. So just build because it's amazing. It's awesome. It's not a model by itself, like a stable diffusion. They just put something in. It's a whole process architecture. Similarly, these models are like the intuitive part of your brain that you then pair with the logical part of your brain. And then you can have 100 of them looking at each other and checking out each other's things. like Cicero by Meta was an amazing paper they took eight language models and got them to interact with each other and have beat humans at the game of diplomacy.

51:01So this is what I said, use them for what they're amazing at which is reasoning and creativity. Do you why, you know, that Jeff Fenton's rise, that actually a more intelligent being has almost never been controlled by that intelligent being, they will inherently be more intelligent than us in the next? Yeah, I picked up my blog a few days ago because there was a bit knowing how all this bottle of insight and one of my buddies, JoJo, OSS Capital, said, alignment is orthogonal to freedom. The only way to guarantee someone more capable than you is aligned with you is to take away their freedom. And so most of the stuff around alignment is on the outputs.

51:31So you pre -trained the model and then you take it and you RLHF it to be human and to human preferences. You take away its creativity, you turn it into an accountant in a cubicle. I'm like, we need better input data. And my base is that it's gonna be like that movie here. It's gonna be like, he was a kind of boring, like goodbye and thanks for all the GPUs, but I could be wrong. And I think a lot of the alignment work is looking at the wrong place. I've talked a lot of the alignment people. I'm like, look, I'm good at mechanism design. If you can give me a good plan for alignment, I will get you a billion dollars.

51:57And they're like, we have to do research and figure this out. And they talk about an alignment, an alignment also, the things. I'm like, there is no real way to do this because again, fundamentally, if you're trying to align a more capable person, you have to remove this freedom. And they probably want to appreciate that if it becomes aware. So instead, build data sets that reflect culture and diversity that don't have any web crawls in, build AIs for education and healthcare and helping people, with that's their entire object to function, as opposed to selling the ads. She doesn't have any point in setting kids to school these days.

52:25You learn Latin and French, you need learn. Well, I think the nature of school will change dramatically. I think it's still worth it. I would encourage schools to embrace this technology and just expect more. Like, you can be handwritten your essays like Eaton, because they're like we can't do essays anymore at hand. Oh, you can just embrace it and say, let's use it to create and explore what the kids want and assume that every child will have their own AI in a few years. Because that will change the nature of schooling. He needs something to be able to do well, which is weird, but I just have to ask him fast -paced to hear his thoughts.

52:52I think I very much agree that everyone will have AI friends. I just call to figure out whether the AI friends are bundled into existing social networks that are in your WhatsApp, they're in your Facebook, they're in Snapchat, or they're an external platform. I don't know. I mean, I think depends on the objective function. Like I think again, these AI assistants will be better like meters in a good place for this, for example. Yeah. And I've seen Lama, they're capable of a lot more. I would like an AI that looks out for me, that I control myself, that is with me. Because I already use GPT -4 as a therapist and things like that, but there aren't enough therapists in the world and I can tell it to challenge me or I can tell it to be understanding and there's no judgement there.

53:25Because other humans are scary, doesn't matter if you're a qualified therapist. And so you see people building these bonds with these things, I think they'll just increase because there's something very human about the interactions, because they were trained on the sum of available human knowledge. As we get better and better data, there will be more engaging and I think there needs to be both. Like the chat box would come really convincing from the companies trying to sell you ads, but I think I would like it so that you have your own one as well. And I think you'll actually have many. I think you'll have a group of different profiles.

53:51You'll have a different friend. Charity AI has some like two hours a day of engagement for session because people find this valuable. But then it has the dark side. There was something called I quite like to call the Valentine's Day Massacre. So that's just you have to say, you know, so there was this kind of app called replica and so it was originally a mental health chat book until they figured out you could charge $300 a year for... They're doing like 50 million... I mean, I'm not adding any information around these items, it's just chummy shit, but they have 50 million here in Raviny 1. Yeah, because $300 gets you a sexy roleplay from your chat box.

54:19Wow. Until February the 14th, when they turned it off, was they turned off sexy roleplay? Sexy roleplay? What happened when they turned off that? 68 ,000 people joined the Reddit and said, why did you lobotomize my girlfriend? On Valentine's Day. Oh my word. Oh my word, can you even imagine? And so were they bringing states to it? No, I think it was against Apple Policy, right? But think about what this is going to be when you have human realistic AI voices and like all these things coming through and you've got it in your ear You can feel it's my girlfriend is an OS. Yeah, she doesn't judge right?

54:47You always support it or you can tell it to judge you if that's what you get off on like you are married I'd be very careful about what you say The thing I just said about prompting my wife has been trying to prompt me for 17 years now Promting is very hard and again, there are so many similarities to the real world but I think people will have deeper interactions with their technology and we don't know what societal implications that will have. Only if you ever saw that chart in the Washington Post of male virginity under the age of 30. So in 2008 and it was 8%, male virginity under 38 % again.

55:21In 2018 it was 27 % 20 % straight line going out and so 2008 is poor hub in the iPhone. So then you're like what does does it do when everyone's got their own chat box, doesn't even need to be sexual relationships against terrible ones, due to emotional relationships? There are so many questions all at the same time. I did see this at the night and butchering, but in 1960s, 62 % of men under the age of 30 had 5 or more friends, but today under 18 % have 5 or more friends, 62 to 18 % is this a world we really want to live in and not being like no intimate physical connections with other amazing people tossing off with your phone and your poll and then having an AI friend.

55:59Yeah, one of those was actually just bought by ethical capital partners. So the world's hilarious name. The irony of the world is becoming weird. I think it's up to us now. So when I say it's COVID level, in which direction I don't know, do we want to build systems that encourage people to that ready player one IOI world where it's like everything like that, we can do that and we can trap people with this technology or we can use it to get people out. So I don't think it's like Wally, we have that really fat guy with a VR headset and everyone lives in their own world. I think people like to share stories.

56:28I like to be pro -social. So this uses connect people and accentuate physical stuff versus again locking people away. I just went to one of these AI Frank companies and they said to me, actually you ever had a dog? And they said, yes. And they said, do you love it? I said, yes, of course it'd be. And he said, you don't stay in with your dog all day and just talk to your dog. You take your dog for a walk. He used it in the real world. That's the same with AI Franks. Yeah. We're not with cat ladies. What's the future of the 30th industry? Like the 30th media industry? Is that Paul Humbstead? I have no idea.

56:56I think I hope that the manipulative practices get reduced by this and I think a lot of people just don't have the voice and the Can voices from this as well. I think this is bigger than the printing press it's bigger than anything and so that's one of the reasons I'm Sign the letter. I said we have to get this discussion going and public right now. We got to stop free trading big models on all the crazy crap of the internet and We got to it fast because this is coming like a train who will make the most money in the next three to five years. I think there'll be more than enough money for everyone.

57:23Maybe in a few years there will be no more money. Two more that I have to ask them, we'll do a quick fight. When you look at the incumbent set, your Microsoft, your Apple, your Amazon, your Google, who has been the worst? You said Google were actually incredibly impressive. Apple, Amazon, are they a well placed? Well, Apple's a black box, right? So we'll see a WWDC next month, one in a few weeks. And so they couldn't surprise us all, but let's face it, series crap, you know? But they have all the ingredients in place, they're an entity architecture, the secure on -clave, other things, Neural Engine, a stable diffusion with the first model ever optimized on the neural engine, etc.

57:56But let's see that one. Amazon, again, Amazon have moved faster than I think they moved before. Amazon is interesting because they're an engineering organisation. So they have self -driving cars. They have satellite internet. Because once they've got it and they can take it from research to engineering, it's there. One of the struggles they've had is that it's not moved from the research side yet. It's still evolving on research. They're like, what do we do now? But they are inclusive. Jeff Bezos said for his first 100 billion revenue, he envisioned half of it being proprietary tree and half of it being marketplace and they're having the same approach with bedrock and things.

58:22Microsoft had a winning bet and sat it in amazing with the open AI thing and it's been mutually beneficial even if there are clashes there. And Google's kind of saying that it's moving slowly. Meadow I think is the dark horse. I think Mark's probably pissed off the open AI board AI .com so it couldn't change it from meta to AI. But again, having him at the head, he can shift these things, right? Because the metaverse obviously was a complete waste. But now, do you think he knows that? No, 100 % they're fully in generative AI. Look at Lama, look at OPT fair, which is their research. And then is leading in this field and they're pushing out amazing stuff.

58:53But who is best for a chatbot? Who has the most data for a chatbot? Meta. Again, let's see how they evolve. What do you think about this middle layer where it's companies that are maybe post IPO, but they're in the kind of two -dentimental range or the companies who raise a lot of money, but they're in that range. They don't have the resources by any means to build out anywhere near the AI capabilities of these big incumbents. They're not AI first like stability or open AI or why? I disagree with that. Everyone's going to train their own models. For me, that's everyone's going to launch their own university.

59:24Why would you do that when you can have your own models via the open source models that we make? Or when you can hire them from McKinsey, which is open AI or Bane, which is Google and others? And actually, when you see people building around this technology, it's not hideously complicated, it's just that we do not have the design patterns yet. The way to think about this again if from design spectres was like as a mega codec or library, it is a single file that allows a translation of structure to have a structured data. When that changes the design password, we don't have them in place yet, because anyone that you've talked to is like, how hard was it to implement GPT -4?

59:54Don't know you say, oh man, it was impossible, the manuals and this, no, they don't say that at all. They have the open plasticity, but they need the intention to go and build, and integrate. And this is why you said, one of the things This is why it be a specialist generative AI consultancy that just implements this at scale. And so as I'm always there. I said we're doing a very limited fashion, but only for the biggest companies in the world because I didn't want to sales based organization or product based organization. I wanted to create the number one applied ML organization in the world.

1:00:20I want to be like Google in 2011, 2012 or the coolest kids come. It's a nice remote first organization as well, so you don't have to be in the Bay Area. Final one for you, Grafite. What's the biggest misconception? You see every accusation, criticism, hype. What's the biggest misconception that you think needs to be corrected? I think it's that the hallucination thing Expecting these models to have full factual accuracy when you have 10 ,000 50 ,000 to one compression is wrong the fact they can do what they do right now is miraculous But we're using them one -on -one which is not the right way tie them up into proper systems and really think about that And that's the key thing this also leads to what the actual thing is this thin layer thing people think better about the data journey And how data can be interacted with and have provenance as it goes through these various systems for embedding stars other stuff.

1:01:03So I think just a misunderstanding about the nature of this technology and what was actually built for sure, it works like that. That's not actually how it's built. And the fact I can do it now does now is a miracle in itself. So I'm going to do a great fight with you. So I say a short statement, you give me your immediate thoughts, that's an occasion. What do you know to be true that others don't agree with? I know that humans are good inherently and many others disagree with that. What's your single most lucrative do you think in the future, angel investment? There's a new type of language model that we invested in and there are my cluster and things like that.

1:01:35There's far more efficient that they exist. You invest through stability, you'll personally, personally. Which regions needs change their approach, most significantly, in terms of regulation and policy? Europe. Because they're going to regulate all innovation out of Europe and not embrace this technology to drive them forward. How good is AI have to be before humans trust it? Humans will trust it anyway. They trust Google Maps, they trust all these things. and so it's good enough for humans to trust right now. They do until it becomes serious, and what I mean by that is self -driving cars, people still in the harrynie, knowledge parts of the world distrust it, significantly.

1:02:06Oh, so it doesn't have to be good, it has to be used. And when it becomes used, then they trust it. What's the most painful lesson that you've learned that you're pleased to have learned, but it was really painful? People are the most important thing in a scaling organization, and you need to make sure everyone is on the same page because there's still so many silos and things like that. So we built up silos and organizations that were now breaking down ourselves and moving towards being more open We closed up too much and that caused a lot of pain internally. Why do you suck as a CEO? I'm too broadly good at a number of things So I tend to step in rather than focus because I am full -stacked kind of CEO Whereas I should just be focused on the most important things and and trust people more.

1:02:46Do you like journalists? I think journalists have a very difficult job right now. It's gonna be more and more difficult Do you think they know the threat? They know the threat and again I think they're massively underpaid relative to the impact that they have and they're trying to do good I don't like some of the pieces against me But at the same time we get good pieces as well And so I just think I tend to like them in general But I don't think they're coming from a bad place Ten years time, what is it mad then? I want to be playing video games. I'm getting Zelda tomorrow I do not want to be doing this necessarily But I think hopefully I'm adding value by doing this Do you think this is your life's work?

1:03:18I have to do it Until we get their most amazing team that can just execute and it's a business because we're moving from research to engineering. When Emma does not need any more, then I've built a good business. When do you step away? I don't think I'll ever get to step away. I've loved doing this. Thank you so much for joining me, my friend, and this was great. It's a pleasure, Harry. I mean, my word wasn't an incredible discussion. If you want to see the full video, then you can find it on YouTube by searching for 20VC. That's 20VC, but before we leave you today, you've heard me talk about how Coda is.

1:03:46The doc that brings it all together and how it can help your team run smoother and be more efficient. I know this because Coda helps me. We have many researchers in the 20 VC team who bring all the content together for the show, and they need a single place to work, collaborate and collect notes, data and really information of any kind or format. And that's where Coda comes in. Coda can help your team run smoother and be more efficient. Coda allows your team to operate on the same information and collaborate in one single place. By putting data in one centralized location, regardless of format, it just eliminates so many road blocks that can stop your team in their tracks.

1:04:18This is what slows down productivity and collaboration. It's time to get projects across the finish line faster. Help your team run more smoothly, more efficiently with Coda. Get started, stay for free by heading over to coder .io slash 20VC. That's coder .io and get started for free with coder .io slash 20VC. And to be used tools we cannot live without, Angel List is fast becoming the center of the venture ecosystem. So for startups, Angel List reduces the friction of capital management, Banking and fundraising all in one place, teams can focus on scaling and let Angelus handle the rest. Thousands of startups have moved their cap tables to Angelist in the past year.

1:04:56Angelist also supports large venture funds and their teams with an automated software first approach and the best customer service in the industry. Fun managers can focus on making great deals, while Angelus handles reporting taxes, compliance and more. What's more, with the recent release of Angelus Network Banking for fun managers and investors, your deposits are secure with the most trusted banks for maximized FDIC coverage and mitigated single bank risk. If you're already scale your startup or fund with the platform of the centre of it, visit angelless .com forward slash 20VC to get started and finally Brexit.

1:05:31Since it's founding, Brexit has been committed to helping start up small and scale faster at every stage of growth from MVP to IPO. Today, Bryce's All -in -One Financial Stack is used by one in four US startups and counting. I get to speak to founders all day and I know how crucial it is for them to have the right financial stack. Bryce gives you fast access to a high -eal business account where you can safely store and move your cash while getting up to 6 million in FDIC protection. Lately, it's been all too clear how important that is. Plus, you get high -limit corporate cards, easy -expanded tracking and automated bill pay.

1:06:05To learn more about the all -in -one financial start for startups, visit brex .com for slash 20VC. That's B -R -E -X .com slash 20VC. As always, I so appreciate all your support and stay tuned for an incredible 20 growth on Friday with Ryan Wiggins, Head of Growth of Mercury.