Bill Gurley and Sunny Madra talk open-source vs. proprietary AI | E1825

0:00You don't want any one party controlling a platform technology. You want it to be open source. You want closed source solutions, private company solutions, open source solutions. You want a range of opportunities. And remember, the last time we had some founder say, trust me, I'll get us some regulation. That was SBF. And he was going to be the one who got us federal regulation for crypto. And he's on trial. That's his very moment for a bunch of Meshuggah. This Week in Startups is brought to you by LinkedIn Marketing. To redeem a$100 LinkedIn ad credit and launch your first campaign, go to linkedin.com slash thisweekinstartups.

0:43Vanta. Compliance and security shouldn't be a deal breaker for startups to win new business. Vanta makes it easy for companies to get a SOC 2 report fast. Twist listeners can get$1 ,000 off for a limited time at vanta.com slash twist. And CLA, innovation takes balance. CLA's CPAs, consultants, and wealth advisors can help you get from startup to where you want to end up. All right, everybody. Welcome back to this week in startups. It is Monday. It is Madra Monday. Yes, that's right. every monday money madra joins us sunny madra of definitive ai and we do our ai demos and uh this week sunny you told me you were having some deep uh weekend discussions we know silicon valley and the tech industry on the weekends that's when everybody goes deep they don't have meetings but the back channels start to light up and the back channel was lighting up this weekend talking about what topic well open source right there was a you know a couple of big threads that kicked up this weekend and you know um the tech start flying and then i think you know we have a special opportunity have a guest with us today oh okay who do we have yeah we've got the legendary the goat bill girly all right bill girly uh i think maybe third time no only second time on this week in startups you were last on 2017 uh episode 722 for folks uh yeah uh so welcome back um bill gurley of course uh from benchmark and um many great companies uh you're particularly passionate about open source why uh in relation to ai bill well i mean obviously there's a lot of answers or answers to that question, but most recently when I did the regulatory capture speech at your conference, I was mentioning at the very end that oddly, some of the, what you might call the early incumbents in AI software, were running out and promoting the idea that open source should somehow be curtailed or kneecapped.

3:00In particular, it's quite notable that all of the loud voices are either executives at these companies and or large investors at these companies. And because of the way these companies have raised money, the largeness of those investments are in the tens and hundreds of millions of dollars. So quite a bit at stake. And there now seems to be a growing populace. And this is what led to the conversation Sonia we're having that are worried that these Companies are basically, you know, at the very start of this AI movement, trying to cut off their biggest competition whatsoever. And the thing that would probably unleash the most innovation and the most prosperity, which would be if open source models were prevalent.

3:49And this is more Sonny's world than me, so I'll let him comment. But I do believe that the stats about the performance of these open source models is actually quite compelling. And what I'm hearing internally from our portfolio is that specifically Lama, too, but Sonny sent me another one today, are really starting to gain market share amongst the startups that are using these tools. So it would be really unfortunate if they found – and when I spoke at your conference, I mentioned this guy, George Stigler, who had won a Nobel Prize in academic mission who had talked about how companies use regulatory capture.

4:29And he said the two things they try and do is store competition and protect pricing. Clearly, if they were successful in getting governments to block open source AI, they would achieve both of those goals. yeah and so that great talk by the way if you haven't seen it just do a google search for all in summit bill gurley and um the name of the talk 2851 miles about regulatory capture so sunny when we look at the space the open source space let's get to you know maybe the the tail of the tape some metrics here on how these open source projects are faring against uh the large language models that are available by api call right and i guess uh paradoxically open ai which is now the most closed of all the ai companies but has opened in the name and started under the mandate that this technology was too powerful for it to be closed it needed to be open to everybody has gone exactly closed and has gone exactly regulatory capture with uh sam altman begging literally uh begging uh for the government to get involved and to regulate so maybe we could talk a little bit about um you know the what are the leading uh open yeah so maybe projects yeah so let's just kind of level set like you know who the players are because you know to bill's point there's many different people that are showing up uh and and also this is kind of a good lay of the land because you see here some startups some real incumbents and you know some some new folks that really kind of risen to the top and so just level setting both entropic and open ai you know uh i'd say kind of leading models in the space definitely closed uh microsoft is kind of in this unique straddle they're they're open sourcing lots of unique pieces of content additional frameworks that you need around um like llms but they're gone all in with open ai google started this whole thing obviously the you know the reason others can get there is the papers are open source and people have written about them but we haven't seen an open model from google in in a while although you know they do support open models in their uh vertex platform and then meta and then databricks uh and there's another one we'll bring up mistrol as well today uh these these folks are fully open uh commercial use case is different commercial use uh differs based on their licenses but in general they're sharing everything and we're seeing really fast innovation so that's the level set um in terms of Just two quick things to add to what Sonny just said.

7:01It's remarkably ironic that OpenAI, when Elon backed it, was open source and moved away from it. And then second, as Sonny mentioned, I've been reading a ton this weekend to try and get a lot deeper in this world. And everyone points to this Google paper, attention is all you need, as the thing that allowed these LLMs to become successful and to really progress. So you do have an academic paper with all Google people on it that actually led to the technology that's being used by Anthropic and OpenAI. So I think that's ironic also. Anyway, sorry to interrupt, Sonny. no no and and going deeper into the playing field what's your analysis of what's happening here what i hear often uh people say is when you're in the lead you're closed and when you're behind you go open so ios was in the lead and they and they kept it very closed google search their algorithm they're in the lead they have a dominant position they'll never open up the search algorithm but then you have somebody like maybe meta they feel like they're pretty far behind on language models So they open source it and the report was they internally leaked it accidentally on purpose.

8:16All right, listen, when you're selling to business to business buyers, you really want to get your pitch in front of decision makers. Why? Because upper level execs are usually the ones making purchasing decisions. Duh. The problem is high level folks can be really hard to find and target on most social media platforms. But on LinkedIn, oh my God, they know all of the CTOs, all of the CFOs. all of the VPs of finance, engineering, HR, recruiting, all those titles are sitting there waiting for you. And now let's just talk about the funnel. LinkedIn's about to hit a billion members. Did you know that?

8:52950 million members at this point in time. There are 180 million of those 950 who are senior level execs. There are 10 million C-level executives in that 180 million senior level execs, which are part of the 950 million members. I am a C-level executive. I am on LinkedIn all day long because LinkedIn equals business business equals LinkedIn and LinkedIn ads are built specifically for b2b marketers. LinkedIn generates two to five times higher return on ad spend than other social media platforms. LinkedIn equals business business equals LinkedIn when people are on LinkedIn, they're ready to do business.

9:27It's that simple. So make business to business marketing everything it can be and get$100 credit on your next campaign from me your boy jay cow i'm sending you the hundy linkedin.com slash this week in startups to claim your credit that's linkedin.com slash this week in startups terms and conditions apply because they're giving you the hundy so maybe some some historical information here on when do companies choose open versus closed and to your point google was behind in cloud services and led a movement to open source kubernetes so there's a company that has played both sides of of the aisle depending on where they are when you when you look at this field open ai and anthropic these are startups uh have you ever seen startups be uh asked for regulation this early uh and this often and be so opposed to open source is this a new trend and what do you attribute it to i mean bill like there's more your area you answer that one and then i can chime in yeah Well, it's funny because I have certainly never seen it, Jason.

10:37I've never seen this early. And maybe it just speaks to the wild success of open source. I mean, the number of venture-backed open source companies today versus 30 years ago is just amazing. It is a very disruptive way to get your technology out there quick and fast. And, you know, one of the other things that Stigler talked about, a phrase I mentioned several times, he said, when you have this type of blocking regulation, you end up with a net loss to society. And I'm one that firmly believes that open source is amazing for society because when you have technology locked up with patents, it's harder for ideas to spread.

11:21It's harder for ideas to spread across borders and to other countries among all these different smart people that can get out and innovate. And so I have never seen something this early. Obviously, there's a ton at stake. There's a ton. Like I mentioned, these companies have raised money at an unprecedented level. So, yeah, you could call them early state startups, but you could also, I mean, they've raised billions of dollars each, you know, and only maybe in the ride sharing market did you have that happen so quickly. And so there's a lot at stake. There's clearly a lot at stake, but I've never seen startups proactively pitch governments and not just ours, but several governments around the world.

12:02I've never seen that. I also find it really suspect that there aren't like technology lead, like academicians out there leading this charge. The people that are leading the charge calling for the regulation and calling and some of them raising this question of whether open source should be allowed are the incumbents. They are the one either the incumbents are their backers. and some of them you know mustafa uh for uh from inflection on our podcast said i know it looks odd me being the one that's that's asking and uh yeah it does it is a it is odd and it would it reeks of protectionism pulling up the ladder behind you um and with microsoft and google they seem to be uh maybe trying to dance along the line here bill they want to have cloud computing uh cert they both have major cloud computing services azure google cloud they want to offer these things but they also have you know a proprietary use for this obviously google search and bard and the chat interface are going to overlap microsoft trying to get bing to break out using ai in the office suite so they see that as a competitive advantage what's your How would you handicap Google and Microsoft's behavior here?

13:27Well, I mean, I think to a certain extent, it's interesting on the chart that I don't know who produced the chart that Sonny put up, but it had Microsoft neutral. I could find myself believing that. I mean, they had to do a rather convoluted deal to get access to the technology that they're using with OpenAI. And I wouldn't be shocked if they're comfortable with a hedge on that, primarily because they already control these creative products that they now believe will be enhanced with AI. And I don't know that whether it's open source AI or someone else's AI that it really impacts them because it's the lock-in they have on the product.

14:13So I wouldn't be surprised by that. And like I mentioned, Google's played this both ways. You know, they pseudo-open-sourced Android because within open source, there's different dimensions on how open it is and whether you've really committed to a third party like the Linux Foundation that runs the regulatory aspect of it. Or not regulatory because I don't want to confuse it with the government. That runs, you know, how it's – the most open projects have a third party that keeps it independent. Open source foundation. Yeah, and a board. There's others other than the open source foundation, but that's the largest one that manage the process.

14:53And Android's not like that. So anyway, I had to do that quick aside, but Google's done. Whereas Kubernetes, wide open, Linux foundation manages it. And so they've been all over the map. And as you mentioned, they once published a paper, we love open source, but it's not right for search. You know? Yeah. Well, once you get that lock in, that's when you don't want to open source it because people can then build competitive products. And as we've seen in search, that space has not seen any changes in 20 years. Like that has been a locked box where nobody's innovated for 20 years. A number of people have tried.

15:28I tried myself. It's very hard to get any kind of a foothold in search. But I don't think, you know, those two aren't at the forefront of this. As you mentioned, it seems to be a battle between these extremely well-funded startups. and really more of a community. I'd say the people that are on the other side of this, based on what I saw going around this weekend, it's more of a community. I mean, it involves, you know, like Jim Zimlin, who runs the Linux Foundation, who's been out talking. At your conference, I spoke to Stephen Wolfram, who told me he thought it was ridiculous that someone would try and ban open source here on a safety reason.

16:04And so that's what I say when, if I might be more open to listening, if I thought it were, you know, some broad group of technologists and big thinkers that were making this argument. But all of the arguments are coming from the people with the most to lose. Yeah. And that seems crazy. Sonny, you want to just give us an idea of how? Well, yeah. I mean, you know, we're going to lose Bill in a couple of minutes here. But like one thing I'd love to get his thoughts on, because I think it really reinforces the point here is that. So, you know, last week we saw a big funding announcement on Mistral.

16:36It's like a European-based group that raised$100-plus million to Bill's point, a lot of money. And they released their model open source, open for commercial use as well. And you can see a couple of key points here. One, you can see that there's 7 billion parameter model uses half the amount of memory. And down here, even just against other open source models, right? There are 7 billion parameter model is outperforming Lama 213 billion. Again, this is just against open source, but like the rate of innovation is moving so quickly here that if some of that regulation were to come into play and these folks couldn't put this out there and they had to do it, you know, through some kind of regulator, which has been, you know, like the FDA or some of that, those are the ideas that we've heard put out there.

17:23I think we just wouldn't see that. And I'll just add one more thing. Like last week, J. Cal, you know, we even demoed GPTV. And, you know, a week later, we get Lava and Lava is an open source implementation of like a vision model in the same example that we did there. you know we have it and i think you know to bill's point i can't get my head around um you know what it is around open source that's bothering people we've seen open source in operating systems we've seen it in databases right we've seen it in mobile phone operating systems you know you get things get patched quicker when the code is open you can find vulnerabilities and so the arguments that around safety no one is providing the sort of the background as to you know what is it that is not safe here fairly obvious what's happening here bill uh the people who have the lead are using job destruction and the fear of ai from science fiction and this you know uh could get out of control the the demon could be unleashed they're using that in order to maintain their lead because they know full well that large uh numbers of startups or mid-sized companies embracing an open source project would lead to the demise and would absolutely evaporate the lead of open ai and this really is about open ai and sam altman let's call it what it is sam is the one who's leading the charge although mustafa and reid hoffman have been perhaps even more vocal or at least openly vocal on podcasts and whatnot so it's not just that but once again hundreds of millions billions at stake yeah i mean i agree with sonny i mean it's ironic but linux is the most stable, most secure operating system that's ever existed.

19:04And I think, you know, I go back to some of the original thesis of why open source would work and more eyes, the better, the more transparency, the better. And the notion that the people that it's just so ridiculous for someone to say this stuff's super scary, like, you should be really afraid, but let me do it. You know, like I'm, I'm, you should trust me, but it's super scary, but it can do really good things, but it's scary. Let me take care of it. You know, help me be the only one that gets to take care of it. And yeah, that's sad. I hope there's quite a few people stepping up. I hope this doesn't happen.

19:46One last thing I would mention before I have to go. I think the cat's out of the bag. So you're not going to stop there from being open source in parts of the globe. So if you shut it down in a particular region, that region is going to fail to innovate relative to the other regions that are out there. There's a really cool piece of open source technology called RISC-V, which you guys have talked about a couple of times in the semiconductor space. And I believe they intentionally moved the governing body outside the U.S. because they were afraid of the restrictions the U.S. was putting on semiconductor technology.

20:26And I think it was smart that they did that. And risk five is going to be wildly successful, you know, regardless of what happens to U.S. regulation. And so I think governments need to be particularly careful that if they take a first step move here, they're going to put their own society in a worse place and their own entrepreneurism in a worse place than others around the globe. I mean, and we've seen this play itself out with social networks and the impact they have on, you know, society writ large, the fact that so much power was consolidated in meta, and oh, trust us, we'll be the will protect you, it doesn't work.

21:08The person who's making a profit and has a profit motive, the incentive is too great to act in the public's interest, whereas a group of people working on the project together, they keep each other in check. Yeah, that's like, it's a governance thing, right, Bill? undoubtedly and this is this is an important message to spread immediately yeah so i appreciate i appreciate you guys we appreciate you checking in here uh and a great continuation of your talk from all in uh everybody that's bill girl we follow him on twitter where he's super active if you're a sas or services company that stores customer data in the cloud then you need to be uh sock to compliant you knew that from a third party and you need that third party to close big deals.

21:49And if you want to get compliant easier and faster, you need to use Vanta, V-A-N-T-A. Vanta makes it so easy for you to get and renew your SOC 2. On average, Vanta customers are SOC 2 compliant in just two to four weeks. Compare that to three to five months without Vanta. And Vanta can save you hundreds of hours of manual work and up to 85 % of compliance costs. This is a total no-brainer. And Vanta does more than just SOC 2 compliance. They also automate up to 90 % compliance for GDPR, HIPAA, and more. You can't afford to lose out on major customers. We all know that. Listen, it's a hard year.

22:24Last year was hard. You can't lose those major customers because you don't have your compliance dialed in. Just work with Vanta. Get your compliance automated and tight, and tight is right. Lock down those big deals. Here's the best part. Vanta is going to give you$1 ,000 off. That's 10 hundies. Get$1 ,000 off at vanta.com. That's vanta.com. For$1 ,000 off, you're so too. okay so let's go deeper into the actual models here yeah um because the demos are great i you you glossed right over the demo of what you called lava yeah which i'm gonna pull that back up i think we should stop we should start there because this is i think where the rubber meets the road if you're wondering why a friend of the pod sam altman might not want competition well if you look at open source last week we had the multi modal chat gpt4 we were playing with it i have it on my phone now it's extraordinary it's amazing you take a picture you upload a picture and you ask it to do things with the picture one of the things we have to do is to make a hamburger recipe based on a hamburger that sunny um was interested in but there is an open source project called lava yeah large language and vision assistant got it so this is built on top of uh llama i take it or it's its own project it's its own project and um it it's you know built to mimic the spirits of multimodal gpt4 and um you know just kind of touching a little bit on you know the microsoft being yellow in that chart that we talked about earlier you can see here it's a uh put together by researchers at uh it was university wisconsin and madison microsoft and columbia and so the kind of really interesting group and and i think it you know does justify that you know microsoft is still supporting you know open source which is important here and um you know it's it's a great paper uh we won't spend too much time um looking at it but i suggest people to go look at the the github url which is here but um you know this doesn't have the multi just type in l l a v a um large language and visual uh and vision assistant and so this interface looks very similar to the chat gpt4 multimodal interface you said give me instructions on how to prepare this this being a brioche bun uh weak egg yolk hamburger that looks absolutely delicious and this is the same image we used last week correct and so yeah how did it do versus chat GPT for?

25:00Yeah, so I'll say, look, chat GPT, you know, V did a better job in terms of describing what it was, and then the instructions. But, you know, for me, what I'll say here is the fact that this is available, and it's open, and we can build from it, it's one week later, that's that that's that race that's starting to collapse. Now, right, we're not like 100 % equal a week later but being you know my grade on this would probably be like you know in comparison to that it's like a you know maybe a b because it doesn't here's here's the results right here yeah if you gave them an a you would give this this student a b which means hey this student applies themselves and yeah if you were let's say a startup would you tie your wagon to chat gpt4 a closed system with deep ties into microsoft or would you fork this and start building with your own hooks into it and wrapping what would you advise a startup you've invested in what would you advise them to do sunny or would you have them split the baby and and it would be like on a use case by use case right like i think it really depends on like what what value you're trying to build like if you're really want to you know have the whole stack in your control because that's how you can provide value i think i would do this like i would use you know lava but if it's just a small feature within my product then i would use chat gpt because i'm going to get there quicker and i don't have to worry about the infrastructure and scaling costs and so the reason i didn't do this one live it probably takes like two or three minutes to run sure because they don't and this goes back to you know the funding they don't have the billions of dollars behind it to have you know huge farms for inference and so this thing it runs a bit slower um and you you can try it out live yourself as well jaykel i'll drop the link for us here um and you know i think they don't have the 10 000 gpu cluster that open ai has um exactly and but you know there's going to be cases where for your business it's important that you build this and you you build some ip around it and so i'll go back to a point that i made a couple of maybe episodes ago which was the API for GPS, right?

27:13The API for GPS on a cell phone been around for a long time. Apple, you know, made it slightly better, but companies were built when they built a lot of infrastructure around that, right? And saying, hey, like, you know, I'm going to build my app there and I'm not, this is just one part of what I'm trying to do. So I think you have to kind of look at in some places, you know, how important is this to the core of what you're trying to build? But I do want to reiterate, like, look how fast, how quickly this is happening on the backs of it. And I do think, it's always great to have Bill here earlier.

27:40This is why there's so much noise around slowing this down. Because if you're this company, you know, being valued at, you know, whatever, 20 billion, 50 billion, 100 billion, these ridiculous valuations right now, to have an open source model right on your heels, it's got to be really challenging. I'll put that back to you as an investor. Anybody who wants to buy shares in OpenAI at 90 billion for common shares, I take it, with a capped upside, you would probably want to monitor these open source solutions and say, well, if they're getting 90 times revenue, or 100 times revenue, whatever it is, for the current valuation of open AI, is that a good bet?

28:19Or are these open source projects going to create downward pressure? And the downward pressure would provide is the API calls, and what chat gpt for open ai could charge for access to their language model is going to go down and because you could just fire up your own open source solution on your own servers and this is where microsoft um google and of course that we didn't talk about amazon's position here which is i think they also invested in anthropic anthropic yeah amazon's been very clear that they would like to be a neutral third party in all of this and so amazon will make money no matter what they're going to have every language model on aws and their interest is in continuing to lower the cost continually and just make up for it in scale but this could be you know a road to nowhere this could make open ai be like seagate you know it's like a hard drive provider like there's just not a lot of value you know in the cloud today you know for the most part obviously you know some people do do this specifically but when you're asking for a server you don't really care whether it's you know Intel or AMD or you know some kind of thing that's virtualized running you know arm underneath it right and so especially you know now if you have specific use cases that require that type of a certain instruction set that only Intel provides for performance, you'll ask for it.

29:50But I think those clouds have all those options available. All right, everybody. Steven Estes is a principal at CLA. Clifton Larson Allen is a professional service provider that specializes in CPA, tax consulting, and wealth advisory. Welcome to the program, Steven. Thank you for having me. So at what point should a startup seek out professional accounting and tax services? I think there's a couple of tipping points, right? One of them is really as soon as funding and equity-based comp come into play. So whether it's a 500K seed or pre-seed or a$10 million Series A, as soon as you're raising and you're looking to hire talent and give equity to those people, you should have quality advisors for both legal and tax.

30:31Like I said, another marker really is that foreign activity, having a foreign subsidiary or foreign founders. You can really find yourselves in hot water real quick if you don't know what you're doing in that regard because all companies have to play by the same rules in the international sandbox as whether it's a startup or coca-cola get started right now at claconnect.com slash tech let them know your boy jake how sent you claconnect.com slash tech to get started right now let's go back to mistral for a second i thought this was fascinating as well yeah i just ran this live while we were sitting here so you know for the folks that were watching yeah but okay so uh suffice to say you run it on your own servers you can go faster and it's going to just get faster every week and then you know these open source projects can sometimes have such a diversity of talent in them and because they don't have a command and control structure they're going to have more interesting insights right that's the nature of open source projects you might have these four or five you know developers in south america and then these 12 in you know japan and then these six in ukraine and they all have some other use case and they contribute to the model in a way that's different than if microsoft or google or open ai are building software they don't need to ask for permission to work on some part of the of the model or the open source project correct yep yeah i mean not at all it's permissionless yeah Yeah.

32:02Now, there might be some permissions that occur when you do want to, you know, actually commit those changes. Exactly. Yeah. So to go now, usually how these projects are governed is, you know, they, they, you know, like Bill was mentioning, they have organizations that are built to, you know, kind of decide like, you know, where the direction of the project is going. And so those then directions, then basically the developers on the open source project use those high level, like think about that as roadmap and vision and say, oh, we want to build this next and everyone aims towards it. but nothing stops you from taking it if you don't agree with that and doing something and then some you know submitting it for commitment into the project and either they can take that or you can fork it many projects have been forked right and so yep um that's happened that's always your option if the if the core project doesn't want to go off on your side quest you can just make a copy of it and you fork it and exactly you you then start a new project we've seen that happen over and over and over again okay let's get back to mistral m-i-s-t-r-a-l yeah got it okay now you mentioned how many uh parameters let's explain that for uh civilians listening yeah yeah so the way the way i kind of put this into like into simple thinking is more parameters equals um Like parameters are like neurons in a brain.

33:26And so if you look at a small organism, it doesn't, you know, has a very small brain doesn't have a lot of neurons. I think it's generally considered that humans have somewhere between 40 and 80 billion neurons. and so um when we think about it the more um neurons exist the more data that this has access to that not not from like what's what well it started from what it was trained on but that data is then processed and held in kind of these weird fragments that we've talked about before and so the more of that that's there the more it's able to basically reason thank provide you really you know incredible results like we've seen um and so these folks at mistral have done an incredible job of creating a smaller model that performs like the larger ones that's you know probably in due to some of the technological um approaches they've taken and also perhaps even the training data and um and they've shown that in uh like knowledge and reasoning and across these different tasks uh which which you know math code uh which is really incredible i thought this was very exciting so the exciting thing here is you know people are taking different approaches they're making um you know these language models more efficient and faster smaller better cheaper all of those things and how many language models uh is the open source community grinding on right now and when i say grinding like making daily progress on like you know of the you know it could be some projects that are abandoned etc yeah so like listing all the language model projects on github or wherever hugging face that's not productive but let's say major ones that are getting daily updates to them right because daily updates would would be a sign that this thing is cooking so how many are there's probably under 10 that are like and so what we really have to do is maybe take a step back because the space is playing out in a really unique way um the folks that are there's a less set of people that are going after really really large models like open ai that are general intelligence models right where more of the energy is going is smaller run in you know kind of more confined uh compute requirements and that area has a lot of models like too too many to count and what many folks have realized is that it's a better approach to go after a smaller model um that's you know tuned or trained for specific tasks then to try to compete with uh general purpose models yeah and so that means that when we talk about this competition and regulatory capture if you're open ai if you're anthropic and you want to lock all this down if you're reid hoffman or musafa uh or sam altman you know greg whatever you're saying hey slow it down trust us we're gonna protect everybody you then have they're under attack that hey i'm gonna make a verticalized one that's just for audio just for video just for um code just for you know a specific language or a specific culture whatever the vertical is you're going to see um you know these large language models be attacked by a thousand cuts uh by verticals and by thousands of people contributing to an open source model is the goal to do smaller more nimble models like and have the best model with the least parameters is that like an attack vector here that people are trying to make these things smaller and more efficient and cheaper to run yeah i i think that's the ultimate goal because what we've already seen in the last year and you know with the rise of nvidia stock was these really really large models require you know compute and energy like less people talk about the energy but compute and energy when you have like a cluster of 10 000 or 100 000 gpus the amount of energy it's using is really substantial I wonder what one of those costs to run a day, like one of those GPUs, you know, an H100 per day, if it's actually doing work, it's doing jobs.

37:47Maybe the guys can look at this one. But honestly, I think it was like the amount of electricity more than like a house. It was like something really substantial that we can probably research in the background here. It might cost$1 ,000 a month or something, you know. So, if these things cost$1 ,000 a month to run. Yeah, yeah. and you've got a thousand of them that's a million dollars a month it's 10 million a month in energy costs for 10 000 of them which i think is what some of these big clusters are now doing yeah it's not that's not nothing you know it's like 120 million dollars a year in electricity cost on top of these things costing 40k each or whatever they cost yeah yeah it's yeah the energy cost might be 25 of the yearly cost so over four years it might be the same yeah it's crazy yeah so just like i'm looking at it here while we're doing it but it basically is saying like um an h100 card runs at 700 watts right and so think about that 24 7 that turns into like a really really big that basically turns into i would say my guess is close to a megawatt a month yeah so we have to then figure out what a megawatt a month cost and here's from reuters um running chat gpt is very expensive for the company each query costs roughly four cents according to an analysis from bernstein if chat gpt queries grow to a tenth the scale of google search require roughly 48 billion worth of gpus initially and about 16 billion worth of chips a year to keep operational which is just crazy but i think this is going to plummet right it's going to go down 50 90 percent a year so uh and yeah i i get the sense this is going to be you know if it's four cents a query it's going to be 0.4 cents and then 0.04 cents and then nobody's going to even think about it kind of like storage uh became so de minimis but the energy cost is pretty crazy um there's been reports that everybody's making chips so open ai supposedly is looking into making chips we heard last week that part of i think the anthropic deal was that amazon was making their own chips and they wanted anthropic to use it obviously apple makes the m1 the m2 and whatever's in the a whatever they're up to a 14 15 16 whatever's in the the iphone uh so we now have everybody's going to make their own chips i heard google is going to make their own chips too so now we have google has been doing it for a long time right they have the tpu right which is uh yeah they've been doing that for a very long time there seems to be some intense uh ramp up of this because not only can't you get nvidia chips there's a line out the door for them um people maybe don't feel great about the pricing of them so now everybody makes their own chip so that is something i didn't anticipate um and so here's the headline from friday microsoft to debut ai chip next month that could cut nvidia gpu costs i don't and that's from the information so i started my career in um actually making chips but on you know for networking equipment and this happened in networking uh at the late 90s uh everyone needed their own chips to basically interface with the immense growth of the internet is on the sort of the core side of the internet with optical you know with fiber optics right and so everyone was making chips there and there was a time and place you know somewhere between say 97 and 2001 where there would have been like hundreds of chip companies making front end chips to interface with optical transceivers that ultimately all consolidated down to maybe like three companies.

41:29And it happened for the same reason, like no one could get these chips, right? And then no one could develop them in ways that each particular vendor required. And I think we're just kind of seeing that movie again here and we'll see an explosion, which will bring the cost down. And then ultimately, I think it'll consolidate as well but those are the waves that we see in tech anyways right where we see like expansion and consolidation and expansion again uh and so a massive amount of investment and then it becomes commoditized or so cheap that people maybe some number of players bow out and let or roll up and yeah exactly that's another way i mean we we saw that i predicted this in gpus because we saw it with fiber people were building out so much fiber that they overbuilt yep and then i think google and some other providers bought up a lot of that fiber um you know pennies on the dollar uh that have been overbuilt so we might be in the over building overbell phase um what other demos do we have for this week um yeah so i just had like mistral which you know we can run through that one actually like we just i had the paper there so i can pull that one up and so this is their playground on on hugging face as well where we spend some time uh all right let's uh give me a give me a drop in here jay cal and i'll send you this link too if you want to try one give me a question here oh question well you know um how about what are the best uh restaurants

42:59in napa that's it i mean this is like some live information who knows uh where they got this information um but it's screaming fast even on hugging face i mean instant answer so that was that's the first thing i'm noticing is that this is and um yeah uh it it of course uh got it exactly right bouchon french laundry soul bar i mean ad hoc yeah these are all great ones uh but i know a bunch of these um and if you asked it um uh what uh add uh put those in a table and add the average cost of dinner i wonder if it has anything there i don't think so uh you know that would be like more live data but it did pretty good here you know depending on what so yeah i don't think it'll have that but like yeah so yeah uh yeah but you know that's a tough one i think even for chat gpt But what I really liked about this one, and I'm glad you did this question, was like, and this is what we're going to get as we put these out there.

44:08It first of all kind of says, hey, in order to provide the best, but here are my assumptions, right? And it talks about, hey, you can define best. And here's like a combination of food quality, good service, and that. And then target audience and research how it got there. I really, really am a fan of, like you said, the speed. You can run this on your laptop if you want to, right? And I think that's what's really incredible here. Yeah. so you could give this language model to an astronaut you know going to mars and if it lost contact with the internet it would still be able to give those answers which is just for people to think about this if you were stranded on a desert island having one of these language models fully trained on a laptop you would be able to have like an increasingly um impressive conversation that could wind up saving your life if you asked it how to start a fire on a desert island what would it say yeah yeah i mean actually ask it you know i'm trying if you were trapped on a desert island how would you on a desert island how would you start a fire now let me bring it you know if you didn't go to survival school and you had this all of a sudden you've got this bot with you on some foreign uh location you i wonder if it will actually even help you build that fire uh you could you find a piece of glass is one technique rubbing two sticks together is another uh obvious technique trying to find flint or stones or metal that could spark something um gather material collect dry leaves small branches of course that's kindling find a suitable location okay create a fire pit prepare the tinder okay this is all the same stuff kindling prepare fuel create a fire layer structure okay ignite the fire there are several methods to start a fire such as using flint and steel magnifying glass and or a battery i forgot the battery one um so that's interesting so tell it to um explain to me the techniques in number 10 let's see if it understands that we're asking it about its existing answer because steps one through nine are preparing a fire putting the wood together but not actually sparking the fire what we care about is actually starting it i wonder if it will be able to teach us how to start a fire oh yeah here we go flint and steel this method involves striking flint again steel rod okay magnifying glass on a sunny day focus sunlight uh onto the tinder using magnifying glass battery and steel wool that works yeah uh lighter matches yeah if you have them um it's pretty good pretty good answer um and what else do we have in the uh demos in the in the demos everybody loves the demos here yeah so last week we kind of uh right at the end we got into uh whatsapp and the model and the ai and i think uh one of the things i wanted to correct so i wanted to pull whatsapp back up because you know it is available in the group chat so i was giving a second here yeah so you can do a group chat and include an agent so because that was one of the reasons you didn't give it a even a higher grade last week yeah and i loved it last week i was super impressed um and i just thought you know if my wife and i had a a chat going and we had slash gordon ramsey and we could you know ask gordon hey here's what they you know here's what we have what should we make hey we got you know some salmon and we've got some pasta uh you know we've got butter and cheese and it was like okay yeah you make some farfalle with salmon in it here's a recipe for you so um this is our ai you know twist uh whatsapp group that i created let get this over the side and in here we have the meta agent right and so um and if you can see here you can do sort of an at and it allows you to add meta ai and say i am thinking about booking a trip to napa what are the top five hotels okay there we go let's see and what's great about this is like i could then ask a follow-up to it to respond back in a second maybe they've they've gone slow as well now there we go well here we go uh yeah or bosh uh uh meadowood four seasons yeah i mean it's not great it's not terrible yeah not not not great not bad what's a fun activity for the three of us to do while in lake tahoe over christmas i mean it's pretty obvious what you do in lake tahoe over christmas right uh i consider ice skating uh taking a sleigh ride uh enjoying a festive atmosphere yeah um yeah we could have also gone skiing but we're snowmobiling not a great answer a lot of snow at christmas sometimes though maybe you know yeah uh how about more answers additionally you explore emerald waters of clear kayak toward not in the not in the winter you could do that uh So you can see the meta AI is like a very rudimentary AI, I think.

49:29I'm not sure who. I'm not sure which model that is. You think that's Lama? It must be, right? Is there a language model? Yeah, it's Lama, right? Lama, yeah. Yeah, it's definitely that one, yeah. Yeah. Yeah, I think that leaves a lot to be desired right now. I think this is where I think verticalized AI is going to be much better. when you ask about travel you really need a travel ai that's just that right um so i think that that's where these models are going to wind up is that they'll be very verticalized and fine-tuned ones by vertical and if you ask about food it should really be narrowing you down and not just giving you the generic one it should be just like when you search now for recipes on google it doesn't just give you 10 blue links it really is thoughtful and they have a lot more information on our listen it's been another amazing episode thanks for bill girley for tuning in it's pretty clear uh the community must fight for more open source anytime anybody says trust us uh we will be the sole source uh you should be wondering uh if you should trust them and just uh listen you need to fight for open source and you don't want one private company it's not a dick to any you know it's not a dick to sam it's not a dick to anthropic You don't want any one party controlling a platform technology.

50:48You want it to be open source. You want closed source solutions, private company solutions, open source solutions. You want a range of opportunities. And remember, the last time we had some founder say, trust me, I'll get us some regulation. That was SBF. And he was going to be the one who got us federal regulation for crypto. And he's on trial. That's his very moment for a bunch of Meshuggah. i mean are you following that sunny you were down the crypto rabbit hole uh yeah and and you know very deeply and following it and it's really sad to see you know i think one of the you know i think one of the more there's a lot of shocking things so not not to say but one of the more shocking things i saw last week is they were running some kind of um insurance service and there was like a counter that was saying oh like this much money is insured by this and that counter was basically like a random number generator that was like picking a number between like you know 7500 and 10 000 and like i mean talk about explicit like fraud i there you go exactly i mean it's deranged your point is if yeah to say that like you know this um was a real business and you know um that they were just in over their heads when they were doing free meditated things like this and this is where like all the chats eventually get dumped all the emails all the slacks everything everybody winds up flipping no matter what the case is it's very rare that a group of people will circle the wagons like even the mob you know and the mafia you know had a hard time maintaining that and this is like their entire lives their families their traditions were around this right yeah um maybe the cartels on the margins can but not a bunch of dopey kids who've been working together for 18 months they were committing fraud after fraud in this case they were using a random number generator you know with some parameters on it to dupe the public when they were getting insurance yeah if i'm understanding it correctly um and listen and it takes like it takes you know like a certain amount of evil to do that right because you're you know there's lots of cases people have gone sideways like you know theranos elizabeth holmes and you know at the core i think she was trying to make a you know like a testing machine it's just yeah kind of that's the case of it maybe got ahead of you right you got ahead of your skis you were a artist you thought ah you know i can i can fake it till i make it this wasn't fake it till you make it this was let's orchestrate a huge crime um and you know listen if you're in the mob and you flip you get whacked with these kids like if they flip there's nobody there to whack them it's not like sbf's parents are like some criminal mastermind cartel they're a bunch of dopey yeah uh stanford professors who you know didn't raise their kid correctly and uh you know they're not going to whack the other members they're all going to flip on each other it's all going to come out i i told everybody i will bet dollars to donuts that he gets over 30 years i'm saying over 30 years one thing we take the over the under 30 year sentence i would take the over yeah if i said 50 we should take the over i think under i think so 40 would be the line we have to think it through what would you take at 40 pardon we take the over the under at 40 years i think the under i think it'll come right around that okay there we go wow look at that i said a pretty good line so the line is probably 36 you know he gets out before retirement kind of thing you know that i think they're gonna bernie made off him you know bernie made have got multiple life sentences so i set the 30 year but i actually think there's a chance he gets life i think this could be a life kind of situation well you know it's very rare that you catch somebody red-handed yeah doing a multi-billion dollar uh crime yeah there's a small number of multi-billion dollar crimes in the history of humanity yeah it's hard to pull off and so if you're going to put you know some you know uh kids in a local city who were dealing drugs 20 years 30 years and they were dealing you know a half million dollars in drugs a million dollars in drugs let's just call it a million dollars in drugs a million dollar drug you know cartel you know in chicago and this kid was stealing over a billion i mean it doesn't feel proportional right if he if he were to get any anything but a magnitude more because it's a thousand x that crime yeah this is serious crime folks yeah i don't know yeah i mean you're you're spot on there i just don't know how to like do that relative calculation i mean like there's people in in jail today for marijuana use that and you know it's it's it's allowed now right yeah no luckily we're we're there is uh consensus trump biden obama before everybody is you know a line that we need to reverse those for non-violent felons um and really think about those all right listen another great episode and uh you know listen um he's guilty i've made my decision already it's the evidence is there the jury will make their decision but i hope he gets a a huge sentence uh and that's it all right sandeep madra they have it folks definitive intelligence if your company is looking for ai and analysis of big data and you need help Sonny's your guy.

56:21Just email Sonny at definitive.io and we'll see you all next time. Bye bye.