RUMOR: GPT-4.5 Coming Before EOY (Maybe Even This Week?)

0:01Today on the AI Breakdown, we're looking at Mistral's latest release. It appears that open source AI might have a new leader. Before that on the brief, rumors are flying that GPT 4.5 could be coming before the end of the year and maybe even this week. The AI Breakdown is a daily podcast and video about the most important news and discussions in AI. Go to breakdown.network for more information about our YouTube, our newsletter, and our Discord. Welcome back to the AI Breakdown Brief, all the AI headline news you need in around five minutes. We start today's episode in a slightly different place than normal, which is in the realm of speculation.

0:38However, one of the interesting quirks of the Twitter slash X space right now is that there are a couple accounts that seem to have insider information on what happens next at OpenAI. The two accounts are the Jimmy Apples account, which we've talked about on the show before, and then Flowers from the Future at Futurist Flower, which only started in October, but has already built a bit of a reputation. All the way back at the end of October, on October 26th, that account had tweeted, OpenAI big announcement in December, which is a fairly generic and open-ended thing. And obviously this was all before the OpenAI drama, so who knows if that would still be true.

1:12But in terms of establishing their bona fides, let's look at another tweet from October 26th, where that same account tweeted, Just verified one of my sources. Gemini will come in the second week of December. As Pedro responded on December 7th, I came from the future, second week of December. This is actually true. Now, of course, one correct prediction does not mean that we should believe everything that this account says, but suffice it to say that there are a lot of people taking it seriously. And so, last week, when on December 7th, that account said, Some of you can look forward to a little something coming next week, a lot of people stood up and took notice.

1:43Now, they expanded on that about a day later when they wrote, Okay, so there's one big thing and one small OpenAI thingy waiting for us. The small thing is currently delayed due to company dramas, and the big thing seems to be progressing according to plan. December remains exciting, especially next week. That was tweeted on the 8th, so next week is in fact this week. That account also posted this cryptic image of a black screen with a blue ChatGPT logo in the middle. The AI Safety Memes account writes, GPT 4.5 next week? Jimmy Apples and Futurist Flower, both of whom have made some accurate leaks, are dropping gossip.

2:17Futurist Flower writes, There's one big thing in one small OpenAI thingy waiting for us. This is the same quote that I just read you. Now back to AI Safety Memes, they continue, the big thing could be GPT 4.5. The small thing? Maybe that was the GPT store that just got delayed to 2024. Now in the same post, they also referenced a Jimmy Apples tweet. Keep an eye out on a potential end of December GPT 4.5 drop and new multimodal from Anthropic. How big of a deal would this be? I don't know. But remember, the jump from GPT 3 to GPT 3.5 was massive. Gemini might look quaint overnight. The AI Safety Memes account also shared a tweet that Jimmy Apples deleted that came out right after Gemini was launched that I had referenced on this show, where he wrote, At Sam Altman, what are you waiting for?

2:59Going to take the mitts off yet and release what you are sitting on or keep drip feeding? Cat emoji. Now, AI entrepreneur Bindu Reddy also tweeted about GPT 4.5, writing, Fascinating. Grok, in response to the prompt, what were the top AI tweets in the last two days? Grok has decided that OpenAI has released GPT 4.5. It is hallucinating, but maybe it knows something we don't know. I'm hearing juicy rumors about 4.5 dropping before end of year. Now, what Grok had come back with was a theoretical tweet from OpenAI that said, Introducing GPT 4.5, the next step in our journey towards more advanced and capable AI.

3:32Stay tuned for more updates and demos. There has been some reporting recently, although it's unverified as of yet, that Grok is actually accessing draft tweets, which obviously would be a big security problem if it's true. But anyways, all of this adds up to probably nothing, but certainly some interesting speculation for a Monday morning. Is this important? Is this something you should plan around? Obviously not. Could be that we just wasted a few minutes on rampant speculation. But the reality is that right now, coming off of this Gemini announcement of an announcement, as well as the OpenAI power struggle a couple weeks ago, this is, for better or worse, something that everyone is interested in.

4:06Meanwhile, in terms of real news, the EU has finally come to agreement over the EU AI Act. Now, if you've been paying attention to this, you're probably confused around exactly how the EU lawmaking process works, because we reported a few months ago that the EU AI Act had been passed, but then there were a few remaining steps. But then it turned out those few remaining steps were very controversial or at least had room for some controversy. And now, once again, an agreement had been reached. But as the New York Times writes, the law still needs to go through a few final steps for approval. So TLDR, this podcast is much too short to try and actually explain the European Union political process.

4:40But suffice it to say that the AI Act has moved through yet another critical step, and one which did see a fair bit of controversy and disagreement. Now, the biggest issue recently had been around what the rules would be for foundation model makers like OpenAI. A lot of the other rules, such as basically banning facial recognition AI, had been there from the beginning. Remember, the EU AI Act has been in the works for many years, in fact, longer than ChatGPT has been around. Part of the reason that there was such controversy around things like ChatGPT is that it was kind of a new addition late in the process, and there were some who wanted to punt it off entirely and basically figure it out later.

5:13Now, when it comes to those rules for foundation models, they did not end up exempting them entirely. but as Axios sums up, foundation model providers will need to submit detailed summaries of the training data they use to build their models. However, Axios also writes, quote, there's a big catch in the EU's landmark new AI law, which is the fact that it doesn't come into effect until 2025, which is of course an eternity in terms of the development of AI systems. Again from Axios, the hiatus leaves plenty of room for the US or others to undercut the EU's plans before they go into effect by, for instance, implementing less restrictive rules before Europe's kick in.

5:46Senate Majority Leader Chuck Schumer has expressed concern that EU-style laws enacted by the US would put American firms at a disadvantage competing with China. Now, EU lawmakers like Thierry Breton hailed the agreement, tweeting out an image that pointed out that the EU was now the only continent with comprehensive AI regulation. Many commentators, however, noted that the EU, screaming about how good at regulating it was, sort of just reinforced the perception of its place as a backwater for technology. Adam Singer tweets, You hate tech and economic growth. No one takes you seriously. Along with the graph showing how few European tech companies there were relative to Asia and the U.S.

6:21Stephen Serges tweets, You need to understand that people want industry that works, and regulation can be required to make sure this exists. But nobody is clamoring for regulation in the absence of industry. Mike Solana writes, I am genuinely so embarrassed for you. And Rohit writes, Genuinely thought this was someone making fun of the EU. Now, all of that said, there are indications that regular citizens in a lot of countries, including America, are not as universally pro-AI as the people tweeting in response to the EU might make it seem. The AI Safety Memes account again writes, A new poll dropped and wow, are Americans anti-AGI, anti-open source, and anti-effective accelerationism.

6:59Three to one want to prioritize AI safety over AI innovation, five to one want to stop the AGI race by banning development of ASI, and 5 to 1 are opposed to EACC. Now, once again, these results do come from the AI Policy Institute, whose whole self-admitted role is to slow down the development of AI, but they're still pretty interesting statistics. The question, I think, is to what extent these numbers reflect people's considered opinions or the incredible amount of fear-based media that has been published this year around the potential risks of AI. In either case, the numbers are so stark that they're probably fairly hard to ignore.

7:31And so really the question becomes, I think, what US policymakers do with all of that. One last note, for those of you who are sad that that Google demo video where it was identifying the duck and then the hand motions about what was going on ended up being faked or fabricated or at least just not exactly what it appeared. Someone wired up GPT-4 Vision to see if ChatGPT-4 could do it, and sure enough, it was able to perform a little bit better. Anyways, friends, that is going to do it for today's AI Breakdown Brief. Up next, the main AI breakdown. Quickly, a brief word from today's sponsor. As a listener of this show, I suspect you like to stay up to date on all things AI and tech, which is why you have to check out the chart-topping podcast Web3 with A16Z Crypto.

8:13Produced by venture firm Andreessen Horowitz, Web3 with A16Z is the perfect companion podcast to the AI breakdown. Web3 with A16Z Crypto is your definitive resource for the future of the internet, whether you're interested in the convergence of AI and crypto or simply curious about what's next. If you need a place to start, they recently released an excellent episode with Stanford cryptography professor Dan Bonet and former Google X engineer Ali Yaya in conversation with host Sonal Choksi about the intersection of AI and crypto. From fighting deepfakes and proving humanity to large language models like ChatGPT, they cover it all.

8:47I highly recommend checking it out, especially if you'd like to learn more about how AI and crypto will impact our everyday lives. Beyond crypto and AI, this show is for creators seeking more ways to truly own their work, for business leaders trying to prepare for the future today, and for innovators exploring trending tech topics. Don't miss out. Follow Web3 with A16Z Crypto on Apple Podcasts, Spotify, or your favorite listening app. Welcome back to the AI Breakdown. Mistral first jumped into the headlines back in June, when the company, which was at the time only four weeks old, announced that they had raised a$113 million dollar seed round.

9:26Yes, a nine-figure seed round, which many at the time took as indication of just how overhyped and crazy the AI world was. However, subsequent to that, the company's first model that they released, their Mistral 7B model, started to become a favorite among open-source developers. Indeed, whereas Meta's Lama had commanded so much of the energy and attention throughout the year, Mistral really became another genuine open-source competitor of the big AI labs at companies like Gemini, Anthropic, OpenAI. Now, as we head into the end of the year, there has been a lot of speculation around what the open-source side of the AI arms race will look like going into 2024.

10:04Aravind Srinivas, the CEO of Perplexity, writes, next year we will see at least two of Meta, Mistral, and Anthropic catch up to Gemini and GPT-4. Going to be exciting times. Well, today we got a bunch of announcements from Mistral that suggest that Aravind may be even more right than he seemed. Now, first of all was the confirmation of the funding round, which was first reported last week. The final tally was 385 million euros or$415 million, and the valuation, which again had been reported last week, was$2 billion. The round was led by Andreessen Horowitz and other investors, including Lightspeed, Salesforce, B &B Paribas, General Catalyst, and more also participated.

10:42Now, given that a big topic of our conversation during today's brief was the EUAI Act, it's interesting to note the extent to which Mistral is using European pride as a selling point for this company, said co-founder and CEO Arthur Mensch. Since the creation of Mistral AI in May, we have been pursuing a clear trajectory, that of creating a European champion with a global vocation in generative artificial intelligence, based on an open, responsible, and decentralized approach to technology. Now, in terms of how Mistral thinks about European regulations, they are one of the companies that has been lobbying for a total exemption for foundation models, with their argument being, as TechCrunch sums up, quote, that regulation should apply to the use cases and companies working on products that are used by end users directly.

11:23Now, part of what makes Mistral interesting is that their 7B model was released under the Apache 2.0 license, which means no restrictions on use or reproduction outside of attribution. Now, for those wondering how Mistral plans to make money, today also came with the announcement of a new API program. Through that, developers will be able to access their new Mixtral 8x 7b model, which they call Mistral Small, and that will be for free again under the Apache 2.0 license. But a third model, Mistral Medium, is only available through the paid API platform. Devendra Traplow from the company writes, Excited to release Mistral Platform, three chat endpoints with competitive pricing.

11:58Mistral Tiny, Mistral 7b Instruct version 0.2, upgraded base model with higher context length, 8k to 32k, and better fine-tuning. Mistral Small, Mistral 8x7b Instruct version 0.1, matches or exceeds GPT 3.5 performance multilingual. Mistral Medium outperforms GPT 3.5 on all metrics multilingual. All endpoints have a 32K context size. Author Carlos Y. Perez writes, Mistral Medium blows away GPT 3.5. What's Mistral Medium? HyperWrite CEO Matt Schumer says, Looks like Mistral has a model that's even better than Mixtral 8X7B, and they're serving it to alpha users of their API. scoring 8.6 on MTBench that's frighteningly close to GPT-4 and beats all other models tested.

12:42This is their medium size. Large will likely beat GPT-4. Carlosy Perez again writes, Mistral has released their report on Mixtral. It's very impressive. It challenges Llama 70b and GPT-3.5 on less hardware and at lower inference latency. Furthermore, it uses only 12 billion active parameters to achieve this. So how does this all work? Well, if you guys are willing, let's wade into the realm of the technical for just a moment, and for this I'm just sharing other explanations from folks who are attempting to provide a little bit of a layman's explanation of how Mistral is approaching things, even if as you'll see it's still a little bit dense.

13:15Dmitry Zulgakov writes, Mistral model is hot, with mixture of experts like GPT-4. It promises faster speed and lower cost than model of the same size and quality. Fascinatingly, the speed-up is unevenly distributed. Running on a laptop or the biggest GPU server benefits the most. Here's why. The new Mixtral 8x7b model has around 47 billion parameters. Each token uses only a subset of those, approximately 25%. Two experts out of eight. But when running multiple requests in the same batch, different requests will activate different experts. For a small batch, speed is determined by how quickly the GPU can read weights from memory.

13:50For a single request, reading two out of eight experts is four times faster. At a small concurrent load and most of the speedup evaporates, all experts have to be loaded. If one can fit a big batch with many requests in a GPU, the situation changes. Speed now mostly depends on how many multiplications, flops, are done. Mixtral MOE still does only 25 % of those. And we can expect speed to approach 4x. But running a big batch comes with its own challenges. The model needs to be aggressively split into many GPUs. Mixture of Experts brings more options here like expert-level parallelism. GPT-4 is rumored to use Mixture of Experts too.

14:24It serves a lot of traffic and thus benefits from this regime. Bindu Reddy also wrote an explainer writing,

14:56on all the experts at the same time. In the case of Mistral for each token, only two experts are used, and this could potentially mean a 4x speedup compared to something like Llama 2. GPT-4 is rumored to be an MOE model as well. It's likely to have a greater number of larger size experts. When it comes to training MOEs, there are complexities around dealing with training the gating network or router and load balancing, avoiding over-reliance on a few experts. Since MOEs are more parameter efficient, you don't require quite so many computational resources. It will be really interesting to see how much the open source community will improve Mistral MOE in the coming weeks.

15:28Already, I am seeing several fine tunes, but nothing very compelling in terms of metrics yet. Anyways, there is probably a much deeper show that could get into all of this if that's something that interests you. But for now, the other piece of the story that I want to talk about is how they launched this. Again, from Bindu, the right way to drop a model, she writes, Mistral AI just dropped a Mistral 8x7b model as a torrent link. Christmas came early for open source AI. Eric Jang writes, Mistral's brand is already becoming one of my favorites in the AI space, releases 87 gigabyte torrent containing an 8x7b MOE model via tweet, and refuses to elaborate.

16:01That was a couple days ago before this whole announcement came out. Nick Dobos pointed out the contrast to what we got from Google around Gemini last week. Mistral launching a torrent link versus Google Gemini ultra blog post waitlist. Mistral actually understands the market it's building for, power users and hacker. Google keeps trying these disappointing consumer demos to convince the uninterested. Don't understand doom to fail. Now, the other thing that some people are discussing is how much Mistral is sucking the oxygen out of the room for other open source competitors like Meta in particular.

16:31Farrell writes, Meta is losing mindshare. Hopefully they release Llama 3 via Apache 2. Accelerate Harder writes, If Meta really wants to make an impact on the open source world, they would choose a fully open source license and they'd do it soon, before Mistral steals their thunder completely. Now, bringing this conversation back to the very beginning of today's brief, The way that the Flowers from the Future account kicked off was actually with rumors about Llama 3. On October 23rd, that account wrote, Llama 3 will come in different sizes, but the most capable one will be around 120B. Oh yes, and it will come February 24.

17:04Will be multimodal on par with GPT-4. Still others don't want to wait for February. Boyan Tungus says, All I want for Christmas is Llama 3 175B. Now, lastly on this misdial announcement, a number of people, including OneLittleCoder, noted that in their announcement post, they had a space for a link to a safety document that they ended up not actually linking. OneLittleCoder writes, is Mistral actually trolling us or maybe they forgot the link? Professor Ethan Mollick writes the significance. For those who don't follow AI closely, one, an open source model, free, anyone can download or modify, beats GPT 3.5.

17:39Two, it has no safety guardrails. There are good things about this release, but also regulators, IT security experts, et cetera, should note the genie is out of the bottle. However, to Mistral's credit, AI Safety Memes writes, quote, it has no safety guardrails. No open source models do since you can just strip them off in two hours. Mistral is just honest about it. Anyways, different people might have different takes on this, but from a technology perspective, what Mistral is putting out is pretty impressive and certainly makes these big investments and big valuations make a little bit more sense.

18:07That, however, is going to do it for today's AI Breakdown. I appreciate you listening or watching wherever you are. Until next time, peace. Thank you.