The Truth Behind DeepSeek

0:03Hello, and welcome to the 404 Media Podcast, where we bring you unparalleled access to hidden worlds, both online and IRL.

0:30I'm your host, Joseph, and with me are two of the 404 Media co-founders, the first being Emmanuel Mayberg. Hello. And Jason Kebler. Hey. Sam will join us for the second half of the show. Right now, there is one story which is basically across all of tech, all of AI, and Emmanuel wrote about it. The headline is, Deep Seek Mania Shakes AI Industry to Its Core. um emmanuel first off let's just keep it super basic to start what is deep seek and where did it come from because to a lot of people it feel like this just happened overnight yeah uh deep seek is an ai model that's developed by uh a chinese company of the same name that itself was spun out from a Chinese hedge fund.

1:27And yeah, it was, I've heard about it several times. I follow the frontier model space, which is a way to refer to the leading AI models. And it released last week. The AI community was very excited about it. Whenever one of these models is released, there's a bunch of benchmarks that people run to see how it compares to all the other models. It scored very well on the benchmarks, was beating OpenAI on some things, was beating Llama, which is the open weight model from Meta. Again, this is normal. Like the newest thing is usually pretty good. And the hype started to build more and more as people learn more about how it was made.

2:14And now we're at the point where I'm surprised. I think my best way to measure how impactful a particular piece of news is, is when I hear about it from my mom, I almost never get a text asking what this or that is in the news. And I got a text from her yesterday, which was just like, hey, do I need to worry about this Chinese AI thing? What was she potentially worried about? And I mean, I'm not trying to put your mom on blast. It's more, it's just, it's symptomatic of, this isn't just hype, there's almost like worry and concern. Yeah, totally. And I think for me, as someone who's like followed it from the beginning, I was totally prepared for everything that was about to happen over the weekend.

3:05It became very clear that this was going to have an impact that is going to escape the AI nerd community and going to have like a real impact on the market. and I think that's why I heard about it from her. I don't watch CNN. She watches a lot of CNN and I'm sure that she just turned on the TV in the morning and it's one of those screens that's showing you all the stocks and everything is red and the news anchors are like, China, AI, America, World War III? And she was like, uh-oh, I'll call my son who knows computers. Wait, but let me take a shot at explaining why people are freaking out because it is true that this is a Chinese-developed large language model.

3:49The model is called R1, the one that they released. And then the ChatGPT competitor is just called DeepSeek, and you can use it on your phone. So it's the top app in the app store. There's the sort of normal like, oh, Americans are using Chinese tech. We're worried about that. But the big thing is that this was developed using old chips. and fewer chips than normally are used. And they spent somewhere around like$5.6 million or something like that on training, which is a number that, as you explain in your article, we should be skeptical of. But the argument that the people who are freaking out are saying is OpenAI and NVIDIA and all American tech companies have spent a ton of money training their AI models.

4:46And by a ton of money, I mean billions and billions and billions of dollars. Here is a Chinese company that comes along and spends, frankly, like$5.6 million. If you take that at face value, which I don't know if you can, that's like peanuts for any AI startup. That's essentially no money. It's performing better than OpenAI's most advanced publicly released model. I think it's the... It's O1 is OpenAI's most advanced model, I believe. And it's free. It's released for free. And they did it using old chips because China cannot get the newest NVIDIA chips because of export controls. So the narrative in people's heads is like China beat the US with less money and old technology.

5:42What the fuck? Is that right, Emmanuel? Is that the argument for why people are freaking out? Yeah, that's totally accurate. And the single headline that dominated Monday morning and what sent the markets into a frenzy is that this impacted NVIDIA's value. Ever since the AI boom took over the world, NVIDIA has been the primary, I mean, you could argue TSMC, which is like further up the supply chain or further down the supply chain, further up the supply chain and chip manufacturing. But NVIDIA has the key designs and chips that are powering this AI revolution. And Meta, Google, Amazon, OpenAI, Microsoft, all these companies, when they're building AI, they are paying NVIDIA for massive amounts of chips in order to train their AI.

6:50So Elon Musk very proudly announced last year that XAI, which is X's or Tesla's or Elon Musk's AI company, has a cluster of 100 ,000 NVIDIA chips. And those clusters just keep getting bigger. And at the moment, all these companies are kind of eyeballing their path to like, can we make a cluster with a million chips? Because that's going to give us a much stronger AI model. And that's been the paradigm we've been in for the past three to four years. More chips, more compute, more training data equals better AI. Whoever has the best and most is going to win the AI race. And if you believe what these people say about how important AI is going to be, you're going to dominate the world.

7:38And more advanced chips, crucially, because NVIDIA will release new ones every single time. And the idea is the American companies will be getting those. So it's not just quantity or expenditure. It's, as you say, the access to the latest and greatest, allegedly greatest. Yeah. And at the highest level, these companies are competing for like, oh, NVIDIA just announced a new design and they're competing like, okay, well, put me down for a million chips, you know, and they're finding out who's going to get them first because it's so important to have these chips in order to win the race. I believe they've called this like a digital moat or something where it's like, US companies will get the awesome chips and China will get like old tech.

8:15And that will naturally just keep American tech companies ahead of Chinese tech companies. If I can give away a crude analogy to explain it, it's like the Americans are only, they get the PS5s. The Chinese companies can only have PS3s, maybe even a PS2, but they made a really sick game for PS3 and PS2. And everybody's like, damn, maybe you just don't need the latest console. I think that's it. I think that's pretty much it. Like, imagine suddenly, you know, a Chinese game developer releases a PS2 game that looks like a PS5 game. Everybody's like, hey, what are we doing wasting all this money? You know?

8:56Right. And they're like, we made this in a weekend. Yeah. With like some spare parts. I think we'll get back to the stocks in a second. But who are some of the big names reacting and how are they reacting? I'm thinking of A16Z. Are they, what are they saying? Yeah, everybody's reacting. I mean, obviously before it became world news, it was huge in the AI space. So everybody had a take. Mark Andreessen, who's one of the co-founders of A16Z, he said, I think it's a, to be seen if he's correct, but I totally understand the sentiment. He called it AI's Sputnik moment. and that is referring to the Russian satellite, which got ahead of the United States in the space race and it was kind of an oh shit moment for the United States to say we're in a more competitive space with Russia than we realized about space exploration and it's the Cold War, so it's the weaponization of space and all that.

10:02And that kind of accelerated the US space program. And I think that is the analogy he's making. It's a wake-up call for American companies to realize that these sanctions that America, or sorry, sanctions is probably not the right word. These export restrictions on AI chips that the Biden administration has put in place for the reason of hobbling their ability to compete in this AI race, right? That's not going to cut it, right? And the idea that we just have a massive amount of capital and we're just going to throw it at these AI problems with these bigger and bigger clusters, which has... People expressed doubt about the technology and theory behind that starting last year.

10:50But this is tangible proof that that is not going to cut it. The US needs to be competitive in more areas than just capital and access to chips. We have to, not we, the U.S. has to have innovative research. They have to have the best talent. They have to realize that the Chinese universities and research that are happening in China are not to be dismissed. These are real competitors that can make real progress that leapfrogs the American AI companies despite all the advantages that they have. so the another take that i've seen is the sort of uh china surveillance and china um censorship take which is like if this if this or you know there's going to be other models that come out very soon like as as you've reported on and as we'll talk about like these models will get leapfrogged this will happen over and over again but like at the moment there's a bunch of people using DeepSeek.

12:00It's the top app in the App Store. People are showing that it's censored in the way that you would expect Chinese technology to be censored. The obvious example is if you ask about Tiananmen Square, it will not answer. And then people also saying, hey, this is Chinese propaganda, Chinese surveillance, blah, blah, blah. I'm not sweeping that under the rug. I'm just saying that's an argument that people are talking about. You raised the very good point that ChatGPT is also censored in several ways. Can you just talk a little bit about that and that aspect of the concern? Yeah, I think the way I think about it is to go back to one of the earliest stories that I did at 4.4 Media, which is about this guy named Eric Hartford, who takes these open weight models like Mehta's Llama, and because they're open weights and he can modify them, he creates uncensored versions of these AI models.

13:12And Joe, I know this always gets to you that they use this terminology, but, you know, OpenAI, they would consider a censored and woke model, right? Because it won't tell you how to make a bomb, and it won't tell you about the Hunter Biden laptop. I don't know if that one's actually true, but it restricts what it says about the election. And something that we bring up all the time is that it kind of refuses to engage with anything remotely sexual. Things like this. and the argument that the people who make the uncensored models have always made, and this is an argument that Marc Andreessen has made as well, and he has funded Eric Hartford, and he has funded these group of small developers who create uncensored models because they argued that, okay, maybe the way that OpenAI censors its models doesn't bother you because you happen to align with their ideology because you live in America and you're American and you don't want the AI to engage with these questions.

14:24But if we accept that paradigm, should another country create another dominant AI model and they have their own censorship based on their culture that we don't agree with, we have to live under that, Right? So that is the argument for the uncensored model. And this moment, this deep seek moment of it, showing people that it is possible that we will live in a world where the AI that is dominant is made in China, really freaks people out because we then have to accept, you know, Chinese restrictions and Chinese culture and the way that impacts their AI model, right? It's very similar to the TikTok argument.

15:07It's like social media is bad, but we accept it. But at the moment it becomes a social media that is dominant as Chinese, people really freak out. And it's the same dynamic here. Facebook does bad things. TikTok does bad things. We are more willing to accept the bad things that Facebook does because it's our social media company as opposed to the Chinese social media company. And you're seeing the same thing here. And I think in a way, though, I have a lot of criticisms of the uncensored model argument and community. I think this really proved their point where if you allow for censorship or restrictions on an AI model, just like as a theory of how AI should be managed, then you also have to accept it when it comes from a different country.

15:56And that obviously really bothers people. Right. And it should be mentioned quickly that DeepSeek is an open weights model, meaning you can tweak how it works and you can run it locally. Although we don't have all the information on how it was trained, so it's not fully open source. yeah it seems really trivial to be fair to to deep seek or the people who i mean are using it and defending it it seems really trivial to get around this tenement square uh restriction you just have to run it locally i've seen people use different languages to get around the restrictions it's like this is a form of jailbreaking but i saw someone uh engage with deep seek in hebrew and it just answered all the questions.

16:42I saw someone get around it by using Leetspeak, where you replace letters with numbers, and then it also engaged with the questions. That's my primary language. I was going to say it's a manual language. It's a restriction that is there. But the idea that the problem with this particular model of DeepSeek is that it's restricting information that the Chinese government doesn't want people to have sort of i don't know unserious because it's it's open weights and you can get around it and yeah um i think with all of these takes flying oh and i'll just say i think perplexity has actually added support for it as well where you can now interact with this model through perplexity and i you know i presume they're running it locally in their own ways right not locally to the user obviously so there's all of these takes flying around but you kind of have one main one in the article and i'm referring to you know even though this is a pretty significant event is it kind of impossible to tell what's going to happen with the ai industry based on it like what's all your main takeaway from this i think my main takeaway is that the ai space is moving very fast and is developing at the rate of rapid scientific discovery.

18:05And that is the schedule that it's on. And that is very hard to predict. We don't know where it's all going to end up. Some people who are kind of cheering and waiting for the AI bubble to burst really hope that this downturn in the market that is instigated by NVIDIA and AI companies is going to... This is the AI bubble popping, and now it's all going to go away. I'm skeptical of that. Maybe that happens later. I don't know. I don't make predictions. If I was able to make predictions about the stock market, I wouldn't be talking to you. I'd be on my yacht.

18:40But... So it's like, AI is moving at the pace of scientific discovery, and the markets are moving emotionally. It's like we're seeing an emotional reaction in the market. I am doubtful that this will hold. I am doubtful that this is terrible for NVIDIA in the long term because they are still the dominant hardware player in this space. It is possible that later on China, especially because we're imposing these export restrictions on them, will develop their own supply chain for the chips that does devalue NVIDIA. But at the moment, NVIDIA is the main player. And one obvious argument about that is, okay, DeepSeek has discovered these new, far more efficient reinforcement learning strategies that allow them to develop models that are better for less.

19:42but the research is out there nvidia is going to figure this out facebook is going to figure this out there was a report in a wall street journal i think that uh meta has several war rooms just trying to reverse engineer recreate this stuff but the american companies will catch up and then they'll have these new efficiencies and the access to chips and now we'll just make them better and more efficient you know um so and i also think you know we talked about leapfrogging um meta gets ahead google gets ahead these companies kind of vie for the top spot um developing uh depending on whose research is latest and best for a while their mistral which is this french ai company was really far ahead uh for a minute uh and people didn't freak out about that i think because it's a European company and the markets didn't freak out about that.

20:42They weren't like, oh my God, OpenAI is doomed and Meta is doomed because this French company suddenly has a better model. When it's a Chinese company, people, I think, react emotionally because China is perceived as this adversary of the US and the West. And that's why I think people are freaking out. And I think it's very plausible that a few weeks from now, OpenAI will release a new model. Everybody will agree that this is the latest and best and we'll go back to where we were which is rah, rah, rah America and the future controlling AI. I'm going to go to my war room to determine what this means.

21:22I think you're in it right now. It's your garage. I'm always in it 24-7. Alright, we'll leave that there and we'll be right back after the break with another story. We'll be back after this.

21:50It's a new year, so it's time to make some moves. Maybe you start a new business, launch a new product line, or organize your existing operations. For us, this meant a total refresh of all 404 Media's merch, which we could do only with Shopify. Make this year different and better with Shopify. The best time to start your business is right now. Shopify makes it easy to create your brand, open for business, and get your first sale. It couldn't be easier. Shopify has thousands of customizable templates, no coding or design skills required. All you need to do is drag and drop. Shopify's powerful social media tools let you connect all your channels and create shoppable posts to help you sell your product everywhere people scroll.

22:33With Shopify, it's easy to manage your growing business, ship your products, collect taxes, and manage payments from a single portal so you can focus on the important stuff, like growing your business. Don't kick yourself when you hear this again in a year because you didn't do anything now. With Shopify, your first sale is closer than you think. Established in 2025 has a nice ring to it, doesn't it? Sign up for your $1 per month trial period at shopify.com slash media all lowercase. Go to shopify.com slash media to start selling with Shopify today. Shopify.com slash media. Support for the 404 Media Podcast comes from Coda.

23:16Picture this. You're starting a new project at work and your vision is clear. You know exactly who's doing what and where to find the data you need to do your part. Everything is organized and you don't have to go searching for everything. That's because everything your team needs, from project trackers and OKRs to documents and spreadsheets, lives in one tab, all in Coda. Coda's collaborative all-in-one workspace gives you the flexibility of docs, the structure of spreadsheets, the power of applications, and the intelligence of AI in one easy-to-organize tab. Coda connects tools and teams so that you spend less time switching gears.

23:54We're a small team, but we do everything for the whole business. With Coda, we're able to combine what would have once taken lots of different siloed pieces of software and puts them in one place where the four of us can see everything, track our progress, and make moves to help our business. More than 50 ,000 teams trust Coda to keep them more aligned and focused. To try for yourself, go to coda.io slash 404media today and get started for free. That's coda.io slash 404media to get started for free. coda.io slash 404media. Today's episode is sponsored by BetterHelp. In 2025, maybe you're ready for a plot twist, or maybe there's a part of your story you've been wanting to revise.

24:38Life isn't about resolutions that fade by February. It's about picking up the pen and becoming the author of your own journey. And therapy, it's like having an editorial partner, helping you craft new chapters, set boundaries, and become the best version of yourself. BetterHelp makes getting that support easy. With over 30 ,000 credentialed therapists and a fully online platform, therapy is affordable, convenient, and designed to fit your schedule. Plus, you can switch therapists anytime at no extra cost. Therapy has certainly helped me find a better version of myself, and with BetterHelp, we've found it's convenient, affordable, and flexible for our busy schedules.

25:17Write your story with BetterHelp. Visit betterhelp.com slash 404 media today to get 10 % off your first month. That's betterhelp.com slash 404 media.

25:43All right, and we are back. And then Sam is now with us. We're going to talk about a story that she wrote. The headline is, memos to federal employees were written by people with ties to Project 2025 metadata shows. A lot going on in that headline. Maybe let's just start with, what are some of these memos that were sent by the Office of Personnel Management, which for those people who don't know is basically the u.s government's big hr department that's how i think of it anyway yeah it's honestly i will admit i had not heard of this office before this week it's like there are so many like literal little like federal offices um that i think a lot of people are learning about for the first time because they're in crisis thanks to the only reason anyone would ever know about opm was it got hacked seven years ago.

26:39Yeah. Yeah, that's the only reason anyone would have ever heard of this office, I feel. Yeah. But they're sending these things which are impacting a ton of people. And what are they? Like, what are they telling people to do or not do, I guess? I mean, so they're sending a bunch of memos, which memos are public on their website. So anyone can go and look at them and download them as PDFs. But they're sent out to federal workers and federal departments. And obviously, this is a big thing right now because Trump is, you know, working on firing a lot of federal workers in order to, I don't know, like, sow chaos and still people who are loyal to him doing all these kind of like loyalty checks with federal workers and seeing if they, you know, gutting like DEI, obviously, is a big one that's been in the news lately.

27:38So these memos go out and the memos say, you know, it's, I can just, I can read you like some of the titles. They're pretty self-explanatory, but some of the ones that we looked at just from this week were, one was called Guidance on Presidential Memorandum Returned to In-Person Work, which one of the things that Trump has done is said, you know, all federal workers have to return to the office. Whereas before they were allowed to work remotely in a lot of cases, now everyone who's a federal worker has to be in the office. And that's what that memo is about. And then there's others that are about, like, obviously diversity, equity, and inclusion programs.

28:18There are, you know, it's just like a... A lot of them are, like, kind of dry. But then some of them are pretty, you know, unprecedented slash spicy in general. Well, there's a hiring freeze as well, right? Yeah, there's a hiring freeze one for, like, federal workers. which is obviously not something that happens every day. I don't know if it's happened, you know, in the last four years, certainly. So yeah, they're going out to federal workers basically and saying, this is what you need to then disseminate to your department. Yeah. Whatever it is. So we'll get to the metadata in a bit and the authors and that sort of thing.

28:59But there's been, I would say, a ton of chatter around these memos. And, you know, there's a lot of posting on social media, I would say. There's all of these Reddit posts. You know, some are going to be more believable than others. Of course, you have to take everything a pinch of salt or, you know, go and verify it as we do in some cases that we can. And, I mean, even just beyond sort of these memos, there are like multiple journalists tweeting that they've never seen sort of this many leaks in a way. And I know these memos are public, but it's just sort of associated with the massive reaction that federal employees are having.

29:40It's like a lot of them, it seems, are leaking to other journalists as well. But my question for you is, how are federal employees reacting to these memos? Like, are you seeing a lot of pissed off people online? Yeah, I mean, we've, I think all of us at this point here at 404 at this point have had folks reach out to us who are federal workers. There's just, there's a ton of them, you know, like they're, and they're across all different sectors, all different industries working on lots of different things. So, you know, they're, like you said, they're hitting up journalists and saying, this is what I'm seeing.

30:14And they're freaked out, obviously. It's like, it's a big deal. They're worried about their jobs. They're being forced to do stuff that they don't agree with. um you know like we'll talk about later like scrubbing websites and things like that uh that have to do with like um you know diversity and inclusion and those sorts of things so yeah they're they're worried um but at the same time it's like there's just not a ton that they can do um from the inside they're federal workers so if they want to like do some kind of protest it's a federal crime um i mean i assume but yeah it's like they're they're very disheartened it seems um about the whole entire situation because they're um you know they're being told to do stuff that they morally um ethically don't don't agree with as people so yeah i don't think everybody's i mean people will be mad understandably depending on your own circumstances about the need to return into office or something like that but broadly many people are simply mad because they don't agree with the incoming administration for various reasons yeah and like not agreeing and they're like not like supporting trump at this point it seems like that's a fireable offense you know which that is very unprecedented um that's you know it's something that um we're seeing we're going to see more of in the coming days i think but like um not like being gung-ho about what he's doing seems to be something that puts you on the chopping block as a federal worker now, which is horrifying.

31:50Yeah. I think one of the tricky things is that the administration has shut down effectively so many different parts of the government and has also made it very difficult for government employees to talk to outside contractors. and for anyone who's not familiar like many many many many like technically third party contractors actually are basically government employees this is just like how the government works and it's like there is so much happening because essentially anyone who works for any of these massive agencies can is being affected in some way shape or form and You know, these are literally millions of people.

32:39And so trying to tell that story is very, very difficult. For us as a small team, but also for like the media in general. And, you know, I think you can create like a coherent narrative of what is happening, where it's, you know, the Trump administration is just like pausing the government and it's having all these knock-on effects. But getting into these specific effects is like essentially impossible because there are so, so many. yeah the scale is almost unfathomable in a way and you can't really communicate it in a single article necessarily or whatever medium you're using just because we're talking about opm i guess we'll touch on this in the subscribers only section as well but considering opm they have created these sort of tip lines right jason where people can email in to i mean basically rat on their co-workers when it comes to them running certain DEI programs.