Founder Eric Steinberger on Magic’s Counterintuitive Approach to Pursuing AGI

0:00The thing that remains to be solved is general domain, long horizon, reliability, and I think you need in friend's time compute test and compute for that. When you try to prove a new theorem in math or when you're writing a large software program or when you're writing an essay of reasonable complexity, you usually wouldn't write a token by token. You'd want to think quite hard about some of those tokens. and finding ways to spend not 1x or 2x or 10x but a million x the resources on that token in a productive way I think it's really important that is probably the last big problem

1:00Hi and welcome to Training Data. I'm delighted to share today's episode with Eric Steinberger, founder and CEO of Magic. Eric has an epic backstory as a researcher, having caught the attention of Noam Brown and becoming one of his research collaborators while still a student in high school. Eric is known for his exquisite research taste as well as his big ambition to build an AI software engineer. We're excited to ask Eric about what it takes to build a full stack company in AI, his ambitions for magic, and what separates a good AI researcher from a legendary one. Eric, welcome to the show. Thank you so much for joining us.

1:37Thank you for having me, Sonny. Okay, so let's start with who's Eric? You're a Vienna -born Wundergind, whose early passion for math turned into a... I think what you described as a full -fledged obsession with AI by age 14. Take us back to age 14, Eric. Like, what were you up to? How did you become so obsessed with AI. Thank you, Zanya. I think I just have my midlife crisis when I was 14. And I was just looking for something meaningful to do. Spent about a year looking at physics, math, bio, medicine, just anything really that seemed valuable to the world. And at some point bumped into just simply the idea of AI.

2:15It hadn't sort of occurred to me until then. And if you could just build a system, a computer system, and it could do all this other stuff for me. Like, great, I don't have to decide. So it felt like my decision paralysis was sort of resolved. It was this weird moment where I could just see the next 30 years of my life unfold in front of me. And I was like, okay, this is clearly what's going to happen. Like I have to do this. And it was quite nice. I like predictability. So it was great to like, no, what the world will look like. And you started loving math. Like, why AI then? I think I'm naturally attracted to math.

2:51It's just what my brain sort of gravitates to. AI just seems useful. The thing that's most important to me is just what is useful for humanity in the world and math is nice but not useful at some point like you know 17 dimensional spheres are probably not going to be the best career choice if you want to be useful so It seemed like something that I could get good at but also just the most important thing ever. And so it was a very clear choice. AI just just like, it was clear 10 years ago. It just, it wasn't close. And now it's close and clear. Can you tell the story of how you got to fair? I think it is such an epic story.

3:32Sure. I mean, so when I started at 14, I didn't really know how to program. I didn't get into programming out of curiosity about computers. as I just wanted to solve AI, basically. So after a couple years of just warming up on my own, I reached out to one of David Silver's PhD students, who was the AlphaGo DeepMind co -founder. And this PhD student, I guess at that point, he was a graduate and sort of worked at DeepMind. I asked him if you could spend a year with me just every two weeks bashing my work, trying to do, so super speed up, mini PhD -like experience where I could just learn how to do research.

4:17I sent him this giant email, you could print it out, I don't know how many pages it would be, but it'd be a lot of pages. I was basically just saying, I want to build this algorithm you made in your PhD. I'm going to beat this algorithm you made in your PhD. Here's a list of 10 ideas. I don't know if they're going to work and I think I need your help to figure that out. And then over a year we eventually got there and he was like as they say Harness, Yannis was kind enough to just patch me every two weeks roughly. And yeah, it was brutal dude. Like, I was like, he always do the standard, you know?

4:52Don't be nice just because I've been high school. Yeah, I was in high school. And then when we were done, I just graduated in high school when I finished. So this is when I finished the project that I was trying to get to with this. And then Noam Brown, who is obviously one of the best RL researchers in the world, reached out. Because he worked on something similar, turns out. And we sort of had some ideas that were very similar. And some ideas that were a little different. And so we just published this and Eri Stoud. And then I got to work with Noam Brown for two years, which was great. And then that continued.

5:25And so I got back to the other year. And you were a high schooler. He was Noam Brown. Well, I mean he published a paper called Deep Counterfactual Regroup Minimization and I published a paper called Single Deep Counterfactual Regroup Minimization and Mine Beat Hissed by a little bit and... So you want to know him? Brown is a high schooler. I think I just graduated and I also took him like three months to write this paper and it took me a couple years. But yeah, I mean slightly. I'm sure he would have gone up with this the next day. but the gap between the two things. But it was just, yeah, it was a obsession.

6:00It's the right word. I do things like 100%. And yeah, so that was a little fun. Keep working in our all with No Brown for a while. And yeah, so that's how I got to fair. No Brown worked at fair at the time, and he reached out. I was actually at university then. And basically just work part time as a researcher at fair file studying. Anyway, that was it. That's awesome. It was a lot of fun. No, it was great. Like the brainstorm ping pong sessions with no brown dude. There's like nothing like this. Or you would just think there's like this problem. And it sort of like, you know, maybe you would like start a six -month research for like no.

6:42No, I mean, I would get on a call and it was like, we just discuss it and it's done. I love that. I love that. What makes him so great as a researcher? I think it's a number of things. As a researcher, generally, from more from a meta -level, he is fantastic at picking the right problems and then spending a long time just grinding to make it better and better and better and better. So he's very good at the whole like compounding thing in research. Also making bets that aren't obviously the right bets when he makes them, because he makes them earlier, I suppose, and making them say differently.

7:16So you're generally very good at picking problems and then attacking them consistently. It's also just very smart, I guess that helps. It works really hard. He used to do 100 -hour week storing his PhD. I don't know if he still does them, but he used to work really, really hard during his PhD. I imagine he still is. Okay, so no more range for you to become a Researcher at Fair while you were still a university student. Yeah, I was from. I was in my first semester, I think, or something. So you were juggling that, you were juggling being a collaborator to know him at the fair, and then you became obsessed with yet another problem, climate change, and I actually started an NGO that is incredibly popular, climate science.

7:56So you just didn't have enough on your plate. That was actually too crazy, that's when I dropped out, I was like this is crazy, this is too much. I can do two things, I cannot do three, it was like my conclusion after I did three months of that, that was terrible. That was fucking awful. Doing all three things. Because you just can't do well at three things. I mean, Elon can, but maybe I'll learn it in 10 years. But I couldn't at the time. So I dropped out at the time. But yeah, I started NGO. I generally think charity stuff is awesome and hugely underappreciated. It's super high status to start a startup.

8:36But I think it should be equally cool to start a charity. You're like helping the world in other ways. And so, yeah, I mean, we started it as a, it's a nonprofit, but we started it like a startup. People were working insanely hard. We had clear objectives. It was, you know, a software product effectively. It was much more similar to a startup with the exception that there was no money and no money out. I can flip. I mean, just very weird, but the, yeah, it's mostly volunteer driven, or I guess is, I just no longer run it. Yeah. Yeah, that was an interesting experience. You'd think I would learn transfer a lot from running a Quote Enquad Company to running a Quote Enquad Company now, but they are so different that I could like, there was no transfer at all between climate science and magic.

9:21Like a thousand volunteer, 20 hardcore engineers. No money at all. I can't tell you how much we raise because it's not enough yet, but, you know, giant. It's like completely different in every imaginable way. So, but yeah, it's not fun. So Eric, climate science became an incredibly successful nonprofit. I wasn't just sending nonprofits. What made you decide to kind of hand over the reins and hand over the torch on that and then go start a company in AI? I just thought AGI was further away when we started it at all. I wouldn't ever have started anything else if I thought AGI was so close. And once I realized it is, there was just like no other.

10:01I mean, my initial thing was always AI. That's what I did as a kid. I care about various issues in the world, but it's none of them are my unique calling in any way. I just, you know, I hopefully have been in a position to donate a bunch of money on whatever, but the thing I care about fundamentally is AGI, and it was like, oh damn it, this is not 20 years away. So I have been running around with this AGI to -do list, which is somewhat of a meme, uh, uh, uh, internally. Because it's sort of like, just like going through it. Then we're trying like to fix all these problems. You have seen it. Yeah.

10:37So we showed it to you. Um, and, uh, I've been running around with a version of this. There's actually like a 2017 or so. I was still in high school. I got it. It's I don't know why but some conference invited me to present my AGI to the list. It was wrong at the time. I was also, I was also sure it was wrong. But, um, at some point, so that was one thing I just couldn't at all figure out. And I don't like Blue Sky Research in the sense of like just staring at a wall and trying to like figure out what the right question is. I really like to have the question and then look for the right answer when starting an intense project because you need to know which direction you run into really plan for it.

11:13And many things seem clear, but it seemed completely unclear how to make these models reason in the general domain. And that became more clear with language models, especially code models. And so, yeah, when I saw just some of these early, some of the early results in this basic, I was like, okay, I know all this stuff from the RL world, I have a bunch of other thoughts. This seems great, like we should just take LMS and make them do the RL stuff. It's a very simple kind of proposal, but I think that's sort of where, I mean, it makes a lot of sense. RL has been going this for 10 years. it works in restricted domains.

11:56If you can make something work in 20 restricted domains, and you have something else that works in a general domain, if you can combine them, maybe you'll get both the X and the Y axis, and then you have your beautiful top right corner of the matrix. It seems like it seems persuable. When something becomes, when something as important as AGI becomes an actually executable to do, it's obviously there are still things to figure out, details of the algorithms, how do you make it efficient, etc, etc. we do everything at all. Many, many things to figure out, but the direction was clear. Yeah. So it seemed like the right moment.

12:32Okay, we're going to circle back to your AGI to do list later because I'm curious about it. Sure. I want to brag about you for minutes because it might be wrong still. I don't know, until we have AGI, it is a hypothetical AGI to do list, but we're frying. I think the research field is tracking pretty closely to your to do list. I want to brag about you for a minute. I think you've been incredibly humble about your background, but you know, as a high school student, you did catch Noam Brown's eye and as, you know, as one of his colleagues at Fair, you became one of his top collaborators, not even just one of many, because there's such talented people that work there, but you're one of his top collaborators.

13:10And, you know, when I speak to folks that know you, they just say extraordinary things about your capabilities as a researcher, your creativity, your work ethic, as far as I can tell you work non -stop. Thank you, texted me at 2am in preparation for this podcast. So I think it's safe to say. No, thank you silent mode. Anyways, I think it's safe to say that you are one of the brightest minds of the current research generation already and will certainly be one of the legends that people talk about for the next decade. And so with that in mind, I'd love to ask you some questions of advice for aspiring researchers.

13:43And so maybe first off, you did it all from a very untraditional background. How did you do it and like do you think that's like what advice would you have to others in your shoes? I can only really speak for it a sort of profile of goals and person I am. I think I was lucky in the sense that I knew very very early with Martinez as we said exactly what I wanted to do with my life. I had no doubt at all and then not certainly can be paralyzing to a lot of people. I also had a very clear sense that I did not at all have a plan B. Like there was no water, a path in life that I would have been even like remotely above the neutral line on.

14:22It had to be build AGI, everything else is completely irrelevant. So, you know, I understand, so for many people like, you know, a well -paying job at Google is a great achievement. I mean, I would, if it's on AGI, it's fine. But you get what I mean, like I just knew that there was nothing else I could do and like be fulfilled in life I look back when I'm 90 and be happy So in a way like burning the boats very very early gives you the opportunity to Just be be like do things that you do otherwise do 10 years later, which again even like I sucked at the beginning It took me two months to understand the first paper.

14:59I tried to understand and I was terrible at programming for a long time. But when you're a teenager, you're like a decent researcher, you don't have to be great. That gets you things like a great mentor who then bachelors you for a year, which was very, very helpful. And then you get better, and you're still young, so you serve your brain shapes more easily maybe. I don't know. So I feel like I benefited a lot from being early. But within that, I'd say just like go for the end goal immediately, doing anything sort of like, oh I'm gonna do a PhD because I need a PhD to get a jub -ass -hole bullshit.

15:36Like you don't, it's just completely bullshit. The other thing is like writing five -page emails to people actually works. Writing like, I get a lot of these like two paragraph meh things now and grateful like get emails but don't, I understand now why people think this stuff doesn't work. It certainly does when you're like, here is how I'm gonna beat your algorithm. Please help me. Five pages, at least in my experience, every single time anyone I I want to help from in this way was very helpful. So I suppose be proactive in seeking like the best people in the world to in a time -efficient manner just distilled their brain into yours and show them that you can make use of that.

16:18If you if you if you tell someone who's very good effectively, hey, I'm gonna make good use of this. If you want to coach someone, I would love to be that person. They'll usually do it. They won't do it for 10 people, but if they do it for one or two, that's enough. You just have to win that seat, I guess. So that's been really helpful in my experience. Also, just not shying away from learning new things. Like, again, I didn't get into programming because I'm curious about computers. I'm not very curious about computers. I just like AI and that computers are the thing that are necessary to do.

16:47So it's fine. I enjoy programming now. It's great. But I wouldn't have gotten into it, I think, if it wasn't But still, you get into it. So don't be shy. We need to be a lot of people who don't know how you'd implement an LLM. And it's kind of crazy to me if you're a researcher, and you couldn't implement charting or whatever. Like it's just insane. So really understanding the whole stack, going down to, but not bottom up, really top down. Here's the thing I care about. This is the problem I want to solve. What do I need? What do I need? And then all the way down. And I know like much more competent people at kernel programming and hardware design or whatever than I could ever dream up to be.

17:27But I understand enough of it to do better work at the top of the stack than I could if I didn't.

17:36So I think fundamentally you need to understand the domain you work in. It's also really good to just read everything. Like I used to read, I don't know, I don't have a precise number, but just every paper, I could, every paper with C basically. And eventually you get so fast at it that you can, like, that's feasible. And you, like, build a database in your head of, like, oh, this is similar to this thing. This is sort of like, this is sort of my eye -opening moment. Or Bill Gates has this interview, like, oh, yeah, you've learned enough things. They're all, like, similar to each other. So it's not linear.

18:03It gets easier. And at that point, I was like, I should read every paper. And so thanks for the advice, Bill. Obviously, it was, like, through a video. I never met him. But the, so I just started reading every paper. And that's really, really helpful because a lot of the best ideas that we had that work really well now at magic. like we're enabled by random things that are like, oh, it would never work without this random thing that I would have to have come up with in tandem. But because I have this database in my head, I can go like, oh yeah, like this. And then so often, like one good idea is enabled by three other ideas that other stuff come up with.

18:35And so it's always just like this composition of stuff. So having a large database is really helpful. Yeah, and then just never stop. Like never, never stop. It takes like ages to do good stuff, to do good work. at any point. I was actually one moment with Johannes, the Deep Rind Research Scientist who mentored me for a year in high school, where we had a version of the algorithm that wasn't very good. It was all right and we're thinking that guys should be published this, like we were both not really happy about it, and he was like close to giving up on me. I was like, well you know, like maybe this is just not gonna work, like I wouldn't want to publish this, and so that was like, dude, fuck you, I'm just gonna get this done, and then we got it done like a one or two later.

19:18And so I think that I remember going on a walk after this and just being like, can I do this? Yeah, I don't know if I can do this. But there is no other option. So I just better get it done. And then I went back home and I started programming again. It was still like sad that day, but the next day was fine again, and they just keep going. So I think you have to, I think that was a pretty formative experience because I actually wasn't sure if I could do it. And then we just did it like super soon after. So I really haven't felt that in the same level of doubt and pressure since then, which has enabled, it's actually beneficial.

19:54You have to be realistic, but if you stop, yeah, I mean, so anyway. So I think those would be the main things. Also be really fucking honest about what you suck at to yourself, because otherwise you're never going to get good at it. like you need to search for the bad things. And instead of like trying, actually, I think like as a researcher, betting on your strengths is good only to the extent that you don't have necessary conditions that are completely missing. Like you can't bet on your strengths if they're not enabled. Is again, like back to the engineering thing, for example. So yeah, I don't know, I'm rambling, but stuff like that.

20:34That's great. That is such a fascinating glimpse since the inner mind of what it takes to be a great researcher, and behind all the glamour of training large models. And so thank you for providing that peak. I'm really glad that you mentioned reading every paper voraciously and having this database in your head. Because one thing I've heard from your collaborators is that your superpower is understanding and absorbing new research. And so I'm curious, do you agree? Do you think that is your superpower as a researcher? What traits do you think have made you such an exceptional researcher. So I think initially in DRL work I did, it was synthesis where I would read every paper and I would go like this thing plus this thing plus that thing with this modification.

21:14I think that's what they would mean. That yes, it was definitely very helpful. I think is a good way to do research. Generally there is enough work for synthesis to be a successful strategy. I guess to an extent it's still that I tried very hard after it. This is actually, in reading, it's like you bring this up. I realized this and I tried very hard to get better at leaps like coming up with totally alien crap that just there's no reference for it at all. And because ultimately like, so if you take like the transformer, for example, like attention existed, the idea of stacking a bunch of LSTM blocks existed and you just have to remove the idea of recurrence, really like that and like a bunch of couple other things that were necessary, right?

21:55Residual streams like the residual update and transformer existence from existed from ResNet. So it's like, it's synthesis, but there is an amount of leap in there to make it all work. It's a little more complex than just taking components and putting them together. You need to come up with new things too. Like, you know, the normalization and the head, square square, the ad rejection, correct? But anyway, everyone now knows this. But roughly, like you should do some organization. So there are some new ideas in there that really help make it work. But it's still a larger amount of synthesis. So I suppose like most good ideas are synthesis, but they're always some like in the best ideas are some leaps and I'm trying to get better at those But still it's mostly I guess like take five things and throw away the stuff that doesn't work in them Make the things work and Don't forget about yeah, I think I think some Some stuff needs leaps but but yeah, I guess like no, that's a recipe like take all of them make them super efficient block context giant, throw our L on it, make it all work together.

22:58It's still mostly synthesis, I guess you're right. Who do you admire most in the research world and like what do you think those folks superpowers are? Shizur, no Shizur. He is. He is. What is his superpower? I guess to an extent synthesis. He is just the best at synthesis. He is also great at everything in the stack. He has no weakness really. He can implement the whole thing on his own if he had to run it. He sees the future in a way that's very unconstrained and I think I run sort of crediting a number of the labs for scaling laws. was this guy made a presentation where he was zipping through essays or a completion of whatever written by like models of various scale.

23:52I was like this is 100 million parameter model. This is a 300 million parameter model. This is a billion parameter model. This is a 5 billion parameter model. This is on YouTube summer. It's hilarious. I knew he could like, well, what if we make this bigger? It's sort of presenting it this hilarious way. And I was also super scientific about it. I think no, this generally just, if I had to put it, it's very intuitive. I think there are a lot of labs and researchers are sort of, I think this is not a bad thing. It's very good. A very eval is driven very mechanical, right? It's sort of very empirical in a way.

24:26No, I'm sort of just nose. He's like, God, this would work. And then it works. So I think that's a superpower. It's just extremely great synthesis. Yes, the larger data base, because he's been around for so long. You just you literally knows everything. I mean he bent the half of the stuff that everyone's doing now There's no one to convince I'd say There are a number of other people I guess Just you you shouldn't feel like out of all the people who are sort of the OGs of deep learning I think you could have hinted there's but deserves by part of most credit just as he like when through all the bashing when And when it was like, oh, it is will never work.

25:08It's like trading tiny, tiny, tiny, tiny, tiny things. They're like, it's will never work. And he's not gonna stuck with it. I think that's that level of grit and belief and something that is now obviously working deserves a huge amount of credit whether Capsuleck's work or not, whatever. Yeah. He, like, you know, it's incredible to come to something like the conclusions that the world is at now. And if you look at some of the older papers, a lot of the ideas that are important now were in there already. So that's important. And I think he just deserves a ton of credit. And Noam Brown had the Army of Noam's.