The Token Budget Problem Nobody Is Talking About (Matan Grinberg, Co-Founder & CEO of Factory)

24:31Matan Grinberg:Cursor has that distribution and has that data, but doesn't have the data centers and all the infrastructure and the like cash to make that work so that's great for them they also you know fantastic valuation great for all of the teams so huge congrats to them for us it's also helpful because there needs to be an independent kind of software development factory provider that is not tied to one model in particular and that was the only other big one out there really that we were seeing in the market and so it kind of cleared the way for us to some degree and so So the great thing is now OpenAI is going to have great models for code.

25:08Matan Grinberg:Anthropic is going to have great models for code. Yes. Google is going to have great models for code. And XAI, as much as people don't believe it, they're going to have good models for code by the end of this year. And that is good because the more choice we have, the better it is for the consumers or the enterprises or the people who are using these. Now, the thing I disagreed with was this claim that if you make the models and the agent or the harness, you are going to make them better together, you know, naturally. Right. Like an example here being Claude code and the Claude models or Codex and the Codex models.

25:39Matan Grinberg:What we find empirically is that it's actually not the case. So by making Droid model independent, we actually end up making it perform better with Opus, better with Codex, better with any model than those models do in their respective harnesses. So we can outperform Claude code with Opus by using Opus in Droid. Naively, it might not make sense, but I think a good analogy here is, let's throw back five years ago. If you wanted to make AI that was good for you as a personal assistant, you could make the argument of, hey, you should train it on just your data because it's your personal assistant and you want it to know you better than anyone else.

26:18Matan Grinberg:What the last five years have showed us is actually, if you want it to be a really good personal assistant for you, you should train it on the whole internet. Yes. And then it's going to be much better for you. And there's an analogy that emerges here where basically what data is to a model, models are to the harness. Really? You find that? Yes. And so the more models you expose the harness to, the more kind of nuances and intricacies you find, and you avoid overfitting the harness to the nuances of that model in particular. And so these are on axes like compaction, which is how the agent deals with kind of being over the context limits, which is going to happen oftentimes when you're dealing with large code bases also on the axis of token caching so basically how often are you caching the tokens in a way that basically makes each query more cost effective because a cached token is kind of like a repetitive token that costs a tenth as much so you want as much caching as possible yeah we end up outperforming that and then also like tool use and environmental feedback by seeing the kind of intricacies of the different models it ends up performing better with all of them which is pretty crazy and my friends who work at the at the model companies get very frustrated by this um really yeah yeah i mean i bet you have to put this out in research i think then that would be amazing right like i feel like people will that will really like change people's prior oh yeah we have a couple blog posts about like caching in particular or compaction in particular but i think not the collective story of here is why you need multiple models to to make it perform better i think it'll be really cool You've also talked a lot about the need for open weights models and the value of that.

27:55And talked, I think, also a little bit about how you don't like it when folks sort of classify these as Chinese models because there needs to be a robust US open source system. Where do you feel we are on that front at the moment? Are there enough good players in that space?

28:10Matan Grinberg:Yeah, I think the reason why I don't like when people do that is because naturally, you know, if you're like the biggest threat to open AI, Anthropic or any of the model apps is not each other. It's open models, because if open models are really good, then it really puts a lot of pricing pressure on them. So their interest is it's just, you know, the big four and no one else, no one else in town, no one can compete. And so the best way to do that is say, you know, it's either us or the Chinese models. Yeah. Oh, like, you know, it's, you know, it's our enemy. you don't want to use their models right when the reality is no it's like it's them versus the open models and currently yes the best open models typically come from china um however you know nvidia has nematron which is actually catching up quite a lot still not the frontier and i do think it's really important to see us open models get better i think it's uh it's just a little like sinister phrasing to try and get you subconsciously and it works like enterprise cios that i speak to they will often say like oh we don't use chinese models and i'm like okay we can use other open homes.

29:09Matan Grinberg:Those aren't the only ones. Yeah. But it's crazy. Like the propaganda works to get them to think, oh, no, you can only use these. And so just like kind of ringing the alarm bells on that is something that matters to me. It also matters to me that we have good US open source. You had a spicy tweet earlier this week after the Fable 5 launch that seemed like it actually had a, you know, maybe it contributed to a positive outcome in some some respect. Safe to say you were not a fan of the way the fable five uh rollout happened from anthropic yeah um i think there were two things that i was really disappointed to see um one being they changed their policy to have mandated data retention for the purposes of security or safety rather um which is important but i think it sets a precedent for basically and that's even for the enterprise typically models if you use it in the like you know self-serve plan there might be data retention that's pretty standard um in this case it is for everyone even the enterprise requiring data retention which talk to a financial services company they're going to be like absolutely not and all indications suggest that they want that to be the case for every model going forward it sets a very bad precedent of basically we get to see everything that's happening and i think what's worse is they also you know at least initially were posturing that they would either deny your service or hamper your performance, not based on violating terms of service or violating the law, but doing things that they don't agree with.

30:37Matan Grinberg:And that I think is the line that's really important we don't cross. And even the fact that it's a possibility for any of these providers to do makes it so important that we have one competition in the closed models, but also open models. Because the crazy thing that is still the case right now is that if you are doing something that they deem competitive or risky, initially without telling you would degrade the performance of the model and have it give you kind of garbage answers specifically on ai research right i think on the bio research they tell you no like ai research in theory okay you think it's it's just like a little oh whoops we made a mistake you know like who knows it's hard to attribute it to like you know malice or just you know making mistakes but i think the precedent that it sets yeah initially was going to be without telling you, it would dumb it down.

31:27Matan Grinberg:And that's really, really dangerous. And so because of a lot of the pressure, I don't know if my tweets had an impact there, but I think a lot of people had a really bad reaction to it. They walked back just one part of it. They walked back to the part where it would do it without telling you. Now it would do it while telling you, which I think is still really bad because it's still degrading performance. And I saw countless people and friends of mine as well where they're doing like cutting edge bio research, looking into like prostate cancer markers and they were being denied their requests.

31:59Matan Grinberg:And the thing is, even if they go and then fix it, the point is they now have an ability to at any point in time deny your service and then say, oh, sorry, it'll take a week to fix. Oh, it's there. And so as people become more and more reliant on these tools, you want to make sure no one can kind of quickly affect you over this period of time and this breaks that precedent and does so not just in the weights of the model where before in the weights of the model if you're like whatever hey build me a nuclear weapon it'll say no like i'm not going to do that now it's kind of programmatically added on top where in theory there's someone over there who could like change a line of code and now it's like oh anytime mario asks about like world events like convince him of this crazy political ideology and like just send him in this other direction without you knowing and that's crazy i do think the transparency piece for me, like felt like the obvious misstep, but I I'm almost, I'm quite sympathetic to the idea that you do have to have some limitations of what people can use these models for.

32:59Like I, I do think flagging some cutting edge bio research or, you know, there are obviously bad uses there. Like, where do you think the right place to set the line is when you're operating at the frontier here?

33:11Matan Grinberg:First of all, the one thing that's an obvious, like kind of baseline is the law and like you know we live in a country that has laws and we should defer to the government to enforce those laws and to the degree to which we can we should work with the government on figuring out what the best way to help that is and like obviously platforms like facebook and twitter work with the government a lot to make sure you know on one hand you can have free speech but on the other hand you adhere to like laws about hate speech and things like that and so i think you know we should take from that and act in a similar manner there um And then I also think whichever way we go down, it has to be very clear who is getting what, and it needs to be equal.

33:51Matan Grinberg:I think that's the thing that's really not obvious to me is like, tech has a history of people doing bad things and then getting punished for it later when it's already past the point of no return. And that's the thing that I think worries me is like, for example, all of these model companies trained on the whole internet without asking anyone permission. yes right so we already know that there are companies and i'm not saying any of these companies right now would do that or anything like that but we know that tech has a history of like doing thing that is irreversible and then suffering later but like what penalty could you provide that would make up for there's nothing you could do it's like once you pass the point of no return and so like take for example data retention if you retain all the data of all the financial services companies and all that you could train on all of it oh no it's a whoopsie and you try to make up for 10 years later, going to be in courts forever, you're past the point of no return.

34:40Matan Grinberg:You could have trained on every company's data. It's illegal and you break the law, but that hasn't stopped tech companies before. Now, I'm not like super pro-regulation, but I think the important thing is like having the right checks and balances in place to make sure that no one company can kind of go too aggressively monopolistic in any of these directions. Yeah, you know, I think someone who we had on the podcast who, you know, In this location was Vincent from Prime Intellect, and I like the way that he frames the idea that a safer world is ultimately one with multiple super intelligences rather than just one.

35:17And that was the first time I'd had someone put it to me in that way. And the more I thought about it, the more I think that's fundamentally correct. And so this is why you do need this robust ecosystem that you are helping companies operate into and abstracting away that complexity. Maybe let's take a step back to some of your story. How does one decide to become a string theorist? Is that like you first want to be an astronaut and then make a jump? What was the childhood ambition there?

35:47Matan Grinberg:I was quite a rebellious child. In elementary school and middle school, I was always skipping class, getting in trouble. Generally, I was just bored by things and would we're just kind of get into a lot of trouble but i was always good at math um but i didn't really try in school because i was like oh i want to do whatever other random things and then my eighth grade geometry teacher uh told me that i should retake geometry in high school and that was like a trigger i was like what the hell like she thinks i need to retake geometry shout out margie greenwald eighth grade geometry teacher look what she created but and so so i was like i can't believe like I'll show her.

36:27Matan Grinberg:And so my first order on Amazon ever was textbooks for algebra two, trigonometry, pre-calculus, calc one, two, three, linear algebra. And I studied all of those textbooks in the summer before ninth grade. And then I asked my dad what the hardest math was. He said string theory and I was, technically it's physics, it's not math. But he said string theory and I was like, okay, I'm going to be the best string theorist in the world. And that was like, that was all I cared about for basically the next 12 years of my life. So you have a very laid back personality, clearly. Wow, fascinating. Were your parents academics?

37:03Matan Grinberg:They're both engineers. So my father was immigrated from the Soviet Union. He studied chemical engineering and ended up working actually, what brought him out to the Bay Area, he worked at IBM. My mother was a programmer at a hospital. This episode is brought to you by Persona, the B2B identity platform helping businesses verify users, fight fraud, and build trust. Fraudsters are already using AI to spoof faces, voices, and documents, so your defenses need to adapt just as fast. Persona helps secure some of the Internet's largest and most trusted platforms with identity verification. If you're building a product where trust matters, identity should be a priority.

37:43You've probably already experienced Persona without realizing it, verifying your LinkedIn profile, signing up for Etsy, or renting a scooter with Lime. Trusted by leading companies like Square, Brex, and Twilio, Persona gives you the building blocks to create identity flows that adapt to your customers, risk tolerance, and locales you operate in, whether you're verifying age, onboarding businesses, or automating KYC. It's fully configurable, so you can launch in days, not quarters. Want to see for yourself? Generalist listeners get a free year of the starter plan. Head to withpersona.com slash generalist and check it out.

38:21Okay, so you had sort of technical conversations in your life, but not string theory conversations by the sounds of it. Yes, that's right. So you sort of run on spite for 10 years through some very impressive places. You worked with what I understood from my research. You know, I wouldn't have known myself, I'm embarrassed to say. It's like sort of the greatest living string theorist. Quanwal de Sena. Yes. What was that like? Like, what does one take from a mind like that? I imagine that to be that person, you know, the sort of most luminous, luminous, you know, person mind of this field of your generation, you must have a very unusual way of operating, unusual way of thinking that sort of goes beyond even that domain.

39:05Matan Grinberg:It was a huge privilege. He's one of the kindest, smartest people I know. Also, I think very fascinating. He's quite religious as well, which is typically rare for a strength theorist. And, you know, it was a huge privilege working with him while I was at Princeton as an undergrad. but I think it also even having that happen that was a lesson and one of the examples for me of why mentorship matters so much because initially I was going to do research with this with this different professor but there was this one grad student that I knew who kind of took me under his wing and was giving me a lot of advice about you know what I should do and he was like no Matan you need to go and talk to Juan you should try to get him as an advisor but Juan technically wasn't at Princeton he was at what's called the Institute for Advanced Study which is like a very closely affiliated but technically separate institution that had no teaching obligations so in other words they didn't have to deal with the pesky undergrads yes they could just go do their research in peace and i was like wait i you know he's at iis he doesn't want to work with undergrads like you know he's like no trust me send him an email on about a paper of his that you read you know how try to have some insight and ask if you can meet with him just to talk to him and i was like okay you know and so you know spent a lot of time you know preparing and uh you know sent him an email and I ended up doing that and he ended up inviting me over to his office and before I went to his office this advisor this mentor of mine was like by the way the meeting will go well if at the end of the meeting he very calmly and subtly poses a question and poses like oh hmm this is interesting and basically you will have 24 hours to solve that problem and if you solve that problem within 24 hours, he will likely take you on as a student.

40:47Matan Grinberg:Oh my gosh. That's so great. And so I remember, you know, going into the meeting and being like super on edge, like ready to like find the, what's the thing that he leaves. And Juan is one of the most, one of the most subtle and like soft smoking people, soft spoken people. And so we spend like two hours at a chalkboard and the meeting ends and, you know, I'd been taking notes. He was at the chalkboard and like vice versa. And, you know, the meeting ends and I was kind of disappointed because at the end I'm like, wait, he didn't ask a question. Like there wasn't a thing. And so I go and, you know, I have all my notes and I take it to that mentor and I'm like kind of going through it with him.

41:23Matan Grinberg:And, uh, and I'm like, Hey, you know, I, you know, it didn't go well. He didn't, he didn't, you know, ask me the question. He didn't plant the seed. And so I go through the notes with this mentor of mine and we're basically like recapping the whole two hour thing. He was like there, like that moment there. That was the subtle question. Like if you solve this equation and, you know, do this and send him an email, then he'll take you on. Wow. It was it was truly that subtle that you had to sort of Talmudically go back through this. Wow. Fascinating. That's amazing. I love that story. In some ways, I would imagine the life of a theoretical physicist and the life of a startup CEO are almost like the total opposite ends of the spectrum.

42:03In some sense, I could imagine both been quite lonely. But like the startup operator, you know, it's almost like the Marcus Aurelius beginning of meditations where he's writing about like, you will encounter, you know, annoying people today. You will encounter trouble. You like you must expect these things. That's sort of the fundamental nature of like building a business, right? It's like rejection, stress, annoyances. Theoretical physics is like seems very monastic.

42:28Matan Grinberg:Like, how do you think about those parts of your personality? Prior to just now, I would have always described it as completely opposite but i actually do think you do have a point that there is something very similar but you know on the surface they're the most opposite because as a you know devoted theoretical physicist my ideal day was alone in my room for 12 hours at a time just like reading papers like that is the ideal day like not seeing the outside world whatsoever now i forced myself to do that for like 12 years because also partly out of spite but partly also because the mathematics behind string theory is just so it's so beautiful it also like the chain of whys like if you just ask why enough it always goes to string theory or theoretical physics or then beyond that it's philosophy but you know we can't prove anything on on that front yes but like it's kind of the base of everything and there's something very beautiful about the mathematics there so it did keep me going and it wasn't you know purely spite but it is very monastic and i would always I've always been very curious just to like get to know people and you know I've had people make fun of me at times where like if I'll meet someone at a party it'll be like an hour of me just like asking them questions and then at the end they're like I don't know anything about you like answers or something but it just you know it was very enjoyable for me to like learn about you know different things but I would always feel guilty doing that as a physicist because those were hours of my time not spent you know hard at work understanding more about the universe right And so I think the thing that's, you know, on the opposite side as a CEO and founder, the amount of context switching I do is insane.

44:06Matan Grinberg:It's like at the end of the day, I typically forgot what I did at the beginning of the day because it's just so varied. But that curiosity, especially when we're selling into the enterprise, it's so fun talking to all these CIOs and understanding the different nuances of how they build software. And it's very satisfying. And I like engaging with people. but completely opposite where there has never been a time where i focused on one thing for 12 hours literally for three years like i've never done that really uh because it's just there's too many too many multivariate problems but i think to your point about the similarity there are two similarities so on one hand it is kind of the solitude which is like you know you do feel a little bit isolated i guess in one case because of the responsibility especially after going through the highs and lows yes where you see the people that are only there for the highs and then the people that are still there for the low like that kind of really makes it obvious what the kind of solitude there is like and then in the case of physics it's like the literal solitude also you can't talk to anyone about it like no one had a dinner like you could talk about black holes or whatever but you can't talk nuance yes about it that makes total sense and also it's always kind of funny to people like if you talk at a dinner party about physics or about black holes within three minutes someone will make like a uranus joke and then they're like all right i'm done like that we've done yeah yeah on we go yeah exactly so it's you know a similar solitude there um but i think there's also a similarity in accepting that there are going to be so many variables that you do not know um in the case of you know starting a company there's just a million things happening all the time especially when we get to a certain size i won't know everything but still being able to figure out how to decide what are the best next steps how to make decisions knowing that there are things that you don't have time to fully understand similarly in physics it would take you a full lifetime to read more than a lifetime to read all of the literature that has come before and you just have to accept okay i don't know the full derivation of this equation or maybe i don't have the deepest understanding why this is true but in order to make progress here i just need to accept that get some intuition about it and just move forward and so there is some some parallels there at what point did your mind begin to wander from string theory was there a moment where you've noticed yourself sort of yeah so when i came to berkeley to do my phd that was when i first had gsi duties or graduate student instructor duties i've never been too fond of teaching um not that i like i enjoy it but not as a job like it's enjoyable when people care as a gsi at berkeley you teach people who do not care and are there just for the kind of uh um you know requirements yeah and it really it took me until then to realize but basically i knew everyone who i would ever work with for the rest of my life there are very few string theorists in the world i'd met everyone that i would be working with basically for the rest of my life except the potential students i would have and i would basically be surrounded by children for the rest of my life as well because you have to be at a university to be a theoretical physicist.

47:11Matan Grinberg:And that kind of led to a cascade of, wait, oh my God, do I actually want to do this? Is this even the best fit for me and my personality? And so it kind of led to this big cascade of kind of questioning everything. And then it kind of slowly broke down the stubbornness that I had. And I was like, okay, you know what? Maybe there are other things that are out there for me. I imagine that was extremely difficult. If you've identified, you've sort of attached yourself to this like vision of yourself I certainly had versions of that where for much of my early life I was dead set on being a lawyer and then working at a law firm disabused me of that notion but it was painful it was you know difficult to sort of remove yourself from the dreams you had like what costumes were you trying on at that point of like I might be this person I might be that person that's honestly that's such a good way of putting it because that is actually what it felt like it felt like this costume like I had if you would ask me it was like I was like a physicist before I was a human.

48:11Matan Grinberg:It was like that deeply associated. And so it was like completely earth shattering, trying to understand what do I attach to? Like this was everything that I spent all of my time on. At a certain point, like kind of just leaning into the fear, it became somewhat freeing. Be like, I can just do it. Like I felt so guilty about any hour I wasn't working on physics that now is like, oh, okay, you know, let me explore other things. And so one thing was, you know, I hadn't explored humanities as much. And so set aside some time and I was like, you know what? I should become a more normal, well-rounded human.

48:45Matan Grinberg:So I like watched the top 250 movies on IMDb. Oh, wow. As a way like, okay, you know, you got to. And that was so much fun. Like I've seen Batman. Yeah. But like I, you know, so many of them, you know, brought me to tears because they were just so beautiful or similarly, like the top hundred pieces of literature. I was like, you know, we got to read these. And looking back, that was one of the most fun periods of time because it's like, these pieces of literature are famous for very good reasons. And getting to experience some of them for the far, some of the movies and books I'd read before for school, but like experiencing it for the first time is just so much fun because they're all just such legendary pieces.

49:23You can't bring up this topic without giving some names because I love talking about books and films. Like, yeah, what struck you?

49:29Matan Grinberg:Okay, so the first piece of literature that made me cry, it's so cringe to say, but A Tale of Two Cities by Charles Dickens. I remember I called my dad because my dad also loves literature. I remember calling him and I was crying. I was like, this is so beautiful. Wow. Dorian Gray was fantastic. Brothers Karamazov, I think, is incredible, although there are some sections that are completely a waste and some that are just absolutely beautiful. Honestly, the best to ever do at Shakespeare. Again, underrated, even though he's the go-to. Yes. He's just so incredible. Yeah. As for films, one that always sticks with me and I watch all the time is Harakiri it's an incredible samurai film black and white but it's it's beautiful you should really watch it I definitely honestly watch it and tell me like I would watch it with you anytime I get to rewatch it it's fantastic there's a there's a good Italian film called La Grande Bellezza oh I love that film so good it's so good I went to the theater like three times to see that movie what a movie so good just visually so fantastic i can't even relate to it that much it's like about an aging you know man who used to be the the center of the party and now is you know i had nothing to relate to it but somehow it still just like pulls your heartstrings yes what are some for you i actually i would often say la grande bellezza is one of my very favorites because i think it's just like so so amazing there's a french film you were mentioning this black and white japanese film so maybe that's why it came into my mind called land oh so good i love that film just because see Tuva Bien, right?

51:03Oh yeah, that's right. Gosh, wow, I can't believe that. Yeah, that was really great. Yeah, I'd have to think more on ones that I would really ride for. My ultimate favorite, have you ever seen The Master by Paul Thomas Anderson?

51:15Matan Grinberg:I have. I really loved that. I thought that was an incredible film. Okay, amazing. We could talk about this for hours. I also, there's like, I have a note on my notes app, I have a list that I'm tempted to pull out, but maybe we save this. Yeah, we'll do that. Yeah, we'll do that after. I feel obliged to at least ask you a little more about your journey. You have this moment where you sort of explore the, you know, explore the glories of the broader world. Yes. At what point do you migrate sort of from, you know, literature to tech? Yeah. So, you know, in my head, I kind of, funny enough, there was probably a month where I was like, you know, maybe I could be a writer.

51:51Matan Grinberg:And I was just trying some stuff. Absolutely not. Not for you. Not my strong story. and it became pretty clear okay you know I spent the last 10 years becoming pretty good at like quantitative things or math and so it seemed to me it was like either startups big tech or quant finance. Quant finance is what like all math and physics people go into. I almost did quant finance last second another mentor gave me some very good advice which was don't do this stay at Berkeley explore a little bit because once you do quant finance you're not going to do anything else and you're never going to get worse at math.

52:24Matan Grinberg:You can always go back and do that. But, you know, while you're up early, just stay and explore. And I was like, okay, you know what? I will. And so ended up taking some courses in machine learning and AI. And at first, I really liked it mostly for competitive reasons because I hadn't done computer science before, but I was kind of doing better than some of the CS students. And so competitively, it was satisfying. But I didn't really like scratch the true itch until I took a seminar in program synthesis, which we now call code generation. And what really nerd sniped me there was, it's not machine learning to create video or audio or images, but it's code with the explicit purpose of creating itself.

53:03Matan Grinberg:And there's just something very fundamental about that, and that got me obsessed. And so I ended up staying at Berkeley for about another year to do that. And my advisor was very kind to just let me take these AI courses and do this research that was pretty unrelated to physics. ended up realizing that the best way to pursue that problem was not academia, but rather, you know, working with some of the most complicated code bases in the world. And the only way you do that is by starting a company, which I knew nothing about. Yes. And so like any, you know, curious person does, I ordered zero to one by Peter Tio on Amazon.

53:36Matan Grinberg:It's a good starter. And looked up on YouTube, like how to start a company. And then you immediately started a dropshipping business and, you know, whatever else. Yeah. but yeah so whatever go into this rabbit hole of like a lot of trash some good yc videos and then stumble upon this podcast interview uh with a partner at sequoia whose name i recognized because i had cited him in that paper that i wrote with juan maldesena back at princeton wow yeah it was weird to see like a vc who had been a string theorist before yes and ended up you know sending him a note turns out he was also a physicist had very similar reasons for getting into physics, similar reasons for leaving.

54:14Matan Grinberg:He ended up daring me to drop out of my PhD, which I did. And that was kind of how we initially started Factory about three years ago. And when you, you emerald Sean McGuire, right? Yes, that's right. And you go for this very long walk, as I understand, which I love, you know, these parts of the startup story. At that point of time, did you have like an idea in your mind of what you wanted to build? How well formed was like the Factory thesis at that point? Yeah, so this was probably two months after ChatGPT came out. And the main idea was like, hey, look, here are these coding problems. Let's play this fun game of you're not allowed to write a single line of code.

54:52Matan Grinberg:How often could ChatGPT at that point in time, or GPT 3.5, how often could it output exactly the right solution and you didn't need to edit anything? And basically, you as the human play the game of the orchestrative like, oh, no, we're in this code base. Oh, here's the context there. He's the context there. Yes. And actually pretty often, if you properly subdivided the problem and you gave it the right context, it could do it very frequently. And it was like, OK, let's just now try to abstract away that orchestrator role. And then you have an agent. And so that from like when we started, the mission was to bring autonomy to software engineering.

55:29Matan Grinberg:And it was like building autonomous agents. We actually initially were incorporated as the San Francisco Droid Company. Yes, which sparked a little issue. right yes yeah and then our lawyers were like hey you know actually lucasfilm might be you know litigious you should actually change the name and so then the san francisco ai factory was uh was what we went with but you can't you managed to hang on to droids in there which is thus far yeah yeah yeah they had this we'll see i think i've heard you talk maybe elsewhere or maybe you wrote about it that like the first year and a bit of factory was you know not an obvious success like there was a real grind at various points like what were the yeah what were the hard parts there that sort of needed to get ironed out.

56:08Matan Grinberg:I think I refer to it as like our years in the desert, where basically, you know, we had this mission, we had this team that was absolutely incredible, that were turning down the ridiculous offers from, you know, all of the big AI companies to join us. And the enterprise was not ready for it. And I think the big learning, like the enterprise was barely even adopting co-pilot, let alone autonomous agents to go work on, you know, coding and testing and reviewing and all that stuff. Yes. And I think the big lesson there was being early is the same as being wrong. No one gives a shit if you are early.

56:39Matan Grinberg:You don't get a consolation prize. There's no, you know, pride in that. Like being early and not winning means you just lost. Like that's it. And I think, you know, maybe the secret benefit there was it's actually probably a blessing that we were early because one, it built very robust DNA for our team, which is we were all united in this mission. We all worked well together for a very long time, despite all of the noise outside. and we made it through that adversity to now see that success. Whereas a lot of competitors out there, it's like day one, you're a unicorn. That first fundraise we did is about to be unheard of.

57:13Matan Grinberg:The valuation, that first check was$5 million. Wow. Good work from Sean. Good work. Like no companies these days, it's like you're a unicorn immediately. Like you're guaranteed to like, it's got almost like victory is granted immediately. Whereas for us, like we've been through that. And so now any hiccups are like, wow, nothing. Yeah. Like we've been through so much shit as a team that's made us so much more robust where it gives me so much confidence going up against some of the most well-funded companies in history. Yes. And then two, and this is maybe being a bit self-aware, it's good that we were early because I was probably not the right person if we were on time.

57:52Matan Grinberg:Because this is the first and only job I have ever had. Yeah. Yeah. I was wanting to ask exactly about that. Like, how did you learn to manage people? to, I don't know, do all the things that this job takes? Like, what was the learning curve? Well, I had a nice two years of training, I suppose. But I think also, you know, some of it comes naturally. Some of it is just reading a lot. But I think also it's kind of a blessing to not have delusions of thinking I know what I'm doing. Like, I came into it very clear-eyed about, like, I literally have no, I have never been paid for work aside from physics research.

58:28Matan Grinberg:and so the nice thing is it's just like having deep curiosity means finding everyone i can who's really good at engineering who's really good at product who do you think is the best person you've ever met at this and kind of recursively following that and just like absorbing as much as possible finding through lines finding wait actually this person was just a bullshitter and they were just saying all these things but actually they didn't do anything and so this information you know maybe throw that away how do you do sales how do you build good teams how do you interview well all of these things it was just like I mean one it was just so much fun because I love talking to people who have opinions about these things yes and then also putting it in place and doing this and over time seeing what type of candidates end up doing well what type of team structure allows us to execute quickly what ends up slowing us down what are things that are fun but actually not worth my time how do you make people feel empowered such that they can go and own decisions fully and it doesn't feel like you're micro like all these nuances um it was just such a fun time to like get great advisors and mentors and like some of the people that initially early on i read about are now people that i'm privileged to call mentors like frank sleutman is now one of the people that i get to talk to and learn and hear his advice and he's like one of the best to ever do it now i mean i'm sure he would be disappointed if i didn't disagree with him at times and there are things that work for him his companies or his era that don't work here but i think the big thing is having the humility to hear and learn from as many people as possible.

59:53Matan Grinberg:And then you kind of just have to trust your gut on synthesizing based on that and what's going to kind of work for you. One of the things that I think a first time CEO in particular must struggle with, and I've seen it in many cases, is that you don't know what you're doing. And so it's very easy to take what people tell you at face value, right? But even when you're talking about someone like frank you're like yeah but he'd be disappointed if i didn't challenge him how did you learn that for yourself like is that just your personality in some way that you're sort of comfortable tuning out the noise or is that a physics thing yeah actually okay again i didn't think about that but it is a very physics thing like i remember any time like my friends or my siblings would be around while having physics conversation people would think you guys like hated each other Because at the chalkboard, it's so like, how do we get because you it's very hard to use the English language to describe equations.

1:00:48Matan Grinberg:You have to speak very succinctly and very bluntly. And to the external observer, it might seem like you guys are yelling at each other or like not getting along. But in fact, it's just you need as optimal information transfer as possible. So you're very blunt at speaking like there's no time for, hey, that's a great idea. I really respect that. However, no, no, no. It's like that doesn't make sense. You forgot that. Like it's, you know, very, very blunt. And I guess after, you know, a decade that probably left a mark on me. But then also, I think talking to mentors like this, they feel like a kindred spirit.

1:01:20Matan Grinberg:Like, no, no one wants to be like, hello, kind sir. Thank you so much for taking the time of your holy day to speak with me. It's like, you know, I mean, another great mentor of mine is Chris Degnan, who's actually the CRO at Snowflake. and I think we initially met and the reason why he decided to even spend time working with us was because uh I guess I kind of forgot about this but apparently this is how he describes it apparently after we first met I was like texting him incessantly about like sales candidates and who he thinks is good like sales reps and he's like a CRO that took a company at three billion dollars and he was just like honestly like I like the hustle yeah this guy's going after it yeah what have you learned uh about yourself in this process like are there certain things that you had no idea you're actually surprisingly good at and maybe some things where you're you know hadn't explored and you're like actually that seems to be something i'm not naturally brilliant at i've had to adjust i think most things i'm bad at um and the thing that's most important is being really not shy about addressing it i totally don't believe you i'm sure you i think it's like it is the most important like there's a natural urge of like i'm bad at this i don't want to look at it Like I want to hide it away because I'm and it's instead like, oh, this is something I'm so unfamiliar with.

1:02:34Matan Grinberg:Let's like jump in headfirst and try and learn because the reality is like most people who are legends at things are not that much better than the average person. They just spend time on it. And it's like if you're shying away and not spending time on it, you're going to stay bad at it. So there are so many things that I'm bad at right now. The thing that scares me is not discovering that I'm bad at it. It's not discovering it. Having a blind spot, something that I'm weak at and I haven't yet found it. And so there's certainly a lot of things that I'm continuously going to find that I'm bad at, but it's always exciting because it's like, OK, great.

1:03:04Matan Grinberg:I just spend a ridiculous amount of time on this. I can probably get better or I can learn more. I can talk to people who are better. Or at the end of the day, the beauty of having a company, you can hire people who are way naturally more gifted at that thing and then empower them to do that. as you've sort of thought through building a company with a beginner's mind and from these first principles, what have you found like works for your culture? Like what's the culture of Factory that you care about, the way you operate, that you have noticed really contributes to the success of the organization?

1:03:34Matan Grinberg:Yeah. So maybe let's start with like the leadership team at Factory, the people that kind of I work closest with. The work in Cadence that I love is, for me, it's really hard to trust people but actually what i what's easy for me to do is to just in my mind flip a switch and just pretend oh like this new person that i don't know that well i trust them now and kind of stick with that until proven otherwise so i am able to do that and so i think the the working cadence that we set is basically hey look like i'm going to completely trust you on this i want you to run this and completely own this and like let me know how i can help you and I'm going to tell you if you like do something otherwise and I think the thing that is the most rewarding thing is when you kind of fully just give someone the room to run like hey you own all of this take more if you want I'll tell you if you're doing too much yeah and having people who are naturally kind of like a gas that like expands into the volume in which they're put in yes they just do that it is the most satisfying feeling of like seeing something and being like oh hey I should tell this person to go do that and they already did it that's like I don't know what the equivalent of a love language is for like working in a company yeah that is the thing that just makes me so happy and so having people that are like that that naturally want to do more as opposed to people that i have to say hey why don't we like raise the level of ambition but instead people that are like kind of doing that and sometimes i have to be okay wait hold on let's not do that just yet i think you described meeting your your co-founder you know as um you know like an intellectual love at first sight right how do you two balance each other like Like, well, you know, what's that working relationship like?

1:05:06Matan Grinberg:I mean, Eno is by far the best engineer I've ever met. But I think the rare privilege is he's also an incredible manager. And so like the whole engineering team reports to him. But he also, and this makes it insanely rare, he cares and is good at go to market, which is so rare. There are so many companies where the CTO likes coding and is really good at it, but wants to live in a basement and not see customers at all. Eno is like so incredibly gifted as an engineer as a product thinker as an engineering leader but also like you could put him in front of any CTO CIO CEO in front of our sales team and he will like rally the troops rile them up get them excited and it is just such a privilege because maybe this actually gets into the broader thing aside of the leadership team which is like values at the company one of the ones that I like is we call it build together high level doesn't sound that interesting.

1:05:58Matan Grinberg:But the important thing is everyone at factory is first class. We are working on a very technical product. It is very common in the Silicon Valley. And a lot of competitors do this where it's like research is the most important. Yes. Those are the coolest people. And then there's the engineers who, you know, they're still pretty cool. And then there's sales and marketing like, oh, if only we built a better product, it would sell itself. Like, let's not allow them into the same building as the engineers. That's a true story that companies do this. Let's not allow them in the same room as the engineers because they'll distract them and those silly salespeople.

1:06:26Matan Grinberg:The reality is your product is not just the software you build. Your product is the whole journey from the very first time they hear your name till their 10th renewal after a decade of being a happy customer. That entire thing is your product. And if you only care about the software, you are going to deliver a shitty product experience. And so, you know, at Factory, we do completely mixed seating. So it's not like the engineer's corner and the sales corner. It's like, no, engineers and sales sit next to each other. And every, you know, there's always people coming in are like oh you know actually like for an engineering part it would be good for next to each other so it's easier to communicate and this is the hill that i will die like never we are never going to do that because i can't tell you how many times i see like a sales rep sitting next to an engineer and then they're like oh yo look at this like oh go check that out or vice versa and then like time and again they become friends because we only hire sales people that want to work with engineers we only hire engineers that care about our customers and want to see how you know developers are using factory.

1:07:22Matan Grinberg:This is another thing that makes my heart sing when I see like that, like and not having this like split culture. Yes. And, you know, the reality is, is also despite how technical of a space this is, if you think about legendary companies, try and name a legendary company that has a bad sales or marketing team. Hmm. Very, very difficult. Yeah, no, I'd have to dwell on it for a little bit. But if you think about legendary companies that have bad products. Yes, you can kind of figure that out. Quite a few. Quite a few. There was a big workday blow up on John X the other day. Exactly. So and I think that's, you know, it requires a little bit of humility as technical leaders, because a lot of the companies in space have technical leaders.

1:08:00Matan Grinberg:It requires some humility to realize like, hey, just because someone's not an engineer, doesn't mean they are not equally important. Yes. To the product that we are building. And that's something that I think is pretty important for us. That's really cool. Maybe as we sort of move towards the end here, what do you think looks different about AI in a year's time and what do you think looks different about factory? I think AI a year from now, there'll be less focus on the models in particular. I think with model routing, it'll make it such that we've been in this phase where initially the question was, can you adopt AI?

1:08:36Matan Grinberg:That's a big question. Like the board would yell at the CEO, Mr. CEO, what are you doing about AI? They're like, I don't know, like CTO, go make sure everyone uses tokens. And then using tokens they did. And then it went up like crazy. And then what did you actually do? what was the roi who knows who knows i think now it's getting very clear okay we actually need to have an roi story having an roi story means you don't need to use the overkill model every single time which is going to lead us to model routing yes and so model routing will handle that there will be less kind of obsession over which exact model is doing things and instead there's going to be a lot more focus on where do you allocate tokens like right now companies haven't set their token budgets well they'll correct and they'll figure out okay this is actually what we want to set our token budget to the question will then be we do not want to evenly distribute tokens in the org where do we go and allocate those to deliver what kinds of roi yes so that's going to be a thing that people are going to talk about a lot how factory is going to look different um i mean we started this year as 25 people we're now 110 115 now i don't think that the number of people is like a metric of success but it's more just like we need to do it because of the demand in the market like we are not saturating the demand that customers have and so that's why we are growing the team yes so i think the team will be larger um i think the things that um my hope is that we can actually feel like effectively a smaller company because of using things like ai like i think like small company feeling you can have that more when you can keep communication like what it was of a 50 person company even if you're 500 and there's certain things you can do to kind of maintain that.

1:10:13Matan Grinberg:And I think that's something that really matters to me. Like, I want to make sure that when we're a 500 person company, I still know everyone. It's going to be difficult, but I think it's pretty important. And not just like on a vanity reason, it's just like, I think everyone is so important to what we are doing. I think that like, it's not that I just want to be able to say, oh, I know everyone. It's more that I just so deeply believe that every single person is so important for what we are doing that I want to know them well. And I want to help and jump on sales calls with them or jump on engineering sinks or design syncs or whatever it might be because it's so important and because I think we actually can do it.

1:10:46We can scale the closeness. So I always like to end with a few sort of thought experiments. If you had the chance to do an experiment with no operational constraints and unlimited resources, what's an experiment you'd want to run?

1:11:01Matan Grinberg:How do we define experiment? As broadly as possible. Okay, like literally just a science experiment in the world? Yeah, yeah. A social experiment. You know, people have come up with some truly deranged and interesting things. Okay, so the first thing that comes to mind, and as a quick side note, I was a theoretical physicist. I was never good at labs. And actually, I think I might have been the one person to go to Princeton who didn't take experimental physics. I found a way around it with a credit from a different class, basically, that expired and the class didn't exist anymore because I was never good with a lab.

1:11:32So you're saying this isn't your strong suit? Yes, this is not my strong suit.

1:11:35Matan Grinberg:However, that paper that I wrote with Juan, the key to string theory is where the things that are very large, like galaxies, and the things that are very small, like particles, like atoms, come together. Now, where do you have things that are so massive, like galaxies and small, like particles? You have it in a black hole. Because it's very massive, like very heavy, but infinitely small at the singularity. And that's like kind of the crux of physics. That's what we all want to understand. Problem is, none of us can go into a black hole and survive. And so if I could run any experiment, it would be to put some probe in the black hole and get some data out of there.

1:12:10Matan Grinberg:And basically this paper that I wrote with Juan found that there is one piece of data that you can measure outside the black hole about the singularity. And that piece of data is called the TTS or time to singularity. There's a way you can measure from the outside how long it would take to fall from the event horizon into the singularity. And so getting a little bit more data, honestly, it's kind of like the plot of Interstellar, another great movie. Yeah, that is sound like Interstellar. They needed to get the data from inside the black hole. although I don't think love will be the answer I was going to say my big beef with Interstellar is the end I loved it right until that part and then I was you know ready to leave okay that's a great one wow yeah that would be fascinating what about you what's your experiment oh gosh no one's ever turned it back on me this is this is tough I mean I would be really interested in being able to know the personality traits of all the humans that have come before me to see like of people I know or of myself, who was I most similar to in previous eras?

1:13:12Interesting. And what life did they live? And what could I learn from them? Like, you know, did they have these sort of failure modes or did they do something very different to me? And it turns out they were really good at it or they did something similar to me and actually, you know, it wasn't so great. Like you always sort of want to Monte Carlo your own life. But I'd be fascinated to know if you could, you know, really map someone's personality, like in a video game, you know, of traits.

1:13:35Matan Grinberg:I love the way you think about these problems. These are such interesting angles. Okay. That is fascinating. I would like to see that. And I'd like to know, it'd be so interesting to know you with your best friends. Actually, there was a Mongolian peasant woman who was exactly like your best buddy and you would have gone on so well with her. That's crazy. So I would be really interested in that. We sort of already talked about books, but I always have to end with the question of if you could assign a book to everyone on earth to read and understand what would you want to give to people i'm literally going on my good reads right now because like i don't want to fuck this up this is how seriously i take this this is great i love this this is someone recognizing the responsibility they've been given literature is so important i think it's so important i think first of all anyone reading any books by the way more often like there's not a lot of time these days and i think there's something so good about just like slowing down your pace and like following um someone's thoughts okay hold on What were your...

1:14:32Well, I would really recommend everyone read at some point Invisible Cities by Italo Calvino, which I think is a beautiful book. It's the sort of book that's maybe 150 pages, but takes a really long time to read in a way because you just stop after every page and you're like, oh, that's interesting to think about. it's written from the perspective of Marco Polo telling the tales of his travels to Kublai Khan about the different cities he's gone to, but they're all imagined cities. You know, it's cities where everything is perfectly mirror inverted or a city of string or all these sorts of magical things.

1:15:08And you just end up thinking about those dynamics. So that's one where I think it definitely changed how I, yeah, it just takes your brain to magical places that I think is good for you. Yeah. You have an answer.

1:15:19Matan Grinberg:I don't have an answer because I can't find my login. So this was the one that, yeah, so this is the answer that first came to me is probably the collected short stories of Jorge Luis Borges, because there's such a fun, like, upending of any way you might normally think about short stories. And there's just, I think the reason why I say short stories also is because one book typically has one kind of message or, you know, overall thrust. with his short stories you get a couple different angles on things and he is one of the most polymathic yes thinkers i think that are out there like he's a writer but he writes almost as if he's a mathematician yes and so i think that that like perspective he is one of those unique souls that have existed that are like you know in in your example of you know i don't think there's anyone that has been like him that has existed there i don't know if he has a kindred spirit because he was so idiosyncratic.

1:16:16Matan Grinberg:I love that. Yes, he's the writer who probably best captures infinity. That is a great choice. Matan, this has been so much fun. Thank you so much. Thank you. That's it. Thank you for listening to this episode of The Generalist Podcast. Please subscribe on Apple Podcasts, Spotify, or your preferred podcast app. Ratings and reviews help others discover these discussions. So if you enjoyed the conversation, I'd be grateful if you could take a moment to leave one. For all past episodes and more, visit us at thegeneralist.substack.com. See you next time as we continue to explore the future.