It’s Tuesday and your tech stack is obsolete (again). Now what? | Theory Venture’s Bryan Bischof

0:04Today, I'm joined by Bryan Bischof, the head of AI at Theory Ventures. Bryan has a PhD in pure mathematics. He teaches AI and data science at Rutgers, and he's built teams and led projects at places like Hex, Weights and Biases, Blue Bottle, and Stitch Fix. And Bryan, I've really been looking forward to having you on our show because I got to see your work in action just last year at your America's Next Top Modeler hackathon, which is an amazing name. We're going to dive a little bit more into the things behind it as well. It was an amazing time, and I learned a lot. That's really influenced my own personal journey and the things that we've talked about on this show.

0:45So it's really great to have a mind in the mix behind all of that that kicked it off. And we're going to get into some of your work at Theory Ventures as well. But first, of course, we have to talk about what's happening right now at the AI Council 2026 conference, where you and I are both on site. And Brian, I'm thrilled to have you here. Congrats again on being named a track chair as well at the conference. Thank you. Thank you. Yeah, I'm really excited about AI Council this year or FKA Data Council. Some of us still think of it as Data Council. Let's just be real. this reminds me of an interesting debate that I was just like clued into is it AI and data or data and AI I think it's a really interesting like you know I think you got to say AI and data that kind of rolls off the tongue a little better doesn't it I think that's probably right AI and data, data and AI I can't even say you can't even say clearly the answer is AI and data I can't speak the other one out loud if the utterance is unutterable then the decisions have been made.

1:50Okay, so this amazing conference of AI and data. Yeah, I'm really excited. Pete likes to throw me a challenge. And this year, his challenge was inference systems. Last year, it was foundation models. But his question to me was, I'd like to put together something about inference systems. Have fun. And I like the challenge, but I also, every time he throws down one of these gauntlets, I find myself thinking like, I don't know anything about this. And so that begins the journey. And so I research it sort of like how I would research an article or how I would research like a book topic. And I basically just like dig in to start trying to learn things.

2:37In this case, the way that I thought about it is not so much like, I want to learn all the things about inference systems during this research, But what are the things that I would try to learn if I were to try to learn this topic anew? And then what I do is I basically jot down like, okay, these are the major holes in my understanding. And then I just ask some of my friends who are very smart and very knowledgeable how much they know. And I try to figure out like, am I the only dumb one in this particular area? And for many of these, the answer is yes. Everyone else seems to know except for me.

3:11And in other ones, it's kind of like, no, I don't really know much about that. or like, oh, you know, I like kind of understand how that works, but not like a deep way. And so one of the benefits of this approach is because I have all these like smart friends that I can ask, I kind of naturally get this exposure to what is known, what is known by like basically everybody. So like I know it. And then what is known by sort of like the really strong people, but not experts in those areas. And then I find the things that are missing from that. And I said, okay, now I need to go and hunt the people for that.

3:45And so that's basically how I engineered both of the tracks the last few years. And so this year, it was very much like a lot of people had questions of, what does it really mean to optimize at the inference layer? What does it really mean to think about cost in a more holistic way? what exactly are we doing when we are sort of choosing an optimizer for training things like that these are questions that a lot of people felt like they had like a little bit of knowledge but not as deep as they wanted and so then i went and i hunted down the best experts i could find it's really key insight on kind of how you think about organizing um a track and all of the talks within it is treating it as if it's like a high level research project where you have a thesis that you're exploring, you're trying to plug the gaps.

4:36You have a wide and educated network that you can draw from as somebody in this vantage. And in doing so, you start to surface these obvious gaps that there's a lot of smart people in the industry and that you know and that are really dedicated to the problem seem to have. And then you get the shift into this like information headhunter. You're like, who is this out here in the world who could teach us all about this gap? and help can make this puzzle a little more complete. And I think like in such a frontier kind of environment as AI, where there's just so much innovation, there's so much opportunity to learn every single day and every kind of thing that it touches, that there's a lot of gaps.

5:19So I think like there's so many pitfalls and so many obvious things that a track chair or somebody in that position could fall into of like, oh, I have a topic, I'm going to create a narrative. Instead, you flip that on its face. I'm not going to tell a narrative. I'm going to get the breadth of this field, and I'm going to find these pitfalls, these hidden dangers that we all need to be more educated about, and then I'm going to bring us all together to learn about them. And so, you know, what do you think ultimately is the story that's emerged from those gaps that you found and the people that you've lined up to teach us?

5:50Yeah, so I would say two things here. So, like, one, like, what is the point of going to a conference? Like, I've gone to a lot of bad conferences over the years. Ones where I found myself wondering, like, why am I here? Like, it sounded like a good idea. And I don't know if it was a good idea to even, like, spend this day doing this thing. I saw some people I like. But, like, man, is it just a social activity? And that's fine. If that's, like, why you're going to conferences, like, respect. I get that. I went to a conference in, like, Denmark a really long time ago that was basically just a social activity.

6:25But, like, what I like conferences to be is educational. and from that perspective i was like okay you know like here's how i want to make sure that everybody has something to go home with but to answer your question about sort of like what kind of narrative that i find myself in i think it's exactly your point where i'm not trying to construct a narrative but i did find myself asking sort of like what is this like structural thread that pulls things together and it's kind of coalescing into this idea that we sometimes talk about the cost of intelligence chart. My friend, Charles Fry, he made this like amazing graphic a couple of years back of like the cost of intelligence chart.

7:08And it's basically sort of like how well the models are doing on one particular famous benchmark, which is now basically like oversaturated and dead versus the cost of tokens or sort of like how achievable is a certain level of intelligence at a certain time. And there's been 18 billion remixes on this chart. Like by now, it feels very easy to find the one that you remember in your head. And you're probably picturing one right now that you've seen most recently. It's probably not Charles's. But Charles, as far as I know, did the first one. And this one has been really influential for a lot of people.

7:49And the reason I bring it up is because Charles' plot essentially has three dimensions. It's plotted in two dimensions to put on your screen, but it's effectively representing three. And those are basically sort of like the metrics that everyone is optimizing to. It's the thing that you're trying to produce on the other side of the function. But I've been finding myself thinking more and more as a mathematician, okay, there's the target of this function or there's the sort of like, you know, the range of this function where we start getting observations. But what about the domain of these functions?

8:30Like, what do we put in and what do we change? You sometimes hear about this in sort of like data science and business ops as levers. Like what kind of levers can you pull? Also, I don't know if your audience says levers or levers. I've gotten like criticized aggressively for calling them levers. So I'm being very conservative by calling them a lever. You covered your bases. Okay. So these levers that you can pull. So one lever that sometimes people pull is they're like, buy big GPU. That's great. That's totally reasonable. Another lever that people pull is sort of like the way that they connect that hardware or the way that they optimize where that hardware lives.

9:08But then we hear about some like obvious things. We hear about things like quantization. Quantization is actually another lever that you can pull. You can take your model and quantize or reduce the precision of the numbers in the sort of weight matrix to make your model take up less memory. That is a thing that you can choose to do. There's some reasons why you would expect that to have an effect on the range of this function. But there's other things too. And there's actually quite a lot of these levers. And some of them get talked about a lot. We get a lot of talk about optimization, but we don't get a lot of talk about optimization at large-scale inference.

9:50And so as an example, I think a lot of your listeners will be familiar with Atom. If you just hear the word Atom in the context of neural networks, they're probably like, oh, I know that's an optimizer, but what about shampoo? And what about other ones? And so there's this other question around, okay, well, what about optimization at different stages of the pipeline. You can optimize for serving inference. That's neat. But what about training inference? There's optimizations to do there too. And that also has an effect on these functions that we've been talking about. You can optimize things like sort of like, what does your job look like?

10:31A lot of people are now familiar with KV caches. This is a thing that like now is a little bit like, you know, familiar for everyday parlance. And some people may know about sort of like caching those KV values for subsequent conversations. They may know that when you ask Cloud Code 15 questions in a row, that some of those vectors during the sort of like forward propagation are stored and they're stuck there so that when you call it the second time, you're using less input tokens. That's neat. And I'm glad that people know that now. But what if you have 150 ,000 LLM calls that are all kind of similar and they only share part of their prefix.

11:16What about that? Can you do anything there? The answer is actually yes. And so I think these are the sorts of things that like they're all hiding behind this chart, this chart of Charles's chart. It's getting into like a Nickelodeon TV show. But like, you know, this amazing chart is just the sort of like the range of this function. But there's so many dimensions in the domain that people don't talk about. And so if I had to say like now post hoc, what's my revisionist history of like, yeah, I'm real smart and here's my track theme. It's this. And of course, like, you know, like I didn't think of this ahead of time.

11:56This is stumbled into by the things that I didn't understand. And then my friends didn't understand as much as they wanted. Right. And so you're really calling to light the fact that like not all of the levers get their time on stage, get their limelight. Some of them are just way more sexy and talked about than others, and others are just still undiscovered because maybe they're less glamorous or they're more like an edge case. And so you also duly saw your opportunity to bring people together and educate, like you said, as like a responsibility to shine a light on some of these less talked about, less understood things.

12:34Because ultimately, then you could start then, you know, a whole new cycle of deeper study and investigation and discussion. And ultimately, instead of recycling ideas in new ways that we've already been talking about, going out and finding some of these new unknowns. Which, to your point, there are just so, so, so many of. It's impossible really to go out right now and use these tools and not feel like everything you're doing is like some mini research project. to an extent you just feel so compelled to study and record everything you're doing because there is so much new to learn. So in that role as someone curating that kind of education, and then we Zoom out, and we're at this really great conference.

13:17There's a lot of other tracks here where everyone is going to come on site and learn about these very dedicated things. What are maybe some of the other stuff at the conference that you've really got circled or That's the next big thing. Anything that's caught your attention or what's on the horizon that we're all going to be talking about that you think will echo beyond data counsel, AI counsel, as it is? So last year, I was a track host and a speaker, actually. It's kind of a weird coincidence, but I had a talk that I was really excited to give, and that was called Failure as a Funnel. And that was talking about sort of like evaluation for data science agents.

13:53And as you know, I am hell-bent on data agents. And so since last year, where we did get some discussion of data agents at Data Council, there have been even more developments. In this crowd, we all know that there was a step change in capability in the models, especially around coding. with that step change in the models. We didn't see quite the same step change for data agents, but we did see sort of like some of this, we could draft off of it a little bit, as you know, because a lot of data agents are ultimately coding agents wearing some additional armor. And so what I am really keen to see this year and the sort of talks that I'm going to make a point to go to are the ones that are talking about sort of like, data agents and what's changing so rapidly in the area.

14:49The other sort of sub theme that I'm looking for is more and more people are coming to the sort of awareness that designing for agents is different than designing for humans. When I was at Hex, I used to tell people a lot, what's great for agents is great for humans. And what's great for humans is great for agents. And this was to remind people that, you know, AI is not a silver bullet or a magic sword. It is this thing that like you have to put work in to make it great. And that was true for quite a while. And that maxim is still helpful to remind people, put in work to give the agent great context like you would a human and you will have more success.

15:30But if I were to put a little like asterisk on that maxim that I used to like wield, the asterisk is okay. But also, you can do some sneaky shit when you're designing for agents. And that sneaky shit can make the agent have even more superpowers. It can let the agent do crazy shit that you would never expect a human to do. But if and only if you actually give it that capability. As a quick example, you hear about search and you hear about search for agents. This is sort of like an emerging trend where people are starting to realize that the way humans search for things is quite different than the way agents search for things.

16:17How can you start to, I sometimes call this ergonomics, like agent ergonomics. How can you build the agent ergonomics in a way that the agent feels really, really natural to do this shit that you would never do but can really level things up? So that's a sort of another emerging trend that I'm very excited about, and I'm specifically looking for talks that address that. Yeah, to find those like secret little hidden things that we're discovering about how these like rocket fuel things that you can hand to the agent to give it some superpower that maybe is a non-obvious or is something that a human, if it were given that same tool, would not be able to wield that same way.

16:58I think that those are the biggest opportunities, what you've highlighted and like this, like almost like with this like devilish glee of like, and if you give it this one small thing, it's just a whole night and day difference. I encounter that all the time. There's been things like understanding how we as humans make tasks and collect information in those tasks and then utilize and drag them around and understand them is so spatial. And it's so burdensome in a way that it obstructs agents from being able to effectively work in that space with you. So it's like non-obvious that if you strip that back to its barest base forms and you make it something small and fast and light and predictable, deterministic for the LMTUs, they are great at being right there in that task management space with you.

17:43They just need to be given, like what you said, the right ergonomics. And you're so right that like not everything that works for humans ultimately ends up transferring to them. And so there is a bit of research and discovery. Like what I'm talking about at the conference is about like from my background in teaching and like pedagogy methods for having a classroom and creating environments where students can learn in structured ways together and by themselves in a way where you ultimately remove and remove support. And how this mirrors what's happening with harnesses, with agents, how we're creating these environments for them, we're setting them up for success, giving them the tools to learn and to adapt and slowly taking away the support.

18:22And so, you know, there's a lot of, you know, there's hundreds of years of education research that can maybe be used as a platform to build what is ultimately finding these new non-obvious things for agents. So really great mindset. That sounds like a fun talk. Yeah, I'm stoked to give it. It's going to be a blast. You know, obviously, all of this gets up on YouTube, so people can check out your whole track. They can see what I talked about as well. So I'm definitely excited to maybe shine some light on some new ways to work with agents. I've definitely been doing some, you know, when I share it with folks, like some non-traditional environments for how I get stuff done.

18:57That's awesome. I just wrapped up teaching my AI engineering class at Rutgers. And this is a real applied AI engineering class. And I can tell you, if you ever want like a real adrenaline rush, try to teach a topic that's really only been around for two years and changes every month to graduate students that want jobs in that domain at a sort of like normie R1 university. Have fun with that. It's a real trip. We'll say that. I had to like rewrite lectures because like something new happened in the industry. And I'm like, well, shit. Like in two weeks, I'm supposed to teach this and this is kind of outdated now.

19:38Allow me to update this. The God, the MCP slash tool lecture was don't even get us started here. How many times did he come alive and died? And we're going to talk to be talking about that year in just a moment, too. So I'm I can only imagine, you know, we actually have a similar pain sometimes here on Dev Interrupted where and where it will be like recording something for our Friday news segment. And like literally while we're recording, like the biggest news of the week will drop. And now suddenly we have to like completely repivot what we're talking about. It's hard to think about that on an even bigger magnified education scale.

20:11Like I have to equip this extremely smart and extremely capable person who probably knows more about this tool than I do to go out into a world that's changing faster than I can understand it. Like talk about a challenge. It's fun. So let's get into a little bit about, you know, in the industry, the trends that are happening, the things, the currents under the water of, you know, what's happening right now at AI. at AI Council. And so recently, I want to pivot to this piece that we talked about a bit on Dev Interrupted. We covered it on a new segment. And you talked about like our software's next epic, our investment into RipGrip.

20:45And this is kind of like a part science, part data science, part satire manifesto about the state of the industry and where things are going. And, you know, you've been talking about as well on LinkedIn pretty extensively about tracking the death of technologies in our industry and the way that we talk about them in the news cycle. You know, I really want to ask you, Brian, because you feel like you're the expert on it. So why is it that in AI, a technology can die 12 times a single year, but still be the main topic of a conference? Yeah. So I think ultimately, the blessing of AI's awareness is that there is the likelihood that more people will benefit from these new changing technologies.

21:36It's not kept in the ivory tower. It's not locked behind an extreme price moat. And people like my mom and my sister might interact with AI and actually get benefit from it. The curse of its visibility is that the TAM is so large for attention that everyone wants a piece. And so one thing that I talked about, and this is a little bit on the spicy side, back in like 22, 23, and even a little bit into 2024, I was sort of early on this opinion that DevRel was killing software and that like developer relations and specifically the craft around like attention optimization, and I don't mean Adam, was really like harming the industry.

22:39You see these like amazing examples of people like Kelsey Piper. Like everyone can look at Kelsey Piper and be like, he did so much for us and we owe him so, so deeply. and then you can see sort of like a new generation of people at his level like Charles Fry who I mentioned earlier as like a standout case unfortunately they're not all like that and I think there's a lot of people that are focused on rage bait and rage bait gets clicks I think no one can deny that if you're an aspiring DevRel with low morals I got a playbook for you and that's rage bait and so ripgrap was meant to be rage bait the rage baiters um that was very much the intention with that like piece but it also did come from a very earnest position of curiosity which is like what like it seems like everything's dead all the time are things actually dead all the time like am i just in some filter bubble that like makes me feel like a myopic moron kind of yes but also not only yes um there's also a very real thing, as you can see in the data.

23:53There's a lot of people calling a lot of things dead. There are a lot of things that people call dead very rapidly and then fall off that. You made this call out to dying 12 times in a single year, whether that's a mathematical aberration or just a reflection of the fact that we do see these hype cycles. This is not new. Like, Hype Sickles are everywhere, pervasive, blah, blah, blah. I'm the last person on earth to be an expert on sort of social media influence. But the one thing I will call out is I do feel like it is hard for me when I see my students ask me questions in the middle of me presenting the tools lecture if we care about function calling anymore.

24:40Because now coding agents can just use the CLI to do everything. Now, you and I and a lot of your listeners are going to understand the nuance. We're going to know that when people say CLIs are really good for agents and sandboxes, we know that, yes, the sandboxes can be valuable. And yes, CLIs are really powerful because coding is a verifiable task. And so training models to get back at coding, getting better at coding is a very tractable thing. But we also know that there's a shitload of applications of language models where plain function calling is super valuable. And then separately, sort of like these connections of tool use.

25:24And then MCP is more about authorization and distribution. And we understand this nuance. But unfortunately, attention is being highly fought over. And so everything is pushed into these extremely spiky statements like MCP is dead. And so even some of the people that I used to really like admire in terms of the way that they presented tech in sort of like on Twitter, for example, I think now are making regrettable comments that piss me off. Like Levels.io saying MCP is dead because he doesn't have the MCP shaped problem. Come on. So I think ultimately, I think what I wished people took away from both RipGrep and FindAll, and if you haven't found the Easter egg of FindAll, I encourage you to go to ripgrep.com.

26:16Um, the kind of idea here is to find people talking about things and sort of desensitize yourself a little bit to these hype cycles, but then also have an opportunity to understand the dialogue and a more meta perspective. Look at the whole field. Because remember, if you say, ChatGPT killed Grok, and then a week later, somebody else said, Grok killed Claude, that gives you this sort of like, teach the whole discussion or argument perspective. And so if there's one real goal with RipGrap, it's to give everybody a moment to look at the entire situation. Yeah, absolutely. And you're really calling out here the commoditization of information within our industry, the social media-ification of all industries on platforms that we post on, that we share discoveries.

27:16And it's unique to our times, and I think sometimes we forget that. If you were to take the same social media world that we have and plop it onto the dot-com bubble, you probably would have gotten a lot of cycles of a lot of people calling a lot of things in the early web dead. And you're exactly right that what this does is it erodes all of the nuance of the conversation so that you get the polarized, like you said, spiky view, the most magnetized on either end of that spectrum kind of thing you could say. And like I said, it erodes the nuance. It prevents us from having constructive conversations.

27:51And it ultimately, it kind of like dampens the ability to share. And so it's like a really tough environment sometimes in that world to like actually navigate like what's real and what isn't. What can I act on and what can I not? And so I think this is a really healthy reminder for everybody that in any kind of thing that you're reading or discovering, if it's constantly a hyperbole of the situation, then you're probably being robbed of the nuance of what's actually happening, which you can only really learn by maybe trying to find counter sources or supporting arguments. Where frankly, at this stage that we're all in, I really just beg everybody, just go to ARCSIV.

28:28Just read the white paper that we're all talking about because they're not that long. And honestly, in some cases, you are going to get much more out of the abstract than you would from any scroll by on LinkedIn. So all of that said, I don't even want to get into how you got ripgrep.com. But it's like, where could you even get that URL from? Surely that was taken. Shockingly, no. So one important caveat on this. So ripgrip.com, both the domain name and the original idea of building a tool for this were Adam Conway's brainchild. He texted me and was like, okay, so this thing we've been talking about, here's how I think we could do it.