Zapier’s Mike Knoop launches ARC Prize to Jumpstart New Ideas for AGI

26:39They all got handily beaten as sort of scale emerged, and they really didn't add to or understanding of how to build useful AI systems. And so what's one of the common questions I've been feeling last couple weeks is like, What's different about ARC? Is that likely to just happen here again? And I think the reason why we're likely to see something much more useful, assuming we get a really good solution to ARC from the first grand prize win, is that the number one problem at the application layer, and we see this with Zapier too, with our new AI bots that we launched a couple months ago, been surprising to me in how that has gotten adopted actually by our users.

27:19There's like how you kind of describe it. There's like concentric rings of use cases that you can use AI automation for. And what we're seeing is people are sort of restricting the use cases for the AI bots, where they're sort of fully automated, totally hands off to the use cases, where there's sort of a very low need for user trust. Or where the sort of, let me say that different way, is if it goes wrong, it's not cast -strapped looking bad. So they deployed for use cases like personal productivity activity or team -based like workflow automation, things where, you know, if it's wrong, or it's right only nine out of 10 times, or it takes me, you know, maybe a couple days to like really work with the system to like do the prompt engineering to steer towards getting maybe 95, 99 % reliability.

28:01That's acceptable because the risk of being wrong is just, you know, quite low. In order to get much higher up and expand the number of concentric rings to, you know, moderate risk to high -risk deployment scenarios where we want these systems working autonomously, we're going to need that the main thing that is missing is user conference and the exacting nature of what it can do and what it can't do. This is what Zapier Core Classic gives us, right, is like a deterministic engine to execute automation. So you know, once you build it and set up, it's going to do the exact same thing every single time.

28:29But that's also what makes it fragile and hard to use. And on contrast, you know, these AI Core -based LM systems that are total autonomous of the opposite set of trade -offs, right? They're much easier to use. just steer them, guide them, and fix them entirely to natural language. But because the accuracy is still in exact confidence is low. And I think that's what art gets us. So solution to art at 85 or 100 % means that you've written a computer program that can generalize from like very simple core knowledge priors to solve with exacting accuracy at high percent reliability. These like site unseen puzzles.

29:00And I think that that tool, as a, there will be a new tool in like the program is still in terms of building products and building systems that can achieve that same thing. We're two weeks in, I think, to when you launched ArcAGI Prize. What have you learned so far? What types of people are working and competing on this? Is it like the pedigrid researchers or the big labs? Is it scrappy hustler types? Like, who's competing? How many teams are seven -thing solutions? Yeah, let's see. So the response by the way after launch was phenomenal, was much bigger than we expected. I think we had, we were trending on Twitter twice, during the launch week, the number one Kaggle competition in the world, over a million social views, I think, over all the launch channels.

29:42So just a very phenomenal, I'm really thankful for everyone who helped sort of promote and helped share Arc. Hopefully we actually like can get a solution here and some short time. I think that like the most interesting thing about the folks that are working towards Arc is probably a historic glance here and then what I've seen over the last two weeks. So the historic Lancers is most of the people that have worked on ARC or outsiders to the field. They, like, this is not actually the first year that there's ever been a contest about. There was a past competition called Archithal and I was not sure what's smaller.

30:12I was hosted out of this lab 42 AI lab in Switzerland. And so last year there was actually 300 teams that worked on trying to be dark and again no one had sort of beat it. And almost to my knowledge, all of those teams were effectively individuals or effective, you know, outsiders in some way. They're not, you know, people with big AI labs, their focus with backgrounds and engineering, mechanical engineering or video game programming or physics. Folks, they just kind of got curious and interested in the problem at hand. And I actually think that's more likely than not where the breakthrough for ARK is going to come from.

30:48I think it's going to come from an outsider. Somebody who, like, somebody just thinks a little bit differently or has a different set of life experience, so they're able to cross -pollinate a couple of really important ideas across fields. That's one of the reasons why I put as much money as we did in ARC prize. I felt like the progress was idea rate limited actually. And one of the best ways to sort of increase the amount of ideas is to try and blow up awareness, which is what the launch kind of did. Over the last two weeks, I've kind of seen like two probably like camps of people, at least on Twitter, Emerge.

31:19I think there's one camp of people who are sort of the, you know, they're in it for the mission. They grew with the underlying concept. they think that we do need some new ideas or excited to try and figure out what those are. And then there's a second group of people that are sort of like, I'm gonna prove you wrong. LMs are definitely enough. Scale is definitely what we did. And I'm gonna do my best to go feed this benchmark just using existing off -the -shelf technology and sort of prove you wrong. So I'm actually quite happy for both those camps to exist. One of those approaches is currently up in Litherboards.

31:48All right. So yeah, we can break some sort of news here. So this week, this Thursday, we're launching. I guess when this comes out, I'll have launched just a couple days in the past, a brand new public task leaderboard. We talked about how ARC doesn't allow internet access and there's compute limits. I know personally how unsatisfying that is to not be able to use frontier models though. I also want to know how good can GPD -4O, how good it can collage on it, like do against this benchmark. And also because no compute owner, it's also a bit of a barrier to entry. You have to use up -and -source models, you have to do, just quite a bit of engineering or get to do before you can start just testing and experimenting.

32:26So we're gonna be, we're launching a new public task leader board. It's gonna be a secondary leader board. We're gonna be committing about $150 ,000 for a reproducible fund towards this secondary leader board. And we'll be officially part of the competition this year. You know, we wanna, we wanna like maintain that aspect of like assurance on, you know, cheating and contamination overfitting with the private test set. And that's also the test set that has sort of the most empirical evidence against over last four years. But the secondary leaderboard is going to allow folks to basically submit scores towards it against the 400 public tasks set.

32:59And we'll verify and reproduce the scores locally to sort of ensure good like fitting with the approach. And we'll publish that. And you're right. I think the top score or one of the top scores on that is this guy Ryan Greenblatt. He came out a couple days after the competition launched with a pretty interesting novel approach actually towards beating it and he's using GPT -40. And but not just GPT -40. I think the interesting thing is he created a like an outer loop around 40 where he is using 40 to generate Programs or sample from GPD 40 these like program these reasoning traces to Beat the tasks or identify the patterns then testing these patterns against Attempts against the demonstrations that then finding the one that works on applying it and that's that approach seems to be getting in the like like low 40s, maybe 40 % or 41, somewhere in that range.

33:51And I think it's pretty interesting because I think it's, you know, someone might look at that and I think in it, sort of at first blush say, well, isn't that evidence that skills all you need? And I do think there is something interesting there, right? It's like it's showing that, hey, the more training data these things have, the more sort of, you know, programs that they can spit out that might be kind of right. But it also shows that I think that new ideas are needed still. Like this outer loop is novel. Well, that might actually be Frontier LLM reasoning that Ryan published. And we're going to make all the approach whenever we put similar dark prize.

34:26We're going to open source all the code for all the reproducible solutions so folks can take these and apply them and try to reproduce them. And so we're using private closed model, open source models for the closed private data set. But yeah, I do think it's pretty interesting. How much innovation do you get when you... How much innovation we've got over the last few weeks? It's just a result of putting even just the awareness against the public. Are the folks at the big research labs like why are they not working towards this benchmark? Because it almost like when you explain the benchmark it seems so clear that obviously this is the thing you want to solve.

35:01You don't want to solve the memorizing of the textbook use case. Why do you think the folks at the big research labs aren't trying to solve this benchmark? Or are they? So I am aware of a handful of big ILAbs that have tried in the past several years ago. So this was perhaps at smaller scale with weaker models and things like that. One of the things I would hope is that actually more do you in the future actually love to see if we could make ARC AGI an actual measure on some of the model cards that get reported against future models. I think that would be a really cool thing. We're willing to do it.

35:34So if anyone is listening to this and wants to reach out and make that happen, more than happy to work with them and find some way to do that. If I had to guess, well, let me say, let me not guess unless, say, but what I have more sort of confidence in. I've been serving, once I got exposed to Francois to work again and was sort of deep, thinking, thinking deeply about our cage. I started serving a lot of my like friends and researchers and SF in the Bay area about had they heard of Francois and they heard of Arc. And Francois was a pretty good name recognition because he's really big on Twitter, been big on Twitter for many years.

36:11Probably nine out of 10 people I talked to like knew who Francois Shullay was. Maybe one in 10, two in 10 had heard about the Arc age at Yval. And probably half those were confused because there's like five other A .I. Yval's called Arc. And I had to like sort of do some, so I had to like, you know, disemphiguate with them. So it had really low awareness. This was one of the first things I asked for and so about when I met him for the first time in person this year I asked him why do you think that is why do you think you have such high -werness but arc has such low awareness and His answer effectively was That it's hard, you know the way that benchmarks Get like gain popularity and notoriety is we make progress towards them right researchers are working against it somebody has an idea, they have a breakthrough, they publish that in a paper, that paper gets picked up and cited by others, that generates awareness and attention.

37:04Other researchers say, ooh, interesting. Okay, something might be possible now on this really hard benchmark. And so you get the snowball effect of attention. And because ARC has endured with very low rates of progress, in fact, decelerating progress over the last four years, I think anyone kind of, in the lab looking at that, I would just say, well, maybe the time's not right for you. Maybe we don't have the idea set in the world. Maybe we don't have the scale we need yet in order to sort of beat this thing. And it looks like a toy and it doesn't like, you know, I don't fully understand why, I don't get the necessary importance or how it's qualitatively different.

37:37I haven't just spent that much time. I got a million other benchmarks I could use. And, you know, I think that's somewhat of the dynamic that has existed in the past. And it is one of the, again, reasons why we launched our price, right? I think there are some, there's lots of like market tools you can use to shape markets and shape innovation. And I think prizes do have a narrow spot where prizes can be outrageously effective. And it's where the idea is small. And it's like idea rate limited. One person or a small team can make that breakthrough. And it's very quickly and easily inspecable, reproducible, and built on top of it rapidly.

38:15And all those boxes got checked. And yeah, one of the reasons why I decided to go to our surprise. And you mentioned curiosity around AI or AGI dating back to college that was sparked a few years ago in the context of Zapier and has kind of been nurtured ever since. Beyond the curiosity, I'm curious why this is important, meaning if you could paint the picture of what life looks like for the world post -age AI, where we've defined it as the ability to efficiently acquire new skills. What do you think that version of the future looks like? Like, why is this an important thing to solve? The thing that I feel like I have a unique insight into at this point, having spent a lot of time thinking about Ark in this AGI definition is I suspect the advent of AGI is going to look very differently than most people expect.

39:10Especially of the group who are in the camp that AGI is undefinedable because it's so mythical and scary or big or awesome that we can't even hope to ever define. It's just going to be this magical special thing. It turns out, or something that I believe quite deeply, is the definitions are really important. Because definitions allow us to create benchmarks. And benchmarks allow us as a society to measure progress and set goals towards things that we care about and what to happen. And this idea of efficiently acquiring skill, one of the, you have talked about a handful of times today, but one of the direct near term things that you get from that is you get systems that can do exacting accuracy generalization from a small set of core priors and apply it towards novel solutions.

39:58That is again the number one problem that rate limits AI adoption for more real world use cases today. And so that's the, that's what you're going to see, you're going to see basically like the application layer of AI get like amazingly good at accuracy consistency, low hallucination rates, which is going to allow us to use it in a much more unfettered way and a much more trusted way because of because of like the underlying way that in which it's built. So I think that's like, and the reason I think that's important,

40:32I think that's the reason So what that set of capabilities is going to build on top of into the future, right? There's lots of unknowns, I think, about AI, AI, AI evolves beyond the actual inception moment of a system that can efficiently acquire skill. But I think it's going to be a much more gradual and incremental rollout, whether it's a lot of contact with reality as we build and engineer these systems, which is going to give us, as like a society, a lot of time to update based on what those capabilities, what it can do what it can do and make decisions at that point about how do we want to like deploy this technology where might we as a society say we don't want to deploy it for this set of use cases.

41:15You know, I think that's one of the reasons why I've been so sort of such a opponent I think of of open source H .I. progress with our prize is like right now I think the what I see happening is there's sort of this mythical story of a very bad outcome once we get to like super intelligence right. It's a very theoretical driven story. It's not grounded in sort of empirical evidence. It's basically based on sort of reasoning our way to like this outcome. And I think the only way that we can really, really effectively and truly set good policies by you have to look at what the systems can and can't do and then regulate or decide make decisions at that point about what what a can or can't do.

41:49I think anything else is sort of like, you know, you're cutting off potential really, really good futures way too early. And that's sort of what's happening I think with a lot of this early air regulation where, you know, I'll try to, you know, be the paint the good side of this picture. It's like, hey, I care, you know, maybe the risk of this bad outcome is so high in the future that we should like pause here. I think the risk of that is you've like trimmed off every possible good path of the future way too early. And the reason it's way too early is because we still need new ideas. We need new ideas for researchers.

42:19We need new ideas for students. We need new ideas from young people. We need new ideas from labs. Otherwise, we're actually, there's like a chance that we'd like never actually reach the like high degree of useful AI that we actually want. And so that's kind of my nuanced take, I think, on probably what the advent of AI looks like. I think it's much less likely to be a moment in time and much more likely to be a stair step of technology that we build on on top of past technology. And that creates a lot of moments to update beliefs based on what it can and can do. Do you have any predictions on when we'll cross 85 % on our price?

42:56You know, before the competition started, there was a, the first data scientist we hired at SAP, giving this idea a long time ago to stuck with Mickey. He said, the longer it goes, the longer it goes. And so it's this idea that like the longer something takes the more you should update your prior about, it's going to take longer. So coming into this year, my expectation was like, hey, at least three or four years, probably before we get to the grand prize mark based on sort of like past, basically in the past track record. Having seen what we've seen over the last two or three weeks, though, I think it is quite likely we get to 50 % during this competition period.

43:32I would be very surprised, you know, I'd be surprised in a good way if we actually get to the 85 % grand prize in this competition period. But I think it is not unlikely that we crest the 50 % mark before the end of our middle of November, which is when the contest period ends for 2024. before. And is there a good why now? Because people have been trying at this for five years now. And you're galvanizing interest around it. And now a lot more resources down the world are interested in AI and solving hard problems. But is there a good why now in terms of enabling techniques, technologies, et cetera, that's different now than five years ago when friends who have first kind of defined the benchmark?

44:11If it is true that deep learning is an important part of the solution, right? a deep learning guided program to this ascension, or a DSL that is generated on the fly through deep learning technique. If that's true, the world has a lot of experience now on building and engineering and scaling such systems over the last three or four years, and there's a lot more compute online, which brings the cost down into a territory where some of these things may have just been out of practical cost before. For example, actually, Ryan Greenblatt's solution right now how is maxing out our cost limits we're going to have against the public leaderboard costs $10 ,000 to generate the 8 ,000 reasoning sample traces from GPD40 that he then deterministically checks.

44:58And so that would have been a technique that would not have been possible three or four years ago in any way it's a guard. So I think if it is true that there's a minimum amount of scale, the necessary to be dark, I think, hey, we've gotten more of that in the last three or four years than we had when the first come to SRAM. And then I think the other why now is just, can't just largely due to awareness. If it truly is, actually let me answer in the opposite. Like I think the risk that it is, the reason we launched our prize is that it is actually not why now, right? It's like actually it's not going to happen is the problem.

45:32It's not a why now. It's not like oh the ideas are in the world we just need to like get people to work on it. The risk is that it's actually not why now. And not why now is I think a much more interesting story right around this like LM driven, you know, focused attention on sort of LM solutions only, the closed research due to the competitive dynamics, all of these things have like shifted and shaped attention away from the new ideas and towards scale, towards LM's, towards like, you know, yeah, application layer AI. And I think that we think we need some shaping, reshaping back towards the new idea set.

46:06So hopefully, The Y now is because ARC has now lots of potential to be seen. Do you think LLMs will be part of the solution? I'm curious what you think of it. It seems like in the big research route labs right now, all of the frontier research is around. Let's merge kind of LLMs with the insights that you get from the inference time computer and the Q -star, AlphaGo stuff. I'm curious what you think of that kind of direction of research. I, there's some like pretty interesting research I've come across with like transformers that the transformers are capable as an architecture of representing very deep deductive rates in change with 100 % accuracy.

46:45I think this is interesting. And the challenge with them is actually we just don't have the learning algorithm. Backpropagation is an ineffective learning algorithm in order to teach a transform architecture, a set of network weights that can do deductive reasoning with 100 % accuracy. And so I think it's possible that the systems that we kind of, or at least the core concepts that underlie language models have sufficient capability in order to do this type of reasoning. And we have not yet discovered the like algorithm that can train the model in the right way. We haven't quite discovered the right outer loop around the transformer that is going to do the program.

47:17Since this is engine or the DSL generator, I feel more confident in saying that like deep learning is almost certainly going to be a part of the grand prize in some way. I bet it's that it won't, I'm pretty confident it wouldn't be just like a pure deterministic program is going to be how it gets solved. I think transformers are effectively the technology that is the most, has the highest degree of awareness. In research like literature, there's a lot of hardware now that's going towards accelerating transformers, models I think actually just a roller that was like an ASIC that got announced to recently that's trying to accelerate like the transform architecture.

47:52So do they agree that actually like some degree of like scalar computers necessary to be dark? I think those are like things that I would say I'm bullish on sort of the transform architecture. sure though, I would point out that the search base of alternative architectures is quite rich. Right? We've had maybe like nine or ten now mainline architectures for transformers to LSTM, CNNs, RNNs, XLSTM, state space models. This would suggest that the search base of those architectures is quite rich actually and they all have like slightly different varying properties. So like I think it's certainly possible that someone comes up with some innovation there.

48:27I'm less confident or like bullish that LOMs in their exact form are going to be part of 85 % solution though. I would think it probably like a subcomponent of the architecture instead of the entire application system itself. When somebody does ultimately hit the 85 % threshold, what do you hope they do with the solution? Like what would you like to see out of that person? other than submit it to the leaderboard. So this is one thing we didn't talk about a ton, but one of our prizes goals is to accelerate open progress towards AGI. So we are going to be requiring that in order to claim the prize money, you do have to publicly share and reproduce publicly share reproducible code and put it into the public domain.

49:11And this goes for both the public leaderboard and the official competition leaderboard as well. You know, this is with the spirit of trying to re -accelerate open progress again so that we have research in small ways out in public that other researchers can build on top of and hopefully steer supper would actually, actually, billing a GI and not getting stuck in sort of the plateau that we're at that we're in today. But I think that's probably my first pick up. I've actually seen a handful of people online that have said, hey, if you've got a solution in our KGI, I'll give you this like, you know, a million dollar offer.

49:43or like, I'm pretty much like a real person. Yeah, exactly. Which, you know, at one hand, I'm like, yeah, okay. That's kind of interesting. But, you know, in the other end, I'm kind of like, you know, I think that's like great awareness. And it shows actually like the importance of solving this. I think more people are starting to become aware of the lack of progress, lack of frontier progress. I think Arca is kind of becoming a lighting rod for folks that want a actual measure towards this, I think growing sentiment in the field today. Should we close out with some rapid fire questions? Yeah, let's do it.

50:17Okay. Who do you admire most in AI? I mean, Fransfalsha lays a bit of a cop, I'm answering. I wouldn't have go find a dark prize if I didn't admire and respect his work over the last four years. I mean, I think the two people that I have learned the most from, like directly, have inspired my own belief and work, Rich Sutton and Shalei. Both of them published papers in 2019, Rich Sutton published The Better Lasting, which I think is fairly well known on the industry at this point. I think his idea set is quite right there with maybe the one aspect that the one aspect that has not yet had scale applied to it is architecture itself.

50:53We certainly have unbiased search and learning applied on the inference side and the training side, but every architecture still is a very human handcrafted story and journey to it, which is an important insight, I think, about that. And then, I'm measuring intelligence from Shilay in 2019. And I think both of these papers are in like, or I guess maybe, sentence was more of a blog post. But both of these pieces of writing, I think are very important because history has proven the right as time has gone on. Language models, transformer scale has sort of shown some ideas to be even more true than they were in 2018.

51:24And I think the endurance of ARC has shown Fratz -Wastephanesche rage eye to be more and more true as time has gone on. What is your most contrarian point of view in AI? I feel like everything we talk about today, we'll be able to go to group with the Hanzo. All right, we'll count it. Like, Gale is not all you need. Well, new ideas are needed. What's your favorite AI app, other than Zapier? Let me look and see, what do I have, I have a handful. I think I'm not gonna, like, surprise you with anything. So I've got chat to BT, perplexity and cloud, and I'm a paying user of all three of those services.

52:06is, you know, one interesting thing actually I'll add, I have gotten way more value out of language model based tooling over the last, like, call it six months. Then I ever did in the first aspect when I was starting to start work on it, Zapier. And it's because one of the things are really, really good at, the thing that they're perhaps like best at is summarizing tons of unstructured text and helping be like an educational tutorial for you. So it's significantly ramped up my learning rate on actually building with AI, built these fundamental different architectures. Starting to do model training, something's after you're less than done, but I started to do myself over the last six months to get a sense of that type of work.

52:51Language model and AI tooling has definitely accelerated my learning process. All right, last question. Let's do something optimistic, something that we can all dream about. What change in the world are you most excited to see over the next five or 10 years as a result of AI? I think the I've always wanted to like live in the future. I think that's maybe a something that has always driven me towards like working on like frontier tech. I've always you know, bought the latest gadget always tried the latest app. I think it's led me to work on Zapier and AI and And it's one of the reasons I'm working a GYR now is because I think it's like the biggest thing that you can potentially have an influence on trying to pull forward that future.

53:35You know, I personally get, I think one of the things that feels very limited for me right now is that with the nearer form of AI that we have, if we never get a GI, What that will mean is that we will always be rate limited on developing things by the human that's in the loop. And that means we will never have AI systems that can invent and discover and sort of innovate alongside humans and really pull forward and push forward the frontier in I think a lot of really interesting ways. I understand more about the universe and then discover new pharmaceutical things. You know, in discovery physics, discover how to build AI.

54:26You know, I think we're always going to be rate limited by the human today. And I think if you just sort of care about living in the future and you want to pull forward the good aspects of the future, some form of age eyes is I started to do that. Awesome. Thank you, Mike. Thank you both for having me.

55:08Swings and