Zapier Co-Founder Mike Knoop on category creation, API evolution & AI architecture | E1769

27 Jun 2023 · 1 h 8 min

Ask about this episode

Ask anything about it. ChatGPT or Claude reads this page and answers with the times it was said.

Connect VO and ask about every podcast you hear, including the moments you saved. Add to ChatGPT · Add to Claude

In short

This Week in Startups - Episode 1769 Summary

Episode Title Zapier Co-Founder Mike Knoop on category creation, API evolution & AI architecture

Key Guests

  • Jason Calacanis (Host)
  • Mike Knoop (Co-Founder and President of Zapier)

Episode Overview In this episode, Jason Calacanis interviews Mike Knoop, the co-founder of Zapier. They discuss the inception of Zapier, its evolution in the tech landscape, the impact of AI on automation, and the future of API integrations. The conversation touches on Zapier's remarkable growth, profitability, and unique business model.

---

Key Discussion Points

  1. Zapier's Origin Story
  2. Inception: Founded by Mike Knoop, Brian Helmig, and Wade Foster during a Startup Weekend in Columbia, Missouri, utilizing their experience with APIs.
  3. Initial Thesis: To make APIs accessible to non-technical users by reducing the complexity involved in integrations.
  1. Growth and Recognition
  2. Inflection Point: Around 2014, Zapier gained significant recognition, resulting in profitability and a burgeoning user base.
  3. User Engagement: Users expressed gratitude and excitement for the product at conferences, indicating strong market demand.
  1. Business Model
  2. Revenue: Currently generating over $150 million in annual revenue with a pricing model starting at $20/month.
  3. Profit Sharing: Unique model where employees were initially compensated through profit sharing before moving to equity options as the company grew.
  1. Evolution of App Integrations and APIs
  2. Market Landscape: Early days involved searching for integrations yielding developer documentation; now, users can find robust solutions through Zapier.
  3. Focus on Reducing Friction: Zapier's mission includes making automation easy and accessible to promote productivity.
  1. AI Integration and Opportunities
  2. AI Workflow Incorporation: AI is becoming crucial in automating tasks, prompting Zapier to enhance its platform to integrate AI tools.
  3. User Adoption: Approximately 20% of Zapier employees have already integrated AI into their workflows.
  1. Challenges in API Management
  2. API Regulations: Discussion around companies like LinkedIn and their restrictive API usage and how it affects integration tools like Zapier.
  3. Market Alternatives: The tightening of API access by major platforms leads to increased reliance on integration services to fill gaps.
  1. The Future of Work and Automation
  2. Workforce Efficiency: Jason and Mike discuss the potential of AI to make teams more efficient, raising questions about the future workforce landscape.
  3. Job Displacement vs. Opportunity: While automation may reduce the need for certain roles, it also creates opportunities for new kinds of work.
  1. AI Regulation and Development
  2. Thoughts on Regulation: Mike expresses a need for thoughtful AI regulation to balance innovation with safety.
  3. Open Source vs. Closed Development: Concerns voiced about the shift toward more closed systems in AI development and its implications for progress.

---

Key Takeaways

  • Zapier's Impact: The platform has democratized access to automation through user-friendly interface and integration capabilities.
  • AI's Role: AI is seen as a game-changer for automation, promoting efficiency but also raising regulatory and ethical questions.
  • Future Trends: The conversation emphasizes the looming efficiency improvements in businesses due to AI and automation, with a cautionary note on potential job displacement.

---

Episode Links

  • [Zapier](https://zapier.com/)
  • [Follow Mike Knoop on Twitter](https://twitter.com/mikeknoop)
  • [This Week in Startups](https://twistartups.substack.com)

---

This episode of *This Week in Startups* provides valuable insights into the intersection of technology and business, focusing on automation and the evolving landscape of APIs and AI.

Written by AI. May contain mistakes. Listen to the episode to check what was said.

Hear the part that matters, and keep it.Open this episode in VO. Double tap your headphones to save a moment as you listen.
Get VO free

Transcript

Automatic transcript. May contain errors.

0:00There's really not that many great local Italian places. Oh, there is a good pizza joint called Centro Pizza on Broadway. Okay. So, you know Broadway and Burlingame? There's like the street and there's the Great Street. Yep. Broadway has a place called Centro Pizza and they make brick oven pizza and it's amazing. It's the best pizza in the peninsula I've found. Centro. Does it good? Yeah. It's pretty great. Okay. There's your cold open, everybody. This Week in Startups is brought to you by Embroker's Startup Insurance Program helps startups secure the most important types of insurance at a lower cost and with less hassle.

0:39Save up to 20 % off of traditional insurance today at Embroker.com slash twist. While you're there, get an extra 10 % off using offer code twist. Lemon.io Need to speed up your product development without draining your budget? hire vetted engineers from europe at lemon.io go to lemon.io slash twist to get 15 off for the first four weeks and eight sleep good sleep is the ultimate game changer now you can add the pod cover to any mattress go to eight sleep.com slash twist to check out the pod cover and get 150 off at checkout all right everybody welcome back to this week in startups my guest today is mike anoop he's the co-founder president and head of labs at zapier if you don't know zapier i'm about to make you happier zapier is an amazing tool i discovered god it's close to a decade ago that helped me do really interesting automations between google docs my email you know basic stuff if somebody signs up for my newsletter put them into this google this uh google sheet if it's somebody's in this google sheet pipe it into my slack room when somebody signs up for launch fund four as an lp and i've been doing these automations over the years uh and i train everybody on my team to learn how to use zapier notion coda the google docs suite um and zapier and all these products because you can automate so many tasks your partner and your co-founder wade's been on the pod i think twice in the past uh but mike is this your first time on the pod i think so yes uh first time thanks for having me well i just wanted to say also congrats i i mean when people saw zapier and i guess your contemporary if this then that was a there was like a couple of companies trying to do this and everybody was like yeah that's a niche business it is not a niche business explain to everybody when you started the company then we'll get into all this ai stuff which is why i wanted to have you on because ai changes everything with what you're doing and we're going to do a bunch of interesting demos and talk about how startups and everybody can be using ai and zapier to plug everything together but when did you when did the company start and then when did you realize that you were onto something big and then what's the footprint of the company now because i hear all kinds of numbers like somebody told me you're making over 100 million in revenue i don't know if that's true but where's the company at today and where did you start yeah Yeah.

3:15Well, I think it surprised me as well, in terms of how big the business could get. When we started it, Brian Wade and my two co-founders, and we got started back in Columbia, Missouri, a small college town at the University of Missouri. We got started at Startup Weekend. So that's what brought the three of us together. And we were all working with APIs in our day jobs and side jobs. I was one of the early moderators and big users of the Facebook API when it came out in 2009, 2010. So you're all using these APIs in contract work, and we're just doing the same things over and over again with them.

3:48And I think Brian was the one who pitched the idea at Startup Weekend. And the idea was like, hey, there's this huge wave of APIs. They're really cool. Wouldn't it be even cooler, though, if more people could actually use them? Because they still have a very technical bench to be able to take advantage of them. And that was the thesis. And as you started looking online, and if you go online and search around for using these APIs, or more commonly, you'd search for how do I connect these two services together. All you would find on the internet back in the 2010, 2011, 2012 era was basically developer documentation.

4:22You'd find a Stack Overflow link of like, oh, yeah, great. Here's a bunch of code you can use to connect Salesforce with Gmail, for example. Or HiRise with Basecamp or something. Popular tools back in the day. HiRise and Basecamp. Yeah. Yeah, really old school there. You do that same search today and like the sort of landscape of the results, it's totally different. But that was sort of the sort of landscape that it looked like back then. And I think our observation was, okay, well, you see all these forums where folks are almost begging the vendors for integrations. You'd go to say the high rise forums and just see these forum threads with like hundreds of their users begging for like, hey, can you add this like random XYZ integration?

5:00And it never really made sense to them that may add more than one, two, three, or four, the top requested ones, just because the long tail, it's a bit of an N-squared problem. Every new app that gets added, there's an integration that wants to get integrated with it. And we realized, well, okay, we're probably never gonna capture the direct native integration experience. The vendors are gonna build those directly themselves, but we can provide this ubiquitous platform. Maybe we can get 5%, 10 % of all of the integration markets out there because we'll be able to service a set of users that just the vendors themselves or never we're going to be willing to service.

5:33That was the original thesis that like, hey, this could be more than just like a small niche SaaS company. Over the first few years, we got started building. I think one of the things that really changed my perspective of what the business was, because I always thought for a long time, I was actually not a personal user of Zapier for the first couple years. I was building for our customers. And I always sort of saw it as boring productivity software. and that was like that's how i viewed the software you know it's cool great business like boring boring b2b software and what sort of started to change my mind about it was several years in we started going to a lot of these like conferences with our users and with partners and we started having a lot of people coming up to us and like sort of like shouting our name like shouting giving us huge high fives and just being so effusive like thankful there was like passion from the user base yeah it was weird that was palatable right of the business what when you double clicked on that because this is really the key you had what um we call in the industry market pull not only were people were looking for this product and this is beyond product market fit you had people searching the internet how do i integrate these two things how do i create some glue how do i solve this problem and then they're so delighted they would scream your name at a conference at you yeah yeah they'd see the big orange t-shirt and like that would they'd run up to us and like this is a great feeling yeah and what what it really was was like these these these users were not and certainly the software can be used this way they were not using the software for pure like optimization time optimization use cases not to like hey save me five minutes a week or save me an hour a week these users were like doing something that was like transformational for themselves or for their team or for their business it was like It was almost like a skill in mind.

7:22Like, hey, I thought I couldn't do this. And because Zapier existed, I was able to do it. So you think of a solopreneur or a one or two small person business that thinks like, hey, it's out of reach for me able to build a business. And because I actually have access to these tools, I can build an inbound lead generation through a Google form and a lead scoring mechanism and an outbound email thing with MailChimp. I can actually do it now. Maybe I was budget constrained to be able to hire a developer and I didn't have the skills or the time to go learn how to be an engineer to stitch together all these tools myself.

7:53So it'll knock a lot of focus, I think, to be able to do things with software that just previously fell out of reach. And I think that feeling was what drove the drove the passion and i don't know it got me way more excited about really trying to grow the business as much as we could yeah and the company's now worth five billion yada yada you've raised a ton of money you've got how many customers how many employees ballpark several hundred thousand paying customers over 10 million folks have checked out and tried zapier over the last decade we've been around quite a while at this point um over 5 000 apps at this point and you've blown past 100 million in revenue that that rumor's true the last number we shared was like 150 million wow that's just mind-blowing it it took a decade or just over i guess right you're kind of on your 10 year past your 10 year anniversary yep um but it really took if you look at that 10 year plus journey at what point did you have that inflection point where hey this is really uh starting to ramp up because i think some people get discouraged during those first couple of years when maybe you have light product market fit and like you said you didn't think it was a big deal.

8:58If you could pinpoint that moment when you said, hey, you went to the conference, people start yelling out your name, they see the orange shirt. What moment in time was that? And then when did the business actually start to crank and make revenue? Yeah, I think 2014 probably was around the year where we started just to get enough recognition in the market from users and customers and partners to get that passion and hear the excitement. That was also the year that we got profitable. So one of the other unusual things about our businesses we've raised very little venture capital only a million dollars back in 2012 once the balance sheet um since then we basically run the business on cash uh from customers um so then any of those fundraising you've done is just secondary or something since then yeah yeah we we've sort of offered we wanted to um offer an equity program for everyone in the organization a couple years ago so yeah we went out to the market to get a we've never raised monies we didn't know the share price it's actually worth so we went out and actually got a share price and said okay now we're going to start we can build our compensation models around that and actually offer that to everyone now going forward in the organization and um i remember salesforce ventures was one of the early investors obviously went to y combinator another great hit by yc um and then you uh just set up a secondary plan for your employees how do you uh everybody has a lot of questions about that um how do you look at executing it um you know this employee stock option plan equitably fairly keep people motivated yada yada Did you have a process there?

10:25Yeah. There was definitely a history too. It's pretty interesting for us. So when we first started the business, we were 3 dudes from Missouri. So we really had more of that, I guess, ethos and how we ran the business, which was like, you sell products, you make money, you scale the business based on the money you make. We just didn't have the Silicon Valley raise$100 million. That wasn't our default operating model coming into the business. And because we were able to get profitable really early, one of the things we thought to do, we actually did offer equity to early employees. We went through YC, so we got the traditional startup advice, like, oh, we'll set up an option pool and offer equity.

11:00So we did. And the reality of all those early employees, because we were hiring out of our networks when we've been remote since 2012 as well, we were hiring out of the Midwest. We're hiring internationally in Europe. And none of those early folks really valued the equity. No, they've never seen anybody make money off equity. In fact, they've seen people get lied to with equity in some of those places that would ever be worth something. So they just are like, hey, give me cash. And if you want to give me a little extra cash. We switched. We switched to profit sharing really early on. Probably.

11:30It was probably around 2014 when we got profitable. I think when we sort of switched over that model and said, you know, this is what our sort of teams are telling us they want. Our employees are telling us they want. So like, let's let's talk about that instead. And it was way less overhead, too, for offering it because it's, you know, we had a global sort of employee base. just like the logistics of offering, perhaps sharing work were a lot simpler. So we actually built that model for a really long time. And up until closer to 2019, going into 2020, where Zapier wasn't a lottery card anymore, like it was in the early days.

11:59Like, okay, we built a real business, north of 100 million recurring revenue. This is not something that's gonna go away. So we said, all right, we wanna start offering equity for everybody and give everyone a chance to participate in the upside of the business at that point. um that's where we kind of kicked off the logistics to like okay let's actually go try to figure out how we're going to create a secondary market can we get a share price for this asset figure out what it's worth build that into our sort of our compensation models so now we still do have like a bonus program that looks more traditional like we kind of pivoted our profiteering into more of a bonus program um but we added in the sort of mailchimp famously you know did this so we we call companies like this internally at our firm alicorns you know know it's like a unicorn and a pegasus and my joke was they fly over traditional funding rounds com.com we invested in that company when it was like a four and a half million dollar company and nobody would invest in it 40 vc said no we said yes and then alex and michael came to me they're like oh we're raising a little bit of money and doing a little secondary you cool with that i'm like yeah whatever and they're like yeah it's at 250 million and then i think the next round after that was 1.x billion and they didn't need the money like you they just did it off money and 37 signals was similar um what's the uh oh survey monkey was another silver so survey monkey and mailchimp both did it this way it is possible um you raised under 2 million and you got to over 150 million in revenue just let that sink in that is the definition of and i'm not like dogmatic about not raising money um no you know i tend to like zapier's done some weird stuff right we got profitably.

13:33We've been a remote company since the very beginning of the business back in 2012. I like to think of Zapier a bit of as an existence proof of alternative ways to grow and scale companies. Now, I'll also be clear. I think a lot of those things got pulled out of us rather than us expressing them intentionally or proactively. Like, hey, we found a niche in the market that was underserved and we were able to actually do this thing. One of the things I think that's under-realized about Zapier is... I actually think it's one of its fundamental innovations is a bit of a business model innovation more than anything else.

14:05Explain the business model, yeah. Well, so like in the 90s and 2000s, like integration is a thing. Like APIs existed, you just had to... Yeah, middleware. Microsoft BizTalk, if you remember. Yeah. That tech, that product. But it just like costed millions of dollars to have like huge fleets of integrators, basically custom engineers and IT to come into your organization and like stitch all this software up together. And our sort of, I think, innovation was, hey, we found a way to actually deliver the software in a usable fashion. And we found a way to reach customers through search, which cost us$0.

14:42And so every time we're adding new integrations to the platform, which by the way, are also built by a majority by our partners for free, there's no money that changes hands there. So partners are building integrations for free to get access and deliver integration to their customers. That opens up and adds new search landing pages to Zapier, which we get new customers then from Google for$0 effectively. So we're able to find this flywheel that meant we were able to acquire customers for very, very low cost, which means we can deliver the software at$10 a month,$15 a month,$20 a month starting price.

15:13And that let us reach a set of customers and users in the world that otherwise just weren't being served historically. And so that is how we make money still through today. We have software as a service. You know, we have starting price plans around$20,$50,$100 goes up from there. You know, we add on later on features around teams and companies and organizations and things like that. We're starting to add more of a traditional sort of sales and go to market plan as well for like upmarket customers and mid-market customers. But by and large, we make money directly by selling software and users and charging for the amount of like tasks and usage they have on Saviour.

15:48Listen, I work with super early stage companies at launch, like literally year zero. They haven't even incorporated yet. And then we hit the Series A, people have thousands of dollars in MRR, and maybe they've only raised a couple of hundred thousand before that Series A, and they don't have their insurance set up. And in fact, we recently had a great startup that didn't have D &O, and we had to really stop everything because they were having board meetings, they were making massive decisions, there were legal issues, and they didn't have the basic D &O insurance that protects directors and officers.

16:19So we send them right to Embroker. Embroker is business insurance built specifically for startups. A single application will help your startup get four quotes for four lines of coverage in 15 minutes. Think about that. Four quotes, four lines, 15 minutes. And they're going to connect you with one of their expert brokers for unmatched service that goes beyond your policy. We use it at launch. It's easy peasy, lemon squeezy. It's easy breezy. What more do I need to tell you? I use it. I love it. A lot of our startups use it. They love it. Try and broker today with the code twist and you'll get 10 % off their startup package in broker.com slash twist.

16:54That's E-M-B-R-O-K-E-R dot com slash twist and use the code twist for 10 % off. Okay, let's get back to this amazing episode. Something interesting happened recently. You mentioned that you were playing with the Facebook API, I believe. They quickly deprecated that. Jamath talked about it on All In, I think, recently when they realized like, hey, wait a second, this is like the core to the business. We don't want you having access to this, Zuckerberg being savvy. and then famously now uh elon and reddit i just had steve huffman on a couple weeks ago uh or last friday i think actually was and he talked about them i don't know if you saw the episode where he talked about turning off access to the api or i'm not turning it off charging a reasonable fee for it monetizing the api so i'm wondering just with this collection of examples they're all happen to be social sites which is interesting um yeah that's my observation as well okay linkedin is another one from earlier you didn't mention that craigslist has no api never did and will in fact sue you if you use any of their data for through a scraper yeah so maybe you can talk about if you want to pick out a business example of mailchimp and shopify also went through sort of an interesting api breakup i think when they started competing with each other like you can go look at their like public blog plus around this but yeah effectively you know mailchimp i think was starting to introduce products in the market in order to compete.

18:16And some of those were going head to head with Shopify. So both of them mutually said, well, it's not great for my business to just be giving away value to my competitors. So we're going to start disallowing our use case or not, not providing the same native integration that they previously had. And one funny outcome from that was we had customers for both of them basically coming to us and say, hey, my vendor of choice is going to stop supporting this native integration can i just use that for and uh that led to both of shopify and also like basically just sending us a lot of their users who depended on the native integration because we have a lot of sort of step in as a bit of a neutral like while you were speaking i typed in mailchimp spotify api and you're the number one result i think on zapier because you're uh neutral you're sweden um but what do you think of this charging for the api because obviously that changes your business uh you now have to i guess um ask people to put in their tokens to uh do this and then i guess with ai and open ai specifically you know they you know they have calls and stuff like that does that dramatically change your business or do people just have to fill up their you know for most like first-party vendors most software providers is actually the preferred way to go like opening i it's kind of introducing a bit of a new way to do product monetization where like hey, you have a direct billing relationship with OpenAI.

19:36And if you want to use a platform product like Zapier to plug in that intelligence layer into a sort of a workflow, you bring your own key, right? You bring your API key to Zapier. My sense is this is actually like the smart savvy way to go about it for a lot of these products. I kind of actually wish that things like, you know, Twitter and all of them would actually adopt more of this model where it's like, okay, if I have a, I'm going to establish my direct billing relationship with my sort of, you know, first party vendor. and then allow that user to bring their token to other tools. Then you just charge for the user to have access to those tools.

20:08You can say, okay, well, allow access to sort of the API for say Twitter in this example. Allow Twitter apps as long as that customer is, you know, Twitter blue and already paying for it, for example. I think that's sort of like, I don't think anybody wants to get disarmeniated. It's like, why would you ever let like a third party company charge on your behalf anyway? I think it's probably not the best place to be. Yeah, it got kind of weird. I think if you look at the time period where you started your company, It started right at the kind of end, tail end of web 2.0 and the web 2.0 movement really where API started in 2005, six, seven, eight.

20:42People were just looking, people were under resourced. Twitter was under resourced as a company. They couldn't raise a lot of money. They couldn't afford to have iOS developers. And you know, when apps came out, so they're like, Hey, you, you, y 'all have at it. Reddit didn't have enough money. It was kind of free outsource development is how the community looked at it. And you're like, Hey, you, make some value for yourself in the world um don't do anything stupid and you know have at it uh and then the problem i guess of course becomes then when you have to go public like reddit does uh or twitter has to turn a profit eventually uh you need to tighten the screws here and then you find out whoa these people were really abusing the api they were taking our data or users and selling it to people and get all these kind of gray markets etc so i think it's actually kind of cool to for the idea that you could just fill up your card and i have a couple of startups who are doing this at open ai they fill up their card and uh yeah then they run it down and it's like okay i need to get more what do they call that uh card that you get when you are in college and you go to the cafeteria whatever that card is called meal card yeah your meal card it's kind of like your meal card like just points on the card get points on the card so uh all right i think for a lot of like b2b companies is apis are are actually in their interest right where social companies have a bigger downside that they have to protect against, which is disintermediation.

22:02Like, you know, I think in Twitter's case, the stories that I had read on the online was, you know, folks, basically, there was like a bunch of first party platform apps that were using the Twitter API that were starting to get consolidated under one owner. And Twitter sort of got spooked and said, well, shoot, we don't want like our business front end to get disintermediated with our users through one owner. So like, we're going to tighten things down. So if you can figure out a clever way to like protect against that outcome from happening, then I think it's all upside from a monetization standpoint.

22:31I think users at this point in time around subscriptions are used to the idea of like, oh, okay, if you're going to provide an ongoing service, there's an expectation that there could be a cost associated with that. Whereas on the B2B side and the prosumer side, integrations are purely upside. And there's almost no disintermediation risk. In fact, these integrations are usually really good. We actually ran a bunch of studies, one with Typeform, that showed integrated users churn like 10 % less. Yeah, I mean, we use... They're super sticky. the fact is we use notion we use type form and type form we love as survey monkey we love we love all these products if they didn't have integrations um yeah we might actually use uh a product like you know google sheets allows you to do forms we might use a product like google sheets instead and just be like it's not as good but it has integration so we'll go with that right it's almost like you would pick we we would not use certain products certain sas products if they didn't have our users to they will select we have a quite a few users lately will actually say i'll go to our like app directory page on zapier and use that to choose software because i can like trust so i at least i know what integrates with things in like they have a sort of the right mindset around that okay listen you got an idea for a tech startup great you think you want to change the world you think you got this this is the one well you've got that same problem that we all do you don't have an engineer or you don't have enough engineers to make this happen and you need product velocity you need to go fast and how are you going to go fast and how are you going to control your burn rate if you got no engineers?

24:00Well, what if you had a partner who could provide you with more than a thousand on demand developers and those developers were all vetted, experienced, result oriented and passionate about startups and building great products? Well, what if they also charge competitive rates? Does this all sound too good to be true? I know it does. Well, then you need to head to lemon.io because your dreams have come true right now. Startups choose lemon.io because they only offer handpicked developers with three or more years of experience, and ones that have strong portfolios, only 1 % of candidates who apply to work at lemon.io actually get accepted.

24:32And if anything goes wrong, lemon will get you a replacement ASAP. A couple of great launch founders have worked with lemon.io people in our portfolio, and they have had great experiences. So I want you to learn more at lemon.io slash twist. And when you go there, you're going to find your perfect developer or any entire tech team. And you're going to do that in 48 hours or less. and twist listeners get 15 % off the first four weeks. I want you to stop burning money. I want you to hire developers smarter and faster. Visit lemon.io slash twist and give it a shot. All right. I want to talk to you about, I want to actually talk to you because you have some big thoughts on AI regulation.

25:06We'll get to that in the tail end of the show in the third act. But for the second act here, let's go over some of the cool stuff people are doing on Zapier with integrations because my core, and I'll let you fire up your, and share your screen while we're talking my core premise here is there's going to be like a permanent hiring freeze at companies because everybody's getting 30 percent more efficient a year using these tools at least i think and they will continue so why add more people if writing a job rec takes more time than writing a script and automating something so that's the core tenant i come to with this you you think that resonates with most businesses be when faced with a hiring rack versus making things more efficient with tools like yours what should you do it's what a lot of users want like users want automation technology to do work while they sleep like that's that is what buyers want that's the dream i don't think we're there yet and all the use cases even even internally um earlier this year in march we held a company-wide hackathon we actually told everyone the company our pencils down we're going to take an entire week and the company needs to re-educate themselves around what is possible with this technology, what's not possible, and figure out ways to work in your workflows.

26:22And we've gotten up 20 % of every individual person at Zapier. 20 % of all of our employees have worked in some sort of AI into a Zapier workflow that they use. As far as I actually have not talked with a company that has a higher percentage than that yet. So I actually think there's perhaps interesting things to learn there. But it's existential for us. We had to do this. AI and automation are essentially synonymous, I think, going forward. And, you know, Zapier's business is basically, hey, software that works while you sleep, right? So there's... And you're, of course, referring to auto GPT or baby GPTs, I guess people call these, where you give a set of instructions to an AI, and it performs them over time, perhaps even getting better at the task with some scripting or instructions.

27:09So let's do some examples. Well, it's not too far apart from even how you think about what Zapier does today. It's just hard to use for most people. Like Zapier, if you set up a Zap, it's a workflow. It is going to do something forever and without you using the keyboard to come interact with it. But it's really limited. It's constraining, right? It's rigid. And it's also hard to set up. You mentioned at the top of this podcast, like, hey, I have to educate my new employees how to use this technology. It's just not easy enough to actually use right out of the box. The penetration rate of this tech isn't very deep yet.

27:41So I think it's still too hard to use. And I think that's where the technology really has a chance to shine. But yeah, the demos I actually have are the first one is actually they're mostly centered around the chat. You can plug in that we've launched back in. We're one of the launch partners with open air back in March. And I'll show this off in this order. We'll go through maybe a couple examples and we can end on one of the APIs, how this actually works. some of the hood. So this is an example of... I think how we've seen most of our users, even in some internal employees, Zapier adopting this stuff is they'll still...

28:18I've heard a lot of anecdotes actually internally where basically folks will have two tabs open all day. They'll have a chat GPT open in one tab and Zapier open in the other tab. And the reason is because the models of how the software works are completely different. Chat GPT is a piece of software you have to interact with in order to get value out of it. So this example is one I've used myself is, you know, hey, grab an email from my inbox that matches a certain format, and, you know, draft a automatically draft a sort of reply to that. Okay, so you're in chat GPT four, you're using the plugins, you pick Zapier, and you say, Hey, I want to check for an email in my Gmail account.

28:54And you've already authorized it to go to Gmail. And now it finds the latest email and does a reply for you. Yeah, this one summarizes the reply. And then I think if I kind of just, zoom forward here. One of the actual downsides with the plugin architecture on how ChatGPT works right now is everything has to go through these confirm flows, which is... I understand why they do it. The safety argument around it. However, I do think that there's probably some edge cases where we actually take a stronger safe stance than their platform does. And it creates a weird system where two safety systems are trying to be in the middle and it creates a really awkward user experience.

29:30So I think there is more stuff we can do there. But yeah, here's an example of where the plugin is going to come back and grab the email response and summarize it back in. Yeah, so this is an example where we actually opened up a tab on Zapier to give a preview of what the plugin action is about to do. I think this is an example where we've actually inserted our own safety things to let the user explicitly know what actions they're going to do on their behalf instead of just letting them roam free on Zapier on your account. and now we're back inside chat gpt after the user's confirmed and it's pulled an email and summarizing uh the email and i think then in this demo we actually even follow up and ask chat gpt hey can you sign that with with my name and rewrite it my tone and then uh send it through gmail as well and you can actually fire off now from the chat gpt interface using the zapier plugin if you've authorized it it makes you go to that step it will actually do the send from the chat gpt interface it will in this case we're creating drafts this is kind of another one of those like probably tips i would have for most folks that are adopting this tech is like you know the technology is really really good at drafting things so you almost want to lean into use cases where you get to get a preview of it and you can add it and mark it up and have sort of control over it before you press the send button yourself you absolutely can hook this directly up to like sending an email directly um but the one we found most folks inside zapier adopting is you know these flows where it goes through creating a draft for you being able to review it and approve it before hitting send.

30:54Basically, the vision for what we want to try and get this to is it feels like an OAuth flow. Whatever product you're in, if you're in ChatGPT or you're in any other ad product, you need to plug in an action library into it. You click a Zapier button. You say that vendor says, Hey, I'd like to get access to your Gmail account, your Salesforce account, and your Typeform account. And the user says, it looks like an OAuth flow pop up. The user says, Yep, that sounds good. And now you're back inside the first party product. and you can go from there. In the initial version that we released, it's like one, there's one extra step, which is in addition to having to approve it and allow you also explicitly today have to choose which actions from those apps you want to allow the chat to have access to.

31:39So today, for example, with the Gmail, when you saw, when we set that up, you had to say, yeah, okay, I want to allow chat to access to Gmail, but I also want to have it allowed to send a draft email. So it's one extra step today we have to choose those and that's somewhat of a limitation of sort of the language model technology and somewhat of a limitation of the api experience overall right now yeah it's it's there is a it's a little kludgy you have to log in i remember doing this you have to log into zapier and then there are some links that you have to go to to make sure you can search your gmail make sure you can send from gmail and uh that i'm sure will be abstracted in the coming weeks and months.

32:17Yeah. Yeah. I don't, I don't disagree with you at all, by the way. Um, and you know, I think if you, and I have, I've looked at and sort of obsessed over some of the usage and some of the numbers from this stuff. Um, you know, I, I do think that a lot of the plugins and folks I've talked to retention is, is a problem right now with them. You know, I think if you kind of look at like, it's almost like a bit of a numbers game. Um, you know, if you're sort of able to spread your user base over enough users, you can sort of find a percentage of them that you're gonna find these sticky use cases to build this like chat and plugin thing into their workflow the reality is most people in the world haven't even worked chat chp t into their workflows yet so like no asking them to then add on a plugin that is also like you know an active development like it's i think we're still quite a boys away before you're going to see like kind of the refinement you needed from a lot of these this like plugin ecosystem and how even just getting the penetration of chat to begin to like legitimate like sticky use cases i think is still still search for most most well you know it's it's it's going to be a slow process and then it's going to be a really fast one because once um you know the first 10 people in an organization figure this out and they become bionic and they're able to do really interesting things with their gmail box like say who are the people that i was you know emailing with back in 2012 to 2020 that i'm no longer emailing with summarize my conversations with them and then suggest which one you know some emails to catch up with them and like whoa that's going to be super powerful or like you know you're a venture capitalist hey what founders um was i talking to 10 years ago what are they doing now and it's like boop like this is going to lead to um being able to do things that would take so much time nobody would ever even consider doing them like you would have to hire a full-time person to hey go through my emails from you know the 2010 to 2020 period find every founder put them into a google sheet and then look up on their linkedin and see which ones are still at the same company boom it's like yeah whoa and it works and it works so it's super powerful we sort of see that exact same thing that's the message we delivered internally, which is how we got so many folks that have started to adopt in real use cases is, hey, this is like a chance to go learn the technology.

34:41The future is not going to be like... And I can't... I really struggle to think of any sort of technological disruption that displace jobs in a quarter. But over the course of 2, 3 years, 5 years, you're going to see folks that accelerate ahead because they understand and use the technology. You're going to have like infants in the job market who natively know this stuff, Especially if you look at the adoption rates of like chat GPT and education coming, like all those folks graduating through college and entering the job market, like they're going to have a skill set that I think a lot of other folks probably won't have.

Read the full transcript

35:11I think that's going to be a leg up for a lot of them. Any other language models now built into Zapier that have integrations, Google Bard? Yeah, we have, well, none like chat GPT where we have a plugin launched yet. There's nothing to announce at this point. Yeah. We do have all of... We have way more language models actually built into Zapier first party though. So this is the brain bender about it, right? When we actually went to go build the ChatSp2 plugin, one of the reasons we built that was we saw this huge influx of AI apps launching on Zapier. We had Hugging Face and HumanLoop are all getting built on Zapier.

35:48And we realized like, oh, wow, there's this huge explosion of AI products that's happening in the market that are not going to get on Zapier. And we wanted to offer an API to them to be able to bring Zapier's integration platform into their products. Just felt like the first time we've ever actually launched a public API. It's a bit of an almost embarrassing point that we're 10 years in. And it's like, we're finally now just launching a public API that other vendors can pull in. But we felt like it was what needed to happen at this point, given the pace of products that were getting released. But in the more traditional way, where you go to Zapier.com and you're building workflow, Although, you know, we have Anthropic now with Claude is on Zapier.

36:27We have the Google, the Bard version. We've got OpenAX integration as well. So, you know, you can build those into more traditional workflows. But I do think some of the more exciting, interesting ones are like the paradigm shifts where you have like a completely different, you know, front end interface for how you build and use this stuff. If you want to get ahead in your career, you need to sleep well. And if you want to be a stallion, if you want to be a workhorse, you need to get a great night's sleep. It's that easy. So do what I do. And that's getting an 8 Sleep, right? Especially over the summer, right?

36:57Temperatures are going to rise. You want to stay cool. And 8 Sleep mattresses allow you to set the temperature. And they're selling more than just mattresses right now. They have something called the Pod Cover by 8 Sleep. And this fits on any bed. It's just like a fitted sheet. So if you don't want to get a new mattress, but you do want 8 Sleep, you just get the Pod Cover. It's going to keep you cool all night, all the way down to 55 degrees. And it's going to improve your sleep by automatically adjusting to the temperature on your side of the bed. And, you know, your partner or spouse, they get to set their own temperature.

37:27So no more battle with your partner or your spouse over the thermostat. Nope. It's going to adjust based on you and your partner's preferences. And you get personalized sleep reports. Eight sleep, I love it. In the winter, oh, I keep it nice and toasty. But in the summer, I like those cool, crisp sheets. And I don't want to have to blast the AC in my house and burn all that extra fuel. No. I just use my eight sleep. I dial it to exactly where I like. And, you know, I get a better night's sleep. Temperature really does correlate with a good night's sleep. So go to 8sleep.com slash twist to get$150 off the pod cover.

37:59Stay cool this summer with 8sleep, please. And they're now shipping not only in the US, but Canada and the UK and some other countries in the EU and Australia. So 8sleep.com slash twist for$150 off the pod cover. Can't go wrong. Talk to me about LinkedIn. they do not allow people to use their api is linkedin very protective of it because it seems to me like nine nine out of ten times somebody gives me examples of where this is going linkedin comes into play um how do you think about linkedin and how do they operate with zapier at this time yeah i'm trying to remember it was a it was five plus years ago i think when they went through their api sort of tightening phase they had a pretty open api at the time and then they sort tightened it down and got rid of a lot of their automatic message sending stuff, the contact scraping stuff.

38:48They sort of segmented out into their... I think it was around the time where they really wanted to go after recruiters as their key customer and buyer. And so they shaped all of their API usage around that persona and just said, all other uses, we're just going to delete and get rid of. We don't care about them. The one that most of our customers and users wanted was the things like lead hydration, where I could take an email address and go get information about that lead, particularly for marketing flows. You talked about, hey, you've got a founder of contact form that you have on that form, and you get an email address, and you want to automatically pull in a bunch of information.

39:23So you don't ask the founder to type in the resume or whatever. So folks were using things like that for those use cases. And the reality is the market now has addressed the gap. There's a billion different lead hydration companies that all sorts of great public information. And, you know, wouldn't be surprised if some of that originally did come from LinkedIn, but, you know, they had that big public scraping case. They had a big public scraping case with, I think, a company based in Israel that they lost. Yeah, it was like back and forth. They won the first one. They lost an appeal. I'm not actually sure what the current status at all is.

39:54Well, I mean, the great irony of this is they tightened their grip and said, hey, you can't do certain things. and so what that does is since there's a need like we saw with napster back in the day if people want something you know uh whether it's a tv show music or to enrich a lead uh enrich an email and hydration i've never heard that it's a great term um you know get an email and then get the person's title and where they worked previously it's really smart um then some gray market company uh doing gray hat stuff is going to do it and they're going to scrape all of linkedin and then they're gonna back into it and you i know this because i've had so many companies do this with facebook data linkedin data and then legal letters get sent and um the only people who get really impacted are the good actors who want to play by the rules and then the people who get who benefit they get punished and the people who get uh rewarded are the gray hats and the black hats who are going to just scrape the information offshore and they don't answer to anybody.

41:02So it's kind of a bummer. If I look at sort of the mass core of our users and customers too, I would, almost every time there's an API application or terms of service change or something big, like those are the folks getting infected. And by large amounts, Zapier's customers are very small businesses. I think it's like one to four sized teams and companies. Talk to me about like the large enterprises. this is there anybody who's taken zapier to like a large organization and coordinated it and do you have that feature because right now i have my team using it but i don't know if we have a central relationship between more like we're like building some offerings here we're trying to figure out how to do this basically so that's sort of the tldr we do have some like examples netflix is probably one of the larger ones where you know they have a very it forward perspective where you know their it organization basically has an identity of saying our job is to make you more efficient more productive.

41:53That's why we get paid. That's why we're here. Versus a lot of more traditional, old-school, mid-market-plus companies where their technology side of the house might be looking at the rest of the business as, hey, I'm a cost center, I'm about protection, I'm about control, I'm about risk management. And those are just very fundamentally different perspectives. I think when you start talking about introducing new technology, language models and AI admission to business process. So we are seeing more and more of IT or is thinking this way. And we're starting to put together some packages of software and services around actually basically doing what we did internally, which is...

42:29I actually think we have some expertise now to figure out how do we actually adopt AI use cases into real workflows that do allow folks to get legitimate time back and allow them to move on higher value activities and use cases and workflows and jobs and actually deploy that into the organization. um and that's like the common thing that i get asked whenever folks come up to me and talk to me these days around all this ai stuff is like how are you guys actually using it what are you seeing your users actually use it for because i think a lot of folks are like it's still the tech there's broad awareness for what language models and ai can do at this point i don't think there's broad penetration for it actually into real use cases yet yeah so talk to me about um how you look at again i guess everybody's got an opinion on this but ai regulation i saw some of your tweets um do you think that this is moving so fast that there's going to be significant negative consequences for humanity uh do you think we need to slow it down or do you think we need some thoughtful regulation how do you look at um this because open ai you know uh here's my i'll speak like most to what i know um certainly like i think there's some interesting philosophical things we could jam on.

43:41I don't feel equipped to have that argument or to even debate at this point. But I can speak to what I know, which is, Zapier, we have millions of legitimate, useful workflows and automation that users have set up over the last decade. I know that. I also know that it is way too hard for most users to use Zapier, even today. I know you gave us a few praise at the top here and said, Hey, I love Zapier. It's easy to use. The reality, of those. For most users, it's not. We fought for a decade on trying to make Zapier easy enough to use for the traditional professional who does not know how to code, does not know, it's not technical.

44:20However, we still have a long way to go. And we fought on that problem for so long that I think we are reaching some limits of the paradigm of traditional software to actually put workflow and automation into the hands of end users. And I think this AI language technology is the first thing I've really seen that I think offers a step function, not just like an acceleration of a smooth curve, but actually like a step function and an adoption rate of how many business users and users can actually set up and use more technical concepts, things like automation. Most of the people use Appier, even though they might not call themselves technical, they're still builders at heart, right?

44:57They have that sort of identity or like, I'm going to go create something. And I think this is where this language model technology helps it drives down the barrier to creation by just a ton. So I get excited, first and foremost. I get really excited about the idea of like, Oh, wow, well, we have 10 million people who've tried Zapier. Maybe this could get us 100 million folks who've tried and used automation successfully. And I think that's a really positive thing. And especially if we model all the use cases on what people are already using Zapier for, that's all great. I just want to make more of those people.

45:32And I think there's a chance to at this point. So first and foremost, that's where my head goes first is I think the technology transformation, particularly in the prosumer business workflow automation space. And it's not just time-saving. These are legitimate... It is an unlocking technology for a lot of users around what they can actually use language models and automation to do. It's not things that they weren't otherwise doing. On the open source side, But I do think like, so we haven't talked about this much. Basically what I actually, I know you introduced me as sort of the president of the company.

46:04I gave up my exec title last year, last summer in July. I quit the exec team. I went to Brian and Wade, my co-founders. And I said, I think we have, I got to go on in this language model, ML, AI stuff. We got to learn what this is going to do. So Brian said he was going to do the same thing around the same time. So both him and I basically said, we're going to get rid of our exec team roles. And we're just going to go full-time and focus on research and engineering for AI, particularly in the context of sort of Zapier. So back to the laboratory. Yeah, basically. Out of the exec suite, no more fancy bathrooms, back down to the garage, right?

46:39Back to the garage, quite literally. So like, and I do think like you really do have to go hands-on to learn what's like possible with this tech. If I look around, I think one of the coolest things is when I like look around at like, you know, my peer group of founders, you know, folks have been around for 10 plus years. all of them are doing something similar which is like going back and actually getting hands on the technology and i think you have to to learn what's possible yeah um and i actually think zapier plays a role here too because i think this point about you have to go hands-on to learn what it can do and what it can't do it is well beyond like hey i'm going to open up a github repo and download code locally and run it things like zapier can be gateways for a lot of users to discover what is possible and what's not same way the chat gp is offering a view of like what's possible what's not.

47:18We have users that are basically experimenting, trial and error with workflows and zaps and plugging a reasoning engine, a language model into the middle of a workflow to say lead score, or draft a reply to a message that I received, or draft a pull request that I received, or score the JIRA tickets, or summarize customer feedback and dump it out into a Slack channel. There's a lot of trial and experimentation around it. And I think those users are figuring out what it can't do at the same time, right? They're figuring out, oh, I shouldn't just automatically send an email. No, not ready for that.

47:52Yeah, be careful. I shouldn't insert this into an HR hiring decision where I'm not going to review the decision. Definitely not. So I have a lot of trust when I look at our users of how they do their own experimentations to find what it's good and bad for. I think we got to put that experimentation mindset in our hands. So that's why I get really excited about the open source thing about, hey, the more we can get this technology diffuse into more individual hands at the end of the day, I think is going to allow more people in the world to understand what it's good at what it's bad at and like calibrate and i don't know i have i have a large a high degree of trust i think in sort of folks ability to figure out that and navigate that chart like you know deal with the antidotes of the technology as long as you give them enough time to and that's where my maybe the philosophical hack comes on it's like okay if you really can like sort of drop a like i don't know some sort of step function technology change in a month and like okay maybe there's like a moment where there's like enough disruption there it's worth asking the question right now but like based on everything i've seen the last 12 months of language models that we're that is not what we're dealing with um what we're dealing with is more hey there's something that would take you 100 hours might take you one hour or something that took you 10 hours might take you one hour or you're still gonna learn how to do it you gotta learn how to do it which is really important i thing um yeah but you do agree that this is going to make companies massively more efficient and you're going to need much much fewer people to do much more i mean it's hard for me not to agree with that statement in the limited edition like based on where folks want this technology to go like as soon as it exists yeah there's a huge demand for it yeah and so then the question becomes you know we as technologists looking at society uh are left to wonder are there still problems to solve because if a 10 person team can do the work of a 20 person team they could solve twice as many problems it's not like there are not a long list of problems to still be solved in humanity uh and so that's where i look at and i mean the sort of classic use case probably even wade might have showed this when he was on the show is like the typist example right uh or even literally the word computer used to be a profession in and of itself back in the 50s and 60s typist was a profession like yeah silly i got trained in middle school how to type on a keyboard right right i was just having this i wrote in my i started uh doing some email newsletters again and i was like you know there used to be i started my career as a pc support specialist what a pc support specialist did was they set up your computer they upgraded the memory they put in a larger hard drive they set up your ethernet card and then they sat there for two days installing software on your computer using cd-roms all those apple engineers took your job jason where they made the beautiful ios onboarding flow when you get a new phone exactly and now it's like yeah you don't need a pc support specialist to come and set up your computer and your microsoft office for you you can simply uh here's my sub stack thank you um i guess there's the section and you know then when i saw you know startups and when you were setting up your startup you were right at the point of cloud computing so did you rack your own servers for zapier and have a colo we no we did not linode if you remember that name linode yeah great i still have a box on there somewhere but yeah that was the very first cloud service we used yeah they were the pioneers right and so you you you were the first generation of startup founders to not have to go order pcs and build a rack and find a co-location facility rent space go down to the co-lo facility have a sysadmin and you're you might not have even had a sysadmin or somebody at your co-location facility um the generation right before you if you were working at flickr or a facebook you were racking servers and you had two or three people on your team who are managing that for you uh yeah linode.com twist and get a 500 credit i forgot i'm like yeah they were a sponsor there's a plug thanks for the plug i do think one other if i can add one other point of regulation side i do think one side thing that i i think is i i i'm fearful this is how it's going to play out i'm not sure we'll have to check back in in six months 12 months and see if this is true but i i am a little disappointed that i think the way that most of the research around ai and lm's heading is towards more closed companies like they're not being as forthright and like forthright and sharing of basically like the technology the progress the architecture like they're we're kind of getting into the space where people are realizing how much value is in i I think the research and they're just a lot more close pulling up the ladder behind them.

52:23I do think is going to, but that's my, like, that's almost my like sort of anti like acceleration viewpoint. Like I think there is a, there is a path where actually progress slows down for a little bit of time right now because we're either sort of one approaching some of the absent limits of what we're going to exploit out of transformers and language, large language model, the kind of current architectures we have. And to like more research is getting sort of closed up. So there's not as much open sharing, not as much progress. And as you know, like the reason open access basically is because of some of the progress that came out of sort of public sharing from another competitive company with google um so i do think that's a bummer that is such a weird move that they went from open ai to closed ai they literally took the reason they existed and reversed it they're like this technology is too powerful for everybody to not have a say in it and for it not to be transparent then they got a couple years in like you know what this technology is so powerful it's too powerful for everybody to know how it works and i i am 100 agree with you do you think that leads to the developer community at a like fundamental i completely trust like sam and greg like sure as stewards of technology i can't think of two other better people that i would like try to put in charge of that problem um but like if i look at the second order effects of what that then leads to like it does i am a little worried that it does lead to like more closed up nature we're going to see less progress we're going to see less sharing we're going to see less like you know technology getting pushed to the edges um and those good news in all of that is it seems like the open source community and the open source models are advancing much faster i don't know if you saw that google memo uh but there was a google engineer who's like listen at the pace that open source the open source community is rocking on this they're going to just beat us and we don't have a moat and neither does open ai so it's almost like they're squeezing um too hard and that's leading to people saying you know what i don't want to build on a closed system which you know you may call it open ai but i don't want to have the risk factor of working with open ai so i'll look at some alternatives and the reality is open ai more than anyone has pushed for the technology to be developed in public though so i will make the argument in their favor i know that i'm sort of you know poking at a few things and decisions I've made.

54:39But I also think we wouldn't be sitting around in this conversation. Zafari would not have shifted its viewpoint had they not made the decision to publish. They've sent mixed messages to the market. Yeah, I think that's fair. Yeah. I think their largest message is we have a product that you can use for free. And I do think that that has changed a lot of folks' opinions around what's possible and catalyzed a lot of energy. And you look at the API, every six months, they seem to just drop the API price 90%. Right? They've done that twice, I think. on i think you can bet on like costs going down i think you can bet on um context windows going up i don't think you can there's not like a smooth ramp on architecture improvements though i i do think that that is like one thing i've been personally spending more and more time my time on lately is explain that to a lay person yeah that's where we understand what your point is i don't proclaim to be an expert in this either so sure but i'll try i'll try this for my best interpretation of what i understand about sort of transformers at the fundamental level is this is basically an architecture that was not necessarily it was like published from a paper at google back in 2017 that paper was the continuation of actually quite a long journey of research as well around what i think originally started around translation like literally translating a sentence from one language say english to french right and the first sort of uh deep neural networks that did this were sort of constrained they they kind of fixed the amount of characters that you could translate from x to y and that led to the invention of this technique called attention which allowed you to of variable length inputs and outputs.

56:04So you could, you know, the word in English is a different length than the word in French. So you could actually kind of deal with that problem. And this led, that like attention mechanism was like its own neural net at one point. And the sort of infamous vapor, attention is all you need, was the dropping of one of these like ancillary or recurrent neural networks, because it wasn't like an important part of building a transformer. It really simplified the architecture stack. And that architecture stack also happened to be one that really ran in parallel, which fits the GPU scaling curves that we've seen over the last decade.

56:35So those two things in parallel allowed to invent progress around GPT. One, two, three, and now four, and so on from that. But there has not been at least a well-established alternative to architecture. All of the AI products, progress, research papers you're seeing, a lot of the momentum and attention has really shifted into what can you build on top of language models? What can you do with a language model that has this seemingly capability of reasoning? This capability of tool use, this generality around being able to generate content. What can you do with that? I would have actually... Up until last summer, I would have said there's like...

57:18I'm like 99.9 % sure large language models are not on the critical path to something like AGI that reaches human level generality of intelligence. I've decayed that prediction to call it 80-85%. And the reason for that is there are some things that you can do with like GPT-4 right now that no one's productizing. And because it's too slow for the language model to generate tokens, you have to let these language models think out loud and they improve their performance. The classic example is the thing that actually inspired me to go all in on AI last summer, which was the Let's Think Step-by-Step paper that came out last January.

57:55and this was a technique that some researchers found where if you put let's think a step-by-step at the top of your prompt and then ask the exact same question again the language model actually boosts its performance because it gives it time to generate tokens almost like an internal monologue where you're thinking about you're letting the model think out loud for what it should do and then letting it reflect over those tokens it spat out to generate its like actual next action um there are some like performance evals that went from like 30 35 percent up to like 18 90 percent crazy step function increases it's really weird if you literally say to the chat gpt let's try that again and uh can you try to find me three more and you just keep doing that you get to like the six or seven back and forth and it's like yeah i got you your answer and i and i did it right and it's like another thing too yeah some partial right correctness is like another big challenge you've even thought that internally um anyway so like there's there's these use cases that like can demonstrate greatness in certain scenarios.

58:52Oftentimes, they're unreliable. Like the example you just mentioned, where you had to ask it six times and it got one out of six right. Okay, the question is, how do we figure out which one is right more consistently? Or the second one is these really deep reasoning chains where you can actually let the models continue. It's like thought process. This is the auto GPT style stuff where you can let the model reason through a tree-based search of reasoning for almost 30 minutes, 45 minutes, an hour in cases where we had demos running last fall. and it can get to the right answer but it's so slow and so expensive like literally it costs probably a thousand dollars just to run that like reasoning search you're only going to do it for use cases where the product experience is like okay if it's offline and completely asynchronous and like there's huge ri attached because like models are expensive so well i mean find me a stock to short and explain your thesis examples yeah um okay now offline utl job would be another one like hey i need to process you know million records of data and i'm okay if it takes three days that's fine just like yeah it was another good use case um so there are examples of of like cases where the model just can do better than how they're getting practiced because generally with like consumer facing pro-sumer facing products latency matters a lot reliability matters a lot so like all these product builders are self-included or chopping off use cases that are slower expensive and as the sort of cost curve comes down as the context window goes up i there could be some interesting techniques around reasoning that are just out of reach from a sort of keep from a not a capability standpoint from a cost and like performance standpoint that might become in reach and you know maybe there's a way to build an having having these auto gpts talk to multiple language models and having dueling language models where they analyze each other's data and they start talking to each other that kind of feels it's a different type of singularity but it certainly feels promising if you were to say hey i'm looking for socks to short um please go to five language models and ask them about you know the stocks that are most shorted right now and uh then put together a thesis based on their five and you start having the check all different gpt models around the world working on the same problem and then some of them asking reinforcement questions to each other i mean this is you know what the rumors of like the gpt4 architecture have you read about those no tell me what you're describing is effectively the the rumor is unconfirmed as far as i know but around how gp24's architecture works which is essentially they have eight different um you know attention heads or model heads that are all trained on different subsets of their eval and uh they're all 200 billion parameter individual models and they essentially like do a mixing mechanism where they like run the input through each one and they mix the output together from the log probs and use that to generate the final tokens so not too far away from what you're describing where you have like it's like a mixture of experts i think is how they would describe it each head is an expert of a different type of reasoning or a different type of input problem and you try to pick their expert right because then yeah and then i don't even know if those are verticalized experts i mean who knows how they chop those up it's like one expert because of wikipedia oh man it sucks that this is closed right that seems so interesting i would love to read about that i think that could probably accelerate progress in some way and the fact that there's only a couple hundred people in the world they probably know the real answer It's probably because they also have some exposure based on who trained that data.

1:02:09So let's say one of them is like, this is Wikipedia, Reddit, Quora, and Twitter data. And it's, you know, consumer, it's a crowdsourced information. This is the SEC, academia, you know, the New York Times, Wall Street Journal, and like a professional, you know, quote unquote professional. This is a journalist answering the question from the journalist framework based on the Wall Street Journal, Washington Post, personas and data sets. So we're informed by those. And if they were to actually say that, then you would be able to make the case of, well, hey, you're literally picking your expertise based on data sets.

1:02:46And that's probably why they circled. That would be an interesting reason to circle the wagons and not share. It's pretty secretive about what data they use. I think they don't consider that pretty proprietary. I actually think this is a problem that goes away over the long run, not because of like some cultural acceptance of this fact, but more about, I think the amount of data that you actually need to do to train these systems just gets dramatically lower um through model innovation ah fascinating i mean there's some existence proofs here like i actually think there's a lot of really good reason to go all the way back down to like fundamental art and do like an architecture search essentially wide as you can we now have two existence proofs in the world of emergent reasoning intelligence behavior one is humans right discovered through sort of genetic evolution over billions of years and large language models which were are invented over our own cord running on our own silicon on algorithms you know sort of invented the fact that n equals two there suggests that like oh wow there's so many more if you run the probability like you would be shocked if like it stopped at n equals two around architectures that let in like the transform architecture is nowhere near what you see anywhere modeled in sort of the human brain completely wildly different in how they work so like it suggests that there's probably more and The thing that's interesting about humans is how comparatively little examples they need and training data they need in order to be sort of generally intelligent.

1:04:07Humans seem to be born with some innate amount of capabilities or abilities. Very strange, yeah. Like past following, pattern recognition. There's some things that show up really, really in toddlerhood that they never get trained on. Fear of predators. like we actually understand predators in some way natively like if you were grown if you were and i think they've done this with studies like you you don't need to have seen a shark coming at you to know you're about to die i don't pretend to be a scientist but like there's just like some pretty compelling like like i said existence proof examples where oh okay yeah they're like there are way more architectures out here that we should go search for and yeah like i think a really good constraint function to go do an architecture search would be to say let's pin the amount of sample data that we put into this architecture search um so we can find like architectures that are just way more cost effective or cost efficient um and performant uh i could talk to you for hours and we've talked for an hour mike you gotta promise me you'll come back uh maybe like six months from now i think this is moving so fast i'm gonna i'm gonna make an executive decision here uh and two things since you and i are uh you know in close proximity to each other number one we got to get some uh gotta get some ramen and then number or a lobster uh sandwich and then uh number two you got to come back on the show in six months yeah thanks so much for office recordings we can do one in person too i know i'm i'm literally looking for i'm selling our office in the city and i'm setting up our incubator and accelerator somewhere in like san mateo area and when i get that we're going to have an in-person studio again and we'll do a live version of this where we get like audience questions and stuff like that so i'm trying to find like a theater like i want to get like a theater or like a warehouse space where i could have like 50 people come and a little more raw like kind of customize yourself i like a raw yeah i hate these fancy space i a lot of people have been emailing oh i got a fancy space for you over here i got a fancy space over here in el camino come to this office like a restaurant that just shut down basically as i'm hearing that's what i'm looking for is that i'm looking for a like a shutdown restaurant like an old mexican joint with a parking lot or something where i can have founder fridays but we have drinks and then i can have you and i just sit and rap out about stuff uh so those two things will be on the uh on the docket uh and thanks for making zapier it's just such a great product and it's made life easier warm welcome a nice intro uh you know what makes you happier which uh i came up with that you guys uh advertised on the pod years ago and um i'm like how do you actually pronounce this and i think i was the one who came up with zapier makes you happier and uh so a long time we uh it ended up in the footer of the website too i think it still might be somewhere in the about page i might have i might have been the origin story of that i'm not sure if i was or your marketing team thank you because like i use it all the time you know everyone always pronounces it zapier zapier zapier is what we used to we used to joke zapier anyway i don't care what you call Yeah, exactly.

1:07:05Go to Zapier.com slash twist. I think the landing page is still up and we'll see you all next time. Bye bye. On behalf of the producers and the partnership team, thank you for listening to episode 1769. We'd like to take one more time to thank our partners and broker. Use code twist to get an extra 10 % off insurance at and broker.com slash twist. Lemon.io get 15 % off your first four weeks of developer time at lemon.io slash twist. and 8sleep. Go to 8sleep.com slash twist to check out the pod cover and get$150 off at checkout. If you are looking to become a partner of This Week in Startups, you can email Hannah at hannahatlaunch.co.

1:07:49That's hannahatlaunch.co. Thanks for listening.

From the publisher

This Week in Startups is presented by:

Embroker. The Embroker Startup Insurance Program helps startups secure the most important types of insurance at a lower cost and with less hassle. Save up to 20% off of traditional insurance today at Embroker.com/twist. While you’re there, get an extra 10% off using offer code TWIST.

Lemon.io - Hire pre-vetted remote developers, get 15% off your first 4 weeks of developer time at https://Lemon.io/twist

Eight Sleep. Good sleep is the ultimate game changer. Now you can add the Pod Pro Cover to any mattress! Go to eightsleep.com/twist to check out the Pod Pro Cover and get $150 off at checkout!

*

Today’s show:

Zapier’s Mike Knoop joins Jason to discuss the early days of Zapier before breaking down the evolution of app integrations and API usage (1:20). They dive into reducing friction for Zapier users, regulating AI, the limitations of present-day AI architecture, and more (43:06).

*

Check out Zapier: https://zapier.com/

Follow Mike: https://twitter.com/mikeknoop

*

Time stamps:

(0:00) Mike Knoop joins Jason

(1:20) Zapier’s origin story

(8:40) Zapier’s key inflection point and its profit-sharing model

(14:05) Zapier’s business model

(15:48) Embroker - Use code TWIST to get an extra 10% off insurance at https://Embroker.com/twist

(17:03) The evolution of app integrations and API usage

(23:41) Lemon.io - Get 15% off your first 4 weeks of developer time at https://Lemon.io/twist

(25:00) Zapier demo + incorporating AI into your workflow

(36:43) Eight Sleep - Go to https://eightsleep.com/twist to check out the Pod Cover and get $150 off at checkout!

(38:14) Linkedin tightening the belt on its API

(41:21) Zapier’s enterprise customers

(43:06) Reducing friction for Zapier users

(51:41) Regulating AI

(55:19) The limitations of present-day AI architecture

*

Read LAUNCH Fund 4 Deal Memo: https://www.launch.co/four

Apply for Funding: https://www.launch.co/apply

Buy ANGEL: https://www.angelthebook.com

Great recent interviews: Steve Huffman, Brian Chesky, Aaron Levie, Sophia Amoruso, Reid Hoffman, Frank Slootman, Billy McFarland, PrayingForExits, Jenny Lefcourt

Check out Jason’s suite of newsletters: https://substack.com/@calacanis

*

Follow Jason:

Twitter: https://twitter.com/jason

Instagram: https://www.instagram.com/jason

LinkedIn: https://www.linkedin.com/in/jasoncalacanis

*

Follow TWiST:

Substack: https://twistartups.substack.com

Twitter: https://twitter.com/TWiStartups

YouTube: https://www.youtube.com/thisweekin

*

Subscribe to the Founder University Podcast: https://www.founder.university/podcast

More from This Week in Startups

All 653 episodes
Zapier Co-Founder Mike Knoop on category creation, API evolution & AI architectureThis Week in Startups · 1 h 8 min
Listen in VO