From 79¢ Tacos to Technological Milestones: Luke Freiler’s Journey

19 Sep 2023 · 43 min

Ask about this episode

Ask anything about it. ChatGPT or Claude reads this page and answers with the times it was said.

Connect VO and ask about every podcast you hear, including the moments you saved. Add to ChatGPT · Add to Claude

In short

Podcast Summary: Talking AI - Episode: From 79¢ Tacos to Technological Milestones: Luke Freiler’s Journey

Episode Overview In this episode of the *Talking AI* podcast, host Matt Paige interviews Luke Freiler, the CEO of Centercode. The conversation revolves around Luke’s journey into the tech industry, his insights on beta and usability testing, and the intersection of artificial intelligence and product development.

Key Themes

  • Career Journey: Luke’s 22-year trajectory from starting at Samsung to becoming the CEO of Centercode.
  • Usability Testing: The importance of beta testing and the differences between quality assurance (QA) and usability testing.
  • Agile Methodology: The advantages of agile development and how it has shaped Centercode's practices.
  • The Future of AI: Predictions about AI’s impact on the industry and reservations about its potential.

---

Key Moments and Discussions

Luke's Career Beginnings

  • Luke shares his early experiences in tech, starting at Samsung and later at Ericsson.
  • He was introduced to beta testing when tasked with running a beta test at Ericsson, which revealed a gap in the market for structured usability testing.

Understanding Usability Testing

  • Differences between QA and Usability Testing:
  • QA focuses on technical performance and test scripts.
  • Usability testing emphasizes user experience and real-world application of products.
  • Centercode’s approach to usability testing includes gathering user feedback over time to identify issues, improvements, and praise for products.

Evolution of Centercode

  • Luke discusses how Centercode started as an idea to fill a market gap and evolved into a platform that supports usability testing.
  • He recounts his experience with a pivotal first deal with Sun Microsystems, which greatly influenced the company’s trajectory.

Agile Development Insights

  • Agile development was initially challenging for Centercode, but it has evolved into a successful strategy, allowing for rapid iterations and constant user value.
  • The company now operates on a model that releases updates every week, encouraging agile feedback loops.

The Role of AI in Product Development

  • Luke expresses optimism about AI, suggesting it can enhance productivity, while also voicing concerns about its maturity and reliability.
  • He highlights the need for careful implementation of AI in usability testing and product development, emphasizing the importance of human oversight.

Future Predictions

  • Luke foresees a future where AI can automate some usability testing processes but believes human involvement will always be essential for nuanced understanding and insights.

---

Key Takeaways

  • Value of Beta Testing: Effective beta testing provides critical insights that can lead to substantial product improvements.
  • Agility in Product Development: Embracing agile methodologies can foster innovation and responsiveness to user needs.
  • AI as an Augmentation Tool: AI can enhance human capabilities but should be implemented thoughtfully to ensure reliability and user trust.

---

Resources and Links

  • [Connect with Luke Freiler on LinkedIn](https://www.linkedin.com/in/freiler/)
  • [Centercode Website](https://www.centercode.com/)
  • [AI Opportunity Finder by HatchWorks](https://hatchworks.com/ai-opportunity-finder/)

---

Final Thoughts Luke's journey illustrates the importance of listening to user feedback and adapting to market needs. His insights on the evolving role of AI in product development highlight both the potential for innovation and the necessity of maintaining a human touch in technology.

Written by AI. May contain mistakes. Listen to the episode to check what was said.

Hear the part that matters, and keep it.Open this episode in VO. Double tap your headphones to save a moment as you listen.
Get VO free

Transcript

Automatic transcript. May contain errors.

0:00Season 3 of the Built Right Podcast is right around the corner, but we've got one big change coming your way. The Built Right Podcast is now the Talking AI Podcast, and we got a lot to talk about in AI. In the Talking AI Podcast, we'll be having in-depth conversations with both AI experts and early adopters of AI. That way you can understand how the technology works and how early adopters are beginning to implement and more importantly, get value from AI. Our guests range from AI research scientists to founders of AI products to industry leaders putting AI to work in their business. While you're waiting for season three, go ahead and subscribe on your favorite podcast platform so you don't miss an episode.

0:41And make sure to leave us a comment about the AI topics that you want to hear about. So get ready to talk some AI in the new Talking AI Podcast, coming your way August 6th.

0:58Welcome to Built Right, a podcast by Hatchworks, where we help you learn to build the right digital product the right way. In each episode, we'll deconstruct the layers of successful product development, break down popular trends, and offer real advice to help make sure your product is built right. We may not have all the answers, but we've built a lot of digital products across a lot of industries, and we've seen a thing or two. Let's get into it.

1:33Welcome, Built Right listeners. Today, we're chatting with Luke Freiler, CEO of CenterCode. And Luke's journey is a bit unique in the fact that he's been running CenterCode for the past 22 years, starting as CTO, then moving into the position of CEO. And we're going to get into his journey and hear the type of stories you only get from a founder, like how he got the product market fit, insights on pricing, go-to-market, and really everything in between. But welcome to the show, Luke. Thank you, sir. All righty. So instead of like, you know, Luke, you're telling us what CenterCode does. I kind of want to take us back 22 years.

2:08Like go back to when you first realized this problem existed. It's a really cool story we were chatting about in advance. I want to get into that and kind of take us there to kind of get into what CenterCode does. Absolutely. So I started my career from a pretty young age in tech. I started out at Samsung, quickly moved with a big group of people over to Ericsson, which at the time was a hundred year old, hundred thousand person tech company. And at the time, the Internet was still pretty fresh and new. And I was young, but excited and pretty invested in it. And specifically, the thing that really lit me up was just the idea of usability and what eventually became user experience.

2:49You know, back then, tech products were pretty much used by tech people and anything that broke that mold was pretty novel. So I, on the other hand, had low patience, despite being a techie person and kind of grew up supporting, you know, the school network and printers and whatnot, and always thought there should be a better way. So that's kind of where my head was. And where I was at Ericsson was running a web development team that handled any sort of web oriented functionality for the product that we were bringing to market. So firmware of the product itself, internal sites, public marketing sites, everything.

3:23So I had a group under me who were all, again, sort of young web developers. And a product manager came to me and he said, hey, I need you to run a beta test. And I kind of lit up and said, OK, well, what does that mean? And he said, well, it's when you get a bunch of customers. And I was like, no, no, I know what a beta test is. I don't know what that means at Ericsson, right? We have a process for everything. I have a four-inch guide on how to use the logo. There's got to be a path for me here so I don't have to just create it. And he was like, no, man, best I can tell, there isn't. You're going to have to just figure it out.

3:57And I said, okay, can I outsource it? Can I go find a vendor? And he said, sure. And I said, okay, I'll do that. Give me a budget. So I started looking around. I started researching and I quickly realized that it was a gap in the market. What I discovered was this rare necessary evil in business, something that everybody sort of universally agrees is important, is positive, but it's hard. It's challenging. The idea of I'm going to get a unfinished product and give it to a bunch of strangers and expect meaningful feedback that I can transform into a successful product. It's just a tall order. So at that point, I saw that opportunity.

4:35And like I said, I was already kind of borderline obsessed with the future of usability and user experience and technology being more useful. And it all just kind of came together. And from there, I decided after talking to many, many people and asking everyone I knew and everyone within their networks how they did this, I kind of realized that the challenge was, it was something that happened episodically. Every once in a while, somebody would run a beta test for a product. Back then, everything was waterfall development. So it was a good couple of years between products. Product managers were heads down for a very long period of time.

5:07They would run a big beta test for eight or 12 weeks at the end of that. But then there'd be another two years before they did it again. So as a result, they kind of forgot everything they learned. And that led me to a lot of different paths that all turned out to be, yeah, there's an opportunity here. There's an amazing business opportunity that not only fits what I'm passionate about, which is incredible, but also is sort of universally positive. The idea of taking a company and helping introduce them to their customers and then sort of orchestrating the communication between them and helping them ultimately just produce a better product for those customers is just a universal win for everyone involved.

5:45So I never expected to spend most of my career in any one place. That certainly wasn't something I set out to do. I also didn't ever expect to be a CEO. I just kind of fell into something I loved and invested heavily in it, you know, my own personal time. And I don't see myself going anywhere. I absolutely love it. And we get to work with many of the greatest brands ever to bring products to billions of people. So it's awesome. Quick break in the pod. If you're listening to this podcast, chances are you've been thinking about how to actually use AI inside your business. And that's exactly why we built the AI Opportunity Finder.

6:18It's a free tool that helps you uncover high impact, tailored AI use cases based on your business, your goals, your pain points, and your industry. No fluff, no generic use cases, just real ideas that fit your business and the ranked by ROI potential. It takes about three minutes to run and it's like having your own personal AI strategist for free. If you want to try it for free, check out the link in the show notes or go to hatchworks.com backslash AI dash opportunity dash finder. yeah it's a lesson in you know we talk about the the built right method at hatchworks it's centered in building something that's valuable for the customer viable for the business and feasible like the marty caking kind of school of thought and you know we'll see clients a lot of the times they'll come to us with an idea but they're kind of missing like what's the problem behind it and you know so that's a key piece that often gets overlooked but i want to get into you mentioned this usability testing?

7:16A lot of people, when they think of testing, they think of just pure kind of QA testing within a solution, which is different though. Yes. And when you talk about usability beta testing, for those that may not know that kind of difference there, like what's the difference between those and kind of purpose of why you're doing these usability tests? So traditional just QA testing is pretty contained to professionals doing it. Now, there are sort of crowdsourced options nowadays, but ultimately you have people who are walking through test scripts and doing very explicit things to basically make sure the product is performing as it was expected to.

7:50On the other side of that, you have UX testing where it's much more focused on the experience. And do people understand it? Is it producing an experience that actually gives them the outcome they want? What we do is a little bit different. And the sense that it's all about real users, the people who actually have the problem, the product solving that will buy the product eventually in their real environments and then over a period of time. So whereas a UX test is often observatory, we're watching somebody do something and taking notes and taking action from that. And it's very qualitative in that sense.

8:24What we're doing is bringing something a little broader and it's more about, okay, how do these real people adopt the product over time? How do they use the product in their real environments, which then naturally leads to a lot of quality issues that you're not necessarily going to find in a lab. And many people do immediately recognize beta as a bug hunt. And that's certainly a critical part of it because the real world does just uncover an enormous amount of situations and scenarios that a business environment wouldn't. But it goes far beyond that. And from our perspective, we're always looking for three things.

8:57And we're looking for issues in the product. What's it not doing that it is supposed to be doing? What's not behaving as the customer expects it to. We're looking for ideas, what could be improved in the product, either small improvements all the way to features that could end up in future versions or what competitors are doing that might be an expectation they didn't see coming. And then the last thing we're looking for is praise. What do people love about the product? What is it that really stands out to them? And what could be promoted to grow the audience? And those three things together, issues, ideas, and praise, we sort of paraphrase it as fix, improve, promote.

9:30That's what we're going for is we're trying to look at this product in that real world and say, okay, how can we maximize the success of this product? And how can we use an audience of enthusiastic volunteers to make that happen? I wonder if this is another one or if it fits into the idea improved category, but we'll see a lot of times when a product actually gets into the hands of end users, they start using the tool in unintended ways. And the initial reaction is like, oh man, that's the wrong way. You're doing it wrong. But they may unearth whole new ideas with how to use the solution and get value from the solution.

10:03Oh, yeah. That promote side and the praise is where you find the things about your product that happen that are just happy accidents. And that happens every time. I mean, virtually every time there are going to be things about your product that are problematic that you didn't see coming. That's the most obvious thing. But to sort of balance out the morale, you're going to see just as many things that are new opportunities to leverage something you did in that product that you sort of emergently came out and you didn't even see coming. Take us back to your first deal. This is such a cool story.

10:36Your first deal, your first conversation when you got this idea of a product you're building. I think it's such a good lesson for folks in terms of really value. I want to say pricing, but it's all about value. I spent decades unpacking this deal and trying to figure out what to do with it. But yeah, it is a great story. So when I started this company, I was an engineer at heart. And I brought together a few people from Ericsson with me. We all left and started this. And I was leading the tech side. As I said, I wasn't a CEO at the time, but I was the product guy. It was my vision, my idea. And my first idea for this company was that this was going to be a platform that we were going to sell online for a couple hundred dollars a month.

11:16And it was going to be sort of SaaS before SaaS and that sort of self-acquisition before product-led growth was a thing. This is, again, going back 20 plus years. But that's how I thought. I wasn't a sales guy. I didn't want to be a sales guy on the phone. I just wanted to sell online and be frictionless so I could go focus on building something cool. And one of the first calls we got was from Sun Microsystems, who at the time was an absolutely enormous entity. They made the Solaris operating system, which was kind of the heart of business. Big deal. Eventually acquired by Oracle. But at that point in time, they were one of the biggest names in tech.

11:49And they called us and they wanted to inquire about the product. And I was in shock. Like I had not handled this. I basically was a kid and I brought someone on the call who I had worked with at Samsung initially, a guy named Scott. And Scott had spent 30 years at IBM, you know, growing to very, very senior positions running B2B sales at IBM. And I basically asked him to come be the adult on the call. I I would be there to answer any of the product questions. But as far as sort of orchestrating this call, I really needed his help. And he did that as a favor to me. He was great. And ahead of this, we hadn't talked about what the product would cost.

12:28For me, it was going to be a couple hundred bucks a month. It was going to be self-serve, credit card, all that kind of stuff. And that wasn't his world. But we didn't really talk about it because we didn't think the price would come up on the first call. My understanding of those large sort of B2B engagements were that they had numerous fact finding calls and so on. And it was going to take a while to even get there. So we get on the call, we show them the product, which wasn't done. It was far from done, in fact. And we were bootstrapping this company and kind of working it out on our own time.

12:56And I started trying to throw every objection possible at these guys. And I was saying like, hey guys, it's not done yet. And they're responsible. So that's great. We can help shape the roadmap. And I felt like they must have thought playing hard to get was my sales strategy, when in reality, I was just scared to death that they were going to overwhelm us and we were going to disappoint them. And, you know, it was going to be crushing. So anyway, we showed them the product. They love it. We get to the point where they eventually said, just flat out, you know, what does it cost? And I'm sitting there deer in headlights.

13:26I don't know what to say. And Scott and I hadn't talked about this. And Scott just immediately blurts out, well, it's$300 ,000 and$60 ,000 a year in perpetuity for support. And they flat out said, that sounds reasonable. We can do that. And I'm like, we get off the phone and I'm like blown away. Like I just found my funding, you know, from my perspective. And Scott's like, damn, it should have said 500. And I'm sitting here going like, yeah, man, I just said$200. So thank you, I guess. And we sold it to them and they ended up a customer for a very, very long time. It was a great experience, great relationship.

14:00And they did, in fact, basically, you know, without knowing it, fund the company for that year. And on top of that, like even it even more further when we said to them, like, hey, we're not on your platform. They're like, well, great. We'll just send you a whole bunch of hardware. And they funded all of our infrastructure as test equipment, like tens of thousands of dollars, which at the time was amazing for us. We were living on 79 cent tacos. So it was it was an incredible first experience. And it did sort of reshape the business in many ways moving forward. Yeah, it's such a good story just on the concept of perceived value.

14:34Right. And that's the key thing is kind of getting to a sense. How do you even get to a sense of what that is without having this serendipitous moment like you did where you had the guy in the room that did this? It's fascinating because I like that story because it's pretty wild and it's totally true when I was there. But it's actually not all good. And I once I spoke at a conference about software pricing once. and the guy that spoke next to me was a guy named Rick Nucci who was amazing. He's the founder of Boomi that sold to Dell for a bunch of money. Very successful guy. He runs a company called Guru now, I believe.

15:13And I remember him standing up before I spoke and said something along the lines of, you know, we've spent as much time on our pricing as we have our product. And I like literally laughed out loud in the audience. And the reason was because what I was writing or talking about to that group that day, this was some sort of SaaS conferences, is sort of our journey of pricing and how every price we ever had, like the pricing we have today could not look more different than that first price. It's actually in a lot of ways closer to my first vision for the company, ironically. But every price we had for about a decade and a half after that had some roots in that original price that we made up on the fly.

15:52And the reason was because that price was accepted, Scott basically threw down the gauntlet and said, look, man, this is what the market will bear. They accepted it. This is what's going to happen. And what we recognized later is that they were actually a unique company in a unique situation. What they had was an operating system to test. And if you think about it, operating systems were basically the first agile developed products in tech because you couldn't just sit on your hands for two years in between updates. There is security problems. There's compatibility issues constantly coming out.

16:25So people have been patching operating systems frequently forever. And where I'm getting at with this is they had a unique need that we didn't understand was a unique need, yet we pinned that as the market. And for years after, and I used to have this deck that kind of beautifully outlined this, every price that we came up with from there, every new model we came up with was somehow traced back to that original price. So if it was first$300 ,000 with$60 ,000 in support, I would make the argument of, oh, let's get it down to$120 ,000 a year, but now it's$120 ,000 a year. So over a certain course, it's the same amount of money.

17:00Then we would get and say, okay, well, our average customer has six seats, six full-time users who live in this product. So let's sell it for$20 ,000 a seat. And I could actually trace back like a decade and a half of pricing strategies that were all hooked to this idea of we don't want to give up or cannibalize that very first customer. And the problem with that thinking is we didn't except for the longest time that that first customer had a very unique model. And again, the irony is they were just early to agile. And because they had agile development, they had a continuous need. They could justify building a team around this in one place.

17:34And therefore, they could justify the budget to support that team. Whereas for the next decade plus, most companies were putting this on individual product managers like we had at Ericsson, which meant they were only actually running a beta test every couple of years. And when you have that, when you're decentralized in that form, you don't have a centralized team, a centralized process. So you don't have tools to support that centralized process. You don't have a budget for those tools. And that actually held us back. Again, we were bootstrapped. So things like awareness weren't something we could invest heavily in.

18:05And as a result, that kind of held us back sticking to those guns. And it was only in recent years after I took over CEO and wanted to get a lot more aggressive with how we approach the market and rethink what our go to market strategy is that we got around that. So but we lived with those. I mean, it funded our first year. I can never say it was a first mistake. It really did help put the business on the map, gave us an incredible customer and a wonderful relationship and all sorts of social proof. But it also kind of stuck us to our guns way longer than we should have because we didn't accept that the context changed.

18:34And again, now any company that's grown up in the last 10 years or so is agile. They're typically doing consistent development of some, you know, most products or services now. So our newer customers, the Pelotons and Sonos and Rokus of the world, they all have teams for this. They all centralize efforts for this because they have a constant need. Therefore, again, it went back to that idea that we need a tool or services to support that. So, yeah, it's been an interesting journey that's been kind of in all directions. That's really interesting. It's interesting that initial frame of reference.

19:06It's just kind of burned into your memory almost like you talked about. You're always working from that point. And it was kind of, you know, it's taking you down the wrong path in a sense. That's really interesting. Let me get back real quick to, you know, the product wasn't fully ready in the beginning either. You mentioned y 'all were bootstrapping this because you were coming out of effectively the dot-com boom and bust. It's not like it was by choice per se. Didn't even try to raise money. No point. I think this is an underserved thing from a lot of companies. Services is a lot of times a very viable way to A, help fund what you're doing, but you also learn as you go.

19:50I know with Hatchworks, we've been bootstrapped as well. We are a services company, so it allows for more of that. But talk me through that journey. I think that's a really interesting story, how you got kind of into the services game, and it's evolved over time. So, yeah, even before that big software deal, we had spent about, I don't know, nine months or a year kind of building up the platform. And at this point, we're taking on separate contract work because we're all like 20, 21 years old and don't have life savings. So at that time, through a friend of a friend, we met this brilliant guy named Mike Fine.

20:21And Mike had spent a bunch of years running beta teams at a few big tech companies. And at the time, he was the most passionate person I had ever met about this whole concept, this whole domain. And when he heard we wanted to start a business about it, he was just instantly all in. It was in his blood. And what his perspective was, and the pitch he gave us that to this day is still alive, was, hey, while you're building the software, why don't we deliver managed services on top of it? There is a need out there, especially if you look back at that sort of episodic need of once every year or two per product manager, they have a product.

20:58He said there's a need out there. And if we're delivering on the software, it doesn't need to be done. Right. The last 10, 20 percent of the product takes 80 percent of the time when you're polishing and getting it ready for users and going through the UX and everything else. So, you know, when we're working close in hand within the company, we don't need that level of polish. We can do the heavy lifting behind the scenes. And he made a compelling pitch. And we started working with them virtually immediately. And we started delivering managed services. And, you know, initially it was challenging because we're now effectively running two companies in one.

21:29We want to be a software company. We also have managed services. But at the same time, those services lean on the software heavily. They're a tech-enabled service. And that means they have pretty great margins. And as we innovate the software, we can directly have an impact on the margin of our services, which is a really great thing. So to this day, we still offer managed services. It's still a part of the business. And most importantly, as you alluded to, it became what we think of as our test kitchen. We knew that we now had this central team that ran a more diverse set of beta tests than virtually any company in the world.

22:02And therefore, we could learn and implement. And from that, we started to recognize that, hey, one of the problems in our space is that there's no framework, that everybody does it differently. Everybody's kind of recreating it from the ground up and everybody's wasting a lot of time and the opportunity cost there for them is huge. So what if we stopped thinking about just technology and started thinking about the process and everything that's layered on top of it that companies could reuse? And being that we are in the center of this and do so many of these tests, we don't even have to go learn from everyone.

22:30We can and we do, but we have it direct. We have the information that they would never want to share with us anyway. It's right in our wheelhouse. So that led us to making a framework and certifications and everything else and just kind of mastering our space and recognizing that if this category is going to flourish and achieve its goals, it can't just be software solving a problem. And therefore, you need, in IT, the typical way is software process and people, put those three sort of things together to achieve something. And we recognize that we have the software and that's what we're focusing on.

Read the full transcript

23:02The people were our customers. And ironically, the process we'd already been doing, but never really thought about it as a gap. So it's been a while now since we implemented that. It's been pretty successful for us. We've been very happy with it. But our goal was to just get rid of that first thousand hours that every company was going to have to use to reinvent their own wheel. And it's been great. So services to this day are a big part of our business and likely will be indefinitely. You kind of hit on the topic there. It's the dogfooding element of it, right? Where you're using your own product to enhance the product.

23:33How do you think of that in terms of, you know, thinking about your own roadmap for your product, how you think about, you know, prioritizing your backlog, like what's going through your mind? Because there's all kinds of different levers from taking care of technical debt to new features to like, what's your approach to that? So I'm uniquely positioned to think about that probably more than most CEOs, because I am very much a product driven CEO, I still lead our roadmap. I'm still, I live in Figma, like a product manager would it is like I wake up Saturday mornings and work in Figma for fun. I'm just that kind of nerd.

24:07So for me, I'm very, very involved. And I beg my accounts team to get me roadmap meetings with literally every customer possible. I show them everything we're working on. I take feedback in real time. I prioritize it from there. It is a process of, you know, we are on a very agile process where we have a release every week, which is typically facing bug fixes and small improvements. And then we have a feature release, a significant feature release every month. And at any given time, we're working on three to four months of those with different subsets of devs. So for us, it's very interactive with our customers.

24:41we're in the process right now of shifting. In fact, we'll be announcing it. It actually might already be on our What's New page, but if not, it will be this weekend. We're shifting to a feature flag-based beta program. So in the past, we've had what we call our Go Early program, which our customers can opt in and they can either opt in their main implementation or they can opt in a stage. And we have a unique challenge in that beta testing, the beta testing platform is a snake eating its own tail. And we don't want to create friction through an unfinished product for their customers who are dealing with an unfinished product.

25:10It multiplies on itself. So what we're doing now is the more what I would consider modern approach of being able to flag on and off certain features on demand for different customers and basically pre-releasing everything at least a month ahead of time, giving them some time to adopt it and give us feedback on it, implement those improvements and changes, and then roll it out in the next month and roll out the next new thing. So this month, actually, as of this weekend, we're going to announce what the first sort of feature flag feature is. everybody's, even the customer is able to opt in and try it out.

25:41And then as of this month on, we'll have those with each new release. So, but you know, our main process is to dog food our own stuff, to use as many customers as we can. There's a lot of challenges with that. Like I said, snake eating gets on tail and our customers are very busy, but thankfully we got a lot of tactics and tools to get past that. So it's been pretty successful for us. And you talked about agile in this process of releasing every week and all of that. How long has that been going on? Like, Were you all agile from the very beginning back when it was like this new novel concept? No, no.

26:14So yes and no. In fact, and I think many companies have gone through some variant of this, and I'm always willing to follow my sword and talk about it. We were agile from a very early age, and I just fell in love with it. I think I was CTO at the time. And I remember my first write-up to the company about how interesting and amazing it would be to finish a feature and get it in people's hands. The main pitch for Agile was rather than doing a big release infrequently where half the features are done and sit on a shelf for six months while the other half get done. And therefore, they're not producing value for your customers.

26:47And when people are paying for something monthly or annually, they want persistent value. They're paying for a service. It just clicked in my head. There's a few things in my career that immediately just like, oh, my God, that's amazing. Why aren't we doing that? Content marketing was one. I absolutely love it. This was another. So we tried it and we built our own system and we did it kind of our own way and we failed miserably. It just did not go well. And part of it was at that point, the enterprise was not super savvy to agile delivery or acceptance. In other words, they didn't want releases to come that quickly.

27:21And when you have a company who builds a process and builds a team on top of a platform, they need a lot of time to redo how that team works. And anything you change can disrupt that team. So we were at this situation where we were capable of releasing monthly, but we have customers who are basically saying, yeah, I only want it quarterly or I only want it every year even. And we're sitting here going, OK, what are you talking about? These guys are going to get it every month and these guys are going to get it once a year and we're going to support the 11 that they don't have. And it was just a nightmare.

27:56So eventually we basically walked away and said, OK, we're not going to do Agile anymore. We're still going to operate Agile internally, but we're not going to release Agile because we'd gotten used to it and started to like it, but our customers weren't accepting of it. And so then at that point, we're releasing every like three to six months. We'll do a big meaty release just because at that time, our customers were pretty much exclusively enterprise. And that's what made sense. And as we built a bigger team and our engineers got more confident and we put project management instruction in place and whatnot, we started to get pretty confident that we could do the rollouts in a better way.

28:30And more importantly, this is what happens. Everybody kind of matured and it became the norm. And it's the companies like Slack and whatnot that don't give you a choice. And therefore, everybody kind of got used to it. So now, and I'd say for about a year now, no, two years now, we've been more than that. I don't know how long. For a long time now, we've been pure agile doing these monthly releases. You'd have to go through the mega backlog that is our what's new site. And our customers seem fine with it. Every once in a while, you'll get a little bit of pushback on something because we didn't do a good job of prepping them for it.

29:03And we've done some interesting things that we question, we constantly experiment. We always release our release notes early now. And I still question if that's the right idea or not. There's pros and cons to it. But at least if those customers are interested, they can get a sense of what's coming. They can ask questions ahead of time because they might not want to be part of the test. They might just want it when it's out, but still want to know. So we're constantly experimenting. It's a challenge. And it is directly proportional to sort of the size of the company. If they're a slow moving, very, very large company with a big infrastructure, chances are they're not going to adopt most of your new functionality anyway.

29:38So you have to find a way to roll it out that doesn't impact them. And the smaller guys want every bit of value and efficiency they can gain. So they'll take it as quickly as you can give it to them. So we try to make the best of both. Yeah, interesting journey there. Kind of back and forth a bit. But so I want to get into the hot topic right now, whether it's hype or not around generative AI. You know, the concept of even large language models been around for a while, but the hype around it and the access to it is just accelerated. I'm curious from your perspective, how does that impact your business, your product, the concept of usability testing?

30:15Are there any just like views you have of the future of how things just completely stepwise change from what they are today related to your world? Or do you see it as this, you know, just it's hype and it'll die down and not much change? I definitely don't see it as just hype. I mean, obviously, there's a hype component to it, but I think it's incredible. And I've been fascinated since like day two. My marketing VP was fascinated from day one and he spent a few days trying to get me interested. And I played with GPT when it was a little earlier on and it wasn't as impressive. And then the thing that turned to me was when, well, two things.

30:51One, he had to write a rap about our company. And what was amazing about that was he didn't tell it anything about our company. It just knew. And, you know, we're a pretty small player for a site we've never heard of at that point in time to know about us. But it wrote a very accurate rap about our company and culture that just broke my brain. And then the next thing he had to do, which again, to this day might be one of the most impressive things I've ever seen it do. This is like last November, was he had it write a social post that was Thanksgiving themed, it was holiday themed. And what was amazing about it is it created its own analogy.

31:25All he did was said, I want a social post that's Thanksgiving themed for my company CenterCode. and it knew that we were in the beta test management space and it made this insane analogy about how beta testing is like the final spicing of your food. And if you don't put the spicing on at the right time, your meal is going to be a failure. And I'm sitting here going, I can't process mentally how it came to that. We've been doing this for 20 years and we've never thought of that analogy. And there's a temporal aspect. I mean, it really is incredible. It was very shortly after that, I basically went to my team and I still subscribe to this line of thought of like, look, guys, AI is not going to replace you.

32:05Someone who uses AI is. So let's be that. Let's be that person. Right. And I kind of put it out there of like, let's imagine that AI is saving each of us, you know, 20 minutes a day. And let's imagine the company over there isn't using it all. How much of an advantage do we have as a company once you sort of roll that up that we really have no choice not to say yes to? So I'm very positive on it in that regard. At the same time, I have all sorts of reservations of just kind of how we mature it is. One of the interesting things is there's just no such thing as a mature AI product right now. We just did our entire performance review program process with an amazing tool.

32:46I will give them a shout out. It's called HyperContext. We use it to manage meetings, but they invited us to try out their new tool that uses AI to write performance reviews and kind of imagine a scenario where you're already collecting a year's worth of meeting notes for everybody in one system. What could you do with that in sort of understanding how people are performing and whatnot? It's just a fascinating thing to think about. But as you might imagine, as it's amazing, it's still immature. So, you know, there is nothing that's mature yet, so we can't rely on it. They approached it very smartly to where it's basically giving you a starting point, which is a great way to think of AI.

33:20They're not trying to say, here's an AI review, just hand it to them and walk away. They're basically saying, here's everything we learned, now edit it and make it your own and make sure it's accurate and whatnot, which I think is the right way to approach AI. We have a bot in our tool that we started developing about three years ago. And there's some religion to this, but we refused ever to call it AI. We never thought of it as AI. We still don't think of it as AI. We certainly think of it as machine learning, kind of a tier below AI, but it's not what ChatGPT is. And on one hand, you could say, oh, what I'm really saying is it's not as impressive as ChatGPT is.

33:54And that's actually not what I'm saying. What I'm saying is it's not as powerful or unpredictable as ChatGPT is. And in our world, what our bot does is effectively manage all tester engagement for you. The biggest problem in our space is getting people to engage and participate and give you feedback. and we built a bot that builds sort of a behavioral profile on every individual, communicates them through the channels that they prefer at the times that they prefer, simply by watching how they perform and what they do. And all of its messaging and everything is unique. And it's a bot in our sort of go-to-market ones.

34:28We're not going to say it's a bot, but we're also not going to say it's not a bot. We're just going to kind of see what happens. And most importantly, we control the code 100%. We know exactly how it works, which is not true of most generative AI and sort of the black box of it. So we know it's not going to go rogue and piss off somebody. But we also know that it's not sending data anywhere. It's not. It's all within our systems. It's all controlled and whatnot, which is another big problem that most of our customers have. So we've ran contests internally and prototyped a million different ideas and whatnot.

35:00We've run contests to think of every single way that AI could benefit our company as a tool, every single way we could implement AI into our platform as a tool. And we've got this enormous backlog of incredible ideas, but we have to be careful with it. It has to be reliable. It has to be somewhat predictable. If people are going to sort of put their brand behind it, they need to be sure that it's not, you know, Skynet. So that's a lot, but I'm in love with it. I think it's fascinating. I was so excited, like in the early days of showing my mom and my, you know, my kids. it's been just fun to watch people's brains explode and try to figure it out.

35:38To me, it basically went the PC, the internet, the mobile phone, and this. Those are the major technological breakthroughs we've had in our lives. And this one is just as big as the rest of my mind and could be bigger. Yeah, I'm with you there. I think it's on that same level. Let me ask you this. Do you see a future where you actually have a generative AI agent or bot, whatever we want to call it, doing the usability testing instead of an actual person? Is there some utopian future where that's the case or do you think it's always you need that human involved in the process? I would not be comfortable at this point in time predicting too far out on anything with AI.

36:26I don't. Yeah, I'm not. I'm looking to hedge my bets, I guess. But, you know, I again, I would have never imagined it could have written a rap that was weirdly sensible about my company without me telling anything about my company. It's just unwinding that is still fun to think about and endless. I mean, I've never felt both more excited and overwhelmed about a technology in my life ever. It's just nothing's happened this quickly. So the rate of switches happened is impressive. But at the same time, I do think we're about to see a lot of diminishing returns. I think they kind of had it under wraps for a long time while they did a lot of the back work and then they exposed it and it moved really quick for a little while.

37:04But again, that's because that was already done. They just hadn't really rolled it out yet. They weren't confident in it. Now I think we're starting to see a little bit of a slowdown. But now what's cool about AI is we're moving from the tech being the impressive part. And now the impressive part is the creativity of all the developers who now have access to that new thing and can think about entirely new ideas that aren't out there. So back to your question, though, I think there will be a level of user testing and product testing that happens in the AI space. I mean, I know there's already companies working on it.

37:37We've talked about it. We've thought about it. It'll happen. Will it ever fully replace? I doubt it. In part, again, not knowing where it goes. Who knows? If it's a robot in your house that can walk around, then it's a whole different game. But there's a lot of things that happen in the physical space and have to do with humans and whatnot that are very, very, very difficult to predict. But could it, like all things, take out the 20 % of stuff that wasn't quite obvious but is predictable? Yeah, it could definitely do that and will. But depending on the product, just the nature of what technology is, I think you're always going to have to be testing what comes next.

38:13And I don't believe that's... Yeah, and anyone looking to go a little deeper on the generative AI side, check out our previous episode with Jason Slachter. You mentioned the idea of like, you know, if everybody could be 20 minutes more productive, he kind of framed it as, imagine you have a thousand interns or a small country trying to get people into thinking, framing outside of their normal working model, which was kind of an interesting or mental map, I guess it's just a interesting way of kind of framing things. That's very savvy. Yeah. But Luke, I really appreciate the time today. And where can folks find you, find CenterCode?

38:52What's the best way to find y 'all? Yeah, absolutely. So centercode.com is the website, as you might imagine. My email is just lukeatcentercode.com. I'm not nearly as active on the social media as I should be. So while I am on LinkedIn and you can connect with me and whatnot, I'm also just kind of an old school emailing guy. So feel free to shoot me a message. Our product is very different than the stories we told. Initially, it was aimed for big teams and whatnot. In recent years, what we've done is evolved it to really target product managers and all the way starting it free to run a whole program on your own to a very inexpensive self-serve.

39:27And within just a few minutes, you're going to be recruiting your own customers. And I think it starts at about 40 bucks a month. It's really, really simple and straightforward. But then as your program matures, if you do choose to centralize and whatnot, the platform can grow with you to more of its traditional base. So kind of options for everybody. If you're looking to engage with your audience or even do internal dog fooding, anything that involves using your product and gathering feedback, again, issues, ideas, praise or whatever else is going to help improve your product. Check us out. You can try it for free.

39:54You can play with it. And my team is built of product managers who are there to help you and talk about it with you. So happy to help. Yeah. And I know we didn't get into that, but that's an interesting piece from the evolution of the 300K to now this PLG motion where you can try it for free. And that was the unique thing too. I know we kind of just ended, but keep going. The fact that you have product managers almost acting as your salespeople just kind of blew my mind a little bit. You actually have knowledgeable people talking to the customers that actually know the customers in a sense. Yeah, when I talked about doing that and talked about why I wanted to do that, a group of peer CEOs thought I was crazy at first.

40:34And as I've gone through it, I've really enjoyed it. There is certainly a place for sales. And for me, it's more about relationships with the customers and growths and making sure their needs are met. But what I don't want is somebody on the phone with a customer who doesn't have the agency or the right motivation. My dream or from a sales perspective, and we're there now, is nobody selling the product as a commission. And that's because I want them to be motivated to sell the right thing to the customer as opposed to the right thing to hit their quota. That quota is valuable to the company. It's how we eat.

41:08But you sell the right thing to the customer, they're going to use it, they're going to grow, and it's not going to be an issue. So, yeah, I'm very proud that we have a product focused team. And yeah, we have the PLG method in to get people going. And then if you do want to move up from there, we have experienced product managers who are literally the product managers of our platform that are stepping in to help with those conversations. And I'm really excited about that. And I'm trying to use as much of my time to have those conversations as I can because they are one of the funner parts of the job.

41:35So it's a good time. That's awesome, Luke. Well, thanks for joining us today. Have a good one. Thank you, sir.

41:46Thanks for listening to Built Right. If you enjoyed the show, give us a follow or subscribe on your favorite podcast platform. And don't forget to leave us a review. For more info on Built Right, visit our website at hatchworksbuiltright.com.

42:06The single biggest mistake we see companies make with AI is they don't properly train their teams. We see it all the time. Companies roll out AI tools and expect people to just figure it out. But using AI effectively requires a totally different mindset and skillset. And that's exactly why we built training for every level of your org, from AI training for teams and executives to training engineering teams on our generative-driven development methodology. Or if you've already identified your AI use cases and want to just prioritize where to start, we offer an AI roadmap and ROI workshop to help you build a quick plan.

42:39It's all about going from we should use AI to actually driving real value with it. Head over to hatchworks.com to learn more.

From the publisher

Beta testing and usability testing are crucial elements of the product development process. We wanted to learn more about the beta testing space, so welcomed Luke Freiler, CEO of Centercode, to the Built Right podcast to talk about his journey and his insights as a CEO and Founder.  

Luke takes us back 22 years to how he first got interested in the space and eventually became the CTO and then CEO of Centercode. We hear his thoughts on agile working, AI, building a framework into the testing stage, and what his company looks for during tests.  

Key Moments: 

  • How Luke began his career 
  • Where Luke saw a gap in the market  
  • The key differences between QA and usability testing 
  • The three things Luke’s company focuses on 
  • What held Centercode back in the early days 
  • Why Luke loves agile development 
  • Luke’s future predictions on AI  
  • His reservations about AI  
  • Centercode’s unique approach to sales  

Key Links: 


Mentioned in this episode:

Talking AI - Conversations with AI experts and early adopters

Welcome to the Talking AI podcast, where we dive deep into the world of artificial intelligence with host Matt Paige. Formerly known as the Built Right podcast, Talking AI brings you insightful conversations with AI experts, founders of AI products, and industry leaders who are leveraging AI in their businesses. Whether you're an AI expert or a beginner, our episodes will help you understand how AI technology works and how early adopters are deriving value from it. New episodes drop starting August 6th.

AI Opportunity Finder

Feeling overwhelmed by all the AI noise out there? The AI Opportunity Finder from HatchWorks cuts through the hype and gives you a clear starting point. In less than 5 minutes, you’ll get tailored, high-impact AI use cases specific to your business—scored by ROI so you know exactly where to start. Whether you're looking to cut costs, automate tasks, or grow faster, this free tool gives you a personalized roadmap built for action. 👉 Try it now at https://hatchworks.com/ai-opportunity-finder/

More from Talking AI

All 84 episodes
From 79¢ Tacos to Technological Milestones: Luke Freiler’s JourneyTalking AI · 43 min
Listen in VO