Here Come the Robo Umps…

0:04Congratulations! You made the varsity of the podcast. My name is John Arand and I am Puck's sports correspondent and the host of this pod. And today, I am happy to be joined by Puck's newest partner, Ian Krietzberg. This week, Ian launched a private email on the trillion-dollar AI industry called The Hidden Layer. You should go to puck.news and sign up for it right now. I'm really eager to get Ian on the pod to talk about where we see AI in the sports business today, how it's going to develop, how fast it's going to take, and really just to talk me through the AI revolution that's coming. But before I get to Ian, today is Wednesday, July 9th, and here's what I'm focused on.

0:49Once again, it's college football. I can't recall covering a business that has gone through so much upheaval as college sports. You have NIL, the health settlement, conference realignment, potential expansion of both the college football playoff and the NCAA basketball tournaments. Well, this week, expansion of the college football playoff is back in the news. It's inevitable that it's going to go to 16 games. But what's interesting to me is that while everybody seems to want expansion, the top four college conferences have serious differences about how the format's going to look. Just this week, the Big 12's Brett Ewermark reiterated his hope that the extended CFP is going to have five automatic bids awarded to the five highest-rated conference champions.

1:37Then, under his plan, it'll have 11 at-large bids. In making his comments at the Big 12 media days in Texas, Ewermark also suggested that ACC Commissioner Jim Phillips supports that plan. But your Mark's comments came soon after the Big Ten's Tony Petitti offered up a different idea. Petitti wants four automatic pids for the SEC, four automatic bids for the Big Ten, and then two each for the Big 12 in the ACC. That would leave four at-large bids. And I don't really care about that structure. What's interesting here to me has less to do with how the selection process actually will work. Rather, it's the increasingly public disputes that are pitting the two most powerful conferences, the SEC and the Big Ten, against everyone else.

2:27I mean, these are issues that typically get solved in a conference room. But now they're being aired in public and showing that with regard to all of the changes in college sports, you ain't seen nothing yet. Now, let's get to Ian Kreitzberg. Ian, thank you for joining us on the pod here. This is the first time I've ever done this, but to prepare for this interview, I went to ChatGPT and I said, I'm interviewing Ian Kreitzberg for the podcast, What Should I Ask Him? And this was ChatGPT's first question. Ian, can you give us a quick overview of your background and how your work intersects with both AI and the sports business, which means I'm going to have a job for a little while.

3:15What a terrible first question. That is. John, that stuff is so funny. I feel like that was so in vogue when right after ChatGPT came out, everyone was like, hey, I wrote this with ChatGPT. Could you tell the difference? It's been a couple of years. I can still tell the difference. And it's more than just the M dash. I think that's what everybody says. You can tell if it's chat GPT, if they, if they overuse the M dash, which by the way, I've overused my entire journalistic career. Don't even get me started on the em dashes. I hate that because you could tell what happened is there was an attempt to, you know, it's like a writerly thing that we use way too many em dashes or sentences are way too long.

3:59This is what we do. I love em dashes. And so you could tell that there was a moment in time where there was an adjustment and over adjustment to put a lot of em dashes in there to make it look more real. And now everyone sees an em dash and they go, that was AI. Well, anyone who ever reads me, that's all me. I just really love MDashes. Yeah. It's so much more effective than just a simple comma, for goodness sakes. I mean, come on. Yeah. Plus it looks cooler. Yeah, exactly. Not an MDash, but the full MDash. First of all, welcome to Puck. You've been at Puck for now a couple of weeks. Your very first private email came out on Tuesday.

4:39What's a big story that's in it. So the big thing that we're looking at, and this was just kind of a nature of the stuff that was going on right when we started getting going, was kind of what's next for AI policy and AI regulation. And so obviously, shortly after I start, you have Trump's big, beautiful bill. And my entry into that is that there was this AI proposed moratorium that would have disallowed states from passing legislation on artificial intelligence for, at first it was 10 years, and then they started adding caveats to it, and they cut it down to five years, and they added language to it, and it ended up getting cut out.

5:22And it was kind of an interesting moment to see, I don't know, what does that mean? Congress hasn't been doing anything for regulation. The states have been trying to pick up the pace. Some states are going really, that they're really taking a big swing at it. Other states are being a little bit more targeted. And so where do we go? The question that I had was, was this a sign that Congress is just going to say, listen, you guys got this. We'll sit back and watch. Or is this a sign that kind of the normal legislative practice is going to happen here where the states are going to experiment, and then maybe in a couple of years, the federal government will do something.

5:59And so we aimed to kind of address those questions and paint a bit of a light in terms of what the heck is going to happen next. It moves so fast. Well, we see this in sports a little bit with the whole college landscape. The NCAA is like begging Congress to regulate it and to sort of step in and help save it from itself. And it seems to me that there's quite a big sector of AI is viewing it the same way. Like we want regulation, please put, put, put, put on some guardrails, uh, out there, which is, uh, unique from just about every other business that I've covered. Yeah, that was the fascinating thing.

6:40Um, and, and this kind of goes back to the very beginning of when this field, I mean, this field has been around for decades. Um, so it's not new, but the, the kind of recent introduction to the public where we talk about the term AI and artificial intelligence, which These are very abstract, very vague terms. It's not clear what they're referring to. Everyone has different definitions. But yeah, for executives of these companies to sit before Congress and say, please regulate me. What's been interesting is that that has kind of started to shift. Um, we've found that, uh, you know, one, one, one source I talked to kind of put it like this, where at a high level, these companies like the idea of regulation more than they like the actual implementation.

7:27No, I, yeah, I asked for this. I didn't ask for that. Yeah. Right. The interesting thing is then you have a different set of companies that are maybe not like the big, massive companies that really need rules of the road. They need to know how to build stuff that is in compliance so that they don't build something that in six months is not in compliance. And so, yeah, for a number of reasons, regulation would not be a bad thing. Every industry is regulated. It's not a bad thing, but there's so many narratives here. and efforts to just complicate the push there. I want to ask you then that question that you sort of brought up, specifically as we're talking about the sports business and bringing AI out in the sports business, what is AI?

8:17How would you define it as it pertains to sports, period? This is a very good question, John. This did not come from ChatGPT, by the way. No, it would not have been able to. I think as it pertains to sports, You're dealing more so with computer vision models and kind of machine learning models to make kind of predictions about certain things. That would be, from what I've seen, the kind of sports definition of it. That's the kind of stuff that we're starting to see getting incorporated into use by different leagues and different teams. Take me through some specifics on that. Like, what are you talking about?

8:59I mean, obviously the thing that's at the top of my mind right now, because I'm seeing so many articles about it, you've got what's going on at Wimbledon right now. By the way, I got to interrupt you right now. AI, I love having you on here because AI terrifies me. So when I read what's happening over at Wimbledon with AI sort of going offline and just not working, I feel good about that. Humans still matter. Humans always matter. I think the funny thing about what's happening in Wimbledon, I don't know, there's kind of a few things that I find funny. But the main thing is that when we think, when we're told to think about AI, where the framework around that is you're dealing with this, you know, otherworldly species that we're creating.

9:47And that's not true. That's not what's happening here. And if you, I mean, AI aside, I feel like we've always been hearing, you know, technology. It's always breaking, right? It does really good things, but then it fails in very annoying ways. Think about Zoom or a fax machine, computers, things crash. Technology does not work all the time. This is more technology that just doesn't work all the time, which makes it really funny to just replace the people with it. And so they're at Wimbledon. They got rid of the line judges, and now it's the electronic line calling, which uses a bunch of cameras and AI processing to automatically call if a ball was out or not.

10:36That's been in use beyond Wimbledon for a couple of years now. For many years, it's been in use in terms of full replacement. That's a little bit more of a recent thing, but the US Open is doing it too. And what we've been seeing, the whole scandal, is that it's not really working. It works most of the time, but then it's making these very specific errors that are at the very least throwing off the rhythm of the game. One kind of silly oversight is that the players have found that it's hard to hear the calls, which is really funny because then, I mean, if you think about the rhythm of an athlete in motion, now you're spending time turning your head going, what?

11:17What was it? Right? That throws you off and then you have calls that weren't called that then the umpire comes in and they go okay like we'll replay that point but a point had already happened you're off your game it it changes the kind of organic flow of the game um and so folks are are not very happy um watching that roll out on on a pretty significant stage the the whole point behind ai is that it's going to be better tomorrow than it is today and it's going to be better next week than it is this week and you can see where it's going to be like they can increase the sound the players can't hear it okay well let's let's increase the sound or you know uh i did like that it was off for three points i think on on a match on sunday and uh the wimbledon officials described it as human error that uh you know which always reminded me of you know that scene in airplane where the uh the one guy unplugs the uh you know all the flight tracking software.

12:19But I mean, it's going to get better, obviously, right? That's the idea. It's just, you know, where could it stop? Where will it stop? How much better is it realistic for these systems to get? And that's the interesting point. You've got some players in the space that are talking about human level intelligence and the pursuit of replicating that artificially. Then you have others who say, we don't want human level intelligence. Humans screw up all the time. Let's go better than human. Some people call that superhuman or super intelligent, which again, these phrases don't make sense because if it's different and better than human, it's not superhuman.

13:08It's not more fundamentally human. It's just better. can we make something that's more accurate? It's kind of a question mark. We don't know. The assumption is these things get better, but often you're dealing with incremental improvements. And in terms of, you know, if we're talking about electronic line calling, these are systems that are at play that have been in use in testing for many, many years. And before Wimbledon decided to go ahead and get rid of their human judges with their awesome outfits. They did a whole testing series on this stuff and they determined it was good to go and here it is making errors.

13:48So if you're trying to pursue 100 % perfection, that's likely just by nature of the complexity of the world, not going to be a realistic goal. And so then it's which place are you plugging this technology in and what's your margin for acceptable error? If the best we can expect to do is 95 % reliability and accuracy, 95 % could be good or it's really bad because it gets you too comfortable and you're not looking for the 5 % thing, the 5 % gap. And this is the challenge. This goes far beyond Wimbledon and into everything else. I think of self-driving cars. I think of AI and finance and the idea of robo-advisors and stuff, which is concerning for the 5 % chunk that unless you're trained to look for it, that you have to be comfortable with an error rate.

14:49Perfection is not really attainable. Let me take a quick break, and then I want to come back and I want to dive down deeper into league executives, team executives, media executives who listen to this pod, like, how should they be approaching AI? That's my next question for you after this break.

15:17All right. I actually already previewed my question to you. How should people that are invested in the sports business, let's start, let's start with leagues and teams before we get to media. How should leagues and teams be approaching AI? I like to think about, and I talked about this before on the powers that be of kind of intentional use and just being very thoughtful with, you know, what specifically are we using? And more specifically, what are we hoping to accomplish? I think if you start with the problem and then go backwards, you're going to be way better off than if you just say, hey, I've been hearing a lot about this thing called AI, and I want to start using it because it'll make me look cool.

16:02That's not going to help you out. Then you're looking for a way to use it. But if you start with a problem, right? Like the NFL has their digital athlete program, which uses machine learning to build this kind of replication of players' experiences so they can make sure, and then they run simulations based on that. and this way they can better understand what players need to do to stay healthy, recover more quickly, perform at a better level. That's kind of an interesting application. This has been around for a while. You're talking about machine learning. So you're dealing with processing a bunch of different data points and the idea of predictive analytics to prevent players from getting injured.

16:48That's an interesting idea to me. I know that's something that Adam Silver is interested in as well. He mentioned that because we've got, it seems like a remarkable number of Achilles injuries in the NBA. And if there were a way to gather all this data, you have all this video footage, there's other software that tracks athletes' movements, and then you can run simulations based on that. Maybe it might be possible to have determined many games in advance, oh, you know, Tyrese Halliburton's calf strain in whatever number of simulations we run, this is leading to an Achilles tear. You know, maybe we should look closer at that.

17:37That might be possible. That's an interesting way to think about it. But in general, I would approach it from very, very targeted. We have a problem and then see if there's some sort of technological solution that might help address that problem. Let's go down to like one area that I've seen the possibility for AI is in coaching. I guess you always have a problem. The problem is the other team and you want to develop a game plan that will work against the other team. That seems to be an area that is ripe for this type of technology. I don't know if I would want to see that happen um i i think you start to get to a point where depending on what areas you're dealing with we collectively as a society of whatever art lovers or sports fans have to think about what we want these experiences to look like um i definitely think it's possible for these technologies to kind of help assist game plans.

18:46But that almost takes away some of the fun of it. You know, if you're using this tech to make sure that players don't get injured, anything you could do to make sure players don't get injured, I think is fantastic. These injuries are, they suck. If you were to do that, then you start getting into an area of why do we have coaches? You know, how far do we take this? Can we automate out the coach? Well, couldn't you see like an owner saying, wow, I can save millions of dollars a year on a coach. Let's just get AI to do the game plan and hire some schlep just to oversee it. I could see that argument, but then, yeah, I don't know.

19:24You get into a point of where does this stop? Because then at a certain point, why even have guys play basketball, just run simulations of it and make it realistic looking with mid-journey and tell people to watch it? It kind of ties to that thing I mentioned earlier where perfection is unattainable. And the idea of sports to me is you want to see imperfect, but really skilled athletes doing their best. And I don't want to see perfection enter into that, even if it's possible to get, you know, if Tom Thibodeau could have had the perfect plan to tackle the Pacers, it almost would have felt somewhat cheapened to me if we We found out, oh, yes, he had AI analyze all these things, and this was the exact game plan.

20:13And then would that even translate to the court? Because there's players making player decisions. It becomes a somewhat slippery slope where maybe it's an interesting point to kind of direct your attention to patterns that happen. but if you run with things a little bit too far, I think you start to lose the spirit of, of the thing that, that we all love so much. And my, my over under for you bringing up the Knicks on this podcast was 15 minutes in. And so I, I, you, you smashed the over on that. I, I had, I had the, I had you mentioning the Knicks easily within the first five minutes. So like 15 was a nice cushion for me.

20:56So congratulations on that. We opened by talking about Wimbledon and the line calls. Baseball is coming through. The robot ump, is that AI or that old Hawkeye technology? Do you consider that AI? I'm getting caught in the definitions of what's actual technology and what's artificial intelligence. That's the thing. That's another thing that I find funny and just sometimes straight up very frustrating about this field. It's just so unclear. It's just so unclear what we're talking about when we talk about AI. And to some people, it encompasses a very wide realm of different technologies. And to other people, it's referring to hypothetical technologies that don't exist.

21:42And for that reason, everyone is going to have their own definitions of it. And this makes it very challenging to look at something and say, is this or isn't it? Hawkeye, they've been around for a while. It definitely seems like some of their things use types of technology that I would bundle under the AI umbrella. I think they had a somewhat recent one, like a skeletal tracking system for tennis players, where it's a computer vision model, it seems like. that is looking at all this footage. And instead of tennis players having to wear wearable technology to track whatever metrics, because that's not allowed in a lot of tournaments, it'll have this model process that footage, identify skeletal joints, and be able to process and find that data.

22:39And so that seems like it's probably using some form of machine learning. but Hawkeye doesn't make clear in a lot of their materials what exactly we're dealing with. It's not clear if they're using any type of newer neural network-based transformer technology, the kind of stuff that underpins stuff like ChatGPT. In other instances, it is very clear that that's what's going on. IBM has done a bunch of work. IBM is working with Wimbledon as well, and they've worked with ESPN on fantasy football. And they have their Watson AI system offering in the case of ESPN for fantasy football insights into players and predictions on their likelihood to perform at the metrics that you think they will for the coming week and all these things.

23:33So talk about not being perfect. That's what you're talking about. Come on. It's just all this mess of different, you know, like some places are very clear about what they're doing and where that offering comes from. And in the case of IBM, right, like so you have Watson, I think even in the ESPN app, it'll tell you this was an AI generated insight. And whether or not you follow that is then up to you and your gut. but the Hawkeye stuff is not clear and I'm not sure. I'm not sure if it fits the category. It might be close enough and I don't know. I find myself arguing about semantics a lot of the time.

24:15How much does it matter whether or not this fits that specific definition? I think it matters a lot. I think the words we use to describe these systems are important because they inform the way we think about these systems. So a long way of answering your question is for a lot of these, I don't know. Some of these, it's more clear. A lot of these, it's hard to say. And so that's your answer to the Hawkeye. Is it AI? I don't know. It might be. Yeah. So it seems like certain things they do are kind of based in that. The electronic line calling, I don't know if that is based in something that we would classify as AI as the same as we classify other things.

25:01This might be a full podcast on its own, but the Wimbledon system, the AI line judges, how is that different than the Hawkeye technology? It seems very similar to me, but one might be AI, but one definitely is AI. I don't know. It all comes down to these kinds of definitional differences and then stuff that people are, and I guess some organizations are less open with, which is what is going on under the hood here, which is a question I love asking. And often that's the proprietary stuff that they don't want to tell you. But I would say that by today's standards, if what's going on under the hood of the Wimbledon AI line judges is some form of a neural network, some form of deep learning, some form of machine learning, then it's under that classifier um it would seem like the hawkeye stuff also would have some form of that but again their their ecl has been around long before these kind of more modern takes on deep learning became very popularized um and so that might not fit under that specific umbrella Ian, one more quick break.

26:25And when we come back, I'm going to ask you to take a role to be my therapist.

26:41Andy Jassy of the Amazon famously said, AI is coming and we're going to be cutting our workforce drastically. you had that story making the rounds about the this one technologist working with AI tried to shut down the AI program he was working on and the AI went through its emails and started blackmailing him about an affair that he was having I think Axios was the first place that that I read this, this whole move to AI seems so destabilizing. What I really appreciated about your first column with Puck is that you sort of like, and even listening to you here, you're much more measured in talking about it because it's an unknown and it's scary.

27:38I mean, you saw how the internet came in and killed newspapers and everything else. It's not as scary as some of these stories that have been coming out. In general, my aspiration is to be measured. I think it's easy to forget that AI is a science fundamentally and a kind of core truth of science is scientists need to be skeptical. They need to punch holes in things. They need to see how well things stand up. So that said, what we've seen recently, there's a lot of hype and there are a lot of things that are tied to this hype. I mean, you're talking about an industry that I think, as we probably mentioned in some of the materials for The Hidden Layer, is a trillion dollar industry.

28:33You're talking about massive stock movements of really big companies, hundreds of millions of dollars in venture capital funding, and investors that want to make that money back. And so there's a clear money and power nexus that surrounds modern AI. And I find that fear is a really potent weapon to get people to not pay as close attention to something. and my idea is maybe we should be paying closer attention. The devil is often in the details. In terms of the challenge with a lot of the systems we're dealing with and the reasons to be afraid, and I'll get to the job loss point that you brought up in a minute, but starting with the idea of these systems will go rogue, we have to start by kind of taking our understanding of AI from science fiction movies and shunting that to the side.

29:34That's totally it. And then the other interesting thing that we have to think about is that we're dealing with language models often. And it's really very difficult to evaluate what is actually going on with language models. And in fact, it's very easy to falsely insert intentionality to something because it generates language. So we have this propensity as humans to see something that speaks in a language that speaks quote unquote, and that generates language and ascribe intelligence to it. It's like us, right? If a system like ChatGPT says, I think this, that doesn't mean that ChatGPT is thinking.

30:22The way it was engineered was A, by processing tons and tons of texts and using statistical methods to output that text, and B, was engineered to be personable. And so we humans respond better to something that says, I think. And analyzing actual intentionality around these models is really, really difficult when there is no intentionality, but it's communicating, so to speak, in personal pronouns. So that's just a challenge that has been part of the chatbot side of the field for a long time. And it becomes very easy to over-index on what a capability actually looks like when that capability deals with language.

31:12And it's very easy, even a lot of researchers fall into this trap as well of, oh, the model said this thing. Therefore, the model must have been trying to accomplish this thing and must have been thinking about this thing. And that is not at all the case. And it's a massive simplification of what intelligence is, which is not understood in biological beings. But I mean, our intellect, I'm not talking about our ability to pass a test but our intellect comes from many more sources than just things we read it comes from lived experiences we have emotion and so even though it might output in words and in using personal pronouns that doesn't make it akin to a biological intelligence and so intentionality is a lot different and in terms of these ideas of it going rogue I think it's not as much of a challenge in terms of what we're dealing with today to create engineering guardrails to prevent the kind of mistakes that we see happen.