Transcript
Emma Atkinson (00:06):
You're listening to RadioEd, the University of Denver podcast. I’m your host, Emma Atkinson.
Crowd roaring, squeaking shoes on court ambient noise
Emma Atkinson (00:18):
It’s NBA playoffs season.
Eight seconds are left in the second half of a game between the Indiana Pacers and the Boston Celtics.
The Pacers are down a point, and Pacers head coach Rick Carlisle has a decision to make: Should he call a timeout?
He decides against it, and the Celtics go on to score another two points, winning the game 114 to 112 and eventually taking the series.
Carlisle defended his decision to reporters afterwards, saying that he had more faith that his players would score on the very next play than if he had called the timeout and reset the offense. Basically, he wanted the final seconds of the game to play out without intervention.
Carlisle gambled—and lost.
Buzzer sound
Much of sports is a gamble. There’s a saying: “Any team can win any game on any given day.” Almost nothing, no outcome, is guaranteed in sports—and that’s the fun of watching and playing.
But players and coaches want to eliminate as many variables as possible, trying to leave less up to chance. And that is where statistics come in.
It might seem like a good idea to call a timeout in the NBA when the opposing team is on a scoring run—it could slow their momentum, change the energy of the game, right? That’s what critics of Carlisle’s decision might say.
But research from a University of Denver data analytics professor indicates otherwise.
Ryan Elmore (01:38):
NBA players are the best three or four hundred players in the world of what they do. They're all really good. They know, if another team is on a run, or a team is on a run; they generally know how to come back. They hadn't forgotten how to play basketball, so my guess is that if you just let the game proceed as it was going to proceed, they probably would have changed things up and maybe made a run themselves.
So that's not to say that the timeout is bad. The coach probably wants to impart some knowledge on the players as well, which is the coach's prerogative. But I just don't – I mean, I think a lot of people just think that the coach is going to fundamentally change what's going on in the game, and I don't think that the data bears that out as well. That just doesn't happen. The natural progression of the game would have... the other team would have caught up, or the other team wouldn't have kept going on a larger run, let's say.
Emma Atkinson (02:42):
That’s Ryan Elmore, the University of Denver professor who studies sports analytics. He’s looked at everything from timeouts in the NBA to the varied temperature of pro golfers’ hands. And his work on timeouts shows that, statistically, calling a timeout in those high-stakes situations is disadvantageous. So the Pacers coach, Carlisle, might have actually been onto something—even though the cards didn’t fall the way he would have liked.
You might think that studying sports stats involves attending a ton of games and meeting famous athletes—Elmore says it’s not exactly as cool as that. Ideally, though, in order to be statistically sound, it would.
Ryan Elmore (03:18):
If you really wanted to answer this question: Does a coach calling a timeout affect what's going to happen in the next two minutes? Ideally, you would have somebody like me, here, sitting behind the bench, and if you saw the other team going on a run, you're going to flip a coin and say, “Alright, heads: I'm going to tell the coach to call a timeout. Tails: we're not going to call a timeout.” So we're going to conduct a randomized experiment to do this. Obviously, that's not going to happen in the NBA, high stakes NBA.
Emma Atkinson (03:48):
It wouldn’t really work to have statisticians running the NBA. So, here’s what Elmore did instead.
Ryan Elmore (03:54)
What we did instead is employ what we refer to as causal methodology, which is: we can't do the randomized experiment. So this causal inference work was developed in mainly the epidemiological world, and people study this in epidemiology because like you have all this stuff that happens, and then you try to recreate what happened going into that. And then hopefully, you can see differences in some treatment of interest. So our treatment in the NBA game was coaches calling timeouts. Now, how do we kind of replicate or create a pseudo randomized experiment? It's not a randomized experiment, but what we do is we try to take instances in games that kind of match when a timeout was called. So we look at the score the game, how much time was left in the game? What a team was... what was the magnitude of a run leading up to that timeout, what was the relative differences in team quality going into that game? So we use some gambling metrics to kind of show that. But we have what we refer to as potential matches for that timeout where a timeout wasn't called. And then we tried to find the closest instance where a timeout wasn't called and then measure what happens after both the timeout when the timeout was called. And then when a timeout wasn't called, and then you kind of create your randomized experiment there and measure the outcomes.
Musical interlude
Emma Atkinson (05:36):
Elmore’s interest in sports analytics began very organically—he started out as a statistician who just happened to love sports.
Ryan Elmore (05:43):
I mean, I've been a sports fan all my life. So I just started looking into what people are doing in the sports analytics world. And I realized, a lot of the problems that they were solving were problems that I had considered at some point in my life, I just never dove into it from a statistical point of view. So then I started thinking about, what are some of the things that I would like to study in sports? And I thought, ‘Alright, well, I have a lot of knowledge on statistics, these are cool questions.’ And I just wanted to bring the two together. And that's sort of how I got started just solving problems.
Emma Atkinson (06:19):
He was doing sports analytics on the side, as a hobby, using a Python program to scrape sports data from various websites and writing about his findings.
Ryan Elmore (06:27):
So I downloaded all this data. And then I just wrote a blog post. I put it out on Twitter or something, and a few people were like, oh, that's super cool... can you... have you looked at this and this and this, and they kind of led me to a series of problems where I was like, I hadn't thought about that. But I will. So I would just take another question and kind of do the same thing.
I generally didn't look at questions that were...well, basically, that weren't interesting to me. So if somebody gave me a question, I'm like, oh, that's kind of boring. I don't want to deal with that. So I would just take the questions that were interesting, and occasionally blog about the results. And so then I just kept doing problems like that. And eventually, I was like, I think there are some deeper results that can be published here.
Emma Atkinson (07:13):
His idea to study timeouts in the NBA was crowdsourced, too.
Elmore and his friends get together every couple of years to ski and catch up, and they usually chat about sports, too.
Ryan Elmore (07:25):
Somebody was like, ‘You know, a lot of the work that you've done is either studying a particular player type analysis, or most of the work that's done in sports is looking at players. Have you ever done anything related to coaches?’ So we started throwing some ideas out there like,’How could you evaluate coaches?’ And one of my friends, he has a PhD in statistics as well. He was like, ‘I mean, has anybody looked at timeouts in a particular sport?’ And so we just... through that weekend, we started talking like, ‘Well, how could you evaluate timeouts?’ And then of course, we have more drinks and probably forgot about that conversation.
Emma Atkinson (08:03):
The conversation wasn’t entirely lost to the guys’ trip, though. One of Elmore’s former undergraduate students reached out to him and asked if he’d be willing to work on something together, and he remembered that après ski chat.
Ryan Elmore (08:15):
And so I was like, ‘You know, about a half year ago or so some friends of mine and I were talking about timeouts in the NBA.’ So she was like, ‘Yeah, that seems like a cool problem.’ She had a great grad student who was wanting to work in sports as well. So he came on board and we started thinking about that particular problem at a very, I would say, a deeper level than just a barroom conversation. But it led to a really good paper, I would say.
Musical interlude
Emma Atkinson (08:47):
If you had a 30 minute meeting with Adam Silver, the Commissioner of the NBA, what would you tell him about your research? What would you want him to take away from that meeting?
Ryan Elmore (08:46):
Gosh, that's a great question. So I would say that I would want him to know that there are a lot of fans and a lot of academic researchers who are doing really interesting work with the data that we have available, namely, my paper that we were just discussing, but there's a host of other people who are doing great work in both the NBA and other sports. But if I'm talking to Adam Silver specifically, I would just say that one: please don't ever reduce the amount of data that you're making public. And two: I know you have other data that are available, please make that public as well, because there's all sorts of player tracking information that people like myself would love to analyze. And it may lead nowhere, but it potentially would lead to additional insights into the game. So if you could – this is more than 30 seconds—but if you could make that data public, I think you would probably advance the game in some potential way.
Emma Atkinson (10:02):
Wow, that's, that's big.
Ryan Elmore (10:04):
I think so. But, I mean potentially it wouldn't, but I think I think people like myself, and other, not just quantitative fans, but people who are just interested in the NBA, they generally get some...I think they would gain some value from the results that would come out of people like me studying this.
Emma Atkinson (10:24):
That's a really cool thing to think about.
Ryan Elmore (10:26):
Yeah, I mean if Adam Silver is listening, then you know, please reach out.
Emma Atkinson (10:31):
Elmore says he’s always looking for more sports questions that can be answered with data.
If I'm listening to this episode, and I think oh, my gosh, I have the craziest question that I think could be answered by a statistician. Would you want them to reach out to you?
Ryan Elmore (10:44):
Sure. Yeah, definitely.
Emma Atkinson (10:46):
Okay.
Ryan Elmore (10:47):
Yeah. 100%. I mean, I would say, don't be offended if I don't respond right away, because there's a lot of stuff going on. But if you reach out and want an answer or just talk sports statistics, I'm always willing to do it.
Emma Atkinson (11:04):
How do you feel like sports, games, athletes, coaches have changed now that stats are more readily available?
Ryan Elmore (11:13):
Well, in the NBA, for example – I mean, this is kind of a classic example – the game has changed fundamentally in the last ten to say twelve, fifteen years, and that teams really understand the importance of taking a three-pointer relative to a two-point shot right next to the three-point line. So you see that if you looked on a map of where all the shots are taken on a game, you'll see kind of like, in the early 2000s or early 2010s or 2010 around there, you would see kind of a more evenly distributed shot map of where they're taking shots. Now, it's like you're taking shots close to the basket, or they go beyond the three-point line. Okay, so that's a clear...the expected value of a shot that's at he three-pointer is much higher than taking it right inside the three-point line. So players know that, coaches know that, and if you're taking a lot of those two-pointers that aren't worth as much, but are almost as difficult as the three-pointer, the coaches tend to not love that. So players react to that.
Emma Atkinson (12:24):
Kind of in that same vein, what do you think fans should know about sports analytics?
Ryan Elmore (12:30):
Great question. I think what fans should know is – I don’t want to necessarily say what I'm doing is not important, because I think it's important, both from an educational standpoint, because sports analytics is a great way to draw people into statistics in general – but I would say that there's still room for debate amongst a lot of stuff. Like if I were to present this research on timeouts in the NBA and I have my conclusion, well, there's still room for debate. People can still have fun debates. My results aren't the end all be all of what's going on in the NBA, this is just how I analyze the problem. And this is what I believe. But I don't want it to take the fun out of the game, and the fun out of being a fan. And part of that fun is debating these things.
Emma Atkinson (13:27):
Elmore has a perfect example of why he doesn’t want to kill the joy of sports debates with stats.
Ryan Elmore (13:34):
Last year during the the NBA playoffs, I guess the Boston Celtics manager is kind of known for not calling timeouts. And so he didn't call a timeout, the Celtics ended up losing the series. And so I wasn't following this, but on their Reddit, the Boston Celtics subreddit, they were debating like, ‘Should the coach have called a timeout? No, blah, blah, blah.’ But he's up in arms about this whole situation of them getting kicked out of the playoffs and the coach not calling timeouts as the fans thought they should. And so a bunch of people were commenting. Somebody found our paper and was like, ‘Here's what the research suggests.’ And so then they started commenting about this paper. And it was kind of funny to read some of the comments. And then the last comment was just, f this nerd s**t. And then that killed the debate. And I don't want that to kill the debate. You know, I want people to use that and debate that as well as the basketball. But generally, I get that people aren't as interested in debating the statistics as they are just like the raw outcomes. But I think my point is, there's room for just being a fan and there's room for taking a more analytical perspective on what's going on in the game. They don't have to be opposed to one another.
Crowd roaring and whistle blowing ambient noise
Emma Atkinson (15:03):
A big thanks to our guest, University of Denver associate professor of business information and analytics Ryan Elmore. More information on his work can be found in our show notes. If you enjoyed this episode, I encourage you to subscribe to the podcast on Apple Music or Spotify—and if you really liked it, leave us a review and rate our work. It really helps us reach a larger audience—and grow the pod.
Joy Hamilton is our managing editor. Madeleine Lebovic is our production assistant and musical genius and James Swearingen arranged our theme. I'm Emma Atkinson, and this is RadioEd.
