by Ben Kantor, Good Judgment Associate
“Adapt or die” – Billy Beane played by Brad Pitt in Moneyball
Good Judgment loves applying better forecasting and decision making to complex problems. So we get excited to hear about innovative ways forecasting is being applied in the real world. Typically, we work with industry leaders in commercial, government, and non-profit organizations. However, many on our team are also keen sports fans, so we closely follow the use of advanced forecasting techniques in sports.
With projections from Forbes that the professional sports industry in the United States will reach $73.5 billion by 2019, sports executives are in a high-pressure pursuit. Franchises are under pressure to find new, innovative methods that will give them an edge to build a championship caliber team.
Industry leaders are embracing quantified forecasting to adapt. In his resignation letter, Sam Hinkie – formerly general manager of the Philadelphia 76ers – quoted Good Judgment co-founder Phil Tetlock as one of his influencers.
Looking at a player with an estimated 10% or 20% chance of being a star over the next three or four years can’t be written to zero—that’s about as high as those odds ever get. That’s surely a very, very high number for any player that is ever available to you to be added to your team. Once you accept that, it becomes clear that shrinking the confidence interval around that estimate (and the estimates of the downside risk at the other end of the spectrum) becomes pretty darn important.
But our well worn thinking patterns often let us down here. Phil Tetlock, from just down the street at Penn, addresses this well in his most recent remarkable book Superforecasting where he quotes the great Amos Tversky saying, “In dealing with probabilities … most people only have three settings:‘gonna happen,’ ‘not gonna happen,’ and ‘maybe’.” Jeff Van Gundy sums it up succinctly on our telecasts, “it’s a make or miss league.” He’s right.
Science is about predictions. Understanding the world until you can make a prediction about what will happen next. If you’re not sure, test it. Measure it. Do it again. See if it repeats.
Hinkie resignation letter from Philadelphia 76ers, April 6, 2016
Daryl Morey is another basketball executive who uses forecasting models when making key decisions on potential future Houston Rockets players. Morey is known for using scientific methods of analyzing players and making personnel decisions.
On Daryl Morey: “Like Bill James, he might use numbers to make better predictions than the experts. If he could predict the future performance of professional athletes, he could build winning sports teams.”
“ ‘How will that nineteen-year-old perform in the NBA?’ was like ‘Where will the price of oil be in ten years?’ A perfect answer didn’t exist, but statistics could get you to some answer that was at least a bit better than simply guessing.” – The Undoing Project, Michael Lewis
Good Judgment Associate Ben Kantor recently caught up with Morey and asked him a few questions about how Morey applies Superforecasting principles to the Rockets. In our conversation, Morey reported that he and his team use forecasting models to capitalize on the mistakes and misjudgments other organizations make when evaluating players. “That’s the job, having a better sense of the value of something more than the market,” he said.
Being the first to break through the wall is never easy. Forecasting in NBA front offices has “penetrated pretty far,” Morey said, “but there’s still a long way to go….” When asked how the Rockets are improving their forecasting methods, Morey described something that sounds a lot like a Superforecaster® post-mortem: “We track our forecasts closely. We track how we do in the past. Each year we do an after-action review where we see how we can do better. Look at the data, talk process.”
Our research has proven that forecasting is a tool that can be learned, and the more practice one has, the better their forecasts tend to turn out. In order to improve his accuracy when making forecasts, Morey and his team “track how well we do closely, compare the type one & two errors. The ones we take and how did we do.”
Morey and his team do not only forecast on players but what their opponents will look like in the future as well. “That’s another thing that we do. We have different analysts who try to anticipate what they (other teams) are doing.” This helps the Rockets plan their strategies to compete with the teams they believe will be at the top of the standings in the coming years.
We asked Morey to forecast a burning question: the likelihood that the NBA will expand by 2022. His response was “60% chance.”
60% probability NBA will expand by 2022.
Forecasting has been key to the success of the Houston Rockets, and holds promise for other franchises. Instead of using traditional scouting methods, NBA leaders such as Daryl Morey and Sam Hickie have asked themselves the questions that others refuse to ask, and gotten positive results from doing so.
We would love to hear how your company is applying sophisticated forecasting techniques to your industry. Get in touch: