Forecasting the Tokyo Olympics

In late July 2020, a year ahead of the Tokyo Olympics (postponed in 2020 and scheduled to open 23 July 2021), we asked the Superforecasters whether the Games will begin as planned. By 7 September 2020, the Superforecasters had a clear answer. Back then, they gave the Games a 63% probability of proceeding and have hardly looked back.

The picture was by no means clear if you followed media reports around the Olympics throughout the past year. The IOC and the organizing committee were adamant that options such as a cancellation or delay were off the table. The Japanese public became increasingly opposed to the event. COVID-19 hit the country with a new wave. A range of dissonant headlines, speculations, public opinion polls, and even allegations that Japan had privately decided to cancel the Games (Times) all contributed to the noise surrounding the future of the event.

Good Judgment’s professional Superforecasters are skilled at separating the signal from the noise. They took into account such factors as the associated costs; the likelihood that a vaccine would be developed, tested, and become available by the time of the event (this was months before any COVID-19 vaccine was found to be effective—and safe—in a large clinical trial); and the increasing international experience with measures to contain risk. See how their forecast evolved over time against the backdrop of media reports throughout the year.

A list of media headlines and key events is provided at the bottom of this article, demonstrating both the signal and the noise surrounding the Tokyo Olympics.

Full access to the Superforecasts and commentary is available through subscription via our FutureFirst™ monitor.

Tokyo Olympics: A Sample of Media Headlines Since July 2020

21 July 2020: USA Today: “As COVID-19 pandemic rages on, experts say it’s unlikely Tokyo Olympics can be held next summer”

20 Aug 2020: Japan Times: “Majority of Japanese firms against holding Tokyo Olympics in 2021”

7 Sept 2020: BBC: “Games will go ahead ‘with or without Covid’, says IOC VP”

1 Oct 2020: The Diplomat: “The International Olympic Committee has ruled out postponing the Tokyo Games for a second time”

1 Dec 2020: “Report: Delay of 2020 Tokyo Olympics cost $3 billion”

15 Dec 2020: Japan Times: “Most in Japan oppose holding Olympics in 2021, polls show”

27 Dec 2020: Kyodo: “Pandemic causing uncertainty, unease for Tokyo Olympic ‘host towns’”

7 Jan 2021: BBC: “Tokyo 2020: No guarantee Olympics will go ahead, says IOC’s Pound”

10 Jan 2021: Kyodo: “About 80% favor canceling, postponing Tokyo Olympics in summer: poll”

11 Jan 2021: Japan declares a state of emergency

13 Jan 2021: Guardian: “Tokyo’s Covid outbreak adds to doubts over hosting Olympic Games”

15 Jan 2021: NYT: “Hopes for Tokyo’s Summer Olympics Darken”

CBS: “Tokyo Olympics 2021: Spike in COVID-19 cases has Japanese officials bracing for possible postponement”

19 Jan 2021: AP: “Tokyo Olympics Q&A: 6 months out and murmurs of cancellation”

BBC: “Tokyo Olympics ‘unlikely to go ahead in 2021’”

21 Jan 2021: Times: “The Japanese government has privately concluded that the Tokyo Olympics will have to be cancelled because of the coronavirus”

22 Jan 2021: Reuters: “Japan and IOC deny that Olympics will be cancelled”

11 Feb 2021: Guardian: “Tokyo 2020 Olympics president expected to resign over sexist comments”

15 Feb 2021: CNN: “An earthquake at the Olympic torch relay start point is just the beleaguered Tokyo 2020 Games’ latest crisis”

17 Feb 2021: Seiko Hashimoto becomes new president of the Tokyo Olympic organizing committee

9 March 2021: Kyodo: “Japan to stage Tokyo Olympics without overseas spectators”

25 March 2021: Tokyo Olympic torch relay begins

15 April 2021: Washington Post: “Olympics could be canceled because of virus, Japan ruling party figure admits”

1 May 2021: Washington Post: “Olympic officials are determined to have a Tokyo Games despite Japan’s growing doubts”

12 May 2021: BBC: “Tokyo 2020: United States track and field team cancels pre-Olympic training in Japan”

14 May 2021: NPR: “Opposition to Tokyo Games Grows Heated amid COVID Concerns”

Guardian: “Hospitals overwhelmed as Covid cases surge in Osaka”

18 May 2021: CNBC: “Tokyo medical association calls for cancellation of Tokyo Olympics due to spike in Covid cases”

19 May 2021: CNN: “Dozens of Japanese towns have canceled plans to host foreign athletes from around the world due to concern over Covid-19”

25 May 2021: CNN: “Canceling Tokyo Olympics is ‘essentially off the table,’ says IOC member Dick Pound”

2 June 2021: AP: “Yes. Tokyo Olympics are ‘a go’ despite opposition, pandemic”

13 June 2021: CBS: “Cancel the Tokyo Olympics? It’s unlikely. Here’s why”

Question Clustering: Ensuring relevance and rigor in business and geopolitical forecasting

Forecasting is an essential part of business. Companies use historical data and economic trends to make informed estimates of future sales, profits, and losses. Amazon and Google rely on predictive algorithms to highlight specific products or search results for their customers. Shopkeepers arrange their display windows based on their predictions of demand. Some of the forecasting is narrow and company-specific. The more challenging part has to do with broader questions, such as the overall market outlook.

Consider this question: What will market conditions be like after the pandemic?

This question is of great relevance to any business decision-maker, but it’s so broad that it could be approached from many different angles, using multiple definitions. “Market conditions,” for instance, involves multiple factors, from industry-specific trends to interest rates, from consumer spending to supply chains. How do we intend to measure this? What time frame are we looking at?

Much more tractable is a narrower question—”On 31 January 2022, what will be the average US tariff rate on imports from China?” But in checking all the boxes to craft a rigorous question, narrow topics may lose their relevance to decision-makers trying to determine the prospects for their business down the line. This is the rigor-relevance trade-off.

To tackle the relevant strategic questions that decision-makers ask with the rigor that accurate forecasting requires, Good Judgment crafts sets of questions about discrete events that, in combination, shed light on a broader topic. Good Judgment calls this technique question clusters. Good clusters will examine the strategic question from multiple perspectives—political, economic, financial, security, and informational. These days, a public health perspective is also usually worthwhile. Aggregating the probabilities that the Superforecasters assign to such specific questions generates a comprehensive forecast about the strategic question, as well as early warnings about the deeper geopolitical or business trends underway.

The Good Judgment team developed this method in a research study sponsored and validated by the US Intelligence Advanced Research Projects Activity from 2011-2015, where the Superforecasters outperformed the collective forecasts of intelligence analysts in the US government by 30%. Since then, the Superforecasters have refined and expanded their approach to evaluate emerging consumer trends, inform product development, and understand the factors driving commodity markets. A similar framework was detailed in a recent Foreign Affairs cover article by Professor Philip Tetlock and Dr Peter Scoblic.

As an example, a question cluster on a critical topic that clients have asked Good Judgment to develop includes political, military, social, and economic forecasts on emerging trends regarding Taiwan:

• • Will Taiwan accuse the People’s Republic of China (PRC) of flying a military aircraft over the territory of the main island of Taiwan without permission before 31 December 2021?
  • Will the PRC or Taiwan accuse the others’ military or civilian forces of firing upon its own military or civilian forces before 1 January 2022?
  • Before 1 January 2022, will US legislation explicitly authorizing the president to use the armed forces to defend Taiwan from a military attack from the PRC become law?
  • Will Taiwan (Chinese Taipei) send any athletes to compete in the 2022 Winter Olympics in Beijing?
  • Will the Council of the EU adopt a decision authorizing the Commission to open negotiations with Taiwan on an investment agreement?
  • Will the World Health Organization reinstate Taiwan’s observer status before 1 January 2022?

As another example, in forecasting the technological landscape, the Superforecasters estimate the emerging trends in electric car sales, hydrogen-fueled vehicles, the growth of Starlink services, and social media regulation, among others.

Independently, these forecasts are crucial for some companies and investors and informative for the discerning public. Taken together, they point to trends that are shaping the future of the geopolitical and business world.

The same technique can be applied to Amazon. The traditional approach of security analysts is to build a fixed model that predicts valuation based on dividends, cash flow, and other objective metrics. A question cluster approach can uncover critical subjective variables and quantify them, such as the risk of a regulatory crackdown or shifts in labor relations, to make the analyst’s model more robust.

“In business, good forecasting can be the difference between prosperity and bankruptcy,” says co-founder of Good Judgment, Professor Philip Tetlock. Successful businesses rely on forecasting to make better decisions. Using clusters of interrelated questions is one way to ensure those forecasts are both rigorous and relevant.

* This article originally appeared in Luckbox Magazine and is shared with their permission.

The Cost of Overconfidence

With SPACs all the rage, it’s important not to be too carried away by the rhetoric. Overconfidence can be expensive. This is true in geopolitics, public health, or the stock market. From the 1961 Bay of Pigs debacle to the slow response to the COVID-19 crisis, to millions of dollars lost speculating in the markets, history is filled with costly examples. And yet, bold statements continue to be overvalued in our culture. Time and time again, the media turn to pundits who speak with conviction despite their spotty track records when it comes to offering real foresight.

Think back to a year ago, when the airwaves were filled with experts and politicians confidently asserting that COVID-19 would swiftly pass. The US president claimed in February 2020 that the coronavirus was under control in the US and would disappear “like a miracle.” It took another month for the administration to acknowledge that the unfolding pandemic was serious.

“A confident yes or no is satisfying in a way that maybe never is, a fact that helps to explain why the media so often turn to hedgehogs who are sure they know what is coming no matter how bad their forecasting records may be,” writes Good Judgment’s co-founder Philip E. Tetlock in his book with Dan Gardner, Superforecasting: The Art and Science of Prediction.

Dr. Tetlock refers to a distinction between “foxes” and “hedgehogs,” a metaphor borrowed from ancient Greek poetry and popularized by the philosopher Isaiah Berlin: “The fox knows many things but the hedgehog knows one big thing.”

Hedgehogs tend to be more confident—and more likely to get media attention—but, as research has found in multiple experiments, they also tend to be worse forecasters. Foxes, in contrast, tend to think in terms of “however” and “on the other hand,” switch mental gears, and talk about probabilities rather than certainties.

For instance, last year Good Judgment’s Superforecasters were estimating with a 67% probability that worldwide cases of COVID-19 would exceed 53 million within a year and a 99% probability that deaths in the US alone would be more than 200,000—a figure many found exorbitant at the time. Superforecasters proved right in both cases. Their judgment was, and continues to be, well-calibrated. In other words, they know what they know and know what they don’t know, and they make their forecasts accordingly.

To avoid overconfidence, Superforecasters consider worst-case scenarios. Instead of relying on hunches and past success, they actively seek out evidence that those hunches may be wrong. They embrace new information and are not afraid to change their mind in light of new evidence.

Alas, pundits and most of the media have yet to join the foxes. “We live in a world that rewards those who speak with conviction—even when that is misplaced—and gives very little airtime to those who acknowledge doubt,” writes Financial Times columnist Jemima Kelly.

The sense of security that comes with confident judgments is comforting. But it is an illusion.

The cost of that illusion can be steep: from the inadequate early response to the pandemic to the investors trading to their detriment because they are overconfident about their ability to predict stock market returns.

Superforecasters know a way to avoid that cost: In a world that overvalues hedgehogs, pay more attention to your inner fox.

* This article originally appeared in Luckbox Magazine and is shared with their permission.