Superforecasting® the Fed: A Look Back over the Last 6 Months
The Federal Reserve’s target range for the federal funds rate is the single most important driver in financial markets. Anticipating inflection points in the Fed’s policy has immense value, and Good Judgment’s Superforecasters have been beating the futures markets this year, signaling the Fed would continue to hike until the June pause while markets and experts alike flipflopped on their calls.
Ahead of the Fed’s March meeting, when Silicon Valley Bank went under, the futures markets priced out a hike and began flirting with a possibility of a cut as early as summer. Leading market observers like Goldman Sachs said the Fed would pause, and Nomura said they would start to cut at that meeting.
Then the futures markets priced in a pause for the May meeting. Experts like Pimco’s former chief economist Paul McCulley also prematurely predicted that the Fed would go on hold. As the date of the meeting approached, the futures markets—as well as most market participants—came to share the Superforecasters’ view that another hike was in the cards, but lagged behind the Superforecasters by nearly a month.
In the weeks heading into the June meeting, the futures were oscillating between a pause, a hike, and possibly even a cut. A stream of stronger economic data led experts such as Mohamed El-Erian to forecast that the Fed would continue to raise rates for at least another meeting and perhaps longer. Not the Superforecasters. They have been saying since 2 April 2023 that the Fed would most likely hit pause—a view that, once again, eventually became the consensus.
When comparing the forecasts of two groups—Good Judgment’s Superforecasters and the futures markets using the CME FedWatch Tool—for the last four Federal Reserve meetings, the Superforecasters assigned higher probabilities to the correct outcome. They were 66% more accurate than the futures (as measured by Brier scores) and had lower noise in their forecasts (as measured by standard deviation).
See our new whitepaper for details. We also provide subscribers with a full summary of all our active Fed forecasts, which is updated before and after each meeting (available on request).
Good Judgment’s Superforecasters have been providing a clear signal on the Fed’s policy well before the futures and many market participants. Subscribers to FutureFirst™ have 24/7 access to evolving forecasts by the Superforecasters on questions that matter, including Fed policy through the rest of the year and beyond, along with a rich cross-section of other questions crowd-sourced directly from users, including questions on Ukraine, China, and the upcoming US elections.
“The Most Important Election in 2023”: Superforecasting the Vote in Turkey
The Superforecasters’ forecast on the outcome of the 2023 presidential election in Turkey as published in The Economist.
When The Economist’s “The World Ahead” issue was being prepared for publication in October 2022, Good Judgment’s professional Superforecasters were assigning a 71% probability to President Recep Tayyip Erdogan’s victory in the 2023 Turkish presidential election. Since then, we witnessed Erdogan’s increasingly unorthodox fiscal policy that resulted in high inflation and a devastating earthquake that killed more than 50,000 people. These developments led many in the West to start betting on the candidate from Turkey’s united opposition, Kemal Kilicdaroglu. Superforecasters, however, stayed the course. With a one-day dip to 50-50 just ahead of the first round, they otherwise remained consistently at 60–70% for Erdogan throughout the seven months of this question period, and they are currently at 98%.
For Foreign Policy Expert and Superforecaster Dr. Balkan Devlen, this question was close to home. Born in Izmir, Turkey, Dr. Devlen felt many in the media were getting carried away by the narrative of an underdog win. In this interview, he discusses the key drivers behind his forecast, factors that Western commentators failed to take sufficiently into account, and his assessment of what lies in store for Turkey—and the region—after the election.
GJ: The Superforecasters started at 71% for Erdogan. They are now at 98%. How has your forecast evolved over this period? What were the key drivers?
BD: I was consistently around 80–85% throughout the question period. In that sense, my forecast didn’t evolve much, primarily because the key drivers of my forecast didn’t really change.
Number one is the makeup of the electorate in Turkey, which consistently suggests that about 40% of the voters support the more conservative nationalist coalition led by Erdogan. And that didn’t change much over the period, despite difficulties when it comes to economics or other issues.
Number two is the fact that Erdogan has been in power over 20 years, and during that period managed to install his own cadres across the state and bureaucracy, as well as to gain control over almost all of the media. This created an information environment in which the opposition had a hard time reaching voters who were not already anti-Erdogan.
Lastly, Erdogan does not have the luxury of losing. The extreme polarization within Turkish society and Erdogan’s increasing authoritarianism, especially in the last 10–12 years, means that retiring after the election is just not an option for him, and his family and cronies have also been implicated throughout his rule in corruption, oppression of free speech and freedom of media, etc. Taking those three drivers together, I did not see the wherewithal that the opposition can come together and push out a win.
GJ: Several polls and many commentators in the West were predicting that the opposition would win. Did you find their arguments convincing?
BD: I did not find the polling or the Western commentary particularly convincing, primarily because they tend to base their arguments as if the elections were taking place without the context that I’ve just discussed. I also think that some were engaging in motivated reasoning and wishful thinking. One argument is that the impact of the earthquake in Turkey could have shifted the balance, but that again is a misunderstanding of the fundamental dynamics in the region.
GJ: What would have made you change your forecast?
BD: Perhaps two things. One, if there had been another candidate, either Istanbul Mayor Ekrem Imamoglu or Ankara Mayor Mansur Yavas, the opposition would have had a much better chance.
Two, if I were to see any high-level defections from the AKP or the close circle of Erdogan prior to the election, that would have suggested that, at least, there is a fracture within the ruling elite, that they were considering their post-election fates, and therefore they were breaking rank and jumping ship. We didn’t see any of that, so I didn’t see any reason to change my forecast.
GJ: Superforecasters as a group are now at 98% probability for Erdogan’s victory. Is there a chance they’re wrong?
BD: Of course, there is a chance that the Superforecasters could be wrong, and I am at 98% myself. There are always black swan events, despite the fact that there are only a few days left before the second round. But I don’t see any particular dynamic today that would suggest that the fundamentals prior to the first round were altered in any meaningful way.
Incidentally, I was expecting that this would go to the second round, partly because of the third-party candidate, Sinan Ogan, who now declared his support for Erdogan. That base will probably break 2-1, at least, for Erdogan. But the implication is that if Ogan weren’t in the running, Erdogan probably would have won in the first round.
GJ: What’s in store for Turkey now and for the region?
BD: Well, that’s a big question and I don’t think that’s enough space to go into a detailed examination. But I can tell you that the region and Turkey will probably need to accept the fact that Erdogan will be in power as long as he’s alive. And actors in Europe, in the Middle East, in the Caucasus, and elsewhere would need to adjust to that particular fact.
For Turkey, the results would not lead to a more democratic system. There are those in the parliament now, as part of Erdogan’s coalition, who are hardcore Islamists, and who are calling for much more Islamist policies, for example.
The election results will also lead to recriminations among the opposition coalition, as the only thing that really unites them is their opposition to Erdogan. Therefore, one can expect turmoil within the opposition parties in the post-election era.
Erdogan will probably consolidate his power. That might provide some stability in terms of regional policies now that the need to play for domestic audiences has decreased. Therefore, we are likely to see a more predictable foreign policy attitude from Turkey. I do not necessarily see, though, much change in the direction in which Erdogan wants to take the country, both domestically and internationally.
I don’t see Sweden’s membership being approved by the Turkish parliament before the end of summer as it’s just not a priority, although timing is very hard to predict in these cases.
But like I said, a proper answer to this question requires much more detail and a much longer exposition than can be provided in a short interview. One thing is quite clear, though, and that is Erdogan will be in power for the foreseeable future.
A Strong Year for Good Judgment Inc: 2022 in Review and Outlook
What a year! With Russia’s invasion of Ukraine, protests in Iran, rising tensions in the Taiwan Strait, and inflation pressures around the world, 2022 was the busiest year yet at Good Judgment. Our esteemed clients and FutureFirst™ subscribers in the private sector, government, and non-profit organizations posed a record 181 questions to the professional Superforecasters. More than 80 questions resolved and were scored in 2022. We launched an additional 422 questions on Good Judgment Open, our public forecasting site and primary training ground for future Superforecasters.
Good Judgment’s Superforecasters’ outlook on the 2022 US midterm elections as of 22 October 2021, published in The Economist.
At the start of 2022, our Superforecasters called the 2022 US midterm election, as can be seen in The Economist’s The World Ahead publication, an ongoing collaboration that showcases how “data-driven approaches are becoming popular in all kinds of journalism.” Other appearances of Good Judgment and Superforecasting in the press and news can be found here. The Superforecasters also nailed the forecasts on Jerome Powell’s renomination to head the Federal Reserve and on the Tokyo Olympics. Looking ahead, their forecasts focus on the Russia-Ukraine conflict, tensions around Taiwan, global economy, and key elections in 2023 (many of which are featured in The World Ahead 2023).
Here are some of the other key developments and projects we worked on in the past year:
We launched an updated version of our subscription-based forecast monitor, FutureFirst. In addition to a brand-new interface, the monitor has been equipped with the following features:
Forecast Channels: Questions are grouped by theme or topical focus. Some of the current channels are US Elections, Ukraine, Geopolitics, Economics, Policy, Markets, China, and the Federal Reserve. These are available as part of FutureFirst or as standalone subscriptions.
Implied Medians: In addition to probability ranges, for specific questions FutureFirst now includes a median as part of a continuous probability distribution.
API Access: FutureFirst REST API enables clients’ programs to retrieve daily updated forecast data for use in their models. Separate entry points deliver Question data, Forecast data, and Forecast Distribution data (for applicable questions). Data can be provided in JSON or CSV formats.
Good Judgment’s Superforecasters called Jerome Powell’s renomination to lead the Fed months in advance.
We have further refined our question cluster methodology to illuminate complex topics with a diverse set of discrete questions which themselves can be valuable but taken together provide decision makers with a robust real-time monitoring tool.
We expanded our workshops to include advanced training sessions on question generation and low probability/high impact “gray swan” events. We resumed offering in-person workshops in 2022 even as we continued to offer virtual trainings for teams scattered across the globe—with an average Net Promoter Score of 71 across all sessions, above Apple. Of all the organizations that had a workshop in 2021, more than 90% came back for more in 2022, now regularly sending their interns and new hires through our training.
The US military continues to lead the way: Superforecasting training has been part of the curriculum for senior officers since 2019, and Good Judgment has been delivering semester-long courses at National Defense University in Washington, DC, continuously since the start of 2020. Over the same period, civil servants and military officers from Dubai have been forecasting during and after our annual two week-training courses.
We launched the Forecasting Aptitude Survey Tool, a screening tool that measures characteristics that Good Judgment’s research found correlate with subsequent forecasting accuracy. It’s been an integral part of our workshops for years, and we’re now pleased to provide it to organizations looking for an additional input to evaluate their existing or prospective staff.
To help advance decision-making skills among high school students, Good Judgment partnered with the Alliance for Decision Education to launch a pilot forecasting challenge for selected schools across the United States. We are continuing and expanding this project in 2023. At the other end of the education spectrum, we collaborated with the Kennedy School of Government at Harvard University in a semester-long course that honed the forecasting skills of top graduate students.
The Superforecasters’ forecast on a peace agreement between Russia and Ukraine.
With climate change becoming a growing concern worldwide, we partnered with adelphi, a leading European think tank for climate, environment, and development, to produce “Seven questions for the G7: Superforecasting climate-fragility risks for the coming decade,” published in 2022. Commissioned by the multilaterally backed climate security initiative Weathering Risk, it is the first report of its kind that applies the Superforecasting methodology to climate-related risks to peace and stability. We also partnered with Dr. William MacAskill, the author of the groundbreaking 2022 book What We Owe the Future, supplying 22 additional forecasts on the long-term impacts of climate change.
Finally, at the end of 2022, Good Judgment’s co-founder Philip Tetlock launched the non-profit Forecasting Research Institute, an exciting additional venture that will advance the frontiers of forecasting for better decisions. We look forward to contributing to their efforts in 2023—and beyond.
It’s already shaping up to be another exciting year of uncertainty in 2023, with gridlock in Washington, rising geopolitical tensions, and fierce global economic headwinds. We look forward to those challenges and contributing our forecasts and insights with the help of a phenomenal staff at Good Judgment Inc, the unrivalled skills of our professional Superforecasters, and the support of our expanding roster of clients and partners around the world.
Schedule a consultation to learn how our FutureFirst monitoring tool, custom Superforecasts, and training services can help your organization make better decisions.
Keep up with Superforecasting news. Discover training opportunities. Make better decisions. Sign up for Good Judgment’s newsletter.