Good Judgment’s team of Superforecasters received full marks from The Economist for their forecasts published last year in “The World Ahead 2023” issue. Now that eight of the nine questions have resolved, The Economist’s editors were able to score the Superforecasters’ performance.
“The Good Judgment team had a good year in 2023, correctly forecasting the outcomes of the eight questions that were resolved,” the editorial team writes in the “The World Ahead 2024” print issue. “Global growth was 3%, China grew by 5%, ruling-party candidates won in Nigeria and Turkey, Vladimir Putin was not ousted, there was no election in Britain, no clash over Taiwan, and no nuclear device detonated by Russia.”
As to the ninth question in the 2023 publication, the Superforecasters continue to see a protracted conflict in Ukraine, likely going beyond 1 October 2024. That question remains open, and, as The Economist team notes, “Events in 2023 did not prove them wrong.”
“The World Ahead 2024” from The Economist is now available online and in print, and once again features the Superforecasters’ take on key questions for 2024. See their forecasts in the newspaper—or subscribe to FutureFirst™ to access all their forecasts that are updated daily.
About Good Judgment
Good Judgment Inc is the successor to the Good Judgment Project, a research team that emerged as the undisputed victor in a massive forecasting competition (the Aggregate Contingent Estimation or ACE tournament) sponsored by the Intelligence Advanced Research Project Activity (IARPA) of the US Office of the Director of National Intelligence (ODNI). Spanning four years, 500 questions, and over a million forecasts, that research project confirmed and refined methods that lead to the best possible forecast accuracy and is credited with the discovery of Superforecasters—people who are exceptionally skilled at assigning accurate probabilities to future outcomes. Good Judgment Inc is now making this winning approach to harnessing the wisdom of the crowd available for commercial use. Good Judgment’s Superforecasters are men and women around the world who go through a rigorous qualification process to demonstrate consistently high accuracy and quality commentary in their forecasting approaches.
About FutureFirst™
FutureFirst, Good Judgment’s exclusive monitoring tool, gives 24/7 access to timely insights on top-of-mind questions from a diverse global team of professional Superforecasters. It combines the advantages of an expert network with model-friendly quantitative forecasts of likely outcomes of key events. Daily forecast updates from our subscription service allow clients to spot emerging risks earlier and see ahead of the competition.
A Strong Year for Good Judgment Inc: 2022 in Review and Outlook
What a year! With Russia’s invasion of Ukraine, protests in Iran, rising tensions in the Taiwan Strait, and inflation pressures around the world, 2022 was the busiest year yet at Good Judgment. Our esteemed clients and FutureFirst™ subscribers in the private sector, government, and non-profit organizations posed a record 181 questions to the professional Superforecasters. More than 80 questions resolved and were scored in 2022. We launched an additional 422 questions on Good Judgment Open, our public forecasting site and primary training ground for future Superforecasters.
Good Judgment’s Superforecasters’ outlook on the 2022 US midterm elections as of 22 October 2021, published in The Economist.
At the start of 2022, our Superforecasters called the 2022 US midterm election, as can be seen in The Economist’s The World Ahead publication, an ongoing collaboration that showcases how “data-driven approaches are becoming popular in all kinds of journalism.” Other appearances of Good Judgment and Superforecasting in the press and news can be found here. The Superforecasters also nailed the forecasts on Jerome Powell’s renomination to head the Federal Reserve and on the Tokyo Olympics. Looking ahead, their forecasts focus on the Russia-Ukraine conflict, tensions around Taiwan, global economy, and key elections in 2023 (many of which are featured in The World Ahead 2023).
Here are some of the other key developments and projects we worked on in the past year:
We launched an updated version of our subscription-based forecast monitor, FutureFirst. In addition to a brand-new interface, the monitor has been equipped with the following features:
Forecast Channels: Questions are grouped by theme or topical focus. Some of the current channels are US Elections, Ukraine, Geopolitics, Economics, Policy, Markets, China, and the Federal Reserve. These are available as part of FutureFirst or as standalone subscriptions.
Implied Medians: In addition to probability ranges, for specific questions FutureFirst now includes a median as part of a continuous probability distribution.
API Access: FutureFirst REST API enables clients’ programs to retrieve daily updated forecast data for use in their models. Separate entry points deliver Question data, Forecast data, and Forecast Distribution data (for applicable questions). Data can be provided in JSON or CSV formats.
Good Judgment’s Superforecasters called Jerome Powell’s renomination to lead the Fed months in advance.
We have further refined our question cluster methodology to illuminate complex topics with a diverse set of discrete questions which themselves can be valuable but taken together provide decision makers with a robust real-time monitoring tool.
We expanded our workshops to include advanced training sessions on question generation and low probability/high impact “gray swan” events. We resumed offering in-person workshops in 2022 even as we continued to offer virtual trainings for teams scattered across the globe—with an average Net Promoter Score of 71 across all sessions, above Apple. Of all the organizations that had a workshop in 2021, more than 90% came back for more in 2022, now regularly sending their interns and new hires through our training.
The US military continues to lead the way: Superforecasting training has been part of the curriculum for senior officers since 2019, and Good Judgment has been delivering semester-long courses at National Defense University in Washington, DC, continuously since the start of 2020. Over the same period, civil servants and military officers from Dubai have been forecasting during and after our annual two week-training courses.
We launched the Forecasting Aptitude Survey Tool, a screening tool that measures characteristics that Good Judgment’s research found correlate with subsequent forecasting accuracy. It’s been an integral part of our workshops for years, and we’re now pleased to provide it to organizations looking for an additional input to evaluate their existing or prospective staff.
To help advance decision-making skills among high school students, Good Judgment partnered with the Alliance for Decision Education to launch a pilot forecasting challenge for selected schools across the United States. We are continuing and expanding this project in 2023. At the other end of the education spectrum, we collaborated with the Kennedy School of Government at Harvard University in a semester-long course that honed the forecasting skills of top graduate students.
The Superforecasters’ forecast on a peace agreement between Russia and Ukraine.
With climate change becoming a growing concern worldwide, we partnered with adelphi, a leading European think tank for climate, environment, and development, to produce “Seven questions for the G7: Superforecasting climate-fragility risks for the coming decade,” published in 2022. Commissioned by the multilaterally backed climate security initiative Weathering Risk, it is the first report of its kind that applies the Superforecasting methodology to climate-related risks to peace and stability. We also partnered with Dr. William MacAskill, the author of the groundbreaking 2022 book What We Owe the Future, supplying 22 additional forecasts on the long-term impacts of climate change.
Finally, at the end of 2022, Good Judgment’s co-founder Philip Tetlock launched the non-profit Forecasting Research Institute, an exciting additional venture that will advance the frontiers of forecasting for better decisions. We look forward to contributing to their efforts in 2023—and beyond.
It’s already shaping up to be another exciting year of uncertainty in 2023, with gridlock in Washington, rising geopolitical tensions, and fierce global economic headwinds. We look forward to those challenges and contributing our forecasts and insights with the help of a phenomenal staff at Good Judgment Inc, the unrivalled skills of our professional Superforecasters, and the support of our expanding roster of clients and partners around the world.
Good Judgment Inc and Metaculus Launch First Collaboration
Metaculus and Good Judgment Inc are pleased to announce our first collaboration. Our organizations, which represent two of the largest human judgment forecasting communities in the world, will compare our results and methodologies in a project comprised of identical forecasting questions that ask about the future of 10 Our World In Data metrics. We plan to share insights, lessons learned, and analysis to contribute to the broader community and to the science of forecasting.
Cohorts of Superforecasters fromGood Judgment Inc and Pro Forecasters from Metaculus will make predictions on their separate platforms on a set of 10 questions about technological advances, global development, and social progress on time horizons ranging from one to 100 years.
A Future Fund grant is supporting both organizations in producing these expert forecasts, as well as apublic tournament on the Metaculus platform, though this collaboration between Metaculus and GJI is distinct, separate, and voluntary.
“Our shared goal is advancing forecasting as a trusted method for leaders to make critical decisions,” said Gaia Dempsey, CEO of Metaculus. “We’re thrilled to be working with our partners at Good Judgment Inc. This is the first time two of the largest players in the field of forecasting have come together in the spirit of collaboration to compare methodologies and to advance the science of forecasting.”
“We’re excited to be partnering with Metaculus to combine our approaches to apply probabilistic thinking to an uncertain future and help individuals and organizations make better decisions about the future,” said Warren Hatch, Good Judgment’s CEO. “We look forward to building on this collaboration for Our World In Data.”
Good Judgment Inc harnesses the wisdom of the crowd, led by Superforecasters, to quantify hard-to-measure risks for smarter strategic decisions for the private and public sectors.
Metaculus is a forecasting technology platform that optimally aggregates quantitative predictions of future events.
Schedule a consultation to learn how our FutureFirst monitoring tool, custom Superforecasts, and training services can help your organization make better decisions.
Keep up with Superforecasting news. Discover training opportunities. Make better decisions. Sign up for Good Judgment’s newsletter.
Good Judgment’s team of Superforecasters received full marks from The Economist for their forecasts published last year in “The World Ahead 2023” issue. Now that eight of the nine questions have resolved, The Economist’s editors were able to score the Superforecasters’ performance.
