Superforecaster Tips: Dealing with Confirmation Bias in Election Forecasting

Written by

Superforecaster Tips: Dealing with Confirmation Bias in Election Forecasting

As the 2024 US election approaches, forecasters are faced with the daunting task of finding signal amid a cacophony of partisan noise, personal biases, and volatile public opinion. One significant challenge is confirmation bias—the tendency to search for, interpret, and recall information in a way that confirms one’s preconceptions. In this blog post, we draw on an internal discussion among seasoned Superforecasters to explore practical strategies forecasters can use to mitigate confirmation bias in election forecasting.

Diversifying Information Sources

“Assign yourself to spend some time reading (reasonably reputable) news sources that disagree with your general perspective on the question.”

Superforecasters highlight the importance of consuming a balanced diet of news sources, including those that challenge one’s beliefs. This approach was systematized by Good Judgment Project (GJP) superforecaster Doug Lorch, who wrote a program to randomize his news intake among a diverse set of sources.

“It certainly didn’t hurt,” recalls Terry Murray, CEO Emeritus of Good Judgment Inc and Project Manager for the GJP at UC–Berkeley. “He was the top forecaster in the whole IARPA tournament that year.”

Engaging in Scenario Analysis and Premortems

“I try to run through various scenarios where [the expected winner] could end up losing.”

Superforecasters routinely consider alternative outcomes by rigorously testing their own assumptions and logic. This involves running through various scenarios where expected outcomes might not materialize and thinking critically about the conditions that would lead to different results.

Embracing Epistemic Humility

“One thing I know is that I don’t know much.”

Acknowledging the limits of one’s knowledge and being open to new information is another tip the Superforecasters offer. This strategy is crucial for preventing overconfidence and being receptive to counterarguments.

Red Teaming

“One of the most important duties for me, as a Red Team member, is not to convince a forecaster that they are wrong… Rather, it’s to test the confidence of the Superforecaster in their own forecast.”

Having a red team to challenge forecasts helps forecasters to re-evaluate the confidence in their arguments and consider why they might be wrong. Red teaming is a standard practice in all Good Judgment’s forecasting.

Leveraging Collective Wisdom

“Sometimes, it pays to listen to the articulated reason of an outlier.”

Some Superforecasters use the median forecast of their group as a benchmark, particularly when their individual estimates deviate significantly from the consensus. This approach can provide a reality check against one’s own extremes. It is important, however, to pay attention to outlier opinions too, to resist conformity and groupthink.

As we dive into another election cycle, the discipline of forecasting reminds us that remaining actively open-minded is more crucial than ever. Combating confirmation bias in election forecasting is no small feat, given the complexity and the emotionally charged nature of politics. However, by employing strategies such as diversifying information sources, engaging in premortems, practicing epistemic humility, employing red teaming, and referencing the collective wisdom of peers, forecasters can enhance the accuracy and reliability of their predictions. Good Judgment’s exclusive forecast monitoring tool FutureFirst™ offers daily forecast updates on election results and trends and many other topics, brought to you by professional Superforecasters.

Learn More about FutureFirst™!

Decoding SCOTUS: Navigating Media Bias in Supreme Court Forecasting

Written by

Decoding SCOTUS: Navigating Media Bias in Supreme Court Forecasting

Superforecaster, GJ managing director, and leader of Good Judgment’s question team, Ryan Adler shares tips on how to approach forecasting Supreme Court decisions.

May is a lovely time of year. Lots of daylight, newborn critters all around, and Supreme Court junkies start to get their fix from now through the end of June. As someone who has watched the Court enthusiastically for more than a quarter of a century, I always look forward to decision season. However, in all those years, I’ve learned a painful lesson. The press is generally terrible at reporting about the Court. For Court junkies like me, it’s not that big of a deal. I learned early in my studies that if I want to know what a case means, I need to dig into the weeds myself. But as someone who has also written dozens of SCOTUS forecasting questions, I know that many forecasters don’t have the benefit of having read thousands of pages of appellate decisions to instruct their analysis. So, as is the case for anybody forecasting on a topic with which they aren’t intimately familiar, they lean on press reporting to fill in the gaps.

“Creating an emotional investment in the outcome of a case is poison for a forecaster.”

Alas, most journalists and talking heads intent to tell readers and viewers “how it is” end up just regurgitating heavily politicized schtick from one side of the aisle or the other. A glaring example is Trump v. United States, the presidential immunity case out of the DC Circuit Court of Appeals. For the vast majority of the press, it’s a question of what the Court will let happen to Trump in Jack Smith’s federal election interference case. For anyone who understands the Constitution and listened to oral arguments in the case, the Court knows it’s grappling with something much bigger and fundamental than Trump’s actions and statements on January 6th. Questions of presidential immunity go to the heart of the structure of the US’ separation of powers, perhaps the greatest innovation from the Founding Fathers that has proven key to good governance in modernity. Framing the case as democracy in peril (you can find such arguments from both Trump’s supporters and detractors) is, in this writer’s opinion, theater. This isn’t to say that the events of January 6th are nothing of concern, nor does it make light of the potential precedents to be set. But let’s be honest: if the party affiliation of the defendant in this case were flipped, who honestly thinks that the arguments from the “legal experts” in the media wouldn’t look far different? That doesn’t make these people liars, but it highlights that these “experts” are doing something very different than what we ask forecasters to do—figure out what will happen irrespective of what they feel should happen. Such “experts” placate and/or infuriate, which is how politics works.

What is to be done?

Creating an emotional investment in the outcome of a case is poison for a forecaster. We can debate all day about the political implications of a Court decision, and the nine robed ones don’t operate in a vacuum. That all notwithstanding, there is a stark difference between being mindful of political factors that influence the conversation among them and acting as though those implications represent the sole axis upon which the case will turn. It may seem hard not to see anything and everything through a political lens, and most media outlets have given up pretending that their reporting isn’t first run through a political prism.

So, what is to be done? As James Madison stated that ambition must be made to counteract ambition, this writer suggests that cynicism must be made to counteract cynicism. Take the words of any legal “expert” or talking head with a large grain of salt. Being a lawyer doesn’t make you an expert on the Constitution. Many lawyers who practice would probably admit that the last time they paid attention to constitutional law was prepping for a bar exam. That fact is not mitigated by gaining the title of “contributor.” If what you read or hear feels like it’s what you may personally want to read or hear, be suspicious. That’s sound advice for anything in the media, but it’s especially crucial for navigating the flood of gibberish that inundates the airwaves and web when a high-profile case makes it to Washington.

Do you have what it takes to be a Superforecaster? Find out on GJ Open!

 

The Power of Precision: Why Top Investors Choose Our Superforecasters

Written by

The Power of Precision: Why Top Investors Choose Our Superforecasters

With increased geopolitical tensions influencing market dynamics, investors today face significant challenges in navigating the turbulent financial landscape. Despite extensive analysis by top hedge funds like Bridgewater, which has 400 specialists dedicated to predicting market impacts of geopolitical and other event risks, even they acknowledge a 40% error rate in their forecasts. At Good Judgment, our Superforecasters are not only equipped to navigate today’s complex global environment, but also to provide early signal, clarity, and actionable insights that can significantly enhance investment strategies.

Unparalleled Accuracy

Our Superforecasters are elite forecasters whose unparalleled track record in forecasting high-impact geopolitical events and economic shifts has been repeatedly proven for more than a decade. Their success is rooted in a methodology that consistently outperforms conventional market predictions. While even the top hedge funds face a significant error rate in their forecasts, our team has demonstrated superior accuracy, giving our clients a competitive edge.

Superforecasters were more accurate and less noisy than the futures in forecasting the Fed’s next move.
Data-Driven Decision Making

At the core of our forecasting excellence lies the Superforecasting process, a robust analytical framework that integrates daily updates to numerical probabilities with qualitative insights, red teaming, and openness to new information. This approach allows Good Judgment’s Superforecasters to discern patterns and implications that others might miss, turning complex data into clear, understandable forecasts. Our clients benefit from this data-driven decision making, which equips them with the knowledge to make more informed investment choices.

Customized Insights

Every investor’s needs and goals are unique. In addition to a generous offering of forecasts on crucial issues of the day, delivered via our forecasting monitoring tool FutureFirst™, our Superforecasters can provide tailored insights on questions that matter to your organization. Whether you’re managing a large portfolio or focusing on niche markets, Good Judgment provides you with the most relevant and strategic information, ensuring that every decision is supported by the highest quality of forecasting.

Early Signal

Geopolitical events can impact markets suddenly and severely. Our Superforecasters not only identify and quantify these risks but also provide an early signal of potential shifts. They can help enhance your approach to risk management to protect investments from sudden downturns and ensure long-term stability and growth.

Investors who choose to leverage the expertise of our Superforecasters are not just staying ahead of the curve in an increasingly uncertain world—they do so with confidence. To see how our Superforecasters can transform your investment approach, contact us today.