Polarization vs Prediction: Writing Questions to Moderate the Debate on Iran

Posted on Posted in Pundit Challenge

By Kathryn Cochran, Good Judgment Inc

Welcome to the official launch of Philip Tetlock and Peter Scoblic’s adversarial collaboration challenge on Iran, “Prediction v. Polarization” now under way at GJOpen. As Phil and Peter set out in their recent piece for the New York Times, forecasting tournaments can elevate polarized political debates by putting accountability front and center: the vague predictions of the opposing sides are replaced with concrete, testable propositions. Before we can make good forecasts,  however, we need good questions.

The act of forecasting itself can moderate the debate: making concrete predictions –  and being held accountable for those predictions –  reduces overconfidence, encourages people to take contrary evidence seriously, and enables people to identify their biases.

Thinking critically about what the important questions are for either side of the debate can also have a moderating effect by moving from abstract concepts  (did Iran negotiate in good faith) to concrete questions (will Iran dilute their stockpile of uranium before June 1, 2016) . It also helps move the focus of the debate away from ideologically-driven assumptions to specific events that can be used to assess the effect of the agreement at a concrete point in time. Writing questions also forces people to make explicit connections between their assumptions and events on the ground. Linking key assumptions, or hypotheses, to concrete events ex-ante makes it much harder to use a different interpretation to bolster your initial beliefs ex-post.

So come participate in question generation as part of the “Prediction vs Polarization” challenge. Take a moment to think about the specific reasons you think the nuclear deal is beneficial or detrimental to US security interests, what key events would support your current assessment of the deal, and what events might prompt you to change your mind.

To help get you started, here are some suggestions for writing solid questions:

  1.  Think about how the question relates to the larger debate: This doesn’t mean that the question has to be on the specific provisions of the nuclear deal, but it does mean that it should be related to larger issue of whether the nuclear deal contributes to the security of the US . So a question about whether Iran will release Jason Rezaian before next October isn’t directly related to the nuclear deal, but it has bearing on whether Iran is becoming more cooperative with the US and that is one of the disagreements dividing supporters and opponents of the deal. Obviously questions directly related to the deal, such as whether Iran provides site access to the IAEA, are also important to include.
  2. The outcome of the question should be obvious: This is an open- source tournament so we must be able to assess whether an event has occurred using open sources. The best questions are ones in which the outcome is not subject to interpretation. Surprisingly, writing questions like that can be challenging, because events often transpire in unexpected ways.  For example, asking about whether Iran will get more involved in Yemen is relevant to the overall debate, but it might not be resolvable because of the paucity of open source reporting on the issue and ambiguity about what would constitute “greater involvement.”
  3. The opposing sides of the debate should have different predictions: Try to ask questions that the two camps will disagree on, because this will generate the most meaningful discussion and also require people to assess their biases when they get predictions wrong. For example, a question about missile tests would be useful because people who believe that Iran will continue to pursue weaponization will likely have differing predictions than those who believe that the nuclear accord demonstrates that Iran has no intention of ever building a nuclear weapon.
  4. Try to ask questions that have a realistic probability of occurring within the timeframe: The challenge runs through the end of 2016 so it would be best to focus on events that are likely to occur in the short- and medium-term, like whether restrictions on Iran’s access to SWIFT will be lifted before next August, or whether Russia will deliver the S-300 missile system to Iran before the end of the year.

So go join the challenge, make predictions on the questions that have already been posed, and suggest what questions we should launch next. We’ll be releasing new questions each month; see how well you can answer your own questions and questions raised by those with opposing views. Let’s see how asking concrete questions – and holding ourselves accountable for our predictions – can shift the debate.

Share on FacebookTweet about this on TwitterShare on LinkedInEmail this to someone

One thought on “Polarization vs Prediction: Writing Questions to Moderate the Debate on Iran

  1. “The best questions are ones in which the outcome is not subject to interpretation.”

    I strongly disagree. I would totally agree with “I would be great if the best questions were the ones in which the outcome is not subject to interpretation”. Alas, everything is subject to interpretation. For example, what constitutes a “best question” is very clearly subject to interpretation. “Subject to interpretation” is similarly ambiguous.

    With this in mind, in the context of forecasting, the best questions are ones that are most interesting and most relevant to persons that desire forecast. Not surprisingly, the best questions are frequently very ambiguous – but that’s just how life is.

    To take a concrete example. “Delivery” of the S-300 missile system can mean any number of things – is this a good reason not to ask the question? No. But there are different reasons similar questions and these reasons influence questions’ interpretation. If one is interested in Iran having fully functional and active S-300 (as it would be when planning an attack), then delivery means full complex delivered, installed and training performed. But if one is interested more in the proof of the commitment to fulfill the delivery contract, then delivery of even a single radar without any missile launchers means will count. So what it mean in the context of asking forecaster right questions? – That question’s intent is critical. Providing intent of the question helps to reduce ambiguity.

Leave a Reply

Your email address will not be published. Required fields are marked *