By Phil Tetlock
What are the usual weapons in policy debates? Here are just a few:
- Sly ad hominem attacks: “Serious observers understand that …. ”
- Convenient counterfactuals: “Things may look bad but you have no idea how bad they would be if… “
- Slippery statistics: “If you make the proper seasonal adjustments and add this variable … “
- Appeals to self-evident truths: “It is now well-established that…”
- And of course the untestable forecast: “If we go down this path, expect things to get worse” – without specifying what “worse” means, on which metric, or when the trend shift will start
There must be a better way to run policy debates. And there is.
Forecasting tournaments create new incentives for debaters. The winners will not be the best put-down artists. They will be those who do the best job of assigning realistic probabilities to testable claims about the future that shed light on whether policy x is working as predicted.
Each side in a debate can nominate a set of questions – let’s cap it at a dozen for now – that it feels can do a better job of answering than the other side. If you think Janet Yellen is a reckless money printer, then ante up with your best probability estimates of interest rates, inflation, and unemployment being above or below a threshold point x by date y. If you think the opposite, take up the critics-of-Janet challenge and propose some questions of your own.
No single question will typically be so probative that it can tip debate decisively. But if you routinely make more accurate predictions than the other side – answering not only your questions better but their questions as well – reasonable people will draw the reasonable conclusion.
This forum is open for nominations of debate topics and questions for pinning down the expectations of each side in the debate. Once we have a critical mass of high-quality questions, we’ll call for volunteer forecasters to play the game in our public tournament website. For more, please click HERE.