Good Judgment research discovered four keys to accurate forecasting: talent-spotting, training, teaming, and aggregation.
Our scientifically validated process uses these four keys to unlock the secret to better forecasts – and better decisions.
Our evidence-based techniques reliably identify the most accurate forecasters.
Combining our training methods with deliberate practice significantly increased accuracy in Good Judgment research.
Forecasters working in teams outperform those working alone or in prediction markets.
Our state-of-the-art aggregation methods extract the maximum possible signal from the noise of crowd-sourced forecasts.
This Knowledge@Wharton podcast features Good Judgment co-founder (and Wharton professor) Barb Mellers and INSEAD professor Ville Satopää.
Prof. Satopää summarized the new research findings:
What we found out, as a simple rule of thumb, is that about 50% of the accuracy improvements that we saw going from the regulars to the [Superforecasters] can be attributed to noise reduction. The remaining 25% is information improvement, meaning that they have more information, and that last 25% will be then bias reduction.
Superforecasters are less noisy — they don’t show the variability that the rest of us show. They’re very smart; but also, very importantly, they don’t think in terms of “yes” or “no” but in terms of probability. They break problems down to their component parts and don’t think holistically.
Cass Sunstein, Robert Walmsley University Professor at Harvard Law School and co-author of NoiseSelected Good Judgment publications on the science of Superforecasting.
Decision, 11(1), 60–85. DOI:10.1037/dec0000211.
International Journal of Forecasting, DOI:10.1016/j.ijforecast.2023.12.009.
Futures & Foresight Science, e173, DOI:10.1002/ffo2.173.
Sports Medicine – Open, 9 (100), DOI:10.1186/s40798-023-00629-w.
Behavior Research Methods, DOI:10.3758/s13428-023-02234-x.
Perspectives on Psychological Science, DOI:10.1177/17456916231185339.
Forecasting Research Institute
Futures & Foresight Science, 2023;e157, DOI: 10.1002/ffo2.157.
Haas School of Business, Berkeley
PsyArXiv (online)
Intelligence and National Security, 38(4), 558-575, DOI: 10.1080/02684527.2022.2142352.
Wiley Online Library
Open Forum Infectious Diseases, ofac354.
BMC Infectious Diseases, 22, 833.
SSRN (online)
Journal of Applied Research in Memory and Cognition, 11(1), 1-18.
SSRN (online)
SSRN (online)
In C. L. Frisby, R. E. Redding, W. T. O’Donohue, & S. O. Lilienfeld (Eds.), Political Bias in Psychology: Nature, Scope, and Solutions. New York: Springer.
SSRN (online)
SSRN (online)
International Journal of Forecasting, 38(2), 688-704.
Futures & Foresight Science, 3(1), DOI:10.1002/ffo2.64.
Journal of Experimental Social Psychology, 96(6004), 104157
SSRN (online)
Organizational Behavior and Human Decision Processes, 160 (September 2020), 19-35.
American Psychologist, 74(3), 290-300.
Journal of Politics, 81(4), 1388-1404.
Cognition, 188 (July 2019), 19-26.
ResearchGate (online)
International Studies Quarterly, 62(2), 410-422.
Journal of Forecasting, 37(3), 259-268.
Frontiers in Psychology, 9, 2640.
Intelligence and National Security, 33(3), 337-356.
Judgment and Decision Making, 12(6), 610-626.
Judgment and Decision Making, 12(2), 90-103.
Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 2348-2357.
Science, 355(6324), 481-483.
Management Science, 63(3), 587-900.
Judgment and Decision Making, 12(4), 369-381.
Electronic Journal of Statistics, 11(2), 3781-3814.
International Journal of Forecasting, 33(4), 817-832.
Management Science, 63(11), 3552-3565.
Judgment and Decision Making, 11(5), 509.
Decision, 3(1), 1-19.
Journal of the American Statistical Association, 111(516), 1623-1633.
Intelligence and National Security, 31(6), 903-920.
Decision Analysis, 13(2), 128-152.
Statistics & Probability Letters, 119, 170-180.
Perspectives on Psychological Science, 10(3), 267-281.
arXiv:1506.06405v2 [stat.ME]
Management Science, 61(2), 267-280.
Perspectives on Psychological Science, 10(6), 753-757.
Journal of Behavioral Decision Making, 28(3), 250-261.
Journal of Experimental Psychology: Applied, 21(1), 1-14.
California Management Review, 57(1), 5-15.
International Journal of Forecasting, 30(2), 344-356.
Current Directions in Psychological Science, 23(4), 290-295.
Proceedings of the National Academy of Sciences, 111(32), 11574-11575.
Annals of Applied Statistics, 8(2), 1256-1280.
Psychological Science, 25(5), 1106-1115.
Decision Analysis, 10(4), 205-326.
Decision Analysis, 11(2), 133-145.
Judgment and Decision Making, 8(3), 188-201.
Critical Review, 24(3), 375-391.
2012 AAAI Fall Symposium Series
Schedule a consultation to learn how our FutureFirst monitoring tool, custom Superforecasts, and training services can help your organization make better decisions.