Bringing the Wisdom of the Crowd to Help the Early Warning Project

Posted on Posted in Uncategorized

Good Judgment Inc (GJI), the Early Warning Project (EWP), and the United States Holocaust Memorial Museum (USHMM) are pleased to announce an expanded partnership in 2017-2018. The Early Warning Project Challenge is now open for forecasters on GJ Open. 

In the 2016 Early Warning Project Forecasting Challenge on GJ Open, forecasters predicted the likelihood of mass killing episodes in Ethiopia, Yemen, Turkey, Myanmar, the Democratic Republic of Congo, and other states identified by EWP as high-risk. This year, the EWP is not only moving its own expert forecasters to GJ Open, but GJ Open forecasts will now serve as the primary way that EWP tracks changes in the likelihood of mass killings in between their yearly statistical reports. Aggregated forecasts will be available on their website and will be used in briefs for policymakers and advocacy groups dedicated to preventing genocide and other mass atrocities. GJI is thrilled to leverage the wisdom of the GJ Open community to support EWP and the USHMM.

To make GJ Open forecasts even more useful to decisionmakers, and to address some of the feedback we received from forecasters in 2016, GJI and EWP have made changes to the format of the questions for this year.


  • We have moved away from asking about the “onset” of a new mass killing episode, and instead will ask about “systematic killing” of 1,000+ civilians, which enables EWP to share predictions on both the onset and continuation of mass violence.
  • Instead of providing narrow definitions of key terms, these new questions use examples to help forecasters understand what types of events would prompt an affirmative resolution by EWP. Relying on narrowly-defined terms shifts the focus away from the big picture risks that EWP is trying to capture. See the “More Info” section and the EWP example page (link) for more information.


  • This year, the questions ask about civilian fatalities suffered during a specific twelve month period in the future, moving away from the rolling timeframe of the 2016 questions.
  • These new questions cover a twelve-month time period, but forecasts will only be collected through the first three months of that time period. Because GJ forecasts decay toward a status-quo resolution as the end date of a question approaches – they often reflect the limited amount of time left before the forecasting question resolves rather than on changes that could prompt mass killing, limiting the the policy relevance of the question as the end date approaches. By limiting forecasting to the first three months of the question’s open period and then releasing follow-on questions each quarter, EWP can share continuous predictions about the likelihood of mass killing that do not reflect arbitrary end dates.

Although these questions are more complex than many of GJ’s standard forecasting questions, the importance of this topic and the utility of these forecasts for the policy community make them well worth asking and forecasting on. Help bring the wisdom of the crowd to this important issue by joining in the 2017-2018 EWP challenge on GJ Open.

Share on FacebookTweet about this on TwitterShare on LinkedInEmail this to someone

Leave a Reply

Your email address will not be published. Required fields are marked *