On July 5th, we sent a short feedback survey to over 26,000 forecasters who had joined Good Judgment Open since the site opened to the public in September 2015, hoping to learn more about how our forecasters use GJ Open and what they hope to gain from participating in a massively open online forecasting site. Since then, we’ve heard from over 700 of you (or about 3% of all forecasters) about why you joined and what you’d like us to work on improving.
By and large, you’ve told us that you joined GJ Open to become a better forecaster, and that you want more feedback and tools to help you do so.
When asked Why did you sign up for GJ Open?, over 72% of respondents rated the answer to find out how good I am at forecasting as “important” or “very important” and over 66% rated the answer “to get feedback and become a better forecaster” as important or very important. You also indicated that you joined to learn about new topics (over 57% rated important or very important) and to follow the news on current events (50%), but placed less emphasis on interacting with other forecasters (29%) and sharing your opinion on current events (25%).
These priorities were reflected in the features you asked for. When asked to rank the importance of four types of features we’re considering developing, you indicated that you would most like to see detailed feedback about your forecasting accuracy and to receive training on how to become a better forecaster, as over 60% of you ranked those two options above features that help you follow the news and learn about new topics and social features that let you interact with other forecasters.
The crowd on Good Judgment Open is incredibly diverse, as are your reasons for forecasting. But the results of this survey tell us that most of you see the site not just as a fun game or a chance to interact with other people, but specifically as an exercise in judgment and as a way to improve your skills so you can apply them to other areas of your life. Given these results, our data science team will be working closely with Cultivate Labs, our partners in developing the current Good Judgment Open forecasting platform, to prioritize the development of new features that provide you with better and more detailed feedback on your forecasting accuracy and that train you to become a better forecaster based on the science of forecasting. And we have hundreds of thoughtful and detailed comments from the survey on specific improvements we can make to achieve those goals.
Of course, we recognize that not all of our forecasters joined GJ Open because they see it as an educational opportunity. We believe that forecasting can be fun and social (in fact, the ninth commandment of Superforecasting, “Strive to bring out the best in others and let others bring out the best in you,” explicitly encourages interaction as a path to improvement) – so we will continue to develop features that facilitate social interaction and to offer challenges that cover fun topics.
The feedback you’ve provided shows us that many of you care about GJ Open just as much as we do, and we thank you for putting in the time and effort to help us make it even better. For those of you who have yet to fill out the survey, we’re still monitoring the results, and you can complete it here.
– The Good Judgment Open Team