Forecasting and group decision making
The accuracy of predictions of events such as the outcomes of political
elections can be improved by a variety of methods for combining the predictions
of individual forecasters.
We are conducting a study with over a thousand of forecasters, each
making hundreds of forecasts, to study how best to
elicit, weight, and combine the judgments of many experts.
We gave each forecaster a variety of intelligence and personality
tests, and then randomly divided them into different
"conditions" with different training (e.g. in probability and
forecasting) or with different ways of interacting
(no interaction, teams, prediction markets, ...). The results show the benefits
of information exchange, as in teams and prediction markets.
We are also developing new statistical algorithms for aggregating the
judgments by many individuals.
Issues include how to model the decision processes of individuals and groups,
how to optimally transform the probability estimates,
and how to best weight individual forecasts based on forecaster
confidence, expertise and cognitive style.
This work is supported under the IARPA ACE project which aims "to
dramatically enhance the accuracy, precision, and timeliness of
forecasts for a broad range of event types, through the development of
advanced techniques that elicit, weight, and combine the judgments of
many intelligence analysts."
Joint work with a large team,
including Barb Mellers, Phil Tetlock, and a host of others.
For more information
- Improving Geopolitical Forecasting with Teamwork, Training and Algorithms
Barbara Mellers,
Lyle Ungar,
Jonathan Baron,
Jaime Ramos,
Burcu Gurcay,
Katrina Fincher,
Sydney Scott,
Don Moore,
Pavel Atanasov,
Sam Swift,
Philip Tetlock
- Combining Multiple Probability Predictions Using a Simple Logit Model
Ville A. Satopaa, Jonathan Baron, Dean Foster, Barbara Mellers, Philip
Tetlock, Lyle Ungar
International Journal of Forecasting, Volume 30, Issue 2, April–June 2014, Pages 344–356.
- Two reasons to make aggregated probability forecasts more
extreme
Jonathan Baron, Lyle H. Ungar, Barbara A. Mellers, Philip E. Tetlock
Decision Analysis
- The Marketcast Method for Aggregating Prediction Market Forecasts
Pavel Atanasov, Barbara Mellers, Lyle Ungar, Philip Tetlock, Phillip
Rescober and Emile Servan-Schreiber
International Conference on Social Computing, Behavioral-Cultural
Modeling, & Prediction (SBP13)
Many of these papers can be found here.
home: ungar@cis.upenn.edu