Israeli Researchers Find Prediction Markets Surprisingly Accurate
Translated & summarized from NEWSru Israel by baba
Israeli scientists from Tel Aviv University analyzed over a million predictions from four platforms, finding collective forecasts to be surprisingly well-calibrated. The study, published in PNAS, covered ten years across various domains, noting that while generally accurate, predictions on conflicts and technology were overestimated, and election forecasts underestimated. Researchers advise using these predictions with awareness of known biases, especially for significant decisions.
The story in 5 lines · by baba
- Tel Aviv University researchers found collective predictions on four platforms to be surprisingly well-calibrated.
- Over one million predictions from ten years across twelve domains were analyzed for the study.
- Predictions related to military conflicts, international relations, and technology were systematically overestimated.
- Forecasts concerning elections were found to be systematically underestimated.
- The study, published in PNAS, suggests using predictions with awareness of known biases.
Scientists at Tel Aviv University have analyzed over a million predictions from four major platforms, investigating whether collective forecasts can accurately predict future events, especially when participants have financial or reputational stakes. The study, led by Professor Michael Gilead, examined 1,014,122 predictions spanning a decade across twelve domains, including elections, geopolitics, technology, and security, with prediction horizons ranging from hours to years. The research, published in the journal PNAS, focused on the calibration of these predictions. Calibration assesses the reliability of a forecaster's stated probabilities over a series of predictions; for example, if a forecaster predicts a 70% chance of rain, that prediction is considered calibrated if it rains approximately seven out of ten times when such a probability was stated.
The findings revealed that the collective predictions on platforms like Polymarket, Kalshi, Manifold, and Metaculus are generally well-calibrated, even early in the prediction period when much information is still unavailable. Accuracy increases as events approach, primarily because participants become better at distinguishing likely outcomes from unlikely ones. However, a detailed analysis uncovered systematic biases in crucial areas. Predictions related to military conflicts, international relations, and technology tended to overestimate probabilities, while those concerning elections consistently underestimated them. This is particularly significant as forecasts in these domains often influence real-world decisions.
"The main conclusion is striking: prediction markets and collective forecasting platforms are surprisingly well-calibrated," Gilead stated. The researchers emphasize that while collective forecasts offer valuable insights into probable outcomes, they should be used with an awareness of these identified distortions, especially when political, economic, or technological decisions hinge on the outcomes.
