I am not a statistician, but this old post from Dominic Cummings (now “disgraced”) leaves a lot to think about. Dominic wrote about ability to forecast.
A review of Tetlock’s ‘Superforecasting’ (2015) – Dominic Cummings’s Blog
Tetlock also found that a small fraction did significantly better than average. Why? The worst forecasters were those with great self-confidence who stuck to their big ideas (‘hedgehogs’). They were often worse than the dart-throwing chimp. The most successful were those who were cautious, humble, numerate, actively open-minded, looked at many points of view, and updated their predictions (‘foxes’). TV programmes recruit hedgehogs so the more likely an expert was to appear on TV, the less accurate he was. Tetlock dug further: how much could training improve performance?
How did they do it?
How did they do it? GJP recruited a team of hundreds, aggregated the forecasts, gave extra weight to the most successful, and applied a simple statistical rule. A few hundred ordinary people and simple maths outperformed a bureaucracy costing tens of billions.
Dominic then throws a tirade on the sagging bureaucracy and suggests this as an alternate.
I have been thinking about the policy space for a long time now. The outliers in any thought process, especially when it presents as an alternative to the status quo, always have an appeal. It is for the same reason why pronouncements on “fixing the healthcare” in the US draw the crowds with frenetic calls for inclusiveness. However, as much as I need healthcare for all, each facility requires forecasting models about probability of people falling ill. Healthcare facilities have long gestation periods and require a considerable time to recover their costs (if at all).
Can we use these forecasting models? I’ll explore them.
RadOncNotes 