AI is now beating the forecasters

[james22]

We have been writing a lot about forecasting failures. We came across a wonderful recent story that should have deserved more attention. Admitted, it did in the narrow world of machine learning and artificial intelligence. It is about the Makridakis forecasting model competition, named after a professor from Cyprus, the latest version of which focused on the stock market.

 

The headline news is that a new generation of modern forecasting tools has gradually been emerging as the most successful over the years. Of the 163 entries, the winner is a model based on a very new method, called meta-learning, which one can translate as learning how to learn. They don't have a method, they pick a method. The second-placed entry follows last year's trend that showed the power of another approach, which goes by the name of gradient boosting.

 

We would like the organisers to go for an economic forecasting competition next time, though we doubt that the central banks and other official forecasters would willingly participate in a competition that risks highlighting their failures. Our hypothesis is that modern machine learning methods would outperform them, and do so by a wide margin. This is quite astonishing because there is absolutely no economics input in those new forecasting methods. The winner of the stock market competition was totally ignorant of financial economics. 

 

The main reason for the relative success of the new generation of forecasting models is lack of bias. Forecasting is very difficult. It is hard to beat the dart-throwing monkey dumb forecast. In the machine learning competition, the dumb forecast occupied 12th place out of 163. Another old-school model, that simply used implied prices from the options market, came eighth. You can think of this as the efficient-market-hypothesis entry. That raises the question of whether the top seven are merely there because of luck. After all, more than 150 entrants were not so lucky.

 

Another important point raised by one of the competitors who did well with a new-generation method is that the big money in stock markets is not made through forecasting, but through deep understanding: market intelligence. Those players are unlikely to take part in any competition. That would be our main conclusion too.

 

Forecasting is, to a very large extent, a mug's game, and that applies to the new world of AI as well. For us, models have some limited value, not because they can predict the future but help us understand the past, and throw up connections that we would otherwise not have made.  

 

But if forecasting is your thing, then you are about to get a lot of competition.

 

https://www.eurointelligence.com/