Conventional models beat machine learning in predicting crises, paper finds

A conventional logit approach works significantly better than machine learning models in predicting financial crises, a working paper published by the Deutsche Bundesbank finds.

In An evaluation of early-warning models for systemic banking crises: does machine learning improve predictions?, Johannes Beutel, Sophia List and Gregor von Schweinitz construct an early-warning system based on a logit approach. They use this as a benchmark to contrast with machine learning models.

They then apply both the logit approach and the machine learning models to data on systemic banking crises in 15 advanced economies over the last 45 years.

The authors find machine learning methods often have very good predictive results for the data sample they were originally applied to. But, the authors find, “they are outperformed by the logit approach in recursive out-of-sample evaluations”.

Conventional models, the authors say, “appear to use the available information already fairly efficiently, and would for instance have been able to predict the 2007–08 financial crisis out-of-sample for many countries”.

They note these models “identify credit expansions, asset price booms and external imbalances as key predictors of systemic banking crises”.

The authors say their findings suggest that “further enhancements to machine learning early-warning models are needed before they are able to offer a substantial value added for predicting systemic banking crises”.