DAAN STEENKAMP AND PIETMAN ROOS: AI economists need to do more than blindly trust the machine

Imagine your doctor prescribes medicine without providing a diagnosis or explaining the treatment and its intended effect. Few adults would accept such a situation, yet decisionmakers have been infantilised by the allure of artificial intelligence (AI) when it comes to economic and financial forecasts.

Before AI, economists needed to understand and assess economic fundamentals to provide an admittedly imperfect picture of what could happen. However, some economists are turning to theory-free AI-generated forecasts, where the thinking seems to be “close your eyes and trust the machine”.

Machine learning, an application of AI, is incredibly good at recognising patterns and therefore automating things such as fraud detection, digital imaging or product recommendations. While it also shows tremendous promise for economic and financial forecasting, we need to get real about the difference between predictive analytics and capturing dynamic and bi-causal economic relationships.

Consider the relationship between a shopper and a loaf of bread. Using predictive analytics, retailers can accurately predict the likelihood that you will buy a loaf when you visit their store. They can use their knowledge about you to inform anything from pricing to how they market their baked goods to you. But this is ultimately a tactical exercise and is valid only if many things remain unchanged.

The economic relationship between a shopper and a loaf of bread, on the other hand, depends on a much larger set of factors, ranging from fiscal and monetary policy to product and labour markets. The multi-causal nature of economic relationships means that impacts from one variable to the next need to be considered over time and in terms of their cumulative effects. Long-term changes to taxes, interest rates and the labour market not only affect the shopper, but also the price of a loaf of bread.

Machine-learning frameworks that “look at all the data” can often do a good job of capturing relationships on a specific day, but they generally struggle to consistently explain the underlying economic drivers over time. Which is to say, using predictive analytics instead of economic models means that business decisions tend to become more tactical and less strategic. 

Traditional modelling approaches seek to capture the statistical properties of data and potentially relevant theoretical relationships. Any (responsible) economist should start their analysis by working to understand data and its limitations. Understanding data must inform modelling choices and how one assesses the usefulness of models and their projections.

Another potential issue with predictive AI models is “overfitting the data” — a mistake that is also common in conventional modelling — which is when a model includes redundant information. Even though overfitted models might accurately explain data trends, they tend to perform terribly at explaining movements in new data or at forecasting. This happens if a model captures noise in the data instead of the underlying relationships.

One sees this a lot with exchange rate forecasts: historical model output that can perfectly explain movements in currencies. However, exchange rates are affected by everything, with their volatility reflecting unpredictable factors, such as political and financial shocks.

Despite economists’ penchant for making confident predictions about where the exchange rate will go over the next year, the accuracy of their forecasts will depend on the extent to which unexpected shocks hit the economy. In this example, it is also particularly important to understand how specific economic and financial shocks would affect currencies so that decisionmakers can manage their risks.

While economists are often skewered for hedging their bets when it comes to forecasts, at least they can explore different scenarios and consider the implications of relaxing an assumption or two. At present, AI models are much less useful than traditional models for constructing economic scenarios as they provide limited structure about economic relationships. Like second-hand dealers in ideas, analysts tend to hype the fact that results are AI generated, as opposed to speaking about the fundamentals underlying their projections.

The irony of the rapid adoption of AI in economic forecasting is that it has not realised the tangible value AI can offer for changing the way economists work. It is already possible to automate a lot of what economists do using AI. Models and assessments of their performance can be kept fully up to date, including analytical slide decks and reports. But humans are crucial to governing these processes. Deep understanding of the models applied and tools used is required to make sure results are not spurious.

Advances in computer processing capabilities mean that sophisticated analysis and scenario planning can now be done in front of executives in the boardroom. Model scenarios can be adjusted in real time to quickly answer questions and advise on the effects alternative strategies would have on the business. Data and model automation can turn every team member into a super-analyst and provide senior management with decision-ready analytics. 

For example, 10 years ago, it could take days to re-estimate a large-scale macroeconomic model. That made debate about the implications of alternative economic assumptions on the economy difficult to have around a boardroom table. Today, the same exercise can take seconds.

Automation will save on resources and transform the way analysts and economists work, dramatically improving their productivity. But realising this potential requires modernisation of data management and ways of work. Economists need econometric and coding skills to be able to understand whether models are working as intended and to clearly communicate the drivers of their forecasts.

The next time you are given any forecast, simply ask, “what are the assumptions?” If the answers is, “well, it’s AI and it is really sophisticated,” you might need a second opinion.

• Steenkamp is CEO at Codera Analytics and a research associate at Stellenbosch University. Roos is an associate with Codera.