CHRISTOPHER LOEWALD: How Reserve Bank’s forecasting fares in uncertain times

Economic forecasting is always a challenge but is made far more difficult when there are random, large shocks and the underlying economic landscape is shifting. Forecasting normally depends on predictability and stability in key economic relationships. Large, unanticipated shocks to variables such as income, spending, oil or global growth force economists to rely more on judgment, experience and incomplete information. 

The past three years alone have seen major negative shocks, including the KwaZulu-Natal unrest (July 2021), the Russian invasion of Ukraine (February 2022), the KwaZulu-Natal floods (April 2022), a major Transnet strike (October 2022), and sustained logistical failures and significant energy load-shedding.  Other than the energy deficits, none of these were anticipated.

The table accompanying this article shows the root mean square errors for the SA Reserve Bank, IMF and a few prominent central banks for annual inflation and GDP forecasts of their respective economies, including and excluding the Covid-19 pandemic period. A smaller root mean square error number shows a more accurate forecast. The table tells us that the Bank’s inflation and GDP forecasts compare well. If we exclude the March 2020 monetary policy review forecast, which was put together before the pandemic shock, again our track record is good.

Sometimes forecasts show larger errors because of the difficulty of getting the starting point right, as the starting data has a significant effect on the short-run trajectory.  Nowcasting — the prediction of the very recent past, the present and the very near future state of an economic indicator — has emerged as one approach to improve on this, though it has its own serious shortcomings derived from incompleteness of data, its volatility (think of monthly output data) and the complexity of the real world.

Individually, nowcasting performances are often poor, leading to averaging of outcomes from various nowcast models. Nonetheless, the Bank employs a range of short-run and nowcasting models covering inflation, specific prices, food, the supply side of the economy, commodity prices, mining sales and production, and the exchange rate.

Moreover, because of our forecasting and monetary policy committee schedule we often need to use these models with limited data, making comparisons with other nowcast models difficult. While it is always convenient to choose a particular forecast to compare ours against, most forecasting reviews will use the mean or median of market forecasts as a benchmark in comparisons, instead of that of an individual institution such as the IMF.

Of course, the data we have access to matters. Whereas the IMF probably has the most extensive database to draw from, the Reserve Bank has access to only publicly available data. We used to have early access to the Bureau for Economic Research (BER) inflation expectations survey, which we sponsor, but we agreed in 2023 to release it to the market at the same time as we get it. However, as wonderful as the BER data is, it is only one input to a broader measure of expectations that makes our forecasting model more accurate.

The Bernanke review of the Bank of England will prove useful for the SA Reserve Bank (“Perhaps it’s time for Reserve Bank to review its modelling and forecasting”, May 29). However, it is clear that since Covid, inflation under-forecasting was common to all central banks, with the main deficits coming from oil and food prices and supply constraints, and inflation in major trading partners.

Global supply variables traditionally feature in forecasts, but not very strongly because they tend not to be volatile. No forecast can anticipate huge supply (or demand) shocks. The supply and demand variables need instead to tell us something about what happens to inflation afterwards.

Of course, the Bank remains open to suggestions that could further improve our analyses and forecasts and believes in continuous improvement of our frameworks. The quarterly projection model (QPM) was introduced as the workhorse model for the Bank’s monetary policy in 2017.  In 2021 we commissioned Laurence Harris (University of London) and Nicola Viegi (University of Pretoria) to evaluate our entire forecasting framework (models and processes).

Better evaluations

We also benefited from the National Treasury-initiated review of monetary policy in SA undertaken by Patrick Honohan (World Bank, Central Bank of Ireland and Trinity College) and Athanasios Orphanides (MIT, Federal Reserve and Central Bank of Cyprus governor). These reviews recommended better evaluations of “judgment” in forecasting, developing a fiscal block in the QPM and making technical adjustments to the output gap. A range of papers detailing how such suggestions were incorporated into our models and forecasting are freely accessible on our website. 

Forecasting plays an important role in the monetary policy committee process, not least by setting a baseline economic trajectory that responds to the repo rate and other shocks. Those other shocks and their potential sizes and probabilities of occurring are articulated before the committee sets out a view as to whether the risks to the baseline favour lower or higher inflation. The market appears to understand this process and outputs quite well.

We sometimes see a difference between what economic analysts say about the inflation outcome and repo path and what the financial institutions’ own traders signal via pricing in the markets. It is normal for analysts and traders to bet incorrectly about what decision a central bank makes, but it is not all that common. Of course, it is harder when there is greater economic uncertainty, but central banks are subject to the same uncertainty as the markets.

Is this overall process a good one? Our approach blends an intensive, if time-limited, engagement of committee members with the forecast and other analyses (global, conjunctural, market-focused). There are other approaches, such as some that have far more prolonged panel-beating of a single forecast by committee members, or that take the output of a relatively inviolable model directly. Another approach features a wide range of models coming from different constituent institutions, with a dispersion of the forecasts made public. Each of these has pros and cons.  Our approach has evolved over time, with adjustments made to deal with inefficiencies that emerge or are observed.

As recommended in the Bernanke review, our forecast team already presents alternative inflation scenarios to the monetary policy committee, but it’s not evident that publishing them would enhance clarity. Rather than overwhelming with hypotheticals, the committee focuses on explaining the most probable risks while using scenarios internally to challenge its assumptions and test potential outcomes. Ultimately, complaints about our approach and communication are really demands for “forward guidance”. However, in practice this was used to signal efforts to prevent deflation rather than to set quarter-by-quarter policy rate projections.

How useful is our research output? Somewhat different from advanced economy central banks, our research seeks primarily to inform policy-making, the monetary policy committee, our MPR, and our broader engagements with the markets, the public and interactions with international financial institutions. The level of output is far beyond where we were 10 years ago, even if some of our publications, say, on nowcasting, get relatively few academic citations. With our inflation rate (and forecasts) comparing well with those of the most advanced economies at present, a good case can be made that our research output is rather efficient. 

• Loewald is chief economist at the SA Reserve Bank.