Oracles
Which analysts are the best at predicting inflation?
Forecasting the future of economic variables has been a crucial topic for investment banks, research firms, and brokers since the dawn of time.
Now, with interest rates close to modern-time highs and the impacts of the pandemic still looming over inflation figures, having an accurate forecast for the CPI has become more important than ever.
Read by almost every researcher, trader, and economic analyst with exposure to the Mexican economy, the famous “Encuesta Citibanamex” collects inflation forecasts from experts each month to provide the market with an overall sentiment on where price increases might head in the near future.
However, to the best of our knowledge, we’ve never seen an in-depth analysis on the accuracy of these forecasts. How reliable are they? Which firm has had the most consistent results in the past few years? And how can we best use this valuable data to make important business decisions?
That’s why, for today’s Margin article, we’ve decided to do something a little bit different than usual and assess these questions.
We hope you find it useful.
Predicting where the market is going to move is, in itself, a very complex task. Economists and analysts, spend countless hours tuning and feeding their models with data to provide their clients and trading desks with the most accurate forecast.
Since 2018, the consensus from research firms covering the Mexican economy has missed the observed inflation rate by about 185 basis points each year, on average.1
This figure is affected, of course, due to the much higher-than-expected inflation rates observed in 2021 and 2022. If we exclude those two years, the average error would be around 62 basis points (bps).
As expected, predictions made a year in advance consistently have higher error rates than those made closer to the forecast’s “target date” — especially in years with considerable volatility.
For example, by looking at predictions made between 2018 and 2023, we find that forecasts made 10 months (or more) prior to the official release of annual CPI figures had an average error of over 150 bps. As the end of the year approaches, the estimates become more accurate, with the average error decreasing to 55 bps for predictions made 6 months prior and 16 bps for those made 1 month prior to the official data release.
When excluding 2021 and 2022 from the sample, we see that the only major improvements in accuracy are 9 months and 1 month prior to INEGI’s press release.
Using 2022 as an example, we can observe how analysts' projections begin to adjust as the year progresses. However, this adjustment is sometimes not enough when there is too much uncertainty in the market.
Notice how at the start of December 2022—just a couple of weeks before INEGI published the country’s CPI figures—all analysts predicted that inflation would still surpass the 8% mark. Eventually, the rate closed the year at 7.8%.
Yes, you might think that a 20 bps error is close to perfect.
However, the stakes are extremely high for professional research teams. If you take a quick look at the previous chart, you can see that the average error for a 1-month forecast in a “normal year” was just about 11 bps.
Now that we've covered how the consensus performs, it’s time to take a look at individual analysts’ results.
But before we proceed, please bear in mind that the ranking may differ greatly depending on the methodology used to evaluate these forecasts. To “balance things out,” we’ve decided to evaluate firms across two categories:
Short-term: Forecasts made 1 month prior to the CPI release.
Long-term: Forecasts made 12 months prior to the publish date.
For a more detailed explanation of our methodology, please refer to the foreword at the end of the article.
Based on short-term forecasts, just 7 firms (4 local, and 3 international research teams) managed to average an absolute error below 20 basis points during the past 4 years: BNP, Actinver, Banorte, UBS, Vector, BBVA and JP Morgan.
If we zoom into the year-per-year rankings, we can see that Finamex claimed the top spot for the data we have so far in 2024—marking an impressive improvement from their previous two-year rankings, which positioned them at the bottom end of the table.
We also believe that a special mention should be made to Actinver and UBS, the only two research firms to have consistently finished in the top 10 for four consecutive years.
Next, let’s take a look at the performance on a long-term basis—a category in which research teams are most competitive due to more conservative “bets” on the 12-month forecast for inflation.
Here, the difference between the top and bottom-ranked firms (Cibanco & Oxford Economics) is just 21%, compared to a 41% difference in the short-term forecasts.
Notably, only two research teams, Actinver and BNP, ranked in the top 5 for both long-term and short-term forecasts.
Finally, here are the year-by-year rankings for the long-term predictions category.
All in all, we believe that long-term inflation forecasts are still subject to considerable errors—with all firms averaging an error above 200 basis points (2 percentage points) for a 12-month forward inflation rate, and should therefore be taken with a grain of salt.
However, as the target date approaches, analyst predictions become impressively accurate. Even during 2021 and 2022, estimates made with a 6-month lag were off by just half a percentage point on average. Taking it further, forecasts made just a month prior to the official inflation figure were off by only 20 basis points on average, making it quite an impressive achievement for analysts covering the Mexican economy.
Foreword
For those interested, this is the methodology we followed to rank research firms’ CPI forecasts:
Data Collection: We collected data from all Citibanamex surveys that included at least January 2021 forecasts for monthly and yearly inflation figures.
Recent Predictions: We retained predictions only from research firms that published at least one estimate in the past two months (i.e., July and August 2024).
Averaging Multiple Forecasts: Since the survey is sometimes published twice a month, if a team published two forecasts during the same month, we averaged them out.
Minimum Predictions: We removed all research firms for which we didn’t have at least 26 predictions (out of a maximum of 52).
Error Calculation: With the selected sample, we computed the absolute error for each forecast and averaged these figures on a yearly basis.
For the “complete” rankings we excluded firms which didn’t have data for all years in the sample, such as: GBM and Bradesco.
This error is computed based on the absolute difference between the forecast made at the start of the year and the year’s observed inflation rate. For example, for 2019, the first forecast for that year’s inflation rate was published on January 7th; we compare that estimate with the observed annual inflation rate for December 2019.