Jul 11, 2022
Forecasting with Limited Historical Data
A Novel Method that is Revolutionizing Demand Planning
By Nate Lanier and Veronika Andreeva
Using historical data to predict future trends is a method that has been around for a long time. When it comes to business applications, supply chain and revenue leaders across industries (from retail and manufacturing to airlines and customer service) adopted the method as an effective and straightforward way to anticipate and prepare for future demand. As a result, using historical data became a standard business practice and might even be perceived as the only reliable forecasting technique available today. However, The latest MIT research shows that it’s more complicated than that.
It’s true that past events can help us make inferences about trends and patterns. For example, consider coffee. If hot coffee was typically the most popular consumer choice during cold months in the past, it’s fair to assume that trend will continue. Yet, aside from seasonality, there are other important factors to consider: coffee’s relationship with other caffeinated beverages and how the demand for those beverages affects the demand for hot coffee.
For instance, suppose a portion of consumers discover that a latte is a milder drink and decide to make it their beverage of choice. Hot coffee sales could be negatively correlated with the sales of lattes. At the same time, imagine that consumers who prefer a latte also like to get a croissant with it. So, if you sell more lattes, you also sell more croissants — and less hot coffee.
In our example above, we first looked at hot coffee as a standalone item and, by analyzing its relationship with time, could predict its future sales patterns. The same would be true for the other two products, lattes, and croissants. Then, once we considered all three products as a cohesive system of relationships rather than unrelated products, we were able to add more dimensions to our analysis (coffee vs time; coffee vs latte; coffee vs croissant). Therefore, we could draw a more nuanced picture of consumer demand.
Thus, the greater the variety of products or services, the more dimensions that can be added, which yields a higher forecasting accuracy. And this is where the most exciting discovery comes in. As it turns out, not only do these inter-system relationships add more accuracy to time series forecasting, but they can also serve to fill in the gaps if historical data is limited to as little as 2–3 weeks.
This novel method has been thoroughly studied and fully developed by MIT researchers recently. Because this method looks at demand forecasting holistically, its forecasting accuracy surpasses existing state-of-the-art techniques, including deep-learning methods.
Forecasting with limited data is especially valuable during times of uncertainty, such as the effects of COVID-19 on virtually all industries, in terms of disrupted supply chains and shifted consumer demand. Ikigai makes this method available at scale in a no-code manner through its operational BI platform, which is already used by forward-looking enterprises as well as fast-growing start-ups within finance, insurance, retail, logistics, and other industries. At the end of the day, forecasting is not the end goal but a means to an end. To learn more about high-quality forecasting techniques that can produce insightful actions for questions — such as “What SKUs should I order today and how many?” — try Ikigai today to ensure the best return on investment.
About the Author
Nate Lanier (Applied Data Scientist)
Nate Lanier is an Applied Data Scientist at Ikigai Labs where he is working to build out the platform’s AI / ML capabilities. He holds a Masters Degree in Data Science from Brown University and a BS in Resource Economics from UMass-Amherst. His interests include all things Machine Learning, Data Science and AI ranging from Natural Language Processing to Time Series Analysis.
Veronika Andreeva (VP, Marketing)
Veronika Andreeva is the VP of Marketing at Ikigai Labs where she helps business leaders learn how to use AI, automation, and data tools to thrive in a fast-paced environment. She holds an MBA from Cornell University.
