Introduction
Demand forecasting has always been a complex and time-consuming exercise. However, since market disruption caused by the Covid-19 pandemic and resulting quarantine measures, many companies are finding it a nightmare to accurately forecast new products launches. This is a critical area to reset — forecasting accurately empowers large, omnichannel manufacturers to get the right product to the right channel in the right store. It ensures you promote those products best positioned for success, to maximize ROI behind your innovation and marketing investments.
New Product Launches: The benefits of improved forecasting
In discussions with our clients and partners, we hear specific benefits around improved new product forecasting, such as:
- Optimizing innovation plans by ensuring market demand is sufficient for a new entrant;
- Selecting and prioritizing the best new concepts for market activation;
- Stretching R&D budgets further to accomplish more with less;
- Providing more accurate predictions of cost and revenue to the finance silo and senior management;
- Reducing wasteful investments on products that fall short of expectations.
Sales forecasting ordinarily predicts demand using historical modeling from point of sales data. For products already on the market, companies depend on historical sales of that product to predict future velocity. The rub, however, comes with new products that haven’t launched yet and have no sales history. As a result, companies deploy a variety of tactics to forecast new product launch sales, many of which no longer work due to excessive “noisy data” following recent, massive market disruptions.
This article will introduce you to the most common methods that global enterprises use, as well as the obstacles and solutions to predicting new product success.
Method 1: Gut Instinct with Excel
The most common method that large corporations use to predict the demand for a new product launch depends on a combination of industry knowledge and Microsoft Excel spreadsheets. An analyst or marketer will go through an Excel spreadsheet that contains historical sales data and, using linear regression models, creating 52-week rolling forecasts. Afterwards, the same person will look at the new product’s description and attempt to select similar products to it. Next, they will try to find a few products that appear similar to the new product that is awaiting launch. The same analyst or marketer will then attempt to average selected similar products’ sales forecasts to create a sales forecast for the new product.
The fundamental issue with this approach is that it is far too dependent on human perception and human error. One analyst or marketer may believe that a certain set of products is similar to the new product, and another may believe that a different set of products is analogical. This dependence on human perception creates inaccuracies in the forecasts, especially when trying to predict what a consumer’s perception will be.
The second issue is that this process reduces the size of data, used in a forecast, substantially. To be more specific, if an analyst or marketer selects 5 products they believe are similar, with similar attributes, to make the forecast and the forecast of the new product depends on only 5 data points,–then it isolates it from the rest of the category. A smaller sample size often means a lower resulting level of accuracy. Therefore, the best solution to this is to focus more on the attributes of a product to enlarge the sample size of analogical products.
Method 2: Accumulating Estimates from Local Teams
Another common method of forecasting new products launches is for the centralized team to ask each of their local sales managers to provide an estimate of sales for that product over a certain period, and to then accumulate those estimates. Sales teams that speak with each retailer, then create their own sales forecast, based on their conversations with the retailer, in terms of potential shelf availability, product placement, and other products they’ve sold that they consider as performing similarly to the upcoming product to be launched.
Unfortunately, there are two major concerns associated with the “salesforce forecast.” Overall, there’s minimal checks and balances to the sales teams’ forecasts, with their opinions ultimately carrying the most weight in impacting the decision to launch the product or not.
The first concern is the probability that a sales team overestimates their expected sales volume resulting from new products launches. Often, sales teams are extremely optimistic about their product’s potential sales volume and, unintentionally, overestimate the sell-through volume expected from the launch.
The second concern is the opposite: underestimating expected sales volume from new products launches. Most salespeople are managed through a set of Key Performance Indicators (“KPIs”) and may receive rewards for exceeding their KPIs. So, a sales team could knowingly, underestimate the sales forecast of a new product launch in order to, seemingly, over-achieve on their KPIs and, therefore, get rewarded.
Method 3: Executive forecast
The third method of sales forecast for new products is, arguably, the most common, yet also the simplest: executive-driven forecasting. In this approach, the company planning new products launches will have its executives make their pre-launch forecasts in a meeting. Their prediction will be based on their experience with the company, relevant Excel worksheets, and knowledge of the upcoming product to be launched.
Often, this approach is used to save time and crowdsource the decision-makers’ opinions. However, similar to the previous two approaches, there are inherent issues here as well.
To start with, the time span involved in forecasting here is extremely narrow. In many cases, the executives will make the new product forecast in a matter of minutes—sometimes hours— while ignoring finer details, like changing consumer demand patterns, uplift in competition, cannibalization from the launch, the uniqueness of the new product, and the overall consumer need for that product.
A second issue, which aligns well with the above methodologies’ flaws, is the dependence on raw human instinct or gut feeling. Executives combine their opinions of market changes—and the upcoming product launch—with their memory and gut instinct, in order to make a rolling 52-week forecast, months before a launch.
The approach isn’t completely flawed, however. Executive-driven forecasting is effective for rapid decision-making, without having to deep-dive into relevant data.
Method 4: Consumer Panel Extrapolation
The fourth method of forecasting demand for a new product is by extrapolating results from a Consumer Panel. Primary research is an important part of new product development. It includes concept testing, where new product concepts are benchmarked against each other, according to the opinions of potential consumers.
Using consumer panels for new product sales predictions involves estimating the trial potential for a concept, by assuming there is 100 percent product awareness and distribution. By integrating price elasticity models, agencies, or Fortune 500 companies, can extrapolate panels to correlate with seasonal trends.
Three obstacles prevent consumer panel extrapolation from being a vital asset for new product demand forecasting.
The first barrier is that surveys and focus groups, typically, represent a small group of consumers who are extrapolated to represent much larger sample size. As mentioned previously, small sample sizes often lead to inaccurate results.
The second concern with forecasting by a consumer panel is its dependence on status quo data. Consumer opinions, today, are not representative of the opinions of last year or next year; consumer interests change rapidly. So, depending on a group of consumers’ opinions on a certain day can be the cause of skewed results. For example, if you’re launching a new ice cream on a particularly hot day, the consumer panel results can display higher purchase interest in the product than when it actually goes to market, which may not be on such a hot day.
A third issue is the cost barrier. Many Fortune 500 companies launch hundreds of products annually. Running consumer panels for each of those launches, when they’re priced in the six to seven-figure range, can be very costly. After all, they have to pay people and consulting agencies large service fees.
Do these obstacles make a meaningful difference in forecasting new products launches?
A few decades ago, these obstacles were relatively meaningless to businesses and Fortune 500. There were far fewer products, consumer interest was much easier to predict and product life cycles were much longer. Since then, the proliferation of eCommerce, social media, and search engines have fragmented consumer demand substantially. This makes accurately forecasting demand for new products launch nearly impossible, with conventional methods.
In the Consumer Packaged Goods category, 30,000 new products are launched every year. According to Clayton Christensen, a professor at Harvard Business School, 95 percent of those 30,000 new product launches fail.
The number one cause of product launch failure is a lack of understanding of consumer interest in a product. This is then relayed to the product launch through inaccurate forecasting.
Simporter makes accurately forecasting demand for new products easy
Simporter’s AI-driven software platform changes the challenge and inaccuracy of forecasting new product launches. The Software-as-a-Service (“SaaS”) tool combines external datasets with historical Point of Sale (“POS”) data. This is used to train Artificial Intelligence algorithms to, accurately and automatically, predict the first 52 weeks of a new product launch, on the market and at the store-level.
First, Simporter’s AI extracts product attributes from their descriptions using Natural Language Processing. This removes the human factor in identifying analogical products, which magnifies accuracy rates.
Second, models analyze Simporter’s large databases of search traffic data, social media sentiment, and consumer reviews. This helps them understand consumer needs and competition for each of the new product’s attributes. These data-points, when trained with historical sales data, are able to further enrich the AI model’s accuracy. They do this by capturing predictive signals that display future demand shifts by consumers.
With millions of data points, Simporter trains its models for 3 weeks before the handover to each client. This process allows the AI models to become domain-specific. As a result, it empowers Fortune 500s to gain from the automation and accuracy of AI, without using generic models. Occasionally, there are product categories that do not have much online data, like milk, for example. Therefore, the domain-specific models are able to adapt to the category’s limitations and still maintain high accuracy, by increasing the model’s influence on historical sales data, for example.
How Simporter Forecasts Demand for New Products Launches
There are two unique components of Simporter’s AI models. The first component is our proprietary use of Light Gradient Boosting Models (“Light GBM”). Light GBMs learn from each forecasting iteration, thus growing the probability that the decision tree fits residual errors.
This means that every subsequent iteration tries to learn the difference between actual output and the weighted sum of predictions from the previous iteration. With the gradient method, errors are minimized.
The second component of Simporters AI models is NLP. Together with the past four years of historical POS data, Simporter uses NLP to extract the attributes and features of each product from its description. With this data, models can begin to identify patterns between attributes and sales and collect external data points around those categorical attributes. If there’s a new category or attribute that doesn’t yet exist in the category, Simporter’s NLP can examine adjacent categories and increase the weights of its external data points, that better classify consumer needs.
Going forward, our models integrate marketing variables, such as packaging, distribution plans, pricing, and promotional plans to identify and retain patterns between marketing variables and historical data points, like sales or search interest.
Accuracy is measured by training models, using 80 percent of the historical data and testing the models with the remaining 20 percent. After the data scientists enrich the domain-specific model to a high enough accuracy point, our backend software engineers then automate these models. This turns them into autonomous, software-functioning pre-launch models. It also allows a Fortune 500 to simulate any range of product launch scenarios, in seconds.
The Outputs of Simporter’s New Launch Forecasting Software H10
Simporter provides an intuitive, easy-to-use interface. It empowers both technical and non-technical users to access highly accurate, automated launch forecasts. In the dashboard, a user is shown their upcoming launches that have already been added to the system. A user may select one of these launches to analyze or create a new launch in the system. This is done by inputting the attribute-level information, necessary marketing variables, manufacturer suggested retail price, distribution levels, and the launch dates for each retailer. Users can then compare their launch with other upcoming product launches, or products that are already on the market.
After a user inputs a launch, the analysis screen first displays a 52-week rolling forecast of the product. It starts at its product launch date for each channel, with predicted revenue and units for each week, and for each retailer. If a comparable product is selected, you can view its historical sales and then compare its sales forecast against that of the new product.
Simporter’s AI models have proven to be between 91 and 96 percent accurate, with lows of 78 percent and highs of 98 percent. These metrics are achieved by measuring the Mean Standard Error of our previous demand forecasting with actual sales results.
If a user decides to change a variable of new products launches, such as an attribute or price in the product, they can do so in just a few clicks—and immediately get an update in their sales forecast.
Furthermore, Simporter software opens up the black box in its predictions and presents the influence of each attribute in making a sales forecast. This allows a user to understand the weights behind the prediction and so confidently validate the model’s methodology.
Further down the page, Simporter’s pre-launch forecasting software shows a user the most likely products the new launch will cannibalize. This is done using its proprietary Similarity Index, which matches products based on their attribute similarities. In addition, the software empowers a user to make price-based decisions by outputting a price-index of the new product, relative to the market average.
For further insights, a user can update their forecasting demand for new products based on simulations of different product promotions, price changes, distribution adjustments, and media mix.
Once a product enters the market, Simporter models begin learning from their pre-launch predictions, with the actual sales data and monitoring the product’s performance. With this updated data, the models start to increase the weight of historical sales data and create updated, accurate forecasts for products on the market.
If you’d like to learn more about Simporter’s prelaunch forecasting software, you can sign up for a quick demo here.
Closing Remarks
Forecasting demand for new products launches is difficult but can be the difference between innovative success or failure. With Fortune 500s launching huge amounts of products every year, upgrading from conventional forecasting techniques to AI-driven techniques, like Simporter’s, represents a valuable method to move ahead of the competition.
Conventional forecasting methods, such as gut instinct, sales managers, executive opinions, and primary research, were effective before the proliferation of online data but are no longer effective today.
By leveraging domain-specific AI models through Simporter’s SaaS platform, Fortune 500s can now take a major step forward in meeting their consumers’ needs and expectations more accurately.