Objective : forecast product movement of individual product category for each store. customer has around 1000 product categories and over 2000 store.
Approach :
- Use ARIMA with regressors (with lead and lag effect) , TBATS, NNET , other models on training dataset and test the accuracy against test dataset
- select best model based on their MAPE and boxtest results.
- use best model identified above to generate forecast.
Challenge : The code is taking around 3-5 minutes to run for one product one store. I need to run this code for around 1000 products across 2000.
Question : How do we achieve scalability when we are running forecasting for each store each category ?
Note : All four cores of machine is utilised while running the code. CPU utilisation 80%.