0

Below is the code of what I'm trying to do, but my accuracy is always under 50% so I'm wondering how should I fix this? What I'm trying to do is use the first 1885 daily unit sale data as input and the rest of the daily unit sale data from 1885 as output. After train these data, I need to use it to predict 20 more daily unit sale in the future The data I used here is provided in this link https://drive.google.com/file/d/13qzIZMD6Wz7e1GpOsNw1_9Yq-4PI2HrC/view?usp=sharing

import pandas as pd
import numpy as np
import keras
import keras.backend as k
import tensorflow as tf
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import Dropout
from keras.callbacks import EarlyStopping
from sklearn import preprocessing
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split

data = pd.read_csv('sales_train.csv')
#Since there are 3 departments and 10 store from  3 different areas, thus I categorized the data into 30 groups and numerize them
Unique_dept = data["dept_id"].unique()
Unique_state = data['state_id'].unique()
Unique_store = data["store_id"].unique()
data0 = data.copy()
for i in range(3):
    data0["dept_id"] = data0["dept_id"].replace(to_replace=Unique_dept[i], value = i)
    data0["state_id"] = data0["state_id"].replace(to_replace=Unique_state[i], value = i)
for j in range(10):
    data0["store_id"] = data0["store_id"].replace(to_replace=Unique_store[j], value = int(Unique_store[j][3]) -1)

# Select the three numerized categorical variables and daily unit sale data
pt = 6 + 1885 
X = pd.concat([data0.iloc[:,2],data0.iloc[:, 4:pt]], axis = 1)
Y = data0.iloc[:, pt:]

# Remove the daily unit sale data that are highly correlated to each other (corr > 0.9)
correlation = X.corr(method = 'pearson')
corr_lst = []
for i in correlation:
    for j in correlation:
        if (i != j) & (correlation[i][j] >= 0.9) & (j not in corr_lst) & (i not in corr_lst):
            corr_lst.append(i)  

x = X.drop(corr_lst, axis = 1)
x_value = x.values
y_value = Y.values

sc = StandardScaler()
X_scale = sc.fit_transform(x_value)

X_train, X_val_and_test, Y_train, Y_val_and_test = train_test_split(x_value, y_value, test_size=0.2)
X_val, X_test, Y_val, Y_test = train_test_split(X_val_and_test, Y_val_and_test, test_size=0.5)
print(X_train.shape, X_val.shape, X_test.shape, Y_train.shape, Y_val.shape, Y_test.shape)

#create model
model = Sequential()
#get number of columns in training data
n_cols = X_train.shape[1]
#add model layers
model.add(Dense(32, activation='softmax', input_shape=(n_cols,)))
model.add(Dense(32, activation='relu'))
model.add(Dense(32, activation='softmax'))
model.add(Dense(1))

#compile model using rmsse as a measure of model performance
model.compile(optimizer='Adagrad', loss= "mean_absolute_error", metrics = ['accuracy'])

#set early stopping monitor so the model stops training when it won't improve anymore early_stopping_monitor = EarlyStopping(patience=3)
early_stopping_monitor = EarlyStopping(patience=20)
#train model
model.fit(X_train, Y_train,batch_size=32, epochs=10, validation_data=(X_val, Y_val))

Here is what I got enter image description here

The plots are also pretty strange:

Accuracy

Loss

desertnaut
  • 57,590
  • 26
  • 140
  • 166
Paul Z
  • 31
  • 5
  • 1
    Hi Paul, and welcome. Could you perhaps add some more clarifications of what you want to do? What is the input data? What is the training target? Have you visualised the input data and labels to ensure that these are correct? – MPA Apr 19 '21 at 11:47
  • Hi MPA! thanks for ur reply. The input of my model: the data from day 1 to day 1885 daily unit sale and the output of my model is the predicted unit daily sale from day‐1886 to day‐1913. However, I don't know how to train my model to make it best fit for the output data provided. Also there are some categorical variables like department, location,etc. I simply numerized them but I'm not sure whether I'm on the right track on not – Paul Z Apr 19 '21 at 12:01
  • For getting constructive replies (and upvotes) it helps a lot to give a detailed context in which you want to solve your problem. From your loss curve it seems you model is actually learning something, so you could try to train it longer. Softmax seems to be the wrong activation function for your problem (which outputs probabilities that sum up to 1). For daily unit sales (which are >= 0), stick to ReLU. – MPA Apr 19 '21 at 15:10
  • For forecasting problems people have had some success with e.g. recurrent neural networks and LSTMs. Also check out [Steve Brunton's lectures](https://www.youtube.com/playlist?list=PLMrJAkhIeNNSVXUvppZTYNHKQUD-oWys9) on Koopman analysis if you think there is some underlying dynamics you want to uncover and use in your predictions. – MPA Apr 19 '21 at 15:10

1 Answers1

0

Two mistakes:

  1. Accuracy is meaningless in regression settings, such as yours here (it is meaningful only for classification ones); see What function defines accuracy in Keras when the loss is mean squared error (MSE)? (the argument is identical when MAE loss is used, like here). Your performance measure here is the same with your loss (i.e. MAE).

  2. We never use softmax activations in anything but the final layer of a classification model; replace both softmax activation functions used in your model with relu (keep the last layer as is, as no activation means linear, which is indeed the correct one for regression).

desertnaut
  • 57,590
  • 26
  • 140
  • 166