So far I have worked with neural networks mainly in Matlab. However, some limitations led me to try with python. My choice fell on the Tensorflow library. I am comparing the accuracy of prediction on the very same data set and ANNs topology in both environments. First Matlab implementation:
net = feedforwardnet([18 12 23],'trainrp');
rng('default') %Just for repeatability
net.divideParam.trainRatio = 75/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;
net.performFcn='mae';
net.layers{1}.transferFcn = 'logsig';
net.layers{2}.transferFcn = 'softmax';
net. Layers{3}.transferFcn = 'softmax';
net.trainParam.showWindow = 0;
[net,tr] = train(net,input,target);
output=net(input);
f=mean(mean(abs(target-output)./((abs(target)+abs(output))/2)))*100;
The final error f=8.0164 which is acceptable value. Now I reproduce the same ANN using tensorflow in Python:
#For repeatability
SEED = 777
random.seed(SEED)
tf.random.set_seed(SEED)
np.random.seed(SEED)
# split the data into train and test set
train_x, test_x, train_y, test_y = train_test_split(
input_array, output_array, test_size=0.15, random_state=7, shuffle=True)
# split the train into train and validation set
train_x, val_x, train_y, val_y = train_test_split(
train_x, train_y, test_size=0.17, random_state=7, shuffle=True)
# Model function definition (21 features in every observation, 11 features in output, the same as in Matlab)
model = tf.keras.Sequential([
tf.keras.layers.Input(shape=(21,), name="Input"),
tf.keras.layers.Dense(18, activation = 'sigmoid', name="Hidden1"),
tf.keras.layers.Dense(12, activation ='softmax', name="Hidden2"),
tf.keras.layers.Dense(23, activation ='softmax', name="Hidden3"),
tf.keras.layers.Dense(11, activation = 'linear', name="Output")])
# Compilation of model
model.compile(optimizer='RMSprop',
loss='MeanAbsoluteError')
# fitting the model
model.fit(train_x, train_y, epochs=200, batch_size=64,
validation_data=(val_x, val_y),verbose=0)
values = model.predict(input_array, verbose=0)
Error=abs(values-output_array)/output_array*100
f=np.mean(Error)
In this case f=94.4482 which is very huge difference. Could you help me figure out why python can't handle it? I was trying to mess with epoaches and batch_size but the difference is like +/- 5% so nothing special. The dataset I use is a bit big and I can't share it, but I hope you can use any dummy data to reproduce my MWE.
Edit:
I still did not solved the problem but I have a clue, that maybe you can use to help me figure it out. Matlab use some kind of automatic data preprocessing (normalization?) that I think Python does not do. When I switch it off by:
net. Inputs{1}.processFcns={};
net.output.processFcns={};
The f=35.5123. Still much lower than in Python but it suggest that my python code my misses some auxiliary operation before the data is send to the Dense layer.
