I am trying to reproduce analytically Matlab's neural network function to understand it better. I have already looked at the following Matlab NN architecture, Matlab input/output processing functions, and a related post but I am still having problems.
Here's my simple example. I want to fit a feedforward neural network with 1 hidden layer and say 10 neurons to estimate the function y = sin(4x) for x in [0,3] given some data. I want to write my own Matlab function to analytically express Matlab's neural network function.
From training the network net, Matlab tells me that the transfer functions are tansig (hidden layer 1), and purelin (output). It also tells me that the input and output processing functions are mapminmax.
Here's the code I wrote which is a simple example comparing: my analytic function, Matlab's prediction, and the true function I want to estimate:
rng(1);
xTrain = 0:0.5:3; % Training data
yTrain = sin(4*xTrain); % Training data
net = fitnet(10); % 10 neurons
net = train(net,xTrain,yTrain);
xPred = 0:0.1:3; % Prediction data
% Analytic
% Scale input
NNinput = mapminmax(xPred,net.inputs{1}.processedRange(1,1),net.inputs{1}.processedRange(1,2));
HL1 = tansig(net.IW{1,1}*NNinput + repmat(net.b{1},[1,size(NNinput,2)]));
HL2 = purelin(net.LW{2,1}*HL1 + repmat(net.b{2},[1,size(HL1,2)]));
% Scale output
PredValsAnlyt = mapminmax(HL2,net.outputs{2}.range(1,1),net.outputs{2}.range(1,2));
% True function
yTrue = sin(4*xPred);
% Matlab prediction
yPred = net(xPred);
figure;
plot(xPred,PredValsAnlyt,xPred,yPred,xPred,yTrue)
legend('My analytic','Matlab prediction','True')
Here's the output comparing my analytic function, Matlab's prediction, and the true function. While I don't expect Matlab's prediction to be close to the true function due to the coarse data, I expect that my analytic function and Matlab's prediction should be the same. What bothers me is that Matlab's prediction actually goes beyond the [-1,1] target range of the output. Am I missing out on something?
Update: March 29, 2019: After digging up some more information around various websites and playing around with Matlab, it seems like in the above code, instead of
PredValsAnlyt = mapminmax(HL2,net.outputs{2}.range(1,1),net.outputs{2}.range(1,2))
we have
PredValsAnlyt = mapminmax('reverse',HL2,net.outputs{2}.processSettings{1})
I am able to reconcile my code with Matlab's output.
So the problem now boils down to understanding mapminmax in Matlab with the reverse option and specified process settings. Is anyone familiar with this? It does not simply do a scaling procedure to achieve the target min and max because this is what I was doing in my original code.
