1

I'm trying to classify 1D data with 3-layered feedforward neural network (multilayer perceptron).

Currently I have input samples (time-series) consisting of 50 data points each. I've read on many sources that number of neurons in input layer should be equal to number of data points (50 in my case), however, after experimenting with cross validation a bit, I've found that I can get slightly better average classification (with lover variation as well) performance with 25 neurons in input layer.

I'm trying to understand math behind it: does it makes any sense to have lower number of neurons than data points in input layer? Or maybe results are better just because of some errors?

Also - are there any other rules to set number of neurons in input layer?

Update - to clarify what I mean:

I use Keras w tensorflow backend for this. My model looks like this:

model = Sequential()
model.add(Dense(25, input_dim=50, activation='relu'))
model.add(Dense(8, activation='relu'))
model.add(Dense(1, activation='sigmoid'))

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(input_data, output_data, epochs=150, batch_size=10)

predictions = model.predict(X)
rounded = [round(x[0]) for x in predictions]
print(rounded)

input_data, output_data - numpy arrays with my data points in former and corresponding value of 1 or 0 in latter.

25 is number of neurons in first layer and input_dim is number of my data points, therefore technically it works, yet I'm not sure whether it makes sense to do so or I misunderstood concept of neurons in input layer and what they do.

Sljder
  • 148
  • 1
  • 9
  • 3
    Are you sure you have your terminology straight? The reason you should have exactly the number of nodes in the input layer as you have data points is that the data enters the network through being fed to the input layer, one variable per node. I don’t understand how one would put 50 values into 25 nodes..? – Amadan Jan 15 '18 at 13:32
  • I'm using Keras with tensorflow back-end in following manner:
    model = Sequential()
    model.add(Dense(25, input_dim=50, activation='relu'))
    model.add(Dense(8, activation='relu'))
    model.add(Dense(1, activation='sigmoid'))
    As I understand from documentation 25 is number of neurons while input_dim is number of data points, therefore it works, yet I'm not sure if corretly.
    – Sljder Jan 15 '18 at 14:08
  • Show us your code, I’m almost certain you’re not doing what you think you are. – Amadan Jan 15 '18 at 14:13
  • I've updated question with more details. – Sljder Jan 15 '18 at 14:22
  • 3
    Your input layer is 50 nodes. Your first hidden layer is 25. Your second hidden layer is 8. Your output layer is 1 node. The input layer does not get explicitly added to the model, it is implicitly shaped in `input_dim` parameter. How many nodes go in hidden layers is a bit of an art, but the basic starting point is usually the average between the number of input and output nodes, which would actually be 25ish in your case, so your code is definitely not too weird; you just didn't connect what's going on in your code to what the mathematical concepts are correctly. – Amadan Jan 15 '18 at 15:15
  • Great response! Thanks! – Sljder Jan 15 '18 at 15:59
  • 1
    This is a common point of confusion for new users - see [Keras Sequential model input layer](https://stackoverflow.com/questions/46572674/keras-sequential-model-input-layer/46574294#46574294) – desertnaut Jan 15 '18 at 18:42

0 Answers0