11

So this question might stem from a lack of knowledge about tensorflow. But I am trying to build a multilayer perceptron with tensorflow 2.0, but without Keras.

The reason being that it is a requirement for my machine learning course that we do not use keras. Why you might ask? I am not sure.

I already have implemented our model in tensorflow 2.0 with Keras ease, and now I want to do the exact same thing without keras.

model = Sequential()
model.add(Dense(64, activation='relu', input_dim=784))
model.add(Dropout(0.5))
model.add(Dense(64, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(5, activation='softmax'))

sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='categorical_crossentropy',
              optimizer=Adam(),
              metrics=['accuracy'])

X_train = X[:7000]
y_train = tf.keras.utils.to_categorical(y[:7000], num_classes=5)
X_dev = X[7000:]
y_dev = tf.keras.utils.to_categorical(y[7000:], num_classes=5)

model.fit(X_train, y_train,
          epochs=100,
          batch_size=128)
score = model.evaluate(X_dev, y_dev, batch_size=128)
print(score)

Here is my problem. Whenever I look up the documentation on Tensorflow 2.0, then even the guides on custom training are using Keras.

As placeholders and sessions are a thing of the past in tensorflow 2.0, as I understand it, then I am a bit unsure of how to structure it.

I can make tensor objects. I have the impression that I need to use eager execution and use gradient tape. But I still am unsure of how to put these things together.

Now my question is. Where should I look to get a better understanding? Which direction has the greatest descent?

Please do tell me if I am doing this stack overflow post wrong. It is my first time here.

Innat
  • 16,113
  • 6
  • 53
  • 101
Anders Stendevad
  • 111
  • 1
  • 3
  • This is the closest I have been to a real explaination of making a model without keras. https://www.tensorflow.org/tutorials/customization/custom_training_walkthrough#define_the_loss_and_gradient_function – Anders Stendevad Nov 19 '19 at 12:10
  • Is the idea to make a custom training loop or does the teacher want you to avoid layers and create the weights yourself? – Daniel Möller Nov 19 '19 at 12:20
  • There used to be tensorflow layers in the past (not keras layers), maybe you could use them plus the custom training loop. – Daniel Möller Nov 19 '19 at 12:20
  • Custom training loops: https://www.tensorflow.org/tutorials/customization/custom_training_walkthrough – Daniel Möller Nov 19 '19 at 12:21
  • Does the course instructor know that nowadays keras is built into tensorflow and is an integral part of most model-building? – The Guy with The Hat Nov 19 '19 at 14:37
  • I have now gone as far as to make the custom training loop. Though I still rely on a Keras model. ex model = tf.keras.Sequential() Thanks Daniel for the link! – Anders Stendevad Nov 20 '19 at 14:04
  • He sadly does not. He is using python 2.7 and tensorflow 1.14. Not that this is a problem, but this is the reason why he stated keras was not to be used :/ – Anders Stendevad Nov 20 '19 at 14:08

1 Answers1

0

As @Daniel Möller stated, there are these tutorials for custom training and custom layers on the official TensorFlow page. As stated on the custom training page:

This tutorial used tf.Variable to build and train a simple linear model.

There is also this blog that creates custom layers and training without Keras API. You can check this code on Google Colab, which uses Cifar-10 with custom layers and training in the same manner.

rvimieiro
  • 1,043
  • 1
  • 10
  • 17