how to enable keras fit() multiprocessing properly?

Question

when I run fit() with multiprocessing=True i always get a deadlock and the following warning:

WARNING:tensorflow:multiprocessing can interact badly with TensorFlow, causing nondeterministic deadlocks. For high performance data pipelines tf.data is recommended.

how to run it properly?

Since it says "tf.data", i wonder if transforming my data into this format will make multiprocessing work. What specifically is meant/how to convert it?

my dataset: (reproducable)

Input_shape, labels =(20,4), 6
LEN_X.LEN_Y = 20000.3000 
train_X,train_Y = np.asarray([np.random.random(Input_shape) for x in range(LEN_X )]), np.random.random((LEN_X ,labels))
validation_X,validation_Y = np.asarray([np.random.random(Input_shape) for x in range(LEN_Y)]), np.random.random((LEN_Y,labels))
sampleW = np.random.random((LEN_X ,1)) 

Answer 1

The multiprocessing doesn't accelerate the model itself. It only accelerates the data loading. And data loading delay is not a problem when all your data is already in-memory.

You could still use multiprocessing, however, but you must make sure that the underlying dataset is thread-safe and you have to carefully craft the data pipeline. That is quite time consuming. So, instead I suggest you speed up the model itself.

For that, you should look into:

• changing all except last layer activations to RELU.
• tweaking batch size. (optimal number depends on your hardware, and is almost always less than or equal to 32)
• using Batch normalization to speed up convergence.
• using higher learning rate (be careful not to overdo this step).
• if you need faster convolutions, consider using Kaggle notebooks or vast.ai for GPU-enabled computations.
• last but not least, try using a simpler, smaller model.

---

Comment down here if you have any additional questions.
Cheers.