I am training an LSTM autoencoder model in python using Keras using only CPU.
I can see that there is an argument called use_multiprocessing in the fit function. Could you please explain in simple terms what does this argument do exactly. I read the explanation on tensorflow.org but I cannot understand from it if I set the parameter to true how would my model be impacted. I am looking for ways to speed up the training of my model and I am wondering if this parameter would help.
The use_multiprocessing (and workers and max_queue_size) parameters apply to the batch data generation. The clue in the documentation is this: "Used for generator or keras.utils.Sequence input only" [ref https://keras.io/api/models/model_training_apis/#fit-method]. Deep under the hood, keras uses an orderedenqueuer to wrap your input.
If use_multiprocessing is True and workers > 0, then keras will create multiple (number = workers) processes to run simultaneously and prepare batches from your generator/sequence. They will try to keep the queue of batches ready for training up to max_queue_size.
If use_multiprocessing is False and workers > 1, then keras will create multiple (number = workers) threads to simultaneously prepare batches, similar to above (but your input data object needs to be thread safe).
If your batch data generation is the bottleneck in your training process, this can speed things up a lot. I have found use_multiprocessing True to speed up batch data bound work linearly e.g. 2 workers = 2x as fast (though there is overhead to start the processes). For use_multiprocessing False and threads, I have found an unpredictable 0%-15% speed increase, without overhead.
Refer also this question with lots of details: How to define max_queue_size, workers and use_multiprocessing in keras fit_generator()?
