14

In TensorFlow's Dataset API, we can use dataset.prefetch(buffer_size=xxx) to preload other batches' data while GPU is processing the current batch's data, therefore, I can make full use of GPU.

I'm going to use Keras, and wonder if keras has a similar API for me to make full use of GPU, instead of serial execution: read batch 0->process batch 0->read batch 1-> process batch 1-> ...

I briefly looked through the keras API and did not see a description of the prefetch.

zimmerrol
  • 4,872
  • 3
  • 22
  • 41
Shouyu Chen
  • 655
  • 8
  • 16

1 Answers1

7

If you call fit_generator with workers > 1, use_multiprocessing=True, it will prefetch queue_size batches.

From docs: max_queue_size: Integer. Maximum size for the generator queue. If unspecified, max_queue_size will default to 10.

Rocketq
  • 5,423
  • 23
  • 75
  • 126
  • This answer seems to be right according to https://stackoverflow.com/questions/36986815/what-is-the-parameter-max-q-size-used-for-in-model-fit-generator – Matias Haeussler Jun 23 '19 at 15:22
  • 3
    From tensorflow 2.2.0 onward, `fit` now has arguments for `workers` and `use_multiprocessing`. I think `fit_generator` is now legacy according to [this tutorial](https://www.pyimagesearch.com/2018/12/24/how-to-use-keras-fit-and-fit_generator-a-hands-on-tutorial/) on pyimagesearch. – aliassaila Oct 27 '20 at 10:55