7
dataset = dataset.batch(50)
dataset = dataset.prefetch(buffer_size=1)

Is it prefetch 1 batch or 1 element?

Per the API document in tensorflow, the buffer_size is the max num of elements prefetch. But it seems it is num of batch after batching the dataset.

CertainPerformance
  • 356,069
  • 52
  • 309
  • 320
Yik Wai Ng
  • 71
  • 1
  • 2
  • https://stackoverflow.com/questions/46444018/meaning-of-buffer-size-in-dataset-map-dataset-prefetch-and-dataset-shuffle – Deepali Aug 02 '18 at 05:51
  • but this post answer didn't clarify whether an element means a batch(if we call .batch() before the prefetch call) or a sample. – Yik Wai Ng Aug 02 '18 at 07:31
  • https://stackoverflow.com/questions/49707062/tensorflow-dataset-api-dataset-batchn-prefetchm-prefetches-m-batches-or-sam – Deepali Aug 02 '18 at 08:37
  • Possible duplicate of [Tensorflow Dataset API: dataset.batch(n).prefetch(m) prefetches m batches or samples?](https://stackoverflow.com/questions/49707062/tensorflow-dataset-api-dataset-batchn-prefetchm-prefetches-m-batches-or-sam) – mrry Aug 06 '18 at 15:27
  • yes, it got answered. – Yik Wai Ng Aug 07 '18 at 16:34

1 Answers1

8

Since you are using dataset.prefetch(buffer_size=1) after dataset.batch(), it means that it will prefetch 1 batch.

Djib2011
  • 6,874
  • 5
  • 36
  • 41