1

I am trying to create a custom generator. I saw in this question Custom Keras Data Generator with yield how no while loop is necessary because they use the sequence API. I am not using a sequence API but my code looks very similar to the code mention. Do I need a while loop because I have not called the sequence API. Also I am using yield to return the image.

import os
import numpy 
import cv2
class image_gen():
    def __init__(self, sdir, batch_size, shuffle):          
        self.batch_index=0  
        self.sdir=sdir # directory containing input images
        self.batch_size=batch_size   #batch size is number of samples in a batch
          # tuple (width, height)  for target image 
        self.shuffle=shuffle   # set to True to shuffle images, False for no shuffle
         # initialize list to hold sequential list of total labels generated
        self.image_list=[] # initialize list to hold sequential list of total images filenames generated
        self.s_list=os.listdir(self.sdir)   # list of images in directory      
        
    def get_images(self):    # gets a batch of input images, resizes input image to make target images        
        input_image_batch=[] # initialize list to hold a batch of target images
         # initialize list to hold batches of  input images 
        sample_count=len(self.s_list)  # determine total number of images available         
        for i in range(self.batch_index * self.batch_size, (self.batch_index + 1) * self.batch_size  ): #iterate for  a batch
            j=i % sample_count # cycle j value over range of available  images
            k=j % self.batch_size  # cycle k value over batch size
            if self.shuffle: # if shuffle select a random integer between 0 and sample_count-1 to pick as the image=label pair
                m=np.random.randint(low=0, high=sample_count-1, size=None, dtype=int) 
            else:
                m=j   # no shuffle         
            path_to_img=os.path.join(self.sdir, self.s_list[m]) # define the path to the m th image 
            input_image=cv2.imread(path_to_img)
            
             #create the target image from the input image            
            input_image_batch.append(input_image)        
        input_image_array=numpy.array((input_image_batch))
        
        self.batch_index=self.batch_index + 1         
        yield (input_image_array)
Grysik
  • 807
  • 7
  • 16
user123
  • 415
  • 2
  • 4
  • 17
  • Please read docs about generators, it seems to me that you don't understand them. Generators do what's in code, until yield statement comes. When they are called next time, they start from next line after yield. If you have nothing after yield, then your generator returns. If you want to operate with batches in keras you need to somehow move from one batch to another, e.g. `while self.batch_index * self.batch_size < sample_count`, so after generator is called next time your loop starts over. – Grysik Sep 17 '21 at 09:40
  • Okay i understood what you mean.so for example if I put while True when my get_images() then that should return and start the loop again until the end of the files right? – user123 Sep 17 '21 at 10:02
  • if you put `while True` how would python know when is end of your files? – Grysik Sep 17 '21 at 10:10
  • i got that idea from this post https://towardsdatascience.com/writing-custom-keras-generators-fe815d992c5a. how do you suggest i loop it over till the file end. – user123 Sep 17 '21 at 10:20
  • I've described it earlier, put while condition which is true until you reach end of data in your batch. You might used `while True` and when you reach end start from 0, depending how you use it (so e.g. you can go through image set few times, if that makes sense) – Grysik Sep 18 '21 at 06:53

0 Answers0