4

I am a beginner training an image dataset on diabetic retinopathy, using the keras_flow_from_dataframe class. But my model has been underfitting. So I tried preprocessing, by writing a custom preprocessing function to be passed in my image data generator class, using OpenCV's adaptive thresholding implementation. The function works very well when I use it outside of Keras, but when I add it to my image data generator class and fit my model, it returns a type error saying bad argument type for built-in operation before my first epoch starts.

Here's the preprocessing code:

def preprocess(im):

    im = cv2.imread(im, 1)
    im= cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
    im=cv2.resize(im, (300,300))
    im.resize(300, 300, 1)
    block_size = 73 
    constant = 2    
# ADAPTIVE GAUSSIAN THRESHOLDING

    thr2 = cv2.adaptiveThreshold(im, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, block_size, constant)
    return thr2

It runs well outside of Keras when I test it with the images in my dataframe, but when I add it to my image data generator class, it throws an error.

train_datagen = ImageDataGenerator(
    rotation_range=30,
    width_shift_range=0.4,
    height_shift_range=0.4,
    shear_range=0.3,
    zoom_range=0.3,
    horizontal_flip = True,
    fill_mode='nearest',
    preprocessing_function = preprocess)

valid_datagen = ImageDataGenerator(preprocessing_function = preprocess)

Then I load in my dataset from dataframe:

from keras.preprocessing.image import ImageDataGenerator

traingen = train_datagen.flow_from_dataframe(x_train, x_col='path', y_col='level',class_mode='other', 
                                             target_size=(300,300), color_mode='grayscale', batch_size=16)

validgen = valid_datagen.flow_from_dataframe(valid, x_col='path', y_col='level',class_mode='other',
                                            target_size=(300,300), color_mode='grayscale', batch_size=16)

Then I fit the model using model.fit_generator, which then throws me the type error: bad argument type for built-in operation.

TypeError                                 Traceback (most recent call last)
<ipython-input-126-30ceb84a2574> in <module>()
      2 
      3 history = model.fit_generator(traingen, validation_data = validgen, epochs=100, steps_per_epoch=10,
----> 4                                   validation_steps=10, verbose=1, callbacks=[lr_reduction])
      5 
      6 

~/var/python/lib/python3.6/site-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
     89                 warnings.warn('Update your `' + object_name + '` call to the ' +
     90                               'Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
     92         wrapper._original_function = func
     93         return wrapper

~/var/python/lib/python3.6/site-packages/keras/engine/training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
   1416             use_multiprocessing=use_multiprocessing,
   1417             shuffle=shuffle,
-> 1418             initial_epoch=initial_epoch)
   1419 
   1420     @interfaces.legacy_generator_methods_support

~/var/python/lib/python3.6/site-packages/keras/engine/training_generator.py in fit_generator(model, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
    179             batch_index = 0
    180             while steps_done < steps_per_epoch:
--> 181                 generator_output = next(output_generator)
    182 
    183                 if not hasattr(generator_output, '__len__'):

~/var/python/lib/python3.6/site-packages/keras/utils/data_utils.py in get(self)
    599         except Exception as e:
    600             self.stop()
--> 601             six.reraise(*sys.exc_info())
    602 
    603 

~/var/python/lib/python3.6/site-packages/six.py in reraise(tp, value, tb)
    691             if value.__traceback__ is not tb:
    692                 raise value.with_traceback(tb)
--> 693             raise value
    694         finally:
    695             value = None

~/var/python/lib/python3.6/site-packages/keras/utils/data_utils.py in get(self)
    593         try:
    594             while self.is_running():
--> 595                 inputs = self.queue.get(block=True).get()
    596                 self.queue.task_done()
    597                 if inputs is not None:

~/var/python/lib/python3.6/multiprocessing/pool.py in get(self, timeout)
    642             return self._value
    643         else:
--> 644             raise self._value
    645 
    646     def _set(self, i, obj):

~/var/python/lib/python3.6/multiprocessing/pool.py in worker(inqueue, outqueue, initializer, initargs, maxtasks, wrap_exception)
    117         job, i, func, args, kwds = task
    118         try:
--> 119             result = (True, func(*args, **kwds))
    120         except Exception as e:
    121             if wrap_exception and func is not _helper_reraises_exception:

~/var/python/lib/python3.6/site-packages/keras/utils/data_utils.py in get_index(uid, i)
    399         The value at index `i`.
    400     """
--> 401     return _SHARED_SEQUENCES[uid][i]
    402 
    403 

~/var/python/lib/python3.6/site-packages/keras_preprocessing/image/iterator.py in __getitem__(self, idx)
     63         index_array = self.index_array[self.batch_size * idx:
     64                                        self.batch_size * (idx + 1)]
---> 65         return self._get_batches_of_transformed_samples(index_array)
     66 
     67     def __len__(self):

~/var/python/lib/python3.6/site-packages/keras_preprocessing/image/iterator.py in _get_batches_of_transformed_samples(self, index_array)
    233                 params = self.image_data_generator.get_random_transform(x.shape)
    234                 x = self.image_data_generator.apply_transform(x, params)
--> 235                 x = self.image_data_generator.standardize(x)
    236             batch_x[i] = x
    237         # optionally save augmented images to disk for debugging purposes

~/var/python/lib/python3.6/site-packages/keras_preprocessing/image/image_data_generator.py in standardize(self, x)
    695         """
    696         if self.preprocessing_function:
--> 697             x = self.preprocessing_function(x)
    698         if self.rescale:
    699             x *= self.rescale

<ipython-input-112-7bddefa5e731> in preprocess(im)
      1 def preprocess(im):
----> 2     im = cv2.imread(im, 1)
      3     im= cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
      4     im=cv2.resize(im, (300,300))
      5     im.resize(300, 300, 1)

TypeError: bad argument type for built-in operation


TypeError                                 Traceback (most recent call last)
<ipython-input-126-30ceb84a2574> in <module>()
      2 
      3 history = model.fit_generator(traingen, validation_data = validgen, epochs=100, steps_per_epoch=10,
----> 4                                   validation_steps=10, verbose=1, callbacks=[lr_reduction])
      5 
      6 

~/var/python/lib/python3.6/site-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
     89                 warnings.warn('Update your `' + object_name + '` call to the ' +
     90                               'Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
     92         wrapper._original_function = func
     93         return wrapper

~/var/python/lib/python3.6/site-packages/keras/engine/training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
   1416             use_multiprocessing=use_multiprocessing,
   1417             shuffle=shuffle,
-> 1418             initial_epoch=initial_epoch)
   1419 
   1420     @interfaces.legacy_generator_methods_support

~/var/python/lib/python3.6/site-packages/keras/engine/training_generator.py in fit_generator(model, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
    179             batch_index = 0
    180             while steps_done < steps_per_epoch:
--> 181                 generator_output = next(output_generator)
    182 
    183                 if not hasattr(generator_output, '__len__'):

~/var/python/lib/python3.6/site-packages/keras/utils/data_utils.py in get(self)
    599         except Exception as e:
    600             self.stop()
--> 601             six.reraise(*sys.exc_info())
    602 
    603 

~/var/python/lib/python3.6/site-packages/six.py in reraise(tp, value, tb)
    691             if value.__traceback__ is not tb:
    692                 raise value.with_traceback(tb)
--> 693             raise value
    694         finally:
    695             value = None

~/var/python/lib/python3.6/site-packages/keras/utils/data_utils.py in get(self)
    593         try:
    594             while self.is_running():
--> 595                 inputs = self.queue.get(block=True).get()
    596                 self.queue.task_done()
    597                 if inputs is not None:

~/var/python/lib/python3.6/multiprocessing/pool.py in get(self, timeout)
    642             return self._value
    643         else:
--> 644             raise self._value
    645 
    646     def _set(self, i, obj):

~/var/python/lib/python3.6/multiprocessing/pool.py in worker(inqueue, outqueue, initializer, initargs, maxtasks, wrap_exception)
    117         job, i, func, args, kwds = task
    118         try:
--> 119             result = (True, func(*args, **kwds))
    120         except Exception as e:
    121             if wrap_exception and func is not _helper_reraises_exception:

~/var/python/lib/python3.6/site-packages/keras/utils/data_utils.py in get_index(uid, i)
    399         The value at index `i`.
    400     """
--> 401     return _SHARED_SEQUENCES[uid][i]
    402 
    403 

~/var/python/lib/python3.6/site-packages/keras_preprocessing/image/iterator.py in __getitem__(self, idx)
     63         index_array = self.index_array[self.batch_size * idx:
     64                                        self.batch_size * (idx + 1)]
---> 65         return self._get_batches_of_transformed_samples(index_array)
     66 
     67     def __len__(self):

~/var/python/lib/python3.6/site-packages/keras_preprocessing/image/iterator.py in _get_batches_of_transformed_samples(self, index_array)
    233                 params = self.image_data_generator.get_random_transform(x.shape)
    234                 x = self.image_data_generator.apply_transform(x, params)
--> 235                 x = self.image_data_generator.standardize(x)
    236             batch_x[i] = x
    237         # optionally save augmented images to disk for debugging purposes

~/var/python/lib/python3.6/site-packages/keras_preprocessing/image/image_data_generator.py in standardize(self, x)
    695         """
    696         if self.preprocessing_function:
--> 697             x = self.preprocessing_function(x)
    698         if self.rescale:
    699             x *= self.rescale

<ipython-input-112-7bddefa5e731> in preprocess(im)
      1 def preprocess(im):
----> 2     im = cv2.imread(im, 1)
      3     im= cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
      4     im=cv2.resize(im, (300,300))
      5     im.resize(300, 300, 1)

TypeError: bad argument type for built-in operation


TypeError                                 Traceback (most recent call last)
<ipython-input-126-30ceb84a2574> in <module>()
      2 
      3 history = model.fit_generator(traingen, validation_data = validgen, epochs=100, steps_per_epoch=10,
----> 4                                   validation_steps=10, verbose=1, callbacks=[lr_reduction])
      5 
      6 

~/var/python/lib/python3.6/site-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
     89                 warnings.warn('Update your `' + object_name + '` call to the ' +
     90                               'Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
     92         wrapper._original_function = func
     93         return wrapper

~/var/python/lib/python3.6/site-packages/keras/engine/training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
   1416             use_multiprocessing=use_multiprocessing,
   1417             shuffle=shuffle,
-> 1418             initial_epoch=initial_epoch)
   1419 
   1420     @interfaces.legacy_generator_methods_support

~/var/python/lib/python3.6/site-packages/keras/engine/training_generator.py in fit_generator(model, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
    179             batch_index = 0
    180             while steps_done < steps_per_epoch:
--> 181                 generator_output = next(output_generator)
    182 
    183                 if not hasattr(generator_output, '__len__'):

~/var/python/lib/python3.6/site-packages/keras/utils/data_utils.py in get(self)
    599         except Exception as e:
    600             self.stop()
--> 601             six.reraise(*sys.exc_info())
    602 
    603 

~/var/python/lib/python3.6/site-packages/six.py in reraise(tp, value, tb)
    691             if value.__traceback__ is not tb:
    692                 raise value.with_traceback(tb)
--> 693             raise value
    694         finally:
    695             value = None

~/var/python/lib/python3.6/site-packages/keras/utils/data_utils.py in get(self)
    593         try:
    594             while self.is_running():
--> 595                 inputs = self.queue.get(block=True).get()
    596                 self.queue.task_done()
    597                 if inputs is not None:

~/var/python/lib/python3.6/multiprocessing/pool.py in get(self, timeout)
    642             return self._value
    643         else:
--> 644             raise self._value
    645 
    646     def _set(self, i, obj):

~/var/python/lib/python3.6/multiprocessing/pool.py in worker(inqueue, outqueue, initializer, initargs, maxtasks, wrap_exception)
    117         job, i, func, args, kwds = task
    118         try:
--> 119             result = (True, func(*args, **kwds))
    120         except Exception as e:
    121             if wrap_exception and func is not _helper_reraises_exception:

~/var/python/lib/python3.6/site-packages/keras/utils/data_utils.py in get_index(uid, i)
    399         The value at index `i`.
    400     """
--> 401     return _SHARED_SEQUENCES[uid][i]
    402 
    403 

~/var/python/lib/python3.6/site-packages/keras_preprocessing/image/iterator.py in __getitem__(self, idx)
     63         index_array = self.index_array[self.batch_size * idx:
     64                                        self.batch_size * (idx + 1)]
---> 65         return self._get_batches_of_transformed_samples(index_array)
     66 
     67     def __len__(self):

~/var/python/lib/python3.6/site-packages/keras_preprocessing/image/iterator.py in _get_batches_of_transformed_samples(self, index_array)
    233                 params = self.image_data_generator.get_random_transform(x.shape)
    234                 x = self.image_data_generator.apply_transform(x, params)
--> 235                 x = self.image_data_generator.standardize(x)
    236             batch_x[i] = x
    237         # optionally save augmented images to disk for debugging purposes

~/var/python/lib/python3.6/site-packages/keras_preprocessing/image/image_data_generator.py in standardize(self, x)
    695         """
    696         if self.preprocessing_function:
--> 697             x = self.preprocessing_function(x)
    698         if self.rescale:
    699             x *= self.rescale

<ipython-input-112-7bddefa5e731> in preprocess(im)
      1 def preprocess(im):
----> 2     im = cv2.imread(im, 1)
      3     im= cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
      4     im=cv2.resize(im, (300,300))
      5     im.resize(300, 300, 1)

TypeError: bad argument type for built-in operation

I have also thought about preprocessing the images and saving them to a folder then I'll load them from that folder to a dataframe, but it's computationally expensive and time consuming.

Georgy
  • 12,464
  • 7
  • 65
  • 73
Victor E. Irekponor
  • 53
  • 1
  • 10

2 Answers2

3

I ran into a problem like yours and my teacher helped me by pointing it to the docs of the tf preprocess_function, it said the preprocess_function argument is an image, you can read more on this.

That is why it gives you the error at cv2.imread(image). You should remove that line because im is an image that the generator gives you. There no need to load it because it's already loaded

My one work well, hope yours will be fine too.

Exelian
  • 5,749
  • 1
  • 30
  • 49
Hòa Bùi Ngọc
  • 31
  • 2
  • I updated your answer to clarify it and added syntax highlighting making it clearer what you're saying. – Exelian Sep 23 '20 at 11:57
0

I think the issue was with Keras parsing the openCV output, Because it ran well when i used another library to perform the processing called ImgAUg. Here's the link. https://pypi.org/project/imgaug/.

So i just scripted a preprocessing function with the library, and then i passed it into the keras imageDataGenerator class. It ran fine and didn't throw any error at me.

Victor E. Irekponor
  • 53
  • 1
  • 10