0

I am trying to train a resnet50 model with EMNIST data which is a dataset containing 300k images of letters and numbers. Resnet50 requires 3 dimensional images as its input and not grayscale so i tried to convert all the grayscale images to RGB but it isnt working like I want it too. When i view them using pyplot.imshow, the RGB image is really different from the grayscale one which is queer because these commands are actually just copy pasting the same grayscale matrix in 3 dimensions.

The 3 commands which I have tried are given below:

> resizedImageRGB = cv2.cvtColor(resizedImage,cv2.COLOR_GRAY2RGB)
> resizedImageRGB = np.repeat(resizedImage[:,:,np.newaxis],3,-1) arr =
> np.expand_dims(resizedImage, axis=2) resizedImageRGB =
> np.concatenate((arr,arr,arr), axis=2)

The grayscale and RGB image of one of the letters are given respectively:

GrayScale Image

RGB Image

Usman Abdur Rehman
  • 334
  • 1
  • 3
  • 13
  • This is hard mathematically. Consider from RGB to Gray: Y = 0.299R + 0.587G + 0.114B. You could imagine that going back requires too much unknown information (1 equation 3 unknowns). People use deep neural networks for such tasks.. Rather what you should do, is change the ResNET input to accept grayscale. not the other way around – WiseDev Jun 20 '19 at 14:34
  • @BlueRineS I would love to do that but from what i have read, resnet's layers already have weights on them so removing the input layer to add my own which accepts grayscale image would affect its performance. Idk I havent tried it yet, maybe it doesnt – Usman Abdur Rehman Jun 20 '19 at 14:43
  • Possible duplicate of [How does one convert a grayscale image to RGB in OpenCV (Python) for visualizing contours after processing an image in binary?](https://stackoverflow.com/questions/21596281/how-does-one-convert-a-grayscale-image-to-rgb-in-opencv-python-for-visualizing) –  Jun 20 '19 at 14:43
  • @BlueRineS Or i could just design my own model but i dont have time. This is part of my FInal year project and it has to be submitted in 3 weeks. Idk if I could do that. Make models, test them, add some layers, remove some layers. One of my teachers said that its best that you use a pretrained model and just train your dataset with it. That would be less time consuming – Usman Abdur Rehman Jun 20 '19 at 14:45
  • See my answer below. Of course, to answer optimally for your case, more info would be required, but it should be enough from a general point of view. – WiseDev Jun 20 '19 at 14:46
  • 1
    @UsmanAbdurRehman -- the "grayscale" image in your link is already and RGB png file. Why not just open it as such? – Mark Jun 20 '19 at 14:54
  • @MarkMeyer nooo that is actually the image i saved of the image from pyplot.imshow – Usman Abdur Rehman Jun 20 '19 at 17:36
  • @UsmanAbdurRehman the image behind the link `GrayScale Image` is an RGB image — that makes your question a little confusing. You should consider editing the question to link to the correct image if that's wrong. – Mark Jun 20 '19 at 17:50
  • @MarkMeyer these pics are the screenshot i took by plotting them using pyplot.imshow. The actual images are in the form of vectors stored in csv files – Usman Abdur Rehman Jun 21 '19 at 05:14

1 Answers1

-1

Going from Grayscale to an RGB approximation is hard mathematically. Consider (one of) the formula(s) for going from RGB to grayvalue Y:

Y = 0.299R + 0.587G + 0.114B

Now you can imagine that going in the other direction, and attempting to derive R, G and B values from Y, well.. requires too much information (1 eq. 3 unknown). People actually use Neural Networks for this stuff..

Rather, the right approach for you is to do it the other way around. That is, if you have access of only grayscale data (or 1-channeled data for that matter) you should modify your network such that it accepts the right input.

I'm not sure what libraries or exact code you are using from your question, but in general this shouldn't be too hard.

Usually the code that you find online has function that create these nets for you with the right input arguments supplied.

def ResNET(shape=(256,256,3), ...):
    some_code()

then you can usually just pass your own input:

net = ResNET(shape=(256,256,1))

Hope this helps.

WiseDev
  • 524
  • 4
  • 17
  • I am importing resnet from Keras using from keras.applications.resnet50 import ResNet50 and then have to initialize it with the input shape of the model like this: model = ResNet50(weights= None, include_top=False, input_shape = (32,32,3)) – Usman Abdur Rehman Jun 20 '19 at 14:46
  • I believe that should be an easy task when you use keras code. I image the original network is defined like in a function as stated above in my answer. Try to supply your input_shape if that is an argument, or simply change the input_layer by hand in the code (few seconds work) – WiseDev Jun 20 '19 at 14:48
  • Thanks I will try that :) – Usman Abdur Rehman Jun 20 '19 at 14:48
  • Great! Let me know if it helps, and if it does, consider upvoting my answer :) – WiseDev Jun 20 '19 at 14:49
  • This does not work, we are not allowed to change CNN input architecture for pretrained model. Reference: https://stackoverflow.com/a/51996037/9162147 – DataFramed Feb 07 '20 at 11:53
  • Who said anything about pretrained models? – WiseDev Feb 07 '20 at 12:10