7

I build a image classification model in R by keras for R.

Got about 98% accuracy, while got terrible accuracy in python.

Keras version for R is 2.1.3, and 2.1.5 in python

following is the R model code:

model=keras_model_sequential()
model=model %>% 
  layer_conv_2d(filters = 32,kernel_size = c(3,3),padding = 'same',input_shape = c(187,256,3),activation = 'elu')%>%
  layer_max_pooling_2d(pool_size = c(2,2)) %>%
  layer_dropout(.25) %>% layer_batch_normalization() %>%
  layer_conv_2d(filters = 64,kernel_size = c(3,3),padding = 'same',activation = 'relu') %>%
  layer_max_pooling_2d(pool_size = c(2,2)) %>%
  layer_dropout(.25) %>% layer_batch_normalization() %>% layer_flatten() %>%
  layer_dense(128,activation = 'relu') %>%
  layer_dropout(.25)%>%
  layer_batch_normalization() %>%
  layer_dense(6,activation = 'softmax')


model %>%compile(
  loss='categorical_crossentropy',
  optimizer='adam',
  metrics='accuracy'
)

I try to rebuild a same model in python, with same input data.

While, got totally different performance. The accuracy even less than 30%

Because R keras is calling python for run keras. With same model architecture, they should get similar performance.

I wonder if this issue caused by preprocess, but still show my python code:

model=Sequential()
model.add(Conv2D(32,kernel_size=(3,3),activation='relu',input_shape=(187,256,3),padding='same'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(BatchNormalization())
model.add(Conv2D(64, (3, 3), activation='relu',padding='same'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(BatchNormalization())
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.25))
model.add(BatchNormalization())
model.add(Dense(len(label[1]), activation='softmax'))

model.compile(loss='categorical_crossentropy',
              optimizer='adam',
              metrics=['accuracy'])

This is a simple classification. I just do as same as most instruction.

Can not find others faced same problem. So want to ask how it happen and how to solve. Thx

Jacky Tsai
  • 129
  • 5
  • 1
    Is the data a public data set? – Ahmed Fasih Apr 18 '18 at 04:04
  • No, it's my private data set – Jacky Tsai Apr 18 '18 at 05:23
  • Why did you use `elu` in R but `relu` in Python? – abarnert Apr 18 '18 at 05:40
  • that's the small mistake when I write this question. I've tried these two activation function. Their performance are basically same. – Jacky Tsai Apr 18 '18 at 06:45
  • 1
    Understood. Can you replicate the problem with a public dataset? That would help us narrow down the problem. – Ahmed Fasih Apr 18 '18 at 11:05
  • 1
    I try cifar10 or mnist and do well in these jobs. So I'm not sure how to replicate the problem. The only difference between R and python model is the preprocess way. I've tried many different resize tools, but can't get same outcome as R. – Jacky Tsai Apr 19 '18 at 07:10
  • If you think that preprocessing is the source of the issue then you can validate that by visualising some of the preprocessed results and make sure it's consistent across R and Python. – Vikash Singh May 04 '18 at 11:43

1 Answers1

3

That is a dramatic difference so perhaps there's a bug in the code or something unexpected in the data but reproducing Keras results from R in Python is more difficult than it may seem since setting the seed on the R side is insufficient. Instead of set.seed you should use use_session_with_seed, which comes with the R libraries for tensorflow and keras. Note that for full reproducibility you need to use_session_with_seed(..., disable_gpu=TRUE, disable_parallel_cpu=TRUE). See also stack and tf docs. Also, here is an example using the github version of kerasformula and a public dataset. Also, watch out for functions like layer_dropout that accept seed as a parameter.

mohanty
  • 78
  • 7