IndexError in py_call_impl

Question

I am a master student, currently enrolled in an R project course. While setting up a deep learning model, I came across some issues. I would appreciate if somebody could help me. I reinstalled R, RStudio, Python, Anaconda, but couldn't resolve the issue.

library (keras);library(tensorflow);
install_tensorflow(); install_keras()
library(ggplot2); library(magrittr);
library(dplyr); library(tm); library(tidyr); 



clothing_reviews <- read.csv("C:/Users/Astrid/Documents/Master BWL/Data Mining mit R/R/Präsentation 2/Womens Clothing Reviews.csv") %>%
+   mutate(Liked = ifelse(Rating == 5, 1, 0),
+          text = paste("Title", "Review.Text"),
+          text = gsub("NA", "", text))
> 

> glimpse(clothing_reviews)
Observations: 28,222
Variables: 13
$ X                             <fct> "0", "1,1080,34,,\"Love this dress!  it's sooo pretty.  i happened to fi...
$ Clothing.ID                   <fct> 767, , , , 847, , , , 1077, , 1077, 1095, , ,  so i would say it's runni...
$ Age                           <fct> 33, , , , 47, , , , 24, , 53, 39, , , , , , 50, , , 41, , , , , , , , , ...
$ Title                         <fct> , , , , Flattering shirt, , , , Flattering, , Dress looks like it's made...
$ Review.Text                   <fct> Absolutely wonderful - silky and sexy and comfortable, , , , This shirt ...
$ Rating                        <fct> 4, , , , 5, , , , 5, , 3, 5, , , , , , 3, , , 5, , , , , , , , , , , , ,...
$ Recommended.IND               <fct> 1, , , , 1, , , , 1, , 0, 1, , , , , , 1, , , 1, , , , , , , , , , , , ,...
$ Positive.Feedback.Count       <fct> 0, , , , 6, , , , 0, , 14, 2, , , , , , 1, , , 0, , , , , , , , , , , , ...
$ Division.Name                 <fct> Initmates, , , , General, , , , General, , General, General Petite, , , ...
$ Department.Name               <fct> Intimate, , , , Tops, , , , Dresses, , Dresses, Dresses, , , , , , Dress...
$ Class.Name................... <fct> Intimates;;;;;;;;;;;;;;;;;;;, , , , Blouses;;;;;;;;;;;;;;;;;;;, , , , Dr...
$ Liked                         <dbl> 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, ...
$ text                          <chr> "Title Review.Text", "Title Review.Text", "Title Review.Text", "Title Re...
> 

> clothing_reviews %>%
+   ggplot(aes(x = factor(Liked), fill = Liked)) +
+   geom_bar(alpha = 0.8) + 
+   guides(fill = FALSE)
> 

> reviews_total<-rbind(clothing_reviews$Liked,
+                      clothing_reviews$text)
> 

> reviews_m<-as.matrix(reviews_total)
> dimnames(reviews_m)<-NULL
> 
> set.seed(123)
> ind<-sample(2,nrow(reviews_m),replace=TRUE,prob=c(0.7,0.3))
> reviews_train<-reviews_m[ind==1,1]
> reviews_test<-reviews_m[ind==2,1]
> reviews_trainlabel<-as.numeric(reviews_m[ind==1,2])
> reviews_testlabel<-as.numeric(reviews_m[ind==2,2])
Warning message:
NAs introduced by coercion 
> 
> maxlen<-100
> max_words<-10000
> tokenizer<-text_tokenizer(num_words=max_words)%>%
+   fit_text_tokenizer(reviews_train)
> sequences<-texts_to_sequences(tokenizer, reviews_train)
> word_index= tokenizer$word_index
> 

> x_train<-pad_sequences(sequences, maxlen=maxlen)
> y_train<-as.array(reviews_trainlabel)
> 
> batch_size <- 20
> epochs <- 12
> 
> model <- keras_model_sequential()%>%
+   layer_embedding(input_dim = max_words, output_dim = 100)%>%
+   layer_lstm(units = 16)%>%
+   layer_dense(1)%>%
+   layer_activation("sigmoid")%>% compile(
+     loss = "binary_crossentropy",
+     optimizer = "adam",
+     metrics = "accuracy"
+     
+   )
> 
> hist <- model %>%
+   fit(
+     x_train,
+     y_train,
+     batch_size = batch_size,
+     epochs = epochs,
+     validation_split = 0.3
+   )

2019-07-21 20:19:34.543600: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2

Error in py_call_impl(callable, dots$args, dots$keywords) : IndexError: list index out of range

Answer 1

While the first 'error' is just an indication, that the TensorFlow binary you installed is not 'optimal' for your CPU. Instead of installing the default TensorFlow you can build form source and thus optimize the CPU usage etc., but this is not an error (when I'm working with Tensorflow / Keras I get this too.)

Further for the second error, a quick search showed: https://github.com/rstudio/tensorflow/issues/238#issuecomment-383189626 where it is discussed that it seems that the py_call_imb() error happens due to dependencies related to RStudio / Keras.

To quote from another answer, apparently one user could resolve the issue:

"Well - I used https://mran.microsoft.com/package/keras, where version 2.1.5, published 2018-03-25 is announced. However, upon installing as indicated, the whole process as described in my last post starts again. (I just repeated it, alas...). Only from devtools::install_github("rstudio/keras") I am able to get a working installation."