6

The goal here is to install the dplyr-snowflakedb R package on an AWS Linux machine. The problem is that the rJava package will not install, it fails with the following error message:

“installation of package ‘rJava’ had non-zero exit status”

The dplyr.snowflakedb package makes the following recommendations:

The rJava package needs to be installed and working with Java 8 to support the SnowflakeDB JDBC requirements. This may require:

  • installing Java 8

  • running R CMD javareconf so R uses the Java 8 for its JAVA_HOME

  • installing rJava from source so it can be linked against Java 8

Despite trying these strategies, rJava is still not able to be installed on AWS linux.

The AWS linux instance is running Java 8:

conda activate ~/anaconda3/envs/R
(R) [ec2-user@ip-xx-xx-xx-xx ~]$ java -version

>>> openjdk version "1.8.0_152-release"
>>> OpenJDK Runtime Environment (build 1.8.0_152-release-1056-b12)
>>> OpenJDK 64-Bit Server VM (build 25.152-b12, mixed mode)

The R CMD javareconf command was executed:

Java interpreter : /home/ec2-user/anaconda3/envs/JupyterSystemEnv/jre/bin/java
Java version     : 1.8.0_152-release
Java home path   : /home/ec2-user/anaconda3/envs/JupyterSystemEnv
Java compiler    : /home/ec2-user/anaconda3/envs/JupyterSystemEnv/bin/javac
Java headers gen.: /home/ec2-user/anaconda3/envs/JupyterSystemEnv/bin/javah
Java archive tool: /home/ec2-user/anaconda3/envs/JupyterSystemEnv/bin/jar

trying to compile and link a JNI program 
detected JNI cpp flags    : -I$(JAVA_HOME)/include -I$(JAVA_HOME)/include/linux
detected JNI linker flags : -L$(JAVA_HOME)/jre/lib/amd64/server -ljvm
x86_64-conda_cos6-linux-gnu-cc -I"/home/ec2-user/anaconda3/envs/R/lib/R/include" -DNDEBUG -I/home/ec2-user/anaconda3/envs/JupyterSystemEnv/include -I/home/ec2-user/anaconda3/envs/JupyterSystemEnv/include/linux  -DNDEBUG -D_FORTIFY_SOURCE=2 -O2  -I/home/ec2-user/anaconda3/envs/R/include -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib  -fpic  -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -I/home/ec2-user/anaconda3/envs/R/include -fdebug-prefix-map=/tmp/build/80754af9/r-base_1570124924484/work=/usr/local/src/conda/r-base-3.6.1 -fdebug-prefix-map=/home/ec2-user/anaconda3/envs/R=/usr/local/src/conda-prefix  -c conftest.c -o conftest.o
x86_64-conda_cos6-linux-gnu-cc -shared -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib -L/home/ec2-user/anaconda3/envs/R/lib -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib -o conftest.so conftest.o -L/home/ec2-user/anaconda3/envs/JupyterSystemEnv/jre/lib/amd64/server -ljvm -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR


JAVA_HOME        : /home/ec2-user/anaconda3/envs/JupyterSystemEnv
Java library path: /home/ec2-user/anaconda3/envs/JupyterSystemEnv/jre/lib/amd64/server
JNI cpp flags    : -I$(JAVA_HOME)/include -I$(JAVA_HOME)/include/linux
JNI linker flags : -L$(JAVA_HOME)/jre/lib/amd64/server -ljvm
Updating Java configuration in /home/ec2-user/anaconda3/envs/R/lib/R
Done.

It is not clear how to install rJava from source or Link it against Java 8.

This related question: Can't install rJava on ubuntu system advises to run the following command:

sudo apt-get install r-cran-rjava

however AWS Linux doesn't use apt-get, it uses yum, and the equivalent command with yum finds no package: 

sudo yum install r-cran-rjava
>>> No package r-cran-rjava available.

Perhaps the issue is that JAVA_HOME is set to the JupyterSystemEnv rather than the R env?

Roko Mijic
  • 6,655
  • 4
  • 29
  • 36
  • Also considering this question as a possible resolution: https://stackoverflow.com/questions/28617617/rjava-can-load-in-rstudio-but-not-in-r-os-is-ubuntu-14-04-1-lts?rq=1 – Roko Mijic Jun 10 '20 at 10:55
  • Thanks Harsh. I will take a look at that. – Roko Mijic Jun 11 '20 at 11:52
  • @Harsh how much custom code will be needed to get the ODBC connection working with dplyr the same way as the dplyr-snowflakedb package does? Ideally I would like to use code already written with dplyr-snowflakedb! – Roko Mijic Jun 11 '20 at 11:56
  • 1
    only indirectly related to your question. But in case you did not already know. For R(Studio) on aws+linux, there is a great image: https://www.louisaslett.com/RStudio_AMI/. I am pretty sure it should be pre-configured there. – Tonio Liebrand Jun 15 '20 at 09:31
  • I wonder for getting R package rJava, why don't you just invoke R and then do `install.packages('rJava')` there.? – abhiieor Jun 16 '20 at 09:29
  • In my experience if you are not able to install rJava from R then having JRE also will help. – abhiieor Jun 16 '20 at 09:30
  • 1
    Try the dev version (use `install.packages("rJava",,"http://rforge.net")` ) and consider downgrading R to v3.4 per this question/answer: https://superuser.com/questions/1498373/installing-rjava-on-ubuntu-18-04-3 – jared_mamrot Jun 17 '20 at 04:00
  • 1
    Countering jared_mamrot, downgrading to R v3.4 seems redundant. Simply ensuring that a proper jre, jdk, the jdbc driver and dependencies for these and R, should be sufficient to make the package work, similar to how one would do for ubuntu. – Oliver Jun 19 '20 at 12:23
  • "similar to how one would do for ubuntu." -- yes, but that doesn't work on AWS linux, or at least I cannot get it to work. – Roko Mijic Jun 19 '20 at 12:27
  • 1
    Which AWS Linux ami do you use? – Vadim Yangunaev Jun 19 '20 at 13:39
  • Whatever the standad AMI for Sagemaker is. Is there an easy way to get the version from the command line? – Roko Mijic Jun 19 '20 at 15:47
  • cat /etc/os-release – Vadim Yangunaev Jun 19 '20 at 22:07
  • 1
    Have you tried a solution from my answer? – Vadim Yangunaev Jun 21 '20 at 20:53
  • NAME="Amazon Linux AMI" VERSION="2018.03" ID="amzn" ID_LIKE="rhel fedora" VERSION_ID="2018.03" PRETTY_NAME="Amazon Linux AMI 2018.03" ANSI_COLOR="0;33" CPE_NAME="cpe:/o:amazon:linux:2018.03:ga" – Roko Mijic Jun 22 '20 at 07:58

1 Answers1

4

  1. Make sure you install not only JRE, but JDK:

    sudo yum install java-1.8.0-openjdk
    sudo yum install java-1.8.0-openjdk-devel

  2. Configure:
    sudo R CMD javareconf
  3. Install rJava in R: install.packages("rJava")
  4. Once you have rJava installed and verified it is using Java 8, you can install dplyr and dependancies. install.packages(c("RJDBC", "DBI", "dplyr"))
  5. Installing dplyr.snowflakedb
    install.packages("devtools")
    devtools::install_github("snowflakedb/dplyr-snowflakedb")

If you have issues with install devtools try to instal dependencies or try another repo:

  • install.packages("devtools", dependencies=TRUE)
  • install.packages('devtools',dependencies=TRUE, repos='https://stat.ethz.ch/CRAN/')
Vadim Yangunaev
  • 1,817
  • 1
  • 18
  • 41