Problems with deployment of shiny app due to reticulate_python

Question

I have an app that I want to deploy on shinyapps.io. I think it is important to note that in the app data is extracted from a database in athena. I used the package Rathena to connect to the database and everything used to run fine locally from my computer. However I tried to deploy it on shinyapps.io and I get a message that an error has occurred. I see this simply in the window that pops up after I deployed it. I went and looked at the log section from shinyapps.io and I see the message that says:

warning: using reticulate but python was not specified; will use python at /usr/bin/python3 Did you forget to set the RETICULATE_PYTHON environment variable in your .Rprofile before publishing?

I have never used an .Rprofile and I don't know what it means by RETICULATE_PYTHON. Anyways they say on stack overflow google is your best friend. I found this tutorial on Github called Tutorial: using Shiny + reticulate to create apps with R and Python 3. I did the initial steps like it says to and then it says at the end:

Confirm that the .Rprofile file is included in your project's directory and was deployed along with server.R and ui.R to shinyapps.io. This file sets the RETICULATE_PYTHON environment variable, which tells reticulate where to locate the Python virtual environment on the shinyapps.io servers.

I looked up what .Rprofile is and I think it's just a script for code but It doesn't tell me what code I should put there. I think I need to mention that package Rathena uses Python code to connect to the database.

I also read somewhere that I should place this line RETICULATE_PYTHON = /usr/local/bin/python3 in my .Renviron file. I did this and now when I try to run the app locally on my computer I get the error message:

Error: Boto3 is not detected please install boto3 using either: pip install boto3 numpy in terminal or install_boto(). If this doesn't work please set the python you are using with reticulate::use_python() or reticulate::use_condaenv()

This doesn't make sense because I already have installed boto3 and even in the terminal it says

Requirement already satisfied: boto3 in ./Library/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (1.14.28)

I feel like I am going in circles here.

Answer 1

This is actually a fairly annoying problem with running software locally and running in another environment and I'd say solutions are generally non-obvious. You probably aren't the first person to run into this.

Here are some facts that I think are relevant:

• reticulate needs to know where your Python executable is on the computer
• The locatl of the Python executable differs between your computer and the virtual computer shinyapps.io runs your Shiny app from
• reticulate seems to determine where your Python executable is from the RETICULATE_PYTHON environment variable but also has some default behavior when the variable isn't set
• The Python executable on your computer is at a different location than the executable on the virtual computer shinyapps.io runs your Shiny app from
• When you set the value of RETICULATE_PYTHON in your .Rprofile to the value shinyapps.io needs, your setup breaks because your Python environment is different

I haven't tested this as a solution but I think the fix is actually in your link! Check out https://github.com/ranikay/shiny-reticulate-app/blob/master/.Rprofile and see how they include a bit more than just a simple assignment of a value to RETICULATE_PYTHON and actually use conditional statements to set various values depending on the result of (Sys.info()[['user']].

The value of (Sys.info()[['user']] will differ based upon whether R executes .Rprofile on your local computer versus the virtual computer shinyapps.io runs your Shiny app on and the linked .Rprofile is doing the work of setting up various values based upon whether R is running on your local computer or on another computer. This all works because shinyapps.io

From the code in the .Rprofile, I can tell that shinyapps.io runs your Shiny app under the user named "shiny",

if (Sys.info()[['user']] == 'shiny'){
  Sys.setenv(RETICULATE_PYTHON = paste0('/home/shiny/.virtualenvs/', VIRTUALENV_NAME, '/bin/python'))
}

and they even add a conditional block in their to support RStudio Connect, which apparently runs your Shiny apps under the user named rstudio-connect instead of "shiny":

} else if (Sys.info()[['user']] == 'rstudio-connect'){
  Sys.setenv(RETICULATE_PYTHON = paste0(VIRTUALENV_NAME, '/bin/python'))
}

Lastly, the .Rprofile uses an else statement to catch any other case and this will match the case when you're running on your own computer:

} else {
  # RETICULATE_PYTHON is not required locally, RStudio infers it based on the ~/.virtualenvs path

From what you've described above and what I see at the linked tutorial, I think you could nearly copy the entire .Rprofile they have there into your project and, with maybe only a minor modification or two, you should be able to get your Shiny app running locally and on shinyapps.io with reticulate.

However, we see that not setting RETICULATE_PYTHON defers to behavior defined by running a virtualenv so I might recommend setting RETICULATE_PYTHON here to a value such as the location or your miniconda-based Python executable. I'm guessing that value might be ./Library/r-miniconda/envs/r-reticulate/bin/python3 or similar.

If you don't want to copy the linked .Rprofile and modify it as you need, you might just change yours to only contain this line:

if (Sys.info()[['user']] == 'shiny'){
  Sys.setenv(RETICULATE_PYTHON = "/usr/local/bin/python3")
}

And then try running locally and deploying.

This pattern of setting environment variables differently based upon the environment a program is running under is actually a fairly widely-accepted best practice in software engineering [1] so it's a pattern that may serve you in the future as you build applications locally and deploy to other environments.

[1] https://12factor.net