Understanding the scalability of RShiny apps hosted on ShinyServer

Question

I am building a series of interactive shiny web apps for a project that I am considering turning into a Company. My background is in data science and I don't have a lot of experience on the web app / server side of things, but these are important aspects for me to consider with my project. I currently have an Amazon Linux AMI EC2 instance with ShinyServer (free, open-source) installed, and I am currently hosting early versions of my web apps there. So far everything works fine, but I haven't made the links public yet.

1. My first question is whether anyone knows if there are certain limitations (scalability limitations, integration with database limitations, security / authentication limitations, etc.) that I will inevitably run into using RShiny apps and ShinyServer? I haven't heard of many successful, super-popular web apps being shiny apps hosted on ShinyServer, but rather my feeling is that ShinyServer is mainly used for hosting RShiny apps that are shared amongst only a small number of people (i.e. shared amongst team members at a company.). Per this thread - Does R-Server or Shiny Server create a new R process/instance for each user? - I am particularly concerned that my app won't be able to handle thousands of users simultaneously since only 1 R process is created for the app regardless of the # of concurrent users of the app. Having 10-20 processes through ShinyServer pro probably doesn't solve the issue either if I ever intend to scale greater than the hundreds or thousands of users. I also noticed that ShinyServer Pro would run me a not-so-negligible $10K per year.

2. My second question is whether RShiny apps can be deployed using other server technologies, such as Heroku. I came across this github page (https://github.com/virtualstaticvoid/heroku-buildpack-r/tree/heroku-16) but haven't dug too deep into it yet. I've been told that heroku makes it easy to update releases to apps whose code is on github (git push heroku:master), amongst other things.

3. My third question involves certain specific considerations of mine. In particular, I am currently working on a script that queries data from an API and writes that data to a (not-yet-setup) database of mine. This is the data my apps use, and I'd be interested in having the apps update in real time as the database updates, without requiring the user to refresh the webpage. A buddy of mine suggested AJAX for this type of asynchronous behavior, and it looks like this may be possible in R with something like this (https://github.com/daattali/advanced-shiny/tree/master/api-ajax).

Sorry that this is such a loaded question, but I hope it doesn't get closed down as I think it is fairly educational. Any suggestions / sources / pointing me in the right direction would be greatly appreciated on this.

Answer 1

Canovice,

I'd recommend you take a look at the following RStudio / AWS support articles. To scale a shiny server you'll need to look at using a load balancer:

• RStudio

  • https://shiny.rstudio.com/articles/scaling-and-tuning.html
  • https://support.rstudio.com/hc/en-us/articles/220546267-Scaling-and-Performance-Tuning-Applications-in-Shiny-Server-Pro
  • https://support.rstudio.com/hc/en-us/articles/217801438-Can-I-load-balance-across-multiple-nodes-running-Shiny-Server-Pro-

• AWS

  • https://aws.amazon.com/blogs/big-data/running-r-on-aws/

• Blog Article:

  • http://mgritts.github.io/2016/07/08/shiny-aws/

Shiny is a great platform, their support is fabulous. I'd recommend you ring them up - they'll be sure to help answer your questions.

That said if your plan is to create a scalable website that will support thousands or hundreds of thousands of people then my sense would be to recommend you also review and consider using D3.js in conjunction with react.js or Angular.js, not forgetting to mention node.js.

My sense is that you are looking at a backend database connected to a logic engine and visualisation front end. If you are looking for a good overview of usage take a look at the following web page and git repo [A little dated but useful]:

• https://anmolkoul.wordpress.com/2015/06/05/interactive-data-visualization-using-d3-js-dc-js-nodejs-and-mongodb/
• https://github.com/anmolkoul/node-dc-mongo

I hope the above points you in the right direction.

Answer 2

I'd like to provide some notes related to your second question: Yes, you can use the mentioned buildback to deploy shiny applications on heroku.

I was in a similar situation with you (asking myself about possible ways of serving Shiny applications in a scalable manner) and decided to go the "heroku way".

You may find these hints helpful when deploying your app to heroku using the buildpack mentioned above:

• Heroku tries to "guess" how to execute your application. But you can also add a special file, named Procfile, to your application to control the process commands you want to execute for your application. In my case I used web: R -f ~/run.R --gui-none --no-save, where this means that a file named run.R is being passed to the R executable for the web server process

• The stack on heroku is based on Ubuntu. If you need additional deb-packages, you can create another special file named Aptfile and add the package names therein, heroku will then automatically install these for you (I needed it for RPostgreSQL)

• You can add another special file named init.R and install all R packages as necessary just as you are used to, i.e. with install.packages etc. You can also add initial configuration material within this file.

As a running example, here is an example toy application that I wrote for myself to remember how a "full-stack" shiny app may look like, including compability with heroku.

Answer 3

1. For a large number of concurrent users, use a load balancer like nginx and enable the autoscaling of your app, e.g. through Kubernetes.

2. You can deploy your app on Heroku. On the paid tiers it includes NoOps autoscaling of your app. See this tutorial on how to deploy a Shiny app in a Docker container on Heroku: https://medium.com/analytics-vidhya/deploying-an-r-shiny-app-on-heroku-free-tier-b31003858b68

3. You can query the table last update timestamp in the Shiny server logic with reactivePoll() and rerun your db query if it changed. It is not "real-time" but depending on your application close enough if you set the time interval small.