4

I am learning full stack development for a basketball-related data viz web app that I want / am trying to make. I have a question relating to the best approach for solving a data problem of mine, given what tech skills I have.

A few months ago I created, using R, the following R Shiny App. My stack / very patchy backend for this app is:

  • I used R Shiny to launch the app, hosted on an EC2 instance with Shiny Server
  • In R, I have a script called scraper.R that queries data daily (once per day) from MySportsFeed, a great sports API, and updates the new daily data into 1 of 8 CSV files I have. (I have a different CSV file for each graph in my Shiny app)
  • My R Shiny app then reads the CSVs into the app.R file that has all of the code for the entire app.

To expand on this app and make it better, I have been learning full stack javascript development, and I am about to begin coding this Shiny app as a React app.

My current challenge / consideration is how I should manage the data in my React app. In particular:

  1. My scraper.R file does A TON of data manipulation with dplyr and other R libraries. Getting the data from the form that MySportsFeed's API provides it in, to the form needed for my Shiny App, for all of my graphs and charts, is quite a bit.

  2. With that said, I would like to keep using R to call the MySportsFeed API and format the data. I would of course like to do away with the CSV files but it is not easy. Each of the 8 CSV files includes the result from hundreds of calls to the MySportsFeed API. (each API call to MySportsFeed may give me 1 NBA game of data, whereas the CSV files have been updated with 1000 games of NBA data).

  3. I believe I want to replace (a) reading data from the CSV files into the R Shiny app with (b) calling my own API endpoints with the data feeds hosted (I can create API endpoints in R or Javascript, R's plumber is good for this)

  4. I believe I want to replace (a) storing data in CSV files with (b) storing the data in a mongo database, which plugs easily into a React app.

My question is then: Is there an intuitive way to connect the following together:

  • my R code (which calls the MySportsFeed API, grabs new data, and manipulates that data into chart-ready format)
  • an API (whether an R plumber API or a javascript API that has all of my data)
  • mongo database (to hold all of my historically scrapped data).

Thanks in advance.

EDIT - A few months ago I made a post here on an RShiny app that I wanted to maybe turn into a React app.

Canovice
  • 9,012
  • 22
  • 93
  • 211
  • 1
    This would be doable with Plumber or, as the other answer mentions, OpenCPU. Here's an example a Plumber app that hosts its own Vue.js client: https://github.com/trestletech/movies-plumber. You could do something very similar with React. – Jeff Allen Mar 19 '18 at 15:26

1 Answers1

3

I think you should look into OpenCPU - it does majority of the stuff for you (HTTP API to your R code) out of the box plus it's scalable at a fraction of cost of shiny server pro. It requires turning your code into R package(s) though.

Connecting to MongoDB is possible via mongolite library, but unless you need unstructured data store for some reason (not sure if the feed you've scraped is stable), SQL databases might be easier to handle when database performance drops (I find debugging performance issues in MongoDB harder).

I cannot comment on React, as I know next to nothing about JS, but apparently people have coupled OpenCPU with ReactJS: https://github.com/sdeboudt/reactr

PSzczesny
  • 373
  • 2
  • 12