Is there a way to read images stored locally into Plotly?

Question

In the code below another user has successfully demonstrated a workaround on how to load an image URL into a Plotly graph. The image data is taken from the data frame and then, by employing the custom data tool, appears as a tooltip on hover at each data point in the graph. Is there a way to do something similar using images stored locally, simply using the file directory?

library(shiny)
library(shinydashboard)
library(plotly)

# Data ------------------------------------------------------------------
dt <- data.frame(
  fruits = c("apple", "banana", "oranges"),
  rank = c(11, 22, 33),
  image_url = c(
    'https://images.unsplash.com/photo-1521671413015-ce2b0103c8c7?ixlib=rb-0.3.5&s=45547f67f01ffdcad0e33c8417b840a9&auto=format&fit=crop&w=667&q=80',
    "https://images.unsplash.com/photo-1520699697851-3dc68aa3a474?ixlib=rb-0.3.5&ixid=eyJhcHBfaWQiOjEyMDd9&s=ef15aee8bcb3f5928e5b31347adb6173&auto=format&fit=crop&w=400&q=80",
    "https://images.unsplash.com/photo-1501925873391-c3cd73416c5b?ixlib=rb-0.3.5&ixid=eyJhcHBfaWQiOjEyMDd9&s=379e4a0fffc6d11cd5794806681d0211&auto=format&fit=crop&w=750&q=80"
  )
)

# Dashboard ----------------------------------------------------------------
ui <- dashboardPage(
  dashboardHeader(title = "Test"),
  dashboardSidebar(),
  dashboardBody(tags$head(tags$style(
    HTML("img.small-img {
          max-width: 75px;
          }")
  )),
  plotlyOutput("hoverplot"))
)

server <- function(input, output, session) {
  output$hoverplot <- renderPlotly({
    plot_ly(
      dt,
      x         = ~ fruits,
      y         = ~ rank,
      type      = 'scatter',
      mode      = 'markers',
      hoverinfo = 'none',
      source = "hoverplotsource",
      customdata = ~ image_url
    ) %>%
      event_register('plotly_hover') %>%
      event_register('plotly_unhover')
  })

  hover_event <- reactive({
    event_data(event = "plotly_hover", source = "hoverplotsource")
  })

  unhover_event <- reactive({
    event_data(event = "plotly_unhover", source = "hoverplotsource")
  })

  hoverplotlyProxy <- plotlyProxy("hoverplot", session)

  observeEvent(unhover_event(), {
    hoverplotlyProxy %>%
      plotlyProxyInvoke("relayout", list(images = list(NULL)))
  })

  observeEvent(hover_event(), {
    hoverplotlyProxy %>%
      plotlyProxyInvoke("relayout", list(images = list(
        list(
          source = hover_event()$customdata,
          xref = "x",
          yref = "y",
          x = hover_event()$x,
          y = hover_event()$y,
          sizex = 20,
          sizey = 20,
          opacity = 1
        )
      )))
  })
}

shinyApp(ui = ui, server = server)

ismirsehregal · Answer 1 · 2020-08-27T13:43:17.143

You'll need to add the directory, in which your local images are stored, as a static resource to Shiny's web server. This can be done via addResourcePath:

library(shiny)
library(shinydashboard)
library(plotly)

ui <- dashboardPage(
  dashboardHeader(title = "Test"),
  dashboardSidebar(),
  dashboardBody(tags$head(tags$style(
    HTML("img.small-img {
          max-width: 75px;
          }")
  )),
  plotlyOutput("hoverplot"))
)

server <- function(input, output, session) {
  
  # create some local images
  if(!dir.exists("myimages")){
    dir.create("myimages")
  }
  
  myPlots <- paste0("myimages/myplot", seq_len(3), ".png")
  
  for (myPlot in myPlots) {
    png(file = myPlot, bg = "transparent")
    plot(runif(10))
    dev.off() 
  }
  
  myImgResources <- paste0("imgResources/myplot", seq_len(3), ".png")
  
  dt <- data.frame(
    fruits = c("apple", "banana", "oranges"),
    rank = c(11, 22, 33),
    image_url = myImgResources
  )
  
  # Add directory of static resources to Shiny's web server
  addResourcePath(prefix = "imgResources", directoryPath = "myimages")
  
  output$hoverplot <- renderPlotly({
    plot_ly(
      dt,
      x         = ~ fruits,
      y         = ~ rank,
      type      = 'scatter',
      mode      = 'markers',
      hoverinfo = 'none',
      source = "hoverplotsource",
      customdata = ~ image_url
    ) %>%
      event_register('plotly_hover') %>%
      event_register('plotly_unhover')
  })
  
  hover_event <- reactive({
    event_data(event = "plotly_hover", source = "hoverplotsource")
  })
  
  unhover_event <- reactive({
    event_data(event = "plotly_unhover", source = "hoverplotsource")
  })
  
  hoverplotlyProxy <- plotlyProxy("hoverplot", session)
  
  observeEvent(unhover_event(), {
    hoverplotlyProxy %>%
      plotlyProxyInvoke("relayout", list(images = list(NULL)))
  })
  
  observeEvent(hover_event(), {
    hoverplotlyProxy %>%
      plotlyProxyInvoke("relayout", list(images = list(
        list(
          source = hover_event()$customdata,
          xref = "x",
          yref = "y",
          x = hover_event()$x,
          y = hover_event()$y,
          sizex = 20,
          sizey = 20,
          opacity = 1
        )
      )))
  })
}

shinyApp(ui = ui, server = server)

As an alternative you can store your images in a www folder (subdirectory of your app folder), then you can access your images without a prefix.

Update: Here is another version using base64enc::dataURI instead of addResourcePath:

library(base64enc)
library(shiny)
library(shinydashboard)
library(plotly)

ui <- dashboardPage(
  dashboardHeader(title = "Test"),
  dashboardSidebar(),
  dashboardBody(tags$head(tags$style(
    HTML("img.small-img {
          max-width: 75px;
          }")
  )),
  plotlyOutput("hoverplot"))
)

server <- function(input, output, session) {
  
  # create some local images
  if(!dir.exists("myimages")){
    dir.create("myimages")
  }
  
  myPlots <- paste0("myimages/myplot", seq_len(3), ".png")
  
  for (myPlot in myPlots) {
    png(file = myPlot, bg = "transparent")
    plot(runif(10))
    dev.off() 
  }
  
  myImgResources <- vapply(myPlots, function(x){base64enc::dataURI(file = x)}, FUN.VALUE = character(1L))
  
  dt <- data.frame(
    fruits = c("apple", "banana", "oranges"),
    rank = c(11, 22, 33),
    image_url = myImgResources
  )
  
  output$hoverplot <- renderPlotly({
    plot_ly(
      dt,
      x         = ~ fruits,
      y         = ~ rank,
      type      = 'scatter',
      mode      = 'markers',
      hoverinfo = 'none',
      source = "hoverplotsource",
      customdata = ~ image_url
    ) %>%
      event_register('plotly_hover') %>%
      event_register('plotly_unhover')
  })
  
  hover_event <- reactive({
    event_data(event = "plotly_hover", source = "hoverplotsource")
  })
  
  unhover_event <- reactive({
    event_data(event = "plotly_unhover", source = "hoverplotsource")
  })
  
  hoverplotlyProxy <- plotlyProxy("hoverplot", session)
  
  observeEvent(unhover_event(), {
    hoverplotlyProxy %>%
      plotlyProxyInvoke("relayout", list(images = list(NULL)))
  })
  
  observeEvent(hover_event(), {
    hoverplotlyProxy %>%
      plotlyProxyInvoke("relayout", list(images = list(
        list(
          source = hover_event()$customdata,
          xref = "x",
          yref = "y",
          x = hover_event()$x,
          y = hover_event()$y,
          sizex = 20,
          sizey = 20,
          opacity = 1
        )
      )))
  })
}

shinyApp(ui = ui, server = server)

Thanks! Very helpful, however, would it be possible to simply read the file directory from the data frame instead of the URL as in the original code? Alternatively, could you please elaborate on your second option of storing in a www folder? — eliotandpipe, May 21 '20 at 09:34
Just updated with a second approach. The file directory is taken from a `data.frame`. The `www` folder of a shiny app provides resources to the app without using `addResourcePath`. — ismirsehregal, Aug 27 '20 at 13:45
@ismirsehregal this is great. Do you know if this is possible to do without a shiny app? Meaning a stand alone plotly graph displayed in RStudio viewing pane or in an R Markdown file? — bmacGTPM, Apr 05 '21 at 15:11
I should note that I saw this but wondering if there is a solution that is similar to yours and doesn't involve javascript https://anchormen.nl/blog/data-science-ai/images-in-plotly-with-r/ — bmacGTPM, Apr 05 '21 at 17:30
@bmacGTPM sorry, as far as I know there is no r-plotly-only-solution without using additional JS. — ismirsehregal, Apr 06 '21 at 08:53
Thanks for the useful solution. I am wondering if it would be possible to place the images in different positions relatively to the position of the mouse depending on the actual position (retrieved with hover_event()), instead than placing them so that the upper left corner of the image is placed at hover_event()$x and hover_event()$y. — Stefano Guidi, Dec 02 '21 at 10:18
Ideally, I'd like that if hover_event()$x < x_midpoint then the image is placed to the right of the crosshair, and to the left if hover_event()$x < x_midpoint. Similarly if hover_event()$y > y_midpoint the image should be placed below the crosshair (like now), and above it if hover_event()$y < y_midpoint — Stefano Guidi, Dec 02 '21 at 10:18
@StefanoGuidi I think this should go into a separate question - which you may link here. — ismirsehregal, Dec 03 '21 at 07:03

Is there a way to read images stored locally into Plotly?

1 Answers1

Linked