Optimizing add_trace() in a for loop?

Question

I'm using the add_trace() function in a for loop to create lines for a 3d network graph in plotly's scatter3d mode. Each add_trace draws an individual line between two nodes in the network. The method is working, but with large number of loops, the speed of the individual loops seems to be slowing down very quickly.

Example data can be downloaded here: https://gist.github.com/pravj/9168fe52823c1702a07b

library(igraph)
library(plotly)

G <- read.graph("karate.gml", format = c("gml"))
L <- layout.circle(G)

vs <- V(G)
es <- as.data.frame(get.edgelist(G))

Nv <- length(vs)
Ne <- length(es[1]$V1)

Xn <- L[,1]
Yn <- L[,2]

network <- plot_ly(type = "scatter3d", x = Xn, y = Yn, z = rep(0, Ne), mode = "markers", text = vs$label, hoverinfo = "text", showlegend = F)

for(i in 1:Ne) {
  v0 <- es[i,]$V1
  v1 <- es[i,]$V2

  x0 <-  Xn[v0]
  y0 <-  Yn[v0]
  x1 <-  Xn[v1]
  y1 <-  Yn[v1]

  df <-  data.frame(x = c(x0, x1), y = c(y0, y1), z = c(0, 0))
  network <- add_trace(network, data = df, x = x, y = y, z = z, type = "scatter3d", mode = "lines", showlegend = F, 
                       marker = list(color = '#030303'), line = list(width = 0.5))
}

This example is fairly quick, but when I include a few hundred edges or more, the execution of the individual loops start to slow down radically. I tried different optimization methods (vectorisation etc), but there seems to be no working around the slowness of the add_trace function itself.

Any suggestions?

Answer 1

The most efficient way to add many line segments in plotly is not as a separate trace each, but to use only a single trace that contains all the line segments. You can do this by constructing a data frame with the x,y coordinates of each node to be connected, interspersed with NA's between each line segment. Then use connectgaps=FALSE to break the trace into separate segments at each NA. You can see another example of this approach, applied to spaghetti plots in this answer.

es$breaks <- NA
lines <- data.frame(node=as.vector(t(es)), x=NA, y=NA, z=0)
lines[which(!is.na(lines$node)),]$x <- Xn[lines[which(!is.na(lines$node)),]$node]
lines[which(!is.na(lines$node)),]$y <- Yn[lines[which(!is.na(lines$node)),]$node]

network <- plot_ly(type = "scatter3d", x = Xn, y = Yn, z = rep(0, Ne), 
                   mode = "markers", text = vs$label, hoverinfo = "text", 
                   showlegend = F) %>% 
  add_trace(data=lines, x=x, y=y, z=z, showlegend = FALSE,
                      type = 'scatter3d', mode = 'lines+markers',
                      marker = list(color = '#030303'), line = list(width = 0.5),
                      connectgaps=FALSE)

Reproducible data for this question

For convenience, here are the data for this question. The OP required downloading a .gml file from github, and installing library(igraph) to process the data into these.

es <- structure(list(
  V1 = c(1, 1, 2, 1, 2, 3, 1, 1, 1, 5, 6, 1, 2, 3, 4, 1, 3, 3, 1, 5, 6, 1, 1, 4, 1, 2, 3, 4, 6, 7, 1, 2, 1, 2, 
    1, 2, 24, 25, 3, 24, 25, 3, 24, 27, 2, 9, 1, 25, 26, 29, 3, 9, 15, 16, 19, 21, 23, 24, 30, 31, 32, 9, 10, 14, 15, 16, 19, 20, 
    21, 23, 24, 27, 28, 29, 30, 31, 32, 33), 
  V2 = c(2, 3, 3, 4, 4, 4, 5, 6, 7, 7, 7, 8, 8, 8, 8, 9, 9, 10, 11, 11, 11, 12, 13, 13, 
    14, 14, 14, 14, 17, 17, 18, 18, 20, 20, 22, 22, 26, 26, 28, 28, 28, 29, 30, 30, 31, 31, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 
    33, 33, 33, 33, 33, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34)), 
  .Names = c("V1", "V2"), row.names = c(NA, -78L), class = "data.frame")

theta <- seq(0,2,length.out=35)[1:34]
Xn <- cospi(theta)
Yn <- sinpi(theta)

Nv <- NROW(Xn)
Ne <- NROW(es)
vs <- data.frame(label = as.character(1:Nv))