Speed up performance of a for loop

Question

This function takes a while to run. So, I would like to know how to improve its performance.

The purpose of this function is to compute several gjr garch dcc regressions and then store the standard deviations and the correlations in a list.

  sigma_function<-function(index_ret, ret){
  sigma<-data.frame(Date=index_ret[,1])
  
  rho<-data.frame(Date=index_ret[,1])
  
  for (i in c(2:ncol(ret))){
    
    x<-data.frame(index_ret, ret[,i])
    
    gjrgarch.spec <- ugarchspec(variance.model = list(model = "gjrGARCH", garchOrder = c(1, 1)), mean.model = list(armaOrder = c(0,0), include.mean = TRUE), distribution.model = "norm")
    
    dcc.gjrgarch.spec = dccspec(uspec = multispec(replicate(2,gjrgarch.spec)), dccOrder = c(1, 1), distribution = "mvnorm")
    
    dcc.fit <- dccfit(dcc.gjrgarch.spec, data = na.omit(x[,-1]))
    
    h<-dcc.fit@model[["sigma"]] #this are the conditional standard deviations
    
    h_1<-data.frame(na.omit(x),h)
    
    names(h_1)[c(4,5)]<-c(paste("sigma_rm",i), paste("sigma",names(ret)[i]))
    
    sigma<-merge(sigma,h_1[,c(1,4,5)], all = TRUE, by="Date")
    
    p<-dcc.fit@mfit$R #this is the conditional correlation matrices (3D matrices)
    #each matrix corresponds to one date
    rho_1<-c()
    for (j in c(1:nrow(na.omit(x)))){
      
      rho_1[j]<-p[[j]][1,2] #extract the correlation between the index and the stock for each date
    } 
    
    p_1<-data.frame(na.omit(x),rho_1) 
    
    names(p_1)[4]<-paste("rho",i)
    
    rho<-merge(rho,p_1[,c(1,4)], all = TRUE, by="Date")
   }
  return(list(sigma, rho))
}

Parfait · Answer 1 · 2021-08-02T22:20:02.607

Avoid growing objects in a for loop with merge. Instead, consider building a list of objects with lapply to chain merge once outside loop.

sigma_function <- function(index_ret, ret){ 
    # COLLECT RESULTS IN A LIST
    results_list <- lapply(2:ncol(ret), function(i){ 
        x <- data.frame(Date=index_ret, ret[,i]) 

        gjrgarch.spec <- ugarchspec(
            variance.model = list(model = "gjrGARCH", garchOrder = c(1, 1)), 
            mean.model = list(armaOrder = c(0,0), include.mean = TRUE), 
            distribution.model = "norm"
        )
        
        dcc.gjrgarch.spec = dccspec(
            uspec = multispec(replicate(2,gjrgarch.spec)),
            dccOrder = c(1, 1), 
            distribution = "mvnorm"
        ) 

        dcc.fit <- dccfit(
             dcc.gjrgarch.spec, data = na.omit(x[,-1])
        ) 
        h <- dcc.fit@model[["sigma"]] #conditional standard deviations 

        h_df <- data.frame(na.omit(x), h) 
        names(h_df)[c(4,5)] <- c(paste0("sigma_rm",i), paste0("sigma_",names(ret)[i]))         

        p <- dcc.fit@mfit$R #conditional correlation matrices (3D matrices)
        # each matrix corresponds to one date 
        x_rows <- nrow(na.omit(x))
        rhos <- vector(size=x_rows, mode="numeric") # PRE-DEFINE VECTOR LENGTH AND TYPE
        for (j in 1:x_rows){ 
            rhos[j] <- p[[j]][1,2] #extract the correlation between the index and the stock for each date 
        } 

        rho_df <- data.frame(na.omit(x), rhos)
        names(rho_df)[4] <- paste0("rho",i) 

        # RETURN NAMED LIST
        return(list(sigma=h_df[,c(1,4,5)], rho=rho_df[,c(1,4)]))
   })

   # CHAIN MERGE OUTSIDE LOOP
   sigma_df <- Reduce(
       function(x, y) merge(x, y, all=TRUE, by="Date"),
       lapply(results_list, "[", "sigma")
   )

   rho_df <- Reduce(
       function(x, y) merge(x, y, all=TRUE, by="Date"),
       lapply(results_list, "[", "rho")
   )

   # RETURN NAMED LIST
   return(list(sigma=sigma_df, rho=rho_df)) 
}

Thank you @Parfait. Were you able to run the code without getting any errors? Because I got the following error: multi-argument returns are not permitted. So I changed return(sigma=h_df[,c(1,4,5)], rho=rho_df[,c(1,4)]) to a vector or a list but then I get the following error: Error in fix.by(by.x, x) : 'by' must specify a uniquely valid column. — Suzy, Aug 02 '21 at 18:35
I do have any reproducible data to test. So, this will need checking. I did correct the `return` line. That last error indicates the `by` column, `Date`, not present for `merge`. I rename first column of `x` data frame which may help. — Parfait, Aug 02 '21 at 22:21

Speed up performance of a for loop

1 Answers1