Create for loop in R ggplot wide format

Question

I'm trying to do tackle two problems to create one complete graph (preferably in ggplot). First, I think I need to develop a for loop for data in wide format, so that I can plot multiple series as individual lines on a single plot.

Second, I need to plot another column (climate data, different y-values) over top of the individual series. I know there are multiple arguments about dual axis graphs, but a client has asked for this specific graph, and I can't get around it.

This is the ideal graph to produce (borrowed from a 2011 publication)

My data is currently in wide format, and my thought is to create a for loop for columns 2:13, to produce individual gray lines for every series. Then I'd like to be able to selectively add a specific month (columns 14:25) in red over top. I'm happy to switch to long format if I can selectively pull months out and plot them over the gray data series. Below is a small subset of data, the actual data set is much larger.

WideDF <- structure(list(Year = c("2008", "2009", "2010", "2011", "2012", 
"2013", "2014"), M3T1045 = c(153.821659726048, 123.081558884975, 
176.771512423309, 181.76302562778, 121.366484915564, 112.166318263156, 
83.430179685548), M3T2045 = c(1979.67428393397, 2304.69652323779, 
2038.57278760525, 1598.21780623635, 1698.10896695776, 1282.44883906715, 
1243.48492202174), M3T3045 = c(2059.61060104009, 2538.3591621577, 
2836.19881815023, 2529.3912848661, 2715.04505989801, 2137.18245498415, 
1914.61513277697), M3T4045 = c(3923.28951818649, 4180.76535402238, 
4214.78156108969, 4295.05332803458, 4522.22790268241, 4472.36745319943, 
4098.5309021999), M3T5045 = c(2018.8160905218, 1997.65271391178, 
2079.82809750919, 2248.83387563616, 2494.15706661903, 1355.10547024274, 
1353.37214047842), M3T6045 = c(4409.16623051298, 4276.51428713093, 
3831.60326756482, 3308.15549785341, 4045.82249844548, 4462.00875828256, 
3572.81219768132), M3U1045 = c(113.83650201514, 118.780241020127, 
114.984700722952, 133.421492210513, 205.600098647934, 122.476010617811, 
97.1844650220458), M3U2045 = c(337.787032910181, 376.487106438174, 
430.035628616836, 414.099557447903, 458.763256619945, 331.056861800248, 
238.741139683364), M3U3045 = c(320.507421943828, 322.708996668388, 
313.17219893837, 270.058319678974, 376.456328254947, 267.987391226506, 
296.912173018184), M3U4045 = c(1095.26781018262, 1138.4400696583, 
1401.22300738868, 1387.70010663431, 1658.03042540523, 1230.4459207838, 
987.787227627777), M3U5045 = c(125.346087984721, 127.324176091091, 
111.68607241898, 164.421987100271, 241.013673561191, 158.550165076745, 
157.942524508873), M3U6045 = c(774.63712785103, 824.750682887739, 
745.467515460673, 666.817790501942, 988.567171986328, 654.042598223899, 
559.472739395766), Jan = c(1.11, 3.38, 3.41, 1.26, 1.02, -1.39, 
1.71), Feb = c(2.32, 2.85, 3.09, 0.97, -1.52, -1.22, 1.55), Mar = c(2.47, 
2.1, 3.59, 1.31, -2.29, -1.62, 1.7), Apr = c(2.45, 2, 2.5, 1.96, 
-1.93, -2.14, 1.45), May = c(1.44, 1.97, 1.47, 2.12, -1.26, -1.35, 
1.48), Jun = c(1.77, 3.53, 1.86, 1.74, 2.06, 1.57, 1.73), Jul = c(2.07, 
4.66, 1.14, 1.1, 1.07, 2.28, 3.09), Aug = c(2.83, 4.74, -1.6, 
2.55, 1.08, 2.79, 3.05), Sep = c(4.04, 3.63, -1.22, 2.26, 1.5, 
3.48, 1.92), Oct = c(3.89, 4.21, 0.97, 2.7, 2.46, 2.31, 2.53), 
    Nov = c(4.09, 3.97, 1.28, 1.67, 1.12, 2.13, 2.16), Dec = c(4.21, 
    3.79, 2.05, 1.38, 1.51, 1.84, 2.71)), .Names = c("Year", 
"M3T1045", "M3T2045", "M3T3045", "M3T4045", "M3T5045", "M3T6045", 
"M3U1045", "M3U2045", "M3U3045", "M3U4045", "M3U5045", "M3U6045", 
"Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", 
"Oct", "Nov", "Dec"), row.names = 65:71, class = "data.frame")

I've tried following the post here, but I would have to create a line of code for every series.

I also tried using the for loop provided by Didzis here, but kept getting the error that each group only consists of 1 observation. This is the code I tried using:

plotAllLayers(WideDF)
fields <- names(WideDF[,1:13])
p <- ggplot(aes(x=Year), data=WideDF)
for (i in 2:length(fields)) {
loop_input = paste("geom_smooth(aes(y=",fields[i],",color='",fields[i],"'))", sep="")
p <- p + eval(parse(text=loop_input))
}
p <- p + guides( color = guide_legend(title = "",) )
p

Any help would be appreciated to tackle this problem.

Why do you insist on keeping your data in a wide format? Why not reshape to long/tall format to make things much easier to plot. — MrFlick, Jun 19 '17 at 14:13
As I mentioned in the post, I'm happy to reshape to long if there's a way to still pull out specific months of climate data to plot over top. When I converted to long format originally, it was plotting all of my data (individual times series, as well as climate data in months). If there's a way to avoid this and still produce the above graph, I'm happy to convert to long. — KKL234, Jun 19 '17 at 14:24
I'd recommend against plotting two data series with different units on the same plot, especially if scaling is done arbitrarily. It tends to imply relations that aren't true. — AkselA, Jun 19 '17 at 16:33

YCR · Answer 1 · 2017-06-19T15:22:34.433

0

Your data is a mess, you can't really use your variable M3T or M3U. Plus, you should definitely switch to a long format and avoid doing a loop in ggplot2.

Here is an exemple of how to do it:

library(tidyverse)

coef_shrink <- 120
month_picked <- month.abb[1]

# convert to long format:
longdf <- WideDF %>%
  gather(key = var, value = measure
         , -Year)

# limit to the monthly variables:
longdf_year <- longdf %>%
  filter(var %in% month_picked) %>%
  mutate(measure = measure * coef_shrink)

# limit to the M* variables
longdf_MTU <- longdf %>%
  filter(!var %in% month.abb)

# plot:
ggplot(longdf_MTU, aes(x = Year, y = measure, group = var)) +
  geom_line() +
  geom_line(data = longdf_year, aes(x = Year, y = measure), color = "red")+
  scale_y_continuous(
    "var 1",
    sec.axis = sec_axis(~ . / coef_shrink, name = "var 2")
  )

edited Jun 19 '17 at 15:22

answered Jun 19 '17 at 14:54

YCR

3,794
3
25
29

Thanks for providing a possible answer. Can you help explain your answer a bit more? To help clarify, I merged 2 separate dataframes together to try to compile all of my data by year (df1=M3U and M3T columns, df2=month columns). If I can keep them separate and overlay 2 graphs, I'm happy to do so. Also, why can't I use my M3U and M3T variables? They are my primary data, and what I'd like to use to create my individual lines in the graph. The months (Jan-Dec) are there so I can selectively choose months to overlay on the M3U and M3T data. – KKL234 Jun 19 '17 at 15:00
1

ggplot2 provide a way to plot your data in nearly anyway you want from the moment they are in a long format and use a stack of layers to create a plot. So you can merge multiple ggplot graphs if you want. Basically, there is no reason in your case to concatenate your df. – YCR Jun 19 '17 at 15:09
Plus, you can add a 2nd axis: https://stackoverflow.com/questions/3099219/plot-with-2-y-axes-one-y-axis-on-the-left-and-another-y-axis-on-the-right – YCR Jun 19 '17 at 15:11
I've looked at the stackoverflow question you posted before, but I've gotten stuck with it every time. If I put my data in long format, how can I plot just specific variables (M3U and M3T data) without plotting the other variables that will be present (Jan-Dec)? – KKL234 Jun 19 '17 at 15:17
Edited the answer with the 2nd axis – YCR Jun 19 '17 at 15:22

Create for loop in R ggplot wide format

1 Answers1