for loop with ggplot only uses last value, even with vector defined outside the loop

Question

I have been working on a for loop that produces 2 different plots generated with ggplot. On each plot, two groups of the same length are defined, and within each group, each individual line has a colour.

Each plot has a different number of lines (see picture) that can be divided by two (both plots represent an equal number of males and females). I managed to obtain my two different plots, but the line colours are wrong. The colour values correspond to the last plot's, instead of adapting to each plot. I tried defining the colours both inside and outside the loop, but the result is the same.

Here is a reproducible example:

# Create the dataframe:
Strain <- rep(c(rep("A", times=2), rep("B", times=4)), times=2)
Sex_ID <- rep(c("M_1", "F_2", "M_3", "F_4", "M_5", "F_6"), times=2)
State <- rep(c("virgin", "mated", "expecting", "parent"), each=6)
Huddling <- seq(from=1.5, to=3.8, by=0.1)

d<-data.frame(Strain, Sex_ID, State, Huddling)

level<-levels(d$Strain)
huddlist<-list()

# How many colours do we need? Different reds for each female, blues for males
len <- c(length(d$Sex_ID[d$Strain=="A"])/8,length(d$Sex_ID[d$Strain=="B"])/8)

for(i in 1:length(level)){
  ss<- subset(d, Strain==level[i]) # subset only for one species at a time
  m <- scales::seq_gradient_pal("cyan2", "midnightblue", "Lab")(seq(0,1,length.out = len[i]))
  f<-scales::seq_gradient_pal("tomato", "red4", "Lab")(seq(0,1,length.out = len[i]))
  fm<-c(f,m)
  ymax <- max(ss$Huddling); ymin <- min(ss$Huddling)

  # The plot
  huddling<-ggplot(ss, aes(x=factor(State), y=Huddling, color=factor(Sex_ID), group=factor(Sex_ID)))+
    geom_point(shape=21, size=3, position=position_dodge(width=0.3))+ 
    geom_line(size=0.7, position=position_dodge(width=0.3)) +
    scale_color_manual(values=fm)+
    scale_fill_manual(values="white")+
    ylim(ymin,ymax)+
    labs(y="Time huddling (s)", x="Reproductive stage")+
    theme_classic()+
    theme(axis.line.x = element_line(color="black", size = 1),
          axis.line.y = element_line(color="black", size = 1))+
    theme(axis.text=element_text(size=17),axis.title=element_text(size=19,face="bold"))+
    theme(legend.title=element_text(size=17))+
    theme(legend.text=element_text(size=15))+
    theme(legend.position="none")+ # if legend should be removed
    theme(plot.title = element_text(lineheight=.8, face="bold",size=22))+
    scale_x_discrete(limits=c("virgin", "mated", "expecting", "parent"), labels=c("Virgin", "Mated", "Expecting", "Parent"))

  huddlist[[i]] <- huddling
}

library(gridExtra)
do.call("grid.arrange", c(huddlist))

Alternatively, inside the loop:

  for(i in 1:length(level)){
  ss<- subset(d, Strain==level[i]) # subset only for one species at a time
  len<-length(levels(factor(ss$Sex_ID)))/2 # number of individuals of each sex in sample
  # this number allows to calculate the right number of reds and blues                       # to plot for females and males, respectively
  m <- scales::seq_gradient_pal("cyan2", "midnightblue", "Lab")(seq(0,1,length.out = len[i]))
  f<-scales::seq_gradient_pal("tomato", "red4", "Lab")(seq(0,1,length.out = len[i]))
  fm<-c(f,m)
  ymax <- max(ss$Huddling); ymin <- min(ss$Huddling)

  # The plot
  huddling<-ggplot(ss, aes(x=factor(State), y=Huddling, color=factor(Sex_ID), group=factor(Sex_ID)))+
    geom_point(shape=21, size=3, position=position_dodge(width=0.3))+ 
    geom_line(size=0.7, position=position_dodge(width=0.3)) +
    scale_color_manual(values=fm)+
    scale_fill_manual(values="white")+
    ylim(ymin,ymax)+
    labs(y="Time huddling (s)", x="Reproductive stage")+
    theme_classic()+
    theme(axis.line.x = element_line(color="black", size = 1),
          axis.line.y = element_line(color="black", size = 1))+
    theme(axis.text=element_text(size=17),axis.title=element_text(size=19,face="bold"))+
    theme(legend.title=element_text(size=17))+
    theme(legend.text=element_text(size=15))+
    theme(legend.position="none")+ # if legend should be removed
    theme(plot.title = element_text(lineheight=.8, face="bold",size=22))+
    scale_x_discrete(limits=c("virgin", "mated", "expecting", "parent"), labels=c("Virgin", "Mated", "Expecting", "Parent"))

  huddlist[[i]] <- huddling
}

library(gridExtra)
do.call("grid.arrange", c(huddlist))

This is what the plots look like. Normally, the first plot should have one red and one blue line, while the second should have two red and two blue lines.

you dont need `i <- i+1` in your code: `i` increases within the `for` loop` — user20650, Mar 24 '17 at 12:26
Most likely, if you want to receive help in finding any problems in your code you should add an example of your data, so that the issues can be reproduced. http://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example should help with creating an example. — user20650, Mar 24 '17 at 12:30
Please, post a new question instead of amending the existing question after you have accepted the answer. Thank you. — Uwe, Mar 24 '17 at 17:52

GGamba · Accepted Answer · 2017-03-25T14:53:18.893

The immediate problem is resolvable transforming the ggplot object to a Grob and use it in grid.arrange. The root problem is probably caused by lazy evaluation (thanks @baptiste(? - comment removed)).
Just change huddlist[[i]] <- huddling to huddlist[[i]] <- ggplot_gtable(ggplot_build(huddling)):

However what you are trying to do is mapping two different dimensions (Sex and ID) to the same aesthetic. What I would do is to separate those dimensions, adding an aes and using the standard faceting method.

For example I'd keep the same color for the same Sex and different point shapes for different ID:

While not entirely different I think this is better, for example in the first plot I assume that the same color applies to the same individual, when that's not the case.

data

set.seed(4887)
Strain <- rep(c(rep("A", times = 2), rep("B", times = 4)), times = 2)
Sex_ID <- rep(c("M_1", "F_2", "M_3", "F_4", "M_5", "F_6"), times = 2)
State <- rep(c("virgin", "mated", "expecting", "parent"), each = 6)
Huddling <- runif(8, 1.5, 3.8)

d <- data.frame(Strain, Sex_ID, State, Huddling)

code for the first plot

level<-levels(d$Strain)
huddlist<-list()

# How many colours do we need? Different reds for each female, blues for males
len <- c(length(d$Sex_ID[d$Strain=="A"])/8,length(d$Sex_ID[d$Strain=="B"])/8)

for(i in 1:length(level)){
  ss<- subset(d, Strain==level[i]) # subset only for one species at a time
  m <- scales::seq_gradient_pal("cyan2", "midnightblue", "Lab")(seq(0,1,length.out = len[i]))
  f<-scales::seq_gradient_pal("tomato", "red4", "Lab")(seq(0,1,length.out = len[i]))
  fm<-c(f,m)
  ymax <- max(ss$Huddling); ymin <- min(ss$Huddling)

  # The plot
  huddling<-ggplot(ss, aes(x=factor(State), y=Huddling, color=factor(Sex_ID), group=factor(Sex_ID)))+
    geom_point(shape=21, size=3, position=position_dodge(width=0.3))+ 
    geom_line(size=0.7, position=position_dodge(width=0.3)) +
    scale_color_manual(values=fm)+
    scale_fill_manual(values="white")+
    ylim(ymin,ymax)+
    labs(y="Time huddling (s)", x="Reproductive stage")+
    theme_classic()+
    theme(axis.line.x = element_line(color="black", size = 1),
          axis.line.y = element_line(color="black", size = 1))+
    theme(axis.text=element_text(size=17),axis.title=element_text(size=19,face="bold"))+
    theme(legend.title=element_text(size=17))+
    theme(legend.text=element_text(size=15))+
    theme(legend.position="none")+ # if legend should be removed
    theme(plot.title = element_text(lineheight=.8, face="bold",size=22))+
    scale_x_discrete(limits=c("virgin", "mated", "expecting", "parent"), labels=c("Virgin", "Mated", "Expecting", "Parent"))

  huddlist[[i]] <- ggplot_gtable(ggplot_build(huddling))
}

library(gridExtra)
do.call("grid.arrange", c(huddlist))

code for the second plot

library(tidyr)
d <- d %>% 
  separate(Sex_ID, c('Sex', 'ID'), sep = '_')

ggplot(d, aes(x = factor(State), y = Huddling, color = Sex, group = ID, shape = ID))+
  facet_grid(Strain ~ ., scales = 'free_y') +
  geom_point(size = 3, position = position_dodge(width=0.3), show.legend = F) + 
  geom_line(size = 0.7, position = position_dodge(width=0.3)) +
  scale_color_manual(values = c('red4', 'midnightblue')) +
  scale_fill_manual(values = "white") +
  scale_x_discrete(limits = c("virgin", "mated", "expecting", "parent"), 
                   labels = c("Virgin", "Mated", "Expecting", "Parent")) +
  labs(y = "Time huddling (s)", x = "Reproductive stage") +
  theme_classic() +
  theme(axis.line.x = element_line(color = "black", size = 1),
        axis.line.y = element_line(color = "black", size = 1),
        axis.text = element_text(size = 17),
        axis.title = element_text(size = 19,face = "bold"),
        legend.title = element_text(size = 17),
        legend.text = element_text(size = 15),
        plot.title = element_text(lineheight = .8, face = "bold",size = 22))

Thank you! How can I visualize a single plot from huddle_list()? Your last plot looks great and I'd rather pick that option. However, I have more data points on my real plots and the shapes overlap (messy). This is why I used a tweak to colour each individual differently, taking sex into account. — Mehdi.K, Mar 24 '17 at 15:33
I'd still be curious if you have a solution for this: I need to be able to conditionally format the points (I used a conditional fill based on the values of an extra binary factor of type Y/N). Building on your last plot, is there a way I could conditionally colour the points based on that extra factor, without changing the lines? For example if for each row you had an extra variable that says either "y" or "n", would you be able to blacken all the dots for which that factor says "y"? — Mehdi.K, Mar 24 '17 at 15:34
Not sure I understand what you are saying, but (using a shape between 21 and 25) it's possible to override the fill `aesthetic` inside `geom_point` — GGamba, Mar 24 '17 at 15:54
I edited my post with a (hopefully) clearer question. Thanks for your help! — Mehdi.K, Mar 24 '17 at 17:46

for loop with ggplot only uses last value, even with vector defined outside the loop

1 Answers1

data

code for the first plot

code for the second plot