Manually adjusting the Z order of `geom_*` layers

Question

This is a follow-up question to my previous question on how to shade regions of a plot based on whether a given condition is satisfied or not.

When adding several geom_* layers to a ggplot2 plot, the layers are apparently drawn, by default, in the order in which they are added to the plot. For instance, in my answer to my previous question, a semi-transparent geom_rect() layer is added after a geom_line() layer. As a result, the line color changes slightly where the rectangles are overlaid; this is particularly evident when zooming in:

I would like to avoid this by putting the geom_rect() layer behind (below) the geom_line() layer.

The obvious solution is to add the layers in that order, that is, ggplot(...) + geom_rect(...) + geom_line(). This would work in the artificial example from my previous question.

In my actual, real-world application, however, this is not so easy. I have a relatively complicated plot, and several different sets of conditions I want to use for highlighting periods in said plot by shading them. What I'm doing thus is to

1. create the (complicated) plot without any shading,
2. save it to a variable (say g), and then, for each set of conditions that I want to highlight separately,
3. perform g + geom_rect(...) + scale_fill_discrete(...) to add the required shading.

Adding geom_rect(...) before the remaining layers would require me to recreate the complicated plot in question for each new version with different highlighting, and I would prefer to avoid that.

It seems to me that the obvious answer would be to explicitly tell ggplot2 the Z order of the layers, something along the lines of e.g. ggplot(...) + geom_line(z.order = 0, ...) + geom_rect(z.order = -1, ...) in my artificial example. But I don't know how to do this, or whether it is even possible. I've searched Google and StackOverflow for a solution, but found nothing really helpful, beyond some suggestions that this may in fact not be possible using current ggplot2 versions.

Any help would be greatly appreciated. As always, thank you!

Answer 1

This is not what you had in mind. But as an alternative approach you could use a plotting function to create your complicated plot and to which you could pass a background layer as an optional argument. Doing so you could avoid duplicating the code to create your complicated plot.

Based on the data and code of your previous question and answer such an approach may look like so.

Note: Of course could you also create a second function to create the background layers.

library(ggplot2)

plot_fun <- function(layer_rect = NULL) {
  ggplot(dl, mapping = aes(x = x, y = bound_vals, color = Bounds)) +
    layer_rect +
    geom_line(linewidth = 1) +
    theme_light()
}

plot_fun()
#> Warning: Removed 9 rows containing missing values (`geom_line()`).

rect <- list(
  geom_rect(
    data = iD, mapping = aes(xmin = iS, xmax = iE, fill = "Inversion"),
    ymin = -Inf, ymax = Inf, alpha = 0.3, inherit.aes = FALSE
  ),
  scale_fill_manual(name = "Inversions", values = "darkgray")
)

plot_fun(rect)
#> Warning: Removed 9 rows containing missing values (`geom_line()`).

DATA

library(tidyverse)
library(RcppRoll)

set.seed(42)
N <- 100
l <- 5
a <- rgamma(n = N, shape = 2)
d <- tibble(x = 1:N, upper = roll_maxr(a, n = l), lower = roll_minr(a + lag(a), n = l)) %>%
  mutate(
    inversion = upper < lower,
    inversionLag = if_else(is.na(lag(inversion)), FALSE, lag(inversion)),
    inversionLead = if_else(is.na(lead(inversion)), FALSE, lead(inversion)),
    inversionStart = inversion & !inversionLag,
    inversionEnd = inversion & !inversionLead
  )
dl <- pivot_longer(d, cols = c("upper", "lower"), names_to = "Bounds", values_to = "bound_vals")

iS <- d %>%
  filter(inversionStart) %>%
  select(x) %>%
  rowid_to_column() %>%
  rename(iS = x)
iE <- d %>%
  filter(inversionEnd) %>%
  select(x) %>%
  rowid_to_column() %>%
  rename(iE = x)
iD <- iS %>% full_join(iE, by = c("rowid"))

Answer 2

I think Stefan's solution is great. This is just to show that you can actually manually change the underlying order of your geom layers. It requires that you really only have on geom_rect layer!

## make your plot
g <- ggplot(dl, mapping = aes(x = x, y = bound_vals, color = Bounds)) +
  geom_line(linewidth = 1) +
  geom_rect(data = iD, mapping = aes(xmin = iS, xmax = iE, fill = "Inversion"), ymin = -Inf, ymax = Inf, alpha = 0.3, inherit.aes = FALSE) +
  scale_fill_manual(name = "Inversions", values = "darkgray") +
  theme_light()

## get index of geom rect layer 
rect_ind <- which(lapply(g$layers, function(x) class(x$geom)[1]) == "GeomRect")

## manually change the order to put geom rect first
g$layers <- c(g$layers[rect_ind], g$layers[-rect_ind])
g
#> Warning: Removed 9 rows containing missing values (`geom_line()`).

here in zoom:

Answer 3

I ended up using Ricardo Saporta's "handy little function" from this answer (h/t to MrFlick for pointing it out) --- note that I removed the default value for after, as I found that it needed to be passed anyway:

library(tidyverse)
library(RcppRoll)

# https://stackoverflow.com/a/20250185/20917815
insertLayer <- function(plotObj, after, ...) {
    if (after < 0)
        after <- after + length(P$layers)

    if (!length(plotObj$layers))
        plotObj$layers <- list(...)
    else
        plotObj$layers <- append(plotObj$layers, list(...), after)

    return(plotObj)
}

set.seed(42)
N <- 100
l <- 5
a <- rgamma(n = N, shape = 2)
d <- tibble(x = 1:N, upper = roll_maxr(a, n = l), lower = roll_minr(a + lag(a), n = l)) %>%
    mutate(inversion = upper < lower,
           inversionLag = if_else(is.na(lag(inversion)), FALSE, lag(inversion)),
            inversionLead = if_else(is.na(lead(inversion)), FALSE, lead(inversion)),
        inversionStart = inversion & !inversionLag,
        inversionEnd = inversion & !inversionLead
    )
dl <- pivot_longer(d, cols = c("upper", "lower"), names_to = "Bounds", values_to = "bound_vals")

iS <- d %>% filter(inversionStart) %>% select(x) %>% rowid_to_column() %>% rename(iS = x)
iE <- d %>% filter(inversionEnd) %>% select(x) %>% rowid_to_column() %>% rename(iE = x)
iD <- iS %>% full_join(iE, by = c("rowid"))

g <- ggplot(dl, mapping = aes(x = x, y = bound_vals, color = Bounds)) +
    geom_line(linewidth = 1)

g %>%
    insertLayer(
        after = 0,
        geom_rect(data = iD, mapping = aes(xmin = iS, xmax = iE, fill = "Inversion"), ymin = -Inf, ymax = Inf, alpha = 0.3, inherit.aes = FALSE)
    ) +
    scale_fill_manual(name = "Inversions", values = "darkgray") +
    theme_light() -> g
g

This gives

with a minimum of fuss in what I feel is a fairly intuitive and flexible manner. The only downside is that you have to use %>% (or |>) rather than +, and that you may need parentheses since the piping operators bind more tightly than +. But this is also a result of ggplot2's choice to use +, and only represents a minor inconvenience.