Unquoting the loop variable in `rlang::expr`

Question

I came across unexpected behavior when using operator !! to iteratively construct expressions that involve a loop variable.

Creating an expression from the loop variable itself, e.g.,

X <- list()
for( i in 1:3 )
  X[[i]] <- rlang::expr( !!i )
str(X)
# List of 3
#  $ : int 1
#  $ : int 2
#  $ : int 3

works as expected. However, attempting to construct any complex expression involving the loop variable, such as

Y <- list()
for( i in 1:3 )
    Y[[i]] <- rlang::expr( 1 + (!!i) )
str(Y)
# List of 3
#  $ : language 1 + 3L
#  $ : language 1 + 3L
#  $ : language 1 + 3L

seems to produce expressions that contain the final value of the loop variable. It's almost as if the expressions get captured as quote( 1 + (!!i) ) during the loop, but the unquoting via !! happens only after the entire loop is executed, as opposed to at every iteration.

Can somebody please shed some light on this behavior? Is this a bug? The intended goal is to capture unevaluated expressions 1+1, 1+2 and 1+3 with a loop. The expected output of Y would therefore be

# List of 3
#  $ : language 1 + 1L
#  $ : language 1 + 2L
#  $ : language 1 + 3L

Side note: the extra parentheses around !!i are there to address the potential issue of operator precedence.

Using rlang 0.2.1.

`rlang::expr( !! (1 + i) )` works (if in need of a quick workaround). Adding `str(Y)` inside `for(...) {...}` reveals that all expressions are indeed evaluated for the last `i`. — Mike Badescu, Aug 27 '18 at 05:06
Thanks, Mike. Unfortunately, `expr( !!(1 + i) )` collapses the expression. The goal is to capture the unevaluated expressions `1+1`, `1+2`, `1+3`. Your suggestion of putting `str(Y)` inside `for(...) {...}` has me scratching my head even more. It looks as if the first expression is correctly captured as `1+1` in the first iteration, but then gets modified to `1+2` at the second iteration and finally becomes `1+3` by the time the loop finishes. However, further changes to `i` outside the loop have no effect. What gives? — Artem Sokolov, Aug 27 '18 at 13:24
This is called lazy evaluation. Check my answer for some ideas. — acylam, Aug 27 '18 at 15:46

Maurits Evers · Answer 1 · 2018-08-27T05:15:11.713

1

I'm not 100% sure I understood you correctly. Are you perhaps after rlang::eval_tidy to evaluate the expression?

X <- list()
for( i in 1:3 )
    X[[i]] <- rlang::eval_tidy(rlang::expr(!!i + 1))
#[[1]]
#[1] 2
#
#[[2]]
#[1] 3
#
#[[3]]
#[1] 4

edited Aug 27 '18 at 05:15

answered Aug 27 '18 at 05:07

Maurits Evers

49,617
4
47
68

Sorry for the confusion, Maurits. The intended goal is to capture unevaluated expressions `1 + 1`, `1 + 2`, and `1 + 3` using a loop. I updated the original question with expected output. – Artem Sokolov Aug 27 '18 at 13:14

score 1 · Accepted Answer · answered Aug 27 '18 at 13:56

It seems that for creates some sort of environment / frame where evaluation takes place at the end (for the last i). One possible way to deal with this is to avoid for and use purrr:: or lapply.

Y <- purrr::map(1:3, ~ rlang::expr( 1 + (!! .) )) 
str(Y)
#> List of 3
#>  $ : language 1 + 1L
#>  $ : language 1 + 2L
#>  $ : language 1 + 3L

Y <- lapply(1:3, function(i) rlang::expr( 1 + (!! i) )) 
str(Y)
#> List of 3
#>  $ : language 1 + 1L
#>  $ : language 1 + 2L
#>  $ : language 1 + 3L

while works in console but it fails when used with reprex (not shown).

Y <- list()
i <- 1L
while (i <= 3) {
    Y[[i]] <- rlang::expr( 1 + (!!i) )
    i <- i + 1L
}
str(Y)
#> List of 3
#>  $ : language 1 + 1L
#>  $ : language 1 + 2L
#>  $ : language 1 + 3L

score 1 · Answer 3 · answered Aug 27 '18 at 15:37

Not entirely sure if this is the case, but I think it has to do with lazy evaluation. The main idea is that R does not evaluate an expression when it is not used. In your example, rlang::expr( !!i ) is "self-evaluating" because an expression that represents a constant is the constant itself. This causes the following to evaluate at every iteration of the for loop:

X <- list()
for( i in 1:3 )
  X[[i]] <- rlang::expr( !!i )

Notice that the elements of X are no longer expressions but integers:

> str(X)
List of 3
 $ : int 1
 $ : int 2
 $ : int 3

identical(rlang::expr(1),1)
# [1] TRUE

Your second example, however, has rlang::expr( 1 + (!!i) ), which remains an expression at each iteration of the for loop without evaluation. Lazy evaluation causes R to only evaluate i at the end of the loop, which takes the last value of i. The way to fix this issue is to force the evaluation of i:

Y <- list()
for( i in 1:3 ){
  force(i)
  Y[[i]] <- rlang::expr( 1 + (!!i) )
}

> str(Y)
List of 3
 $ : language 1 + 1L
 $ : language 1 + 2L
 $ : language 1 + 3L

Note that lazy evaluation used to also affect functions like lapply as discussed in this question: Explain a lazy evaluation quirk, but it has since been fixed in R 3.2.0. Higher order functions like lapply now forces the arguments to the inner function. See @jhin's answer in the same question. This is why @Mike Badescu's lapply solution now works.

Thanks for this, @avid_useR. Very insightful! I suspected that lazy evaluation was at play, but what gives me pause is @Mike Badescu's suggestion of doing `for( i in 1:3 ) { Y[[i]] <- rlang::expr( 1 + (!!i) ); str(Y) }` I would expect that `str(Y)` forces evaluation of `!!i` the same way `force(i)` does, but perhaps that's not the case? I suppose `str` simply displays the content of its argument, without necessary forcing its evaluation? — Artem Sokolov, Aug 27 '18 at 16:39
@ArtemSokolov What you really want is to force `i` to evaluate instead of `Y` (force the argument to a function instead of the function itself). If you do `str(Y)`, lazy evaluation still applies because `Y` is still being modified at each iteration. Try instead: `for( i in 1:3 ) {Y[[i]] <- rlang::expr( 1 + (!!i) ); str(i) }`. This forces evaluation of `i` at each iteration. — acylam, Aug 27 '18 at 16:50

Unquoting the loop variable in `rlang::expr`

3 Answers3