capturing convergence message from lme4 package in R

Question

I was wondering if there is a way to write a logical test (TRUE/FALSE) to show whether a model from lme4 package has converged or not?

An example is shown below, I want to capture if any model comes with the convergence warning (i.e., Model failed to converge) message?

library(lme4)

dat <- read.csv('https://raw.githubusercontent.com/rnorouzian/e/master/nc.csv')

m <- lmer(math ~ ses*sector + (ses | sch.id), data = dat)

Warning message:
In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv,  :
  Model failed to converge with max|grad| = 0.00279 (tol = 0.002, component 1)

score 3 · Answer 1 · answered Dec 16 '21 at 13:22

3

> sm=summary(model)
> sm$optinfo$conv$lme4$messages
[1] "Model failed to converge with max|grad| = 0.0120186 (tol = 0.002, component 1)"
>

answered Dec 16 '21 at 13:22

Yossi Levy

79
1
5

2

Can you clarify what this adds to the existing answers? – Ben Bolker Dec 16 '21 at 15:52
I think that the previous answers are complicated with no apparent reason. Why would one use purrr, grep, write a special function, and all that jazz? It's just a matter of looking at a string in the model object. I read the existing answers and could not figure out what's going on there, at least at the beginning. I think that a simple answer is better. – Yossi Levy Dec 19 '21 at 07:23

Ronak Shah · Accepted Answer · 2020-09-29T08:16:10.420

We can use tryCatch, using withCallingHandlers taking inspiration from this post.

dat <- read.csv('https://raw.githubusercontent.com/rnorouzian/e/master/nc.csv')

m <- tryCatch({
          withCallingHandlers({
            error <- FALSE
            list(model = lmer(math ~ ses*sector + (ses | sch.id), data = dat),
                 error = error)
          },warning = function(w) {
              if(grepl('failed to converge', w$message)) error <<- TRUE
          }
          )})


m$model
#Linear mixed model fit by REML ['lmerMod']
#Formula: math ~ ses * sector + (ses | sch.id)
#   Data: dat
#REML criterion at convergence: 37509.07
#Random effects:
# Groups   Name        Std.Dev. Corr
# sch.id   (Intercept) 1.9053       
#          ses         0.8577   0.46
# Residual             3.1930       
#Number of obs: 7185, groups:  sch.id, 160
#Fixed Effects:
#(Intercept)          ses       sector   ses:sector  
#     11.902        2.399        1.677       -1.322  
#convergence code 0; 0 optimizer warnings; 1 lme4 warnings 

m$error
#[1] TRUE

The output m is a list with model and error elements.

If we need to test for warning after the model has been created we can use :

is_warning_generated <- function(m) {
  df <- summary(m)
  !is.null(df$optinfo$conv$lme4$messages) && 
           grepl('failed to converge', df$optinfo$conv$lme4$messages)
}

m <- lmer(math ~ ses*sector + (ses | sch.id), data = dat)
is_warning_generated(m)
#[1] TRUE

Thanks, Ronak! So for our logical test, can we write a function where the function gets a model like `m` and outputs `TRUE` if the model has that warning and `FALSE` otherwise? — rnorouzian, Sep 29 '20 at 07:56
Your solution needs the entire model to be retyped inside the list! Like I said I just need a function that accepts a model object like `m` in my question and checks whether there is that warning (return `TRUE`) or otherwise `FALSE`. — rnorouzian, Sep 29 '20 at 08:07
See updated answer to pass model `m` to function `is_warning_generated` — Ronak Shah, Sep 29 '20 at 08:19

score 1 · Answer 3 · answered Oct 01 '20 at 00:29

We can use safely from purrr. It will also return the error as a list element and captures the error. If there are no error, it will be NULL

library(purrr)
safelmer <- safely(lmer, otherwise = NA)
out <- safelmer(math ~ ses*sector + (ses | sch.id), data = dat)

score 1 · Answer 4 · answered Nov 06 '21 at 02:16

1

I'm just going to say that @RonakShah's is_warning_generated could be made slightly more compact:

function(m) { 
    w <- m@optinfo$conv$lme4$messages
    !is.null(w) && grepl('failed to converge', w) 
}

answered Nov 06 '21 at 02:16

Ben Bolker

211,554
25
370
453

score 0 · Answer 5 · edited Nov 04 '21 at 08:08

I applied Ronak's solution to my own simulation data and found a problem. The message may be a vector of multiple entries, leading also grepl() to have multiple entries. However, the && operator compares the string only to the first entry, such that further occurrences of 'failed to converge' are unobserved. To avoid this behavior, I changed && to &.

Now a problem occurred if there was no message at all. In this case the !is.null() part becomes correctly FALSE (i.e., no warning generated), but the grepl() part becomes logical(0) and the function value becomes FALSE & logical(0) which is logical(0). In fact it would work for FALSE && logical(0) which is FALSE (correct).

A solution that worked for me is if(is.null(mess)) FALSE else grepl('failed to converge', mess)

which in case of a failure to converge provides a vector with a TRUE at the entry where the warning was placed. This vector may be evaluated, for example, by building the numeric (or Boolean) sum which becomes greater 0 or TRUE.

Thanks to Dylan for editing! – Gerta Rücker Nov 05 '21 at 18:08 — Gerta Rücker, Nov 05 '21 at 18:08

capturing convergence message from lme4 package in R

5 Answers5

Linked

Related