expose_functions from brms throws $ operator error

Question

I was working on some code assembled by a colleague and ran into some problems with the expose_functions() function in brms. When I got the error (once I had made sure I hadn’t fat-fingered something), I started a new R session, made sure all of my packages, etc. were updated, and then tried it again - same error.

At this point, I went to the vignette (here) and ran that code - where the error repeated itself exactly.

Here are the relevant parts of the vignette:

# Data
data("cbpp", package = "lme4")

# Custom family definition
beta_binomial2 <- custom_family(
        "beta_binomial2", dpars = c("mu", "phi"),
        links = c("logit", "log"), lb = c(NA, 0),
        type = "int", vars = "vint1[n]")

# Stan function definition
stan_funs <- "
        real beta_binomial2_lpmf(int y, real mu, real phi, int T) {
                return beta_binomial_lpmf(y | T, mu * phi, (1 - mu) * phi);
        }
        int beta_binomial2_rng(real mu, real phi, int T) {
                return beta_binomial_rng(T, mu * phi, (1 - mu) * phi);
        }
        "

# Stan variables
stanvars <- stanvar(scode = stan_funs, block = "functions")

# Model fit
fit2 <- brm(
        incidence | vint(size) ~ period + (1|herd), 
        data = cbpp, 
        family = beta_binomial2, 
        stanvars = stanvars
)

# Expose functions
expose_functions(fit2, vectorize = TRUE)

None of this is original code - it’s taken directly from the vignette. When I run this, I generate 32 warnings and one error:

Error: $ operator is invalid for atomic vectors

This is exactly the same thing that happens when I run my other code. I’m sure it’s me and not the code, but I don’t see where I’ve gone wrong here.

Just in case it helps anyone find the solution, here is a list of the various warnings (unique warnings only). Most of them seem "harmless" (based on this), but some may not be.

In file included from file656664715164.cpp:8:
In file included from /Library/Frameworks/R.framework/Versions/4.1/Resources/library/RcppEigen/include/RcppEigen.h:25:
In file included from /Library/Frameworks/R.framework/Versions/4.1/Resources/library/RcppEigen/include/RcppEigenForward.h:40:
In file included from /Library/Frameworks/R.framework/Versions/4.1/Resources/library/RcppEigen/include/unsupported/Eigen/SparseExtra:51:
/Library/Frameworks/R.framework/Versions/4.1/Resources/library/RcppEigen/include/unsupported/Eigen/../../Eigen/src/Core/util/ReenableStupidWarnings.h:14:30: warning: pragma diagnostic pop could not pop, no matching push [-Wunknown-pragmas]
    #pragma clang diagnostic pop
                             ^

/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/stan/math/prim/err/check_matching_dims.hpp:33:8: warning: unused variable 'error' [-Wunused-variable]
  bool error = false;
       ^

/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/stan/math/prim/err/hmm_check.hpp:33:7: warning: unused variable 'n_transitions' [-Wunused-variable]
  int n_transitions = log_omegas.cols() - 1;
      ^

/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/stan/math/prim/fun/gp_matern52_cov.hpp:275:10: warning: unused variable 'neg_root_5' [-Wunused-variable]
  double neg_root_5 = -root_5;
         ^
 /Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/stan/math/prim/fun.hpp:183:
/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/stan/math/prim/fun/log_mix.hpp:86:13: warning: unused variable 'N' [-Wunused-variable]
  const int N = stan::math::size(theta);
            ^

/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/stan/math/prim/prob/double_exponential_cdf.hpp:82:10: warning: unused variable 'N' [-Wunused-variable]
  size_t N = max_size(y, mu, sigma);
         ^

/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/stan/math/prim/prob/gaussian_dlm_obs_rng.hpp:98:7: warning: unused variable 'n' [-Wunused-variable]
  int n = G.rows();  // number of states
      ^

/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/stan/math/prim/prob/hmm_marginal.hpp:26:13: warning: unused variable 'n_states' [-Wunused-variable]
  const int n_states = omegas.rows();
            ^
/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/stan/math/prim/prob.hpp:315:
/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/stan/math/prim/prob/std_normal_rng.hpp:23:22: warning: unused variable 'function' [-Wunused-variable]
  static const char* function = "std_normal_rng";
                     ^
/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/stan/math/prim/prob.hpp:335:
/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/stan/math/prim/prob/von_mises_cdf.hpp:72:10: warning: unused variable 'ck' [-Wunused-variable]
  double ck = 50;
         ^

/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/src/stan/io/dump.hpp:778:15: error: no member named 'validate_dims' in namespace 'stan::io'
    stan::io::validate_dims(*this, stage, name, base_type, dims_declared);
    ~~~~~~~~~~^

/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/src/stan/model/indexing/rvalue.hpp:610:13: warning: unused variable 'cols' [-Wunused-variable]
  const int cols = rvalue_index_size(idxs.tail_.head_, x_ref.cols());
            ^

/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/src/stan/model/indexing/rvalue_varmat.hpp:241:22: warning: unused variable 'x_cols' [-Wunused-variable]
  const Eigen::Index x_cols = x.cols();
                     ^

/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/stan/math/rev/core/operator_subtraction.hpp:88:43: warning: lambda capture 'b' is not used [-Wunused-lambda-capture]
                            [avi = a.vi_, b](const auto& vi) mutable {
                                          ^
/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/stan/math/rev/core/operator_minus_equal.hpp:24:16: note: in instantiation of function template specialization 'stan::math::operator-<double, nullptr>' requested here
  vi_ = (*this - b).vi_;
               ^
/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/stan/math/prim/core/complex_base.hpp:136:9: note: in instantiation of member function 'stan::math::var_value<double, void>::operator-=' requested here
    re_ -= x;
        ^
/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/stan/math/prim/core/operator_subtraction.hpp:24:5: note: in instantiation of function template specialization 'stan::math::complex_base<stan::math::var_value<double, void> >::operator-=<int>' requested here
  y -= rhs;
    ^
/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/stan/math/prim/core/operator_subtraction.hpp:55:20: note: in instantiation of function template specialization 'stan::math::internal::complex_subtract<std::__1::complex<stan::math::var>, int>' requested here
  return internal::complex_subtract(x, y);
                   ^
/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/stan/math/prim/fun/acosh.hpp:105:31: note: in instantiation of function template specialization 'stan::math::operator-<stan::math::var_value<double, void>, int>' requested here
  auto y = log(z + sqrt(z * z - 1));
                              ^
/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/stan/math/rev/fun/acosh.hpp:94:32: note: in instantiation of function template specialization 'stan::math::internal::complex_acosh<stan::math::var_value<double, void> >' requested here
  return stan::math::internal::complex_acosh(z);
                               ^

/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/stan/math/rev/core/operator_addition.hpp:84:43: warning: lambda capture 'b' is not used [-Wunused-lambda-capture]
                            [avi = a.vi_, b](const auto& vi) mutable {
                                          ^
/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/stan/math/rev/fun/grad_inc_beta.hpp:45:43: note: in instantiation of function template specialization 'stan::math::operator+<int, nullptr>' requested here
    grad_2F1(dF1, dF2, a + b, var(1.0), a + 1, z);
                                          ^

/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/stan/math/rev/core/operator_addition.hpp:84:43: warning: lambda capture 'b' is not used [-Wunused-lambda-capture]
                            [avi = a.vi_, b](const auto& vi) mutable {
                                          ^
/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/stan/math/prim/fun/trigamma.hpp:63:31: note: in instantiation of function template specialization 'stan::math::operator+<double, nullptr>' requested here
    value = -trigamma_impl(-x + 1.0) + square(pi() / sin(-pi() * x));
                              ^
/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/stan/math/rev/fun/trigamma.hpp:23:44: note: in instantiation of function template specialization 'stan::math::trigamma_impl<stan::math::var_value<double, void> >' requested here
inline var trigamma(const var& u) { return trigamma_impl(u); }

It works for me. When I run the function, I am immediately prompted with 'do you want to build the package'? If I choose 'yes', I get tons of warnings. If I want to read the output, I need to set it to an object and pull the object after it finishes. If I choose no, the function immediately returns the output with no errors or warnings. Recently, I answered a question on SO that was unrelated to `brms`, but the error was essentially from packages that used RcppEigen--but only affected Windows devices. Is your OS Windows? Are you asked about the build? Have you set it to an object? — Kat, Feb 18 '22 at 20:17
@Kat - I don't see any of that. I'm running on a Mac (OS 10.15.7). I am not asked about a build. — Karl Wolfschtagg, Feb 20 '22 at 01:01
On a hunch, add the code `library(RcppEigen)` before you call `expose_functions()`. When I did that (and it's not a package that requires compilation, so --*weird*--), I no longer received errors or the pop-up about a build. However, when I ran the function, it processed but didn't render any output. I had to set the results to an object and retrieve them that way. If that doesn't work, look for library conflicts. Perhaps try to only call those that you need (so in this case, only `brms` and `RcppEigen`). — Kat, Feb 20 '22 at 07:04
@Kat - If I remove the argument `vectorize = TRUE` then that line of code generates 27 errors (mostly different errors than before) and no errors. If I keep going in the vignette, the code fails two blocks down (in the `loo()` function). — Karl Wolfschtagg, Feb 21 '22 at 20:46
Those are the errors I see when I don't include the call to the `library(RcppEigen)`. I'm not sure if they're exactly the same, but they look the same. I didn't have to change the call `vectorize = T` though. Have you tried removing the argument and adding the library? What error happens when you get to the `loo()` function? The same one about vectors? — Kat, Feb 21 '22 at 21:34
I tried removing the argument and adding the library - same results. The error that the `loo()` function gives is `Error in beta_binomial2_lpmf(y, mu, phi, trials) : Expecting a single value: [extent=4000].` — Karl Wolfschtagg, Feb 21 '22 at 23:42

score 1 · Answer 1 · answered Feb 22 '22 at 16:19

I added this as an answer because it was just too much for the comment. I want to help you figure this out.

1) Versions

First, please compare the versions of brms and the dependencies. I know you wrote that you checked this, but these errors are odd! It can't hurt to compare them. It's probably important to note that I have a Mac (OS 12.2.1).

I used this code to gather the information; here are my versions.

# brms and it's dependencies
dPkg <- c("brms", "rstan", "ggplot2", "loo", "posterior", "Matrix", "mgcv", "rstantools",
          "bayesplot", "shinystan", "bridgesampling", "glue", "future", "matrixStats", 
          "nleqslv", "nlme", "coda", "abind", "stats", "utils", "parallel", "grDevices", 
          "backports")

cbind(dPkg, purrr::map(dPkg, ~as.character(packageVersion(.x))))
#       dPkg                      
#  [1,] "brms"           "2.16.3" 
#  [2,] "rstan"          "2.21.3" 
#  [3,] "ggplot2"        "3.3.5"  
#  [4,] "loo"            "2.4.1"  
#  [5,] "posterior"      "1.2.0"  
#  [6,] "Matrix"         "1.4.0"  
#  [7,] "mgcv"           "1.8.38" 
#  [8,] "rstantools"     "2.1.1"  
#  [9,] "bayesplot"      "1.8.1"  
# [10,] "shinystan"      "2.5.0"  
# [11,] "bridgesampling" "1.1.2"  
# [12,] "glue"           "1.6.1"  
# [13,] "future"         "1.24.0" 
# [14,] "matrixStats"    "0.61.0" 
# [15,] "nleqslv"        "3.3.2"  
# [16,] "nlme"           "3.1.155"
# [17,] "coda"           "0.19.4" 
# [18,] "abind"          "1.4.5"  
# [19,] "stats"          "4.1.1"  
# [20,] "utils"          "4.1.1"  
# [21,] "parallel"       "4.1.1"  
# [22,] "grDevices"      "4.1.1"  
# [23,] "backports"      "1.4.1"

2) `expose_functions` and `rstan`

I have found that expose_functions is a regurgitation of the function rstan::expose_stan_functions(), so I went there to look for answers.

I found this website. If you haven't seen it, you might find it pretty insightful. It doesn't help with the immediate issue, though.

In the rstan package, they had a straightforward example of this function:

library(rstan)
model_code <-
  '
  functions {
    real standard_normal_rng() {
      return normal_rng(0,1);
   }
  }
'
expose_stan_functions(stanc(model_code = model_code))

It doesn't give output, either (just like the issue I saw with expose_functions). I received a warning that it expected a list of integer vectors--but it worked. Does it work for you? If it does, we have to go back to the models.

I'm not sure what the real purpose of this function is. The output is the name of the function. So the output of this last bit is:

# [1] "standard_normal_rng"

And the output of expose_functions in your question is written in the code when stans_funs was created. This is the output of the original expose functions:

# [1] "beta_binomial2_lpmf" "beta_binomial2_rng"

When you run the library rstan, it tells you that you are recommended to add a few things to your script. I followed the recommendation.

options(mc.cores = parallel::detectCores()) # added as rec by lib rstan
rstan_options(auto_write = T)               # added as rec by lib rstan

3) Back to the models

If that call for the function from rstan works, then I would like to know what you got for the output of fit1 and fit2. I didn't think the entire output of summary() needs to be added, so I highlighted a few things from the model summaries:

Fit 1

family: binomial; link: mu = logit
draws: 4 chains w/ iter = 2K, warmup = 2K, thin = 1, post-warmup = 4K

Fit 2

family: beta_binomial2; links: mu = logit & phi = identity
draws are the same as fit1

As far as the loo function, that is derived from the package loo It may be better to wait to dive into that until the other parts are checked, eh?

Thank you again for all your help. Regarding #1 above, my versions are the same as yours with four exceptions: `stats`, `utils`, `parallel`, and `grDevices` are all version 4.1.2 on my machine - you report 4.1.1. I don't think that is the issue. — Karl Wolfschtagg, Feb 22 '22 at 18:05
In addition to this, I was also working on a very strange problem with `tidyverse` and `dbplyr`, the solution to which was (oddly enough) uninstalling RStudio and re-installing it. While I was writing up my response to your points above, the code magically started working - I didn't make any changes. The original statement with `expose_functions` and `vectorize = TRUE` works now. Completely frustrating since I have no idea what happened. Thanks for your help! — Karl Wolfschtagg, Feb 22 '22 at 20:50
Well, that's awesome, yet not! Glad it's working. Wish I knew why it wasn't to begin with! — Kat, Feb 22 '22 at 22:23

expose_functions from brms throws $ operator error

1 Answers1

1) Versions

2) expose_functions and rstan

3) Back to the models

2) `expose_functions` and `rstan`