2

In the R script Fit12_for_stack.R, I call rstan package's stan() function. When I run the Fit12_for_stack.R code in an interactive R session, I get these warning messages from stan():

Warning messages: 1: There were 13 divergent transitions after warmup. Increasing adapt_delta above 0.8 may help. 2: Examine the pairs() plot to diagnose sampling problems

When I run the script Fit12_for_stack.R on the command line with the command:

Rscript Fit12_for_stack.R

I get output, but not the warning messages. How can I capture the stan() warning messages when running the R script that calls stan() on the command line?

From the post How to save all console output to file in R?, I tried adding 

con <- file("test.log")
sink(con, append=TRUE)
sink(con, append=TRUE, type="message")

to the top of the script, but test.log again showed output, without stan() warning messages.

This is what Fit12_for_stack.R looks like:

con <- file("test.log")
sink(con, append=TRUE)
sink(con, append=TRUE, type="message")

library("rstan")

J <- 2
L <- 3

X <- matrix(c(98, 22, 42, 99, 68, 61), nrow = L, ncol = J)
N <- matrix(100, nrow = L, ncol = J)

fit <- stan(file="try8.stan",
            data=list(J, L, X, N),
            iter=100, chains=4, seed = 1)

This is what try8.stan looks like:

data{

   int<lower=0> J;
   int<lower=0> L;

   // Declare arrays with integer entries.

   int X[L,J];
   int N[L,J];

}

parameters {

   // Add parameters:
   // - pi_vec = [pi_1, ..., pi_L]
   // - C_vec = [C_11, ..., C_JJ]

   vector<lower=0,upper=1>[L] pi_vec;

   vector<lower=0,upper=1>[J] C_vec;

   matrix<lower=0,upper=1>[L,J] Alpha;

}

transformed parameters {

}

model {

    for (i in 1:L) {
        pi_vec[i] ~ uniform(0,1);
    }

    for (j in 1:J) {
        C_vec[j] ~ uniform(0,1);
    }

    for (i in 1:L) {
        for (j in 1:J) {

            Alpha[i,j] ~ normal(pi_vec[i], sqrt(C_vec[j]*(pi_vec[i])*(1 - pi_vec[i]))) T[0,1];

            // For the (Like = 1) test, have X[i,j] ~ U(0,1),
            // i.e. set the likelihood's density to 1 so that posterior density = prior density.

            X[i,j] ~ uniform(0,1);

        }
    }

}
Community
  • 1
  • 1
Qian Zhang
  • 41
  • 4
  • Newest update (2.12.1 on Sept. 10, 2016) says rstan now throws warnings even if not in interactive mode. That change might let you capture warnings now. – TJ Mahr Sep 13 '16 at 13:35
  • Thanks TJ. Did you check the latest version of the manual in response to this post, or is there a way to be automatically alerted every time they come out with a new manual? – Qian Zhang Nov 08 '16 at 17:17
  • 1
    I read the package news whenever I update RStan https://cran.r-project.org/web/packages/rstan/NEWS – TJ Mahr Feb 06 '17 at 14:39

2 Answers2

1

I tried adding stan(..., cores = 2) and the warning messages were logged. I skimmed through the source, and my guess (I could be wrong) is that when cores = 1, the warning is only thrown if R is in interactive mode (right at the very end of the script).

When cores is more than 1, sink() does not appear to log the output from the workers, but redirecting the output seems to work, e.g.

Rscript Fit12_for_stack.R  > test.Rout
Weihuang Wong
  • 12,868
  • 2
  • 27
  • 48
  • Thanks for commenting! I tried your solution like so: (1) "cp Fit12_for_stack.R Fit12_for_stack_Wong.R". (2) I deleted the top 3 lines of Fit12_for_stack_Wong.R involving con and sink(). (3) I added "cores = 2" to the stan() call. (4) I ran Rscript Fit12_for_stack_Wong.R > test_Wong.Rout, but the stan() warnings printed to the console and not to test_Wong.Rout. Maybe if I used qsub to submit "Rscript Fit12_for_stack_Wong.R > test_Wong.Rout" to a cluster, the output file for the qsub command would capture the stan() warnings, but I haven't tried that. – Qian Zhang Aug 25 '16 at 23:49
  • When I tried it, I retained the lines related to `sink()` (i.e. didn't do your step 2 above). Anyway, seems like your method works, so it's all good! – Weihuang Wong Aug 25 '16 at 23:53
  • Yup, and my mild adjustment of your method works too when sink() is removed. I did steps (1)-(4) and then qsub -q [queue name here] -j yes -e [folder name here] -o [folder name here] run_Wong.sh, where run_Wong.sh calls "Rscript Fit12_for_stack_Wong.R", and the output file run_Wong.sh.o1551988 gets saved to [folder name here] with stan() warnings. – Qian Zhang Aug 26 '16 at 00:28
0

So I removed the lines

con <- file("test.log")
sink(con, append=TRUE)
sink(con, append=TRUE, type="message")

from 

Fit12_for_stack.R

and ran the program with the command

R CMD BATCH Fit12_for_stack.R

This produced an output file

Fit12_for_stack.Rout

with the stan() warnings included.

Qian Zhang
  • 41
  • 4