0

I ran vowpal wabbit on a full dataset and got the final coefficients. And I ran the same data with batch learning ( glm in R ) method to get the coefficients. The coefficients I got from vowpal wabbit is hugely different from the batch learning coefficients.

I was under the impression that vowpal wabbit uses gradient descent algorithm for any given model ( squared loss, logistic loss ). So I expected that the end results would match to some extent. But one is in the order 10^-1 ( online ) and other is 10^4 ( batch ). Could someone please explain the difference? I even used multiple passes ( the same number of iterations batch learning used )

========================

some info: in glm I used binomial family and in vowpal wabbit I used loss_function logistic.

DarkHorse
  • 137
  • 8
  • did you use any kind of regularization in glm that could bound the weight magnitude? – Thomas Jungblut Aug 11 '15 at 13:51
  • The result I posted is without any regularization.. I even used lasso (glmnet with alpha=1).. But that too did not help.. the lowest coefficient in absolute terms is still > 50 – DarkHorse Aug 12 '15 at 04:59
  • with ridge (alpha = 0) also it is same.. – DarkHorse Aug 12 '15 at 05:14
  • This may be related to this answer: https://stackoverflow.com/questions/24822288/correctness-of-logistic-regression-in-vowpal-wabbit/24832382#24832382 -- If not, could you post a full reproducible example? A data-set and full command line in `vw`, full data & script in `R`. That would go a long way towards understanding the problem. If the data is big, you may trim it to any size that is sufficient to demonstrate the difference. Thanks. – arielf Aug 13 '15 at 05:52

0 Answers0