Questions tagged [log-likelihood]

Only questions related to the implementation and usage of the mathematical function - log-Likelihood should use this tag.

Given a sample and a parametric family of distributions (i.e., a set of distributions indexed by a parameter) that could have generated the sample, the Likelihood is a function that associates to each parameter the probability (or probability density) of observing the given sample.

The log-Likelihood a function which is the natural logarithm of the Likelihood function

For many applications, the log-Likelihood, is more convenient to work with as compared to the Likelihood. This is because we are generally interested in where the Likelihood reaches its maximum value. Since the logarithm is a strictly increasing function, the logarithm of a function achieves its maximum value at the same points as the function itself, hence the log-likelihood can be used in place of the likelihood for maximum likelihood estimation and related techniques.

Finding the maximum of a function often involves taking the derivative of a function and solving for the parameter being maximized, and this is often easier when the function being maximized is a log-likelihood rather than the original likelihood function, because the probability of the conjunction of several independent variables is the product of probabilities of the variables and solving an additive equation is usually easier than a multiplicative one.

232 questions
10
votes
1 answer

Maximum Likelihood estimation for Inverse Gaussian distribution

I am trying to reproduce the mean squared error between the actual and estimated parameter 'tau' (for over a month :(). The estimated 'tau', namely 'tau_hat' is obtained through the maximum likelihood estimation (MLE) , shown below. The joint…
nashynash
  • 375
  • 2
  • 19
8
votes
1 answer

How to implement sentence-level log-likelihood in tensorflow?

I want to implement the sentence-level log-likelihood as described in Collobert et al., p. 14. To compute transition scores, I could use CRF, but I don't know how to integrate it in tensorflow. I thought about using…
anamar
  • 649
  • 2
  • 8
  • 16
7
votes
2 answers

Small bug in MATLAB R2017B LogLikelihood after fitnlm?

Background: I am working on a problem similar to the nonlinear logistic regression described in the link [1] (my problem is more complicated, but link [1] is enough for the next sections of this post). Comparing my results with those obtained in…
Alexandre C-L
  • 132
  • 11
6
votes
1 answer

Goodness-of-fit for fixed effect logit model using 'bife' package

I am using the 'bife' package to run the fixed effect logit model in R. However, I cannot compute any goodness-of-fit to measure the model's overall fit given the result I have below. I would appreciate if I can know how to measure the…
Eric
  • 528
  • 1
  • 8
  • 26
6
votes
1 answer

Should the "perplexity" (or "score") go up or down in the LDA implementation of Scikit-learn?

I'd like to know what does the perplexity and score means in the LDA implementation of Scikit-learn. Those functions are obscure. At the very least, I need to know if those values increase or decrease when the model is better. I've searched but it's…
Guillaume Chevalier
  • 9,613
  • 8
  • 51
  • 79
4
votes
2 answers

Looping over np.einsum many times... Is there a faster way?

I have a likelihood function that I am trying to sample with MCMC. I have used no for loops in the log likelihood itself, but I do call np.einsum() once. Here's a sample of what my current code looks like: A = np.random.rand(4,50,60,200) # Random…
Jsn
  • 107
  • 6
4
votes
1 answer

AttributeError: 'str' object has no attribute 'parameters' due to new version of sklearn

I am doing topic modeling using sklearn. While trying to get the log-likelihood from Grid Search output, I am getting the below error: AttributeError: 'str' object has no attribute 'parameters' I think I understand the issue which is:…
4
votes
1 answer

How should I write the loss function with Keras (and TensorFlow) for negative binomial distribution?

I am modeling variables with the negative binomial distribution. Instead of prediction expected mean value, I rather would like to model two parameters of the distribution. So the output layer of my neural network is composed of two neurons. For…
gugatr0n1c
  • 377
  • 6
  • 21
3
votes
1 answer

negative values for mean squared errors in sae package for R

I have been using "sae" package for R to use small area estimations with spatial fay-herriot models (SFH). Using different distance matrices I occasionally obtained negative values of Mean Squared Errors (MSE). The following link may reference a…
jsollari
  • 33
  • 4
3
votes
1 answer

Quick evaluation of binomial likelihood in Rcpp

I need to evaluate a large number of binomial likelihoods very quickly. Therefore, I am thinking of implementing this in Rcpp. One way to do it is the following: #include // [[Rcpp::depends(RcppArmadillo)]] using namespace…
yrx1702
  • 1,619
  • 15
  • 27
3
votes
1 answer

How to build AIC model selection table using mlogit models

I'm trying to build an AIC table for my candidate set of models in R, which were run using mlogit. I've used glm and glmer in the past, and have always used the package AICcmodavg and aictab to extract values and create a model selection table. This…
3
votes
1 answer

Calculating loglikelihood of distributions in Python

What is an easy way to calculate the loglikelihood of any distribution fitted to data?
dejoma
  • 394
  • 1
  • 6
  • 18
3
votes
2 answers

log_loss from sklearn gives nan, while tensorflow.losses.log_loss works

I have a binary classification problem. I am using the log_loss from tensorflow.losses.log_loss. To check, I use sklearn.metrics.log_loss. Most of the times, the two functions give the same result (only difference in dtype). In some instance, the…
skjerns
  • 1,905
  • 1
  • 16
  • 25
3
votes
0 answers

Hessian matrix with optim or numderiv package?

I do maximum likelihood estimation for a loglikelihood function of a poisson distribution. After the estimation i want to compute the standard errors of the coeffients. For that i need the hessian matrix. Now i dont know which function i should use…
Dima Ku
  • 243
  • 2
  • 11
3
votes
1 answer

Poisson Regression by hand

I want to do a poisson regression by hand and define a function that can be used for estimation of an arbitrary number of coefficients. I have 2 questions: First: How can I get a matrix of betas and don't have to write every beta explicity. I want…
Dima Ku
  • 243
  • 2
  • 11
1
2 3
15 16