3

For the Gaussian Mixture model with k components:
enter image description here

Fisher information matrix is given by,

enter image description here

How do I calculate the Fisher Information Matrix? Is there an R function available for this calculation?

Débora
  • 5,816
  • 28
  • 99
  • 171
  • what R package/function have you used to get a model + data that is a gaussian mix? – Mossa Nov 07 '21 at 21:17
  • @Mossa, I m a newbie. I don't much idea about R package or lib to use. Thanks. – Débora Nov 09 '21 at 20:02
  • Please not that SO is for coding. I tend to believe this should be migrated to SE where the mathematical concept will be included before the code provided – Onyambu Nov 14 '21 at 23:22
  • This question seems to lack enough context. What have you tried? What have you searched for? – eduardokapp Nov 16 '21 at 18:10
  • The coding here will be incomplete. `g` is not given, `pi` is not given. etc. If you want the hessian, having assumed what `g` is, You will have to estimate ``the `pi` vector via EM, then you coudl use the `mle.tools` package to compute the I – Onyambu Nov 16 '21 at 18:28

1 Answers1

2

Take a look atmle.tools package. For example, in normal distributions,

library(mle.tools)
## Normal distribution
pdf <- quote(1 / (sqrt(2 * pi) * sigma) * exp(-0.5 / sigma ^ 2 * (x - mu) ^ 2))
lpdf <- quote(-log(sigma) - 0.5 / sigma ^ 2 * (x - mu) ^ 2)
x <- rnorm(n = 100, mean = 0.0, sd = 1.0)
expected.varcov(density = pdf, logdensity = lpdf, n = length(x), parms = c("mu", "sigma"),
                mle = c(mean(x), sd(x)), lower = '-Inf', upper = 'Inf')

$mle
         mu       sigma 
-0.01889498  0.96256386 

$varcov
                 mu         sigma
mu     9.265292e-03 -6.989460e-13
sigma -6.989460e-13  4.632646e-03




##Normal distribution
lpdf <- quote(-log(sigma) - 0.5 / sigma ^ 2 * (x - mu) ^ 2)
x <- rnorm(n = 100, mean = 0.0, sd = 1.0)
observed.varcov(logdensity = lpdf, X = x, parms = c("mu", "sigma"),
                mle = c(mean(x), sd(x)))

$mle
        mu      sigma 
0.02476464 1.12332365 

$varcov
                mu        sigma
mu    1.261856e-02 2.617048e-19
sigma 2.617048e-19 6.405360e-03

For more details, see mle.tools.pdf

Park
  • 14,771
  • 6
  • 10
  • 29
  • Thanks for your answer. Actually I m a newbie for stats. Can you please let me know how to define 'pdf' and 'lpdf' for the likelihood of the gaussian mixture model for my given formula above. – Débora Nov 08 '21 at 10:29
  • This is not for mixture models but rather for normal distribution. – Onyambu Nov 14 '21 at 23:21