4

Let's suppose x ~ Poisson(2.5); I would like to calculate something like E(x | x > 2).

I assumed that this could be done with the .dist.expect operator, i.e.:

D = stats.poisson(2.5)
cond_expect = D.dist.expect(lambda x: x, D.args,lb=2)

This evaluates to cond_expect = 2.29478750344

However, if I just calculate the mean of a random sample from that distribution

D = stats.poisson(2.5)
test = D.rvs(size = 100000)
empirical_expectation = np.mean(test[test>=2])

empirical_expectation evaluates to 3.20875563063.

If anyone could clarify what I'm misunderstanding about the API, it would be greatly appreciated.

torgos
  • 73
  • 1
  • 6

1 Answers1

6

The method expect takes a Boolean parameter conditional, which is False by default. Set it to True:

cond_expect = D.dist.expect(lambda x: x, D.args, lb=2, conditional=True)

returns 3.219839256818051 in agreement with empirical result.

What this does:

conditional : bool, optional

If true then the expectation is corrected by the conditional probability of the summation interval. The return value is the expectation of the function, func, conditional on being in the given interval (k such that ul <= k <= ub). Default is False.

So, if False then you get E(X if X >= 2 else 0) instead of conditional expectation, which is adjusted by division by P(X >= 2): E(X | X >= 2) = E(X if X >= 2 else 0) / P(X >= 2)

I don't know why you would ever want conditional=False when providing an upper or lower bound, but it's the default.

Community
  • 1
  • 1
  • To the last: the initial usage of the bounds was to help in numerical integration, e.g. https://mail.python.org/pipermail/scipy-user/2018-February/037405.html – Josef Feb 16 '18 at 12:38