6

I have the following panel stored in df:

state district year y constant x1 x2 time
0 01 01001 2009 12 1 0.956007 639673 1
1 01 01001 2010 20 1 0.972175 639673 2
2 01 01001 2011 22 1 0.988343 639673 3
3 01 01002 2009 0 1 0 33746 1
4 01 01002 2010 1 1 0.225071 33746 2
5 01 01002 2011 5 1 0.450142 33746 3
6 01 01003 2009 0 1 0 45196 1
7 01 01003 2010 5 1 0.427477 45196 2
8 01 01003 2011 9 1 0.854955 45196 3
  • y is the number of protests in each district
  • constant is a column full of ones
  • x1 is the proportion of the district's area covered by a mobile network provider
  • x2 is the population count in each district (note that it is fixed in time)

How can I run the following model in Python? model

Here's what I tried

# Transform `x2` to match model
df['x2'] = df['x2'].multiply(df['time'], axis=0)
# District fixed effects
df['delta'] = pd.Categorical(df['district'])
# State-time fixed effects
df['eta'] = pd.Categorical(df['state'] + df['year'].astype(str))
# Set indexes
df.set_index(['district','year'])

from linearmodels.panel import PanelOLS
m = PanelOLS(dependent=df['y'], exog=df[['constant','x1','x2','delta','eta']])

ValueError: exog does not have full column rank. If you wish to proceed with model estimation irrespective of the numerical accuracy of coefficient estimates, you can set rank_check=False.

What am I doing wrong?

Arturo Sbr
  • 5,567
  • 4
  • 38
  • 76

1 Answers1

7

I dug around the documentation and the solution turned out to be quite simple.

After setting the indexes and turning the fixed effect columns to pandas.Categorical types (see question above):

# Import model
from linearmodels.panel import PanelOLS

# Model
m = PanelOLS(dependent=df['y'],
             exog=df[['constant','x1','x2']],
             entity_effects=True,
             time_effects=False,
             other_effects=df['eta'])
m.fit(cov_type='clustered', cluster_entity=True)

That is, DO NOT pass your fixed effect columns to exog.

You should pass them to entity_effects (boolean), time_effects (boolean) or other_effects (pandas.Categorical).

Arturo Sbr
  • 5,567
  • 4
  • 38
  • 76
  • Thanks, are you saying that "delta" in this example should not go in "exog" but that it should go in "other_effects"? – asd Oct 31 '21 at 12:11
  • No. Delta are the "individuals" if you may. Hence, you make delta the index and set `entity_effects = True`. In my example, `eta` is passed as `other_effects`. – Arturo Sbr Oct 31 '21 at 13:23
  • Thank you. How could I see the formula of the model? I tried "mod.formula". It ran but nothing happens – asd Nov 01 '21 at 09:21
  • Perhaps you could try `print(mod.formula)`, but to be honest I don't know. – Arturo Sbr Nov 01 '21 at 14:08
  • You can only see the formula when you create a model using the formula interface (e.g. `PanelOLS.from_formula`). If you create it using `DataFrame`s then there is no formula. – Kevin S Nov 01 '21 at 23:18