3

I want to run a regression in statsmodels that uses categorical variables and clustered standard errors.

I have a dataset with columns institution, treatment, year, and enrollment. Treatment is a dummy, institution is a string, and the others are numbers. I've made sure to drop any null values.

df.dropna()    
reg_model = smf.ols("enroll ~ treatment + C(year) + C(institution)", df)
.fit(cov_type='cluster', cov_kwds={'groups': df['institution']})

I'm getting the following:

ValueError: The weights and list don't have the same length.

Is there a way to fix this so my standard errors cluster?

tower489
  • 45
  • 1
  • 1
  • 5
  • Turns out dropna() wasn't catching some nulls, which I had to replace using `fillna(, inplace=True)` and then it worked fine – tower489 Jan 24 '19 at 18:55
  • 1
    Your use of `dropna` is flawed. Without `inplace=True` argument, `df.dropna()` just returns a copy of your DataFrame without nulls - it doesn't save it to the `df` object. Moreover, if there are more variables than you listed but you only want to drop nulls among the subset in your regression, you need the `subset` argument too. You could instead `reg_model = smf.ols("enroll ~ treatment + C(year) + C(institution)", df.dropna(subset = ['enroll','treatment','year','institution']))` – Gene Burinsky Oct 15 '19 at 23:58

1 Answers1

3

You need cov_type='cluster' in fit.

cov_type is a keyword argument and not in the correct position when keywords are used as positional arguments. http://www.statsmodels.org/stable/generated/statsmodels.regression.linear_model.OLS.fit.html

In general, statsmodels does not guarantee backwards compatibility when keyword arguments are used as positional arguments, that is keyword positions might change in future versions.

However, I don't understand where the ValueError is coming from. Python has very informative tracebacks, and it is very useful when asking questions to add either the full traceback or at least the last few lines that show where the exception is raised.

Josef
  • 21,998
  • 3
  • 54
  • 67