42

Is there a module for Python to open IBM SPSS (i.e. .sav) files? It would be great if there's something up-to-date which doesn't require any additional dll files/libraries.

Shoe
  • 74,840
  • 36
  • 166
  • 272
Lamps1829
  • 2,231
  • 3
  • 24
  • 32
  • possible duplicate of [Exporting to SPSS files in Python Django?](http://stackoverflow.com/questions/6463763/exporting-to-spss-files-in-python-django) If you want there is also a [recipe](http://code.activestate.com/recipes/577650-python-reader-for-spss-sav-files/) on active-state – Bakuriu Feb 01 '13 at 13:09
  • Hi, Bakuriu. It's not a duplicate, as I'm not referencing the Django framework, I'm talking about opening, as opposed to exporting/writing a file, and I mentioned the preference for something recent which doesn't require external libraries/dlls. There's some common ground between the questions, but they can elicit different, as well as similar, responses. Thanks for the link, but again, I'm trying to avoid dll files, if possible. – Lamps1829 Feb 01 '13 at 17:23
  • The other answer cites Django, but it actually has nothing to do with it. Since Exporting requires the ability to write a file, the chances that you can also read it are high. Reading around I strongly believe you have only one choice: use the `.dll` released by IBM. I can't find any open specification for that file format, which means that the only way to read those file is to use IBM's libraries. You can always try to reverse-engineer the format, but that would take much more time and effort. – Bakuriu Feb 02 '13 at 08:14
  • 1
    Thanks, Bakuriu. It's unfortunate, but as you said, it is looking likely that IBM's .dll release is the thing to use. – Lamps1829 Feb 02 '13 at 12:53

9 Answers9

41

I have released a python package "pyreadstat" that reads SPSS (sav, zsav and por), Stata and SAS files. It is a wrapper around the C library ReadStat so it is very fast. Readstat is the library used in the back of the R library Haven, which is widely used and very robust.

The package is autocontained. It does not require using R (no need to install an aditional application) and it does not depend on IBM dlls or other external libraries.

For example, in order to read a SPSS sav file you would do:

import pyreadstat

df, meta = pyreadstat.read_sav("/path/to/sav/file.sav")

df is a pandas dataframe. Meta contains metadata such as variable labels or value labels. read_sav reads both sav and zsav (compressed) files. There is also a function read_por for old por (portable) files.

You can find it here: https://github.com/Roche/pyreadstat

Otto Fajardo
  • 3,037
  • 1
  • 18
  • 26
  • That is why I love Python. Messed around a lot of places, finally thought lets use Python. And it worked the first time. Thanks. – mradul dubey Jun 13 '21 at 12:58
  • fantastic library with amazing performance! thanks a lot. – Martin Bucher Jun 22 '21 at 13:40
  • I am late to this, however I would like to add my two cents. For me, I am working on a remote server, and I occassionally end up breaking things when trying to pip install new packages, etc., so there was the ```readsav``` function in ```scipy.io``` that worked for me, and was already included. The other top answer of using ```pandas.rpy.common``` didn't work for me either, as apparently that wasn't an attribute that ```rpy``` included. – Steven Thomas Nov 08 '21 at 16:09
  • @StevenThomas notice however that scipy.io.readsav reads and IDL sav file, not an SPSS sav file (the topic of this thread). IDL is a completely different programming environment. Pyreadstat does not read IDL files, only SPSS. – Otto Fajardo Nov 09 '21 at 10:38
  • @OttoFajardo thanks for the clarification, I didn't realise there were different types of .sav files. Perhaps I should remove my comment? – Steven Thomas Nov 09 '21 at 11:41
  • many people get confused by that, so let's leave the comment, I would say – Otto Fajardo Nov 09 '21 at 14:06
13

When you have pandas >= 0.25.0 you can now finally just do pd.read_spss():

# you need pandas >= 0.25.0 for this    
import pandas as pd
df = pd.read_spss('your_spss_file.sav')

This has library pyreadstat as a requirement, so you might have to install that first:

pip install pyreadstat

Extra info on the parameters of pd.read_spss():

Parameters
----------
path : string or Path
File path

usecols : list-like, optional
Return a subset of the columns. If None, return all columns.

convert_categoricals : bool, default is True
Convert categorical columns into pd.Categorical.

Returns
-------
DataFrame

Sander van den Oord
  • 10,986
  • 5
  • 51
  • 96
11

Depending on what you want to do--process data using R-related commands from rpy2, or switch to Python--the solution provided by @Spacedman on a related thread might easily be adapted to suit your needs.

Otherwise, Pandas includes a convenient wrapper for rpy2. Here is an example of use with Peat and Barton's weights.sav data set:

>>> import pandas.rpy.common as com
>>> filename = "weights.sav"
>>> w = com.robj.r('foreign::read.spss("%s", to.data.frame=TRUE)' % filename)
>>> w = com.convert_robj(w)
>>> w.head()
     ID  WEIGHT  LENGTH  HEADC  GENDER  EDUCATIO              PARITY
1  L001    3.95    55.5   37.5  Female  tertiary  3 or more siblings
2  L003    4.63    57.0   38.5  Female  tertiary           Singleton
3  L004    4.75    56.0   38.5    Male    year12          2 siblings
4  L005    3.92    56.0   39.0    Male  tertiary         One sibling
5  L006    4.56    55.0   39.5    Male    year10          2 siblings
Community
  • 1
  • 1
chl
  • 27,771
  • 5
  • 51
  • 71
9

As a note for people findings this later (like me): pandas.rpyhas been deprecated in the newest versions of pandas (>0.16) as noted here. That page includes information on updating code to use the rpy2 interface.

Savage Henry
  • 1,990
  • 3
  • 21
  • 29
  • 3
    Thanks for the sharing this. So `com.convert_robj(rdf)` should be replaced with `pandas2ri.ri2py(rdf)`. But what about `com.robj.r('foreign::read.spss("%s", to.data.frame=TRUE)' % filename)`? – Pyderman Mar 28 '16 at 20:42
3

But the benefit of using the IBM libraries is that they get this rather complex binary file format right. They are free, relieve you of the burden of writing code for this format, and the license permits you to redistribute them. What more could you ask?

JKP
  • 5,419
  • 13
  • 5
3

Here're packages you probably interested in

4ilin
  • 143
  • 1
  • 5
2

I had the same question as @Pyderman about how to update this for pandas (>0.16). This is what I came up with:

from rpy2.robjects import pandas2ri, r
filename = 'weights.sav'
w = r('foreign::read.spss("%s", to.data.frame=TRUE)' % filename)
df = pandas2ri.ri2py(w)
df.head()
Courtney
  • 231
  • 1
  • 6
0

You could use a python interface to R and then import the data using read.spss in library(foreign).

Community
  • 1
  • 1
Jeromy Anglim
  • 33,939
  • 30
  • 115
  • 173
0
from scipy.io import readsav

import pandas as pd

sav_df = readsav('file.sav')

df=pd.DataFrame(sav_df)
jmoerdyk
  • 5,544
  • 7
  • 38
  • 49
Anisha
  • 1
  • 1
  • 4
    While this code may solve the question, [including an explanation](https://meta.stackexchange.com/q/114762) of how and why this solves the problem would really help to improve the quality of your post, and probably result in more up-votes. Remember that you are answering the question for readers in the future, not just the person asking now. Please [edit] your answer to add explanations and give an indication of what limitations and assumptions apply – jmoerdyk Apr 14 '23 at 15:22