7

I am working on a personal project in Python where I need some form of persistent data. The data would fit in 2-3 tables of 10-20 columns and 100-200 records each. I have a basic understanding of SQL, so a database seems to make some sense.

I am new to Python, so I am not familiar with the options for database interface from Python. I have also heard about pickling and am not sure if that would be a better solution for my project size. Can anyone recommend a good solution?

Brian Tompsett - 汤莱恩
  • 5,753
  • 72
  • 57
  • 129
astay13
  • 6,857
  • 10
  • 41
  • 56

6 Answers6

8

Or, if you just want to persist data between executions - for such a small data set you could have a look at the pickle module for persistency, and just load the data into memory during execution.

It's a simple solution - but for a personal project it might be enough.

ThomasH
  • 22,276
  • 13
  • 61
  • 62
Abizern
  • 146,289
  • 39
  • 203
  • 257
6

You should use sqlite3 module for this, it is included in Python.

Also you may want too look for an ORM solution.

Community
  • 1
  • 1
sorin
  • 161,544
  • 178
  • 535
  • 806
3

This sounds like very few data. An SQL DB might be overkill, especially with an ORM on top. I'd check whether JSON could do the job...

mhyfritz
  • 8,342
  • 2
  • 29
  • 29
  • what are the advantages of JSON over pickle? From my cursory understanding, it seems like it would be better to use Python's own pickle format. – astay13 May 28 '11 at 11:15
  • 1
    It's widely supported (i.e. practically platform/language-independent), plus it's human readable. – mhyfritz May 28 '11 at 11:18
  • But Json is not persistent on its own. You need to write and read it to a file/files, serialize/deserialize on reads/writes. Pickling might be faster on that. But pickling really pays when you have more complex data types to persist, like class instances. – ThomasH May 28 '11 at 11:41
  • @ThomasH Agreed. In case of small, simple data I would however opt for json - I just like the flexibility that comes with txt files, plus I might be a bit biased anyway as I use JS at times in addition... – mhyfritz May 28 '11 at 12:13
  • json is pretty much the exact same code as pickle, so I wouldn't necessarily choose one because it's "easier" or anything. Pickle can handle more types, but it's not as human readable as json and it's limited to python. You could also consider YAML, which is similar to json but even more human readable (IMO). – chmullig May 28 '11 at 12:27
1

I agree with using sqlite3. It is very easy to use, you don't need to worry about having to set up a database server. You should check out the SQLAlchemy library too.

josh-fuggle
  • 3,097
  • 2
  • 26
  • 27
1

The real question is really what kind of operations you want to do with your data.

As far as storage possibilities, the simplest solutions are indeed sqlite3 and pickle.

The solution that you will choose depends basically on whether using SQL or Python is the easiest way for you to manage your data. SQL is probably better at complex operations than Python, but Python is definitely more lightweight and simpler, and therefore is a good choice for simple operations. So, if using pickle+Python is too cumbersome, then sqlite3 is a very good choice.

Eric O. Lebigot
  • 91,433
  • 48
  • 218
  • 260
0

Peewee is another ORM that works with SQLite. It is an alternative to SQLAlchemy. If using SQLite, I would consider Peewee for pet projects and SQLAlchemy for professional work. I typically would not use SQLite directly.

Asclepius
  • 57,944
  • 17
  • 167
  • 143