2

Very newbie programmer, sorry if this is stupid or if my English is wrong. So, I have this command line address book that I am writing. It consists of a dictionary that holds an object with the key as the name variable, and each object has variables associated with it like the name of the person, the email, etc... It works, but now I'm trying to make it store the dictionary persistenly in memory using pickle. 

def create_person():
    """Adds an instance object of the Person class to the dictionary persons. persons is a global variable, that has been created previously. DATA is a variable that points to a file named test.data that exists in the same directory as the script."""
    name = raw_input("Enter the person's name here: ")
    email = raw_input("Enter the person's email here: ")    
    phone = raw_input("Enter the person's phone here: ")
    address = raw_input("Enter the person's address here: ")
    f = open(DATA, "rb")
    persons = pickle.load(f) #assign whatever is saved in the dictionary in persistent memory to global variable persons, which is empty at this point in the beginning
    f.close()
    persons[name] = Person(name, email, phone, address)    
    f = open(DATA, "wb")
    pickle.dump(persons, f)
    f.close()

However, I'm getting this error: 

Traceback (most recent call last):
File "testpickle.py", line 85, in <module>
  main()
File "testpickle.py", line 40, in main
  create_person()
File "testpickle.py", line 20, in create_person
  persons = pickle.load(f)
File "/home/pedro/anaconda/lib/python2.7/pickle.py", line 1378, in load
  return Unpickler(file).load()
File "/home/pedro/anaconda/lib/python2.7/pickle.py", line 858, in load
  dispatch[key](self)
File "/home/pedro/anaconda/lib/python2.7/pickle.py", line 880, in load_eof
  raise EOFError
EOFError

I don't understand this. I had actually already written this program, and it was working with memory saving, but I accidently deleted it. What is happening?

msw
  • 42,753
  • 9
  • 87
  • 112
chilliefiber
  • 571
  • 2
  • 7
  • 18
  • 2
    Are you sure the definition of class `Person` is exactly the same as it was before the crash? That's one possible cause. – martineau Jul 15 '15 at 19:11
  • 2
    I strongly suggest that you not use `pickle` for almost any purpose because the mechanism is very fragile. As an early Pythoner, the concept of the pickle interface was seductive: "I can save my objects and get them back with no effort". Unfortunately, because pickled data is implemented as programs for a simple stack machine, a single bit error will break the whole thing. Your code appears correct, but [silent errors](http://stackoverflow.com/questions/10263564/python-pickling-dictionary-eoferror) can invade. Use [sqlite3](https://docs.python.org/2/library/sqlite3.html) instead. – msw Jul 15 '15 at 19:16
  • @martineau the class Person isn't exactly the same, but nothing should change from it. All I did was remove one of the variables (relationship). – chilliefiber Jul 15 '15 at 19:19
  • @msw I don't know sql yet. I'm only a newbie programmer. I'm so sad with this situation, yesterday everything was just fine I have no idea what is happening. I don't even understand the damn error message ( I know what EOF is kinda, but the rest I don't understand) – chilliefiber Jul 15 '15 at 19:21

2 Answers2

4

pickle hitting EOF is a sure tell the file is corrupt (probably truncated, maybe a write operation failed).

If you upload it somewhere, we might be able to deduce what exactly is wrong with it. In the worst case, you'll have to peek inside and deduce the data that was there to re-input it by hand.

That's your price for using an unreadable format and not paying attention to its integrity (e.g. writing to another file and only moving it over the original one after saving succeeded).

UPDATE:

If you want to start from a new, "empty" file, then handle the case when the file is missing and produce an empty dict. After all, the file is supposed to be missing initially, isn't it?

An empty file is not valid pickle data (it has to at least contain information about the object's type).

Here's "one obvious way" ((c) Python Zen) to handle a missing file:

import errno    
try: f = open(DATA, "rb")
except IOError,e:
    if e[0]==errno.ENOENT: persons={}
    else: raise
else:
    persons = pickle.load(f)
    f.close()
    del f
Community
  • 1
  • 1
ivan_pozdeev
  • 33,874
  • 19
  • 107
  • 152
  • What? No. The file I had previously has been eliminated. DATA is a completely new file. I have no interest with what was in the file, nor with what is going to be in this new one. I'm just writing this to practice my python. – chilliefiber Jul 15 '15 at 19:23
  • 1
    `pickle` hitting `EOF` is a sure tell the file is corrupt. Why it is corrupt is another question. – ivan_pozdeev Jul 15 '15 at 19:25
  • What if I create a new file, something like newfile.data, put it in the same directory as the script, and then pointing DATA at it, will it work? – chilliefiber Jul 15 '15 at 19:26
  • Ok, just tried with a newly created file called newfile.data, get the EOF anyhow. It is with my script I think. – chilliefiber Jul 15 '15 at 19:28
  • 2
    You'd better handle the case when the file is missing and produce an empty `dict`. The file _is_ supposed to be missing initially, isn't it? – ivan_pozdeev Jul 15 '15 at 19:28
  • yes, in the beginning the file DATA is completely empty. What do you mean produce an empty dict? – chilliefiber Jul 15 '15 at 19:29
  • Ok, now I understand what you and the other answerer meant. I just need to check for the case that DATA is empty, and if it is empty, drop an empty dictionary there. – chilliefiber Jul 15 '15 at 19:35
  • It works now, I just had to check for the file's emptiness. Great! – chilliefiber Jul 15 '15 at 19:38
2

I don't see anything obviously wrong with your code, but if you just want to store key/value pairs like this, you probably ought to use the shelve module instead of a home-brewed solution.

import shelve

persons = shelve.open(DATA)

def create_person():
    """Adds an instance object of the Person class to the dictionary persons. persons is a global variable, that has been created previously. DATA is a variable that points to a file named test.data that exists in the same directory as the script."""
    name = raw_input("Enter the person's name here: ")
    email = raw_input("Enter the person's email here: ")    
    phone = raw_input("Enter the person's phone here: ")
    address = raw_input("Enter the person's address here: ")
    persons[name] = Person(name, email, phone, address)    

while more_people:
    create_person()

persons.close()
hosford42
  • 376
  • 2
  • 16
  • I honestly have no idea what is going wrong with pickle. Just yesterday, it was working fine. Then, I lost the working version, started from scratch today and it doesn't work, for reasons unknown. Should I post my whole code somewhere? Will that help? – chilliefiber Jul 15 '15 at 19:13
  • 2
    @pedromatias You may have corrupted your file by hitting Ctrl-C or closing the script window during execution. Try deleting the file and starting over with a clean slate. Also, I would recommend backing up the file regularly (or better, automatically), whether you choose to use pickle or shelve. You can have your script fall back to the latest good version in case the primary copy gets corrupted. – hosford42 Jul 15 '15 at 19:23
  • I don't care about what is in the file. I actually started with a new DATA file, the DATA file is new and empty. I don't care about the things I'm putting there, or with the things that were in the old file. This is just a project for me to work on to practice Python. I just want my code to work to learn, not because of what I'll put in the DATA file. The old DATA file only had the information of 1 person as a test, and I was that person xD – chilliefiber Jul 15 '15 at 19:25
  • 1
    @pedromatias An empty file is going to cause this error. The error you mentioned happens when the pickle module is trying to load the file and reaches the end before it sees what it is expecting. This will always happen with an empty file. Instead, create a new file by pickling an empty dictionary and dumping it to that path. – hosford42 Jul 15 '15 at 19:28
  • Ok, now I understand what you and the other answerer meant. I just need to check for the case that DATA is empty, and if it is empty, drop an empty dictionary there. – chilliefiber Jul 15 '15 at 19:35
  • 1
    Yes, only I would wrap the `open()` and `pickle.load()` statements in a try block and have it use an empty dictionary if either fails for any reason. – hosford42 Jul 15 '15 at 19:37
  • It works now, I just had to check for the file's emptiness. Great! – chilliefiber Jul 15 '15 at 19:38
  • I didn't use a try block, just a simple if statement in the beginning of the code, after the import statements and the defining of DATA and persons, where I do`if os.stat(DATA).st_size == 0:` and if so, I assign the empty persons to DATA. – chilliefiber Jul 15 '15 at 19:39
  • 1
    That will only handle the case where the file is empty. If the file doesn't exist, or has been corrupted as I described above, you will still get an exception. – hosford42 Jul 15 '15 at 19:41
  • actually, I wanted to make that a piece of this code. To allow for multiple address books to be written, ask the user for the path to the file. If the file doesn't exist, create the file. This should be possible in python, right? I believe os.mkdir creates a directory, something must create files. – chilliefiber Jul 15 '15 at 19:55
  • 1
    To create a file, just use open() with mode 'wb' instead of 'rb'. This will overwrite the file if it exists, though, so be careful. – hosford42 Jul 21 '15 at 18:57
  • yeah the file is working now! I just check to see if the file exists with os.path.isfile() and create it with open if it doesn't, and I also check to see if the file is empty with os.stat(DATA).st_size and put an empty dictionary if it is empty. – chilliefiber Jul 25 '15 at 14:02