140

I am trying to write a dictionary to a txt file. Then read the dict values by typing the keys with raw_input. I feel like I am just missing one step but I have been looking for a while now.

I get this error

File "name.py", line 24, in reading
    print whip[name]
TypeError: string indices must be integers, not str

My code:

#!/usr/bin/env python
from sys import exit

class Person(object):
    def __init__(self):
        self.name = ""
        self.address = ""
        self.phone = ""
        self.age = ""
        self.whip = {}

    def writing(self):
        self.whip[p.name] = p.age, p.address, p.phone
        target = open('deed.txt', 'a')
        target.write(str(self.whip))
        print self.whip

    def reading(self):
        self.whip = open('deed.txt', 'r').read()
        name = raw_input("> ")
        if name in self.whip:
            print self.whip[name]

p = Person()

while True:
    print "Type:\n\t*read to read data base\n\t*write to write to data base\n\t*exit to exit"
    action = raw_input("\n> ")
    if "write" in action:
        p.name = raw_input("Name?\n> ")
        p.phone = raw_input("Phone Number?\n> ")
        p.age = raw_input("Age?\n> ")
        p.address = raw_input("Address?\n>")
        p.writing()
    elif "read" in action:
        p.reading()
    elif "exit" in action:
        exit(0)
Yu Hao
  • 119,891
  • 44
  • 235
  • 294
Radioactive Head
  • 1,519
  • 2
  • 11
  • 11

6 Answers6

197

Have you tried the json module? JSON format is very similar to python dictionary. And it's human readable/writable:

>>> import json
>>> d = {"one":1, "two":2}
>>> json.dump(d, open("text.txt",'w'))

This code dumps to a text file

$ cat text.txt 
{"two": 2, "one": 1}

Also you can load from a JSON file:

>>> d2 = json.load(open("text.txt"))
>>> print d2
{u'two': 2, u'one': 1}
KFL
  • 17,162
  • 17
  • 65
  • 89
  • 6
    should you open the file for reading/writing in a `with` statement to ensure it closes after? – ecoe Jun 16 '14 at 12:11
  • 4
    I experienced problems using this method on some complex dictionaries such has having nested set values (e.g. `d = {"one": {"1" : 1, "a": {"i"}}}`). Therefore, I recommend @blender's pickle method that works in my case. – Gürol Canbek Jun 02 '16 at 08:02
  • 4
    Note that python's json translates dict keys to **strings** when doing json.dump, so `json.dump({1: 'one'}, file)` will read back as `{'1': 'one'}` when doing json.load. Better to use pickle if you are not going to read the saved file using other languages than Python. – igr Jan 23 '17 at 11:04
  • 1
    Is there a way to using something like json.dumps( dict_x, indent=4) to format the way the dict is written to the file to make it more legible? When I use your code the entire dict is written to one line, no matter how large it is, and it's not easily human readable. – herteladrian Oct 16 '17 at 07:33
  • Nvm! I figured it out. I read the file like so `dict_x = json.loads(database_file.read())` and write to it like so `database_file.write(json.dumps(dict_x, indent = 4))`. It's much more legible now. Perhaps I was using your code incorrectly and yours produced a legible text file from a large dict too but this worked for me for a large dict. `database_file` is a `File` object FYI – herteladrian Oct 16 '17 at 07:45
  • PSA: `json.loads` and `json.load` cannot decode multiple dicts from a file – starscream_disco_party Oct 25 '17 at 20:20
112

Your code is almost right! You are right, you are just missing one step. When you read in the file, you are reading it as a string; but you want to turn the string back into a dictionary.

The error message you saw was because self.whip was a string, not a dictionary. So you need to convert the string to a dictionary.

Example

Here is the simplest way: feed the string into eval(). Like so:

def reading(self):
    s = open('deed.txt', 'r').read()
    self.whip = eval(s)

You can do it in one line, but I think it looks messy this way:

def reading(self):
    self.whip = eval(open('deed.txt', 'r').read())

But eval() is sometimes not recommended. The problem is that eval() will evaluate any string, and if someone tricked you into running a really tricky string, something bad might happen. In this case, you are just running eval() on your own file, so it should be okay.

But because eval() is useful, someone made an alternative to it that is safer. This is called literal_eval and you get it from a Python module called ast.

import ast

def reading(self):
    s = open('deed.txt', 'r').read()
    self.whip = ast.literal_eval(s)

ast.literal_eval() will only evaluate strings that turn into the basic Python types, so there is no way that a tricky string can do something bad on your computer.

EDIT

Actually, best practice in Python is to use a with statement to make sure the file gets properly closed. Rewriting the above to use a with statement:

import ast

def reading(self):
    with open('deed.txt', 'r') as f:
        s = f.read()
        self.whip = ast.literal_eval(s)

In the most popular Python, known as "CPython", you usually don't need the with statement as the built-in "garbage collection" features will figure out that you are done with the file and will close it for you. But other Python implementations, like "Jython" (Python for the Java VM) or "PyPy" (a really cool experimental system with just-in-time code optimization) might not figure out to close the file for you. It's good to get in the habit of using with, and I think it makes the code pretty easy to understand.

steveha
  • 74,789
  • 21
  • 92
  • 117
  • `dict()` expects a dictionary or a list of tuples. It won't evaluate a string. – Blender Jun 14 '12 at 05:21
  • @Blender Oh, you are right and I'm tired. I'll edit the answer. – steveha Jun 14 '12 at 05:23
  • @Nick: Try running it first ;) – Blender Jun 14 '12 at 05:24
  • @Blender, thank you for the heads-up. And since I am tired I think I will stop posting on StackOverflow for the night! :-) – steveha Jun 14 '12 at 05:31
  • Thank you for your answer. Would you please give an example of a 'tricky' string on which using eval() is dangerous? – arman_aegit Jul 17 '22 at 19:09
  • @arman_aegit I did a quick search and found a StackOverflow answer giving examples: https://stackoverflow.com/a/37081082 The examples show how a correctly crafted string can run an arbitrary command (with the privileges of the user). Also it would be possible for it to overwrite any file that the user can write to, opening all sorts of privilege-escalation attacks. – steveha Aug 12 '22 at 06:29
75

To store Python objects in files, use the pickle module:

import pickle

a = {
  'a': 1,
  'b': 2
}

with open('file.txt', 'wb') as handle:
  pickle.dump(a, handle)

with open('file.txt', 'rb') as handle:
  b = pickle.loads(handle.read())

print a == b # True

Notice that I never set b = a, but instead pickled a to a file and then unpickled it into b.

As for your error:

self.whip = open('deed.txt', 'r').read()

self.whip was a dictionary object. deed.txt contains text, so when you load the contents of deed.txt into self.whip, self.whip becomes the string representation of itself.

You'd probably want to evaluate the string back into a Python object:

self.whip = eval(open('deed.txt', 'r').read())

Notice how eval sounds like evil. That's intentional. Use the pickle module instead.

Blender
  • 289,723
  • 53
  • 439
  • 496
  • no it gives me this error File "name.py", line 24, in reading print whip[name] TypeError: string indices must be integers, not str I thought p = Person() defined it – Radioactive Head Jun 14 '12 at 05:16
  • That text isn't in your code. Before asking a question, explain what's wrong. – Blender Jun 14 '12 at 05:17
  • 2
    _To store Python objects in files, use the pickle module_ Unless these are config files so you want them to be editable by human - see [Why bother with python and config files?](http://www.robg3d.com/?p=1000) – Piotr Dobrogost Jul 21 '12 at 16:44
  • 2
    Perfect. This is exactly what I need: to store native python structure state in a file that is not intended to be user editable. – bahamat Aug 13 '13 at 23:43
  • 1
    This method works perfectly for complex dictionaries as I described in my comment for the JSON method suggested by @KFL – Gürol Canbek Jun 02 '16 at 08:05
12

Hi there is a way to write and read the dictionary to file you can turn your dictionary to JSON format and read and write quickly just do this :

To write your date:

 import json

 your_dictionary = {"some_date" : "date"}
 f = open('destFile.txt', 'w+')
 f.write(json.dumps(your_dictionary))

and to read your data:

 import json

 f = open('destFile.txt', 'r')
 your_dictionary = json.loads(f.read())
Eric Leung
  • 2,354
  • 12
  • 25
8

I created my own functions which work really nicely:

def writeDict(dict, filename, sep):
    with open(filename, "a") as f:
        for i in dict.keys():            
            f.write(i + " " + sep.join([str(x) for x in dict[i]]) + "\n")

It will store the keyname first, followed by all values. Note that in this case my dict contains integers so that's why it converts to int. This is most likely the part you need to change for your situation.

def readDict(filename, sep):
    with open(filename, "r") as f:
        dict = {}
        for line in f:
            values = line.split(sep)
            dict[values[0]] = {int(x) for x in values[1:len(values)]}
        return(dict)
PascalVKooten
  • 20,643
  • 17
  • 103
  • 160
  • Great code if your values are not too long. My dictionary included multiple long strings which just got cut off with '...' – Anne Sep 14 '18 at 08:05
6

You can iterate through the key-value pair and write it into file

pair = {'name': name,'location': location}
with open('F:\\twitter.json', 'a') as f:
     f.writelines('{}:{}'.format(k,v) for k, v in pair.items())
     f.write('\n')
Aravind Krishnakumar
  • 2,727
  • 1
  • 28
  • 25