IDLE "not responding" when building a dictionary

Question

I am building a trigram tagging decodes with a pre-trained model I have a file that contains on each line (feature,weight) for tag features or trigram features. I need to build two dictionaries to access those values from my main program, the main program will loop through features and look-up the weights to use them in calculations in a Viterbi algorithm. The code below runs ( I inserted print statements, etc) but when it ends, IDLE says not responding and I have to break with control C. The file has 613 pages in Word and a total of 32K data pairs. Why is the program not responding? is there a more efficient way of doing this?

def tag_tri_form():
    f_tag=open('tag.model','r')
    tag_dic={}
    trigram_dic={}
    for line in f_tag:
        lines=line.split()
        if 'TAG' in lines[0]:
            tag_dic[lines[0]]=lines[1]
        else:
            trigram_dic[lines[0]]=lines[1]
    return tag_dic,trigram_dic

Maybe it's just a massive amount of memory to allocate, and IDLE can't handle it. Can you try running this in the terminal to see if that also hangs? — 2rs2ts, Jun 20 '13 at 17:28
Good call, it does not hang on the terminal after printing the dictionaries, but it is not practical to work in the cmd window, how can I make IDLE work? — freman1952, Jun 20 '13 at 19:32
"how can I make IDLE work?" ahahaha I'm dying! Well, [this SO question](http://stackoverflow.com/questions/10264874/python-reducing-memory-usage-of-dictionary) may give you some hints on how to reduce your memory overhead. — 2rs2ts, Jun 20 '13 at 19:35
Also, for what it's worth, you can work in the terminal just fine - edit your file in your text editor of choice, and leave a terminal in the directory in which your file is located. After making any edits and saving, just run the file again. (I do this 99% of the time on any *nix system, unless for some reason I need tkinter.) — 2rs2ts, Jun 20 '13 at 19:37
great article!, just to make sure I understand, when you said terminal, you mean the cmd window or a terminal emulator utility? — freman1952, Jun 20 '13 at 22:26

score 0 · Answer 1 · answered Mar 27 '17 at 13:39

To fix, unfortunately a shell is the best thing to use. I can confirm when trying to build a dictionary that contained over 70,000 characters that IDLE was not working and Terminal did (Mac OSX).

For others having this problem:

Open shell for your operating system
Go to appropriate folder (cd...)
python (py) nameofscript.py

IDLE "not responding" when building a dictionary

1 Answers1