Python restoring an interrupt iteration of two lists

Question

Here the code:

import itertools
# import list from file
a = [line.strip() for line in open("real.txt", 'r')]
b = [line.strip() for line in open("imag.txt", 'r')]

# create file contain restore informations 
f = open("restore.txt","w+")
loop = 0
try:
    for r in itertools.product(a,b):
         loop +=1
         print "processing : "+ r[1] + ": "+ r[0] + "  ("+str(loop)+")"

except Exception, e:
    print str(e)
    f.write("iterazions seccession: " +str(loop)+"\n")
    f.write("real number : " +r[1]+"\n")
    f.write("imaginary unit: " + r[0]+ "\n")

example output

processing: 1 : 1i (1)
processing: 1 : 2i (2)
processing: 1 : 3i (3)
processing: 1 : 4i (4)
...
processing: 2000 : 174i (348000)
processing: 2000 : 175i (348001)
...and so forth
(it does all combinations of two lists)

the question is if an error stop the iteration, there's a way to restart the script from last iteration without start from scratch? I try saving the last value in a file but I do not know how to use it.

p.s. I know the cmath function for complex number but I’m more interested to restore issue

more specific

if an error stop the iteration at: processing: 2000 : 175i (348001) there is a way to restart the script from 2000 : 175i (348001) iteration?

Answer 1

Put the try/except INSIDE the for loop - if the except doesn't raise then the for loop will continue.

for r in itertools.product(a,b):
    try:
         loop +=1
         print "processing : "+ r[1] + ": "+ r[0] + "  ("+str(loop)+")"
             #enable for testing error
             #if loop == 15:
                 #loop = loop/0
    except Exception, e:
        print str(e)
        f.write("iterazions seccession: " +str(loop)+"\n")
        f.write("real number : " +r[1]+"\n")
        f.write("imaginary unit: " + r[0]+ "\n")

Answer 2

Typically you'll see the use of try ... except blocks, which are relatively inexpensive in Python.

This won't stop execution of your loop, as seen in this example (which can be applied to your situation):

numbers = [1, 3, 2, 5, 0, 4, 5]

for number in numbers:
  try:
    print(10/number)
  except ZeroDivisionError:
    print("Please don't divide by 0...")

You'll see the output to this in Python 2.7.11 is:

10
3
5
2
Please don't divide by 0...
2
2

If your code can throw multiple types of exceptions, you can chain them:

numbers = [1, 3, 2, 5, 0, 4, 5]

for number in numbers:
  try:
    print(10/number)
  except ZeroDivisionError:
    print("Please don't divide by 0...")
  except ValueError:
    print("Something something ValueError")
  except KeyError:
    print("Something something KeyError")

If you know all the types of exceptions your code can raise, you can handle them appropriately, which is typically a better solution than a general solution for exceptions.

---

General notes about your code:

Use more representative names for variables

• Using more representative names will help the user/maintainer understand what's going on with your code and help you catch bugs. a and b don't mean much in terms of real and imaginary numbers - why not use real_nums and imag_nums? Not much more typing and much more descriptive.

Use open(file) as f

• This will close your file automatically so you don't need to worry about it later, which can be problematic for writing/appending

Use enumerate(iterable)

• this will keep track of iteration number and the object at the iteration

Iterate over tuples

• You won't need to access with [] notation - you can just use the elements within the tuples directly.

Use format strings

• These will make your code look a little nicer when printing

Putting all this together:

import itertools
# import list from file

reals = [line.strip() for line in open("real.txt", 'r')]
imags = [line.strip() for line in open("imag.txt", 'r')]

# create file contain restore informations 
with open("restore.txt", "w+") as f:
    for i, (real, imag) in enumerate(itertools.product(reals, imags)):
        try:
            print "processing: {0}, {1}, ({2})".format(real, imag, i) 
            # Process data here

            #enable for testing error
            if i == 15:
                i = i/0
        except ZeroDivisionError:
            # Handle exception here
            continue
        except Exception as e:
            print str(e)
            f.write("iterazions seccession: {0}\n".format(i))
            f.write("real number: {0}\n".format(real))
            f.write("imaginary unit: {0}\n".format(imag))

Answer 3

My solution

so, my question was "how restart the script from last iteration without start from scratch?". First I try in the hard way and even if it works fine I realize a way much more simple than that (kiss right?). Here the code:

import itertools
import pickle  #for restoring last data session
fname = "restore.pck"

reals = [line.strip() for line in open("real.txt", 'r')]
imags = [line.strip() for line in open("imag.txt", 'r')]

seed_count = 0 #number last processing cycle
count_ses = 0  #number current session cycle
count_tot = 0  #number total cycle processing so far
it_tot=len(reals)*len(imgs)

ret = raw_input("recover last session?: y/n")
if ret =="y":
    with open(fname,"rb") as f:
        rec= pickle.load(f)
        print  "last session data {0} {1} {2}".format(rec[0],rec[1],rec[2])
        seed_count=rec[2]
        seed_img=rec[1]
        seed_rea=rec[0]
try:
 print "total iterations", it_tot

 for r in itertools.product(reals,imgs):
    # do nothing until you get last interrupted cycle
    count_tot+=1
    if count_tot >= seed_count:
    # and now start processing
     count_ses +=1
     print "processing: {0}, {1}, (total : {2}), (this session: {3}) {4}% )".format(r[0], r[1], count_tot, count_ses,(count_tot/len_tot*100))

except Exception as e:
    print str(e)
    #save data if an error occurred
    store = r[0], r[1], count_tot
    with open(fname,"wb") as f:
     pickle.dump(store,f,pickle.HIGHEST_PROTOCOL)
     f.close()

It simply let run the iteration without processing anything until It reaches the last processing cycle and then restart computing. Of course some "wiseguy" will say this is not the pythonic way or this code it's a mess, is not fancy like mine but this is my first python program, it works and I'm happy:)