0

i wanna use all cpu in a python script i find some code same :

def do_sum():
    min = 0
    max = 100000000
    while min < max:
        min += 1
        file = open('mytext.txt','a')
        file.write(str(min))
def main():
    q = Queue()
    p1 = Process(target=do_sum)
    p2 = Process(target=do_sum)
    p1.start()
    p2.start()
    r1 = q.get()
    r2 = q.get()
    print r1+r2

if __name__=='__main__':
    main()

but it's not match cpu together p1 start write from 1,2,3,4,5 .... and p2 not continue p2 also start from begin 1,2,3,4 so result is : 1122334455

how i can match 2 core of cpu together ? i want write file with fastest my PC can do it , i use i7 cpu ,how can i use all

thefourtheye
  • 233,700
  • 52
  • 457
  • 497
user2244393
  • 3
  • 4
  • 5
    You're bottleneck here is hardly cpu, most likely you're bound by file write speed. And note that file writing is not thread safe, you'll get mangled data without some sort of locks. – alko Dec 20 '13 at 13:38
  • possible realated to [Python: Why different threads get their own series of values from one generator?](http://stackoverflow.com/questions/20042534/python-why-different-threads-get-their-own-series-of-values-from-one-generator) – alko Dec 20 '13 at 13:39
  • i think this thread work same that code and have different values – user2244393 Dec 20 '13 at 14:01

1 Answers1

2

You need a lock mechanism : http://en.wikipedia.org/wiki/Lock_%28computer_science%29 and references for (min, max), not local copies. The multiprocessing lib has already a Lock() object to avoid overwriting and a Value() object to share a mutual state between several process.

from multiprocessing import Queue, Process, Lock,Value

def do_sum(id, counter, lock):
    MAX = 50
    while counter.value < MAX:    

        lock.acquire()
        counter.value += 1

        file = open('mytext.txt','a')
        file.write(str(counter.value))
        file.write("\n")
        file.close()

        lock.release()


def main():

    counter = Value('d', 0.0)
    lock = Lock()

    #f = open('mytext.txt','w')
    #f.close()
    print 'atat'
    q = Queue()
    p1 = Process(target=do_sum, args=(0, counter, lock,) )
    p2 = Process(target=do_sum, args=(1,counter, lock,) )
    p1.start()
    p2.start()
    r1 = q.get()
    r2 = q.get()
    print r1+r2

if __name__=='__main__':
    main()

Anyway, you can harness the power of your cpu all you want, the perfs' bottleneck of your algorithm is located in the I/O operations (which are inherently sequentials).

lucasg
  • 10,734
  • 4
  • 35
  • 57