0

I'm using this code as a template (KILLING IT section) https://stackoverflow.com/a/36962624/9274778

So I've solved this for now... changed the code to the following

import random
from time import sleep

def worker(i,ListOfData):
    print "%d started" % i
    #MyCalculations with ListOfData
    x = ListOfData * Calcs
    if x > 0.95:
        return ListOfDataRow, True
    else:
        return ListOfDataRow, False

callback running only in main

def quit(arg):
    if arg[1] == True:
        p.terminate()  # kill all pool workers

if __name__ == "__main__":
    import multiprocessing as mp
    Loops = len(ListOfData) / 25
    Start = 0
    End = 25
    pool = mp.Pool()
    for y in range(0,Loops)
        results = [pool.apply(worker, args=(i,ListOfData[x]),callback = quit) 

    for y in range(0,len(ListofData))]

        for c in results:
            if c[1] == True
                break

        Start = Start + 25
        End = End +25

So I chunk my data frame (assume for now my ListOfData is always divisible by 25) and send it off to the multiprocessing. I've found for my PC performance groups of 25 works best. If the 1st set doesn't return a TRUE, then I go to the next chunk.

I couldn't use the async method as it ran all at different times and sometimes I'd get a TRUE back that was further down the list (not what I wanted).

sethmac86
  • 9
  • 1
  • 2
  • 1
    Your code makes little sense and have many errors—it's not even close to functional—so it's very difficult to understand what you want it to do and fix it. Please [edit] your question and fix it. – martineau Jan 27 '18 at 00:32
  • There is no logial link between any condition and the fact of calling callback. From documentation: "When the result becomes ready callback is applied to it". You are asking to solve a wrong problem – dgan Jan 27 '18 at 01:47
  • Also, after looking the link you have provided, there is a clear difference between your worker function, and the once which have been shown. Your function returns immediatly, either returning ListOfData if x>0.95, either returning None. While the worker as shown in the link, It loops indefinitly (while True), and returns ONLY when x>0.95. So again, make your worker work correctly, and there is no need to fix the callback – dgan Jan 27 '18 at 01:52
  • Note that the reference warns about using this code in production. For the purpose you describe I don't see why not to stick with the clean solution described above **KILLING IT** – Darkonaut Jan 27 '18 at 02:32
  • Sorry for confusion... I'll edit the post for further clarity – sethmac86 Jan 27 '18 at 16:20

0 Answers0