Questions tagged [apply-async]
20 questions
2
votes
1 answer
Overridden __setitem__ call works in serial but breaks in apply_async call
I've been fighting with this problem for some time now and I've finally managed to narrow down the issue and create a minimum working example.
The summary of the problem is that I have a class that inherits from a dict to facilitate parsing of misc.…
Isaac Wolf
- 317
- 2
- 9
1
vote
1 answer
Python: if I wrap pool.apply_async commands in try...except, are they still executed in parallel?
I've taken over some code a former colleague wrote, which was frequently getting stuck when one or more parallelised functions through a NameError exception, which wasn't caught. (The parallelisation is handled by multiprocessing.Pool.) Because the…
CrowsNose
- 83
- 1
- 10
0
votes
0 answers
How do I use apply_async with a class method?
I am new to multiprocessing in python and am trying to update a shared dictionary in parallel using the apply_async method. But, when I call the class method that does this (fill_contig_matrix) with the apply_async method, it does not seem to do…
rvijay1302
- 1
- 1
0
votes
0 answers
the apply_assync is not working in certain documents
The apply_assync is not working. If i try to call the function outside of the apply it works perfectly, but in the apply_assync neither one of the functions works and the program blocks, not allowing me to manually stop it either, i have to restart…
Maria Carvalhosa
- 9
- 3
0
votes
0 answers
Why is my multiprocessing in python not performing the task?
I want to put the function abc running in parallel, and the task ppm.abc in the apply_async doesn't perform and I'm only getting the error_calback value.
I've tried to run the function outside the apply_async, and it runs perfectly. Does anyone know…
Maria Carvalhosa
- 9
- 3
0
votes
0 answers
python multiprocessing apply_async dosen't run in parallel
I got a problem when trying to use multiprocessing in python to prepare batchs of my data, I used a multiprocessing.Pool to start multiple processes each of which processed a batch of my data, and I used a multiprocessing.Manager to make a Queue for…
yi yang
- 11
- 2
0
votes
2 answers
multiprocessing.Pool's 'apply_async' in Python not running the target function and no error is displayed
I was using the 'apply_async' function in Python's 'multiprocessing' library, and using a 'Thread Pool' to run certain functions for my program.
The function sent as argument to 'apply_async' did not even execute. I was looking for an error, and did…
Abdullah Sheriff
- 21
- 3
0
votes
1 answer
how to "poll" python multiprocess pool apply_async
I have a task function like this:
def task (s) :
# doing some thing
return res
The original program is:
res = []
for i in data :
res.append(task(i))
# using pickle to save res every 30s
I need to process a lot of data and I don't…
arron
- 73
- 5
0
votes
1 answer
Python: apply_sync() and tqdm() printing new line
I have the following code that has 2 variables running 2 separate pools to process a bunch of tables and the progress should be reflected via the tqdm bar. I belive I have accomplished that but my problem now is there are new lines of progress bars…
bakaneko
- 1
- 2
0
votes
3 answers
How can I run two(or more) selenium's webdrivers at the same time in Python?
I'm trying to run two(or more) selenium webdrivers with Python at the same time
I have so far tried using Python's Multiprocessing module, I have used it this way:
def _():
sets = list()
pool = Pool()
for i in range(len(master)):
…
Zakaria
- 1
- 1
0
votes
0 answers
Why does it not execute my func using apply_async? - multiprocessing python
from multiprocessing import Pool
def function1(var1, var2):
print("function1 executing")
var3 = var1 + var2
return var3
if __name__ == "__main__":
pool = Pool()
pool.apply_async(function1, args=(7, 5))
…
Wurstlaster
- 1
- 1
0
votes
1 answer
Python: pass an argument in apply_async by reference
In the code below I am trying to acsess the dictionaries in the list S:
import multiprocessing
from multiprocessing import Pool, cpu_count
def test(s):
s["key"]+=1
print( s )
S = [ { "key": 0 } for i in range(2) ]
pool = Pool(processes = 2…
0
votes
0 answers
Celery: apply_async wait for response
There are two services running in my application (Srvice1, Servie2).
A request is sent from the client to Service1 and Service1 calls a task as follows.
result = my_task.apply_async (kwargs = data)
And my_task calls an operation on server…
Saeed
- 3,294
- 5
- 35
- 52
0
votes
0 answers
Python: Can not get results from simple apply_sync call
I'm trying to use apply_sync() inside a class object in order to process live signal from external sensors.
The processes are created but I'm stuck as soon as I'm trying to get() the results.
I've tried to use a function inside the class or…
Charles Chapus
- 5
- 1
- 4
0
votes
0 answers
UnboundLocalError: local variable 'date_pool_async_results' referenced before assignment
The following returns a list of the desired payload in a PDF for a single site:
import multiprocessing as mp
def process_dates(dates, serviceAreaId, serviceAreaName, siteId, siteName):
model_results = []
# for loop to process dates
return…
ericdwkim
- 37
- 5