More compound wise object sharing in python multiprocessing

Question

Can I share compound objects, like e.g. pandas dataframe between parallel processes in python multiprocessing?

score 1 · Accepted Answer · edited May 23 '17 at 11:43

1

You can try to use the multiprocessing.Manager, like it was described here.
There's also PySpark, which has its own dataframe structure with inherent parallelization, and it can be converted into pandas and vice versa.

edited May 23 '17 at 11:43

Community

1
1

answered Jul 04 '15 at 22:42

hellpanderr

5,581
3
33
43

More compound wise object sharing in python multiprocessing

1 Answers1