Creating a dictionary with multiprocess

Question

I am trying to make a dictionary with dataframe as values using multiprocess module. I am using the instruction that was provided here. However,it outputs an empty dictionary. Any help is highly appreciated.

def parallel_main(summary_res_all_type,st,df_x,df_y,max_number):
   res_all 
   =modified_mlr.do_mlr(df_x,df_y,soil_type=st,max_namber=max_number)    
   summary_res_all_type[st]=pd.DataFrame(index=pars_keys_all,columns=[st] )
   summary_res_all_type[st]=res_all['dict_cor_par'].values()

if __name__ == '__main__':
   manager = Manager() 
   summary_res_all_type = manager.dict() 
   job = [Process(target=parallel_main, args=(summary_res_all_type, 
   i,df_x,df_y,max_number))  for i in set(df_x["soil_type"])] 
   _ = [p.start() for p in job]
   _ = [p.join() for p in job]
   print(summary_res_all_type)

This code is not a [complete example](https://stackoverflow.com/help/mcve); many variables are not initialized, so we are left to guess what they are. Post a complete minimal code, so that we can run it and then help you. — Ralf, Oct 05 '18 at 20:17

score 0 · Answer 1 · answered Oct 06 '18 at 10:29

0

Here is a shot in the dark but the following line:

res_all = modified_mlr.do_mlr(df_x,df_y,soil_type=st,max_namber=max_number)

looks like it has a typo max_namber=max_number. This would cause all the processes to fail with hidden exceptions.

answered Oct 06 '18 at 10:29

SargeATM

2,483
14
24

Creating a dictionary with multiprocess

1 Answers1