0

I'm using multiprocessing to calculate the correlation between factor and returns month-by-month. But there is an error

TypeError: cannot pickle 'staticmethod' object

However, this is one of the solutions I searched to run multiprocess in class.

How should I correct this?

I'm using Python 3.9.

global calc_ic_help
@staticmethod
def calc_ic_help(month, df_factor, df_ret_1):
    print(month)
    month_filter = pd.Series(df_factor.index).apply(lambda x: x[:6] == month)
    month_filter = month_filter.values

    arr1 = df_factor[month_filter].values.flatten()
    arr2 = df_ret_1[month_filter].values.flatten()

    df1 = pd.DataFrame([arr1, arr2],
                       index=['factor', 'future_ret']).T

    df1.dropna(how='any', inplace=True)

    ic = df1.corr().iloc[0, 1]
    return ic


def calc_ic(self, df_factor):
    df_open = pd.read_csv(os.path.join(self.field_path, 'open.csv'),
                          index_col=0, header=0)
    df_open.index = df_open.index.astype('str')
    df_factor.index = df_factor.index.astype('str')
    df_ret = (df_open.shift(-1) / df_open) - 1
    df_ret_1 = df_ret.shift(-1)

    month_list = pd.Series(df_ret_1.index).apply(lambda x: x[:6]).unique()

    with concurrent.futures.ProcessPoolExecutor() as executor:
        results = [executor.submit(calc_ic_help, (i, df_factor, df_ret_1)) for i in month_list]
        for f in concurrent.futures.as_completed(results):
            print(f.result())
mkrieger1
  • 19,194
  • 5
  • 54
  • 65
Zhaoyan Zhou
  • 1
  • Does this answer your question? [Can't pickle static method - Multiprocessing - Python](https://stackoverflow.com/questions/21111106/cant-pickle-static-method-multiprocessing-python) – mkrieger1 Apr 28 '22 at 13:15
  • I don't really think so. Our code is different and I wanna use multiprocessing inside the class – Zhaoyan Zhou Apr 28 '22 at 13:38
  • Are you saying that defining a module-level function instead of a method, and then converting it to a static method, as shown [here](https://stackoverflow.com/a/21114450), doesn't work? Why not? Have you tried it? – mkrieger1 Apr 28 '22 at 13:57
  • Why are you using `staticmethod` outside a class at all? (And if you are not, please provide a [mcve].) – chepner Apr 28 '22 at 14:05
  • It's not outside, just because class code is large, i dont wanna show them all. These are all in one class – Zhaoyan Zhou Apr 28 '22 at 14:19

0 Answers0