I am getting following error while trying to use tensorflow with parallel processing. I saw multiple answers like Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2 but, is not related to parallel processing. I tried and is not working in my case.
here is the piece of code I used to do parallel processing
from multiprocessing import Pool
feature_dic = {}
if __name__ == '__main__':
max_number_process = 30
p = Pool(max_number_process)
fet, result = p.map(get_feature_vector_average_matrix, list_compounds)
print(result)
p.close()
p.join()
If I call the function get_feature_vector_average_matrix, without parallel processing it give me the desired result and no warnings. Could you please help me on this. I am using anaconda. Thanks in advance.
Error
Using TensorFlow backend.
2019-06-10 10:53:49.553787: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2019-06-10 10:53:49.563624: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2394415000 Hz
2019-06-10 10:53:49.567835: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x559fa6cf2660 executing computations on platform Host. Devices:
2019-06-10 10:53:49.567876: I tensorflow/compiler/xla/service/service.cc:158] StreamExecutor device (0): <undefined>, <undefined>
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 0-39
OMP: Info #156: KMP_AFFINITY: 40 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 2 packages x 10 cores/pkg x 2 threads/core (20 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 0 maps to package 0 core 0 thread 0
OMP: Info #171: KMP_AFFINITY: OS proc 20 maps to package 0 core 0 thread 1
OMP: Info #171: KMP_AFFINITY: OS proc 1 maps to package 0 core 1 thread 0
OMP: Info #171: KMP_AFFINITY: OS proc 21 maps to package 0 core 1 thread 1
OMP: Info #171: KMP_AFFINITY: OS proc 2 maps to package 0 core 2 thread 0
OMP: Info #171: KMP_AFFINITY: OS proc 22 maps to package 0 core 2 thread 1
OMP: Info #171: KMP_AFFINITY: OS proc 3 maps to package 0 core 3 thread 0
OMP: Info #171: KMP_AFFINITY: OS proc 23 maps to package 0 core 3 thread 1
OMP: Info #171: KMP_AFFINITY: OS proc 4 maps to package 0 core 4 thread 0
OMP: Info #171: KMP_AFFINITY: OS proc 24 maps to package 0 core 4 thread 1
OMP: Info #171: KMP_AFFINITY: OS proc 5 maps to package 0 core 8 thread 0
OMP: Info #171: KMP_AFFINITY: OS proc 25 maps to package 0 core 8 thread 1
OMP: Info #171: KMP_AFFINITY: OS proc 6 maps to package 0 core 9 thread 0
OMP: Info #171: KMP_AFFINITY: OS proc 26 maps to package 0 core 9 thread 1
OMP: Info #171: KMP_AFFINITY: OS proc 7 maps to package 0 core 10 thread 0
OMP: Info #171: KMP_AFFINITY: OS proc 27 maps to package 0 core 10 thread 1
OMP: Info #171: KMP_AFFINITY: OS proc 8 maps to package 0 core 11 thread 0
OMP: Info #171: KMP_AFFINITY: OS proc 28 maps to package 0 core 11 thread 1
OMP: Info #171: KMP_AFFINITY: OS proc 9 maps to package 0 core 12 thread 0
OMP: Info #171: KMP_AFFINITY: OS proc 29 maps to package 0 core 12 thread 1
OMP: Info #171: KMP_AFFINITY: OS proc 10 maps to package 1 core 0 thread 0
OMP: Info #171: KMP_AFFINITY: OS proc 30 maps to package 1 core 0 thread 1
OMP: Info #171: KMP_AFFINITY: OS proc 11 maps to package 1 core 1 thread 0
OMP: Info #171: KMP_AFFINITY: OS proc 31 maps to package 1 core 1 thread 1
OMP: Info #171: KMP_AFFINITY: OS proc 12 maps to package 1 core 2 thread 0
OMP: Info #171: KMP_AFFINITY: OS proc 32 maps to package 1 core 2 thread 1
OMP: Info #171: KMP_AFFINITY: OS proc 13 maps to package 1 core 3 thread 0
OMP: Info #171: KMP_AFFINITY: OS proc 33 maps to package 1 core 3 thread 1
OMP: Info #171: KMP_AFFINITY: OS proc 14 maps to package 1 core 4 thread 0
OMP: Info #171: KMP_AFFINITY: OS proc 34 maps to package 1 core 4 thread 1
OMP: Info #171: KMP_AFFINITY: OS proc 15 maps to package 1 core 8 thread 0
OMP: Info #171: KMP_AFFINITY: OS proc 35 maps to package 1 core 8 thread 1
OMP: Info #171: KMP_AFFINITY: OS proc 16 maps to package 1 core 9 thread 0
OMP: Info #171: KMP_AFFINITY: OS proc 36 maps to package 1 core 9 thread 1
OMP: Info #171: KMP_AFFINITY: OS proc 17 maps to package 1 core 10 thread 0
OMP: Info #171: KMP_AFFINITY: OS proc 37 maps to package 1 core 10 thread 1
OMP: Info #171: KMP_AFFINITY: OS proc 18 maps to package 1 core 11 thread 0
OMP: Info #171: KMP_AFFINITY: OS proc 38 maps to package 1 core 11 thread 1
OMP: Info #171: KMP_AFFINITY: OS proc 19 maps to package 1 core 12 thread 0
OMP: Info #171: KMP_AFFINITY: OS proc 39 maps to package 1 core 12 thread 1
OMP: Info #250: KMP_AFFINITY: pid 11661 tid 11661 thread 0 bound to OS proc set 0
2019-06-10 10:53:49.586954: I tensorflow/core/common_runtime/process_util.cc:71] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.
OMP: Info #250: KMP_AFFINITY: pid 11794 tid 11794 thread 0 bound to OS proc set 0
OMP: Info #250: KMP_AFFINITY: pid 11796 tid 11796 thread 0 bound to OS proc set 0
OMP: Info #250: KMP_AFFINITY: pid 11795 tid 11795 thread 0 bound to OS proc set 0
UPDATES: Before I did not get any output. I tried to install tensorflow from source as explained in the link but did not help. Finally, to get the output I imported tensorflow inside the working function and run different session for each process and I get the output. But the warning is still there, but is working.
