1

I use two different models to to predict two different values. The prediction is computed by a CPU. My time is restrained in a way that one of the two predictions is too late when run sequentially. Is there a way to run two predictions parallel on a CPU?

I use Keras 2.3.1, Tensorflow 2.0.0 and Python 3.6.9.

My attempt so far (not as a functional code):

from tensorflow.keras.models import load_model
import concurrent.futures

model_one = load_model(path_to_model_one)
model_two = load_model(path_to_model_two)

with concurrent.futures.ProcessPoolExecutor() as executor:
    f1 = executor.submit(model_one.predict, [input_values_one])
    f2 = executor.submit(model_two.predict, [input_values_two])

prediction_one = f1.result()
prediction_two = f2.result()

I get this error:

Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/queues.py", line 234, in _feed
    obj = _ForkingPickler.dumps(obj)
  File "/usr/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <class 'tensorflow.python.keras.saving.saved_model.load.Model'>: attribute lookup Model on tensorflow.python.keras.saving.saved_model.load failed
Pm740
  • 339
  • 2
  • 12
  • Hi. Almost the same problem. Did you find any solution? – Vahid the Great Dec 27 '21 at 16:08
  • I think so, but my solution did not improved performance and might be outdated. I think that a scheduler inside of Keras still keeps everything single threaded. Still interested? – Pm740 Dec 27 '21 at 18:26
  • I'm not sure if I understood correctly. But could you please check my question here and see if these are relevant? https://stackoverflow.com/q/70496446/11411596 – Vahid the Great Dec 28 '21 at 12:51

0 Answers0