0

I am working on a project that requires me to use a pre-trained Tensorflow model (MLP model) and make predictions. But while creating the executable file using PyInstaller, the size of the executable file becomes more that 2GB. I inspected and found that Tensorflow alone takes up approximately 1.3GB of that.

So my main question is whether I can reduce the size of this executable, by using an API for serving. However every solution that I see requires tensorflow as a dependency, which would require tensorflow hooks and make my executable larger.

The closest I found was this question in stackoverflow, Serve Tensorflow model without installing Tensorflow. However this requires that I already know what my model is, and since it uses numpy I can't use CUDA acceleration in the future. Also if there are other non-GPU optimizations in tensorflow (besides basic multithreading in numpy), I would be missing out on them if I use just numpy.

So is there a tensorflow API that can just serve the model and keep my executable small in size. Some additional information, this is how my hook-tensorflow.py looks like,

from PyInstaller.utils.hooks import collect_all

def hook(hook_api):
    packages = [
        'tensorflow',
        'tensorflow_core',
        'keras',
        'astor'
    ]
    for package in packages:
        datas, binaries, hiddenimports = collect_all(package)
        hook_api.add_datas(datas)
        hook_api.add_binaries(binaries)
        hook_api.add_imports(*hiddenimports)
Karthik Prakash
  • 153
  • 4
  • 11

1 Answers1

0

Is Tensorflow Lite what you are looking for?

'TensorFlow Lite is a mobile library for deploying models on mobile, microcontrollers and other edge devices'

https://www.tensorflow.org/lite

David Harris
  • 646
  • 3
  • 11
  • I don't think that is enough since, tflite requires tensorflow to be loaded. I think the best option is tflite-runtime, but that is only available for some platforms. – Karthik Prakash Aug 23 '22 at 06:14