Deploy python app to Heroku "Slug Size too large"

Question

I'm trying to deploy a Streamlit app written in python to Heroku. My whole directory is 4.73 MB, where 4.68 MB is my ML model. My requirements.txt looks like this:

absl-py==0.9.0
altair==4.0.1
astor==0.8.1
attrs==19.3.0
backcall==0.1.0
base58==2.0.0
bleach==3.1.3
blinker==1.4
boto3==1.12.29
botocore==1.15.29
cachetools==4.0.0
certifi==2019.11.28
chardet==3.0.4
click==7.1.1
colorama==0.4.3
cycler==0.10.0
decorator==4.4.2
defusedxml==0.6.0
docutils==0.15.2
entrypoints==0.3
enum-compat==0.0.3
future==0.18.2
gast==0.2.2
google-auth==1.11.3
google-auth-oauthlib==0.4.1
google-pasta==0.2.0
grpcio==1.27.2
h5py==2.10.0
idna==2.9
importlib-metadata==1.5.2
ipykernel==5.2.0
ipython==7.13.0
ipython-genutils==0.2.0
ipywidgets==7.5.1
jedi==0.16.0
Jinja2==2.11.1
jmespath==0.9.5
joblib==0.14.1
jsonschema==3.2.0
jupyter-client==6.1.1
jupyter-core==4.6.3
Keras-Applications==1.0.8
Keras-Preprocessing==1.1.0
kiwisolver==1.1.0
Markdown==3.2.1
MarkupSafe==1.1.1
matplotlib==3.2.1
mistune==0.8.4
nbconvert==5.6.1
nbformat==5.0.4
notebook==6.0.3
numpy==1.18.2
oauthlib==3.1.0
opencv-python==4.2.0.32
opt-einsum==3.2.0
pandas==1.0.3
pandocfilters==1.4.2
parso==0.6.2
pathtools==0.1.2
pickleshare==0.7.5
Pillow==7.0.0
prometheus-client==0.7.1
prompt-toolkit==3.0.4
protobuf==3.11.3
pyasn1==0.4.8
pyasn1-modules==0.2.8
pydeck==0.3.0b2
Pygments==2.6.1
pyparsing==2.4.6
pyrsistent==0.16.0
python-dateutil==2.8.0
pytz==2019.3
pywinpty==0.5.7
pyzmq==19.0.0
requests==2.23.0
requests-oauthlib==1.3.0
rsa==4.0
s3transfer==0.3.3
scikit-learn==0.22.2.post1
scipy==1.4.1
Send2Trash==1.5.0
six==1.14.0
sklearn==0.0
streamlit==0.56.0
tensorboard==2.1.1
tensorflow==2.1.0
tensorflow-estimator==2.1.0
termcolor==1.1.0
terminado==0.8.3
testpath==0.4.4
toml==0.10.0
toolz==0.10.0
tornado==5.1.1
traitlets==4.3.3
tzlocal==2.0.0
urllib3==1.25.8
validators==0.14.2
watchdog==0.10.2
wcwidth==0.1.9
webencodings==0.5.1
Werkzeug==1.0.0
widgetsnbextension==3.5.1
wincertstore==0.2
wrapt==1.12.1
zipp==3.1.0

When I push my app to Heroku, the message is:

remote: -----> Discovering process types
remote:        Procfile declares types -> web
remote:
remote: -----> Compressing...
remote:  !     Compiled slug size: 623.5M is too large (max is 500M).
remote:  !     See: http://devcenter.heroku.com/articles/slug-size
remote:
remote:  !     Push failed

How can my slug size be too large? Is it the size of the requirements? Then how is it possible to deploy a python app using tensorflow to Heroku after all? Thanks for the help!

Tensorflow can be run on Heroku. For very small application it can be okay but during runtime you will run into memory problem. You can solve the memory problem by paying for Heroku services thus getting more RAM. The resulting size of your slug is normal. Machine learning is complicated and requires a lot of code. It gets even bigger during runtime. — Tin Nguyen, Apr 06 '20 at 14:55
I would recommend to check the Heroku Slug that your build script produces. There could be an accidentally left over large file/folder (e.g. .git folder for large projects — JME, May 21 '20 at 21:29

score 59 · Accepted Answer · answered Jul 11 '20 at 09:43

59

I have already answered this here.

Turns out the Tensorflow 2.0 module is very large (more than 500MB, the limit for Heroku) because of its GPU support. Since Heroku doesn't support GPU, it doesn't make sense to install the module with GPU support.

Solution:

Simply replace tensorflow with tensorflow-cpu in your requirements.

This worked for me, hope it works for you too!

answered Jul 11 '20 at 09:43

Surya Chereddy

1,031
8
8

Thats great. I was dealing with the slug size by hours but just changing "tensorflow" by "tensorflow-cpu" in requirements the md decreased like 200 md. Thanks you! – Aibloy Oct 15 '20 at 13:46
Worked like a charm! Thanks a ton. – Sarthak Rana Dec 11 '20 at 06:35
1

This saved my life. I use Tensorflow 2.4.x before and it's size nearly 500 MB. Cange to `tensorflow-cpu` it's only 144 MB – Philip Purwoko Jan 23 '21 at 14:54
Hi, do i need to only update tensorflow in the requirements.txt? There are some other ones lik tensorboard,tensorboard-data-server, tensorboard-plugin-wit etc which are automatically installed when i installed tensorflow in my local – anandhu Jul 13 '21 at 14:57
Whatever you dream should come true. Thank you @SuryaChereddy – Blasanka Nov 20 '21 at 12:55

score 4 · Answer 2 · edited Oct 01 '21 at 13:14

4

In requirements.txt file, I replaced tensorflow==2.6.0 to tensorflow-cpu==2.6.0 and it worked perfectly

edited Oct 01 '21 at 13:14

Yaakov Bressler

9,056
2
45
69

answered Oct 01 '21 at 07:41

Vaibhav Taneja

41
1

Working one. It decreased almost 300MB for me, from 680 to 380 – KasRoudra May 05 '22 at 08:17

score 2 · Answer 3 · edited May 21 '20 at 17:19

2

Just replace your TensorFlow version to 2.0.0 by doing:
tensorflow==2.0.0
It has a much lighter whl file and will fit your memory limit. Also, you can use 1.7.0 or 1.5.0 versions.

edited May 21 '20 at 17:19

bhristov

3,137
2
10
26

answered May 21 '20 at 16:48

Googr

166
2
10

I am getting No matching distribution found for tensorflow==2.0.0 – anandhu Jul 13 '21 at 14:48

score 1 · Answer 4 · edited May 22 '20 at 18:57

1

just use tensorflow version 1.14 as it is almost half the size of tf2.0+ and servers most purposes.
In my case the slug size almost halved from tf1.14(580MB) at place of tf.2.1(280MB)

edited May 22 '20 at 18:57

Green

2,405
3
22
46

answered May 22 '20 at 16:21

Anant Dev

11
1

score -2 · Answer 5 · answered Apr 10 '20 at 14:37

-2

it means when downloaded all your requirements are summing upto 600mb but you have only 500mb of memory

answered Apr 10 '20 at 14:37

Vijay

1

Yeah, I know that. But I'm asking if there is any other possibility to deploy a Tensorflow app to Heroku without paying if the dependency alone kills the Slug size, but apparently it isn't – Noltibus Apr 11 '20 at 18:02

Deploy python app to Heroku "Slug Size too large"

5 Answers5

Solution:

Linked