1

sklearn fit, MinMax scaler etc run into memory error when I am running the jupyter notebook on ec2-instance of m5a.xlarge (16gb ram). Whereas when I run the same code on my local machine (Macbook air, 8gb RAM), the code runs fine although slowly.

I have checked that my python version is 64bit.

Am I missing something?

from sklearn import preprocessing

scaler = preprocessing.MinMaxScaler(copy=True)
df2_scaled = scaler.fit_transform(df2)

this gives following eror:

MemoryError                               Traceback (most recent call last)

This has been marked possible duplicate of another question here. Although I get the memory issue even if I run normal .py file. In both cases looking at outcome of 'top' command I can see more than 5gb of free memory

Hardik Malhotra
  • 114
  • 1
  • 10
  • see this: https://stackoverflow.com/questions/43866413/jupyter-notebook-only-memory-error-same-code-run-in-a-conventional-py-and-wo?rq=1 – seralouk Jun 15 '19 at 17:27
  • Possible duplicate of [Jupyter Notebook (only) Memory Error, same code run in a conventional .py and works](https://stackoverflow.com/questions/43866413/jupyter-notebook-only-memory-error-same-code-run-in-a-conventional-py-and-wo) – seralouk Jun 15 '19 at 17:27
  • I have updated my question with explanation @serafeim – Hardik Malhotra Jun 15 '19 at 18:55
  • Your MacBook can use swap. Add the same amount of swap to the EC2 instance and try again. – Sergey Kovalev Jun 16 '19 at 09:10

0 Answers0