3

I keep getting this error when running some of my steps: Container [pid=5784,containerID=container_1482150314878_0019_01_000015] is running beyond physical memory limits. Current usage: 5.6 GB of 5.5 GB physical memory used; 10.2 GB of 27.5 GB virtual memory used. Killing container.

I searched over the web and people say to increase the memory limits. This error is after I already increased to the maximum allowed on the instance I'm using c4.xlarge. Can I get some assistance about this error and how to solve this?

Also, I don't understand why mapreduce will throw this error and won't just swap or even work slower but just continue to work ...

NOTE: This error started happening after I changed to a custom output compression so it should be related to that.

Thanks!

refaelos
  • 7,927
  • 7
  • 36
  • 55
  • Have you increased the memory on the VM or have you increased the heap space? If you don't increase the heap space it will continue with the errors – Ed Baker Dec 19 '16 at 19:54
  • Also - Check this link: https://stackoverflow.com/questions/21005643/container-is-running-beyond-memory-limits – Ed Baker Dec 19 '16 at 19:55
  • @EdBaker I did increase the heap space and mem on VM. Didn't help. What I don't understand is if it has so much more virtual mem to use why won't it use that instead of letting the physical mem fill up? – refaelos Dec 20 '16 at 07:58

0 Answers0