0

I normally resolve this by allocating more memory, however sometimes I find mys elf allocating quite a lot and I wonder if there are other alternatives I can use to solve this, is there a way to make the map jobs smaller, perhaps in aggregate they use same memory, but not all the time

I am mostly trying to learn. I can provide an example but I am not asking about a query or a specific instance, more in general

user1617979
  • 2,370
  • 3
  • 25
  • 30
  • Normal way to resolve it is rewriting the query and increasing parallelizm on mappes and reducers. I'd suggest to use memory allocation as a last resort. Have a look at these answers: https://stackoverflow.com/a/48296562/2700344 https://stackoverflow.com/a/38475807/2700344 https://stackoverflow.com/a/51061613/2700344 https://stackoverflow.com/a/42842117/2700344 – leftjoin Sep 24 '18 at 16:40
  • thanks I will review – user1617979 Sep 27 '18 at 16:10

0 Answers0