Container exited with a non-zero exit code 1 error during mapreduce task

Question

On executing jar in hadoop, I get the following error:

16/11/04 18:32:59 INFO mapreduce.Job: Task Id : attempt_1478261728730_0005_m_000000_2, Status : FAILED
Exception from container-launch.
Container id: container_1478261728730_0005_01_000004
Exit code: 1
Stack trace: ExitCodeException exitCode=1: 
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
    at org.apache.hadoop.util.Shell.run(Shell.java:455)
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
    at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)


Container exited with a non-zero exit code 1

16/11/04 18:33:09 INFO mapreduce.Job:  map 100% reduce 0%

This is application log:

Native code library failed to load.
java.lang.UnsatisfiedLinkError: no opencv_java2411 in java.library.pathopencv_java2411

I don't know what it mean, can anybody help with this please?

Update your post with application logs – BruceWayne Nov 04 '16 at 13:45 — BruceWayne, Nov 04 '16 at 13:45
I have updated the post..@KrishnaChaitanyaReddy – Mahek Nov 05 '16 at 04:29 — Mahek, Nov 05 '16 at 04:29

score 2 · Accepted Answer · edited May 23 '17 at 10:27

2

You are missing opencv on your cluster nodes. See here for all the details on how to handle this.

Long story short though, you need to install opencv on your executors. You cannot really compile it into your job's .jar in a portable way since it's C and not Java code.

Update: Note that the environment on your Hadoop executors is set by your hadoop-env.sh. So it needs to contain a line like:

JAVA_LIBRARY_PATH=${JAVA_LIBRARY_PATH}:/etc/opencv/lib

edited May 23 '17 at 10:27

Community

1
1

answered Nov 05 '16 at 22:17

Armin Braun

3,645
1
17
33

Yes, I got to know that, I am already loading opencv native path, but somehow its not able to load it. @Armin Braun – Mahek Nov 06 '16 at 03:53
@Mahek 3 questions then :) 1. what Version of OpenCV did you install did you install? 2. Also did you do so on all executors? 3. How did you install it/ensure it's loaded? – Armin Braun Nov 06 '16 at 09:01
OpenCV-2.4.11 and I have posted my issue regarding loading of library here http://stackoverflow.com/questions/40435812/how-to-set-native-library-path-for-opencv-in-ubuntu?noredirect=1#comment68119207_40435812 @Armin Braun – Mahek Nov 06 '16 at 09:16
@Mahek see above edit, likely it's missing in your `hadoop-env.sh` – Armin Braun Nov 06 '16 at 09:19

Container exited with a non-zero exit code 1 error during mapreduce task

1 Answers1