I am running Sqoop 1.4 with Hadoop-2.7.3. And also using MySQL 5.7 as metastore of Hive-2.1.1. When running any Sqoop eval command or HDFS operation it is working fine. But when importing data from MySQL to HDFS it is generating following errors. At some point it is showing like :
"Container [pid=8424,containerID=container_1522677715514_0003_01_000002] is running beyond virtual memory limits. Current usage: 109.8 MB of 1 GB physical memory used; 2.1 GB of 2.1 GB virtual memory used. Killing container. "
But I have assigned 8Gb ram to my VM and There is still 23 GB free space in in VM HardDisk .And The data I want to import is of 3 line, like:
mysql> select * from mytbl;
+----+----------+
| id | name |
+----+----------+
| 1 | Himanshu |
| 2 | Sekhar |
| 3 | Paul |
+----+----------+
Sow how could it take my 2.1 GB virtual memory? How can I solve this?
Here is the log generated fromm Sqoop import command .
bigdata@bigdata:~$ sqoop import --connect jdbc:mysql://localhost/test --username root --password paul --table mytbl --target-dir /sqoop8
18/04/02 20:01:02 INFO sqoop.Sqoop: Running S**qoop version: 1.4.6**
18/04/02 20:01:02 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
18/04/02 20:01:02 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
18/04/02 20:01:02 INFO tool.CodeGenTool: Beginning code generation
18/04/02 20:01:03 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `mytbl` AS t LIMIT 1
18/04/02 20:01:03 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `mytbl` AS t LIMIT 1
18/04/02 20:01:03 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /home/bigdata/Work/hadoop-2.7.3
Note: /tmp/sqoop-bigdata/compile/72216fe6b30a45210956d41dc13e7906/mytbl.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
18/04/02 20:01:07 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-bigdata/compile/72216fe6b30a45210956d41dc13e7906/mytbl.jar
18/04/02 20:01:07 WARN manager.MySQLManager: It looks like you are importing from mysql.
18/04/02 20:01:07 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
18/04/02 20:01:07 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
18/04/02 20:01:07 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
18/04/02 20:01:07 INFO mapreduce.ImportJobBase: Beginning import of mytbl
18/04/02 20:01:08 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
18/04/02 20:01:11 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
18/04/02 20:01:11 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
18/04/02 20:01:19 WARN hdfs.DFSClient: Caught exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1252)
at java.lang.Thread.join(Thread.java:1326)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.closeResponder(DFSOutputStream.java:609)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.closeInternal(DFSOutputStream.java:577)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:573)
18/04/02 20:01:19 WARN hdfs.DFSClient: Caught exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1252)
at java.lang.Thread.join(Thread.java:1326)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.closeResponder(DFSOutputStream.java:609)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.endBlock(DFSOutputStream.java:370)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:546)
18/04/02 20:01:20 WARN hdfs.DFSClient: Caught exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1252)
at java.lang.Thread.join(Thread.java:1326)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.closeResponder(DFSOutputStream.java:609)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.closeInternal(DFSOutputStream.java:577)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:573)
18/04/02 20:01:23 WARN hdfs.DFSClient: Caught exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1252)
at java.lang.Thread.join(Thread.java:1326)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.closeResponder(DFSOutputStream.java:609)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.endBlock(DFSOutputStream.java:370)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:546)
18/04/02 20:01:23 WARN hdfs.DFSClient: DataStreamer Exception
java.nio.channels.ClosedByInterruptException
at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:478)
at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:63)
at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:159)
at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:117)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at java.io.DataOutputStream.flush(DataOutputStream.java:123)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:508)
18/04/02 20:01:24 WARN hdfs.DFSClient: Caught exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1252)
at java.lang.Thread.join(Thread.java:1326)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.closeResponder(DFSOutputStream.java:609)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.closeInternal(DFSOutputStream.java:577)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:573)
18/04/02 20:01:24 WARN hdfs.DFSClient: Caught exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1252)
at java.lang.Thread.join(Thread.java:1326)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.closeResponder(DFSOutputStream.java:609)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.closeInternal(DFSOutputStream.java:577)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:573)
18/04/02 20:01:24 WARN hdfs.DFSClient: Caught exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1252)
at java.lang.Thread.join(Thread.java:1326)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.closeResponder(DFSOutputStream.java:609)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.closeInternal(DFSOutputStream.java:577)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:573)
18/04/02 20:01:25 INFO db.DBInputFormat: Using read commited transaction isolation
18/04/02 20:01:25 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(`id`), MAX(`id`) FROM `mytbl`
18/04/02 20:01:26 WARN hdfs.DFSClient: Caught exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1252)
at java.lang.Thread.join(Thread.java:1326)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.closeResponder(DFSOutputStream.java:609)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.endBlock(DFSOutputStream.java:370)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:546)
18/04/02 20:01:27 INFO mapreduce.JobSubmitter: number of splits:3
18/04/02 20:01:29 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1522677715514_0003
18/04/02 20:01:31 INFO impl.YarnClientImpl: Submitted application application_1522677715514_0003
18/04/02 20:01:31 INFO mapreduce.Job: The url to track the job: http://bigdata:8088/proxy/application_1522677715514_0003/
18/04/02 20:01:31 INFO mapreduce.Job: Running job: job_1522677715514_0003
18/04/02 20:01:49 INFO mapreduce.Job: Job job_1522677715514_0003 running in uber mode : false
18/04/02 20:01:49 INFO mapreduce.Job: map 0% reduce 0%
18/04/02 20:02:19 INFO mapreduce.Job: Task Id : attempt_1522677715514_0003_m_000001_0, Status : FAILED
Container [pid=8438,containerID=container_1522677715514_0003_01_000003] is running beyond virtual memory limits. Current usage: 110.6 MB of 1 GB physical memory used; 2.1 GB of 2.1 GB virtual memory used. Killing container.
Dump of the process-tree for container_1522677715514_0003_01_000003 :
|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
|- 8547 8438 8438 8438 (java) 607 32 2246705152 27562 /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx200m -Djava.io.tmpdir=/home/bigdata/Work/hadoop-2.7.3/HadoopTemp/nm-local-dir/usercache/bigdata/appcache/application_1522677715514_0003/container_1522677715514_0003_01_000003/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000003 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 127.0.1.1 37965 attempt_1522677715514_0003_m_000001_0 3
|- 8438 8427 8438 8438 (bash) 0 1 13094912 750 /bin/bash -c /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx200m -Djava.io.tmpdir=/home/bigdata/Work/hadoop-2.7.3/HadoopTemp/nm-local-dir/usercache/bigdata/appcache/application_1522677715514_0003/container_1522677715514_0003_01_000003/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000003 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 127.0.1.1 37965 attempt_1522677715514_0003_m_000001_0 3 1>/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000003/stdout 2>/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000003/stderr
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
18/04/02 20:02:40 INFO mapreduce.Job: Task Id : attempt_1522677715514_0003_m_000001_1, Status : FAILED
Container [pid=8656,containerID=container_1522677715514_0003_01_000006] is running beyond virtual memory limits. Current usage: 102.4 MB of 1 GB physical memory used; 2.1 GB of 2.1 GB virtual memory used. Killing container.
Dump of the process-tree for container_1522677715514_0003_01_000006 :
|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
|- 8694 8656 8656 8656 (java) 520 17 2244608000 25476 /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx200m -Djava.io.tmpdir=/home/bigdata/Work/hadoop-2.7.3/HadoopTemp/nm-local-dir/usercache/bigdata/appcache/application_1522677715514_0003/container_1522677715514_0003_01_000006/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000006 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 127.0.1.1 37965 attempt_1522677715514_0003_m_000001_1 6
|- 8656 8654 8656 8656 (bash) 0 0 13094912 749 /bin/bash -c /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx200m -Djava.io.tmpdir=/home/bigdata/Work/hadoop-2.7.3/HadoopTemp/nm-local-dir/usercache/bigdata/appcache/application_1522677715514_0003/container_1522677715514_0003_01_000006/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000006 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 127.0.1.1 37965 attempt_1522677715514_0003_m_000001_1 6 1>/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000006/stdout 2>/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000006/stderr
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
18/04/02 20:02:44 INFO mapreduce.Job: Task Id : attempt_1522677715514_0003_m_000000_1, Status : FAILED
Container [pid=8708,containerID=container_1522677715514_0003_01_000007] is running beyond virtual memory limits. Current usage: 104.1 MB of 1 GB physical memory used; 2.1 GB of 2.1 GB virtual memory used. Killing container.
Dump of the process-tree for container_1522677715514_0003_01_000007 :
|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
|- 8746 8708 8708 8708 (java) 547 22 2244608000 25906 /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx200m -Djava.io.tmpdir=/home/bigdata/Work/hadoop-2.7.3/HadoopTemp/nm-local-dir/usercache/bigdata/appcache/application_1522677715514_0003/container_1522677715514_0003_01_000007/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000007 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 127.0.1.1 37965 attempt_1522677715514_0003_m_000000_1 7
|- 8708 8706 8708 8708 (bash) 1 0 13094912 745 /bin/bash -c /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx200m -Djava.io.tmpdir=/home/bigdata/Work/hadoop-2.7.3/HadoopTemp/nm-local-dir/usercache/bigdata/appcache/application_1522677715514_0003/container_1522677715514_0003_01_000007/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000007 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 127.0.1.1 37965 attempt_1522677715514_0003_m_000000_1 7 1>/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000007/stdout 2>/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000007/stderr
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
18/04/02 20:02:48 INFO mapreduce.Job: Task Id : attempt_1522677715514_0003_m_000002_1, Status : FAILED
Container [pid=8760,containerID=container_1522677715514_0003_01_000008] is running beyond virtual memory limits. Current usage: 108.3 MB of 1 GB physical memory used; 2.1 GB of 2.1 GB virtual memory used. Killing container.
Dump of the process-tree for container_1522677715514_0003_01_000008 :
|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
|- 8760 8758 8760 8760 (bash) 0 2 13094912 761 /bin/bash -c /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx200m -Djava.io.tmpdir=/home/bigdata/Work/hadoop-2.7.3/HadoopTemp/nm-local-dir/usercache/bigdata/appcache/application_1522677715514_0003/container_1522677715514_0003_01_000008/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000008 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 127.0.1.1 37965 attempt_1522677715514_0003_m_000002_1 8 1>/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000008/stdout 2>/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000008/stderr
|- 8800 8760 8760 8760 (java) 610 28 2246705152 26964 /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx200m -Djava.io.tmpdir=/home/bigdata/Work/hadoop-2.7.3/HadoopTemp/nm-local-dir/usercache/bigdata/appcache/application_1522677715514_0003/container_1522677715514_0003_01_000008/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000008 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 127.0.1.1 37965 attempt_1522677715514_0003_m_000002_1 8
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
18/04/02 20:03:05 INFO mapreduce.Job: Task Id : attempt_1522677715514_0003_m_000001_2, Status : FAILED
Container [pid=8851,containerID=container_1522677715514_0003_01_000010] is running beyond virtual memory limits. Current usage: 108.1 MB of 1 GB physical memory used; 2.1 GB of 2.1 GB virtual memory used. Killing container.
Dump of the process-tree for container_1522677715514_0003_01_000010 :
|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
|- 8851 8849 8851 8851 (bash) 2 0 13094912 749 /bin/bash -c /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx200m -Djava.io.tmpdir=/home/bigdata/Work/hadoop-2.7.3/HadoopTemp/nm-local-dir/usercache/bigdata/appcache/application_1522677715514_0003/container_1522677715514_0003_01_000010/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000010 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 127.0.1.1 37965 attempt_1522677715514_0003_m_000001_2 10 1>/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000010/stdout 2>/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000010/stderr
|- 8889 8851 8851 8851 (java) 582 27 2246705152 26928 /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx200m -Djava.io.tmpdir=/home/bigdata/Work/hadoop-2.7.3/HadoopTemp/nm-local-dir/usercache/bigdata/appcache/application_1522677715514_0003/container_1522677715514_0003_01_000010/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000010 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 127.0.1.1 37965 attempt_1522677715514_0003_m_000001_2 10
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
18/04/02 20:03:09 INFO mapreduce.Job: Task Id : attempt_1522677715514_0003_m_000000_2, Status : FAILED
Container [pid=8915,containerID=container_1522677715514_0003_01_000011] is running beyond virtual memory limits. Current usage: 109.6 MB of 1 GB physical memory used; 2.1 GB of 2.1 GB virtual memory used. Killing container.
Dump of the process-tree for container_1522677715514_0003_01_000011 :
|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
|- 8953 8915 8915 8915 (java) 599 30 2246705152 27300 /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx200m -Djava.io.tmpdir=/home/bigdata/Work/hadoop-2.7.3/HadoopTemp/nm-local-dir/usercache/bigdata/appcache/application_1522677715514_0003/container_1522677715514_0003_01_000011/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000011 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 127.0.1.1 37965 attempt_1522677715514_0003_m_000000_2 11
|- 8915 8913 8915 8915 (bash) 1 0 13094912 761 /bin/bash -c /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx200m -Djava.io.tmpdir=/home/bigdata/Work/hadoop-2.7.3/HadoopTemp/nm-local-dir/usercache/bigdata/appcache/application_1522677715514_0003/container_1522677715514_0003_01_000011/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000011 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 127.0.1.1 37965 attempt_1522677715514_0003_m_000000_2 11 1>/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000011/stdout 2>/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000011/stderr
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
18/04/02 20:03:12 INFO mapreduce.Job: Task Id : attempt_1522677715514_0003_m_000002_2, Status : FAILED
Container [pid=8978,containerID=container_1522677715514_0003_01_000012] is running beyond virtual memory limits. Current usage: 108.3 MB of 1 GB physical memory used; 2.1 GB of 2.1 GB virtual memory used. Killing container.
Dump of the process-tree for container_1522677715514_0003_01_000012 :
|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
|- 9016 8978 8978 8978 (java) 582 24 2246705152 26988 /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx200m -Djava.io.tmpdir=/home/bigdata/Work/hadoop-2.7.3/HadoopTemp/nm-local-dir/usercache/bigdata/appcache/application_1522677715514_0003/container_1522677715514_0003_01_000012/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000012 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 127.0.1.1 37965 attempt_1522677715514_0003_m_000002_2 12
|- 8978 8976 8978 8978 (bash) 0 2 13094912 741 /bin/bash -c /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx200m -Djava.io.tmpdir=/home/bigdata/Work/hadoop-2.7.3/HadoopTemp/nm-local-dir/usercache/bigdata/appcache/application_1522677715514_0003/container_1522677715514_0003_01_000012/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000012 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 127.0.1.1 37965 attempt_1522677715514_0003_m_000002_2 12 1>/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000012/stdout 2>/home/bigdata/Work/hadoop-2.7.3/logs/userlogs/application_1522677715514_0003/container_1522677715514_0003_01_000012/stderr
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
18/04/02 20:03:28 INFO mapreduce.Job: map 100% reduce 0%
18/04/02 20:03:29 INFO mapreduce.Job: Job job_1522677715514_0003 failed with state FAILED due to: Task failed task_1522677715514_0003_m_000001
Job failed as tasks failed. failedMaps:1 failedReduces:0
18/04/02 20:03:29 INFO mapreduce.Job: Counters: 12
Job Counters
Failed map tasks=10
Killed map tasks=2
Launched map tasks=12
Other local map tasks=12
Total time spent by all maps in occupied slots (ms)=256493
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=256493
Total vcore-milliseconds taken by all map tasks=256493
Total megabyte-milliseconds taken by all map tasks=262648832
Map-Reduce Framework
CPU time spent (ms)=0
Physical memory (bytes) snapshot=0
Virtual memory (bytes) snapshot=0
18/04/02 20:03:30 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
18/04/02 20:03:30 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 138.8135 seconds (0 bytes/sec)
18/04/02 20:03:30 INFO mapreduce.ImportJobBase: Retrieved 0 records.
18/04/02 20:03:30 ERROR tool.ImportTool: Error during import: Import job failed!
