I am getting error when importing MySQL data into HDFS through sqoop java. But i gave all permissions to hdfs then also i am getting the same issue. can anyone give solution to this problem. Hadoop version:2.6.0 Sqoop:1.4.6 Kindly suggest exactly what permissions i should give to the local file system. Thanks in advance.
16/12/28 13:14:54 WARN sqoop.ConnFactory: $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
16/12/28 13:14:54 INFO manager.SqlManager: Using default fetchSize of 1000
16/12/28 13:14:54 INFO tool.CodeGenTool: Beginning code generation
16/12/28 13:14:55 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM SHIPPER AS t WHERE 1=0
16/12/28 13:14:56 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM SHIPPER AS t WHERE 1=0
16/12/28 13:14:56 INFO orm.CompilationManager: HADOOP_HOME is /usr/local/hadoop
Note: /tmp/sqoop-hadoop/compile/779c01454836e023b4d99d8ebb4e3add/SHIPPER.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
16/12/28 13:14:57 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hadoop/compile/779c01454836e023b4d99d8ebb4e3add/SHIPPER.jar
16/12/28 13:14:57 INFO mapreduce.ImportJobBase: Beginning import of SHIPPER
16/12/28 13:14:58 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
16/12/28 13:14:58 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
16/12/28 13:14:58 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM SHIPPER AS t WHERE 1=0
16/12/28 13:14:58 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
16/12/28 13:14:58 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
16/12/28 13:14:58 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
16/12/28 13:14:58 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
16/12/28 13:14:59 INFO mapreduce.JobSubmitter: number of splits:1
16/12/28 13:14:59 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local1242973550_0001
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099728/avro-mapred-1.7.5-hadoop2.jar <- /home/hadoop/Workspaces/BigData/avro-mapred-1.7.5-hadoop2.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/avro-mapred-1.7.5-hadoop2.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099728/avro-mapred-1.7.5-hadoop2.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099729/kite-data-hive-1.0.0.jar <- /home/hadoop/Workspaces/BigData/kite-data-hive-1.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/kite-data-hive-1.0.0.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099729/kite-data-hive-1.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099730/parquet-format-2.0.0.jar <- /home/hadoop/Workspaces/BigData/parquet-format-2.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/parquet-format-2.0.0.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099730/parquet-format-2.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099731/sqoop-1.4.6-hadoop200.jar <- /home/hadoop/Workspaces/BigData/sqoop-1.4.6-hadoop200.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/sqoop-1.4.6-hadoop200.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099731/sqoop-1.4.6-hadoop200.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099732/parquet-common-1.4.1.jar <- /home/hadoop/Workspaces/BigData/parquet-common-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/parquet-common-1.4.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099732/parquet-common-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099733/sqoop-1.4.1-incubating.jar <- /home/hadoop/Workspaces/BigData/sqoop-1.4.1-incubating.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/home/hadoop/.m2/repository/org/apache/sqoop/sqoop/1.4.1-incubating/sqoop-1.4.1-incubating.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099733/sqoop-1.4.1-incubating.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099734/mysql-connector-java-5.1.40-bin.jar <- /home/hadoop/Workspaces/BigData/mysql-connector-java-5.1.40-bin.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/mysql-connector-java-5.1.40-bin.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099734/mysql-connector-java-5.1.40-bin.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099735/ojdbc6-11.jar <- /home/hadoop/Workspaces/BigData/ojdbc6-11.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/ojdbc6-11.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099735/ojdbc6-11.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099736/commons-logging-1.1.1.jar <- /home/hadoop/Workspaces/BigData/commons-logging-1.1.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/commons-logging-1.1.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099736/commons-logging-1.1.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099737/parquet-encoding-1.4.1.jar <- /home/hadoop/Workspaces/BigData/parquet-encoding-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/parquet-encoding-1.4.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099737/parquet-encoding-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099738/mysql-connector-java-5.1.40.jar <- /home/hadoop/Workspaces/BigData/mysql-connector-java-5.1.40.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/home/hadoop/.m2/repository/mysql/mysql-connector-java/5.1.40/mysql-connector-java-5.1.40.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099738/mysql-connector-java-5.1.40.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099739/opencsv-2.3.jar <- /home/hadoop/Workspaces/BigData/opencsv-2.3.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/opencsv-2.3.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099739/opencsv-2.3.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099740/snappy-java-1.0.5.jar <- /home/hadoop/Workspaces/BigData/snappy-java-1.0.5.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/snappy-java-1.0.5.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099740/snappy-java-1.0.5.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099741/commons-codec-1.4.jar <- /home/hadoop/Workspaces/BigData/commons-codec-1.4.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/commons-codec-1.4.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099741/commons-codec-1.4.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099742/parquet-column-1.4.1.jar <- /home/hadoop/Workspaces/BigData/parquet-column-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/parquet-column-1.4.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099742/parquet-column-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099743/jackson-mapper-asl-1.9.13.jar <- /home/hadoop/Workspaces/BigData/jackson-mapper-asl-1.9.13.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/jackson-mapper-asl-1.9.13.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099743/jackson-mapper-asl-1.9.13.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099744/jackson-databind-2.3.1.jar <- /home/hadoop/Workspaces/BigData/jackson-databind-2.3.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/jackson-databind-2.3.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099744/jackson-databind-2.3.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099745/parquet-generator-1.4.1.jar <- /home/hadoop/Workspaces/BigData/parquet-generator-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/parquet-generator-1.4.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099745/parquet-generator-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099746/commons-io-2.4.jar <- /home/hadoop/Workspaces/BigData/commons-io-2.4.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/commons-io-2.4.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099746/commons-io-2.4.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099747/commons-jexl-2.1.1.jar <- /home/hadoop/Workspaces/BigData/commons-jexl-2.1.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/commons-jexl-2.1.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099747/commons-jexl-2.1.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099748/jackson-core-2.3.1.jar <- /home/hadoop/Workspaces/BigData/jackson-core-2.3.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/jackson-core-2.3.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099748/jackson-core-2.3.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099749/parquet-avro-1.4.1.jar <- /home/hadoop/Workspaces/BigData/parquet-avro-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/parquet-avro-1.4.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099749/parquet-avro-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099750/kite-data-mapreduce-1.0.0.jar <- /home/hadoop/Workspaces/BigData/kite-data-mapreduce-1.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/kite-data-mapreduce-1.0.0.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099750/kite-data-mapreduce-1.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099751/avro-1.7.5.jar <- /home/hadoop/Workspaces/BigData/avro-1.7.5.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/avro-1.7.5.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099751/avro-1.7.5.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099752/ant-contrib-1.0b3.jar <- /home/hadoop/Workspaces/BigData/ant-contrib-1.0b3.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/ant-contrib-1.0b3.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099752/ant-contrib-1.0b3.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099753/paranamer-2.3.jar <- /home/hadoop/Workspaces/BigData/paranamer-2.3.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/paranamer-2.3.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099753/paranamer-2.3.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099754/slf4j-api-1.6.1.jar <- /home/hadoop/Workspaces/BigData/slf4j-api-1.6.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/slf4j-api-1.6.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099754/slf4j-api-1.6.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099755/parquet-jackson-1.4.1.jar <- /home/hadoop/Workspaces/BigData/parquet-jackson-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/parquet-jackson-1.4.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099755/parquet-jackson-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099756/xz-1.0.jar <- /home/hadoop/Workspaces/BigData/xz-1.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/xz-1.0.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099756/xz-1.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099757/parquet-hadoop-1.4.1.jar <- /home/hadoop/Workspaces/BigData/parquet-hadoop-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/parquet-hadoop-1.4.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099757/parquet-hadoop-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099758/jackson-core-asl-1.9.13.jar <- /home/hadoop/Workspaces/BigData/jackson-core-asl-1.9.13.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/jackson-core-asl-1.9.13.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099758/jackson-core-asl-1.9.13.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099759/kite-hadoop-compatibility-1.0.0.jar <- /home/hadoop/Workspaces/BigData/kite-hadoop-compatibility-1.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/kite-hadoop-compatibility-1.0.0.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099759/kite-hadoop-compatibility-1.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099760/kite-data-core-1.0.0.jar <- /home/hadoop/Workspaces/BigData/kite-data-core-1.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/kite-data-core-1.0.0.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099760/kite-data-core-1.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099761/jackson-annotations-2.3.0.jar <- /home/hadoop/Workspaces/BigData/jackson-annotations-2.3.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/jackson-annotations-2.3.0.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099761/jackson-annotations-2.3.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099762/ant-eclipse-1.0-jvm1.2.jar <- /home/hadoop/Workspaces/BigData/ant-eclipse-1.0-jvm1.2.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/ant-eclipse-1.0-jvm1.2.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099762/ant-eclipse-1.0-jvm1.2.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099763/commons-compress-1.4.1.jar <- /home/hadoop/Workspaces/BigData/commons-compress-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/commons-compress-1.4.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099763/commons-compress-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099764/hsqldb-1.8.0.10.jar <- /home/hadoop/Workspaces/BigData/hsqldb-1.8.0.10.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/hsqldb-1.8.0.10.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099764/hsqldb-1.8.0.10.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099728/avro-mapred-1.7.5-hadoop2.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099729/kite-data-hive-1.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099730/parquet-format-2.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099731/sqoop-1.4.6-hadoop200.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099732/parquet-common-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099733/sqoop-1.4.1-incubating.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099734/mysql-connector-java-5.1.40-bin.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099735/ojdbc6-11.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099736/commons-logging-1.1.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099737/parquet-encoding-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099738/mysql-connector-java-5.1.40.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099739/opencsv-2.3.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099740/snappy-java-1.0.5.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099741/commons-codec-1.4.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099742/parquet-column-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099743/jackson-mapper-asl-1.9.13.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099744/jackson-databind-2.3.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099745/parquet-generator-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099746/commons-io-2.4.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099747/commons-jexl-2.1.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099748/jackson-core-2.3.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099749/parquet-avro-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099750/kite-data-mapreduce-1.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099751/avro-1.7.5.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099752/ant-contrib-1.0b3.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099753/paranamer-2.3.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099754/slf4j-api-1.6.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099755/parquet-jackson-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099756/xz-1.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099757/parquet-hadoop-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099758/jackson-core-asl-1.9.13.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099759/kite-hadoop-compatibility-1.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099760/kite-data-core-1.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099761/jackson-annotations-2.3.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099762/ant-eclipse-1.0-jvm1.2.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099763/commons-compress-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099764/hsqldb-1.8.0.10.jar
16/12/28 13:15:00 INFO mapreduce.Job: The url to track the job: http://localhost:8080/
16/12/28 13:15:00 INFO mapreduce.Job: Running job: job_local1242973550_0001
16/12/28 13:15:00 INFO mapred.LocalJobRunner: OutputCommitter set in config null
16/12/28 13:15:00 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
16/12/28 13:15:00 ERROR output.FileOutputCommitter: Mkdirs failed to create file:/shipper/_temporary/0
16/12/28 13:15:00 INFO mapred.LocalJobRunner: Waiting for map tasks
16/12/28 13:15:00 INFO mapred.LocalJobRunner: Starting task: attempt_local1242973550_0001_m_000000_0
16/12/28 13:15:00 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]
16/12/28 13:15:00 INFO mapred.MapTask: Processing split: com.cloudera.sqoop.mapreduce.db.DataDrivenDBInputFormat$DataDrivenDBInputSplit@3d93720d
16/12/28 13:15:00 INFO mapred.LocalJobRunner: map task executor complete.
16/12/28 13:15:00 WARN mapred.LocalJobRunner: job_local1242973550_0001
java.lang.Exception: java.io.IOException: Mkdirs failed to create file:/shipper/_temporary/0/_temporary/attempt_local1242973550_0001_m_000000_0 (exists=false, cwd=file:/home/hadoop/Workspaces/BigData)
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.io.IOException: Mkdirs failed to create file:/shipper/_temporary/0/_temporary/attempt_local1242973550_0001_m_000000_0 (exists=false, cwd=file:/home/hadoop/Workspaces/BigData)
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:442)
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:428)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:908)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:889)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:786)
at org.apache.sqoop.mapreduce.RawKeyTextOutputFormat.getRecordWriter(RawKeyTextOutputFormat.java:98)
at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.<init>(MapTask.java:644)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
16/12/28 13:15:01 INFO mapreduce.Job: Job job_local1242973550_0001 running in uber mode : false
16/12/28 13:15:01 INFO mapreduce.Job: map 0% reduce 0%
16/12/28 13:15:01 INFO mapreduce.Job: Job job_local1242973550_0001 failed with state FAILED due to: NA
16/12/28 13:15:01 INFO mapreduce.Job: Counters: 0
16/12/28 13:15:01 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
16/12/28 13:15:01 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 3.3552 seconds (0 bytes/sec)
16/12/28 13:15:01 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
16/12/28 13:15:01 INFO mapreduce.ImportJobBase: Retrieved 0 records.
16/12/28 13:15:01 ERROR tool.ImportTool: Error during import: Import job failed!
Here is the code:
package org.sqoop.com;
import java.io.FileInputStream;
import java.util.Properties;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import com.cloudera.sqoop.SqoopOptions;
import com.cloudera.sqoop.SqoopOptions.FileLayout;
import com.cloudera.sqoop.tool.ImportTool;
public class MysqlExportHDFS {
public static void importMySQLToHDFS() throws Exception {
try {
String driver = "com.mysql.jdbc.Driver";
Class.forName(driver).newInstance();
Configuration config = new Configuration();
config.addResource(new Path("/usr/local/hadoop/etc/hadoop/conf/core-site.xml"));
config.addResource(new Path("/usr/local/hadoop/etc/hadoop/conf/hdfs-site.xml"));
Properties properties = new Properties();
properties.load(new FileInputStream("/home/hadoop/Workspaces/BigData/resources/sqoopImport.properties"));
SqoopOptions sqoopOptions = new SqoopOptions();
sqoopOptions.setDriverClassName(driver);
sqoopOptions.setHadoopHome("/usr/local/hadoop");
sqoopOptions.setConnectString(properties.getProperty("db_connection_string"));
sqoopOptions.setTableName(properties.getProperty("db_table_name"));
sqoopOptions.setUsername(properties.getProperty("db_username"));
sqoopOptions.setPassword(properties.getProperty("db_password"));
sqoopOptions.setNumMappers(1);
sqoopOptions.setTargetDir(properties.getProperty("db_output_directory"));
sqoopOptions.setFileLayout(FileLayout.TextFile);
new ImportTool().run(sqoopOptions);
} catch (Exception e) {
// TODO: handle exception
}
}
public static void main(String[] args) throws Exception{
MysqlExportHDFS mySqlImport = new MysqlExportHDFS();
mySqlImport.importMySQLToHDFS();
}
}
