SQOOP Not able to import table

Question

I am running below command on sqoop

sqoop import --connect jdbc:mysql://localhost/hadoopguide --table widgets

my version of sqoop : Sqoop 1.4.4.2.0.6.1-101
Hadoop -- Hadoop 2.2.0.2.0.6.0-101

Both taken from hortonworks distribution. all the paths like HADOOP_HOME, HCAT_HOME, SQOOP_HOME are set properly. I am able to get list of databases, list of tables from mysql database by running list-database, list-tables commands in sqoop. Even able to get data from --query 'select * from widgets'; but when i use --table option getting below error.

14/02/06 14:02:17 WARN mapred.LocalJobRunner: job_local177721176_0001
java.lang.Exception: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class widgets not found
        at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:403)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class widgets not found
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1720)
        at org.apache.sqoop.mapreduce.db.DBConfiguration.getInputClass(DBConfiguration.java:394)
        at org.apache.sqoop.mapreduce.db.DataDrivenDBInputFormat.createDBRecordReader(DataDrivenDBInputFormat.java:233)
        at org.apache.sqoop.mapreduce.db.DBInputFormat.createRecordReader(DBInputFormat.java:236)
        at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.<init>(MapTask.java:491)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:734)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:339)
        at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:235)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
        at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.ClassNotFoundException: Class widgets not found
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1626)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1718)
        ... 13 more

score 6 · Answer 1 · edited May 08 '14 at 05:49

6

Specify the --bindir where the compiled code and .jar file should be located.

Without these arguments, Sqoop would place the generated Java source file in your current working directory and the compiled .class file and .jar file in /tmp/sqoop-<username>/compile.

edited May 08 '14 at 05:49

brasofilo

25,496
15
91
179

answered May 08 '14 at 04:56

user1639487

61
1
2

I searched the whole internet and finally found only your solution works. Tears! Thank you so much! The command is this: sqoop import --connect jdbc:postgresql://:/ --username -password --table --bindir /tmp/sqoop-/compile/ --delete-target-dir – Harvey Dent Feb 13 '19 at 18:56

score 6 · Answer 2 · answered May 20 '15 at 17:04

6

Use the --bindir option and point to your current working directory.

sqoop import --bindir ./ --connect jdbc:mysql://localhost/hadoopguide --table widgets

answered May 20 '15 at 17:04

igaritano

61
1
1

score 1 · Answer 3 · answered Feb 07 '14 at 11:01

1

The problem is resolved after i copied the .class file from /tmp/sqoop-hduser/compile/ to hdfs /home/hduser/ and also the current working directory from where i am running sqoop.

answered Feb 07 '14 at 11:01

user236575

95
1
3

score 0 · Answer 4 · edited Jul 03 '17 at 12:43

For importing a specific table into hdfs, run:

sqoop import --connect jdbc:mysql://localhost/databasename --username root --password *** --table tablename --bindir /usr/lib/sqoop/lib/ --driver com.mysql.jdbc.Driver --target-dir /directory-name

Make sure that /usr/lib/sqoop/* and /usr/local/hadoop/* should be owned by the same user otherwise it will give error like "Permission denied".

PS: Make sure that you have installed mysql-java connector before you run the command. I installed hadoop version 2.7.3 and connector 5.0.8

score 0 · Answer 5 · answered Nov 06 '19 at 18:36

Another fix for ClassNotFoundException is to tell Hadoop to use the user classpath first (-Dmapreduce.job.user.classpath.first=true). This can be on command line or in Options file. The top of an import Options file would be:

#Options file for Sqoop import
import

-Dmapreduce.job.user.classpath.first=true

This fixed ClassNotFoundException for me when trying to import data as-avrodatafile

SQOOP Not able to import table

5 Answers5

Linked