0

I have tried to import data from oracle to hive or HDFS using sqoop command. It worked fine.

I want to use sqoop inside java code. I have tried few things from different links in stackoverflow. But i am not getting proper sqoop.jar file and it says Sqoop.runTool(str); class not found. , please let me know the steps to do this. How to start with java code for this.

Thanks.

timma
  • 223
  • 1
  • 5
  • 15
  • do you want to run sqoop import from java program? – Ankur Shanbhag Nov 07 '13 at 13:28
  • Yes. absolutely. I tried code from different links in stackoverflow but nothing worked, please let me know the step by step flow of how java code should be. – timma Nov 07 '13 at 13:36
  • what exception does it throw? It is hard to make suggestions before knowing the actual problem. – Ankur Shanbhag Nov 08 '13 at 06:08
  • I have tried this code for mysql included mysql drivers in eclipse i am new to java my requirement is to run the sqoop command `sqoop-import --connect jdbc:mysql://xxxxx/sqoopdb --username sqoopuser --password sqoopuser --table sqooptable` inside a java program `import com.cloudera.sqoop.*; public class driver { public static void main(String[] args) { String[] str = { "import", "--connect", "jdbc:mysql://192.168.0.129/xx","--username", "xx", "--password", "xx"}; Sqoop.runTool(str); } } } It says sqooprun tool class not found` – timma Nov 08 '13 at 07:29

1 Answers1

0

Major problem with running sqoop command from Java program is that you need to make sure that all the libraries required by sqoop are on the class path. If libraries are missing it would result in ClassNotFoundException. Sqoop requires hadoop libraries, hbase libraries if you are performing hbase import and hive libraries if you are creating tables in hive. It also uses libraries from SQOOP_HOME/lib folder.

It is very tricky to get the sqoop import working by directly calling the sqoop API's. Because you are bypassing the sqoop.sh command, which sets the environment (like adding required libraries to class-path etc.) before calling the sqoop API's.

What I would suggest you is to try using ProcessBuilder.exec() to run sqoop import from java program.

Ankur Shanbhag
  • 7,746
  • 2
  • 28
  • 38
  • I wish i could use ProcessBuilder.exec() but my task is to use java code api to do this, any idea on this. Thanks for reply. – timma Nov 08 '13 at 13:19
  • You can use Java Api to do so. Just you need to make sure you add all the necessary hadoop, hbase, hive and sqoop/lib jar files on the class-path. – Ankur Shanbhag Nov 11 '13 at 05:32
  • Thanks for reply i tried and the code from this link http://stackoverflow.com/questions/9229611/how-to-use-sqoop-in-java-program and got succeded. I have did it in vm ware in eclipse, as said my ankur make sure you include all libraries ie jar files needed. Please let me know how to add these jar files about 12 jar files in EC2(no eclipse there), should i copy each jar into classpath or any other method should i follow. – timma Nov 12 '13 at 04:27