3

I am new to Hadoop. Our team is writing a hadoop job scheduler. Currently I have hadoop well installed. I can submit Hadoop job in the command line using 

hadoop jar wordcount.jar input output

Now I want to run this command in a java program. In other words, using java to do the same thing as hadoop jar wordcount.jar input output. Is there any API I can call to submit jobs from Java?

Raptor
  • 53,206
  • 45
  • 230
  • 366
user3034824
  • 47
  • 4

1 Answers1

4

Yes, you can use ProcessBuilder or Runtime.exec to execute arbitrary system commands

EDIT: The solution I proposed is to just launch the hadoop command as you have specified. Hadoop does have an API for launching jobs, discussed in Calling a mapreduce job from a simple java program.

Community
  • 1
  • 1
Jeff Storey
  • 56,312
  • 72
  • 233
  • 406
  • Updated to add an alternative. – Jeff Storey Nov 26 '13 at 04:15
  • Could you please show me some example code? Thank you very much. – user3034824 Nov 26 '13 at 04:15
  • See this question http://stackoverflow.com/questions/3468987/executing-another-application-from-java. Give something a try - if you have specific questions, post them. – Jeff Storey Nov 26 '13 at 04:16
  • Thanks very much for your answer. Acually I check the API you mentioned. But I am not sure whether it is what I am looking for. In the Job API, the user can specify Mapper class, reduce class, etc. But in my case, we do not care about it. Everything is already packaged in the JAR file. We simply throw it to Hadoop like using "hadoop jar wordcount.jar input output" – user3034824 Nov 26 '13 at 04:22
  • user3034824, i want same thing as you said : Everything is already packaged in the JAR file. We simply throw it to Hadoop like using "hadoop jar wordcount.jar input output. If you get answer than tell me. please. – Divyang Shah Apr 17 '15 at 13:45