Add all JARS in Hadoop FS to MapReduce classpath

Question

Is there any method that takes in a directory path and adds all jar files to the classpath of my map-reduce job?

The one I know is DistributedCache.addFileToClassPath(file, conf) but it seems like this only works for single file. In my case I have all the JAR files required in a single dir and I want all these in the classpath of my MR job.

Also I want this to be done via my java program in which I am using hadoop API to submit my job and not submitting via command prompt.

score 0 · Answer 1 · answered Jan 14 '16 at 12:46

0

You can use "-libjars" to give hadoop all your libraries. You only need to add it and the location of your libraries in your execution command

answered Jan 14 '16 at 12:46

EMBorque

145
9

Duplicate: http://stackoverflow.com/questions/16340158/include-third-party-jars-in-hadoop – EMBorque Jan 14 '16 at 12:51
I am using Hadoop API to submit my job so I want this to be done via my java program – Harinder Jan 14 '16 at 15:54

Add all JARS in Hadoop FS to MapReduce classpath

1 Answers1