4

I wrote a hadoop MR job using spring-data-hadoop and used maven-shade-plugin to pack all necessary jars. But when I ran the job from comand line:hadoop jar , it gave me Exception for ClassNotFound for the spring-beans. Resulting problem in loading spring application context. There after I used maven-assembly-plugin and packed all required jars in a lib folder. This time it worked.

I fail to understand why did it failed with maven-shade-plugin. Can someone help me understand my observation.

user1516083
  • 81
  • 7

2 Answers2

1

I don't know how you used a shade plugin but I'd assume that you may get into trouble with meta info like META-INF/spring.handlers and META-INF/spring.schemas which have same path in all spring jars.

Janne Valkealahti
  • 2,552
  • 1
  • 14
  • 12
1

From http://maven.apache.org/plugins/maven-shade-plugin/

This plugin provides the capability to package the artifact in an uber-jar, including its dependencies and to shade - i.e. rename - the packages of some of the dependencies.

I'm guessing your configuration of the shade plugin had renamed some packages such that the class files could no longer be found.

Phil Hayward
  • 1,113
  • 12
  • 10