4

I'm a new configuration manager at my company and I am starting a project to clean up much of our build process.

We have oodles of jars we have put away in one Subversion project. Some people copy these jars to their projects. Others put an SVN:External on their project to this directory. Others have a convoluted way of copying in these jars. These are the only jars that we're suppose to use.

I'm moving over to Ivy and maybe to Maven. I've setup a Maven repository, and have identified these jar's GroupId, ArtifactId, and Version and have documented how to access the ones you need via Ivy and Maven configurations.

I want to go through all the projects' source files, find all the import statements, and compare those imports against all the classes these jars contain.

I want to find all the jars that a given set of sources call directly. For example, if a jar contains:

import org.apache.commons.beanutils.*

I know it needs commons-beanutils.jar (version 1.6). I also know that it needs commons-logging.jar, but I'll let Maven and Ivy handle that issue. I just want to tell the developer when they create their pom.xml or ivy.xml, they'll need to include common-beanutils#common-beansutil#1.6 in their pom.xml or ivy.xml file.

I know this won't be a complete list. I know there will be jars needed for runtime, but not compile. I also know there are classes that can work with multiple jars. I also now developers could do this:

import org.apache.commons.*;

which would match with dozens of jars and tell me nothing. (Maybe there's a way to figure out what classes are actually used in the Java source files).

I could write a Python or Perl script that could probably do this, but I was hoping there was something already around that could save me several hours of work. I know about TattleTale by JBoss, but the documentation seem to say it only works with jars and shows dependencies between jars, etc. I'm not really interested in that. I just want to know what jars our sources are calling and let Ivy and Maven handle deeper dependencies.

Any such tools?

Addendum

Just to be more specific. This is an old project. In fact, there are 50 or so projects. We know that a particular project needs 20 jars to run, and tools like TattleTail can go through and show you the whole data analysis.

However, let's look at it from a developer's view point. I have a problem, and I find an open source project that will solve my problem. I read the docs, write my code and import the classes I need. I know the jar for that open source project that contains the classes, and that's pretty much all I know.

What I want to do is get back to the original developer's mindset: I used these classes in this project, and I need these 5 jars. That those 5 jars may need another 15 to run is something that Ivy/Maven will now handle. We know the 20 jars that program needs. I just want the 5 that the developers originally referred to in their code.

This is to help the developers write their pom.xml or ivy.xml. I don't want them putting all 20 jars in that pom.xml or that ivy.xml file -- just the five they need.

This is just a starting point. They might have imported com.foo.bar.bar.foo and that class is in foo-all.jar, foo-client.jar, and bar-talk.com. The developers will have to decide which one of these their project needs.

This is a massive paradigm shift in our programming, but we are losing track of jars and versions, and making sure all of these projects can talk to each other. Using Ivy/Maven will greatly simplify this mess.

David W.
  • 105,218
  • 39
  • 216
  • 337
  • `grep` seems almost appropriate here...then again, you may be looking for dependency analysis. – Makoto Aug 15 '12 at 21:15
  • 1
    You might want to take a look at this [question](http://stackoverflow.com/questions/786708/how-to-check-dependencies-between-jar-files). It has some pretty good answers that might help. – Jacob Schoen Aug 15 '12 at 21:18
  • The source code will not tell you much, especially with global imports as you mentioned. Because an import statement doesn't mean that anything that's being imported is actually being used. That problem can be solved by looking at the bytecode. There is also the problem with different versions of the JARs, nothing in the source code can tell you which ones are out there on your developers' machines and which ones you want. – Jochen Aug 15 '12 at 21:28
  • I have NO idea if this is a viable option, but maybe you can somehow hook into the classloader and "paint" the jars that get referenced at run time. I've recently used this target painting analogy to reduce our *.properties files, where most of the properties were actually no longer used. – Jason Dunkelberger Aug 15 '12 at 22:23
  • @Jochen I'm interested in is the developer's point of view. If a developer imports a class, then they had to look at a JavaDoc on that class, and know the jar that class is in. This is true even if that class was never used. Most byte code tools don't distinguish the 1st level dependencies (what the developer wants) vs. the dependencies of dependencies. Developer wants a class in Jar #1, but Jar #1 depends upon Jar #2 and #3. Jar #3 depends upon Jar #4. I only want Jar #1 and let Ivy and Maven handle the rest. That's why I am interested in the source. Ivy/Maven will take care of the rest. – David W. Aug 16 '12 at 01:37
  • @jschoen I saw that page from before. The tools do complete analysis which is great, but not what I need. We know this code needs 20 jars. What we don't know is that the developer refers to classes contained in 5 jars and the other 15 are dependencies on those 5 jars. I want to know the 5 jars that the code refers to and put them into a `pom.xml` or `ivy.xml` file. Let Ivy/Maven figure out the other 15. This is a big change for our development. The report I'll produce is a mere starting point in the analysis to help the developers create their `pom.xml`/`ivy.xml` files. – David W. Aug 16 '12 at 01:43
  • @DavidW. The accepted answer there points to [Tattletale](http://www.jboss.org/tattletale) which as the first bullent point is `Identify dependencies between JAR files`. I think you may be able to derive what you are looking for from that. Might be worth trying it out just to see if you can. It seems pretty easy to use based on their docs. – Jacob Schoen Aug 16 '12 at 02:54
  • @jschoen I've tried tattletail, but the report shows dependencies that simply don't exist in the source code. I know this because I scanned the source for them. I was also able to build the source without these jars. These might be dependencies based upon other dependencies in another jar, but it's not something I would put in my `pom.xml`. Using Tattletail was pretty easy. I had to place all jars in the same directory and include my product's jar. 20 minutes later, a whole bunch of reports were produced. Useful reports, but not for this particular task. – David W. Aug 16 '12 at 14:28
  • @jschoen Okay, it seems like Tattletale did work. The jar I was given to test Tattletale against was not the version I was told. I notice that some of the class files in the jars didn't seem to match the source files in the project. This is the type of mess I was hired to clean up. Tattletale does give me the first layer of dependencies. – David W. Aug 17 '12 at 02:04
  • Awesome, I have been meaning to play around with Tattletale and just have not had a change. It seems like a pretty cool tool over all. – Jacob Schoen Aug 17 '12 at 02:44

1 Answers1

1

Tattletale ended up giving me the first level of dependencies which is what I need to build my ivy.xml or pom.xml file. It misses one or two occasionally, but otherwise it works pretty good.

fglez
  • 8,422
  • 4
  • 47
  • 78
David W.
  • 105,218
  • 39
  • 216
  • 337