2

The problem scienario is as follows (Note: this is not a cross-jar dependency issue, so tools like JarAnalyzer, ClassDep or Tattletale would not help. Thanks).

I have a big project which is compiled into 10 or more jar artifacts. All jars depend on each other and form a dependency hierarchy.

Whenever I need to modify one of the jars, I would check out the relevant source code and the source code for projects that depend on it. Modify the code, compile, repackage the jars. So far so good.

The problem is: I may forget to check one of the dependent projects, because inter-jar dependencies can be quite long, and may change with time. If this happens some jars may go "out-of-sync" and I will eventually get a NoSuchMethodException or a some other class incompatibility issue at run-time, which is what I want to avoid.

The only solution I can think of, the most straighforward one, is to check out all projects, and recompile the bunch. But this takes time, especially if I re-build it every small change. I do have a continuous integration server, that could do this for me, but it's shared with other developers, so seeing if the build breaks is not an option for me.

However, I do have all the jars so hypothetically it should be possible to verify jars which depend on the code that I modified have an inconsistency in method signature, class names, etc. But how could I perform such check?

Has anyone faced a similar problem before? If so, how did you solve it? Any tools or methodologies would be appreciated.

Let me know if you need clarification. Thanks.

EDIT:

I would like to clarify my question a little bit.

The ultimate goal of this task is to check that the changes that I have made will compile against the whole project. I am looking for a tool/technique that would aid me perform such check.

Consider this example:

You have 2 projects: A and B which are deployed as A.jar and B.jar respectively. A depends on B.

You wish to modify B, so you check it out and modify a method signature that A happens to depend on. You can compile B and run all tests by itself without any problems because B itself does not depend on anything. So you happily commit your changes.

In a few hours the complete project integration fails because A could not be compiled!

How do I avoid this?

The kind of tool I am looking for would retrieve A.jar and check that all dependencies in A on the new modified B are still fine. Like a potential compilation error that would happen if I were to recompile A and B sources together.

Another solution, as was suggested by many of you, is to set up a local continuous integration system that would recompile the whole project locally. I don't mind doing this, but I want to avoid doing it inside my workspace. On the other hand, if I check-out all sources to another temporary workspace, then I need to mirror my local changes to the temporary workspace.

This is quite a big issue in my team, as builds break very often because somebody forgot to check out (or open in Eclipse) the right set of projects. I tried persuading people to check-out source and recompile the bunch before commits, but not only it takes time, it needs running quite a few commands so most people just find it too troublesome to do. If the technique is not easy or automated, then it's unusable.

rodion
  • 14,729
  • 3
  • 53
  • 55
  • Sounds like call (scream) for continous integration - it's easy to start having one. – Hurda Feb 25 '11 at 07:44
  • @Hurda. I don't mind having local CI server running on my machine, but I have problems finding a way to set it up (see penultimate paragraph in the **EDIT**) – rodion Feb 25 '11 at 09:44

6 Answers6

2

If you do not want to use your shared continuous integration server you should set up a local one on your developer machine where you perform the rebuild processes on change.

I know Jenkins - it is easy to setup (just start) on a local machine and I would advice to run it locally if no one is provided in the IT infrastructure that fits your needs.

FrVaBe
  • 47,963
  • 16
  • 124
  • 157
  • Thanks for the advice. This approach is nice, but still a bit too "brute force" for my PC. I would have to invest in additional hardware for that one, I feel. – rodion Feb 21 '11 at 12:24
  • I noticed that my PC is mostly bored and not very busy when compiling my code - so I run a Wiki (WebServer), [Quality Management Platform](http://www.sonarsource.org/) and Continuous Integration Server on my local machine. Some of them I only start if necessary but I doubt that they will use to much resources because they also mostly idle around. Of course you need a bit of memory but as a developer you should be already propper equipped. Try it - don't feel :-) – FrVaBe Feb 21 '11 at 13:44
  • I would give it a try but I have just realised that there's still one problem. I am using subversion, so I cannot create a local repository from which Jenkins could check out my temporary sources. I would have to make Jenkins look in my Eclipse project, but that seems rather odd (and would break if I was to check out another branch elsewhere). I could make a script to copy my sources to where Jenkins could see it, but that does not appear very elegant. Any suggestions? – rodion Feb 21 '11 at 14:31
  • You should commit your changes to subversion and let Jenkins make an update in each project before build (checkout somewhere else than your eclipse project). If you broke something you have to fix it. Therefore you are using subversion :-) – FrVaBe Feb 21 '11 at 14:40
  • What I meant was that I am checking in/out from a shared repository (as part of my team's development cycle), therefore there's no easy way to commit that to my local repository. If we used a distributed VCS like darcs or mercurial, I could commit changes to my local repo and then, when I am happy with them, push them to the shared repo. But in svn there is not easy way of doing that afaik. Or is there? – rodion Feb 22 '11 at 00:28
  • I can get you no help on this. I would just recommend not to be to afraid to check in your changes in a shared repository frequently. – FrVaBe Feb 22 '11 at 08:34
  • I don't mind personally, but it's a matter of etiquette, since I am not the only person using it. Besides, whenever build fails, tests and other metrics cannot be run, so the whole development cycle gets delayed. I am sure there's a better way of doing this. Thanks for the advice, I appreciate it! The Sonar tool seems awesome as well. I'll be sure to get that running:) – rodion Feb 22 '11 at 09:01
  • I did not mean that you SHOULD break the build but frequently commit seems to be a kind of best practise. Neverthelss its up to you! If you have all maven projects and eclipse than you can import them all in your IDE and `Enable Workspace Resolution` on each project. Dependencies will than be resolved on your open projects if possible. I suppose you must see dependency failures very quick. – FrVaBe Feb 22 '11 at 16:19
  • Unfortunately the reason why I am asking this question is that there are some many projects that opening them all at the same time kills my machine (it's not a bad machine, the project is huge). So I want to have a "workspace resolution" effect, without actually opening the projects, and if possible, without checking out the source code. – rodion Feb 23 '11 at 00:49
2

Checking signatures is unfortunately not enough. Having the correct signatures does not mean it'll work. It's all about contracts and not just signatures. I mean what happens if the new version of a library has the same method signature, but accepts an ArrayList parameter now in reversed order? You will run into issues - sooner or later. I guess you maybe consider implementing tools like Ivy or Maven:

http://ant.apache.org/ivy/

http://maven.apache.org/

Yes it can be pain to implement it but once you have it it will "guard" your versions forever. You should never run into such an issue. But even those build tools are not 100% accurate. The only proper way of dealing with incompatible libraries, I know you won't like my answer, is extensive regression testing. For this you need bunch of testing tools. There are plenty of them out there: from very basic unit testing (JUnit) to database testing (JDBC Proxy) and UI testing frameworks like SWTBot (depends if your app is a web app or thick client).

Please note if your project gets really huge and you have large amount of dependencies you always not using all of the code there. Trying to check all interfaces and all signatures is way too much. Its not necessary to test it all when your code use lets say 30 % of the library code. What you need is to test what you really use. And this can be only done with extensive regression testing.

lzap
  • 16,417
  • 12
  • 71
  • 108
  • I completely agree with every word! It's just the problem is that I already have Ivy and extensive testing, so the contract problem is not really an issue. The issue is that I want to make sure the my check-in into the shared repository will compile, and so far the only way to check that is to check-out all sources and recompile everything. I don't really mind doing it, as long as it's not in my Eclipse. But ideally, I want a cool tool to do that for me! – rodion Feb 24 '11 at 12:53
  • Oh I see. I only know Tattletale - sorry :-( Cant help here. – lzap Mar 01 '11 at 11:08
  • Though Maven, Ivy, or Gradle do not remove this problem. Say A depends on B and C, and B and C depend on different version of D. One version will be chosen (it depends what system you are using), but there is not guarantee that it is compatible with the both B and C. In this case, it would (again) be useful to check the signatures in this way. – Paul Draper Jun 26 '14 at 20:17
1

I'm also think that just setup local Jenkins is a best idea. What tool you use for build? Maybe you can improve you situation with switching to Maven as build tool? In more smart and don't recompile full project if you don't ask it directly. But switch to in can be HUGE paint in the neck - it hardly depends on how you project organized now...

And about VCS- exist Mercurial/SVN bridge - so you can use local Mercurial for you development .... check this link: https://www.mercurial-scm.org/wiki/WorkingWithSubversion

Vadim Kotov
  • 8,084
  • 8
  • 48
  • 62
Dmytro
  • 195
  • 5
  • I am using Ant+Ivy. All projects are wired-up using dependency management, and everything works like a charm (except the little problem I have described). So switching to Maven itself is not going to solve my problem. I need a special kind of dependency set-up for locally built jars, or something clever like that. As for the Mercurial/SVN bridge, it sounds like a great idea. I will give it a try. I just hope it doesn't screw up my SVN repo:(( – rodion Feb 24 '11 at 12:45
1

I have finally found a whole treasure box of answers at this post. Thanks for help, everyone!

The bounty goes to K. Claszen for the quickest and most input.

Community
  • 1
  • 1
rodion
  • 14,729
  • 3
  • 53
  • 55
0

There is a solution, jarjar, which allows to have different versions of the same library to be included multiple times in the dependency graph.

mindas
  • 26,463
  • 15
  • 97
  • 154
  • I don's see how exactly this helps to solve my problem. As far as I understood, jarjar simply unjars dependent jars and concatenates all contents into one single jar, for easier distribution. – rodion Feb 24 '11 at 00:28
  • I thought the source of the problem you're having is because you need different versions of the same library in your dependency tree. Because if you are recompiling on each update, things wouldn't compile, would they? – mindas Feb 24 '11 at 09:50
  • The problem is in the opposite: that it *does* compile. Other jars that depend on my changes will give no signs of a problem, unless I explicitly recompile them too against my new changes. This explicit recompilation is something I am trying to avoid, because it's expensive. – rodion Feb 24 '11 at 12:29
  • You could still use jarjar to give other jars (which depend on yours) an older version of your library. Just not sure if that is in line with what you are trying to achieve. – mindas Feb 24 '11 at 13:25
  • I want to make sure that compiling all **newest** versions of all jars together will succeed. – rodion Feb 26 '11 at 16:14
0

I use IntelliJ, not Eclipse, so maybe my answer is too IDE-specific. But in IntelliJ, I would simply include the modules from B into A, so that when I make changes to A, it breaks B immediately when compiling in the IDE. Modules can belong to multiple projects, so this is not anything like duplication, it's just adding references in the IDE to modules in other projects.

Jesse Barnum
  • 6,507
  • 6
  • 40
  • 69
  • The problem is that I want to modify B, not A. Modifying A will break for me too, so that's not a problem. – rodion Feb 25 '11 at 09:41
  • Sorry, I got the letters backwards. So include A in the B project, so that when you make changes to B, the IDE tries to compile A at the same time. A good IDE will only try to recompile A if there are changes in B that A depends on, so it shouldn't take longer. – Jesse Barnum Feb 25 '11 at 17:14
  • I get your point, but the source of my problem (see 4th paragraph) is that I may forget to check A out. Actually, in most cases, I don't want to check it out if I am not planning to make changes to interfaces A depends on. But I still want to make sure everything will compile just in case. It's a tricky problem: I need to make sure it will compile, without actually doing the compilation. I believe this should be possible with the right tool. – rodion Feb 26 '11 at 16:07