10

I have multiple unit tests and component tests which I would like to test against several versions of Java (1.6, 1.7, 1.8, etc).

I wonder what’s the best methodology to it?

Should I run tests several times using Jenkins and each time I change the path of the JAVA_HOME variable to a different JRE?

Can I do using Maven plugin? Maybe different profiles?

In addition, is there a way using TestNG to exclude tests based on the JRE version? I can handle it using an if statement on a the java.version system property, however I wonder if there’s a more elegant way.

In case this will affect the answer, I would like to mention that looking forward I would like to test it against several OS types and multiple versions of the product (same tests against v1.0, v2.0, etc).

Palec
  • 12,743
  • 8
  • 69
  • 138
nadavy
  • 1,755
  • 1
  • 18
  • 33
  • as far as I know in linux(ubuntu) you can have several version of java installed then change the default – nafas Dec 16 '15 at 09:06
  • Something like a simulator perhaps? – smac89 Dec 16 '15 at 09:08
  • i'm not sure what is best methodology, but i would go with maven. you can pass java version via parameter. hence you could easy create different jenkins tasks for each version – user902383 Dec 16 '15 at 09:25
  • Why bother? Java is designed to be backward compatible, so 1.6 code runs in a 1.8 environment. It is not at all forward compatible: 1.8 code will not run in a 1.6 environment. – Raedwald Dec 16 '15 at 09:28
  • Yes, there can be JVM bugs that affect backward compatibility. But they are so uncommon you would be better off spendingyour testing resources testing something else. – Raedwald Dec 16 '15 at 09:30
  • If not protected by bounty, this question would be closed as "primary opinion-based" pretty fast. – Oleg Estekhin Dec 20 '15 at 08:14
  • @ Raedwald - Thanks for your comment. Let's say that I'm in a situation which requires it. Do you have any concrete suggestion? – nadavy Dec 20 '15 at 15:38
  • Couple of things you need to observe: If the code uses 1.8-only features then you cannot compile it for 1.7 target nor run it on 1.7 runtime. So adding test conditions is useless if it is to exclude tests that use 1.8-only language features. – Rudi Angela Dec 22 '15 at 09:43
  • @Rudi Angela - thanks for pointing it out. This is something that we are aware of it. – nadavy Dec 22 '15 at 12:26
  • @Raedwald, .. I'm just wondering is there a way to catch major-minor exception in case of forward compatibility ? anyways as said, java is backward compatible and it shouldn't cause much noise during testing.. – harshavmb Dec 26 '15 at 08:23

3 Answers3

6

To run the same code under different conditions jenkins can simply run the same scripts with different JAVA_HOME or other settings. Doing this on different triggers or as part of one multistage job is a decision for you

As for TestNG, see this answer How to disable testng test based on a condition

Community
  • 1
  • 1
Vorsprung
  • 32,923
  • 5
  • 39
  • 63
0

A possible approach is to use maven profiles, combined with the maven toolchains plugin .

In this approach you will configure several JDKs on your build server, and configure your pom xml to have a distinct profiles that run your unit tests with each JDK.

You may want to consider testing not only different java runtimes, but also different target class versions; this can also be combined with the profile approach by having different profiles with different compile settings.

last, exclusion of sure fire tests can also be done from profiles, allowing you to control that as well.

Nitzan Volman
  • 1,809
  • 3
  • 17
  • 31
0

I am aware of one of the methodologies used in many open source projects which use TravisCI. In those projects, .travis.yml file (metadata file for TravisCI) contains this information. Example :-

jdk:
  - oraclejdk7
  - oraclejdk8
  - openjdk6
  - openjdk7

I also used it in one of my open source project (see here). I am not aware if in industry we have some other streamlined process for it but it may be something which open and may vary from organisations to organisations and projects to projects.

Siddharth
  • 2,046
  • 5
  • 26
  • 41