3

In my current project I'm testing one web application against different browsers. so I've got one selenium/maven project and 16 jobs in jenkins: tests in IE8, 9, FF, Chrome, windows 7, windows xp, 32 and 64 bit environments

Jobs results are generating sourcefire reports and I've got nice and clean overall charts.

But the problem is comparison of single tests results: for example I'd like to know which tests are failing under Internet Explorer and not in Firefox or which browser is stable and which is not.

Currently I'm forced to open X windows with different results and manually compare results

enter image description here

is there a plugin that compares results? or my jenkins configuration is wrong?

mleko
  • 11,650
  • 6
  • 50
  • 71
Rurouni Kenshin
  • 101
  • 1
  • 2
  • 5

2 Answers2

1

You should probably be using the "matrix" job type, called "Build multi-configuration project" in the New Job page.

There you configure the job once, and it gets executed automatically multiple times across all the combinations of axes you supply

I'm not sure about the specifics of Selenium with Jenkins, but you would probably want to set up two different axes:

  • User-defined axis: browser (FF, IE...)
  • Slave axis: build slave (XP_32, XP_64, W7_32...)

Overall, documentation is a bit thin for this feature, but see this answer for some info:
https://stackoverflow.com/a/424343/234938

Community
  • 1
  • 1
Christopher Orr
  • 110,418
  • 27
  • 198
  • 193
  • Thank You Christopher for reply, ive tested it and it works great. But it still does not show compared tests results. Please see screenshot: http://imgur.com/ym1UI as 'X' axis i want tests, not slaves – Rurouni Kenshin Dec 19 '11 at 12:24
0

Have you tried Test Results Analyzer plugin in Jenkins?

https://wiki.jenkins-ci.org/display/JENKINS/Test+Results+Analyzer+Plugin

AKS
  • 16,482
  • 43
  • 166
  • 258