XRAY and WebdriverIO integration Issue - Partial result imported into XRAY

Question

I have integrated XRAY with webdriver IO , mocha using the guide below

https://docs.getxray.app/display/XRAYCLOUD/Testing+web+applications+using+Mocha+and+WebdriverIO#tab-API https://docs.getxray.app/display/XRAY/Import+Execution+Results+-+REST#ImportExecutionResultsREST-JUnitXMLresults

WDIO Config for JUnit reporter:

reporters: ['spec',
        ['junit', {
            outputDir: './',
            outputFileFormat: function(options) { 
                return `results.xml`
            }
        }],

Curl to import results.xml into XRAY:

curl -H "Content-Type: multipart/form-data" -u "UserName":"PASSWORD" -F "file=@results.xml" "${URL}/rest/raven/1.0/import/execution/junit?projectKey=${projectKey}&testPlanKey=${testPlanKey}"

Commands to run test suite(s):

Run single suite:     npm run --suite mysuite1
Run multiple suites:  npm run --suite mysuite1 --suite mysuite2

When single suite is executed, result.xml is created and successfully imported into XRAY. But when multiple suites are executed as above, result.xml has test result of the last suite only and thus only test results for the last suite are imported into XRAY.

As XRAY import API needs projectkey and testplankey, a result file should be created for each suite and import API to be invoked for each result file with right file name, project and plan.

What could help is a way to amend result file name which could be associated with the test plan e.g. result_mysuite1.xml.

Please let me know if more information is needed.

Thanks in advance, Mahima.

score 0 · Answer 1 · answered Aug 24 '22 at 15:53

0

The example we have available in the page you referred is in fact for a simple use case with one test, if you want to have multiple results please include a dynamic id in the name so that it will create multiple files (we will include this suggestion in the tutorial in the future):

[
    'junit',
    {
        outputDir: './',
        outputFileFormat(options) {
            return `results-${options.cid}.xml`;
        },
    },
],

Then you can push those results to Xray.

answered Aug 24 '22 at 15:53

Cristiano Cunha

381
1
3

1

Thanks @cristiano. I tried above. This creates multiple result xmls. But still no clue how to map it to respective test plan. Any idea? – Mahima Aug 24 '22 at 22:33
On the upload call to the API you must pass the TestPlan Key to associate those executions to the test plan. Depending on your strategy or usage you can use tags to group tests that you want to execute (using for example: https://www.npmjs.com/package/mocha-tags). You can also try the js client Xray have available to push the results from the code, giving you more control: https://www.npmjs.com/package/@xray-app/xray-automation. – Cristiano Cunha Aug 26 '22 at 08:46
Hi @Cristiano, I am already passing keys while uploading the result file. My issue is while mapping the right result file with the test plan key. If I run multiple suites I want to create separate result.xml for each and a way to associate with respective test plan kay (as shown in the example above) – Mahima Aug 30 '22 at 01:26
What could help is a way to amend result file name which could be associated with the test plan e.g. result_mysuite1.xml. – Mahima Aug 30 '22 at 01:33

score 0 · Answer 2 · answered Aug 30 '22 at 09:54

The Junit Reporter creates one JUnit XML file per runners, as per the documentation. What you can do is to configure WDIO (wdio.conf.js) to generate a distinct file based on an id that is available.

reporters: ['spec',
        ['junit', {
            outputDir: './',
            outputFileFormat: function(options) { 
                return `results-${options.cid}.xml`
            }
        }],

If you have 2 workers, you'll have 2 files like results-0-0.xml, results-0-1.xml. Then you can either upload them, one by one, or you may merge them using an utility such as junit-merge.

To upload one by one, using some shell script, you could something like:

for n in `ls results-*.xml`; do curl -H "Content-Type: multipart/form-data" -u "UserName":"PASSWORD" -F "file=@$n" "$BASE_URL/rest/raven/2.0/import/execution/junit?projectKey=$projectKey&testPlanKey=$testPlanKey"; done

If you prefer to merge the files and upload them in a single shot (my preferred approach), you would so something like:


npm install junit-merge
node_modules/junit-merge/bin/junit-merge  -o junit.xml results-0-0.xml  results-0-1.xml
# or if you have them in a directory, junit-merge  -o junit.xml -d output_dir

curl -H "Content-Type: multipart/form-data" -u "UserName":"PASSWORD" -F "file=@junit.xml" "$BASE_URL/rest/raven/2.0/import/execution/junit?projectKey=$projectKey&testPlanKey=$testPlanKey"

Note: another option could be forcing WDIO to have just one runner; however, it seems that a runner is created per each spec, at least from what I could assess

Thanks @Sergio. I have tried above. Problem is import API needs test plan key as input and I don't see any way to associate results-0-0.xml with the test plan key. How can I relate results-0-0.xml to particular test plan ? I appreciate your help. — Mahima, Aug 31 '22 at 00:09
testPlanKey is an optional parameter on the API request. You don't need to specify it. In case you want to, the two examples I've shown above in my answer includes it on the curl request. It uses a shell variable with the same name. You can either replace the variable and hard-code the test plan key or you can define the variable before the request, for example as a environment variable using "export testPlanKey=..." — Sérgio, Aug 31 '22 at 07:20

XRAY and WebdriverIO integration Issue - Partial result imported into XRAY

2 Answers2