1

In our Appium test automation project, we have a customized configuration that does not use the default wdio.conf.ts file. So far we did not encounter any issues with this: the driver configuration is set up in a separate class and tests run as expected.

Recently we wanted to try out Browserstacks Observability functionality. This works out of the box whenever we try it on a project running with a wdio.conf.js file, but so far I haven't found a way of activating it in our automation project: the configuration is accepted, but no data shows up in Observability. I've spend a few days now messing around with the configuration to get it working, but to no avail.

Note that we also contacted Browserstack support for it: they say our configuration in itself is correct, so right now they also don't know why the data won't show up.

I managed to create a small sample project that produces the same issue, which can be found here. I send it to Browserstack so they can investigate, but if anyone here happens to have some experience or insights in this, then we'd love to hear it! If a solution is found, then it will be posted here as well.

TLDR: can anyone figure out why the data in this sample project does not show up in Observability, and how to fix this?

Thanks in advance!

Tybs
  • 512
  • 4
  • 23

3 Answers3

1

You need to have a static name for your project and build name I see the are different in your code.

enter image description here

The same is mentioned in Browserstack's official documentation: Here is the screenshot and link for your reference: https://www.browserstack.com/docs/test-observability/quick-start/webdriverio enter image description here

Please incorporate the same in your test.

This should resolve the issue you are facing.

Thanks.

grandEL-dsouza
  • 50
  • 8
  • I did mess up the build name in the sample project: apologies, and thanks for pointing this out! Has now been corrected. Sadly this is not the issue in itself, despite the correction I don't see the test data appearing in Observability :( – Tybs Jun 02 '23 at 08:40
  • It will possibly be because the format of your config file will be incorrect. – grandEL-dsouza Aug 11 '23 at 05:30
0

I checked the shared repo and could see that you are using different names in projectName and buildName within the observability service config and the test capabilities. Not sure if this would be helpful but suggestion is to maintain same name for projectName and buildName in both places so that observability picks up that name and populates data based on it in Observability dashboard

Guest123
  • 1
  • Asked Browserstack the same, but according to them this does not matter, which I also noticed at some point in the behavior with wdio.conf.js. Apparently categorization between browserstack runs and reporting is handled differently. Thanks for thinking along though! – Tybs Jun 01 '23 at 12:31
0

The Browserstack servicedesk investigated the issue and eventually escalated it to their product team. A few days after, we got the following response:

The team have had a look at your project and how it is structured and have said that this setup is a standalone setup and that services are not supported in this setup. The wdio-browserstack-service is only supported in TestRunner mode. You can find more information on that here:

https://webdriver.io/docs/setuptypes

We won't be able to support this as of now (I can however; file product feedback on this point). The recommendation is to use a setup that supports the wdio-browserstack-service or you can generate a JUnit XML report for your test and upload that using, https://www.browserstack.com/docs/test-observability/quick-start/junit-reports, to get partial observability features supported.

I know this isn't the answer you were looking for but please reach out if you have any questions

As it turns out, the TestRunner mode (wdio.conf.js) is a requirement for this, which was not included in the documentation (yet). Right now it simply cannot be done by normal setup.

However, at first glance, it seems like the suggested workaround with JUnit XML reports is feasible. We already had JUnit set up, and I can quickly upload these reports and view them in Observability with the following code:

static async uploadReportsToBrowserstack(browserstackUsername: any, browserstackAccessKey: any){
    const reportPath = 'C:/report.xml';
    this.logger.logDebug({reportPath});
    this.logger.logInfo('Upload reports to BrowserStack starts...');
    
    const form = new FormData();
    form.append('data', fs.readFileSync(reportPath), reportPath);
    form.append('projectName', 'Junit report uploads');
    form.append('buildName', 'Observability Sample');
    form.append('tags', 'junit_upload, regression');
    form.append('frameworkVersion', 'mocha, 10.3.2');

    const response = await axios.post('https://upload-observability.browserstack.com/upload',form,
        {
            headers: {
                'Content-Type': `multipart/form-data; boundary=${form.getBoundary()}`
            },
            auth: {
            username: browserstackUsername,
            password: browserstackAccessKey
            }
        }
    );
}

Obviously I'll need to refactor this a bit, but the concept seems to be working fine. Thanks to everyone who gave their input on this!

TLDR: As of the time of writing, Test Observability only supports TestRunner (wdio.conf.js) mode, but the intended result is possible with the workaround described above.

Tybs
  • 512
  • 4
  • 23