2

Have a requirement to capture network traffic from the browser while running automation test cases.

Came across below post but not sure if it has been included as part of Watir 7. Appreciate if there is any example for tapping on to devtools and capture network traffic like api calls using Watir. Thanks!!

https://github.com/watir/watir/issues/721

Megatherium
  • 31
  • 3

2 Answers2

2

Watir is not currently wrapping those features. You have access to everything by getting the selenium driver with the #wd method. So like this:

requests = []
browser.wd.intercept do |request, &continue|
    requests << request
    continue.call(request)
end
browser.goto page
expect(requests).not_to be_empty

There isn't a great guide in the selenium documentation for these, but here are other examples from the specs: https://github.com/SeleniumHQ/selenium/blob/trunk/rb/spec/integration/selenium/webdriver/devtools_spec.rb

titusfortner
  • 4,099
  • 2
  • 15
  • 29
  • Appreciate your inputs. Is there any plans in future for wrapping those within Watir ? – Megatherium Apr 21 '22 at 19:35
  • I mean, I guess we could delegate the various methods from browser to driver? But there's no additional logic watir adds to it at this point, so it's not a priority. – titusfortner Apr 22 '22 at 02:18
  • Works locally with above code + Selenium-devtools – Megatherium Apr 22 '22 at 05:43
  • Thanks!! but with remote execution seeing below error Selenium::WebDriver::Error::WebDriverError: DevTools is not supported by the Remote Server /usr/share/rvm/gems/ruby-2.7.4/gems/selenium-webdriver-4.0.0/lib/selenium/webdriver/remote/driver.rb:56:in `devtools_version' /usr/share/rvm/gems/ruby-2.7.4/gems/selenium-webdriver-4.0.0/lib/selenium/webdriver/common/driver_extensions/has_devtools.rb:34:in `devtools' /usr/share/rvm/gems/ruby-2.7.4/gems/selenium-webdriver-4.0.0/lib/selenium/webdriver/common/driver_extensions/has_network_interception.rb:64:in `intercept' – Megatherium Apr 22 '22 at 05:50
  • Devtools only works with a server that provides a `se:cdpVersion` value in the returned capabilities. Selenium 4+ Grid provides it, so you're likely still using a Selenium 3 server. – titusfortner Apr 22 '22 at 15:22
  • Oh ok, thanks. Appreciate your inputs. Will check with Sauce team. Seeing this issue with executing tests in sauce labs. Thanks!! – Megatherium Apr 22 '22 at 15:31
  • Oh, Sauce. No, Sauce doesn't support devtools at all right now. Doesn't give access to socket connections. What Sauce does have is accessible through "magic strings" and the `#execute_script` method: https://docs.saucelabs.com/insights/debug/#intercept-network-requests – titusfortner Apr 22 '22 at 19:58
  • Yeah, I came across that and tried driver.execute_script('sauce:log', { type: 'sauce:network' }): but didn't get all the info what I got in my local execution with devtools, only got few items from the log, enabled extendeddebugging and See network tab in sauce but the log with above command gives only few requests. – Megatherium Apr 22 '22 at 20:19
0

This should work if you're interested in response as well

log_name = "/path/to/logs/network.log"
log_file = File.open(log_name, 'w')
browser.wd.intercept do |request, &continue|
    continue.call(request) do |response|
      log_file << "#{request.id} \t
                   #{request.method} \t
                   #{response.code} \t
                   #{request.url} \n"
    end
end

and indeed it works, but something is exceptioning

undefined method `each_with_object' for nil:NilClass
/selenium-webdriver-4.1.0/lib/selenium/webdriver/devtools/response.rb:38:in `from'
/selenium-webdriver-4.1.0/lib/selenium/webdriver/common/driver_extensions/has_network_interception.rb:109:in `intercept_response'
/selenium-webdriver-4.1.0/lib/selenium/webdriver/common/driver_extensions/has_network_interception.rb:68:in `block in intercept'
/selenium-webdriver-4.1.0/lib/selenium/webdriver/devtools.rb:155:in `block in callback_thread'```
Luka Vladika
  • 76
  • 1
  • 5
  • Ah, good catch, the code is assuming `responseHeaders` will be there, but that's optional according to https://chromedevtools.github.io/devtools-protocol/tot/Fetch/ would you mind filing a bug with the project? If your use case is reproducible that would be great, otherwise link this SO question and `@titusfortner` in it - https://github.com/SeleniumHQ/selenium/issues/new/choose Thanks! – titusfortner Apr 23 '22 at 01:49
  • Yup seeing the same issue while trying to intercept response `Minitest::UnexpectedError: NoMethodError: undefined method `each_with_object' for nil:NilClass ` – Megatherium Apr 25 '22 at 14:49
  • user name: @titusfortner My chrome version got updated to 101 and now seeing below error message do we need to have new selenium-devtools gem version ? `Minitest::UnexpectedError: LoadError: cannot load such file -- selenium/devtools/v101 C:/Ruby27-x64/lib/ruby/gems/2.7.0/gems/selenium-devtools-0.100.0/lib/selenium/devtools.rb:28:in `require' C:/Ruby27-x64/lib/ruby/gems/2.7.0/gems/selenium-devtools-0.100.0/lib/selenium/devtools.rb:28:in `load_version' C:/Ruby27-x64/lib/ruby/gems/2.7.0/gems/selenium-webdriver-4.0.0/lib/selenium/webdriver/common/driver_extensions/has_devtools.rb:35 – Megatherium Apr 28 '22 at 03:32
  • try running only `require "selenium/devtools"` (without '/v101'), but it seems you don't have it installed (run `gem install selenium-devtools -v0.101`) – Luka Vladika Apr 28 '22 at 10:47
  • can't see it added to rubygems yet though https://rubygems.org/gems/selenium-devtools/versions – Luka Vladika Apr 28 '22 at 10:53
  • yeah, seems like there is an recent update to https://github.com/SeleniumHQ/selenium/blob/trunk/rb/lib/selenium/devtools/version.rb , not sure if need to have a new gem with the same version update – Megatherium Apr 28 '22 at 12:38
  • Sorry, I've been traveling, and the v101 release isn't working for Diego right now, I'm going to get him sorted with it tomorrow. Thanks for your patience. – titusfortner Apr 28 '22 at 23:47
  • No worries. Thanks @titusfortner appreciate it! – Megatherium Apr 29 '22 at 04:35