0

How can you use a proxy with authentication with Watir?

From the documentation

proxy = {
  http: 'my.proxy.com:8080',
  ssl:  'my.proxy.com:8080'
}
chrome_browser  = Watir::Browser.new :chrome, proxy: proxy

My proxy is structured like ip:port:username:password and doesn't appear to be working. How can I use this proxy with Watir?

user2954587
  • 4,661
  • 6
  • 43
  • 101

1 Answers1

1

For posterity:

You can use https://github.com/SebastianJ/proxy-chain-rb to setup a proxy on localhost without authentication that redirects to your actual proxy.

real_proxy = "http://username:password@host:port"
server = ProxyChainRb::Server.new
generated_proxy = server.start(real_proxy)

proxy = {
  http: generated_proxy,
  ssl:  generated_proxy
}

Watir::Browser.new(:chrome, proxy: proxy)

* do your scraping *

server.stop

If running headless mode is not a requirement, you can also use an extension.

Create the zip file described in Selenium using Python: enter/provide http proxy password for firefox (obviously, with your own proxy information).

Create a .crx file from the zip. You can do this in Chrome -> Settings -> Extensions -> toggle Developer mode -> Pack extension

options = { extensions: ['path of .crx file'] }
browser = Watir::Browser.new :chrome, options: options

Do note that this will not work in headless Chrome.

Manfred21
  • 11
  • 1