How to use proxy when using URI.open in Ruby on Rails

Question

I am parsing a web page using nokogiri gem like the code below:

require 'open-uri'

url = 'https://www.google.com/' # example url
Nokogiri::HTML(URI.open(url))

# ... some other codes

I would like to use proxy with open-uri. I have looked through the documentation, but the example code is without using proxy.

How can I use proxy when using URI.open? I would appreciate any help with syntax explanation or some example code. Thank you!

score 2 · Answer 1 · answered Nov 06 '21 at 16:50

URI.open is designed to be an easy to use wrapper for Net::Http but I think you have at least a couple of options:

URI.open (and URI generically) supports the use of an HTTP_PROXY environment variable, like: Automatically adding proxy to all HTTP connections in ruby
Net:Http has options that would allow you to fine tune this and use different types of authentication with the proxy, and it's not particularly hard see: How to set a proxy in rubys net/http?

1 Answers1