0

I take code from, because I need get content with US proxy and good user agent.

How to GET a URL with User-Agent and timeout through some Proxy in Ruby?

require 'rubygems'
require 'nokogiri'
require 'open-uri'

    user_agent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_0) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.854.0 Safari/535.2"

    url = "google.com"

    @doc = Nokogiri::HTML(open(url, 'User-Agent' => user_agent, 'read_timeout' => 10 ), nil, "UTF-8")

But I got error:

/usr/lib/ruby/2.3.0/net/http/header.rb:18:in `block in initialize_http_header': undefined method `strip' for 10:Fixnum (NoMethodError)
    from /usr/lib/ruby/2.3.0/net/http/header.rb:16:in `each'
    from /usr/lib/ruby/2.3.0/net/http/header.rb:16:in `initialize_http_header'
    from /usr/lib/ruby/2.3.0/net/http/generic_request.rb:44:in `initialize'
    from /usr/lib/ruby/2.3.0/net/http/request.rb:15:in `initialize'
    from /usr/lib/ruby/2.3.0/open-uri.rb:320:in `new'
    from /usr/lib/ruby/2.3.0/open-uri.rb:320:in `block in open_http'
    from /usr/lib/ruby/2.3.0/net/http.rb:853:in `start'
    from /usr/lib/ruby/2.3.0/open-uri.rb:319:in `open_http'
    from /usr/lib/ruby/2.3.0/open-uri.rb:737:in `buffer_open'
    from /usr/lib/ruby/2.3.0/open-uri.rb:212:in `block in open_loop'
    from /usr/lib/ruby/2.3.0/open-uri.rb:210:in `catch'
    from /usr/lib/ruby/2.3.0/open-uri.rb:210:in `open_loop'
    from /usr/lib/ruby/2.3.0/open-uri.rb:151:in `open_uri'
    from /usr/lib/ruby/2.3.0/open-uri.rb:717:in `open'
    from /usr/lib/ruby/2.3.0/open-uri.rb:35:in `open'
    from work6.rb:10:in `<main>'
the Tin Man
  • 158,662
  • 42
  • 215
  • 303
Denis
  • 1
  • 1
  • Pass 10 as string not integer value – Vrushali Pawar Jun 14 '17 at 06:33
  • Nokogiri doesn't know anything about user-agents, proxies, or any other HTTP related issues because it isn't an HTTP client so this isn't a Nokogiri question. Notice that in the trace that Nokogiri isn't mentioned at all? – the Tin Man Jun 14 '17 at 22:13
  • strange but I open my host with script and in logs I see normal UA. Maybe not proxy. But UA. – Denis Jun 15 '17 at 04:24
  • What HPP client you recomend to me to use together with proxy and UA? – Denis Jun 15 '17 at 04:25

0 Answers0