23

I've a task to test different user agents on a URL through automation. I'm using ruby to code, and I've been trying to set an user agent using the following method, but it doesn't seem to recognize the user agent. 

@http = Net::HTTP.new(URL)
response = @http.request_get(URL, {'User-Agent' => useragent})

Is there any other way to do this, or what am I doing wrong?

Nakilon
  • 34,866
  • 14
  • 107
  • 142
rubytester
  • 339
  • 1
  • 3
  • 5

5 Answers5

31
http = Net::HTTP.new("your.site.com", 80)
req = Net::HTTP::Get.new("/path/to/the/page.html", {'User-Agent' => 'your_agent'})
response = http.request(req)
puts response.body

Works great for me.

Riba
  • 1,108
  • 8
  • 10
  • Is there a way to set it globally so you don't have to set the hash on each call? –  Apr 24 '14 at 17:34
23

Also another that work for me :

require 'open-uri'
html = open('http://your.site.com/the/page.html', 'User-Agent' => 'Ruby').read
puts html

Hope this will help you.

Riba
  • 1,108
  • 8
  • 10
3

The included Net::HTTPHeader has the initialize_http_header method:

@http = Net::HTTP.new(URL)
@http.initialize_http_header({'User-Agent' => useragent})
response = @http.request_get(URL)

HTH

Hertzel Guinness
  • 5,912
  • 3
  • 38
  • 43
1

I wasn't able to find a solution that works for both https and supplying a header. Here is a version that works:

require "net/http"

uri = URI("https://pokemongolive.com/events/community-day/")

request = Net::HTTP::Get.new(uri)
request["User-Agent"] = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"

response = Net::HTTP.start(uri.hostname, uri.port, :use_ssl => (uri.scheme == 'https')) {|http|
  http.request(request)
}

puts response.code
puts response.body

The Ruby version being used was: ruby 2.6.3p62 (2019-04-16 revision 67580)

nonopolarity
  • 146,324
  • 131
  • 460
  • 740
0
require 'net/https'

uri = URI('https://example.com')
params = {
  'user-agent' => '...'
}

res = Net::HTTP.get_response(uri, params)
puts res.code
puts res.body
Ray
  • 238
  • 5
  • 8
  • Please read "[answer]" and "[Explaining entirely code-based answers](https://meta.stackoverflow.com/q/392712/128421)". It helps more if you supply an explanation why this is the preferred solution and explain how it works. We want to educate, not just provide code. – the Tin Man Mar 20 '22 at 23:05