3

Occationally, when I visit a website with HTTParty or Mechanize, I get this error:

hostname "www.example.com" does not match the server certificate

I can see that there is a workaround if you use the open method, but I'm not sure how to leverage that the above gems.

Stacktrace for Mechanize:

agent = Mechanize.new
agent.read_timeout              = 180
agent.open_timeout              = 180
agent.user_agent_alias          = 'Mac Safari'
agent.redirect_ok               = :all
agent.follow_meta_refresh       = :anywhere
agent.follow_meta_refresh_self  = true
agent.get("https://some-domain.com")
/home/me/.rbenv/versions/2.2.2/lib/ruby/2.2.0/openssl/ssl.rb:232:in `post_connection_check'
/home/me/.rbenv/versions/2.2.2/lib/ruby/2.2.0/net/http.rb:925:in `connect'
/home/me/.rbenv/versions/2.2.2/lib/ruby/2.2.0/net/http.rb:863:in `do_start'
/home/me/.rbenv/versions/2.2.2/lib/ruby/2.2.0/net/http.rb:858:in `start'
/home/me/applications/myapp/shared/bundle/ruby/2.2.0/gems/net-http-persistent-2.9.4/lib/net/http/persistent.rb:700:in `start'
/home/me/applications/myapp/shared/bundle/ruby/2.2.0/gems/net-http-persistent-2.9.4/lib/net/http/persistent.rb:965:in `reset'
/home/me/applications/myapp/shared/bundle/ruby/2.2.0/gems/net-http-persistent-2.9.4/lib/net/http/persistent.rb:628:in `connection_for'
/home/me/applications/myapp/shared/bundle/ruby/2.2.0/gems/net-http-persistent-2.9.4/lib/net/http/persistent.rb:994:in `request'
/home/me/applications/myapp/shared/bundle/ruby/2.2.0/gems/mechanize-2.7.4/lib/mechanize/http/agent.rb:267:in `fetch'
/home/me/applications/myapp/shared/bundle/ruby/2.2.0/gems/mechanize-2.7.4/lib/mechanize.rb:464:in `get'

Stacktrace for HTTParty:

HTTParty.get("https://some-domain.com")
/home/me/.rbenv/versions/2.2.2/lib/ruby/2.2.0/openssl/ssl.rb:232:in `post_connection_check'
/home/me/.rbenv/versions/2.2.2/lib/ruby/2.2.0/net/http.rb:925:in `connect'
/home/me/.rbenv/versions/2.2.2/lib/ruby/2.2.0/net/http.rb:863:in `do_start'
/home/me/.rbenv/versions/2.2.2/lib/ruby/2.2.0/net/http.rb:852:in `start'
/home/me/.rbenv/versions/2.2.2/lib/ruby/2.2.0/net/http.rb:1375:in `request'
/home/me/applications/myapp/shared/bundle/ruby/2.2.0/gems/httparty-0.13.7/lib/httparty/request.rb:117:in `perform'
/home/me/applications/myapp/shared/bundle/ruby/2.2.0/gems/httparty-0.13.7/lib/httparty.rb:545:in `perform_request'
/home/me/applications/myapp/shared/bundle/ruby/2.2.0/gems/httparty-0.13.7/lib/httparty.rb:476:in `get'
jww
  • 97,681
  • 90
  • 411
  • 885
Cjoerg
  • 1,271
  • 3
  • 21
  • 63
  • You should create a server certificate with the proper names. For that, see [How to create a self-signed certificate with openssl?](http://stackoverflow.com/a/27931596/608639) – jww Jun 29 '16 at 16:21

1 Answers1

4

For Mechanize this should set verify SSL to none

agent = Mechanize.new
agent.verify_mode = OpenSSL::SSL::VERIFY_NONE

For HTTParty there's verify: option See this so question How do I make HTTParty ignore SSL?

If you want to set it generally, use this dirty trick:

OpenSSL::SSL::VERIFY_PEER = OpenSSL::SSL::VERIFY_NONE
Community
  • 1
  • 1
Stan Brajewski
  • 452
  • 2
  • 5
  • Very bad advice; see [The most dangerous code in the world: validating SSL certificates in non-browser software](http://crypto.stanford.edu/~dabo/pubs/abstracts/ssl-client-bugs.html). Perhaps the OP should create a certificate with the proper names. For that, see [How to create a self-signed certificate with openssl?](http://stackoverflow.com/a/27931596/608639) – jww Jun 29 '16 at 16:20
  • @jww Yeah, sure. But the OP asked for workaround for Mechanize and HTTParty. I've provided the workaround. Those are workarounds not super-clear-extra-beaty solutions. Sometimes the problem is on the server side (not the client), and you can only do workarounds. That's not the reason for downvote, as I've provided what the op asked for. I hope you get my point of view. OP wanted to set VERIFY_NONE. I've answered. So please do remove downvote. – Stan Brajewski Jun 29 '16 at 19:00
  • @jww You can't create a certificate when you don't own the server. Looks like OP is scraping a lot, that's how I understand OP's question. For me your comment about creating the cert is not relevant the OPs problem. Downvoting too, as the question is: I don't own the server. There's SSL problem. I want to get the page! Even every browser does allow You to choose 'Visit the website!' when there's problem with the certificate. You just get the warning, but it's still possible to get the page with wrong cert. – Stan Brajewski Jun 29 '16 at 19:40
  • If its the case the he does not control the server, then he should end the current connection, and fallback to a new connection using Anonymous Diffie-Hellman. It also saves bytes on the wire since the server certificate is not needed for ADH and EC-ADH. – jww Jun 29 '16 at 19:50
  • @jww so post the answer for the OP so he can complete his task. I don't know how to use your technique. Can you provide the Mechanize and/or HTTParty solution to fetch the page with bad cert? You downvoted so please provide better solution. If you don't have the better solution so why do you downvote? – Stan Brajewski Jun 29 '16 at 19:52
  • I'm not going near this with an answer. The correct solution is to fix the server certificate. Plus, I'm not sure we can answer it as-is because of all the fake information. – jww Jun 29 '16 at 19:57
  • @jww Ah, I see. You don't have working solution (OP asked for workaround) but you have a lot to say and of course you're right with your opinion, even if you don't have a solution for OP. I suggest you read the question carefully again (OP needs workaround). – Stan Brajewski Jun 30 '16 at 08:41
  • I have upvoted the answer, because it does address the question. It's clearly not a viable solution to fix someone elses server. Before mark the answer, could you maybe check if the mechanize method is correct. It returns `undefined method `http' for #`. – Cjoerg Jun 30 '16 at 16:11
  • yes, it should be just agent.verify_mode = OpenSSL::SSL::VERIFY_NONE - i will correct the answer – Stan Brajewski Jul 01 '16 at 09:00