14

I'm trying to do the following:

page = Mechanize.new.get "https://sis-app.sph.harvard.edu:9030/prod/bwckschd.p_disp_dyn_sched"

But I only get this exception:

OpenSSL::SSL::SSLError: SSL_connect returned=1 errno=0 state=SSLv2/v3 read server hello A: sslv3 alert illegal parameter
    from /Users/amosng/.rvm/gems/ruby-1.9.3-p194/gems/net-http-persistent-2.7/lib/net/http/persistent/ssl_reuse.rb:70:in `connect'
    from /Users/amosng/.rvm/gems/ruby-1.9.3-p194/gems/net-http-persistent-2.7/lib/net/http/persistent/ssl_reuse.rb:70:in `block in connect'
    from /Users/amosng/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/timeout.rb:54:in `timeout'
    from /Users/amosng/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/timeout.rb:99:in `timeout'
    from /Users/amosng/.rvm/gems/ruby-1.9.3-p194/gems/net-http-persistent-2.7/lib/net/http/persistent/ssl_reuse.rb:70:in `connect'
    from /Users/amosng/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/net/http.rb:755:in `do_start'
    from /Users/amosng/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/net/http.rb:750:in `start'
    from /Users/amosng/.rvm/gems/ruby-1.9.3-p194/gems/net-http-persistent-2.7/lib/net/http/persistent.rb:511:in `connection_for'
    from /Users/amosng/.rvm/gems/ruby-1.9.3-p194/gems/net-http-persistent-2.7/lib/net/http/persistent.rb:806:in `request'
    from /Users/amosng/.rvm/gems/ruby-1.9.3-p194/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:258:in `fetch'
    from /Users/amosng/.rvm/gems/ruby-1.9.3-p194/gems/mechanize-2.5.1/lib/mechanize.rb:407:in `get'
    from (irb):549
    from /Users/amosng/.rvm/rubies/ruby-1.9.3-p194/bin/irb:16:in `<main>'

How can I get the webpage to load in Mechanize?

wrongusername
  • 18,564
  • 40
  • 130
  • 214

1 Answers1

27

Sometimes you need to tell mechanize to use sslv3:

page = Mechanize.new{|a| a.ssl_version, a.verify_mode = 'SSLv3', OpenSSL::SSL::VERIFY_NONE}.get "https://sis-app.sph.harvard.edu:9030/prod/bwckschd.p_disp_dyn_sched"

Notice that I use OpenSSL::SSL::VERIFY_NONE. That means you are theoretically vulnerable to man-in-the-middle attack, but that's not something I generally worry about when I'm scraping a website.

pguardiario
  • 53,827
  • 19
  • 119
  • 159
  • Could you give me the doc link,from where you got this? – Arup Rakshit Jan 21 '13 at 18:49
  • 1
    It might not be there, I think I noticed it while inspecting Mechanize#methods – pguardiario Jan 22 '13 at 00:04
  • 1
    A similar error I received that had the same solution was: `~/.rvm/gems/ruby-1.9.3-p374/gems/net-http-persistent-2.8/lib/net/http/persistent/ssl_reuse.rb:70:in \`connect': SSL_connect returned=1 errno=0 state=SSLv2/v3 read server hello A: (null) (OpenSSL::SSL::SSLError)` – miknight Jan 28 '13 at 12:21
  • 2
    setting `ssl_version = 'SSLv3'` solved my immediate problem, but I'm curious to understand more about why this is the case sometimes. Especially given that the error message says `state=SSLv2/v3` – Rob May 20 '13 at 15:29
  • I trid this and got.. SSL_connect returned=1 errno=0 state=error: no protocols available (OpenSSL::SSL::SSLError) – Lefty Feb 17 '21 at 18:49