1

Getting a 416 error when trying to GET a website with HTTParty. Works just fine in the browser.

I have never gotten this error before, so I went online and found this:

It occurs when the server is unable to fulfill the request. This may be, for example, because the client asked for the 800th-900th bytes of a document, but the document is only 200 bytes long.

The request includes a Range request-header field, and not any of the range-specifier values in this field overlaps the current extent of the selected resource, and also the request does not include an If-Range request-header field.

Wondering if anyone has gotten 416 with HTTParty before and if there is a way to prevent this form happening. Thanks

Example website where error occurs: http://www.bizjournals.com/jacksonville/blog/morning-edition/2014/07/teens-make-up-less-of-summer-workforce-than-ever.html

Community
  • 1
  • 1
Petr Gazarov
  • 3,602
  • 2
  • 20
  • 37

1 Answers1

1

It appears that bizjournals is able to detect you are a bot (not accessing in the browser) and therefore returns a 416.

irb(main):005:0> HTTParty.get('http://www.bizjournals.com/jacksonville/blog/morning-edition/2014/07/teens-make-up-less-of-summer-workforce-than-ever.html').body
=> "........As you were browsing <strong>http://www.bizjournals.com</strong> something about your browser made us think you were a bot. There are a few reasons this might happen........"

You could either ask bizjournals to allow you to make requests or try to change the headers to make bizjournals think you are not a bot.

kcdragon
  • 1,724
  • 1
  • 14
  • 23