2

How can I force Ruby to respect an underscore in a Net::HTTP header?

uri = URI.parse(url)
headers = { 'api_key': 'my_private_key' }
request = Net::HTTP::Post.new(uri, headers)
http = Net::HTTP.new(uri.host, uri.port)
response = http.request(request)

In this case, the header that gets posted is actually Api-Key and the API I am trying to integrate with actually requires api_key -- case sensitive with the underscore.

Tom Rossi
  • 11,604
  • 5
  • 65
  • 96
  • This is going to be tricky. See the [source](https://github.com/ruby/ruby/blob/master/lib/net/http/header.rb) here. Notice that all keys get made case-insensitive on their entry into the headers hash, and then a call to `each_capitalized` reconstitutes them. At best, you might be able to monkeypatch the call to `each_capitalized` in `GenericRequest` and replace it with your own logic. – simonwo Mar 24 '20 at 21:21
  • 1
    If you intend to create a hash with symbol keys use colons. `{ api_key: 'my_private_key' }`. If you want strings use hashrockets `{ 'api_key' => 'my_private_key' }`. Don't do `{ 'api_key': 'my_private_key' }` which looks like a hash with string keys but where they are actually coerced to symbols. – max Mar 24 '20 at 22:10

1 Answers1

1

Net::HTTP forces headers to meet the spec with regard to capitalization and punctuation. You can monkey patch it in a variety of ways (depending on the version of Net::HTTP) but those solutions are all fairly old at this point. Regardless, monkey patching third party libraries is a recipe for disaster.

Any client that relies on Net::HTTP, like HTTParty, has the same problem. You can read about some of these workarounds at https://github.com/jnunemaker/httparty/issues/406, but again I don't recommend them.

You can read some more about issues with underscores in HTTP headers at Why is my custom header not present sometimes? and Why do HTTP servers forbid underscores in HTTP header names.

The easier solution is to use typhoeus which wraps libcurl rather than relying on Net::HTTP. Here's the quickest demonstration of how this works in typhoeus:

require 'typhoeus'
request = Typhoeus.get('www.example.com', headers: {'foo_bar' => 'baz'})
=> #<Typhoeus::Response:0x00007fdf3aa717e8 @options={:httpauth_avail=>0, :total_time=>0.336714, :starttransfer_time=>0.336496, :appconnect_time=>0.0, :pretransfer_time=>0.26573, :connect_time=>0.265662, :namelookup_time=>0.00133, :redirect_time=>0.0, :effective_url=>"www.example.com", :primary_ip=>"93.184.216.34", :response_code=>200, :request_size=>129, :redirect_count=>0, :return_code=>:ok, :response_headers=>"HTTP/1.1 200 OK\r\nAge: 459799\r\nCache-Control: max-age=604800\r\nContent-Type: text/html; charset=UTF-8\r\nDate: Tue, 24 Mar 2020 21:29:22 GMT\r\nEtag: \"3147526947+ident\"\r\nExpires: Tue, 31 Mar 2020 21:29:22 GMT\r\nLast-Modified: Thu, 17 Oct 2019 07:18:26 GMT\r\nServer: ECS (ord/4CD5)\r\nVary: Accept-Encoding\r\nX-Cache: HIT\r\nContent-Length: 1256\r\n\r\n", :response_body=>"<!doctype html>\n<html>\n<head>\n    <title>Example Domain</title>\n\n    <meta charset=\"utf-8\" />\n    <meta http-equiv=\"Content-type\" content=\"text/html; charset=utf-8\" />\n    <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\" />\n    <style type=\"text/css\">\n    body {\n        background-color: #f0f0f2;\n        margin: 0;\n        padding: 0;\n        font-family: -apple-system, system-ui, BlinkMacSystemFont, \"Segoe UI\", \"Open Sans\", \"Helvetica Neue\", Helvetica, Arial, sans-serif;\n        \n    }\n    div {\n        width: 600px;\n        margin: 5em auto;\n        padding: 2em;\n        background-color: #fdfdff;\n        border-radius: 0.5em;\n        box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);\n    }\n    a:link, a:visited {\n        color: #38488f;\n        text-decoration: none;\n    }\n    @media (max-width: 700px) {\n        div {\n            margin: 0 auto;\n            width: auto;\n        }\n    }\n    </style>    \n</head>\n\n<body>\n<div>\n    <h1>Example Domain</h1>\n    <p>This domain is for use in illustrative examples in documents. You may use this\n    domain in literature without prior coordination or asking for permission.</p>\n    <p><a href=\"https://www.iana.org/domains/example\">More information...</a></p>\n</div>\n</body>\n</html>\n", :debug_info=>#<Ethon::Easy::DebugInfo:0x00007fdf3b3dbf40 @messages=[]>}, @request=#<Typhoeus::Request:0x00007fdf3aa72c60 @base_url="www.example.com", @original_options={:headers=>{"foo_bar"=>"baz"}, :method=>:get}, @options={:headers=>{"User-Agent"=>"Typhoeus - https://github.com/typhoeus/typhoeus", "foo_bar"=>"baz", "Expect"=>""}, :method=>:get, :maxredirs=>50}, @on_progress=[], @on_headers=[], @response=#<Typhoeus::Response:0x00007fdf3aa717e8 ...>, @on_complete=[], @on_success=[]>>

Then validate that the headers in your request were set properly:

request.request.options[:headers]
=> {
    "User-Agent" => "Typhoeus - https://github.com/typhoeus/typhoeus",
       "foo_bar" => "baz",
        "Expect" => ""
}

But even so, pay attention to the full stack that may be processing these headers as underscores are still at times problematic for various components in the stack.

I answered a similar question once before at https://stackoverflow.com/a/58459132/3784008.

anothermh
  • 9,815
  • 3
  • 33
  • 52