Header parameter missing when making POST request using Rest-Client Ruby Gem

Question

Hi I am successfully able to post from Post man , but unable to do so from Ruby Rest client.

Post Man Request

POST /endpoint HTTP/1.1
Host: host:11400
Accept: application/json
HTTP_USER: userid
fname: fname
lname: lname
Content-Type: application/json
Cache-Control: no-cache
Postman-Token: 589a5345-e384-bd71-d690-60987165487b

{ "rtdt":"09/08/2016","jobs":[{"pid":53} , {"pid":54}]}

the rest code I ve tried multiple ways including below

method tried

p RestClient.post 'http://url_details',
                  Accept: 'application/json',
                  'HTTP_USER'.to_sym => 'userid',
                  fname: 'name',
                  lname: '',
                  'Content-Type'.to_sym => 'application/json',
                  payload: JSON.parse('{ "rtdt":"09/08/2016","jobs":[{"pid":53} , {"pid":54}]}')

method tried

p RestClient.post 'http://url_details/job', http_user:  'userid', content_type: :json, accept: :json

method tried

p @uber_ride = (RestClient::Request.execute(
    :method => :post,
    :url => 'http://url_details/job',
    'HTTP_USER' => 'userid',
    :headers => {:content_type => 'application/json', :accept => 'application/json', :HTTP_USER => 'userid', :fname => 'name', :lname => 'lname'}
))

method tried

p '++++++++++++++++++++++++++++++++'
p RestClient.post($Current_Api_Endpoint,
                  $Current_Payload,
                  :http_user =>'userid',
                  headers:
                      {:accept => 'application/json',
                       :content_type => 'application/json',
                       :http_user =>'userid',
                       :fname => 'name',
                       :lname => 'lname'}
                )


p '++++++++++++++++++++++++++++++++'

method tried

p env1 = ENV['http_proxy']
# p a = {:method => :post, :url => 'http://url_details/job', :headers => { :accept => 'application/json', :content_type => 'application/json', :http_user => 'userid', :fname => 'name', :lname => 'lname' }, :payload => { :rtdt => "06/10/2016",:jobs => [{:pid => 53} , {:pid => 54}]} }
p a = {
        method: 'post',
        url: 'http://url_details/job',
        proxy: ENV['http_proxy'],
        headers: {accept: 'application/json', content_type: 'application/json', http_user: 'userid', fname: 'name', lname: 'lname'},
        payload: { rtdt: '06/10/2016', jobs: {pid: 53}}

      }

I am getting error HTTP Status 500 - HTTP_USER header not found in request torg.springframework.security.web.authentication

I Found out the issue since then. All headers are able to be sent, except this one - This one in CAPS. rest of them in small characters. How to solve it

Why did you call it `heaaaaaaaders`? If you're going to try at least use the right key. — tadman, Sep 22 '16 at 22:51
the headers spelleing is fine in my requst. For Stack overflow I modified values to post it — Kiul Logoto, Sep 22 '16 at 22:53
Also do try and avoid using global variables like `$Current_Api_Endpoint`. The Ruby way of doing this is a constant such as `CURRENT_API_ENDPOINT` or even better, a namespaced one wrapped by a module method like `ApiEndpoint.current_url`. — tadman, Sep 22 '16 at 23:06

score 1 · Answer 1 · edited May 23 '17 at 10:34

Ruby's Net::HTTP implementation does not transmit ALL_CAPS_UNDERSCORED header names. RestClient relies on Net::HTTP. So, for example, the HTTP_USER header is actually transformed into Http_user when transmitted.

Alertnative: curb

Curl, however, does not have this limitation. It transmits header names verbatim. Therefore instead of using RestClient, you could use the curb gem, which is a wrapper around libcurl. This seems to work:

require "curb"
Curl::Easy.http_post("http://URL_REDACTED") do |http|
  http.headers["Accept"] = "application/json"
  http.headers["Content-Type"] = "application/json"
  http.headers["HTTP_USER"] = "userid"
  http.post_body = '{ "rtdt":"09/08/2016","jobs":[{"pid":53} , {"pid":54}]}'
end

Alternative: excon

The excon gem is another possibility. It implements HTTP in pure Ruby, without relying on Net::HTTP. It does its own header processing and allows ALL_CAPS headers. So this should work:

require "excon"
Excon.post(
  "http://URL_REDACTED",
  :headers => {
    "Accept" => "application/json",
    "Content-Type" => "application/json",
    "HTTP_USER" => "userid"
  },
  :body => '{ "rtdt":"09/08/2016","jobs":[{"pid":53} , {"pid":54}]}'
)

Anyway, I would argue that the server you are trying to connect to is ultimately at fault here, because headers are supposed to be evaluated as case-insensitive: Are HTTP headers case-sensitive?. If the server requires a header be in all uppercase, that is a bug.

Plenty of error and proxy and firewall issues getting Curl libraries in Windows. Could not install. Any way can we push Caps header at all in Rest Client ? — Kiul Logoto, Sep 23 '16 at 17:59
No, RestClient relies on Ruby Net::HTTP, which is where the header capitalization occurs. I suppose you could monkey-patch Net::HTTP, but that seems severe. https://github.com/ruby/ruby/blob/5a121a4f0b8fb9d416566f75047a190166d98609/lib/net/http/generic_request.rb#L323-L335 — Matt Brictson, Sep 23 '16 at 21:33
I updated my answer with a suggestion to use the excon gem, which does not rely on Ruby Net::HTTP. It should work. — Matt Brictson, Sep 23 '16 at 21:40
Thanks Matt, I will check on the same. I need to get both Excon and CURB working on both mac/linux/windows to ensure we are able to hit the same. I dont have a mac now , so working with colleagues in their free time. @MattBrictson can the question be upvoted too? — Kiul Logoto, Sep 26 '16 at 14:37

score 0 · Answer 2 · answered Sep 22 '16 at 23:34

The RestClient README examples indicate that the correct usage is:

RestClient.post(url, payload, headers)

So all you have to do is:

RestClient.post(
  "http://URL_REDACTED",
  '{ "rtdt":"09/08/2016","jobs":[{"pid":53} , {"pid":54}]}',
  content_type: :json, 
  accept: :json,
  "HTTP_USER" => "userid"
)

I tested against http://requestb.in and got the expected headers and body:

HEADERS

Total-Route-Time: 0
Connection: close
Via: 1.1 vegur
Connect-Time: 0
Http-User: userid
Content-Type: application/json
Content-Length: 55
User-Agent: rest-client/2.0.0 (darwin15.4.0 x86_64) ruby/2.3.1p112
X-Request-Id: c7cd819f-8901-41b9-8cbb-3b6d1895cd67
Accept-Encoding: gzip
Host: requestb.in
Accept: application/json

RAW BODY

{ "rtdt":"09/08/2016","jobs":[{"pid":53} , {"pid":54}]}

I tried this Matt, not working. Also same error, and is HTTP_USER for spring framework needs special handling or something? — Kiul Logoto, Sep 23 '16 at 14:02

Header parameter missing when making POST request using Rest-Client Ruby Gem

2 Answers2