Can I reduce duplication in the serialization of a ruby hash?

Question

I have a hash of the format

{com: 1234, users: [{nid: 3, sets: [1,2,3,4]}, {nid: 4, sets: [5,6,7,8]}]}

which I am sending to a remote server. I am using the HTTParty gem to do this. The code looks like this

class Api
    include HTTParty
    attr_accessor :headers

    def initialize
      @headers = { 'Content-Type' => 'application/json' }
    end
    
    def post_com(hsh)
      response = self.class.post('some_url', query: hsh, headers: headers, format: :plain)
    end
end

When I do

api = Api.new.post_com({com: 1234, users: [{nid: 3, sets: [1,2,3,4]}, {nid: 4, sets: [5,6,7,8]}]}

at the remote server, the hash is being sent in the following format

POST "/some_url?com=1234&users[][nid]=3&users[][sets][]=1&users[][sets][]=2&users[][sets][]=3&users[][sets][]=4&users[][nid]=4&users[][sets][]=5&users[][sets][]=6&users[][sets][]=7&users[][sets][]=8

This means for every entry in set, duplicate characeters users[][sets][] are being sent. In operation, there can be many entries in set, and the result is the server rejects the post as having too many characters.

Is there anyway I can have the hash serialized with far less duplication. For instance if I just do

{com: 1234, users: [{nid: 3, sets: [1,2,3,4]}, {nid: 4, sets: [5,6,7,8]}]}.to_json

I receive

"{\"com\":1234,\"users\":[{\"nid\":3,\"sets\":[1,2,3,4]},{\"nid\":4,\"sets\":[5,6,7,8]}]}"

which has far fewer characters.

are you sure it's being sent as json? ( i mean the content_type, does the receiver identify this request as json?) — Joel Blum, Aug 08 '21 at 08:44

rmlockerd · Accepted Answer · 2021-08-19T06:29:14.430

1

HTTParty, by default, converts the :query hash into what it calls 'rails style query parameters':

For a query:
get '/', query: {selected_ids: [1,2,3]}

The default query string looks like this:
/?selected_ids[]=1&selected_ids[]=2&selected_ids[]=3

Since you are doing a POST, it is possible/preferable to send your hash in the body of the request rather than in the query string.

def post_com(hsh)
  self.class.post('some_url', body: hsh.to_json, headers: headers, format: :plain)
end

This has the advantage that it doesn't do any transformation of the payload and the query string length limit doesn't apply anyway.

For the record, you can disable the default 'rails style' encoding like this:

class Api
  include HTTParty
  disable_rails_query_string_format

  ...
end

You can also roll your own custom query string normalizer by passing a Proc to query_string_normalizer.

edited Aug 19 '21 at 06:29

answered Aug 08 '21 at 06:04

rmlockerd

3,776
2
15
25

I had a chance to try this and just replacing :query with :body in the post request does not work. As explained here https://stackoverflow.com/questions/19461333/how-can-i-implement-this-post-request-using-httparty, the hsh needs to be converted json. So the correct request should read: self.class.post('some_url', body: hsh.to_json, headers: headers, format: :plain). Will I edit the question or you do want to do it? – Obromios Aug 19 '21 at 03:23
I don't mind changing it, but I'm puzzled since that was a straight copy out of a test project I set up. – rmlockerd Aug 19 '21 at 06:28
Perhaps we have different settings. Perhaps note that depending on the setup, you may need to convert the hash to json. – Obromios Aug 21 '21 at 23:04

Can I reduce duplication in the serialization of a ruby hash?

1 Answers1