How to convert Nokogiri HTML node into JSON

Question

I output the HTML node (Nokogiri::XML::Element < Nokogiri::XML::Node), which looks like this:

<prog>
  <prog_name>Barclay CTA Index</prog_name>
  <prog_id>9</prog_id>
</prog>

I want to get the hash like:

{
    prog_name: "Barclay CTA Index"
    prog_id: 9
}

The to_hash method is not working:

[3] pry(main)> prog_element.to_hash
[]

score 0 · Answer 1 · answered Sep 25 '15 at 08:20

0

Try this one:

Hash.from_xml(prog_element.to_xml).to_json

answered Sep 25 '15 at 08:20

Arsen

10,815
2
34
46

1

isn't `from_xml` `rails` specific? – ptierno Sep 25 '15 at 08:40
If you're going to recommend using a Rails method in a non-Rails question, you need to explain how to access it. Simply throwing the methods into code only adds confusion. – the Tin Man Sep 25 '15 at 16:14

the Tin Man · Answer 2 · 2015-09-25T17:36:18.270

It's easy to build a hash if your XML is that trivial:

require 'json'
require 'nokogiri'

xml = <<EOT
<prog>
  <prog_name>Barclay CTA Index</prog_name>
  <prog_id>9</prog_id>
</prog>
EOT

doc = Nokogiri::XML(xml)

prog_name = doc.at('prog_name').text
prog_id = doc.at('prog_id').text.to_i

hash = {
  prog_name: prog_name,
  prog_id: prog_id
}

hash # => {:prog_name=>"Barclay CTA Index", :prog_id=>9}

puts hash.to_json
# >> {"prog_name":"Barclay CTA Index","prog_id":9}

If your XML isn't that simple (which should have been reflected in the example) then it's a little more involved:

require 'json'
require 'nokogiri'

xml = <<EOT
<xml>
  <prog>
    <prog_name>Barclay CTA Index</prog_name>
    <prog_id>9</prog_id>
  </prog>
  <prog>
    <prog_name>foo</prog_name>
    <prog_id>1</prog_id>
  </prog>
</xml>
EOT

doc = Nokogiri::XML(xml)

hash = {
  'prog' => []
}

doc.search('prog').each do |prog|
  prog_name = doc.at('prog_name').text
  prog_id = doc.at('prog_id').text.to_i

  hash['prog'] << {
    prog_name: prog_name,
    prog_id: prog_id
  }
end

hash
# => {"prog"=>
#      [{:prog_name=>"Barclay CTA Index", :prog_id=>9},
#       {:prog_name=>"Barclay CTA Index", :prog_id=>9}]}

puts hash.to_json
# >> {"prog":[{"prog_name":"Barclay CTA Index","prog_id":9},{"prog_name":"Barclay CTA Index","prog_id":9}]}

XML tends to be verbose, whereas JSON is pretty terse, so often there isn't a need for a one-to-one mapping of the XML nodes to JSON, allowing us to pick and choose how the JSON will look. That's a big advantage if the XML is huge and contains a lot of data you don't need to pass on.

If you want to convert the XML verbatim, you can take advantage of other gems or use Rails' from_xml which is found in Active Support's core extensions to Hash but not documented there. It's documented in the Active Support Hash documentation though:

require 'active_support/core_ext/hash/conversions'
require 'json'

xml = <<EOT
<xml>
  <prog>
    <prog_name>Barclay CTA Index</prog_name>
    <prog_id>9</prog_id>
  </prog>
  <prog>
    <prog_name>foo</prog_name>
    <prog_id>1</prog_id>
  </prog>
</xml>
EOT

hash = Hash.from_xml(xml)
# => {"xml"=>
#      {"prog"=>
#        [{"prog_name"=>"Barclay CTA Index", "prog_id"=>"9"},
#         {"prog_name"=>"foo", "prog_id"=>"1"}]}}

puts hash.to_json

# >> {"xml":{"prog":[{"prog_name":"Barclay CTA Index","prog_id":"9"},{"prog_name":"foo","prog_id":"1"}]}}

Notice that using from_xml doesn't convert the prog_id values from strings to ints like they probably should be, which means that they'll be passed as strings by the JSON serialization too. Also, if there are unneeded nodes in the XML, they'll get passed on to the JSON, bloating it and slowing the transfer, so you need to decide whether you want to inflict that on the user's browser or the receiving end.

Working with Nokogiri to convert XML into hashes isn't that hard once you understand XML and Nokogiri, and having that knowledge is a good set of tools for your tool chest. Relying on something like from_xml is OK because it's convenient, but you're trading off that knowledge and the ability to do custom things and slice and dice. Manipulating and accessing data in hashes often turns out to be worse than doing it directly with a tool like Nokogiri, hence its existence.

How to convert Nokogiri HTML node into JSON

2 Answers2