2

In my Rails application, I wanted to gunzip a group of XML files I downloaded from an FTP server.

Each XML file is associated with a merchant and consists of product details. I am trying to unzip each file using GzipReader and then want to parse it with Nokogiri::XML::SAX::Document.

The "gz" files are of variable sizes and I was able to unzip files of sizes around 140 MB, but when the file is larger (~428 MB), the application is throwing this error:

cj_unzipping.rb:10:in `read': negative re-allocation size (NoMemoryError)
from cj_unzipping.rb:10:in `block in <main>'

I read "Why do I get a Nokogiri crash and MemoryError: negative re-allocation size?", but there is lot of space left in my disk.

require 'rubygems'
require 'zlib'

Zlib::GzipReader.open("/home/elmiya/project/mywebapp/cj1.xml.gz") { |gz|
  g = File.new("/home/elmiya/project/mywebapp/cj1.xml", "w")
  g.write(gz.read)
  g.close()
}

the line gz.read creates the issue.

I'm running:

Rails version: 4.0.2
Ruby version: 2.0.0p643

Can anyone please help me resolve this issue?

the Tin Man
  • 158,662
  • 42
  • 215
  • 303
Elmiya Agnes
  • 243
  • 2
  • 11
  • 1
    Why is this a Nokogiri or XML problem? Your code doesn't reference Nokogiri at all, you just want to eventually parse XML using Nokogiri. Don't throw every tag you can think of into a question, only the ones that are directly involved. – the Tin Man Nov 19 '15 at 23:14

0 Answers0