EOF with Nokogiri

Question

I have the following line in a long loop

page = Nokogiri::HTML(open(topic[:url].first)).xpath('//ul[@class = "pages"]//li').first

Sometimes my Ruby application crashes raising the "End of file reached " exception in this line.

How can I resolve this problem? Just a begin;raise;end block?

Is a script that performs a forum backup, so is important that doesn't skip any thread.

Thanks in advance.

Answer 1

In addition to @Phrogz's excellent advice (in particular about at_css with the simpler expression), I would pull the raw xml [content] separately:

page = if (content = open(topic[:url].first)).strip.length > 0
  Nokogiri::HTML(content).xpath('//ul[@class = "pages"]//li').first
end

Answer 2

I would suggest that you should first to fix the underlying issue so that you do not get this error.

• Does the same URL always cause the problem? (Output it in your log files.) If so, perhaps you need to URI encode the URL.
• Is it random, and therefor likely related to a connection hiccup or server problem? If so, you should rescue the specific error and then retry one or more times to get the crucial data.

Secondarily, you should know that the CSS syntax for that query is far simpler:

page = Nokogiri.HTML(...).at_css('ul.pages li')

• Not only is this less than half the bytes, it allows for cases like <ul class="foo pages"> that the XPath would miss.
• Using at_css (or at_xpath) is the same as .css(...).first, but is faster and simpler.