1

I'd like to create a program that generates content from very large XML files, upwards of 500mb in size. I'll need to get data from the files at various times, but the user is willing to wait for a bit, and it's all done on the local machine.

I was womdering if anyone had any advice regarding:

Implementation languages Whether x path is enough for light querying on an (admittedly huge) database Any other advice

I probably only need to use less than 1% of the data, and i can't expect to do processing beforehand to prepare the file.

Any tips?

In response to the comment: I could break the file up, but only by reading it in, and writing it out again. So not really, essentially. I only use the file once, to generate this 'content' using select (and nondeterministically chosen) entries in the xml file given. Then I never need that file again.

mtrc
  • 1,317
  • 3
  • 16
  • 39
  • Do you have any control over the XML file? Can you break it down into smaller pieces? How often do you process the file? Do you ever process the same file more than once? – Brandon Boone Aug 05 '10 at 00:31

0 Answers0