2

I'm facing a situation where I have to write 300 MB of data into XML. Using Streams I have perfectly executed the task without facing OutOfMemory exception.

Now I have to convert the FileOutputStream object into actual data and pass it as String to other method.

I have tried and I'm not getting the expected result. Kindly let me know how to convert FileOutputStream object to String value?

Kaidul
  • 15,409
  • 15
  • 81
  • 150
macOsX
  • 447
  • 1
  • 9
  • 19
  • So you need to use the data you just wrote to pass data to another object or is the data elsewhere too? – Skepi Apr 02 '13 at 08:34
  • I have to read this FileOutputStream object and take that string value and pass it to another method for other process. – macOsX Apr 02 '13 at 08:37
  • so you want: read line from file and this line pass to another methods argument, true? – Simon Dorociak Apr 02 '13 at 09:10
  • what are you using to write data to XML? – linski Apr 02 '13 at 09:42
  • @linski - I'm using stream to write data into the XML. – macOsX Apr 02 '13 at 10:10
  • yes, but how are you generating the XML structure? are you using any XML API - [like this](http://stackoverflow.com/questions/4142046/create-xml-file-using-java)? Or you are creating the XML structure manually? – linski Apr 02 '13 at 10:25
  • Can you say a bit more about your original task? From my experience - it is almost always possible to avoid using String explicitly while working with XMLs, I mean - possible to avoid storing the whole file in one String. So, what's the task? – Alex Kreutznaer Apr 02 '13 at 16:06

3 Answers3

3

If your String is too big and you are trying to keep it in memory at a time, it will always cause you OutOfMemoryError independent of the fact which reader you use.

So, the problem is not your Reader. The problem is your String object (the one you want to pass).

There is definitely something wrong with your architecture, if you need to store/pass such a huge file as a String variable.

Your mistake is that you are trying to put your XML text in a Java String variable. You should avoid storing big objects in memory, that's the key point here.

Generally texts are a subject to compression with the compression ratio of 0.1 - 0.01.

Try to use ZipOutputStream/ZipInputStream (or some similar libraries) - they work fine with streams reading/writing to files.

In my practice it works perfectly with huge XML files.

Alex Kreutznaer
  • 1,170
  • 8
  • 18
  • Thanks for your reply. As you mentioned we need to change the architecture. There is no othe option. +1 for explaining it clearly. – macOsX Apr 03 '13 at 05:13
0

How are you writing in file? FileOutputStream creation is OK, but when you write using FileOutputStream you have to pass something to it. When you are passing the data to FileOutputStream.write(some_data), why don't you write the same in a StringBuffer at the same time?

Abhinaba Basu
  • 361
  • 1
  • 9
  • StringBuffer internally uses append method. When writing huge data it will give you OutOfMemory exception. – macOsX Apr 02 '13 at 08:46
  • Are you trying to append whole data at a time? How are you writing in the outputStream, all at a once or byte by byte? In case you are writing byte by byte, inside the same loop if you appent to StringBuffer does it throw exception? you can also check this: [link](http://stackoverflow.com/questions/2431040/java-outofmemoryerror-with-stringbuilder) – Abhinaba Basu Apr 02 '13 at 08:56
0

You cannot read from a FileOutputStream. Try FileReader to read the data from a file.

Couldnt you use the data you originally used to create the file? If not you could use an XML Parser like SAX or JDOM to read the data

Skepi
  • 478
  • 3
  • 12
  • Now I have FileOutputStream object and I have to convert it to a string. How we can do that? – macOsX Apr 02 '13 at 10:13
  • I am confused as to what you want to do. Have you written to a file yet? What are you expecting by converting the FileOutputStream object to a String? Why do you need to do this? – Skepi Apr 02 '13 at 10:42
  • Actually the method which I'm doing this logic should return a string value, which will be used in some other methods. I tried fileOutputStream.toString(), which is returning the object. But I need a string value to be returned. – macOsX Apr 02 '13 at 12:36
  • What is the reason for this String value? Why does it need it? It's hard to get exactly what you ask with the limited information above – Skepi Apr 02 '13 at 13:33