0

I have streams which contains above 1 to 17 milion records. For every record I doing some processing and I want save result which is BigDecimal value.

  • What would be the fastest way to save those data? I don't want keep it all in memory and save on end. I want save every result when record processing is done.
  • What would be the most efficient structure on disk? .txt/.csv file? Futher I want load data for charts. For exemple from result 2000 to 10000. What data structure or read data solution provide fastest work?
Mr Jedi
  • 33,658
  • 8
  • 30
  • 40
  • I would try something simple first, like described here: http://stackoverflow.com/questions/1062113/fastest-way-to-write-huge-data-in-text-file-java - basically BufferedWriter / FileWriter. Modern systems are blazingly fast and 17M BigDecimals is not that much data. – miku Apr 18 '15 at 16:57
  • For saving tons of BigDecimal to disk: consider ObjectOutputStream. Conversion to and from String can be avoided. – laune Apr 18 '15 at 17:05

0 Answers0