1

I am currently facing a situation where i have a table with almost 80 millions data and i have to take a dump of that table and store it into a csv file. Currently i am using a not so professional approach( with a perl script+DBI interface , printing the values to stdout and redirecting to a csv file). Now i am planning to use java threading approach. Can you suggest a way forward. Thanks in advance

Community
  • 1
  • 1
rchakraborty
  • 51
  • 6
  • 3
    What's wrong with your current approach? What benefits do you expect from using Java and multiple threads? – Joni Jun 29 '13 at 16:20
  • how exactly do you plan to gain anything from MT writing to a file? are all rows of the exact same length? – radai Jun 29 '13 at 16:21
  • 3
    What you're trying to do is completely I/O-bound. There will be no performance gain by multithreading. I guess Java will not be significantly faster than perl in this case ... if you did your perl script right, it can be a professional solution. – Xie Jun 29 '13 at 16:23
  • you have to do this periodically, isn't it? – surfealokesea Jun 29 '13 at 16:26
  • U will require some R&D over it.. though using Thread Pool may help you where some number of threads will take care of your entire data in batch and write them in file.. – Aman Gupta Jun 29 '13 at 16:28
  • I am not sticking to perl approach because i am trying to utilize my hardware a bit .(Exalogic+Exadata Quarter stack) . And i am actually printing the values in stdout and redirecting to a file which seems very very unprofessional. And in many cases ( disconnected from lan/wifi) i dont even get any error notification and i am left out with an incomplete file. And yeah the rows are of equal length. Thread Pool seems like a solution – rchakraborty Jun 29 '13 at 16:52
  • possible duplicate of [How to implement several threads in Java for downloading a single table data?](http://stackoverflow.com/questions/8325745/how-to-implement-several-threads-in-java-for-downloading-a-single-table-data) – rogerdpack Apr 07 '15 at 21:42
  • All the databases I've used come with a dump utility. You don't have to write code for this. – user207421 Sep 22 '17 at 05:15

0 Answers0