0

Recently I was trying to write pointcloud data to .txt file by using C++ in Visual Studio 2010. At the beginning I used ostream to output the data, but I found it slow when writing the data.

My code :

std::ofstream outfile;
outfile.open(filename.c_str());
for(int index = 0;index < pointcloud.size();index++){
  outfile<<pointcloud[index].x<<pointcloud[index].y<<pointcloud[index].z
  <<pointcloud[index].r<<pointcloud[index].g<<pointcloud[index].b<<'\n';
}
outfile<<std::endl;

The output pointcloud is very huge, almost 0.5G. It takes minutes to write into .txt file. How can I improve the speed of writing down the data? I thought it might be the problem of the size of cache buffer, but not sure. Can someone help me with this?

Paul R
  • 208,748
  • 37
  • 389
  • 560
Durden
  • 1
  • 1
  • You could use std::copy to improve the performance. – N. Gerontidis Mar 08 '17 at 14:00
  • Is the source data (`pointcloud`) all in-memory, or is it a container wrapper to "cloud-based" data? If it's the latter, your issue may be there??? – franji1 Mar 08 '17 at 14:01
  • 3
    Get a faster hard drive, and more RAM. Your performance is limited by your hard drive's speed. Short of changing the laws of physics of this universe, the only answer is faster hardware. – Sam Varshavchik Mar 08 '17 at 14:04
  • Possible duplicate of [How to get IOStream to perform better?](http://stackoverflow.com/questions/5166263/how-to-get-iostream-to-perform-better) – Robert Andrzejuk Mar 08 '17 at 14:09
  • You could write chuncks of a size well suited to the drive/os's buffer size, but the stream is probably already doing that for you. Have you researched whether the write speed is appropriate for the amount of data? Just because it's slow doesn't mean there's a problem with the write. – Kenny Ostrom Mar 08 '17 at 14:12
  • Here are some benchmarks. http://stackoverflow.com/questions/18308793/explanation-information-sought-windows-write-i-o-performance-with-fsync-flus/18434227#18434227 – stark Mar 08 '17 at 16:07
  • I have found a way to solve this problem myself.The performance bottlenecks is not caused by IO.If I use outfile<<(std::string)str.c_str(); （str is a Long string like 200MB），This may take less than one second. – Durden Mar 10 '17 at 01:05

1 Answers1

0
I have found a way to solve this problem myself.The performance bottlenecks is

not caused by IO.If I use outfile<<(std::string)str.c_str(); （str is a Long string like 200MB），This may take less than one second.so I use multithread to splice the data into a long string and output through IO stream.The speed is increased by about 6 times on a 4core computer.

Durden
  • 1
  • 1