14

I am currently in the process of learning the Boost framework, and I have found out how to list all folders and files on my system, using 

#include <boost/filesystem.hpp>
#include <boost/foreach.hpp>
#include <iostream>
using namespace std;
int main()
{
    for ( boost::filesystem::recursive_directory_iterator end, dir("C:\\");
       dir != end; ++dir ) {
       cout << *dir << std::endl;
    }
    return 0;
}

but the only problem I'm having is how slow the process is... am I doing something wrong or is it just that Microsoft's .NET version of listing all files is much faster? Thanks!

Barney
  • 2,355
  • 3
  • 22
  • 37
mahamatali0
  • 151
  • 1
  • 4
  • 2
    I'm not familiar with the boost filesystem API, but shouldn't you assign a value to the 'end' iterator? – Tom Knapen Dec 16 '12 at 00:15
  • 4
    @TomKnapen: the default constructor for the iterator gives the end iterator. – Borgleader Dec 16 '12 at 00:16
  • 4
    @OP: I ran the code on my machine and it seemed to run at a decent speed. What do you consider "slow"? Do you have comparative benchmarks? – Borgleader Dec 16 '12 at 00:19
  • @Borgleader I see, then feel free to ignore my previous comment. – Tom Knapen Dec 16 '12 at 00:19
  • 16
    Try replacing `std::endl` with `\n`. `std::endl` causes a buffer flush on every output operation - this (may) be slowing things down. – Yuushi Dec 16 '12 at 01:22
  • @Borgleader I am comparing it to c# code of wrapped Directory.GetFiles and Directory.GetFolders... it is incredibly faster than boost. Tho .@Yuushi suggestion did help alot. – mahamatali0 Dec 16 '12 at 01:35
  • 3
    You might also want to try this: http://en.cppreference.com/w/cpp/io/ios_base/sync_with_stdio. If you're not going to mix cout & printf that is. See here: http://stackoverflow.com/questions/9371238/why-is-reading-lines-from-stdin-much-slower-in-c-than-python – Borgleader Dec 16 '12 at 01:37
  • 2
    @mahamatali0, use [Process Monitor](http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx) to see what the .Net and boost variants do differently. AFAIK boost::filesystem uses `FindFirstFile`/`FindNextFile`, but may be doing one extra call per file to determine its status (directory, symlink) instead of reusing the information returned by the `FindXXXFile` API. See `basic_recursive_directory_iterator` in `boost/filesystem/convenience.hpp` – vladr Dec 16 '12 at 03:38
  • if you have the latest MSVS try to use tr2::filesystem and compare it with boost. – ixSci Feb 22 '13 at 08:49
  • Are you maybe reading files from a network drive? This might be slow depending on the network quality. – Coert Metz Mar 03 '13 at 11:38
  • 11
    @mahamatali0 Don't mess up a benchmark with console IO(because console IO is slow). If you do want to compare the speed of boost::filesystem with .NET Directory, please remove all the console IO code from both of c++ and c# code. Then you will get a much more accuracy result. – Chen Mar 06 '13 at 01:29
  • @CHEN I would agree, as console/file IO implemenations are known to run slower in c++ vice c#. Quick google search turns up [benching](http://www.neowin.net/forum/blog/460/entry-3883-c-vs-c-performance-file-io/) of file IO in c++ vs c# – Dwight Mar 19 '13 at 20:51

3 Answers3

1

Your question implies a comparison, but you've only provided half the information i.e. where is the code to which you are making the comparison? There are many ways you can increase the performance of the code you have supplied, some of which have been supplied in the comments above.

That said, it's likely that the reason you are observing a performance difference in the first place can probably be traced to the very managed environment on top of which the C# codes runs. It's quite likely that your filesystem is indexed in .Net's memory space while your C++ code and the Boost library are going directly to the filesystem and not benefitting from one of the ways Microsoft has tried to make the .NET environment more efficient. Without the efficiencies, it seems more likely that C# code would be orders of magnatude slower than compiled C++ code of the same quality.

alpartis
  • 1,086
  • 14
  • 29
0

It also depends how many files are present in the folder. If there are many files, then it does take a lot of time. Did you try with a folder containing very few files?

Hrushikesh Kulkarni
  • 301
  • 2
  • 5
0

The .NET version might be indexed and perhaps only needs to read filenames from a flat format. What you propose needs to open every single directory.

frickskit
  • 624
  • 1
  • 8
  • 19