33

I want to measure/optimize the "cold boot" startup performance of an application, and it's difficult to do this without an actual reboot, which is obviously not an ideal solution.

Is there a way I could invalidate entire system's file cache, so that mapped page accesses actually cause a disk access, so that I can measure the time my program takes to start up?

Information:

I pretty much need FSCTL_DISMOUNT_VOLUME's functionality, but for the system volume.

Community
  • 1
  • 1
user541686
  • 205,094
  • 128
  • 528
  • 886
  • How about reading a file that is bigger than the system's memory? – David Heffernan Sep 13 '11 at 17:21
  • 2
    See [this answer][1] [1]: http://stackoverflow.com/questions/478340/clear-file-cache-to-repeat-performance-testing – OKeez Sep 13 '11 at 17:25
  • 1
    Actual reboot will give you the most acurate results. If you somehow manage to invalidate all the cache you will incure page-faults for code that is typically not paged-out, and will not see contention with background system startup activity. – John Sep 13 '11 at 17:26
  • @David: That means I'd need to read 6 GB worth of data, which takes longer than a reboot. – user541686 Sep 13 '11 at 21:17
  • @John: Not if I just start a couple of different programs first, to page-in system code before I measure my app code. – user541686 Sep 13 '11 at 21:17
  • @David: Lol, the maximum memory boot option is more practical than that. :P But that's not so much a solution -- I need at least 1 GB for Windows to run, and copying a 1-GB file is not too different from just a reboot. – user541686 Sep 13 '11 at 21:23
  • My personal solution was to write a number to the whole memory (using a x64 application) so that Windows had to flush its read- and write cache but solutions below are much more sophisticated. – gast128 Nov 29 '18 at 08:51

5 Answers5

35

At least on Windows 7, it seems that attempting to open a volume handle without FILE_SHARE_WRITE sharing permissions causes the file system cache to be invalidated, even if the creation fails.

Thus I made a program that simply calls CreateFile to this end.

Download the program* from its Base64 version here:

<!-- Click "Run Snippet", then Right-Click -> Save As... -->
<a href="data:application/octet-stream;base64,TVqQAAMAAAAEAAAA//8AALgAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAsAAAAA4fug4AtAnNIbgBTM0hVGhpcyBwcm9ncmFtIGNhbm5vdCBiZSBydW4gaW4gRE9TIG1vZGUuDQ0KJAAAAAAAAAABdRXZRRR7ikUUe4pFFHuKTGz/ikQUe4pMbOqKRBR7ilJpY2hFFHuKAAAAAAAAAABQRQAATAECAMEgRlkAAAAAAAAAAOAAIwELAQkAAAgAAAACAAAAAAAAoBIAAAAQAAAAIAAAAABAAAAQAAAAAgAABQAAAAAAAAAFAAAAAAAAAAAwAAAAAgAAlr4AAAMAAIQAABAAAAAQAAAAEAAAEAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAACAAAMABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAC50ZXh0AAAAeQYAAAAQAAAACAAAAAIAAAAAAAAAAAAAAAAAACAAAGAucnNyYwAAAMABAAAAIAAAAAIAAAAKAAAAAAAAAAAAAAAAAABAAABAAAAAAAAAAABVi+yLTQiLQTyLVQyD7BBTVot0CHgz2wPxVzvTdBc5XRB1EoldEDgadAv/RRCLRRA4HAJ19YtGGEiJXfiJRfQ703QNi0X0K0X40egDRfjrA4tFEIt+GDvHcgyL2PfTO99yBDP/6xWL+PfXO8dzAov4i14gjTy7izwPA/mF0nRvg2X8AIl98ClV8ItN/ItdEAPKi1XwihQKOV38dASKCesCMskPvtkPvsory3UUhNJ0O4tN/DtNEHQz/0X8i1UM68mLVfg7VfR0TjPbO8t+BkiJRfTrBECJRfiLRfQ5Rfh3NotVDItNCOlU////i00Ii1YYO8JzF4tWJI0EQg+3BAiLVhyNBIKLBAgDwesM99A7whvAI8frAjPAX15bycNVi+yD7ByLVQhkoTAAAABTM8lWV2aJTeSF0nQVZjkKdBBmg0XkAkEPt/FmgzxyAHXwi3gMg8cMiw+JTfA7z3UKM8BfXlvJw4tVCIXSD4SRAAAAD7dBLGY5ReRzBg+3ReTrAw+3wINl+ABm0egz0maJRf5mO9BzXItxMA+3RfgDwA+3DAaJTeyLTQgPtwQIiUX0M9KNTfSF0nUDjU3sD7cBjVi/ZoP7GXcDg8AgD7fAQmaJAYP6AnXbi0XsZjtF9HI1dzP/Rfhmi0X4ZjtF/nKqi03wD7dBLGY5ReR3HBvA99jrBzsPdQeDyP+FwHULi0EY6Uz///+LTfCLCYlN8DvPD4VB////6TX///9Vi+yD7BhTag6NRehQ/3UIM9vGRehfxkXpX8ZF6nfGRetnxkXsZcZF7XTGRe5txkXvYcZF8GnGRfFuxkXyYcZF83LGRfRnxkX1c4hd9uiG/f//g8QMO8N0Eo1N+FFTVo1N/FdR/9CDxBTrB4kfiV38iR6LRfxbycNVjWwkiIHsoAACAFNWV2ptWGpzZolF5FhqdmaJReZYamNmiUXoWGpyZolF6lhqdGaJRexYai5miUXuWGpkZolF8FhqbFtmiUXyi8Nqa2aJRfRmiUX2WMZFdF/GRXVpxkV2b8ZFd2LGRWhmxkVpd8ZFanDGRWtyxkVsacZFbW7GRW50xkVvZsZFNEfGRTVlxkU2dMZFN0zGRThhxkU5c8ZFOnTGRTtFxkU8csZFPXLGRT5vxkU/csZFQEOIXUHGRUJvxkVDc8ZFRGXGRUVIxkVGYcZFR27GRUhkiF1JxkVKZcZFTEPGRU1yxkVOZcZFT2HGRVB0xkVRZcZFUkbGRVNpiF1UxkVVZcZFVldmiUUQamVYanJmiUUSWGpuZolFFFhqZWaJRRZYZolFGGozi8NmiUUaWGaJRRxqMlhmiUUeai5YZolFIGpkWGaJRSKLw2aJRSRmiUUmM8BmiUUojUUQUOgX/f//iUVwM8CNfTCrjUXkahSJRTBYM/ZWiXVkZolFLGaJRS7o8vz//2oKjU1YUVDGRVhMxkVZZMZFWnLGRVtMxkVcb8ZFXWHGRV5kxkVfRIhdYIhdYeij+///g8QUjU1kUY1NLFFWVv/Q/3VkjXXYjX386K79//+JRfhqCI1FaFD/dWTodfv//4lFbGoMjUU0UP91cOhk+///iUUwaguNRUBQ/3Vw6FP7//+JRdxqC41FTFD/dXDoQvv//4lF4GoEjUV0UP91ZOgx+///g2VwAIvwM8BAg8RAg8ZAOUX4iUV0D46EAQAAM8k5TXAPhXkBAACLVXSLRfyLBJAPtxBqQV9mO/p3SmaD+lp3RGaDeAI6dT0Pt1AEZoXSdAxmg/pcdS5mOUgGdShqXFlqLmaJjdj//f9miY3a//3/WWpcZomN3P/9/1lqBGaJjd7//f9Zi9APtwAz/2Y7x3QWD7fAZomETdj//f9BQkIPtwJmO8d17VdXagNXM8BqAWaJhE3Y//3/agGNhdj//f9Q/1Xgg/j/dAlQ/1Xc6b8AAAD/VTBqIIvQWYlVcDvRD4SsAAAAg/oFD4SjAAAAakVYanJmiUUAWGpvi/hmiX0CZol9BF9qJWaJfQaL+GaJfQiL+WaJfQpfamRmiX0MX2pjZol9Dov5Zol9EF9qZWaJfRKL+2aJfRRfamFmiUUaZol9Fl9qaVhqbmaJRRxYamdmiUUeWGolZolFIFhqc2aJRSRYZolFJmoKWGaJRSgzwGaJRSqLRfxmiU0ii010Zol9GP80iI1FAFJQVv9VbIPEEP9FdItFdDtF+A+MfP7//4tFcF9eW4PFeMnDAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAQAYAAAAGAAAgAAAAAAAAAAABAAAAAAAAQABAAAAMAAAgAAAAAAAAAAABAAAAAAAAQAJBAAASAAAAFggAABlAQAA5AQAAAAAAAA8YXNzZW1ibHkgeG1sbnM9InVybjpzY2hlbWFzLW1pY3Jvc29mdC1jb206YXNtLnYxIiBtYW5pZmVzdFZlcnNpb249IjEuMCI+DQogIDx0cnVzdEluZm8geG1sbnM9InVybjpzY2hlbWFzLW1pY3Jvc29mdC1jb206YXNtLnYzIj4NCiAgICA8c2VjdXJpdHk+DQogICAgICA8cmVxdWVzdGVkUHJpdmlsZWdlcz4NCiAgICAgICAgPHJlcXVlc3RlZEV4ZWN1dGlvbkxldmVsIGxldmVsPSJyZXF1aXJlQWRtaW5pc3RyYXRvciIgdWlBY2Nlc3M9ImZhbHNlIj48L3JlcXVlc3RlZEV4ZWN1dGlvbkxldmVsPg0KICAgICAgPC9yZXF1ZXN0ZWRQcml2aWxlZ2VzPg0KICAgIDwvc2VjdXJpdHk+DQogIDwvdHJ1c3RJbmZvPg0KPC9hc3NlbWJseT5QQURQQURESU5HWFhQQURESU5HUEFERElOR1hYUEFERElOR1BBRERJTkdYWFBBRERJTkdQQURESU5HWFhQQURESU5H" download="FlushFileSystemCache.exe">FlushFileSystemCache.exe</a>

Source

// Usage: ClearCache C: D:
#include <tchar.h>
#include <stdio.h>
#include <windows.h>

int _tmain(int argc, LPTSTR argv[]) {
    LPCTSTR DOS_PREFIX = _T("\\\\.\\");
    for (int i = 1; i < argc; i++) {
        LPTSTR arg = argv[i];
        LPTSTR path = (LPTSTR)calloc(
            _tcslen(arg) + _tcslen(DOS_PREFIX) + 1, sizeof(*arg));
        __try {
            if (_istalpha(arg[0]) && arg[1] == _T(':') &&
               (arg[2] == _T('\0') ||
                arg[2] == _T('\\') && arg[3] == _T('\0')))
            { _tcscat(path, DOS_PREFIX); }
            _tcscat(path, arg);
            HANDLE hFile = CreateFile(path,
                FILE_READ_DATA, FILE_SHARE_READ, NULL, OPEN_EXISTING, 0, NULL);
            if (hFile != INVALID_HANDLE_VALUE) { CloseHandle(hFile); }
            else {
                DWORD le = GetLastError();
                if (le != ERROR_SHARING_VIOLATION && le != ERROR_ACCESS_DENIED)
                {
                    _ftprintf(stderr, _T("Error %d clearing %s\n"), le, argv[i]);
                    return le;
                }
            }
        } __finally { free(path); }
    }
    return 0;
}

* Just for fun, see if you can figure out what the executable does by disassembling it. It's not your typical executable. :)

Matthias Braun
  • 32,039
  • 22
  • 142
  • 171
user541686
  • 205,094
  • 128
  • 528
  • 886
  • It works on one Windows 7 PC but not on another one. Care to share the source? – Chris Bednarski Oct 25 '11 at 08:38
  • @ChrisBednarski: Oh, interesting, I'm not sure why that would happen... the source code just calls `CreateFile(path, FILE_READ_ACCESS | SYNCHRONIZE, FILE_SHARE_READ, NULL, OPEN_EXISTING, 0, NULL)` and then `CloseFile` on the returned handle (if successful); nothing else. (Well, I also had a little fun with the executable in obfuscating it, since it was something I learned to do recently, but this is all there is to the actual logic.) – user541686 Oct 25 '11 at 10:48
  • 3
    Thanks. This is a great little gem when it works. Takes a couple of seconds to execute and saves a reboot. :) I read through a few of your previous posts and thought that it might link to ntdll.dll only or something. Dependency walker says - No DOS or PE signature found. Very clever. – Chris Bednarski Oct 25 '11 at 20:37
  • @ChrisBednarski: Haha nice, I'm glad it works. :) When it doesn't work, does it at least give an error, or does it simply quit without doing anything? (And yeah, you're onto something regarding ntdll, but it's not quite that simple, since the imports would be visible. :P) – user541686 Oct 25 '11 at 20:41
  • When it works, it takes a few seconds to execute. When it doesn't work, there are no errors and shuts down just about immediately. – Chris Bednarski Oct 25 '11 at 23:09
  • @ChrisBednarski: Ah, I see -- I'll try and make it return some kind of error code instead when I get the chance. :) Thanks for letting me know. – user541686 Oct 26 '11 at 01:57
  • 1
    @ChrisBednarski: I've updated the program; this one should print an error message and also return the Win32 error code. – user541686 Oct 26 '11 at 05:02
  • Opening a volume handle requires an Administrator right. If UAC is on, you need to run as administrator. Even better news, the trick works on XP (at least SP3) as well. Could you embed a minifest that requires Admin into the exe? – Chris Bednarski Oct 26 '11 at 22:27
  • 1
    Please don't post base64 encoded data of that size in the future. It wreaks havoc on browsers if someone from our community needs to edit your answer in the future. – Tim Post Nov 13 '11 at 18:35
  • @Mehrdad Think "crappy fanless atom netbook with a bunch of tabs open" This wasn't that bad, I agree. More cautionary than anything else :) However, a nice way of not having to worry about a link. – Tim Post Nov 13 '11 at 19:21
  • 3
    @ChrisBednarski (and @ everyone else who sees this): I posted a working version of the source code (although it doesn't include the fun regarding the disassembly :P). – user541686 Nov 19 '11 at 01:51
  • @Mehrdad what did you use to pack that executable? Is it something custom? – Jonathon Reinhart Mar 10 '13 at 01:17
  • @JonathonReinhart: It's not packed at all, actually! I *have* made it do some tricks, yes :) but it's not by packing anything. – user541686 Mar 10 '13 at 01:23
  • 1
    Right, I didn't really mean "packed" - I meant post-processed in the way that packers perform IAT lookups manually etc. Some cool stuff in there. – Jonathon Reinhart Mar 10 '13 at 02:02
  • @JonathonReinhart: Haha, you're in the right ballpark, but that's not quite what's happening. It's actually the direct output from Visual C++, I've done no post-processing on that executable. But the way I created it was definitely tricky, I played around with the object files and used assembler. :) – user541686 Mar 10 '13 at 02:07
  • Great solution! It works for me but is returning a non-zero exit code nevertheless, which is hampering my ability to accurately determine (within my program) whether it succeeded or not. I'm exec'ing this from a Java program (running as admin), and it's returning an exit code of 32, with no output on stdout or stderr. Any ideas why? – JimN May 25 '13 at 02:04
  • @JimN: Error 32 is `ERROR_SHARING_VIOLATION`. I think it's fine -- like I said in the answer, even if the creation fails, it still seems to invalidate the cache. – user541686 May 25 '13 at 02:10
  • @Mehrdad yes I understand this, but if it fails entirely (for example, due to some other issue such as exec or admin permissions), then my automated program would not be able to distinguish this true failure from the false positives that I'm currently getting. Anyway, I've implemented a workaround which assumes success for non-zero exit codes if the exec running time exceeds 500ms. Not ideal, but probably good enough. Thanks again for the help. – JimN May 25 '13 at 02:58
  • @JimN: Yeah sorry, I don't think there's any way to figure out if it "really" succeeded aside from timing. I'm not sure if it's a good idea to make this automated :P but glad it helped. – user541686 May 25 '13 at 03:13
  • 8
    Being able to clear the filesystem cache turns out to be very useful in automated performance tests (benchmarks) which involve the filesystem. ;) – JimN May 25 '13 at 03:18
  • 2
    This still works great on Windows 8.1, BTW. Thanks for this! It saved me a bunch of reboots during performance testing. :D – dbeachy1 Apr 26 '14 at 17:31
  • @dbeachy1: You're welcome! You should also thank Microsoft for "supporting" this undocumented behavior up to 8.1! :) – user541686 Aug 07 '14 at 07:57
  • 1
    Awesome (the solution, the execution, the sensibility to include it as base64), this is one of the rare times one upvote really isn't enough... For users in a hurry (and/or not knowing what to do with the base64 string): you could simply create this hyperlink `FlushFileSystemCache.exe` and replace `*` with the base64 string given. Now you can (right)click the link and download the file as binary and all you'd need is a browser :) Perhaps it might be a suggestion to add such a link to the post? – GitaarLAB Oct 27 '17 at 08:17
  • @GitaarLAB: Thank you! :) Unfortunately that doesn't seem to work -- links are created with the `[text](link)` Markdown syntax for HTTP(S), not the HTML syntax or with anything else apparently, so that ends up rendering as plain text. – user541686 Oct 27 '17 at 08:23
  • not saying you should (my comment above still might help someone): I just tested it by putting it in a 'snippet'. Usage: `run code snippet` > rightclick link > save as ... – GitaarLAB Oct 27 '17 at 08:37
  • Unfortunately, this code doesn't seem to affect the time spent enumerating folders at all. – user626528 May 10 '20 at 17:49
5

I've written a simple command-line utility to do that: FlushFileCache

It relies on the undocumented NtSetSystemInformation functions, and can flush the various other memory pools as well.

Eric Grange
  • 5,931
  • 1
  • 39
  • 61
3

This solution worked great: Clear file cache to repeat performance testing

More specifically, I'm doing this:

// Open with FILE_FLAG_NO_BUFFERING
auto hFile = CreateFile(path.c_str(),
                        GENERIC_READ,
                        FILE_SHARE_READ,
                        nullptr,
                        OPEN_EXISTING,
                        FILE_FLAG_NO_BUFFERING,
                        nullptr);

/// Check
if (hFile == INVALID_HANDLE_VALUE){
    //_tprintf(TEXT("Terminal failure: unable to open file \"%s\" for read.\n"), argv[1]);
    cout << "error" << endl;
    return;
}

// Close
CloseHandle(hFile);

// Now open file with regular C++ API, and caching disabled
ifstream file(path, ios::binary | ios::ate);
JamesThomasMoon
  • 6,169
  • 7
  • 37
  • 63
Markus
  • 2,174
  • 2
  • 22
  • 37
2

What David said. Create a large file, however many GB you need, and each time you want to reset your file cache, make a copy of the file. Then make sure you delete the old file.

So, create BIGFILE1.DAT, copy it to BIGFILE2.DAT, and then delete BIGFILE1.DAT (which removes it from the disk and the cache). Next time, just reverse the process.

Addenda:

Well, the other option is to take the files that are mapped, and copy them to new files, delete the old ones, and rename the new files back to the old ones. The cache is backed by a file. If the file "goes away" so does the cache.

If you can identify these files, and they're not shared by the system/other running programs, this should be simple to script and, ideally, run faster than copy 6 GB of files around.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Will Hartung
  • 115,893
  • 19
  • 128
  • 203
  • 2
    I have 6 GB of RAM. Copying 6 GB worth of data has two problems: (1) Windows 7 doesn't use all of it to evict data from the cache, and (2) even if it did, it would take longer than an actual reboot. – user541686 Sep 13 '11 at 21:16
  • 2
    SetSystemFileCacheSize() could be quicker. – Hans Passant Sep 13 '11 at 21:33
  • @HansPassant: I never saw that comment... it seems like it's a new function in Vista and XP x64, and seems like it would be very useful. Thanks! – user541686 Oct 09 '11 at 07:02
  • I’ve used a similar, but **simpler** method for years. Whenever I need to be sure I’m reading files from the disk instead of the cache, I just read a large file first (e.g., search a large file for a string that doesn’t exist in it). Of course this is still a kludge as opposed to simply running a command meant specifically for this purpose, but at least it works. – Synetech Dec 01 '13 at 01:21
  • 3
    The problem with reading large files is it assumes the cache's behavior is basically FIFO, and does not consider frequency counts and other metrics in deciding what to evict. This can even differ between versions of an OS. You might think you have cleared the cache, but you have only evicted 50% of what you believe. – Cameron Oct 29 '15 at 17:37
  • Perhaps filling the whole memory without file I/O is also sufficient. I did some tests in the past and I think my results where close to results after a reboot. – gast128 Nov 29 '18 at 08:55
0

You can use a VM and take a snapshot right after the VM boots. Resuming from a snapshot will be faster than a reboot.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
John
  • 5,561
  • 1
  • 23
  • 39
  • 3
    Using VM's for performance test is not the best advice. There will be some unpredictable events on the host system which can and will invalidate your measurements. – Tuto Aug 25 '14 at 13:27