12

Possible Duplicate:
How to invalidate the file system cache?

In order to be able to so some benchmarks I need to cleanup Windows disk read cache. How can I do this?

In fact I want to compare if loading a big Unicode file (UTF-8 or UTF-16) from disk is faster or not, considering that in memory I do keep UTF-16.

I know that it should be no significant difference but in order to benchmark it I need to be sure that that file is not cached - I need to see if size on disk has more or less impact than decoding the file.

Community
  • 1
  • 1
sorin
  • 161,544
  • 178
  • 535
  • 806
  • 1
    What you are looking for is also called "dropping the cache". Under Linux you can do it by writing to `/proc/sys/vm/drop_caches`. No idea about Windows though ... – sleske Aug 26 '10 at 10:50
  • @Mehrdad you did realize that this question if from 2010 and that one from 2011? – Jack Dec 14 '12 at 00:37
  • @Jack: Yes sorry, indeed I did, but at the same time, I'd posteed the answer to this question on the other question many months ago... so either that question should be closed as a dupe of this one, or the other way around. I decided to opt for the latter because I saw *tons* of copies of this question (many from 2010) when I Googled, and thought it'd be a lot less trouble to mark them all as a dupe of that one than to try to find the earliest copy of this question on StackOverflow (there are tons). :\ Sorry.. – user541686 Dec 14 '12 at 00:58
  • @Mehrdad I think the official correct procedure is for the *"best"* question to be left and the others marked as dupes. To be honest I didn't really read through or compare the two to see which one is *"better"* – Jack Dec 14 '12 at 01:27
  • @Jack: I feel a little weird saying this because I was the one who answered that question, but I really do think my answer is the only one that really works... I looked at all the [other](http://stackoverflow.com/questions/3276126/how-can-i-force-windows-to-clear-all-disk-read-cache-data) questions and [answers](http://stackoverflow.com/questions/478340/clear-file-cache-to-repeat-performance-testing) but none of them were as convenient (and caveat-free) as mine. :\ That's why I marked these as dupes. – user541686 Dec 14 '12 at 01:32

3 Answers3

4

AFAIK, it's unfortunately not possible to discard the read cache under Windows. I spent some time looking into this some years ago, and only found out how to flush the write cache.

As I see it, you have three options, unless somebody else has found some magic:

  1. If possible, do your read file I/O in unbuffered mode.
  2. Each time you want to benchmark, create a new copy of the test data specifying unbuffered mode when creating the new copy (this should keep the copy out of read cache, but I haven't tested).
  3. Allocate enough memory that windows has to discard the disk cache (ugh!).

EDIT: it is indeed possible to flush the read cache, at least on Vista and later: Disable or flush page cache on Windows. It requires a call to the undocumented NtSetSystemInformation NT API, though. Also, for a single file, read cache can be flushed simply by opening the file with FILE_FLAG_NO_BUFFERING specified and closing the handle again.

Community
  • 1
  • 1
snemarch
  • 4,958
  • 26
  • 38
  • 1
    Note that the program `flushmem` from Sron Sbarnea's answer implements solution 3. It works, but allocating that much memory is apparently quite slow (probably because it causes a lot of paging/swapping). – sleske Aug 30 '10 at 10:51
  • 1
    @sleske: yep, it's not a pretty solution - it forces Windows to trim the working sets for all processes, doing pageout to swap... and then there'll usually immediately be pagein activity afterwards. – snemarch Aug 30 '10 at 11:01
  • 1
    `Note that the program flushmem from Sron Sbarnea's answer implements solution 3. It works, but allocating that much memory is apparently quite slow (probably because it causes a lot of paging/swapping).` Not to mention it could crash stuff. I ran it and Chrome (including this very page) just disappeared. Doh! Fortunately when I ran Chrome, it let me restore my session, so it was just a minor inconvenience; nevertheless, be advised to be safe, you should exit everything before using it (which of course makes it a poor solution). `:-/` – Synetech Dec 01 '13 at 00:59
4

Someone from Microsoft helped me out with this a few years ago (back in the days of Windows XP) and gave me a solution that met my needs at the time. I already had two drives in my machine, and what I needed to test was already on the D drive, and fortunately I didn't need to keep any files open on my D drive when I wanted to flush the cache. From the Windows Disk Management interface, I could change my D: drive letter to another letter, and then back again, and the D drive would perform as if had a cold cache in the OS.

Jeff Grills
  • 41
  • 1
3

The only solution I found so far was http://chadaustin.me/2009/04/flushing-disk-cache/ but this ones takes too much time so I hope we'll find a better one.

sorin
  • 161,544
  • 178
  • 535
  • 806