Invalidating the file cache (2024)

Permalink

Uwe Sieber

2007-03-14 17:24:53 UTC

Post by Ivo Beltchev
What I really need is to invalidate what's already in the cache. I don't want to lose the benefits of cached access.

Writeing small blocks at random position there might be
benefits of cached access. Otherwise it's useless buffering
that can be done more efficient in the application because
cache can only guess what will happen to the data in fututre
while the application usually knows better.

Post by Ivo Beltchev
I was thinking about FILE_FLAG_NO_BUFFERING. http://msdn2.microsoft.com/en-us/library/aa363858.aspx says "This flag does not affect hard disk caching.". Does that refer to the internal HDD cache, or caching in general when accessing HDD?

It does not affect the internal HDD cache.

Post by Ivo Beltchev
I am using file mapping and reading whole sectors, so the restrictions don't affect me. I am reading the file sequentially. Is FILE_FLAG_NO_BUFFERING going to cause any performance problems (aside from not reading from the cache what's already there before I open the file)? For example maybe with buffering the OS will read ahead guessing I will need that data later, and without buffering it won't?

Writing large blocks FILE_FLAG_NO_BUFFERING is faster because
of less overhead. Of course you have to check then if
source and target are on the same physical disk or not and
read/write sequentially if yes and parallel if not.

Uwe

Skywing [MVP]

2007-03-14 18:07:40 UTC

As Uwe said, bulk sequential transfers of fixed size are usually best done
uncached. Disabling caching for your accesses in this scenario will also
help prevent the entire cache from being polluted with your file contents
(unless you were using FILE_FLAG_SEQUENTIAL_SCAN), potentially degrading
performance as more useful data gets removed from the cache.

As long as you are transferring data relatively frequently then whether the
data gets prefetched as part of a read ahead operation or by your explicit
read won't really matter much in the end.

If you stick with keeping caching enabled for your accesses,
FlushFileBuffers and FlushViewOfFile can be used to commit any cached
`dirty' data to disk (however it can still remain in memory in the cache
even after, e.g. for the benefits of improving read performance). IOW,
these will ensure that your data is written out from the cache and file
system, but it can still remain `non-dirty' in the cache after (and of
course, if there is any disk-level caching that might be going on, there are
no guarantees given to you about that in any case).
--
Ken Johnson (Skywing)
Windows SDK MVP
http://www.nynaeve.net
"Ivo Beltchev" <ivo ***@_ adelphia.net> wrote in message news:***@adelphia.com...
What I really need is to invalidate what's already in the cache. I don't
want to lose the benefits of cached access.

I was thinking about FILE_FLAG_NO_BUFFERING.
http://msdn2.microsoft.com/en-us/library/aa363858.aspx says "This flag does
not affect hard disk caching.". Does that refer to the internal HDD cache,
or caching in general when accessing HDD?

I am using file mapping and reading whole sectors, so the restrictions don't
affect me. I am reading the file sequentially. Is FILE_FLAG_NO_BUFFERING
going to cause any performance problems (aside from not reading from the
cache what's already there before I open the file)? For example maybe with
buffering the OS will read ahead guessing I will need that data later, and
without buffering it won't?

Ivo

Post by Skywing [MVP]
Perhaps you should open the file with FILE_FLAG_NO_BUFFERING. Be warned
that this carries various restrictions on how you can access the file.
Also, be warned that if the hardware is doing any caching, then this won't
guarantee that you aren't reading from a HW cache. FILE_FLAG_NO_BUFFERING
only applies to the OS-level filesystem cache.
--
Ken Johnson (Skywing)
Windows SDK MVP
http://www.nynaeve.net
Hi
I'm working on a backup software. After I create the backup (to optical disk
or USB drive for example) I want to verify it. But if I just start reading
from the backup file the data is most likely still in the cache. How do I
invalidate the read cache or otherwise make sure I really read from the
backup media and not from the cache?
Thanks
Ivo