Disk caching and data reading (was Re: create())

[Kenneth Brody]

On 9/6/2013 10:24 AM, Reggie Freedman Bellsouth wrote:
> Windows...
>
> I have had the problem of 'delay writes' often during the written/saved
> output to a text file to disk from filePro. Quickly opening the newly
> created file, I have found its full contents is not always all there. I have
> fixed this sometimes by, on Windows machines, drive write caching is enabled
> by default, turning it off. Boxes and drives are pretty fast these days. I
> have not observed a performance hit doing this. Also a battery backup should
> be used. Sometimes by adding one, I've solved random problems.

Then there is a major problem with your system.  (And, regardless of your 
thoughts about Bill Gates and Microsoft, this is not a problem with the 
Windows design, or else everyone would be experiencing data loss all the time.)

It doesn't matter when (or even "if", for the case at hand) the data is 
written from the cache to the physical device.  As soon as the data is 
placed in the cache (which will happen prior to any[*] write call), it is 
available to be read by any other program.

There seems to be a misconception among some people that reads will somehow 
circumvent any write cache, and that modified data won't be seen until the 
cache is flushed to the physical device.  This is simply not true -- if the 
data is already in cache, then any[**] read will fetch the data from that cache.


[*] There are asynchronous writes available, where the function may return 
prior to the write completing.  However, filePro does not use any such 
calls, and any program which does use them is ("should be") designed to 
handle it properly.

[**] Yes, there are programs which do direct-to-disk I/O, but those are 
special utilities, and any "normal" program will use the "normal" I/O calls.

[...]
> Another thing, keep the 'temp' folder to a low item count. A full (over,
> say, 2,000 items) temp folder I have seen a full temp folder really slow
> things down. A full 'temp' folder will cause delayed or none-write problems
> too.

The only problems I am aware of with directories with lots of files is 
slower open/close functions.  Once open, it doesn't matter how many files 
are in that directory.  (In fact, on *nix systems, you can even delete the 
open file itself, and continue using it.)

[...]
> A badly fragmented disk will
> cause an incomplete write problem or a delay.

Yes, a badly fragmented file will usually have a performance hit.  However, 
it has zero effect on the situation at hand.  ("Data written to the file 
won't be seen by other programs immediately.")

> If you're chasing a virus problem, empty the temp folder. Some antivirus
> software delete temp stuff too.

I would call such an AV program "horribly broken".  Unless it detects a 
virus in one of those files, it has no business touching *anything*.

[...]
> To turn write caching off, in explorer, right click your local disk, click
> the hardware tab, click and select the disk you are using, click the
> properties button near the bottom, remove the check on the policies tab
> "Enable write caching on the device".
>
> Some ideas to consider.
>
> If MS just 'fixed what we have' and called it a new OS, I'd buy.

I find it hard to believe that Windows is "broken" in this regard.  If it 
were, it couldn't function.  And, in my more than 25 years experience with 
Windows, I personally have never seen the situation described.

-- 
Kenneth Brody