Problem with lockfiles, and file ownership(?)

Mon Feb 8 19:30:50 PST 2016

Over many years, I have experienced the problem described.

Adding a delay after a write sometimes was a fix. Not much so if several 
users.

The following usually always fixed it (I have always had backup power on all 
equipment; router, switches, computers, etc.

To work around this problem (windows), follow these steps:
Exit all programs.
Click Start, and then click Control Panel.
Click Performance and Maintenance.
Click System.
On the Hardware tab, click Device Manager.
Right-click your drive device(s), and then click Properties. Do all drives 
as windows itself also caches.
On the Policies tab, click to clear the Enable write caching on the disk 
check box, and then click OK.
On the File menu in Device Manager, click Exit.

NOTE: WINDOWS WRITE CACHING IS ON BY DEFAULT. ON OUR FAST COMPUTERS
            TODAY, YOU CANNOT REALLY EXPERIENCE A DIFFERENCE IF DRIVE CACHE 
IS OFF.
            IF YOU UPDATE TO WIN 10, CHECK THE SETTINGS AGAIN.

I have always disabled write caching on all systems running filePro, the 
server too. I use backups everywhere too.

I never did like the thought/idea of 'data to be written', delayed and NOT 
written, even if seconds later.

Nancy's comments are very valid.

Reggie Freedman
CPI of South Florida
Deltona Florida 32725

-----Original Message----- 
From: Nancy Palmquist via Filepro-list
Sent: Monday, February 08, 2016 2:39 PM
To: filepro-list at lists.celestial.com
Cc: Nancy Palmquist
Subject: Re: Problem with lockfiles, and file ownership(?)

Del,

I have seen random errors on Windows with programming that does I/O and
I have often resolved it by adding a short SLEEP command to give the
"stupid" Windows OS time to write the data and update the lockfile.
This has fixed the issue so often for me, I have this on my list of
"things to do" when I get errors that indicate I/O.

Of course, be sure you WRITE or CLOSE files appropriately but you said
you checked that over very carefully.

Make sure you have CACHING set appropriately.  I was told that it should
not matter if data is cached or not, that it should update correctly,
but I never found that to be true.

If you are using fileA to start a session in fileB, are you properly
exiting fileB when you are finished.  Check your OPEN files list in
Computer Management.  See if all the fileB files are closed after you leave.

I have also seen my Windows Server to get a Record Lock stuck in its
little head and nothing we can do will release it without a system
reboot.  This also has happened more than once, so I am sure it is
possible.

Just my suggestions.  I hope they help.  While adding a SLEEP will slow
down the process a small bit, having it crash is usually much more
annoying.  If the process is running without a human it usually does not
matter much that it is slower.

Nancy

On 2/8/2016 11:25 AM, Walter Vaughan via Filepro-list wrote:
> I know Ken asked you some specific questions, but I also have seen 
> problems similar to this with Windows based applications that read/write 
> to files (never with MS-SQL though, and I don't use filePro except on 
> FreeBSD).
>
> I did come across this utility that maybe you can use without mucking too 
> deep.
>
> https://technet.microsoft.com/en-us/sysinternals/bb897438.aspx
>
> Otherwise your enemy is "Delayed Write Failure", and the fix can be 
> everything from registry settings to changing type of data cable to drive.
>
> -----Original Message-----
> From: Filepro-list 
> [mailto:filepro-list-bounces+wvaughan=steelerubber.com at lists.celestial.com] 
> On Behalf Of Del via Filepro-list
> Sent: Sunday, February 07, 2016 3:49 PM
> To: filePro Mailing List <filepro-list at lists.celestial.com>
> Subject: Problem with lockfiles, and file ownership(?)
>
>      I recently coded a really nifty, but very complex process involving 
> an input (dclerk) process on one file repetitively issuing a system 
> command to execute a dclerk process on another file.   This “called” 
> process (as I will refer to it, even though it is executed via a system 
> command) starts up in add records mode and, in the course of adding a 
> record, also does lots of lookups and updating of records on multiple 
> files.  When the called process is finished, it releases control back to 
> the calling process via an Exit command, at which the calling process (or 
> Mother task?) grabs another record and re-calls the “called” process. 
> Also, this “called” input process, which is normally used directly by the 
> end users, in this case runs without human intervention, using the pushkey 
> command to simulate human input.
>     To make a long story short, I began to run into lockfile and file 
> ownership problems.  That is, the whole process would run a couple of 
> times without any problem, very fast, doing a bunch of input and record 
> updates that would take a human operator way, way more time to input 
> manually – and then, on the third or fourth try, after restoring the very 
> same data files, it would blow up with an error message of “lockfile not 
> found”.  {This is the message I got most frequently, but I also sometimes 
> got a Windows message 32, saying a file was currently in use by another 
> process, or something to that effect, which was obviously untrue since no 
> other processes were running.)  So I took a look at the lockfile, and it 
> was there ok, so why didn’t filepro find it?  After a lot of messing 
> around, I realized that this message really meant that the lockfile was 
> still owned by by an earlier iteration of the same process and for that 
> reason was not available later when the process again issued a lookup or 
> update or whatever it was that prompted the error.
>      Every time I tested, I first restored the data files (data, key, 
> index, and lockfiles) from a backup directory that never changed, and it 
> would usually work just fine the first two or three times, and then it 
> would pop up with some error or other, in different places and completely 
> randomly.  In other words, it was not predictable or deliberately 
> repeatable.
>      So I got desperate and started deleting lockfiles, just to see if 
> that would solve the problem, and I put in commands to execute  “dprodir 
> (filename) –l” in a lot of places where it was blowing up, and that seemed 
> to reduce the frequency of the problem, but it still happened on the third 
> or forth run through the same process with identical data.
>      After seeing it work so many times with identical (and even sometimes 
> different) data, I decided that this was not an error in my code, but had 
> to be some kind of dclerk5.0.14DN9/Windows 7 PRO operating system problem, 
> maybe a timing problem of some sort, where files, like a lockfile, were 
> not being released by Windows to keep up with the very fast running 
> process, thus causing subsequent attempts to reuse the same file to fail. 
> In order to test my theory further, I switched from the multi-user version 
> of filepro to a single user version that I own.  The problem with the 
> lockfiles went away completely, and I ran the whole thing successfully a 
> few times, but then it still failed.  It blew up with a system message 
> saying an index was unavailable because it was in use by another program. 
> This pretty much confirmed my opinion about the problem with Windows not 
> releasing ownership of files when the “called” routine exited, because it 
> ran perfectly five times out of six, after restoring the data files each 
> time.  So the question is, what to do about it?
>      Well, no doubt I could convert the whole process to use dreport 
> instead of repetitive system calls to dclerk – that should get rid of the 
> “file is in use by another program” problem – which by the way should not 
> be happening, since the files should all be released when the “called” 
> process exits.  But that would entail a LOT of extra work at this point, 
> and I like the way this works when it does.  Besides, I am stubborn when 
> it comes to stuff like this.  However, it seems that there is not much 
> that I can do, since the error is not in my code and getting it fixed, 
> whatever it is, is highly unlikely.
>      One thing I will try is to run it on a client machine that is faster 
> and with Windows 10.  I don’t really expect that to work any differently, 
> but it is worth a try.  I am also wondering if I could take the input 
> processes that I am executing using the system command, and convert them 
> to called programs (using the call instead of the system command) or 
> subroutines of the “calling” process (using gosub), so that I won’t get 
> the Windows error 32 problem.  That would also take a lot of work, but 
> maybe less than converting to dreport, and I think it would probably solve 
> the problem.  I would prefer to avoid the extra time that would take, so I 
> would have to be sure it would solve the problem before I commit to doing 
> it.
>      I would like to understand WHY this is happening, because I have used 
> this technique before and never had this kind of problem with it.   If 
> anyone can enlighten me, I would appreciate it.  If anyone has other 
> suggestions about ways to get around the problem, I would be glad to hear 
> them.
>      If you have read all the way through this admittedly long email, 
> thank you for your patience.
>
> Sincerely,
> Del Neroni
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: 
> <http://mailman.celestial.com/pipermail/filepro-list/attachments/20160207/ce8a439d/attachment.html>
> _______________________________________________
> Filepro-list mailing list
> Filepro-list at lists.celestial.com
> Subscribe/Unsubscribe/Subscription Changes 
> http://mailman.celestial.com/mailman/listinfo/filepro-list
>
> _______________________________________________
> Filepro-list mailing list
> Filepro-list at lists.celestial.com
> Subscribe/Unsubscribe/Subscription Changes
> http://mailman.celestial.com/mailman/listinfo/filepro-list

-- 
Nancy Palmquist         MOS & filePro Training Available
Virtual Software Systems    Web Based Training and Consulting
PHONE: (412) 835-9417           Web site:  http://www.vss3.com

_______________________________________________
Filepro-list mailing list
Filepro-list at lists.celestial.com
Subscribe/Unsubscribe/Subscription Changes
http://mailman.celestial.com/mailman/listinfo/filepro-list 