filepro and the SYSTEM command on SuSE Linux

[Brian K. White]

----- Original Message ----- 
From: "Nancy Palmquist" <nlp at vss3.com>
To: <filepro-list at lists.celestial.com>
Sent: Friday, November 09, 2007 2:45 PM
Subject: Re: filepro and the SYSTEM command on SuSE Linux


> Kenneth Brody wrote:
>> Quoting Jay R. Ashworth (Fri, 9 Nov 2007 11:12:04 -0500):
>>
>>> On Fri, Nov 09, 2007 at 09:56:38AM -0500, Nancy Palmquist wrote:
>>>> Here is the framework:
>>>>
>>>> I am logged in as filepro - I run a filePro 5.0 dreport to read the
>>>> names of the files and do a series of processes on each file, the first
>>>> of which will decrypt the file and write it to my work folder.
>>>>
>>>> I make a command string that looks like this:
>>>> gpg --passphrase-fd 0 --output
>>>> /var/appl/cccwork/test_PR1194478701017kptestmetropr05.xml --decrypt
>>>> /home/cccinfo/test_PR1194478701017kptestmetropr05.xml.pgp <
>>>> /myblessedpassph.txt
>>>>
>>>> I then issue the command using SYSTEM
>>>>
>>>> system noredraw pgpcmd  (contents of pfpcmd is shown above)
>>>>
>>>> It will not run.
>
> My definition was that after issuing the command there was no output
> file.  I finally figured out what the issue was and have gotten it
> working.  It was timing.  I did not wait long enough for the output file
> to be written.  Another case of filepro running just too fast.
>
> I added a SLEEP "5000" before I checked the output destination and now
> it is working, just like I stated above.

I have never seen a case yet where sleeping was actually needed or reliable 
or the correct answer even when it seemed to be the fix for the original 
coder.

If you think you need to sleep, then what you really need to do is one or 
all of:
* if you have control over both the writing and reading processes, close the 
file in the writer before trying to open it with the reader.
* if you don't have control over the writing procss (incoming ftp/http/etc), 
impliment a lockfile scheme so that you don't try to read until you know 
that the writer is finished.
One quick-n-dirty trick for file transfers, simpler than lockfiles, is have 
the sender transfer the file under one name, then rename it, and on your end 
only look for the 2nd form of the name.
Since the rename operation is atomic, and since it isn't issued until after 
the put command has finished, as a seperate cmmand, it's safe for you to 
just look for the final name and if you see it, it's ok to read it.
* if you can't do a lockfile scheme, and if the writer has locked the file 
sensibly while writing,
then you can do a try/sleep/try/sleep loop where you try to open the file in 
write mode, and if it fails, sleep a second or a tenth of a second, try 
again, etc... and when the open does not fail, proceed to do whatever you 
were doing with it. close from write mode and reopen in read mode to import 
it, or close and run some external system command on it etc... Incriment a 
counter in the try/sleep loop so that you can give up after a sensible 
amount of time instead of risking waiting forever sometimes.

When you do that, things work as close to 100% as anything ever can, and you 
don't care if it takes 5 seconds or 5 minutes some day, it won't break you. 
And, conversely, when you

As far as the myth of not-yet-flushed disk buffers, that is a myth. Once 
process A closes a file, process B may open it and see every bit of data 
exactly as process A left it, regardless if that data has physically made it 
to the disk yet. When you see otherwise, it's because process A has not in 
fact yet released the file.

If an immediate read gets incomplete data, and a 5 second sleep gets 
complete data, that just means you will be lucky as long as whatever you are 
waiting for just happens to never exceed 5 seconds, and very little in unix 
or in tcp/ip ever makes any promises about how long things will take. Things 
that are generally instantaneous may under various circumstances take any 
amount of time, minutes, hours, etc..
With certain exceptions, things generally wait indefinitely for other 
things, and it's merely normal for many things happen quickly, not expected 
or required or promised at all.

Aside from that, things such as the sync command (both at the os level and 
in filepro) and the printer flush command in fp, might be a necessary step 
in place of closing and releasing the file in unusual cases where for some 
rason you explicitly can't or do not want to close & release the file and 
yet you want some other process to read it right now anyways. That's still 
not sleep.

Brian K. White    brian at aljex.com    http://www.myspace.com/KEYofR
+++++[>+++[>+++++>+++++++<<-]<-]>>+.>.+++++.+++++++.-.[>+<---]>++.
filePro  BBx    Linux  SCO  FreeBSD    #callahans  Satriani  Filk!