File Pro And Bus Errors

[Brian K. White]

----- Original Message ----- 
From: "Nancy Palmquist" <nlp at vss3.com>
To: "J. Ryan Kelley" <ryan.kelley at trinitytransport.com>
Cc: "fplist3" <Filepro-list at lists.celestial.com>
Sent: Wednesday, July 06, 2005 9:43 AM
Subject: Re: File Pro And Bus Errors


> J. Ryan Kelley wrote:
>>
>>
>> Steve Wiltsie wrote:
>>
>>> ----- Original Message ----- From: "J. Ryan Kelley" 
>>> <ryan.kelley at trinitytransport.com>
>>> To: "fplist3" <Filepro-list at lists.celestial.com>
>>> Sent: Wednesday, July 06, 2005 7:07 AM
>>> Subject: File Pro And Bus Errors
>>>
>>>
>>>
>>>> Good Morning,
>>>>
>>>> I've been working with Filepro for about 8 months now after recently
>>>> graduating from college with a degree in Computer Science.  I've had
>>>> experiences working with everything from C, and C++ to Fortran, to 
>>>> Lisp,
>>>> and countless other languages, however never with FilePro until I
>>>> started at my current position.  When I first started working with
>>>> Filepro, I'll admit, I was impressed with how easy it was to step in 
>>>> and
>>>> start working.  In fact, I was doing some programming in my first few
>>>> hours on my first day.  The primary thing that I have found unsettling
>>>> from day one, however, is the presence of the dreaded Bus Error.  I'm
>>>> quite familiar with Bus Errors (aka Segmentation Faults) from my days
>>>> working in C, where attempting to access a memory location you don't
>>>> have permissions to access would cause this problem and dump you out.
>>>> It was usually caused by an improperly indexed array, or an erroneous
>>>> pointer.  This all made sense in C because of how low level it was,
>>>> giving you direct access to the memory.
>>>>
>>>> Filepro, at least relative to C, is much more high level.  There is no
>>>> point at which you are attempting to manipulate the system memory
>>>> directly so these Bus Errors should cease to exist, unless being caused
>>>> by a compiler/interpreter.  When I've questioned co-workers and other
>>>> filepro developers in the past about what causes these errors, and how
>>>> they can be avoided, I always get one reply, "Remove your comments."
>>>> When I was first told this, I disputed it, and disputed it.  After all,
>>>> in all (or at least i thought all) compilers/interpreters comments are
>>>> completely disregarded and treated as if they were simply white space.
>>>> Sadly, I have seen several instances in which removing some comments 
>>>> can
>>>> remedy these errors.  In my opinion, comments are a very important part
>>>> of the code itself, especially when working in a system with several
>>>> developers who need to be able to easily digest what another programmer
>>>> was attempting to do.  For a comment to attempt to access an
>>>> unauthorized memory location is illogical, and I'm assuming that there
>>>> is a better explanation for all of this.  As the days wear on, the bus
>>>> errors seem to become more and more prevalent and are creating some
>>>> serious issues that take me away from my daily duties.  Can anyone
>>>> provide some helpful insight onto this problem, do comments really 
>>>> cause
>>>> Bus Errors?  If So why and how?  Is there a patch in the works? and 
>>>> what
>>>> else in FilePro can cause these nasty errors.
>>>>
>>>> Thank you,
>>>>
>>>>
>>>
>>>
>>> Ryan,
>>>
>>> Sorry you are getting "bus errors" in filePro.  However, you have failed 
>>> to
>>> provide anyone with enough information to help you.  For example:  what
>>> version of filePro are you using?  is there an update available to a 
>>> later
>>> release of the same version you are using?  what operating system(s) are
>>> involved?  do the systems get bus errors running anything else?
>>>
>>> I have been working in filePro doing development for my business 
>>> customers
>>> for over 20 years and I don't think this has been a problem I've seen 
>>> for a
>>> long, long time.  I think you have another problem that is showing 
>>> itself
>>> when filePro is running or you are running a very old version on a new
>>> operating system.
>>>
>>> That's my 2 cents worth.
>>>
>>> Steve Wiltsie
>>> microCONCEPTS Div. multi/user/inc.        Since 1981
>>> (812) 474-0127                          Fax: (812) 474-0147
>>>
>>>
>>>
>>>
>>>
>> Steve,
>>
>> Thanks for the quick reply.  I'm running on FP 5.0.13R4 on Solaris 9.  We 
>> see no bus errors outside of filepro, although this server runs only a 
>> few other applications such as a fax server.  We see these bus errors at 
>> different times, sometimes, before the processing even appears to begin, 
>> and other times after certain fields are updated, the fields will be 
>> updated, but some automatic indexes may need to be rebuilt to reflect the 
>> change because of a bus error.
>>
>> Thanks,
>
> This error may be related to a corrupt file.  It could be the edits, 
> screenss, automatic, input or output processing.  Since you say it seems 
> to happen sometimes before you hit processing.  Do the following:
>
> Remove or rename the processing both auto and input (make a safe copy of 
> course),  Remove all indexes (each and every one - document the indexes 
> before).  Then try the file and see if it functions correctly for access , 
> update and add (without the programming it might hot be fully functional 
> but you will be able to tell if it is giving you bus errors.
>
> Then add back the indexes, create them from scratch.  and try the file 
> again.  If it still works - add back the auto processing .  See if it 
> still works.  (WRITE commands in the auto table could cause the problem 
> you indicate.)
>
> Finally, put back the input/output processing and try it again.  If it 
> crashes, now it will indicate the processing is causing it.
>
> Run the processing in debug mode and follow it till it crashes.  That will 
> tell you where to look in the code.
>
> I have had crashes in the past from stuff like ab(5,.0) which was typed by 
> mistake ab(5,.o).  Or conflicts with labels, array names, lookup names, 
> selection table names, long variables.
>
> FilePro can check for syntax specific things but some errors can not be 
> anticipated.  Those tend to cause the Bus Error type crashes.
>
> I have seen bad edit tables (local or global) or exceeding the 200 limit 
> can cause a crash.  (local + global edits can not exceed 200)
>
> Good luck,
>
> These are a real pain.  But there is a cause and if you persist you will 
> track it down.


Pretty good hints Nancy, especially when you know some facts that neither J, 
Ryan nor his senior co-developer Tim Fischer may be aware of because their 
history doesn't go back much further than when that Solaris box was put in. 
I however can say that the app they are running never got bus errors on the 
ReadHat 7.x box it started out and ran for about a year on, nor on the SCO 
5.0.6 box it was moved to and ran for another year or two. The bus errors 
arrived with the migration to Solaris.

How/why this ties in with your commens so well is, The original linux 
install really was a platform change from our SCO development box, and then 
a year or so later a platform change back to SCO, and neither of those 
involved the xfer program or an endianness change. switching between 
linux/sco/freebsd requires nothing but simple file transfer of any type you 
like. tar/ftp, rsync, etc...

Switching to Solaris was a big project that was pretty difficult. I didn't 
do it but I helped by answering questions and providing possible 
work-arounds for Tim when the official way didn't work. Things like using dd 
to byte-swap an output format or screen or other file that xfer simply 
ignored.

I think the recommendation to take a known misbehaving file and copy it and 
start re-creating it's parts from scratch is a solid one.

Unfortunately, I aslo think that filePro just plain isn't perfect and there 
may be a weird filepro/solaris interaction that won't get tracked down 
because Solaris is not a common & popular platform for filepro.

I have my own entirely demonstrable problem with filepro, a weird 
interaction between filepro system command and ksh on sco.
ksh works fine
filepro works fine
sco works fine
but put a ksh script (or and un-specified script while your login shell was 
ksh and nothing chaged it along the way to the system command) inside a 
filepro system command on a sco box and it'll fail some percent of the time. 
The percent is random too in that, some things will fail almost all the 
time, and sometimes the same script, in some other process table, will 
almost never fail.
complexity/largeness appears to have little to do with likelyhood of 
failure.
Time and again I've received the bug report, saw that as system command was 
failing, and did nothing but switch the shell to plain sh or bash and it 
starts working 100%.

This wasn't always the case. I didn't decide to write all my scripts in ksh 
and then proceed to do so for a few years in a vacuum.

In my case I even went so far as to put together a small process table and 
fp file and shell script that proved and demonstrated the failure 100% 
repeatably.
Remember a year or so ago when I posted that tar file and talked about 
nesting levels of fp system commands? That test didn't produce random 
failures, it produced a seemingly random pattern of nesting levels that 
failed, but it was the same exact pattern every time you run it. bizarre.

But, my usage is apparently not common. Yes a lot of people use shell 
scripts in system commands, but from what I've seen not too many know they 
are supposed to specify the shell at the top, and those that do, not too 
many specify anything other than /bin/sh or write scripts that need anything 
better than plain sh. On sco, by default, users & roots login shells are 
/bin/sh, and all else being equal, any shell scripts you run, if they don't 
specify a shell at the top, will run using whatever shell you are in at that 
time, which will be /bin/sh. Not too many people either change their login 
shell to ksh or specify ksh at the top of shell scripts, and so for most 
people, their shell scripts work because they end up being sh.

And so i don't expect the weird interaction will ever get tracked down and I 
simply have to give up on using ksh as my universal scripting language.
Which sucks because sh is too simple for my needs, and ksh is the most 
universal shell and it's older and more consistent than bash, the next most 
common shell with similar feature set.
But I can live with bash since by now it's available on all platforms I'm 
ever likely to use and if I run into a box with such an old version of bash 
that it behaves differently, I will probably always be free to upgrade it.

Unfortunately, Trinity can't just grumble and use another OS the way I can 
use another shell.
Someone should have advised them about the dangers of moving to a 
not-well-supported platform.
(Actually, someone _did_, but fP themselves should have.)

Brian K. White  --  brian at aljex.com  --  http://www.aljex.com/bkw/
+++++[>+++[>+++++>+++++++<<-]<-]>>+.>.+++++.+++++++.-.[>+<---]>++.
filePro  BBx    Linux  SCO  FreeBSD    #callahans  Satriani  Filk!