xfer problems sco-sun

[Bill Vermillion]

On Fri, Apr 02, 2004 at 10:57:28AM -0800, Bill Campbell thus spoke:
> On Fri, Apr 02, 2004, Kenneth Brody wrote:
> >Bill Campbell wrote:
> >[...]
> >> >Would that it were that simple: it's the twenty byte header at the
> >> >beginning *of each record* that needs bytes swapped.

> >> Complicated by the fact that the FP key and data records are
> >> fixed length with no \n linefeed delimiter so one has to
> >> read fixed length.

> >I've never heard of a binary file that used text-based record
> >delimiters.

> FP files have few binary parts, the file header record, and the
> 20 byte header on each key file record, the data itself is
> straight ascii (or at least it was the last time I dug into FP
> files in depth).

> Ending the files with a line feed would make it easier to
> manipulate them with standard tools.

What tools can be defined as standard.

I've seen many thing over the years with built in line length
limits.  One problem I recall with at least earlier MS
implementations was line length limitations in the order of 500
bytes.  I don't know if that has been changed.

I remember on old Unix systems when there were few tools going into
a hex editor to add 'nl's ever so often in a large file in order to
get it to read into vi with the the 'line too long' error.

I don't know what the current limit is in many, but I just brought
a key file into vi to look at it and find that it is 3 lines long.
Using $ to go to the end of the the line shows that line to be
22071 lines long.

My personal opinion of the MS line terminated approach to
fixed-length databases, is that they didn't seem to grasp the idea
of fixed length, as so many in the MS world early on seem to come
from a sequential access world, and the speed, or lack thereof,
showed this approach.

However many databases are still stuck in the past and I've seen at
least a couple that take advantage of more modern approaches and
the standard editing tools would never work.

One [whose name I've forgotten and maybe it has passed into
history] was on the Unix side of the tree and all you really had
to do was define field names and the edits for those fields.

No more did you have to decided what the longest length of a field
for a name, or city, or ??? would be, because it would use the
length you entered for each record.

The database used two delimiters in it.  A field delimiter and a
record delimiter.  As I recall these two were in the vicinity
of 0xfb and 0xfc.  And there were other defining bytes also at
the top end of the 8 bit range.

The records were essentially mapped against a hashed table just like
bit-maps for used blocks in a FS so access was very fast.  

So you could not access it as a fixed length as traditional
databases, or sequential with the standard comma separated fields.

As databases grow larger standard access tools just don't work too
well.  Changing fp's layout would surely break a great many
existing ways of doing things, and IMO all it would do is change
one out-moded way of accessing data for another.

Even Oracle broke the traditional method of layout when you had to
add new fields or extend a field length.  As databases grew larger
and larger the having to reorg/rebuild the database took far too
long and you could not afford to have the data inaccessible for long
periods of time.  Oracle would just use the new lengths on new
records, and when/if you modified an old record with would be
rewritten to a place where space was available, and as old space
was freed up from older records that were moved, it was cleaned and
added to be available for use.

On the old 3B2 I remember it was using the largest 5.25" HD drives
available at that time - 70MB CDC drives.  If you needed larger
capacity your only alternative was the 8" hard drives. This was
because the maximum amount of data on a disk platter was limited by
the technology of the day.

But as I look at things changing things for reasons such as 'easier
to use', 'use standard tools' 'everyone else does it like that' is
always the successful way.   My POV is that if you are going to
change don't follow the world, find a better way of doing things,
as being a copycat is not usually the way to financial success.

Bill
-- 
Bill Vermillion - bv @ wjv . com