Indexing a file with 3,000,000 records FAST ?

[Kenneth Brody]

Quoting John Esak (Fri, 10 Mar 2006 12:31:30 -0500):

>
> I don't believe anyone is/can give you much more hope with PF variables.
> Could be, setting PFIXBUILD=2 might help... but I would, of course, test
> this. Also, I'm not sure what is meant by the "old 4.1 sorting". Doe it
> mean you can't have multi-segment indexes?  Don't know. Ken, can
> illucidate.

>From the "technical papers" on fPTech's website:

    Tells dxmaint to use 4.1 (multi-pass) style index build. Use this
    on HUGE (i.e.: multi-million record) files to improve performance -
    (will cause poor performance on smaller files).

In other words, the "old" style of building an index was to read a
chunk of the records, sort that chunk, save it to disk, and then
continue on to the next chunk.  When the entire file had been read,
then a second pass was made to merge the pre-sorted chunks into the
final sorted index.  (Even older versions of filePro would then take
a third pass to convert this sorted list into auto-index format.)

With the "new" style indexes, it is possible to build the index in a
single pass.

However, the single-pass algorithm is slightly less efficient than the
multi-pass method.  But, being a single pass, the overhead of the
second pass is completely eliminated.  As the number of records gets
into the millions, the loss of efficiency catches up with the savings
of a single pass, and this method becomes less efficient overall than
the old multi-pass method.

[...]

--
KenBrody at BestWeb dot net        spamtrap: <g8ymh8uf001 at sneakemail.com>
http://www.hvcomputer.com
http://www.fileProPlus.com