FW: Indexing a file with 3,000,000 records FAST ?

[Kenneth Brody]

Quoting John Esak (Fri, 10 Mar 2006 16:26:32 -0500):

>
> > In addition to trying PFBIXBUILD=2 you may want to try
> > to build multiple indexes at the same time.
> >
> > Jeff Harrison
> > jeffaharrison at yahoo.com
>
> Have you actually tried this? Maybe you are just throwing this out as an
> idea? I have tested this suggestion thoroughly on all sorts of machines.
> Never once has building multiple indexes simultaneously beaten building
> the rebuild of the same indexes one-at-a-time sequentially. Unlesa, you
> have multiple CPU's in which case, there is some benefit... but not
> always. If there is a very quiescent condition on the system and the
> first index build grabs the first CPU and holds it pretty successfully
> for long periods of time, the next build of an index startst on the 2nd
> CPU... then, sometimes, I've seen some improvement. Otherwise, building
> sequentially has always worked out as fastest "overall".
>
> On a single CPU, with even a quiescent situation, I do not think
> building more than one index at at ime will save any time at all. Only
> think this because of long experimentation... but YMMV.

I have no empirical data to support this, but...

Remember that dxmaint will not use 100% of the CPU most of the time.
A lot of the time, it will be I/O bound waiting for disk I/O.  It is
possible that other dxmaints can use that time for their processing.
And, depending on the system's disk cache, it may be possible for the
additional dxmaints to pull the data from the cache once the first
dxmaint has requested it, thereby reducing the I/O wait time for these
additional copies.  (Remember, all copies of dxmaint will be reading
from the same key+data files, and they will all be reading them
sequentially.  This will cause a lot of overlap in the reads for all
of the copies of dxmaint.)

--
KenBrody at BestWeb dot net        spamtrap: <g8ymh8uf001 at sneakemail.com>
http://www.hvcomputer.com
http://www.fileProPlus.com