Suggestions for "latency-based" throughput issue?

[Boaz Bezborodko]

On 2/20/2014 1:09 PM, Richard Kreiss wrote:
>
>> -----Original Message-----
>> From: filepro-list-bounces+rkreiss=verizon.net at lists.celestial.com
>> [mailto:filepro-list-bounces+rkreiss=verizon.net at lists.celestial.com] On
>> Behalf Of Boaz Bezborodko
>> Sent: Thursday, February 20, 2014 10:37 AM
>> To: Filepro-List at Lists. Celestial. Com
>> Subject: Suggestions for "latency-based" throughput issue?
>>
>> I'm running Windows Filepro off files served on a Linux server using
> Samba.  I
>> find that there seems to be a hard limit on my throughput of about 4MB/s
> per
>> operation.  That is, each instance of Filepro is limited
>> to 4MB/s.   I see this on many operations that simply update files and
>> also on index rebuilds.  If I run, say, three Filepro windows, each
> rebuilding it's
>> own file index, each is still running at 4MB/s, my computer sees a total
>> throughput of about 12MB/s.  I've seen this limitation even when I
> replaced
>> the server with a somewhat faster server and it was the same when I had
>> fewer disks (and lower local throughput) on the older server.  I figure
> that the
>> limitation is not the server or the PCs, but the latency between the two
> when
>> doing the database operations.
>>
>> I know that switching to Filepro on Linux would make everything faster,
> but
>> I'm not ready to go that route yet and in most cases this is not a
>> problem.   In the meantime there are occasional large jobs that I would
>> like to speed up, if possible.  Does anyone have any suggestions on how to
>> program it so that large jobs are automatically broken up and started in
>> separate windows?
>>
>> --
> Where does your configuration point to PFTEMP, is this on a local machine or
> the server?
To a file on the server.  But these are file operations that don't 
create files in PFTEMP.  (At least to my knowledge.)

> What anti-virus software are your using?  Have you configured it not to scan
> fp and filepro directories (folders)?
Yes, they are excluded.
>
> Is your configuration using a mapped drive or UNC?  The mapped drive will be
> faster the UNC.
We're using a mapped drive.

> How fast are your local drives?
Local drives are pretty fast, but I don't think we are writing to them.  
In any case, the network throughput scales with the number of windows 
being run and, I believe, exceeds the speed of the single local drive 
when doing so.
>
> Depending on the size of your indexes, you may need to tune them using some
> of the filepro index configuration variables.
>
> Try
>   set pfbixnodesize=10
> set pfbixbuild=2
>
> and then test your rebuild time.
The limitation exists with both the build operations and regular 
database operations.  I'm more concerned with regular database 
operations since I rarely have to rebuild indexes.

>
> Richard Kreiss
> GCC Consulting
>
> Office: 410-653-2813
>

One possibility I was thinking about was to run some of these operations 
on an index with a unique # (like order# or some such), count the number 
of selected records, and, after a set number has been reached, stop the 
selection and execute the same command while passing the last number 
reached.  The new operation would then start at that number plus 1 and 
again count up from there.

I realize that this will only work in operations where the operations on 
each record is unique and stands on its own, but that is where I'm 
looking to save time.

Any thoughts?

---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com