limits on called program's variable and/or array space?

Wed Jul 4 03:49:57 PDT 2018

Have you made sure that records are locked prior to updating?

Even if you rebuild index/chain prior to each run, not locking the 
record prior to updating will likely cause the same problem again ...

I have come across so many processes where -p is not used in routines 
where records are being updated - This is even worse if the index key(s) 
is/are being altered.

On 07/03/2018 05:30 PM, Richard Veith via Filepro-list wrote:
> I made a number of the changes that had been suggested, including increasing the sizes of PFTOKSIZE and PFFORMTOKSIZE (though to 500000 rather than 900000), and remade the indexes involved, but after 6 hours or so of running, and definitely getting farther along in the process than it had before, it still aborted before the end.   I will increase those settings to 900000 and see if that makes the difference.
>
> As for Jeroen's suggestion for using Windows Performance Monitor, I have not used it before but guess it is time to dig into it.
>
> Thanks,
> Rich
>
> -----Original Message-----
> From: Filepro-list [mailto:filepro-list-bounces+richard.veith=smrresearch.com at lists.celestial.com] On Behalf Of Bruce Easton via Filepro-list
> Sent: Tuesday, July 03, 2018 11:55 AM
> To: 'filepro-list at lists.celestial.com' <filepro-list at lists.celestial.com>
> Subject: Re: limits on called program's variable and/or array space?
>
> If that's what was printed there, Richard, I suspect it's a typo, and it should have read 999999.  I don't ever remember reading about such a limit in any of the printed manuals, but I may have missed it. I do believe I've had to use values well above 100,000 to get past issues before.
>
> Note that in filepro 5.6 and later the default is 100000 for PFAUTOTOKSIZE, PFTOKSIZE, and PFFORMTOKSIZE. (From online manual at
> fptech.com.)
>
> Bruce
>
>
> On 7/3/18 11:31 AM, Richard Kreiss wrote:
>> 	
>>    
>> For the PFFORMTOKSIZE-nnnnn  - default is 15,000 bytes PFTOKSIZE =
>> nnnnn - default is 20,000 bytes
>>
>> PFAUTOTOKSIZE - nnnnn - default is 15,00 bytes
>>
>> Referring to filePro Developers reference fourth edition
>>
>> It appears that the maximum value would be 99999.  I have used values higher than this, for debugging purposes, but don’t know how effective in adding additional memory space these high values were.  I tended to increase the values by 25,000 until I reached a value that didn't give me an error.
>>
>> It may be useful to have a blank call table. This might release memory being used by the original called table which stays in memory until the next call is made.
>>
>> Call "foobar"; Call "Blank"
>>
>>
>> Richard Kreiss
>> GCC Consulting
>>
>>
>>
>>
>>
>> , I'm not sure if there that these can be
>>> set to.  I have used values of 900,000 frequently.
>>>
>>> On 7/3/18 8:34 AM, Richard Veith via Filepro-list wrote:
>>>> Thanks to those who replied.  Before responding to some of the
>>>> individual
>>> questions, there is a little more I could add to the story.   First, it takes about 6
>>> hrs of processing before the abort, so testing various things takes time.  If we
>>> use smaller sets of data, it does not abort.   And second, there is also a Sort
>>> involved for the calling program, maintained in the program's "out"
>>> file.  It is a 3- field sort with a total of 41 chars per record, for all 22 million records.
>>>> Here are some of the questions from the people who responded:
>>>>
>>>>>     it would help to know if you get an error of sort and/or if
>>>>> anything is logged in event logs
>>>> When the abort happens, Windows puts up a message "dreport.exe has
>>> stopped working".  And the FilePro screen behind it shows the number
>>> of records selected, and the number for "generating output".  The
>>> latter number is never the same but always seems to be within 20,000
>>> or 30,000 of the previous abort points (and sometimes a smaller gap
>>> of just 5 or 6 records).  If there is some "event log" we can look at
>>> (either Windows or FilePro), please point us in the right direction.
>>>>> Have you tried  a much higher PFFORMTOKSIZE and PFTOKSIZE?
>>>> How high can we go?
>>>>
>>>>> Are you using an auto table?
>>>> No.
>>>>
>>>>> Have you considered adding this as a sub routine rather the a called program?
>>>> Yes, but there are one or two other points in our entire procedure
>>>> where the
>>> called program's functionality is needed, and the intent was to use
>>> the same called program in the various places.  Also wanted to keep
>>> the variables separate between the calling program and the called program.
>>>>>     If you are updating records and in these updates an index key is
>>>>> changed,
>>> make sure you are locking your lookups.
>>>> This is a standalone PC doing only one thing at a time, and nobody
>>>> else is doing
>>> any updating.  Both the calling program and the called program do
>>> update records but nothing that would change an index.  And the two
>>> indexes involved are demand indexes.
>>>>> Have you tried running the process using both dreport and rreport?
>>>> Have not tried rreport.  Anything in particular to look for?
>>>>
>>>> I will let you all know how this gets resolved, but as I said, it
>>>> takes 6 hours or
>>> so before hitting the abort, so it is a slow testing process.  And I
>>> will be away for the 4th and the rest of the week.
>>>> Thanks again for any insights.
>>>> Rich
>>>>
>>>> -----Original Message-----
>>>> From: Richard Kreiss [mailto:rkreiss at verizon.net]
>>>> Sent: Monday, July 02, 2018 6:43 PM
>>>> To: Richard Veith <richard.veith at smrresearch.com>
>>>> Subject: Re: limits on called program's variable and/or array space?
>>>>
>>>> Have you considered adding this as a sub routine rather the a called program?
>>>>
>>>> Richard
>>>> Sent from my iPhone
>>>>
>>>>> On Jul 2, 2018, at 5:58 PM, Richard Veith via Filepro-list
>>>>> <filepro-
>>> list at lists.celestial.com> wrote:
>>>>> We have a database of about 20 million records, where each record
>>>>> is just
>>> over 2200 characters.  As part of our processing, we run an output
>>> process table on the entire database, and this process calls another
>>> output process table.  The calling process is about 1500 lines, and the called process is about 300 lines.
>>>>> Up until a recent change, this all worked fine.  The change was to
>>>>> add a
>>> subroutine to the called process, and this subroutine included a few
>>> more variables (declared with sizes, as were the previous variables)
>>> as well as two arrays.  One of the arrays is 700 elements consisting
>>> of 4 digit integers, and the other array is 700 elements consisting
>>> of 20 character strings.  The arrays are not mapped to any existing fields.
>>>>> The puzzling thing is that if the called program, with the new
>>>>> subroutine, is
>>> run by itself as a standalone program against the entire database, it works fine.
>>> But when being called from within the calling program, it aborts
>>> about  85% of the way through the database.
>>>>> Since the calling program/called program arrangement worked fine
>>>>> before
>>> the new subroutine was added, and the called program works fine on
>>> its own, our suspicion is that there is some limit on the memory
>>> allocated for called program variables and arrays, and we exceeded
>>> that.  Or those arrays are not being treated as dummy variables and
>>> instead, as each record is processed, a new version of each array is created and all prior ones are still hanging around.
>>>>> Do you think it is a memory issue?  Is there a way to specify more
>>>>> memory for
>>> the called program (we have PFFORMTOKSIZE set to 200000 on the
>>> assumption it might be relevant since the documentation for CALL says
>>> "the called processing table uses the token area reserved for
>>> printing a form from Inquire, Update, Add")?  Is there a way to
>>> remove the arrays as processing finishes one record and moves to the next record? (We are using "clear" on the arrays but
>>> that probably does not change the space already allocated.)    Are there other
>>> things we should be looking at?
>>>>> Our environment is FilePro 5.7.00D9 on a standalone Windows 7 PC
>>>>> with a
>>> 64-bit OS and 16 GB of RAM.
>>>>> Thanks for any suggestions.
>>>>>
>>>>> Rich
>>>>>
>>>>> -------------- next part -------------- An HTML attachment was
>>>>> scrubbed...
>>>>> URL:
>>>>> <http://mailman.celestial.com/pipermail/filepro-list/attachments/20
>>>>> 18
>>>>> 0
>>>>> 702/d7549a78/attachment.html>
>>>>> _______________________________________________
>>>>> Filepro-list mailing list
>>>>> Filepro-list at lists.celestial.com
>>>>> Subscribe/Unsubscribe/Subscription Changes
>>>>> http://mailman.celestial.com/mailman/listinfo/filepro-list
>>>> _______________________________________________
>>>> Filepro-list mailing list
>>>> Filepro-list at lists.celestial.com
>>>> Subscribe/Unsubscribe/Subscription Changes
>>>> http://mailman.celestial.com/mailman/listinfo/filepro-list
>>>>
>>>
>>> ---
>>> This email has been checked for viruses by Avast antivirus software.
>>> https://www.avast.com/antivirus
>>>
>>> _______________________________________________
>>> Filepro-list mailing list
>>> Filepro-list at lists.celestial.com
>>> Subscribe/Unsubscribe/Subscription Changes
>>> http://mailman.celestial.com/mailman/listinfo/filepro-list
>
>
> _______________________________________________
> Filepro-list mailing list
> Filepro-list at lists.celestial.com
> Subscribe/Unsubscribe/Subscription Changes http://mailman.celestial.com/mailman/listinfo/filepro-list
> _______________________________________________
> Filepro-list mailing list
> Filepro-list at lists.celestial.com
> Subscribe/Unsubscribe/Subscription Changes
> http://mailman.celestial.com/mailman/listinfo/filepro-list