Long running output

[Craig Tooker]

Yes - Marcel Proust has nothing on this one...


Craig


On 03/04/16 19:22, Jose Lerebours via Filepro-list wrote:
> Heck, just right a left join query and let it rip!  Then take the
> associative array and dump it by converting it to a string or imploding
> it using a comma as an element separator.  2 or 3 lines of code and
> you're done!!!  Of course, make sure your server is set to handle 20MB+
> per process so that it does not choke.
>
> As per Saturday night 4 hour (faster) time - Simple ... Happy hour
> starts at 5:00 and ends by 8:30 so, gotta get to the bar timely. ;-)
> Tuesdays are a drag and even Servers feel it!
>
>
>
>
> On 03/04/2016 07:00 PM, Richard Kreiss via Filepro-list wrote:
>> One of my client's has an output (creates a csv file) that takes a
>> long time to run.  Last Saturday, on their system, it took just over 4
>> hours.  On Tuesday the job has to be cancelled as it was running over
>> 6 hours and was only 1/4 of the way through.  This output is a task
>> that runs Saturday afternoon (2 PM- no one working) and on very early
>> on Tuesday morning( 2 AM) .
>>
>> The parameters of this output.
>>
>> Process all of the active items in the item master (just over 4300 items)
>>
>> As each item record is selected, do a look to the order detail file
>> and run a set of calculations (dummy fields) for each order detail
>> record selected.  Once all of the order detail records have been
>> processed, export the line detail to the csv file.  Simple enough.
>> The problem is, on average, there are 40,000 order detail line to loop
>> through using a getnext.  this is using an indexed lookup on the item #.
>>
>> There is a posting procedure which would reduce the number or order
>> line to process, but I can't get them to run it.
>>
>> My real question is why would the output complete a about 4 hours on
>> Saturday and run for over 6 hours on Tuesday morning.
>>
>> I have disabled all of the tasks I had created which run on Monday
>> night and early Tuesday morning.  I did the same things for Wednesday
>> & Thursday tasks.  There are a few that still run but they take under
>> 15 minutes to run and complete will before the report runs.
>>
>> Also, this report is started remotely on the office managers
>> computer.  We installed an SSD and the csv file is created on that drive.
>>
>> Any suggestion as to what I should look for to see why the Saturday
>> output completed and the Tuesday morning output ran too slow?
>>
>> I am currently running this output on my system with my client's data
>> (downloaded last night)/  I modified the output process slightly so that:
>>
>> 1. The start time is displayed
>> 2.The end time will be displayed
>> 3. the total run time will be displayed
>> 4. The count of order details is displayed after each 50 records are
>> processed.  This was to that I knew something was being processed;
>> otherwise it looked like  the program had hung up.
>>
>> My workstation is running Windows 10 (64bit) with an intel core
>> I5-4690K 3.56Ghz, 16GB RAM and outputting to a 256GB SSD.  Server
>> Window server 2008 R2 4GB memory, Network 1GB. FilePro 5.8 64bit.
>>
>> Roughly speaking, this output is processing some 175,480,000 records.
>> (4387*40,000).
>>
>> I stated this run at 1:06 PM and it has processed 1425 item records or
>> 57,000,000 records.  Based on a quick calculation, 4.3 item records a
>> minute are being processed. By 6:15PM
>>
>> My system is not running any background program.  No Norton on this
>> system.
>>
>> Short of running the posting remotely, any suggestion, in programming
>> I could use to speed thing up.
>>
>> Within the next few weeks we will be installing a new server with a
>> faster processor and more memory.  It may take just running this
>> output directly on the server to have it finish in a reasonable amount
>> of time.
>>
>> Thanks for any suggestions.
>>
>> Richard Kreiss
>> GCC Consulting
>>
>>
>>
>>
>> _______________________________________________
>> Filepro-list mailing list
>> Filepro-list at lists.celestial.com
>> Subscribe/Unsubscribe/Subscription Changes
>> http://mailman.celestial.com/mailman/listinfo/filepro-list
>
> _______________________________________________
> Filepro-list mailing list
> Filepro-list at lists.celestial.com
> Subscribe/Unsubscribe/Subscription Changes
> http://mailman.celestial.com/mailman/listinfo/filepro-list