Computer arithmetic
Richard D. Williams
richard at appgrp.net
Thu Jun 13 13:22:31 PDT 2013
On 6/13/2013 3:18 PM, Jean-Pierre A. Radley wrote:
> I'm generating a CSV file for Excel comprising about 6500 invoices.
>
> Each invoice (row) has charges allocated to one or the other of six
> category columns (e.g., goods, tax, freight...), and then there's a
> total column. (I'm using filePro's '$' edit for the CSV entries.)
>
> At the bottom, the columns are totaled, but the sum of the first six
> columns does not equal the total in the seventh column; the error is
> slightly different if I let filePro do all the totalling, or if I
> use Excel SUM formulas.
>
> The error is typically a few thousand bucks out of a grand total over
> six million dollars.
>
> Is this an inherent result of representing decimal numbers in a computer's
> binary language?
>
> --
> JP
> _______________________________________________
> Filepro-list mailing list
> Filepro-list at lists.celestial.com
> Subscribe/Unsubscribe/Subscription Changes
> http://mailman.celestial.com/mailman/listinfo/filepro-list
>
>
>
Jean-Pierre,
Have you tried doing it with out the $ edit?
Are you exporting a =sum from filepro in the last row?
Richard
More information about the Filepro-list
mailing list