Computer arithmetic

[Jean-Pierre A. Radley]

Richard D. Williams propounded (on Thu, Jun 13, 2013 at 03:22:31PM -0500):
| On 6/13/2013 3:18 PM, Jean-Pierre A. Radley wrote:
| > I'm generating a CSV file for Excel comprising about 6500 invoices.
| >
| > Each invoice (row) has charges allocated to one or the other of six
| > category columns (e.g., goods, tax, freight...), and then there's a
| > total column.  (I'm using filePro's '$' edit for the CSV entries.)
| >
| > At the bottom, the columns are totaled, but the sum of the first six
| > columns does not equal the total in the seventh column; the error is
| > slightly different if I let filePro do all the totalling, or if I
| > use Excel SUM formulas.
| >
| > The error is typically a few thousand bucks out of a grand total over
| > six million dollars.
| >
| > Is this an inherent result of representing decimal numbers in a computer's
| > binary language?
| >
| > --
| > JP
| >
| Jean-Pierre,
| 
| Have you tried doing it with out the $ edit?

I did at first, but when the CSV fle was displayed as a spreadsheet, the
results were ugly, in that values like

	100.00	200.50	300.75

showed up as

	100	200.5	300.75

i.e. dollar amounts were not lined up.

| Are you exporting a =sum from filepro in the last row?

No, because the report format has no grand total section -- it's a
"processing only" format.

-- 
JP