weird import behavior

[Mitch Theophila/MGT Computer Solutions]

hello all, 

since imports and exports are being discussed currently, i 
thought i'd share a strange occurrence.

when importing ascii text files, has anyone ever encountered 
any weird problem losing data from fields having special 
characters?  Filepro 4.8.09K2D4, SuSE 6.3. 

i recently imported a file which included some special 
characters in one field of some records.  The filePro records 
that resulted from these were created with blank character 
fields; the numeric fields were imported properly. 

it gets weirder.  here is the background. 

the import process reads an ascii text file containing three 
fields; records are separated by 0x0A and fields separated by 
the tilde ~.  import file fields 1 and 3 contain character 
data of variable length 25 or less import file field 2 is a 
price which would fit into a 10,.2 format 

the import process creates a new fP record for each record 
imported.  the contents of imported field 1 go into target file 
field 1, which is defined as (25,ALLUP) .  imported field 2 
goes into target field 4 defined (10,.2).  field 3 goes into 
field 20, defined as (30,ALLUP). 

the target file has hundreds of fields, but only these three 
values are needed to create a record.  the import process 
calculates and looks up values to populate the rest of the 
fields. 

field 1 in the target file is the basis for index A.  

out of 13000+ records imported, a dozen or so had 0xF8 or 0xB0 
characters in import field 3.  fields 1 and 2 had no such 
instances. 

here comes the weirdness.  

after the import, in only the new records created from these 
dozen or so, only the numeric value from import field 2 (now 
in file field 4) had been imported.  when viewing these records 
in *clerk via a defined screen, fields 1 and 20 (from import 
fields 1 and 3) appeared to be blank. 

even stranger, index A had been updated properly; when browsing 
the file in *clerk, these records presented in their proper 
sequence.  

strangest of all - when browsing over these records, the values 
that should be in field 1 are visible!  both fields 1 and 20 
are included in the browse format.  and field 20 (from import 
field 3, the one that had the bad data) is blank in the browse, 
just as it is in the screen view.  Field 20 is the basis for a 
demand index which was not used during any of the processing 
mentioned; the demand index was not rebuilt after the import.  
but view the record through any of the defined screens, and 
fields 1 and 20 are both blank, as already noted. 

this is posed as an observation of general interest, not as a 
request for assistance.  once the problem was identified we 
just edited the incoming data to remove the funny characters.  
furthermore, the version of fP and its OS are both pretty
stale; if this is a known and fixed problem the issue is moot. 

Thanks for reading,

.mitch theophila/MGT Computer Solutions