Strange behavior with IMPORT

[Boaz Bezborodko]

On 8/20/2013 8:38 AM, filepro-list-request at lists.celestial.com wrote:
> Message: 2 Date: Mon, 19 Aug 2013 14:34:29 -0400 From: "Brian K. 
> White" <brian at aljex.com> Subject: Re: Strange behavior with IMPORT To: 
> filepro-list at lists.celestial.com Message-ID: 
> <52126535.4030005 at aljex.com> Content-Type: text/plain; 
> charset=ISO-8859-1; format=flowed On 8/18/2013 4:17 PM, Boaz 
> Bezborodko wrote:
>> I am trying to import a file and it's giving me weird results.  I import
>> using the following command:
>>
>> IMPORT ASCII ord=(File_Location{"INBOUND\ "{ae) R=\r F=, O=" C="
>>
>> The file I am getting is a little strange in that it has CR at the end
>> of each line except the last.  What is happening is that when Filepro
>> reads the last line then 'ord' is populated with data, but 'not ord'
>> shows as true.
>>
>> If I check for 'not ord' to break out of the import loop then the last
>> file is not processed.  If I let it go through then Filepro keeps
>> reading the last line over and over with each loop.
>>
>> I can save the data from the last line and check against that to break
>> out, but is this a bug?
> You can test for not ord and something else like ord(@rn) to tell if you
> are really on a non-empty, *new*, record before proceeding to ignore the
> not-ord.
>
> That way should handle the current files and still work fine when files
> come in with a normal trailing linefeed some day.
>
> then: rn = "0"
> then: import ord = ...
> if: not ord and ord(@rn) eq rn
> then: goto fini
> then: rn = ord(@rn)
>
> I don't really know that @rn will behave the way this expects. It might
> increase every time even though the file position is not really
> increasing any more. You might be able to check something else that you
> think you can depend on in the data itself that is guraranteed to change
> with every record. Only you know what that might be. Really only the
> people sending know.
>
> Myself I would do this:
> In @once, use file i/o commands to read the last byte of the file, and
> if it is not already a LF, then append a LF. All in @once so it
> automatically only happens once without having to check for it every
> record, and it all happens before the first record is even looked at.
>
> No pre-processing outside of filepro needed, everything stays self
> contained and self documented right in the table, and you won't end up
> creating blank records if these start coming in with normal trailing
> line-endings some day.
>
> @once
> ' get file name from cmdline
> then: fn(128,*,g) = File_Location{"INBOUND\ "{ae
> ' open the file in both read and write mode, binary mode
> then: h = open(fn,"rw")
> ' seek to one byte before end of file
> then: x = seek(h,"1","2")
> ' read one byte
> then: x = read(h,d,"1")
> ' if it is a CR, then skip the rest
> if: asc(d) eq "13"
> then: goto endini
> ' write a CR
> x = write(h,chr("13"),"1")
> endini:
> then: close(h)
> then: end ' end of @once
>
>
> In the interests of keeping the example simple, I'm not checking errors
> here. Those x variables you could check each time, they tell you if the
> open failed, how many bytes were actually read in the read, if the write
> failed, etc.
>
> Since we are getting the file name into fn in @once, you should change
> the import line and any other references to the file name to use fn.
> import ascii ord = fn ...
>
> This is all based on you're statement that the line-endings are really
> just CR and not LF or CRLF. If that's not really the case, well, garbage
> in -> garbage out.
>

Thanks for the code, Brian, I'll try it out.  I don't expect any errors 
since the filename is determined by a directory listing of files.

Boaz