OT: using pscp to *move* files

Brian K. White brian at aljex.com
Mon Jul 16 16:20:20 PDT 2007


In that case it would be both simpler and more reliable & robust to to push 
the file onto the windows box.
create a share on the win box and use smbclient to upload the files and you 
then can do all the scripting on your end. You could then go even further 
and download the file after upload, compare, and only delete if the 
downloaded file matched the original. smbclient in it's ftp-like mode not a 
mounted/mapped file share.

Or, use cgi instead of sftp (https if the ssh part of the equation was 
important, but you were using a plain windows share before so...) write a 
cgi that echos the content type for csv and just cats the data out. To the 
client it just looks like a file download. The cgi can even have a filename 
of foo.csv to satisfy really simple clients. Then since it's a script, you 
can do your file management along the way.

Or use ftp/ftps instead of sftp, which allows the use of many clients that 
do have extra features like automatic delete after download, or can be told 
to via command line options.

Or, really, I don't see why, if you can hand them a psftp binary, you can't 
also hand them a batch file you wrote for them just as easily. I've done 
that a few places and it's worked out ok.
If you use a lockfile scheme you can get away with very simple scripting on 
the client side that doesn't need anything but stock command.com/cmd.exe and 
stock ftp client and basically 4 static ftp commands. create lock, get 
files, delete files, delete lock, all in a single .bat file and no other 
dependancies.

The server only creates files outside of the globbing pattern *.csv used in 
the download, and renames them after finished writing, and does not rename 
if lockfile exists.

----------------------
@echo off
REM getcsv.bat / getcsv.cmd
REM non-interactive batch script to ftp download files
REM impliment lockfile, delete files after download
REM Brian K. White - brian at aljex.com - Aljex Software

REM         CONFIG SECTION
REM ###############################
set FTPHOST=myhost
set USERNAME=someuser
set PASSWORD=somepass
set GETFILES=*.csv
REM ###############################

REM lock file in temp
set LOCKFILE=client.lck
set LF=%TEMP%\%LOCKFILE%
echo. |time >%LF%

REM ftp script in temp
set TF=%TEMP%\getcsv.ftp
echo user %USERNAME% %PASSWORD%   >  %TF%
echo binary                       >> %TF%
echo hash                         >> %TF%
echo put %LF% %LOCKFILE%          >> %TF%
echo mget %GETFILES%              >> %TF%
echo mdel %GETFILES%              >> %TF%
echo del %LOCKFILE%               >> %TF%
echo quit                         >> %TF%

REM run tmp ftp script
REM this ftp command might need to be modified slightly on different
REM versions of Windows. Specifically the -n and -i options.
ftp -n -i -s:%TF% %FTPHOST%

REM delete local tmp files
del %TF%
del %LF%
-----------------------


On the server you create files named *.csv.t instead of .csv
Rename a file from .csv.t to .csv only after done writing and only if 
lockfile does not exist.

The ftp script is actully static, excatly the same every time, I only write 
it out fresh each time like this for simplicity. This way the batch file is 
a single self contained file that can be emailed/copied/run etc..

There is no special reason for the gimmick to write the date/time into the 
lockfile either.
Could just be a static echo command.

How to handle it on the server side when it actually encounters the lockfile 
depends on whatever is convenient for your process. Wait & loop to finish 
renaming the file, or just rename it to a 3rd form like .csv.w and forget 
about it. Have a cron job rename *.csv.w to *.csv, also only if lockfile 
doesn't exist.

Looks like a lot when written down, but it does work well, is simple for the 
end user, and in this case requires nothing from the end user but run this 
batch file, no special ftp/sftp client.

Then it's a relatively small step to go from this to the equivalent but 
using an ftps capable ftp client so you regain the security you had with 
psftp.

Really, lately I've gone to supplying a nice stand-alone zsh.exe binary and 
real scripts.
There are freestanding awk.exe and several other utils too which can all be 
put in a nice simple (for the end user) self-extracting rar/zip. But by the 
time you go that far it's probably better to learn some native windows 
script language and use that instead. I haven't done that yet but I don't 
feel too bad because I can't find a single windows script language that's as 
small, light weight, consistently available, consistently behaved across 
versions, windows version independant, and simple to install as a zip file 
with a few standalone binaries. Some require .net, some require 3 or more 
different MS update downloads, some huge, and the latest & greatest windows 
power shell thats supposed to be the windows answer to unix shell, _requires 
that stupid WGA or else you can't install it!_ Yeah that's painless to 
inflict on a mass of ultra end users desktops... idiots...

Brian K. White    brian at aljex.com    http://www.myspace.com/KEYofR
+++++[>+++[>+++++>+++++++<<-]<-]>>+.>.+++++.+++++++.-.[>+<---]>++.
filePro  BBx    Linux  SCO  FreeBSD    #callahans  Satriani  Filk!


----- Original Message ----- 
From: "Jay R. Ashworth" <jra at baylink.com>
To: <filepro-list at lists.celestial.com>
Sent: Monday, July 16, 2007 4:35 PM
Subject: Re: OT: using pscp to *move* files


> On Mon, Jul 16, 2007 at 03:57:39PM -0400, Kenneth Brody wrote:
>> Quoting Jay R. Ashworth (Mon, 16 Jul 2007 15:18:24 -0400):
>> [...]
>> > Non-atomicity: I can't just have teh windows side say "delete
>> > everything" after I've grabbed everything, as I'll end up with a race
>> > condition -- if the Unix side drops a new packet, it will get eaten.
>>
>> Yes, my first thought was ftp's "mget" followed by "mdel", but that
>> wouldn't work if a new file appeared.
>>
>> [...]
>> > So I need either a -d option to pscp (which doesn't appear to exist),
>> > *or* I need to punt and go back to figuring out why Samba stopped
>> > working between these two boxes (at which point I can go back to a
>> > drive letter and ren).
>>
>> Can't you use pscp to get a directory listing, and then get/delete
>> the files one at a time?  That eliminates any race condition with new
>> files appearing after the gets.
>
> There are lots of ways I could do this, if I was permitted to specify
> scripting on the Windows side.  But I can't.
>
> Cheers,
> -- jra
> -- 
> Jay R. Ashworth                   Baylink 
> jra at baylink.com
> Designer                     The Things I Think                       RFC 
> 2100
> Ashworth & Associates     http://baylink.pitas.com                     '87 
> e24
> St Petersburg FL USA      http://photo.imageinc.us             +1 727 647 
> 1274
> _______________________________________________
> Filepro-list mailing list
> Filepro-list at lists.celestial.com
> http://mailman.celestial.com/mailman/listinfo/filepro-list
> 



More information about the Filepro-list mailing list