OT: using pscp to *move* files
Brian K. White
brian at aljex.com
Mon Jul 16 16:20:20 PDT 2007
In that case it would be both simpler and more reliable & robust to to push
the file onto the windows box.
create a share on the win box and use smbclient to upload the files and you
then can do all the scripting on your end. You could then go even further
and download the file after upload, compare, and only delete if the
downloaded file matched the original. smbclient in it's ftp-like mode not a
mounted/mapped file share.
Or, use cgi instead of sftp (https if the ssh part of the equation was
important, but you were using a plain windows share before so...) write a
cgi that echos the content type for csv and just cats the data out. To the
client it just looks like a file download. The cgi can even have a filename
of foo.csv to satisfy really simple clients. Then since it's a script, you
can do your file management along the way.
Or use ftp/ftps instead of sftp, which allows the use of many clients that
do have extra features like automatic delete after download, or can be told
to via command line options.
Or, really, I don't see why, if you can hand them a psftp binary, you can't
also hand them a batch file you wrote for them just as easily. I've done
that a few places and it's worked out ok.
If you use a lockfile scheme you can get away with very simple scripting on
the client side that doesn't need anything but stock command.com/cmd.exe and
stock ftp client and basically 4 static ftp commands. create lock, get
files, delete files, delete lock, all in a single .bat file and no other
dependancies.
The server only creates files outside of the globbing pattern *.csv used in
the download, and renames them after finished writing, and does not rename
if lockfile exists.
----------------------
@echo off
REM getcsv.bat / getcsv.cmd
REM non-interactive batch script to ftp download files
REM impliment lockfile, delete files after download
REM Brian K. White - brian at aljex.com - Aljex Software
REM CONFIG SECTION
REM ###############################
set FTPHOST=myhost
set USERNAME=someuser
set PASSWORD=somepass
set GETFILES=*.csv
REM ###############################
REM lock file in temp
set LOCKFILE=client.lck
set LF=%TEMP%\%LOCKFILE%
echo. |time >%LF%
REM ftp script in temp
set TF=%TEMP%\getcsv.ftp
echo user %USERNAME% %PASSWORD% > %TF%
echo binary >> %TF%
echo hash >> %TF%
echo put %LF% %LOCKFILE% >> %TF%
echo mget %GETFILES% >> %TF%
echo mdel %GETFILES% >> %TF%
echo del %LOCKFILE% >> %TF%
echo quit >> %TF%
REM run tmp ftp script
REM this ftp command might need to be modified slightly on different
REM versions of Windows. Specifically the -n and -i options.
ftp -n -i -s:%TF% %FTPHOST%
REM delete local tmp files
del %TF%
del %LF%
-----------------------
On the server you create files named *.csv.t instead of .csv
Rename a file from .csv.t to .csv only after done writing and only if
lockfile does not exist.
The ftp script is actully static, excatly the same every time, I only write
it out fresh each time like this for simplicity. This way the batch file is
a single self contained file that can be emailed/copied/run etc..
There is no special reason for the gimmick to write the date/time into the
lockfile either.
Could just be a static echo command.
How to handle it on the server side when it actually encounters the lockfile
depends on whatever is convenient for your process. Wait & loop to finish
renaming the file, or just rename it to a 3rd form like .csv.w and forget
about it. Have a cron job rename *.csv.w to *.csv, also only if lockfile
doesn't exist.
Looks like a lot when written down, but it does work well, is simple for the
end user, and in this case requires nothing from the end user but run this
batch file, no special ftp/sftp client.
Then it's a relatively small step to go from this to the equivalent but
using an ftps capable ftp client so you regain the security you had with
psftp.
Really, lately I've gone to supplying a nice stand-alone zsh.exe binary and
real scripts.
There are freestanding awk.exe and several other utils too which can all be
put in a nice simple (for the end user) self-extracting rar/zip. But by the
time you go that far it's probably better to learn some native windows
script language and use that instead. I haven't done that yet but I don't
feel too bad because I can't find a single windows script language that's as
small, light weight, consistently available, consistently behaved across
versions, windows version independant, and simple to install as a zip file
with a few standalone binaries. Some require .net, some require 3 or more
different MS update downloads, some huge, and the latest & greatest windows
power shell thats supposed to be the windows answer to unix shell, _requires
that stupid WGA or else you can't install it!_ Yeah that's painless to
inflict on a mass of ultra end users desktops... idiots...
Brian K. White brian at aljex.com http://www.myspace.com/KEYofR
+++++[>+++[>+++++>+++++++<<-]<-]>>+.>.+++++.+++++++.-.[>+<---]>++.
filePro BBx Linux SCO FreeBSD #callahans Satriani Filk!
----- Original Message -----
From: "Jay R. Ashworth" <jra at baylink.com>
To: <filepro-list at lists.celestial.com>
Sent: Monday, July 16, 2007 4:35 PM
Subject: Re: OT: using pscp to *move* files
> On Mon, Jul 16, 2007 at 03:57:39PM -0400, Kenneth Brody wrote:
>> Quoting Jay R. Ashworth (Mon, 16 Jul 2007 15:18:24 -0400):
>> [...]
>> > Non-atomicity: I can't just have teh windows side say "delete
>> > everything" after I've grabbed everything, as I'll end up with a race
>> > condition -- if the Unix side drops a new packet, it will get eaten.
>>
>> Yes, my first thought was ftp's "mget" followed by "mdel", but that
>> wouldn't work if a new file appeared.
>>
>> [...]
>> > So I need either a -d option to pscp (which doesn't appear to exist),
>> > *or* I need to punt and go back to figuring out why Samba stopped
>> > working between these two boxes (at which point I can go back to a
>> > drive letter and ren).
>>
>> Can't you use pscp to get a directory listing, and then get/delete
>> the files one at a time? That eliminates any race condition with new
>> files appearing after the gets.
>
> There are lots of ways I could do this, if I was permitted to specify
> scripting on the Windows side. But I can't.
>
> Cheers,
> -- jra
> --
> Jay R. Ashworth Baylink
> jra at baylink.com
> Designer The Things I Think RFC
> 2100
> Ashworth & Associates http://baylink.pitas.com '87
> e24
> St Petersburg FL USA http://photo.imageinc.us +1 727 647
> 1274
> _______________________________________________
> Filepro-list mailing list
> Filepro-list at lists.celestial.com
> http://mailman.celestial.com/mailman/listinfo/filepro-list
>
More information about the Filepro-list
mailing list