FTP folder upload via command line / macro madness & WinSCP

I was pair-programming today and we made a change to a fundamental data type which required all the web-services to be rebuilt and republished.

We’d done this a couple of times before. The build is OK, but what then follows is a torterous process of manual FTPing 10 webserservices to a pre-live environment. Find folder locally, find folder remotely, select all..NOT web.config(!), double check you’re in the pre-live folder, copy, repeat.

The pragmatic part of me thought there had to be a better way of doing this. Why not just script it. Surely every man and his dog has a handy FTP upload script to recursively upload a folder of stuff?

Apparently not.

I tried PowerShell first. I like PS, but I’m a novice user trying to get better. Last time I used it (to find a string in a collection of files) I was blown away by how simple and elegant it was. 3 commands, piped together on one line, no fuss.

Turns out you can’t FTP in PowerShell.

OK, let me clarify… According to StackOverflow, you can, but only really by getting .NET to do it for you, and only then one file at a time. And the code…

# create the FtpWebRequest and configure it
$ftp = [System.Net.FtpWebRequest]::Create("ftp://localhost/me.png")
$ftp = [System.Net.FtpWebRequest]$ftp
$ftp.Method = [System.Net.WebRequestMethods+Ftp]::UploadFile
$ftp.Credentials = new-object System.Net.NetworkCredential("anonymous","anonymous@localhost")
$ftp.UseBinary = $true
$ftp.UsePassive = $true
# read in the file to upload as a byte array
$content = gc -en byte c:\me.png
$ftp.ContentLength = $content.Length
# get the request stream, and write the bytes into it
$rs = $ftp.GetRequestStream()
$rs.Write($content, 0, $content.Length)
# be sure to clean up after ourselves
$rs.Close()
$rs.Dispose()

That’s not a solution.

You could also do it using NetCmdlets, some sort of add-on for PowerShell, but seriously, even if it is free for one concurrent connection, registering, providing email addresses, downloading, installing … just to put some FTP away? In a scripting language?!

Where else did we try? Oh, we tried NAnt as well – I thought they must have a decent <ftp> task. Apparently not. The only one anywhere close was this NAnt FTP Task – but don’t waste your time getting excited about the example because although it looks like it does exactly what we needed, the link is broken and there’s no trace of an alternative.

We tried DOS’ FTP mput as well … but this isn’t recursive.

We did think that it would be pretty cool if there was an FTP client which recorded your commands, allowing you to play them back later, or save them to a script file, but alas, such a client doesn’t exist. Add it to the list of programs I will one day get around to making (yeah, right).

By this point, things were getting pretty desperate, and we started to consider just writing a small .NET Console Application to automate the transfer for us, although that would have had its problems as well.

Finally, however, we came across WinSCP. At one level, this is just another FTP Client. However, you can run it in command line mode, and you can send scripts to it. And it’s clever enough to know that when you specify a folder instead of a file, it needs to copy everything in that folder.

Within 10 mins of downloading, we’d thrown together a simple script using WinSCP’s put command which worked perfectly. The GUI isn’t too shabby either, so I’m pretty sure that WinSCP will be replacing CoreFTP as my new FTP Client of choice. It’s supposed to be a SSH file copy client (Win SecureCoPy), but it does FTP pretty well too, and is a true open-source app under GPL.