chendo

S3: Going commando line

Dr Nic needed to upload his screencast to S3 so he could distribute his propagandapresentation about Javascript testing to the rest of the world. We tried to upload it from our work connection, but it kept failing with random errors, timing out, and killing the response time of the net since it was choking the upstream.

And so I suggested uploading the 400mb or so file to a remote server via a protocol that can easily be throttled, then uploading it to S3 from there. Since it was my idea, I was ‘honoured’ with handling the logistics of this.

The first step was picking a method to get the file on the remote server without killing your net. My first thought was rsync, and a quick Google search said that there’s an option called –bwlimit which allowed you set the I/O speed (file I/O, not network I/O, but it worked anyway). So about three hours later, the screencast was on the server and then it was down to getting this on to S3.

I found a tool named s3cmd, which is a Python-based command-line tool for S3 management. After you install it (apt-get install s3cmd is what I used on our Ubuntu server), then, you need to run s3cmd --configure, where you enter in your access key, secret key and an optional encryption password (I didn’t do this). Then, I ran s3cmd put < filename> s3://<bucket name> and it was done in less than a minute. The wonders of a 100mbit connection

My first Yahoo Pipe: Brisbane Traffic Notices

This is my first Yahoo pipe I made for KIB216: Brisbane Traffic Notices. It takes data from http://131940.qld.gov.au/, runs it through a geolocation service and plots them on a map. It’s rather slow at the moment, probably all the geolocating it’s doing, and the geolocator sometimes gets it wrong.

This should be useful for anyone who drives, as they can easily see what roads to avoid, etc.

Photosketch

This is probably the next coolest thing since content-aware resizing: Photosketch.

What it does is converts a hand-drawn sketch with annotated text to a composite image that looks pretty damn good. You have to watch the video at the bottom of their page to see how awesome it is.