Amazon AWS S3 and Setting Response Headers

One of the previously annoying limitations of Amazon S3 was that youcould not specify things like Content-Disposition on a per-request basis. If we use an MP3 as an example, if you uploaded it without a Content-Disposition header set, viewing it in a browser via the S3 URL would likely have your browser playing it rather than downloading. This behavior varies based on browser, but the big point is that it lead to inconsistency and confused some less technical users (“What do you mean I need to Right-Click then Save As?”).

Response Headers over GET to the Rescue!

This is recent, but breaking news. Googling around, it seems like this went largely unnoticed by many, so I’ll bring it up again here. You can now specify headers for GET requests for keys/objects as URL parameters. The response from S3 will have your provided headers instead of whatever was set at the time of upload (or not at all).

See their announcement forum post for more technical details. As a general note, boto, my weapon of choice for stuff like this, does not support this yet. We’re working on that, though.

How do YOU deploy Django?

There are a good number of choices for deploying Django projects,whether they be Fabric, Chef, Puppet, Paver, or something else. However, each may be more suitable for some types of deployments than others. For example, Fabric’s lack of parallel execution may leave you frustrated if you have many dozens of hosts to deploy to, and Chef’s complexity and scattered documentation may be somewhat discouraging, as powerful as it seems.

Show and Tell

We (DUO Interactive) have been using Fabric for our deployments. We’ve got a few projects in development and maintenance, most of which are single app server situations (which Fabric is awesome for). To keep our deployments consistent across our various projects, we wrote a smattering of management commands and Fabric methods, and tossed it up on GitHub as django-fabtastic. There is no real documentation aside from comments, but we figured we’d share what has worked for our simple deployments.

We’ve been extremely happy with Fabric and fabtastic for our simple deployments, but a new project looks to be moving towards the more complex end of the deployment spectrum in the future. We’re looking at the possibility of at least five app servers by year’s end, and are frustrated with our Fabric-based deployment (with a handful of app servers) for a few reasons:

• It can be pretty slow with a good number of app servers.
• Hosts are deployed to one at a time, with no abiiity to execute deployments in parallel.
• We sit our EC2 app servers behind one of AWS’s Elastic Load Balancers. At any given time during deployment, users might get an updated app server, or one that has not been deployed to yet. This inconsistency is troubling. We’d still get this even with parallel execution (as it stands right now), but it would possibly be less of a problem.

In an Ideal World

I personally would love to see a deployment system that could do the following:

1. Fire off an individual deployment step to a large number of nodes at a time (in parallel, not one at a time).
2. Have the ability to wait until all nodes have completed certain steps before progressing onwards with the deployment. For example, wait until all nodes have checked out the latest version of the source before restarting the app servers. There would obviously still be a small period in time where not all nodes have reloaded the code, but it would potentially be a lot better than what we’re seeing right now.

What do YOU do?

So I ask the readers: What tools do you use to deploy your projects with, and how many Django app servers do you deploy to? What works well for you with your deployment strategy, and what irritations do you face?

Amazon EC2 and long restart delays

For the benefit of others either considering Amazon’s EC2, or who arealready there, I thought I’d point something out. I am not sure if this is an Ubuntu EC2 AMI issue, an EC2 issue, an EBS issue, or some combination of all of these, but we are experiencing some really erratic restart times. We run our EC2 instances with the following basic configration:

• Size: small and medium high-cpu
• Root device: EBS
• Distro: Ubuntu 10.10 (the latest)

Symptoms

The behavior we are seeing is that even our simplest, no-extra-EBS-volumes instances periodically hang on boot when restarted. The restart can range anywhere from less than 60 seconds, to 5 minutes, to not at all. In the case of ‘not at all’, something bad happens to the point where we the instance fails to reach the point where we can even SSH in, and the syslog that the web-based management console shows looks to be out of date.

In the case of a complete hang on restart (it seems to be 50-50 right now), we have to reboot the machine again from the web-based AWS EC2 management console. This second reboot usually results in a full startup.

Our hunch

From what we can tell, this may be EBS-related. Even though we specifically have nobootwait on our swap partition (which is an included ephemeral drive that is standard with the Ubuntu AMI), it seems like Ubuntu may be freaking out when it can’t reach the root EBS drive. It’s also possible that even despite the nobootwait in the fstab, this same thing could be happening with the swap partition as well. We haven’t had a lot of time to try different combinations, as we’re insanely busy right now.

If anyone else has experienced something similar, please chime in.

First impressions: Android Appstore

I took a moment to give Amazon’s new Appstore a quick run-throughtoday, and thought I’d mind dump a few assorted thoughts.

Competition is a good thing

One of the worse thing about monopolies is the lack of motivation to change, adapt, and listen to the needs of customers. Why bother bending to the wills of your users when you’re the only option? What if I (Joe Consumer) don’t agree with the selection process a store uses to determine which apps make the cut? When there’s only one option, I’m out of luck, and at the provider’s mercy.

Even if Amazon Appstore ends up being an also-ran (which I suspect it will), it’s putting some amount on Google to continue improvement of their Market. I sincerely hope that Amazon improves to the point where the competition pushes both parties to innovate.

The concern of fragmentation is somewhat unfounded

“Fragmentation” is a term used in a lot of FUD campaigns. It is important to keep in mind the fact that Google’s Market will be the default (much like IE is for Windows), and will likely always be. Most of us will probably look there first. But just like I can navigate away from eBay to Amazon while looking for something, I can now do the same with apps. Maybe one has something priced cheaper. Maybe the superior quality of one causes policy changes in the other. A win-win just for having to tap a piece of glass a few times (instead of clicking).

It’s also important to realize that we deal with fragmentation every day. Have you ever downloaded software from somewhere other than Microsoft Update, Mac’s App Store, or one of Linux’s many package managers? We get software from all kinds of different sources, and I’m glad for the fact.

Amazon may not ultimately succeed, but this is a victory for the consumer. Developers that are annoyed at having to submit to multiple places will probably have their concerns answered by a service that handles submissions to multiple sources. Or the process in general will be streamlined to the point where it’s not an issue.

As far as the store itself

There were some definite positives, and there were the expected early-life issues. First, the good:

• I liked being able to select apps for download from the web and have them pulled by my device, like Google Market does (as long as the Amazon Appstore app is running).
• Amazon’s recommendation engine is pretty nice. Google’s is not nearly as good. This is one of those places that I expect Google will have to improve to compete.
• The web interface is decent.

Now the bad:

• The Android app’s UI is pretty clunky. I suspect they’ll fine tune this over time, just like all other similar products have done. This is not a deal-breaker.
• As expected, the selection is limited. Not a deal-breaker yet, we’ll give them some tme.

A delicate balance

The only thing out of all of this that sends up red flags is that Amazon is instituting an approval process similar to Apple’s. However, I cringe less at that is the fact that Amazon isn’t also the platform provider (as Google is to Android). Amazon doesn’t care if your Photo Gallery app competes with Android’s, as long as it adds a quality replacement.

I did see a statement that anything pornographic will be denied, which is fine, but we’ll see what constitutes pornographic material soon. There was also mention that the approval process might take a week, which might dissuade some developers.

Summary

Amazon’s Appstore is the first attempt that comes to mind for an alternate phone app store at this scale. They are very much pioneers of this for Android, and maybe the smartphones in general. The Appstore does not have enough clout to make it something I’ll spend a ton of time thumbing through just yet, but I will certainly peek at it from time to time.

Fragmentation is a non-issue, and I welcome the competition that Amazon brings against Google and others thinking of doing the same thing. The consumer wins when we have more than one option. Just like individual app stores will evolve and improve, I expect the whole alternate app store thing to become a lot more natural with time. Perhaps Google will even embrace this and offer some additional programmatical facilities for developers.

Evennia gets Southy

The Django-based Evennia MUD Server took another great step forwardtoday in adding support for the excellent South migration app. This should knock down another barrier for those considering beginning development on their own games. The kinds of schema changes from this point forward should be fully covered by the South migrations that we provide.

For those that aren’t familiar with Evennia, it is the first well-established MUD server built with Django (perhaps the first, period). This makes game development extremely simple, and gives us a lot of power to webbify games (take a look at the included web-based client, for an example). Twisted handles our network layer, and some scheduling. Another unique thing (as far as MUDs go) is that there is actually a good bit of documentation.