Amazon AWS S3 and Setting Response Headers

One of the previously annoying limitations of Amazon S3 was that youcould not specify things like Content-Disposition on a per-request basis. If we use an MP3 as an example, if you uploaded it without a Content-Disposition header set, viewing it in a browser via the S3 URL would likely have your browser playing it rather than downloading. This behavior varies based on browser, but the big point is that it lead to inconsistency and confused some less technical users (“What do you mean I need to Right-Click then Save As?”).

Response Headers over GET to the Rescue!

This is recent, but breaking news. Googling around, it seems like this went largely unnoticed by many, so I’ll bring it up again here. You can now specify headers for GET requests for keys/objects as URL parameters. The response from S3 will have your provided headers instead of whatever was set at the time of upload (or not at all).

See their announcement forum post for more technical details. As a general note, boto, my weapon of choice for stuff like this, does not support this yet. We’re working on that, though.

How do YOU deploy Django?

There are a good number of choices for deploying Django projects,whether they be Fabric, Chef, Puppet, Paver, or something else. However, each may be more suitable for some types of deployments than others. For example, Fabric’s lack of parallel execution may leave you frustrated if you have many dozens of hosts to deploy to, and Chef’s complexity and scattered documentation may be somewhat discouraging, as powerful as it seems.

Show and Tell

We (DUO Interactive) have been using Fabric for our deployments. We’ve got a few projects in development and maintenance, most of which are single app server situations (which Fabric is awesome for). To keep our deployments consistent across our various projects, we wrote a smattering of management commands and Fabric methods, and tossed it up on GitHub as django-fabtastic. There is no real documentation aside from comments, but we figured we’d share what has worked for our simple deployments.

We’ve been extremely happy with Fabric and fabtastic for our simple deployments, but a new project looks to be moving towards the more complex end of the deployment spectrum in the future. We’re looking at the possibility of at least five app servers by year’s end, and are frustrated with our Fabric-based deployment (with a handful of app servers) for a few reasons:

• It can be pretty slow with a good number of app servers.
• Hosts are deployed to one at a time, with no abiiity to execute deployments in parallel.
• We sit our EC2 app servers behind one of AWS’s Elastic Load Balancers. At any given time during deployment, users might get an updated app server, or one that has not been deployed to yet. This inconsistency is troubling. We’d still get this even with parallel execution (as it stands right now), but it would possibly be less of a problem.

In an Ideal World

I personally would love to see a deployment system that could do the following:

1. Fire off an individual deployment step to a large number of nodes at a time (in parallel, not one at a time).
2. Have the ability to wait until all nodes have completed certain steps before progressing onwards with the deployment. For example, wait until all nodes have checked out the latest version of the source before restarting the app servers. There would obviously still be a small period in time where not all nodes have reloaded the code, but it would potentially be a lot better than what we’re seeing right now.

What do YOU do?

So I ask the readers: What tools do you use to deploy your projects with, and how many Django app servers do you deploy to? What works well for you with your deployment strategy, and what irritations do you face?

Removing some dust

After sufficient embarrassment, I have finally forced myself to updatethe site. In addition to a visual overhaul, I’ve updated the codebase to use Django 1.3, and brought in a lot of the pieces of our stack we use at DUO Interactive.

I’ve also made the updated source for this site available in its entirety on GitHub. As a disclaimer, I don’t have the time or will to support the codebase, and it’s mostly there just so I can get this thing under version control. Perhaps it will be of use to someone as an example, though.

Django work available

A quick tip for anyone with time to spare: The company I work for, DUOInteractive, LLC, is in need of some extra Djangonauts for some small to mid-sized projects. The work is payable on an hourly basis, and can scale up or down based on availability. Most of the projects are for the Ligonier Ministries site, which sees about 10,000 visitors a day.

Those with a firm understanding of Python/Django (and some free time) may send resumes to jobs@duointeractive.com.

django-eve-db is Ready to Roll

While by no means at all ready to be deemed “mature”, django-eve-dbis now at a state that I feel fine about announcing its presence to the wider Python/Django community.

django-eve-db is a set of Django models that wrap CCP’s EVE Online data dump. This makes it extremely easy to query the dump and rapidly develop software around it. Little to no SQL is required for many usage cases.

I could really use some help testing the models, importers, and admin classes. If you are a current or potential EVE Online developer, I encourage you to take a peek. If you’d like to hear discussions and announcements about the project, see our mailing list.