Why you should donate to the Django fellowship program

Disclaimer: I do not represent the Django Software Foundation in any way,nor has anything below been endorsed by the DSF. The following opinions are my own, unsolicited rambling.

If you hadn’t been looking for it specifically, you may have missed it. The Django Softare Foundation is running a fundraising effort for the new Django Fellowship program. It sounds like they’re still trying to figure out how to get the word out, so I wanted to do what I could to tell you why you should chip in.

This particular blog post is going to focus on encouraging (peer-pressuring) commercial Django users in particular, though enthusiasts are welcome to read along!

Humble beginnings

Django is free and open source. Just provide the expertise and the infrastructure and you can build just about whatever web powered contraption you’d like. So you end up doing just that.

Your first stop is the Django tutorial, written and maintained by a community of volunteers (just like the rest framework itself). You stumble along, slowly at first. Perhaps you find yourself frustrated at times, or maybe things move along at a faster pace. In no time, you’ve got “Hello World!” rendering, and here comes a business idea!

One hundred lines of code turns into a thousand, then five thousand, and beyond. You start seeing signups, and revenue begins to trickle in. You toil away at your codebase, making improvements and dealing with the “accidental features” that crept in during one of your late night dev sessions.

You could have built your business on one of any number of frameworks, but you chose Django. You like how it’s a very productive way to build a web app. You appreciate how it’s not impossible to find Django developers to work with you. There are probably some things you don’t like, but you might not have the time to work on fixing them yourself. You’re just busy shipping and growing.

But it could be better still!

You’re happily using Django, it serves you well. There are a few things you’d love to see fixed or improved, but you don’t really have the time or expertise to contribute directly. As luck would have it, all of the Django core developers have day jobs themselves. Things would progress much more quickly if we had someone working full-time on Django…

Enter: Django Fellowship Program. The idea is to fund at least one Django developer to work for the DSF part or full-time for a while. During this fellowship, said developer sets aside some or all of their other responsibilities to focus on improving Django. The DSF, in turn, pays the developer a fair (but low rate) for their work.

As per the Tim Graham’s recent retrospective blog post, we’ve see some huge leaps forward for the project during these fellowships. These are periods of focus and rapid improvement that everyone (including your business) benefit from.

The only problem is that we’re not going to see the benefits of this program unless it gets (and stays) funded. A well-funded fellowship program could mean one (or more) developers working on Django full-time at any given point in time. That would be huge for the project (and you and I).

Why you should donate

As a business, we are donating to the fellowship program to see one of our critical components improved. Due to the fellowship application process, you can be assured that your money will be paying a capable, trusted developer to get things done.

Consequently, you can view a donation to the Django Fellowship program as an investment with an almost assuredly positive return. If you are making money with Django, consider making a (potentially tax-deductible) investment in what may be the foundation of your business.

At the end of the first full day of fund-raising, there are precious few commercial donors listed in the “Django Heroes” leaderboard. Let’s help change that!

If you don’t hold the purse strings at your business, get in touch with someone who does and tell them about this investment with near-guaranteed returns.

Let’s play: python-gotalk

A recent HackerNews post announced Gotalk, a simple bidirectional protocol.I can imagine your collective eyeballs rolling. “Oh great, yet another half-baked way for… things to talk to one other”. But keep following along, maybe you’ll see something you like. Here are some highlights:

• By Rasmus Andersson - You may know him from his work at Facebook, Spotify, and Dropbox.
• Bidirectional — There’s no discrimination on capabilities depending on who connected or who accepted. Both “servers” and “clients” can expose operations as well as send requests to the other side.
• Concurrent - Not too earthshattering, but this is a non-blocking, multiplexed protocol.
• Debuggable - ASCII-based wire format. You can, of course, encode your payloads any number of way, but you’ll have an easy time with your packet sniffers and debug tools.
• Pretty damned simple - There’s not a whole lot to Gotalk. A handful of message types. A pretty simple, easy-to-parse wire format. Nothing crazy for terminology or concepts. If you were curious about network protocols and wanted a gentle intro, this would be a great one to look at.

The official Gotalk repo has a bunch of examples and even some helper libraries in Go and JS. The official implementation is in Go, with the JS libraries being built on top of WebSockets.

But what about Python?!?

“But Greg, I too want to play with this fledging version 0.0 protocol! And I want to do it in Python!”

You, my friend, are in luck! I have a rough, version 0.0 Python module to go with this new, version 0.0 protocol.

For now, most of the effort is on message serialization and deserialization. We’ll be keeping that separate from any of the naughty bits (sockets, IO, and other things). The goal is to provide some reusable components that people can tinker with.

And more importantly, I haven’t ever bothered to mess around at the protocol level very often. This has been a great excuse to play around with a spec that is just getting started.

Pull requests, issues, suggestions, and the whole lot are welcome. Tell me how bad I screwed up!

Closing notes

If it wasn’t already evident, do not use this for anything but tinkering right now!.

Also, to stem the tide of “Well, why didn’t you just use X instead?”, this is a fun little experiment for me. Yes, I wrote a protocol serialization/deserialization for funzies. No, I’m not mentally ill.

EMDR Relay 1.1 Released

EVE Market Data Relay (EMDR) has been chugging along behind the scenesin the EVE Online developer community, quietly delivering large volumes of player-supplied market data. But the winds of change are arriving, as CCP has released a set of HTTP APIs for obtaining much of the data directly. EMDR will continue to function for those who don’t want to poll on their own.

The first step in putting a fresh coat of paint on EMDR is to freshen up the relays. We have done just that, updating and modernizing a few things.

Some hilights of the 1.1 release:

• I have built a Docker image that is now the officially endorsed way to set up and run an EMDR relay.
• We’ve upgraded (and now require) ZeroMQ 4.x. If you use the Docker image, you don’t need to worry about this.
• We now auto-restart the process every 12 hours. This works around some ZeroMQ edge cases where connections aren’t restored correctly. Most of our relay operators are already doing this, but now we all will, just in case.

See our Docker Hub repo for full setup instructions. Current relay operators are encouraged to upgrade, though nothing will break if you don’t.

python-colormath 2.1.0 released

python-colormath 2.1.0 haslanded, bringing with it some excellent new features and bug fixes. See the release notes for a more detailed look at the changes.

The headlining feature is the replacement of our hardcoded conversion tables with NetworkX-based resolution of color conversions (courtesy, Michael Mauderer). Color Appearance Models and IPT round out the rest of the new features.

Installation

The easiest way to get python-colormath is through pip/easy_install:

pip install colormath

Alternatively, grab it from PyPi.

Halp!

If you get stuck, create an issue in the issue tracker and we can figure it out.

Networked, multi-container image crawling with Docker and fig

Like many, I’ve been playing with Docker as it has been developing. I feltreasonably comfortable creating images and running isolated containers, but I had yet to tinker with automatically provisioning and networking multiple containers together.

To remedy this, I spent a weekend writing a simple distributed dockerized image crawler using Twisted, Docker, and fig. The full source (under a BSD license) can be found in the Github repo.

The README covers a lot of this in more detail, but the application is comprised of three container types:

• An HTTP web API for enqueuing jobs. For the sake of this example, you only run one of these, but it’d be easy to adapt the code to support multiple instances.
• A worker container that carries out the image crawling. You can run as many as these as you’d like.
• A Redis container that is used for tracking job state and results.

The worker daemons connect to the HTTP web API container over ZeroMQ, PULLing jobs that the HTTP daemon PUSHes. If this were anything but a toy app, you’d probably want a proper message broker that could offer stronger deliverability and persistence guarantees.

Assorted Observations

• I may have been doing something wrong, but boot2docker seems to be pretty rough with volume mounting. It looks like it’d be pretty easy to mount a directory within the boot2docker VM, but the host machine->container sharing didn’t work in real-time. I expect this will improve with time, or I could have missed something.
• fig is a great idea, and is very handy. I’m looking forward to seeing where this concept is taken over the next few years.
• There is a great python:2.7 image on Docker Hub that serves as a great starting point. They’ve got all sorts of other versions and alternative implementations (PyPy) covered.