Tip: Pointing pylibmc at a unix domain socket

This is a quick tip that will hopefully save someone else a few minutesat least. If you’re using Django 1.3, using pylibmc and want to point at a local unix domain socket for memcached, this is what your CACHES setting will look like:

CACHES = {
        'default': {
                'BACKEND': 'django.core.cache.backends.memcached.PyLibMCCache',
                'LOCATION': '/tmp/memcached.sock',
        }
}

This also assumes that you have configured memcached to use a unix domain socket (with the -s option in memcached.conf).

A MUD behind a proxy is… potentially great

My perpetual tinker project is a MUD server that may or may not eversee the light of day. In recent adventures, I pursued using Exocet to make my goal of a mostly interruption-less experience for the players a reality. The attempt worked in most cases, but failed horribly in a few others. The failures were bad enough to make me scrap the idea. The next best thing I could think of is to stick a proxy server in front of the MUD server.

The proxy would handle all of the telnet protocol and session stuff, and just dumbly pipe input into the MUD server through Twisted’s AMP (a really neat, simple, 2-way protocol). When a user inputs something, the proxy says to MUD server “A session attached to an Object with an ID of “a20dl3da” input this. The MUD server would then have any object matches route the input through whatever command tables they are subscribed to, causing things to happen in-game.

Communication back to the proxy would happen whenever an an Object’s emit method (IE: print to any sessions controlling this object). The proxy would see if it had a session attached to the given object, and call the TelnetProtocol’s msg() method with the output.

Neat thing #1: Strictly enforced separation

Convention typically dictates that connection and protocol-level things be kept separate from business logic and other more interesting things. However, having separate proxy and MUD server sections of the codebase really enforces that separation in my mind.

Keeping session and protocol-level gunk confined to the proxy makes the MUD server easier to understand, maintain, and test. I find this layout a little easier to mentally digest.

The other cool thing in the future is that adding support for other protocols (websockets, anyone?) can be handled in the proxy, hooking the input/output into AMP commands. Protocols are already in their own island with Twisted, but this separation is much more strictly enforced under this arrangement (which again, I like). The MUD server can speak in a protocol-agnostic format like Markdown or BBcode, and the protocols can format the output for whatever they are serving.

Neat thing #2: Neither proxy nor MUD server care if the other dies

Consider the following two scenarios:

• MUD server dies, proxy stays up.

  The proxy accepts connections, but all input is left with an error message telling the user to stay put until the MUD comes back up. All sessions are maintained, and Twisted’s auto-reconnection facilities continuously tries to get back in touch with the MUD. When it does come back up, business continues as usual without interruption. The MUD server doesn’t care about sessions, and the proxy doesn’t care about in-game objects, rooms, and etc.

• The proxy goes down, but the MUD server stays up.

  This one isn’t quite as neat. In theory, this scenario should be extremely rare. If the proxy goes down, the user is unable to connect to the running game. They’ll need to re-connect once the proxy comes back up. However, the MUD server continues about its business in the meantime, so mobs are moving, the economy is ticking, etc. Once the proxy is back, it re-connects and players can interact with the game world again.

Neat thing #3: We don’t need to bother with code re-loading

The last, and most important, neat thing is that because of neat things #1 and #2, we don’t need to implement code re-loading. If both proxy and MUD server are monitored/auto-restarted by something like Supervisor, the latest version of the game code can be loaded by silently shutting down the MUD server (but leaving the proxy up). Supervisor (or runit, or launchctl, or whatever) sees the server process down, restarts it, and the proxy automatically re-connects as soon as it’s back up.

The end result is that the user may get an error or two if they’re trying to type stuff while the server is down, but the outage should be short and potentially completely unnoticed by some of the players. We don’t need to worry about all of the messyness associated with code reloading, and we can keep the MUD server focused on game logic.

Code to come

I’ve got a proof-of-concept for this arrangement “working”, but it’ll be some time before I am able to restore the existing features of the MUD server to work with the new proxy + MUD server model. I’ll continue to write posts about progress as it happens.

GoLang gets it App Engine on

Today, Google anounced the general availability of its Goprogramming language on App Engine. It will be interesting to see how this pans out, but this very well may be the kind of endorsement needed to catapult the language out into the semi-mainstream. While still young, this language is looking very promising.

My initial tinkerings liken it to a cross between Python, C, and a little bit of Java. Visually, it seems to look the most like Python and C, which is a big plus for someone who loves the syntactical minimalism of Python (not so much C).

While there are certainly going to be some issues right out of the gate, it will be very interesting to watch App Engine and Go evolve separately and together.

Exocet makes code reloading easy

One of the big philosophical “pillars” I’ve been building my tinkerPython MUD on is that cold restarts (restart the process, clients are disconnected) should be exceedingly rare. Code re-loading in Python can be challenging, but Allen Short has nailed it with his Exocet module.

The coolest part for me (as it pertains to my game) is as follows:

“Exocet is a new way to load Python modules. It separates the act of naming a dependency from the act of creating a module object. As a result, more than one instance of a module can be created from the same source file.”

As a result of this, we can deal with Python modules as objects, and can easily replace them. This is how it ends up looking for me (lots of things removed for the sake of brevity):

import exocet
# Load the general commands module, stuff it in a variable.
general_cmds = exocet.loadNamed(
   'src.game.commands.general',
   exocet.pep302Mapper
)
# Add a command to the command table.
self.add_command(general_cmds.CmdLook())

This is an excerpt from my game’s global command table. The general command module is loaded by exocet instead of Python’s regular import statement. Since we’ve tossed the module reference in a variable, the normal rules of Python garbage collection apply.

While my game does this a little differently, running the line that populates general_cmds again would mean that the old general command module’s reference count would drop to zero, hence it would be garbage collected. In its place, you have the newly loaded general commands module with any new code updates. Here is all I’d need to do to re-load the general commands module after some modifications:

import exocet
# Re-load the general commands module, replacing the old one.    general_cmds = exocet.loadNamed(
   'src.game.commands.general',
   exocet.pep302Mapper
)

Some disclaimers

Of course there had to be a catch, right? I’m still learning how to best use this, but here are a few pointers:

• Your Exocet-based imports will probably always look strange to you, because they are. However, the power you get from them is well worth the “different” look to them. This is a pretty silly point, but I am pretty silly about code cosmetics.
• Be careful about storing references to exocet-loaded modules. If you reference an exocet module from other modules/classes, you may find yourself in a situation where the old module’s reference count never drops to zero, and it isn’t garbage collected. I get around this by using properties to replace things that were previous instance attributes. I probably did a bad job explaining this, sorry!
• Exocet is still in development. It is only available on Launchpad (sadface). However, some pip+bzr magic can make that less of a problem.
• Documentation is limited to a series of blog posts by the author. They are pretty useful, but you’ll have to wade through them if you’re looking for something.

Of Markdown, REST, and others

As I’m working on my side project (You should probably watch it onGitHub to stroke my ego), I’ve been toying with how I’ll eventually handle colors and web content. I’m almost fully set against using MUX/MUSH color codes, as those don’t do the best job of handling where color starts/ends. They’re also not very friendly for web-based output.

I’ve been considering a few different approaches:

• Restructured Text
• HTML
• Something resembling customized BBcode

I really like Restructured Text, but am not quite sure how I’d substitute in the color sequences on the MUD side. HTML (or a subset of it) is possible, but I’m not sure this is a very clean solution. BBcode could work, and is probably the most simple to implement, but I wonder if it’d prove too limiting.

I’ll open this up to other MUD developers for commenting. How do you handle color in a way that plays nice with a variety of different clients/protocols (I’m very interested to hear how games that have both a Telnet and web-based service handle this).