Read the Docs: A huge thanks!

When a free service or module proves to be invaluable to yourproject(s), it’s only right to thank the authors for their excellent work. I thought I’d take a brief moment to thank all of the contributors behind Read the Docs for this simple but wonderful site. I have found it to be extremely useful for my hobby and real-world projects.

For those who have yet to see the light

If you have Sphinx-based documentation and haven’t played with Read the Docs yet, I can’t recommend it enough. Not only do you not have to hassle with hosting your compiled HTML documentation yourself (or on a specially named branch in VC), you can point many software forges’ post-commit hooks at a specific URL to enable automatic doc updating with each commit/push.

I no longer have to manually udpate the docs with each change, or juggle the annoying gh-pages branch on GitHub. Each commit posts to a project-specific URL on Read the Docs, and RTD pulls the latest source, runs Sphinx, and posts the result up within a minute or two.

Small victories in efficiency

If all of this doesn’t seem like a big deal, that’s because it isn’t. This is a small, incremental improvement to my development process that leads to less context switching, less manual labor, and less stuff to worry about. This is not the cure for cancer and it won’t win a Nobel Peace Prize.

However, it is just the very small, simple improvement that makes a noticeable impact. Low-hanging fruit. The tastiest kind.

Evennia MUD server grows a proxy

I’ve obviously been on a bit of a proxied MUD server tangent of late(see here and here). I typically haunt #evennia on FreeNode, and brought the point up to the current maintainer, Griatch, a few weeks ago. As is typical for Griatch when he finds something he likes, he cranked out his take on an AMP-based proxy (the Evennia term is “portal”) very quickly, and announced the specifics a few days ago.

The biggest benefit in the case of Evennia is that the old, complicated code reloading system is now completely gone. This is a big win in simplicity and avoiding nasty edge cases where reloading doesn’t work as expected.

For those that haven’t heard of Evennia, it’s a Python + Twisted + Django MUD server that gives you a solid foundation to immediately start adding your game-specific stuff to. The documentation (particularly the developer section) is outstanding, and is amongst the very top tier of MUD servers. If you are even partially curious, consider joining the Google Group or lurking in #evennia on FreeNode. There are lots of interesting discussions on both from people of varying levels of involvement.

dott - now with more proxy

As mentioned in my previous post about putting a MUD behind a proxy,I’ve been really intrigued by some of the things such a setup would allow. In fact, my motivation was enough to warrant actually trying to implement my own interpretation. My implementation of this split proxy/MUD setup has now been merged into my master branch of dott on GitHub (my tinker MUD that I work on when I get the urge).

A quick refresher

For those who haven’t read the first article, or are too lazy to do so, what I’m doing is sticking a proxy in front of a MUD server. The proxy is the only one of the two that handles telnet connections, maintaining them even if the MUD server is down or restarting. In my case, the proxy also handles authentication and account creation, but this is entirely optional, and just something I wanted. The MUD server only handles game world related stuff: rooms, objects, combat, AI, etc.

The primary benefit of this setup is that instead of dealing with messy live, “seamless” code reloading, I just shutdown and restart the MUD server without alerting the players. The proxy maintains the connections, and the MUD server’s startup procedures are fast enough to where very few players will ever notice that the game went down and back up. If the MUD server is down, the player will get a “The MUD is currently re-loading” message telling them to try again in a few seconds.

How it works

Since my tinker project, dott, is built with Python and the excellent Twisted framework, I had access to AMP, a very simple async messaging protocol. This allows for bi-directional communication between the proxy and the MUD server. The proxy can do stuff like pipe commands to the MUD server through various objects, and the MUD server can tell the proxy to emit messages to any player controlling an object.

Beginning with the startup process, the proxy fires up, starting a telnet server factory to accept telnet connections from players. It imports the MUD server‘s proxyamp module, which contains protocol definitions and an auto-reconnecting client factory used to communicate with the MUD server. Both the proxy and the MUD server use the same ProxyAMP class and its commands contained within.

The MUD server is started up, and the proxy auto-retries its AMP client-> server connection and finds the MUD server reachable over AMP. A connection is established, and the two pieces return to working together.

Nuances

• Both the proxy and MUD server are monitored with supervisor. If either of the two go down, supervisor immediately re-starts them. My @reload command simply shuts down the MUD server. Supervisor sees the exit and restarts it for me with very little delay. The proxy maintains connections and things automatically pick right back up where they left of, when the MUD server returns.
• The MUD server and proxy can be restarted independently and in any order. Shutting down and starting up the proxy leads to no ill effects for the MUD server. It’d just boot the players. Restarting the MUD server leads to no discernible interruption, unless the player types a command. In that case, they get an error message telling them to try again in a few seconds.
• The proxy server handles any protocol-specific display stuff. I’ll eventually be doing room descriptions in some form of markup. The proxy would parse the markup and replace it with the appropriate color/formatting codes for the protocol. For telnet, this would be ANSI color, for a theoretical future WebSocket-based client, this might be JSON or HTML. The important thing to grasp here is that the MUD server is just handling game stuff, which is really neat.

AMP enables the building of distributed MUDs

The following is rambling just for the sake of illustration. I don’t have any immediate plans to do any of the following, since MUDs are not at all resource-intensive, and are best kept simple. With that said…

If one were so inclined, next steps might be breaking out various complicated systems into their own separate services. For example, maybe we have an authentication service that the proxy communicates with over AMP for logging users in. Your website could then also communicate with this authentication service (just like the proxy), instead of hitting the MUD server directly (I don’t really like the idea of hitting a MUD server with web-related traffic). Creating new accounts from the web uses the same plumbing as the proxy.

Or maybe you run a Twisted IMAP service for your game’s mail. Users within the game using mail commands would be reading their mail via AMP messages to the mail service that runs outside of the game. Those with IMAP-enabled clients could hit the mail service separately, instead of hitting a baked-into-the-mud service with potential denial of service or security issues.

Another possible external service would be AI and/or a real-time combat system. The combat system service could handle the ticks, coordinates, and AI stuff that may be resource-intensive, messaging the MUD server when stuff happens. Users steering or moving things around would send AMP messages to the combat system letting it know to change states. This is a cheap way to make use of multi-core machines, if you really need it.

Feedback, ideas, and whatever else

Coming back to reality for a second, I’d love to get some more eyes on the code (on GitHub), or hear any ideas you may have. Please feel free to reply below with a comment. I am writing this stuff for myself for my own selfish use, but constructive criticism is welcome and encouraged. The unit testing is marginal at best at the moment, as I’m trying to figure out the best way to test AMP. However, I’ve tried to make sure the docstrings are detailed.

If there are any slick capabilities that this setup enables, but I haven’t mentioned them in either post, do speak up in case I missed them. There hasve already been some comments that pointed out things I never thought of.

Tip: Pointing pylibmc at a unix domain socket

This is a quick tip that will hopefully save someone else a few minutesat least. If you’re using Django 1.3, using pylibmc and want to point at a local unix domain socket for memcached, this is what your CACHES setting will look like:

CACHES = {
        'default': {
                'BACKEND': 'django.core.cache.backends.memcached.PyLibMCCache',
                'LOCATION': '/tmp/memcached.sock',
        }
}

This also assumes that you have configured memcached to use a unix domain socket (with the -s option in memcached.conf).

A MUD behind a proxy is… potentially great

My perpetual tinker project is a MUD server that may or may not eversee the light of day. In recent adventures, I pursued using Exocet to make my goal of a mostly interruption-less experience for the players a reality. The attempt worked in most cases, but failed horribly in a few others. The failures were bad enough to make me scrap the idea. The next best thing I could think of is to stick a proxy server in front of the MUD server.

The proxy would handle all of the telnet protocol and session stuff, and just dumbly pipe input into the MUD server through Twisted’s AMP (a really neat, simple, 2-way protocol). When a user inputs something, the proxy says to MUD server “A session attached to an Object with an ID of “a20dl3da” input this. The MUD server would then have any object matches route the input through whatever command tables they are subscribed to, causing things to happen in-game.

Communication back to the proxy would happen whenever an an Object’s emit method (IE: print to any sessions controlling this object). The proxy would see if it had a session attached to the given object, and call the TelnetProtocol’s msg() method with the output.

Neat thing #1: Strictly enforced separation

Convention typically dictates that connection and protocol-level things be kept separate from business logic and other more interesting things. However, having separate proxy and MUD server sections of the codebase really enforces that separation in my mind.

Keeping session and protocol-level gunk confined to the proxy makes the MUD server easier to understand, maintain, and test. I find this layout a little easier to mentally digest.

The other cool thing in the future is that adding support for other protocols (websockets, anyone?) can be handled in the proxy, hooking the input/output into AMP commands. Protocols are already in their own island with Twisted, but this separation is much more strictly enforced under this arrangement (which again, I like). The MUD server can speak in a protocol-agnostic format like Markdown or BBcode, and the protocols can format the output for whatever they are serving.

Neat thing #2: Neither proxy nor MUD server care if the other dies

Consider the following two scenarios:

• MUD server dies, proxy stays up.

  The proxy accepts connections, but all input is left with an error message telling the user to stay put until the MUD comes back up. All sessions are maintained, and Twisted’s auto-reconnection facilities continuously tries to get back in touch with the MUD. When it does come back up, business continues as usual without interruption. The MUD server doesn’t care about sessions, and the proxy doesn’t care about in-game objects, rooms, and etc.

• The proxy goes down, but the MUD server stays up.

  This one isn’t quite as neat. In theory, this scenario should be extremely rare. If the proxy goes down, the user is unable to connect to the running game. They’ll need to re-connect once the proxy comes back up. However, the MUD server continues about its business in the meantime, so mobs are moving, the economy is ticking, etc. Once the proxy is back, it re-connects and players can interact with the game world again.

Neat thing #3: We don’t need to bother with code re-loading

The last, and most important, neat thing is that because of neat things #1 and #2, we don’t need to implement code re-loading. If both proxy and MUD server are monitored/auto-restarted by something like Supervisor, the latest version of the game code can be loaded by silently shutting down the MUD server (but leaving the proxy up). Supervisor (or runit, or launchctl, or whatever) sees the server process down, restarts it, and the proxy automatically re-connects as soon as it’s back up.

The end result is that the user may get an error or two if they’re trying to type stuff while the server is down, but the outage should be short and potentially completely unnoticed by some of the players. We don’t need to worry about all of the messyness associated with code reloading, and we can keep the MUD server focused on game logic.

Code to come

I’ve got a proof-of-concept for this arrangement “working”, but it’ll be some time before I am able to restore the existing features of the MUD server to work with the new proxy + MUD server model. I’ll continue to write posts about progress as it happens.