Lighttpd and Django
There was a discussion at work today about replacing Apache on an ebedded system with something a little lighter weight. Someone had recommended checking out Premium thttpd using FastCGI to communicate with our python web interface which was written directly against mod_python.
Premium thttpd seems to be a commercialized version of thttpd with more configurability. Since I'd never heard of it, and it costs money, I figured I'd recommend something I had heard of that doesn't cost money. I recommended lighttpd as a web server that meets the vague requirements I assumed they had.
Of course, my knowledge of lighttpd was pretty much limited to what I'd seen on their home page and the various places where people were saying it kicks a lot of ass. So I had a new quest.
In my usual easily-distracted manner, I set out to learn a lot about this thing I was recommending to people.
First thing I did was download the source code to it to get a feel for what it looked like. I was surprised to see a lot of mod_* files. This thing has modules in a way similar to Apache, and it has a lot of them in common. Some of them would seem to simplify a few of the other problems I've got (such as mod_secdownload). After having read a pretty decent set of documentation, I was interested in seeing if I could move all of my new sites over to it. I figured Django would be the biggest challenge.
Lawrence.com and the other major sites run by the Django people are all about mod_python and postgres, but other configurations are supported. Some of my Django sites are backed by postgresql, and some are backed by SQLite, but I've been using Apache with mod_python for as long as I've heard of Django. Seeing as how there's no mod_python for lighttpd, I got to play around with a FastCGI configuration.
Django has their official documentation on configuring Django to run under Apache or Lighttpd as a FastCGI. It didn't work very well for me, though. It was close enough that I got what I needed at least. In particular, I needed to have lighttpd spawn the server using the threaded method, have it not daemonize, and not declare a socket in order to work from the django side, and on the lighttpd side, I needed to tell it to spawn exactly one process to prevent it from practically filling up my process table instantly (both manage.py and lighttpd wanted to prefork servers).
The following is a snippet of my lighttpd config showing how the virt is configured:
$HTTP["host"] == "www.example.com" {
server.document-root = "/path/to/web/root"
fastcgi.server = (
"/django.fcgi" => (
"main" => (
"socket" => "/some/path/example.socket",
"check-local" => "disable",
"bin-path" => "/some/path/runfcgi",
"min-procs" => 1, "max-procs" => 1,
"bin-environment" => ( "VIRT" => "example", ),
)
),
)
url.rewrite-once = (
"^(/media.*)$" => "$1",
"^/favicon\.ico$" => "/media/favicon.ico",
"^(/.*)$" => "/django.fcgi$1",
)
}
Technically, I could remove the rewrites and turn on check-local, but that'd hopefully give me the same effect with more stats (but less config). I figured I'd keep it since Django recommends it, and it already works. If I start seeing lighttpd get bogged down with CPU, or I get tired of looking at those regexes, I'll try it the other way.
I'm passing the VIRT environment variable to have a single common shell script to start all of the instances. It looks something like the following:
#!/bin/sh
base=/some/path
s=/some/path/$VIRT.socket
p=/some/path/$VIRT.pid
cd $base/$VIRT
if [ -f $p ]
then
kill `cat -- $p`
rm -f -- $p
fi
exec /usr/bin/env PYTHONPATH="../python:.." python2.4 \
manage.py runfcgi \
method=threaded daemonize=false pidfile=$p
This seems to be running smoothly, and took much less time to get working than last night's Django conversion. I used some of my extra time to get sitemaps generated for several of my Django sites with dynamic content as well as a static site.
You can see some of my generated sitemaps for this site or Bracher PTA. The latter is also including flatpages.