Nginx came with this HTTP stub status module that you can get the current server status in a HTML page. However it’s not pretty comparing to Lighttpd’s and Apache’s. Moreover, the information it provides is a bit cryptic.

For example, when you browse to the stub status page (please refer to the wiki document linked earlier on how to set it up), all you get is 4 lines of text:

$ curl http://myserver/nginx_status
Active connections: 183
server accepts handled requests
 914384 914384 2725561
Reading: 3 Writing: 2 Waiting: 178

Not very meaningful. Instead, I wrote a small Python script (download here: nginxstats.py (2.3kb)) that does something like this:

$ ./nginxstats.py http://myserver/nginx_status
Conn     Conn/s     Request/s  Read  Write Wait
-------- ---------- ---------- ----- ----- -----
     157       9.57      31.07     0     2   155
     140      10.20      36.13     0     1   139
     147       9.33      33.60     0     4   143
     189      12.60      40.07     1     4   184
     164      13.07      41.53     5     2   157

Basically it fetches the data from the stub status page every 30 seconds (configurable in the script) and then do some calculation to find out how many connections have been established within the time frame, and what’s the average connections/second and requests/second. Similar to tools like vmstat, it basically runs continuously (until interrupted).

Very useful when you are getting slashdotted :)

I have a mystery and I’m not sure how to solve it. My 360 Linode is running several WordPress-based sites using nginx. Everything is great, except I find that, every few days, fastcgi fails and I start getting 502 gateway errors when I try to post a comment or upload an image, etc. Restarting fastcgi fixes the problem.

It looks like a very similar issue I had with Ubuntu 8.04 (and PHP 5.2.4), which was fixed when I migrated to Debian 5 Lenny. It appears that the OP is indeed running the problematic Ubuntu 8.04. Well. I still have no idea what the real issue was (thus do not know how to report a bug), except my PHP FastCGI process will crash multiple times during busy time of the day (but it’s only 2-3 requests/second average).

Before switching to Lenny, I have tested multiple methods to keep my PHP FastCGI processes alive, including running check/restarting script every 5 minutes, etc. However at the end even 5 minutes downtime felt a bit excessive, so I wrote a simple daemon in Python to keep an eye on Nginx’s error log, and restart PHP/FastCGI processes whenever something bad has been detected. The problem here is that the PHP/FastCGI processes actually do not die — Nginx simply cannot communicate with them. So I need to restart PHP whenever Nginx splits a dummy.

I put the code on Linode Pastebin here. Linode has appeared to remove that script from their Pastebin, but you can Download it here, modify some variables in the code (for example the email addresses, start/stop script location, etc), and then run it as root.

What it does:

• It will damonize first.
• Scanning /var/log/nginx/error.log every 2 seconds, using similar algorithm as tail -f.
• If “104: Connection reset by peer” or “111: Connection refused” is detected in the log,
  • Run killall -9 php-cgi
  • Run /etc/init.d/spawn-fcgi-php start to restart the PHP FastCGI processes

It should use minimum memory + CPU/IO (around 3.5MB RSS, and stat calls are pretty fast). At least it keeps me peace at night that I know if my PHP processes are crashing, there’s a guardian to bring them back up. It still runs on my VPS at Linode — even though with Debian 5 I am getting crashes once every 2 weeks instead of 4-5 times a day.

Hopefully it will be useful to some.

Here is the configuration code Ignor demonstrated in his email:

location /images/ {
   root                 /data/www;
   error_page           404 = /fetch$uri;
}

location /fetch {
   internal;

   proxy_pass           http://backend;
   proxy_store          on;
   proxy_store_access   user:rw  group:rw  all:r;
   proxy_temp_path      /data/temp;

   alias                /data/www;
}

Basically proxy_store or fastcgi_store saves the backend result into files on the local file system, so subsequent requests to the same URI can be handled by on-disk cache without forwarding the request to the potentially-expensive backend again. Traditionally it requires the backend being able to write to local files (for example, one of my small project does just that), however it is not always possible as they might not share the same file system.

You might still need a separate cron job to periodically invalidate the cache. However it just makes writing the backend much simpler.

Anyway. What happened was on a website I am developing, I have provided some API via Javascript, so this guy I am partnering with can just include my dynamically generated Javascript to produce content on his site. However, his site runs entirely on HTTPS but mine is not, so you get that dreadful This Page Contains Both Secure and Non-Secure Items error message in some IE versions.

I guess the easiest way for me to fix it up is actually running the site on HTTPS as well. So I went out and bought a certificate from GoDaddy ($18/year — why so much price difference?), but it wasn’t that trivial as GoDaddy does not have any installation instruction for Nginx, which my site is running under. Why not?! Consider Ngnix already has a sizable market penetration (especially if are a Russian malware distributor). Well, here are the steps.

1. Buy the SSL Certificate

I got mine from GoDaddy‘s TurboSSL Certificate for $20/year, although there are lots of coupons out there that gives you 10% off. I know you can also get very cheap SSL certs from resellers of RapidSSL. Seriously I have no idea about the pricing difference from a technical point of view. These are just your public keys signed by the certificate authority, aren’t they? But I guess its probably the verification process that makes the difference.

I am not running an ecommerce site, so the cheapest one suits me fine. Nor do I want any phone/fax verification because (1) I do not live in the US (2) I need my cert right now!

2. Download the Certificate

GoDaddy provides information on how you can use OpenSSL to generate a private key for your webserver, create a certificate request, and then paste this request onto GoDaddy’s site for verification.

After the request has been verified, a certificate will be generated for you to download. As there is no Nginx option, choose Apache 2.x. You will get a ZIP containing two files:

1. <Common Name>.crt — Your certificate.
2. gd_intermediate_bundle.crt — GoDaddy Certificate Intermediates Bundle

If you don’t see the second file, you should be able to download it from their repository.

3. Upload Them onto Webserver

You need to concatenate your certificate file and GoDaddy’s intermediate certificate to form the final certificate. I usually name my files domain.crt. Then you need to upload both your private key file (generated in step 2) and the certificate file to your server running Nginx.

For example if your domain is example.com:

# cat example.com.crt gd_intermediate_bundle.crt > /etc/nginx/example.com.crt
# cp example.com.key /etc/nginx/example.com.key

Note that concatenating the two files together is important. SSL certificate issuers are chained, and the intermediate bundle provides the “missing link” between well-known certificate authorities in your browser’s root CA list, and your website’s own certificate.

4. Turn on SSL Module in Nginx

Now you can activate your SSL site in Nginx. Use the following configuration in your nginx.conf:

server {
    listen 443;
    server_name example.com;
    ssl on;
    ssl_certificate /etc/nginx/example.com.crt;
    ssl_certificate_key  /etc/nginx/example.com.key;
    ...
}

Restarting Nginx, and your web server will now be serving encrypted content.

Since I have briefly touched on the “URL rewrite-ability” of Nginx and Lighttpd in my previous post, I think it might actually be useful to have some examples showing how rewrite rules are written on these web servers to support clean URLs. I will take the open source CMS Drupal for example, as it is what Hosting Fu runs on. Btw, Drupal 5.0 has just been released and it rocks.

Prerequisite

These are the things that I assume you would know before reading this article.

• Why clean URL. Pick your reason. SEO or general dislike of query string.
• Setting up PHP. I won’t be talking about how to set up PHP/FastCGI on Nginx or Lighttpd. Here’s one for Nginx and one for Lighttpd.
• Installing Drupal. Check related section in Drupal handbook.

I am only going to discuss the rewriting rules needed to enable clean URL in Drupal on either Nginx or Lighttpd.

Apache’s Mod_Rewrite

Like most open source PHP applications, Drupal came with a .htaccess file assuming Apache is serving the pages. We will use it as the reference on how the rewrite rules can be written for the other two web servers.

Here’s the bit in .htaccess that does rewrites:

RewriteEngine on
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ index.php?q=$1 [L,QSA]

What it does is:

• If requested file exists, serve it.
• If requested directory exists, serve it depending on how index option is configured.
• Otherwise, send all requests to index.php, setting parameter ‘q’ as the path of the original request, and then append the rest of the query string.

Simple. Now let’s see how you can do the same with the other two web servers.

Lighttpd

This site runs on Lighttpd 1.4.13, and there seems to be a lot of different ways to do clean URL with Drupal on Lighttpd. Although it came with its own “mod_rewrite”, but I found the functionality is very limited. The biggest problem I found is the lack of conditional rules to check whether a file or directory already exists. At the end you have to either (1) make lots of exceptions in Lighttpd’s rewrite rules, or (2) modify Drupal so it works better with Lighttpd.

I went with the second option.

Well, here are the steps.

1. After Drupal has been installed and tested with clean URL disabled, add the following rules to lighty’s configuration file:

   server.error-handler-404 = "/index.php";
   

   It basically makes index.php the 404 error handler, so any request that does is not handled by a local file or directory will be sent to Drupal.

2. Add the following PHP code to Drupal. I just append them to the end of sites/default/settings.php.

   if (strpos($_SERVER['SERVER_SOFTWARE'], 'lighttpd') !== false) {
       $_lighty_url = $base_url.$_SERVER['REQUEST_URI'];
       $_lighty_url = @parse_url($_lighty_url);
   
       if ($_lighty_url['path'] != '/index.php' && $_lighty_url['path'] != '/') {
           $_SERVER['QUERY_STRING'] = $_lighty_url['query'];
           parse_str($_lighty_url['query'], $_lighty_query);
           foreach ($_lighty_query as $key => $val)
               $_GET[$key] = $_REQUEST[$key] = $val;
           $_GET['q'] = $_REQUEST['q'] = substr($_lighty_url['path'], 1);
       }
   }
   

   Let me explain what it does:

   • If we are behind lighty, turn on this hack (I use Nginx for my development box so this code does not apply).
   • Try to parse the REQUEST_URI. If invoked as 404 error handler, we will parse the QUERY_STRING ourselves and copy the values to PHP’s $_GET and $_REQUEST variables.
   • Also set the path bit of REQUEST_URI as query argument q.

   The reason why we have to parse QUERY_STRING is, Lighttpd deliberately does not set QUERY_STRING if FastCGI is invoked as 404 error handler.

3. Restart lighty, go to Drupal to enable clean URL and see whether it works.

Well, it has been working fine for me, but YMMV.

Nginx

Nginx comes with conditional code for rewrite rules so it is much easier. I basically have the following code in my Nginx configuration file to emulate Apache’s behaviour.

if (!-e $request_filename) {
    rewrite ^/(.*)$ /index.php?q=$1 last;
}

That’s it! Restarting Nginx, and you can now turn on clean URL in Drupal.

However, Nginx is not perfect, and so far I have found one small issue with its rewrite engine. When you use regular expression in Nginx’s rewrite rules, it will try to encode the matches in the replacement URL. So far I have seen it broke Drupal’s search module. For example, if you search for “Hosting Fu”, Drupal will use the following URL:

GET /search/node/Hosting+Fu HTTP/1.1

However, Nginx will rewrite that to:

GET /index.php?q=search/node/Hosting%2BFu HTTP/1.1

Note it encoded ‘+’ to ‘%2B’, which confuses Drupal, who thinks that you are actually searching the phrase ‘Hosting+Fu’. In case of Apache, ‘+’ passed through rewrite rules untouched.

Conclusion

Many open source PHP applications that I have experience with always assume the existence of Apache, and provide clean URL to only Apache users. On the other hand, developers of other frameworks like Ruby on Rails, Django, Webpy, etc take clean URL for granted because it is something handled right inside the framework. It makes the life of the web server guy much easier — what rewrite? Just proxy or pass through the whole damn thing!

I am hoping more and more PHP applications will use simplified rewrite rules. Let applications themselves take care of parsing the REQUEST_URI, instead of generating a million lines of Apache mod_rewrite rules and dumping them into .htaccess files.

Meanwhile, Nginx users will have much easier time porting those rules then the Lighttpd users.

I will be writing more about Lighttpd and Nginx later during the year, but will try to use this post to draw some comparison between Nginx, the new darling of these light-weight web servers, and Lighttpd, many Web 2.0 developers’ all time favourite.

Lighttpd

I have been running Lighttpd (pronounced “lighty”) on my home servers and development boxes since the beginning of 2006. It is a great replacement for Apache if you have the whole box to yourself, i.e. you don’t need to worry about supporting .htaccess files that your users might use. Currently this website is hosted on lighttpd-1.4.13 on a Gentoo VPS.

Pros

• Light weight. Clean restart of 1.4.13 takes no more than 2Mb RSS on this 64bit VPS. It binds the port, drops the privilege and that’s it! A single process does all the tricks even when you have hundreds of concurrent connections. No more pre-fork MPM with mis-configured MaxClient that sends you to swap hell.
• Speed. Very fast static file serving. Very fast FastCGI serving. Very fast proxy serving.
• Modules, and lots of them. Good comprehensive documentation as well. It even has SCGI for your Quixote apps.
• Mod_magnet. Wanna a scripting engine right inside your web server? Mod_magnet integrates Lua into lighttpd, so your World of Warcraft scripting skillz can be put into better use.
• Community. It has got a Blog, a Wiki/bug tracker and a forum. It is easy to find help when you need one.

Cons

• Stability (or lack of according to the RoR folks). I had quite a lot of issues using Lighttpd as proxy+HTTPS front-end for our Python stuff, but the same app runs fine with just lighttpd + proxy without HTTPS.
• Mod_rewrite (or again, lack of it). Built-in rewriting engine sucks, and porting Apache mod_rewrite rules over can be non-trivial sometimes. Update: Here’s an article I have written on Drupal clean URL on Nginx and Lighttpd, which looks at the URL rewrite options of these two web servers.
• Memory leak. The RSS of my lighty process grows by about 1.5Mb per day, but then I don’t have lots of traffic (less than 50k requests a day). At the end I just need to restart it once a week. Many people have far worse memory leaking issues I heard.

Nginx

I have been running Nginx (pronounced “engine X”) on my development box and two of my VPS’s since December 2006. It is Russian, fast and very configurable. I am currently using 0.5.5 for my sites, but don’t be deceived by its version number — it is very stable.

Pros

• Light weight. It is not as light weight as lighttpd when it clean-starts. At least two processes are needed — one master process running as root that binds to the port, and one or more worker processes that handle the actual requests. Around 7Mb RSS together on my 64bit VPS (and only 4.5Mb on 32bit VPS). Still beats Apache hands down.
• Fast. Some benchmarks have shown that Nginx has a slight edge over Lighttpd, but so far I haven’t been able to notice any. Again, much faster than Apache over static file serving or proxying, especially when you turn up the value of keep alive (more than 1 minute for example).
• Modules. There are many modules available on Nginx. Some very useful, and some are just plain weird. While lighttpd has Lua embedded, you can now also embed the whole Perl interpretor inside Nginx.
• Better Rewrite Module. A much better rewrite module than Lighttpd that supports complex conditions. Porting mod_rewrite rules from Apache is actually now feasible without touching the apps themselves.
• Stable and not leaking. Been running Nginx on a production site doing PHP-FastCGI, and have no issue what so ever.

Cons

• Lack of community. Where can I find help regarding Nginx? There’s only IRC as far as I know. And while the lead developer writes beautiful code, all documentation were initially in Russian which was a big stumbling block before the English docs came along.
• No CGI support. Oh well, maybe I am the only one who still hacks small CGI scripts. Apparently Nginx does not spawn CGI or FastCGI processes, which means you need to either (1) convert it into external-spawn FastCGI, or (2) proxy to another web server that does CGI.
• No simple virtual host support. Lighttpd has mod_simple_vhost and mod_evhost to let you quickly deploy lots of name-based virtual hosts. You can somehow do the same with using $server_name in root and a wild-card in server_name, but it’s still not as clean as lighttpd. At the end you will find Nginx configuration files much more verbose if you run lots of small sites off a single web server.
• No X-sendfile support. I found Lighttpd’s X-sendfile support very useful when my scripts need to send back large files, and was disappointed to find out that Nginx does not have it. X-Accel-Redirect is different as it requires extra configuration on web server, which makes your web-app less portable.

Conclusion

I don’t think I am a suitable judge to say which one is better, as (1) I have only been running Nginx for a month, and (2) my level of traffic does not really stress test these high-performing web servers. At the moment I think I like Nginx better purely because it does not leak, and its rewrite module that enables me to run many off-the-shelf open source PHP apps with clean URL.

Again, I might change my mind in 3 months time when I find out more warts about Nginx. We will see.