Judd Vinet of ArchLinux (one of my favourite, btw) has recently written an article to solve this very issue, Web clustering with Amazon EC2, where extra servers are hired from Amazon EC2 on part-time basis to serve surge in traffic. A semi-automated system has been built to make the task of “summoning new servers” much easier, and has been discussed in the article.

It’s quite a detailed write up with a lot of good information. Basically it is what Judd is trying to achieve.

• 2 existing web server cluster using LVS. Use Pound for web balancer.
• Start EC2 instances, and upon start up each instance will phone home to register itself.
• Master server (where Pound is) will add new EC2 instances, and grant database permission to allow connection from EC2.

Seems to be quite a solid approach to scale up your web site instantly. Fire up a few EC2 instances before expected traffic surge, and shot them down after the storm — you are only paying for the time and bandwidth you have used!

At the end of the article Judd mentioned that a fully-automated system will be their next step, so they don’t need to manually activate/deactivate EC2 instances — they should just start/stop whenever the traffic builds up/slows down. Nice for unexpected traffic like Slashdotting or Digging.

Who said the dedicated server vendors shouldn’t be worried?

However, a few problems are also encountered during this exercise — web traffic load balancing and database access. Pound has been used in the example but I think other “web fronts” like Nginx should have no problem either. I have never used Pound but from the example given, a 2 second delay is needed to cleanly stop the old process before new one can be started. I guess it would be unnecessary if Nginx or Lighttpd is used, where you can gracefully reload the server without causing any delay.

However the biggest problem I have with the proxy approach is — which is actually an issue with EC2 itself — that you are effectively paying double the data transfer. You are paying Amazon for data transfer coming in or going out from EC2 at $0.20/Gb. At the same time you are paying whoever is hosting your reverse-proxy servers for traffic going out to your customers. Problem wouldn’t exist if you can run your reverse proxy right inside EC2, as traffic between EC2 instances are free. Except it is not trivial. Not until persistency and static IP are implemented anyway.

Database access over public Internet is also not something trivial to get it right. For a busy website, even 20ms latency between the web server and DB server can cause quite a noticeable degrading in performance. A locally replicated DB will definitely be much better. But somehow “part-time” “on-demand” “MySQL replication” all sounds oxymoron to me.

Conclusion? I guess that’s why EC2 is still in beta, and is still pretty much “developers only”. But no doubt it is full of potential.

It is basically a cluster of web servers/application servers connecting to a fast array of storage (SAN?), and load balancers or routers placed in front of the web servers to evenly distribute the requests across individual physical servers in the grid. A new measurement, Grid Performance Unit, is introduced to measure how much CPU time your account has used.

Michael Arrington has also interviewed the MediaTemple guys on TalkCrunch, talking about the ideas behind their latest offerings.

It is definitely an interesting product that is targeted to solve scalability issue of many growing web sites that has exceeded the capacity of the single-box shared hosting, and it comes at a very attractive price point — $20 a month for 1,000 GPU, 100Gb “premium” storage and 1Tb monthly data transfer. It also has one of the most flexible PHP configuration you can have with shared hosting — choice between PHP4 or PHP5, no stupid safe-mode that limits you in every possible way, and options to use your own php.ini.

Another innovation is their Ruby on Rails Containers technology. Not to be confused with Solaris Containers, MediaTemple’s RoR containers look like a jailed execution environment for your Ruby on Rails applications. Memory is limited to 64Mb, and you can run multiple rails apps. There are also options for you to purchase more memory for your container. Sounds like a light-weight VPS optimised for RoR applications, without the overhead of management.

Here are some of my random thoughts.

Cluster Hosting is Not New

For one thing, MediaTemple is definitely not the industrial first. Cluster hosting has been around for a while. TechCrunch mentioned Rackspace’s Mosso. Even in the land of Down Under, Ilisys has been offering cluster web hosting for years.

What sets MediaTemple apart? Buzz. Being on TechCrunch and interviewed by Michael Arrington certainly helps.

Very Competitive Priced

Another thing that differentiates MediaTemple is their pricing model. I have no idea on how much processing power is 1,000 GPU, but from MediaTemple’s description that it will be sufficient for most of their customers. That means $20/month for up to 100 sites on clouds of servers with huge storage and bandwidth — very attractive. Certainly compares very well with Mosso’s $100/month with 80Gb storage and 2Tb data transfer.

Both as potential replacement for low-end dedicated servers, however, I think Mosso is more geared towards web hosting resellers. Mosso comes with add-on client services that includes billing and support. And with an infrastructure that can utilise both Linux and Windows application platforms, Mosso is best for those starting out reselling web hosting to be able to host both Linux and Windows applications. Whereas Grid-Server is Linux-only with no reselling options.

Still, it’s only $20 a month for all your sites. I am looking forward to other cluster hosting services to catch up and drop their prices.

Clusters are Good for Customers

There are many reasons why clustered shared hosting is way better than single-box model that most hosting companies are running today. These are not specific to MediaTemple’s Grid-Server.

• Centralised storage on fast SAN. Because all cluster nodes have access to the same pool of data, hosting companies need to deploy those high-end storage solutions that are faster and more reliable. At the end the customers benefit.
• Higher uptime guarantee. Server hardware failure no longer means hour-long down time, and from DreamHost status you’ll learn that hardware do fail, and quite often too!
• Scalable performance. What happen if you site got slashdotted or dugg that requires the horse power of more than one physical servers for 12 hours? A cluster will automatically load balance the requests to other servers. Or if your neighbour gets traffic spike, you don’t get to suffer with him as other cluster nodes will take care the traffic.

I think the day of single-box shared hosting, aka the cpanel model, is numbered.

Clusters are Good for Hosting Providers

While developing this technology might be costly, I think the benefit should easily pay off the cost in the long run, as clustering is just so obviously the better solution.

• Overselling CPU time. Already overselling storage and bandwidth? Why not CPU time as well? I think clustering actually makes it easier, according to the law of large numbers. It is much easier to oversell from a big pool of clustered CPUs than separate units of individual boxes — just like you can oversell from your pool of gigabit bandwidth and network attached terabyte storage. End result? More customers on less hardware, which leads to more profit.
• Easier traffic spike management. If you are the host you’ll hate clients getting slashdotted. You have to find spare servers and migrate the accounts around. No longer the case with a clustered backend. Again, law of large numbers works in the advantage of the hosting providers here.
• Easier server management. All the servers in the clusters can be provisioned and managed with the same configuration. No account specific info needs to be stored on these servers. You can even buy cheap hardware for your front-end computational boxes, aka the Google data centre style, as long as your storage is high quality and scalable.

I am sure there are others. At the end, all these leads to happier customers, which lead to longer stays, more referrals and more profit.

Smart move MediaTemple! Have I said that the day of single-box shared hosting is numbered?

Grid Server is Not Competing Against Amazon EC2

Even the MediaTemple guys said so, that (gs) and EC2 are very different. One is fully managed shared hosting platform, and the other is unmanaged Xen virtual dedicated server. They serve different purposes, even though many people have confused the two.

I’ll even take another step saying MediaTemple’s Grid-Server product is not replacing unmanaged VPS either. It does not give you root access, does not let you run things like VPN tunnels, Jabber servers, VoIP software, etc. I’ll say a VPS is still a better development platform, whereas (gs) is a great deployment platform for 95% the case.

Grid Server is Competing Against Managed VPS/Low-End Dedicated

Besides traditional shared hosting, I think managed VPS/low-end dedicated is where (gs) will steal most of the customers from. Many people buy low-end managed solutions because:

• They have out grown the managed shared hosting account.
• They can’t manage the servers themselves.

A scalable managed shared hosting in cluster servers is a much better solutions for those customers. It will be cheaper, more scalable, and much better looked-after than a managed VPS.

I am hoping that (gs) will also attract those who are supposed to get managed VPS, but were too cheap and went for unmanaged solution. After all, running a Linux server, having it all patched up, setting up custom applications on it, etc — these are not for everyone. A mis-configured unmanaged server is actually more harmful on the Internet, if it has been exploited.

If you don’t have the time to keep the software up to date, not having ability to, or simply don’t want to, please go for a managed service. If a standard LAMP stack fits the bill, then MediaTemple’s Grid-Server might actually be a better choice.

Software Compatibility Might Be An Issue

If you have been in software development for clustered applications, you know scaling the system up is much more than just adding more parallel hardware into the grid. As HTTP itself is stateless, you need to somehow keep the system states between individual requests (data, session, etc). Some software packages might not expect different requests handled by different hardware nodes. Anything related to persistency of states warrants a second look.

PHP, due to its share nothing architecture, makes it easier to scale, provided you can also scale the backend database. Does (gs) give you clustered MySQL/PostgreSQL servers with replication? Not from what I can see. The database itself might become the bottleneck when the site tries to scale up.

MediaTemple kept a list of applications known to be grid compatible. However, take WordPress or Drupal for example, can you be 100% sure that every single module or plugin is compatible? I’ll still suggest test, test and more test.

Localised optimisation is also something difficult to do with clusters where adjacent requests can be served by completely different nodes. For example, your PHP intermediate code cache (like APC) might get a lot of cache misses, if requests to your sites are handled all over the places, and caches are trashed by requests from other sites.

Conclusion

Clustering is nothing new, but MediaTemple being today’s web hosting buzz, hopefully more people will aware the benefit of grid computing. At this price point I think it would be great for those who want to have good scalable and reliable website.

As web hosting has become a commodity, it is great to see MediaTemple, being a low-budget shared hosting company, introduced something that sets them apart from the rest. Hopefully we will see the more clustered shared hosting at an affordable price, when this market starts gaining momentum.

Update: Tim Dorr of A Small Orange reported some interesting find on (gs) service, including:

• A site with 10% discount.
• Load balancing server structure.
• MySQL servers layout.
• Foundry Networks ServerIron is used for load balacning/app switching.

Update 2: Ben Rockwood from Joyent blogged about Grid-Server internals, after bought an account and tried to figure it all out himself.

As Google and Microsoft continue to invest in their Internet infrastructure, might they ever reach an economy of scale where it’d make sense to “lower the barrier of entry” by giving away not just APIs, but hosting resources on which third party developers could build and run complex, high traffic mashups?

Imagine this — instead of renting a $99/month Intel/AMD dedicated box, you rent CPU slices in Google’s 400,000+ cluster/server-farm, where it provides high availability (when was the last time you can’t access google.com?), great parallel computing technology and almost limitless bandwidth. Whether it runs single-threaded PHP script for your WordPress blog is another thing…

Well, I will not say that a big clustering account would replace the dedicated server market today, as there are many reasons people opt for dedicated servers. It is like reseller account verses virtual private server — those who picked reseller accounts for their bigger storage, more bandwidth, more CPU resource and an easy to use control panel so they can just focus on content creation. VPS customers (or when they graduated, dedicated server or co-location customers) want their root (or Administrator) access, flexibility of software installation, custom configuration etc.

Mosso is an interesting hosting product. 80Gb storage + 2Tb bandwidth. Moreover, it sits behind inside farm of application servers (PHP, ASP, JSP, MySQL, etc) behind a cluster of load balancer. 100% SLA on network and site availability. Redundant everything. You don’t know who serves your web page, just that it would be served by the cluster member with least load and fastest response time. Even if your neighbour gets slashdotted or digged, it is not going to affect your sites’ performance much. Great infrastructure for $100 a month.

It’s good for the host as well. Overselling CPU time is now closer to reality. I imagine the CPU utilisation would be much better distributed, if you have 500 websites over a 10 node cluster, than 500 websites over 10 dedicated servers.

Maybe in 3 years, all major shared hosting companies will operate this way. We’ll see.

However, it is also known that not all web applications work in cluster. What if the PHP app uses a different way to track sessions and cannot afford to have different requests served by different cluster node? How many support load balanced MySQL servers? However, when there’s demand, we’ll also see more web-based apps designed for this type of environment.

Also looking forward to have a cluster-ready Linux distribution for web hosting companies. Boot straight up from SAN, configured itself from the master nodes of the cluster, and read to serve requests. Hmmm.