Daughter: “Daddy, why don’t we have one toilet for daddy, one toilet for mummy, one toilet for baby (my other daughter who is 10 month old) and another toilet for me?!”

Well. Because that will be excessive! We only have one toilet in a semi-detached house with 3 bedrooms (actually there is another toilet “at the back” that needs a bit of “maintenance” but we won’t go into all the smelly details here). However the chances that everyone needs to use the toilet at the same time is pretty rare. Even if such rare chance arises, just remember that patience is a virtue — just wait and it will soon be your turn (actually somehow I always found myself at the lowest of priority order at home). Build an exclusive toilet for every single person in the household that might have a need to use it is simply not economical!

I guess everyone over 3 year old should understand why don’t we have a dedicated toilet bowl for everyone living in the house. But apparently not everyone agrees that web hosting companies don’t need to have dedicated disk space and traffic allocation for every customer on the box.

(Sorry about the toilet analogy. I promise I won’t use this kind of illustration again on HostingFu, finger crossed :)

How do all us shared web hosting companies sell more disk space and bandwidth for ten bucks then the dedicated server providers sell for hundreds?

Its an easy concept really. Every web host has a terms of service with CPU and memory limits. If your website consumed too much of its share of CPU or memory then most web hosts will require you to upgrade. When you purchase a dedicated server you cant get shut down for CPU or memory abuse so they have to sell you a plan based on what your site could use with less restrictions.

Together with responses in this WHT discussion thread, it is great to see Brent/HostGator being very honest about it. This comment sums up well:

To sum it up…

Cause downtime = in trouble
Not causing downtime = eat your heart out

That is pretty much the policy with all the mega shared hosting oversellers with hundreds or thousands of servers and almost unlimited disk space and bandwidth at their disposal. Disk space and bandwidth are no longer the metrics for web hosting plans. Use as much as you can, except when your dynamic websites started to use too much CPU/disk IO resources.

So, why don’t we stop using data transfer and storage for hosting plans, but measurable CPU units? It is all about marketing — after all to grow in astronomical speed in shared hosting business, you need to attract clients to fork out a lump sum for yearly plans so you get enough cashflow to build infrastructure ahead of time. Easiest way to do it? Promise them the unlimited (guarded by strict ToS) for a price cheaper than your competitors (if you pre-pay for 2-3 years).

I have seen HostGator mentioned it somewhere (damn, can’t find my reference) that they have observed sign up slowed down when their competitors increased their plan offerings. A counter-offering seems to be the best weapon here. They have also added 2/3 year hosting plans with reduced equivalent monthly price so they can look good on hosting review affiliation sites.

Yes. It is all marketing.

Overselling: A term used by numerous WHT members as an apparent attempt to ‘bring down the big bad hosting companies’. Often used in a derogatory manner without substantiated foundation, i.e. actual previous experience with said host.

The domain was registered back in April, and currently hosted on DreamHost, one of a few web hosting companies that actually make overselling work.

Overselling is indeed a very touching subject in web hosting. I previously wrote against it earlier this year when I first discovered this terminology. Then I read DreamHost and Site5‘s take on this issue, and was convinced that overselling is a “business strategy”, rather than a fraud or trickery.

However, with popularity of today’s dynamic database-driven web-based applications in hosted shared servers, “overselling” is no long as simple as “having less bandwidth/disk space than the sum of all users’ allocation”. I will try to take another shot on it in this article.

What is overselling?

First of all, can we define what is overselling? This is what Wikipedia has to say.

Overselling is a term used in web hosting to describe a situation in which a company provides hosting plans that are unsustainable if every of its customers uses the full extent of them. The term is usually referred to the usage of webspace and bandwidth.

I shall take Dan’s example (which, by the way, is excellent and has very good analysis on the pros & cons of overselling). If you are a web host with 80Gb hard disk, and you are selling more than 80x 1Gb storage web hosting accounts, then you are overselling! However, Dan’s definition differs slightly. He said in his opening paragraph:

Overselling basically means to sell beyond the means of delivery.

It seems that “overselling” is actually not just about the actual numbers, but the ability to deliver. Let’s take the following two cases:

1. If a host promises almost-unlimited space and data transfer, but has always delivered what its clients have requested, is it still overselling?
2. If a host does not “oversell” on paper, but still becomes unsustainable even before its customers uses the full extent of advertised resources, is it overselling?

Many web hosting customers are actually happy about (1) — as long as it delivers what I need, I do not really care. After all it is the very benefit of overselling — better utilisation on the resources so “some” customers can get more bang for the buck. Not everyone on your servers will get digged/slashdotted/farked at the same time. Not everyone is running a YouTube clone. But companies have advertised as though they can — to attract all those YouTube wannabes and those who love to have big buffers.

It is a fine business strategy by all means. Tried and tested by all types of service industries over the years, and they worked great — as long as resources are properly managed. A bit of statistics, planning and a big credit helps.

However, there is another case, where a web hosting service can become unsustainable even if it does not oversell advertised resources. How can that be the case?! Because running a server or farm of them is a complicated task, and the limiting resource can be much more than the usual advertised “disk space” and “monthly data transfer”.

What are the constraining resources?

Web hosting firms usually sell plans defined by a combination of disk space and monthly data transfer (or bandwidth, if you are on a unmetered dedicated server/VPS). While they can be easily understood by the customers, they do not represent the true picture of requirement to run a dynamic web site. There are many more types of resources that a web host needs to consider, when “server-side scripts” were introduced. Some of the hosts put a soft limit on these resources, but many seldom publicise about it. Thus it is quite possible to exhaust certain resources without exceeding advertised disk space/monthly transfer, even when they are small reasonable amount.

I shall give some examples. My cases would be a bit extreme, but they are there just to prove the point.

CPU time

CPU time on the server is by far the most obvious example of measurable resource that some hosts have actually started to advertise them and charge excess (MediaTemple Grid Server‘s GPU for example). Many content management systems without careful caching are bounded on CPU or IO, and you can run many “busy” CMS sites that bring a server to its knee without consuming too much bandwidth and/or disk space.

Or take this example — a web-based distributed.net client, written in PHP just to show off my 1337-ness! Not much data transfer is needed (input=key range + cipher text, output=hit or miss). Not much disk space is needed either (a 5k PHP file). However I can be sure that I will be the most unwelcoming customer if I run it 24×7 on a shared host.

Memory / RAM

Physical memory on a hosted server is also a finite resource shared by all accounts on that server. Servers using excessive amount of memory usually swap heavily, which degrades the overall performance (or entering into “spiral of death”). Hosting companies usually limit the amount of memory each single process by setting PHP’s memory_limit option or ulimit -v to prevent run-away CGI process.

But how often do you see web hosting companies explicitly state the maximum virtual memory size for a single FastCGI application? What about the maximum memory for all processes? Across the cluster? It gets more and more difficult to measure, monitor and enforce the policy.

Disk IO

Majority of database applications are disk IO-bound, and disk access can often be the biggest bottleneck on a shared hosting environment — your disk cache can be easily trashed when it tries to serve hundreds of independent websites. Try joining a few medium size SQL tables on non-indexed columns. Do it a few times. Now imagine half the people on your shared server doing the same thing again and again — you can easily stall a server without exhausting your storage and bandwidth, while your CPU is only mildly loaded.

However disk IO utilisation is difficult to quantify on shared hosting. Have you seen any web hosting company putting “thou shalt not do more than 50 block IO/sec” into their terms of services?

There are many others. Number of files (on some file systems large number of files can degrade performance), number of concurrent TCP connections (large number of slow connections -> many Apache processes -> bad), number of opened file descriptors, etc — they are all finite resources on the server. System integrators and developers would track them, monitor them and optimise their applications for them, but how often do you see hosting companies publicly advertise their limit on the usage of these resources? That would be insane. But as it is in their best interests to keep the servers running smoothly, they have to police the shared hosting accounts to prevent them hogging CPU/RAM/IO/etc.

And they don’t need to oversell their bandwidth and storage for the policing to happen. A host might still fail to deliver what it has promised because it oversold its CPU time, as “amount of CPU time” is never spelt out in the plan specification. Or maybe it is the amount of disk IO. Or the number of inodes.

So, is there any non-overselling shared hosting?

If “non-overselling” is strictly defined to mean disk storage and bandwidth, then the answer is probably yes. There are companies who claimed to be non-overselling to be distinctive. Some of them are doing it to limit the number of accounts on the server to reduce the chance where other resources might be constraining.

However,

1. Is there a way to verify the claim? How do you know that they are in fact having only 100 accounts, instead of 500 accounts on the same server?
2. Do you have an overselling upstream provider? How do you know that the bandwidth to the dedicated server hasn’t been oversold? Or the co-location service? Or the carrier bandwidth?
3. Are they overselling the CPU, IO, and other server resources?

Another alternative to “non-overselling hosts” is “pay-only-what-you-have-used” hosts. For example, Amazon S3 which charges you according to the exact amount of disk space and data transfer you have used. NearlyFreeSpeech.net is another one that I am aware of who charges you on the amount you have used.

The biggest issue still remains. What about the resources that are not advertised? Can a non-overselling host guarantee you to use all the storage space + disk transfer, regardless how much CPU time and MySQL connection your apps chew?

Should companies advertise all possible limits?

Maybe they should, just to protect themselves from being accused as fraudsters and liars on WHT. However I don’t think it will ever happen in reality. Those numbers don’t mean anything to a regular web hosting customer, and everyone hates complexity. We want simplicity! Which one would you sign up to? The host with 10 numbers that you should not cross, or the host with two? Or better, the one with none, before it got shot dead with flaming arrows labelling “die! unlimited bandwidth!”

Okay, maybe I should wrap up

The more I dig into it, the more I realise the impossibility to not oversell in a shared hosting environment. Shared web hosting is in fact a tricky business, and keeping servers running smoothly requires both proactive (minimise overselling, plan ahead, clustering etc) and reactive (reduce bottleneck, fast provisioning to scale up, etc) approaches.

Life of being a developer that just writes the software is so much simpler :)

If I am signing up a shared hosting account today, these are the things I will seek after regarding overselling.

1. $10/month for 50Gb or 2Tb does not bother me. I do not have enough popular sites to use 50Gb a month anyway, so a plan with smaller allocation might be more suitable. Then again, there is no guarantee they haven’t oversold their other resources.
2. Check the size of the hosting company. 2Tb/month offered by a 100+ servers company is actually different from 2Tb/month offered by a 2-server one-man shop. Overselling actually works with larger numbers.
3. Ask about their policy on other resources. I prefer if the company has set limits on resources like CPU time, MySQL connections and number of files. Better if those limits are publicly visible.
4. Search around whether someone’s account has been suspended from that host, and for what reason. I actually prefer hosts that suspend their customers and can justify with numbers. They are the one taking reactive approach on server loads.

Well, that’s enough for today. More later :)

Automattic’s Matt Mullenweg responded to Joyent’s point on grid, showing how big a package they can get from LayerTech with under $2.5k per month. It’s indeed quite an impressive list — 15 boxes sharing 27Gb of RAM, 500Gb of RAID’ed storage and 18Tb of public data transfer. At the end he asked:

What do you guys think? Anyone want to work out what the same would cost on Amazon? Which would you go with?

Let’s take out the calculator.

Amazon Web Services is Expensive!

We will just look at the bandwidth cost alone. How much does it cost to pump out 18 terabytes of data from Amazon (EC2 or S3)? At the current rate of 20cents per Gb, 18Tb is going to cost you $3,600! It is already more expensive than LayeredTech’s offering, and we are not yet counting storage and computation power yet!

Sure, your 500Gb file server would only cost $75/month on S3. Running 26 instances of EC2 24×7 would cost around $1,900 per month (26 because Matt’s setup has 12 dual core + 2 single core servers). 45Gb of RAM total, but it still falls short of total amount of “CPU power”.

But the bandwidth cost?!

Amazon: You (only) pay what you have used

I guess the main difference is, most web hosting companies oversell. Most LayeredTech dedicated servers come with 1.5Tb of monthly data transfer, but I am sure many servers actually don’t use anywhere near that.

When Linden praised Amazon S3 for saving the Second Life download, my initial reaction was — “hey that’s crazy”. 1Tb of data transfer? $200 on S3, where as my $100 LayeredTech dedicated server would be sufficient! Then I realised, someone else is actually subsidizing me for using all my allocated data transfer.

S3′s model is also more flexible when the usage pattern is less than regular.

For example, the following month traffic dropped to 500Gb. With a dedicated server however, it will still be $100 (with CPU mostly ideal as file servers are IO bound). With S3, you only pay what you have actually used — $50.

So is it expensive?

Back to Matt’s question — I think AWS can be more expensive when you try to spec out something similar to your average dedicated server offerings. However I do not think Amazon is pushing AWS as a “cheaper dedicated server replacement”.

Instead, AWS is making CPU power, storage, bandwidth, etc a “commodity”. They are “cheap enough”, yet easy and fast to provision, deploy and use. For example, you might be capable to build an economical power generator in your own backyard, it is usually more convenient to just connect to the electricity grid — it will make your life much easier, and much more capable to scale up and down with your demand.

Update

Err. Fixed a few spelling mistakes as I initially wrote this post way too late at night yesterday.

Also with reflection on what Isabel has said, AWS scales great both ways. Not only that it will cost you less during the “not-so-busy” season, it also gives you flexible bandwidth without the need to deploy additional servers during busy seasons. Managing these servers take time, and time is expensive.

At the end, a fleet of 15 servers might cost $2,500/month. A good sysadmin costs much more than that.

So “guess” what?! We’ve changed our “cpu minute” “policy” “for the better!” We no longer HAVE any limits on “cpu minutes”. Maybe it’s just semantics, and maybe it’s just “crazy overselling”, but as long as your site or scripts aren’t causing problems with the server, you are IN THE CLEAR!

Let’s say you’re not IN THE CLEAR though.. don’t worry! We’ll work with you! We’re adding a BUNCH of new servers to help “get” the average load per web server down, and we’ll work with high-load people to get their usage down or their butts onto a new server that can handle it. What a sweet web host.. let no one accuse US of “crazy overselling,” and we won’t accuse THEM.

Please ignore Josh Jones’ sarcastic tone :)

What does that mean to DreamHost users?

• If no one really visits your site, or you have not discovered the world of “dynamic content”, this announcement means pretty much nothing to you. You are no where near DreamHost’s abuser radar. Take easy. Build some traffic.

• If your daily CPU usage is approach the old 60 minute limit (for example, 3,000-4,000 visitors a day on a typical WordPress installation), then this is a good news. DreamHost will no longer automatically trigger an email to inform you that you have exceeded your limit.

  However, it is still a good idea to optimise your site. Consider wP-Cache for your blog, or don’t use WordPress at all. Just in case a sudden traffic surge pushes you into the next level…

• If you occasionally gets DDoS’ed, Digged or Slashdotted, then DreamHost might actually actively assist you to optimise your site. They also reserve the right to temporarily disable your site, or move you onto a semi-dedicated server until the effect dies away. From their updated CPU minute FAQ,

  Once your site starts to adversely affect the shared hosting server you’re on though, we may have to take action. If your site is just causing the load to creep up on your machine, we’ll try and contact you and give some suggestions on how to make your site less resource-intensive. If your site (or just a particular script) is causing the entire server to become unresponsive, we may have to take immediate action and temporarily disable whatever was bringing the machine down. Hopefully the problem was just a one-time thing that can be fixed! If it turns out to be an ongoing issue, we may need to move you to a different server eventually.

  That sounds fair. Instead of getting dreaded account suspended email straight away, at least they are working with their customers to provide a temporary solution.

  Now, how to get onto Digg or Slashdot is another matter though…

• If your site is constantly getting hammered with high CPU usage, then DreamHost might want to negotiate a deal with you to have you upgraded to a higher plan (Crazy Insane CPU for $30 a month?) or even suggest you to move onto a dedicated box. As DreamHost does not provide dedicated at the moment, we can only speculate that it means “please find a new host because you are too costly”.

  From the same FAQ it states:

  If, even after moving you to a new server, your site is still causing problems, we may try and work out some sort of agreement where you get a close-to-empty server all to yourself in exchange for upgrading to a higher plan. But again, this is all on a case-by-case basis!

  Yes — it will be on a case by case basis because we can imagine those who use that much CPU time are minorities.

So, after all, it is not an “unlimited CPU time” announcement. After all, those dual Xeons still have the same 1,440 minutes everyday. DreamHost is here to tell us that the old “hard limit” has gone soft, and they are willing to work with their customers to provide the best solution. If your web site runs silly scripts that hog CPU and cause other sites on the same shared server to perform badly, then there still is a chance that they will force you to leave. Rare, but possible.

Conclusion

Would I change what I have written about the dark side of DreamHost? No. It still holds true, and it holds true for all other shared hosting as well. There is no way to use up all the storage and bandwidth if you are running heavy CMS scripts. You are hogging the resource that was supposed to be shared. Not kicking you out is not playing fair with many other customers on the same box, and DreamHost here is being generous by moving offending site to a less crowded server.

And I have to state it again. When I wrote my previous article, I did not write it with an intention saying “DreamHost sucks”. I have never received CPU minute letter myself. I wrote it to warn those potential budget hosting customers — don’t think the host is going to satisfy your needs because it provides sufficient storage and bandwidth, because you also need to watch out how CPU intensive your website is. DreamHost wasn’t actively stating out this limitation previously in the sign up process, but although the hard limit has now been abolished, web hosting customers still need to understand that soft-limit still exists — they are getting a shared hosting account, not a dedicated box.

I have to applause DreamHost for having such open discussion on their blog site. It is not say, “hey now you can use whatever you want”, but helping their customers to realise why such limit exists and what is their approach to offenders.

Is CPU overselling possible? With virtualization, SAN and real-time server migration, it might become a reality to hosting companies. Maybe I’ll discuss it next time.

Update: Someone in WHT received a warning on CPU resource over-usage, and it is after DreamHost’s announcement of abolishing the CPU minutes. Oh well.

Conclusion? To quote Duncan’s words,

Shared hosting is nearly always a scam, no matter who you host with, because it’s marketed with big figures that are really irrelevant in the delivery of your content, particularly from a blog, to the marketplace.

Duncan then hammered Site5 for their incompetency in resolving his resource usage issue. And later on he suggested

At the end of the day, you do get what you pay for, and if you intend of maintaining or hosting more than a blog or two, don’t use shared hosting.

Yes indeed, “you do get what you pay for”. It is true in our everyday life, and web hosting world is no exception. However it is definitely not what most people are thinking when they sign up a shared hosting account (at least not mine when I had my first). We normally just check (1) price — lower the better (2) storage and bandwidth — more the better, and pretty much nothing else. Many people got stung by bad hosts. Fortunately I haven’t had such experience (probably due to I have never had a popular site).

However sometimes it is also unrealistic to expect a budget share hosting to do everything for your site. They probably have hundreds of $5/month accounts on that dual Xeon, where your badly designed scripts is taking 50% of all CPU time and satuated all out-bound pipes. They also have bills to pay, and from their point of view, your $5/month account with 50,000 visitors a day is no more important than the other $5/month accout with 50 visitors a day. Turning off your website is probably their best interst to protect their other hard earned customers.

Some lessons learnt:

• Always back up your website, and keep at least daily backup of the database if it is on a dynamic CMS. Keep the backup off-site, i.e. not in the same web hosting account. You’ll never know whether your current host will terminate your account and hold your website for ransom.

• Good prevention would be doing thorough checking before signing up a shared hosting account. Google for “<web host> sucks” helps, and Duncan’s post has obviously already tainted Site5′s reputation. However do read through good unbiased reports as well (where you can’t find an affiliate links). Visit WHT and read through articles at Web Hosting Reviews can be very useful.

• If your site is on a blog software or CMS, make sure it is optimised. Got a caching plugin? Give it a try. Bandwidth and storage are cheap those days, and they are flexible in an overselling environment. Server’s CPU power is not, and it is usually the first thing hosts will try to protect.

• Sometimes unexpected traffic can really be, hmm, “unexpected”. Be that special Google love, Slashdot/Digg effect, or your enemies sending down DDoS attacks. Sometimes it is not even your fault — but your hosting company probably won’t care when they mourn over their dead server caught on fire. So, if your site is important to you, always prepare the worst.

• Be ready to switch hosts quickly. That means files and database are always backed up and ready to upload (to a new host). Don’t buy domains and hosting from the same shop — it is just going to make switching web hosts more difficult. Ideally, you’ll already have a redundant site up and running, and just need to switch the DNS at your registrar to get going.

• You don’t need to spend $100+ a month on a dedicated server to host your busy blog. If you are a seasoned Linux user, a cheap VPS on a minimum-share guaranteed technology (like Xen) is probably a more suitable alternative. I am hosting this site on a Xen VPS running Gentoo Linux at unixshell# which costs me around the same as a high-end shared hosting, but much more flexible. Even if it got Slashdotted (I wish), it probably will not affect other VMs on the same host.

• If you have multiple high-traffic sites, I will rather go with multiple VPS at different hosting company/data centre than a single powerful dedicated box. Don’t put everything in one basket, and unless you co-locate, “your” dedicate server is still lended to you by the hosting companies.

Well, I think I have gone too far. At the end, having multiple backups — not just data but also data centre — helps to keep your site alive. Costly? Yes, but great investment if your business depends on your on-line presence.