15Amazon Web Services is expensive, if you compare them with the overselling dedicated server market.
Automattic’s Matt Mullenweg responded to Joyent’s point on grid, showing how big a package they can get from LayerTech with under $2.5k per month. It’s indeed quite an impressive list — 15 boxes sharing 27Gb of RAM, 500Gb of RAID’ed storage and 18Tb of public data transfer. At the end he asked:
What do you guys think? Anyone want to work out what the same would cost on Amazon? Which would you go with?
Let’s take out the calculator.
Amazon Web Services is Expensive!
We will just look at the bandwidth cost alone. How much does it cost to pump out 18 terabytes of data from Amazon (EC2 or S3)? At the current rate of 20cents per Gb, 18Tb is going to cost you $3,600! It is already more expensive than LayeredTech’s offering, and we are not yet counting storage and computation power yet!
Sure, your 500Gb file server would only cost $75/month on S3. Running 26 instances of EC2 24×7 would cost around $1,900 per month (26 because Matt’s setup has 12 dual core + 2 single core servers). 45Gb of RAM total, but it still falls short of total amount of “CPU power”.
But the bandwidth cost?!
Amazon: You (only) pay what you have used
I guess the main difference is, most web hosting companies oversell. Most LayeredTech dedicated servers come with 1.5Tb of monthly data transfer, but I am sure many servers actually don’t use anywhere near that.
When Linden praised Amazon S3 for saving the Second Life download, my initial reaction was — “hey that’s crazy”. 1Tb of data transfer? $200 on S3, where as my $100 LayeredTech dedicated server would be sufficient! Then I realised, someone else is actually subsidizing me for using all my allocated data transfer.
S3’s model is also more flexible when the usage pattern is less than regular.
For example, the following month traffic dropped to 500Gb. With a dedicated server however, it will still be $100 (with CPU mostly ideal as file servers are IO bound). With S3, you only pay what you have actually used — $50.
So is it expensive?
Back to Matt’s question — I think AWS can be more expensive when you try to spec out something similar to your average dedicated server offerings. However I do not think Amazon is pushing AWS as a “cheaper dedicated server replacement”.
Instead, AWS is making CPU power, storage, bandwidth, etc a “commodity”. They are “cheap enough”, yet easy and fast to provision, deploy and use. For example, you might be capable to build an economical power generator in your own backyard, it is usually more convenient to just connect to the electricity grid — it will make your life much easier, and much more capable to scale up and down with your demand.
Update
Err. Fixed a few spelling mistakes as I initially wrote this post way too late at night yesterday.
Also with reflection on what Isabel has said, AWS scales great both ways. Not only that it will cost you less during the “not-so-busy” season, it also gives you flexible bandwidth without the need to deploy additional servers during busy seasons. Managing these servers take time, and time is expensive.
At the end, a fleet of 15 servers might cost $2,500/month. A good sysadmin costs much more than that.
Hiya! Torley (from Linden Lab) here… hehe I gotta let Jeff know you linked to us, he worked hard on and got us to S3, as you know. Cheers.
Hey Scott, 1 TB of data transfer is $100 on a dedicated server, and $200 on S3. But what if your traffic went up to 10 TB? You could set up 9 more servers at a total cost of $1K, but you’d have to make sure no individual machine exceeds 1TB. Otherwise you’d be billed $0.50 to $1 per GB (depending on the hosting provider) in overage fees. In this case, S3 might still be twice as expensive ($2K for 10TB), but you’d also have to take into account the time and effort of managing 9 extra servers. So oversold hosting plans are a great deal – up to a certain level of usage. I think Amazon’s target market is people whose requirements are above that.
10 bandwidth-oversold dedicated server would still turn out to be cheaper, and you get spare cycle which you can then resell :)
But you are right, Isabel, that the cost of commissioning and decommissioning servers, managing them, making sure traffic spreads across (you get 10×1Tb instead of 1×10Tb) — all these at the expense of developers and sysadmins, which would turn out to be more expensive. AWS scales up well in this case.
But then it comes down to one problem I have with the web hosting business. Why the combination of (CPU utilisation + data transferred + storage) (almost) always mark as inseparables unit? Running out of disk space? Upgrade to the next “unit” up. Running out of bandwidth? Upgrade to the next “unit” up. Running out of CPU power? Upgrade to … Yes there’s overage charges, but they are usually ridiculously expensive to be an feasible option. And you usually have applications that run out one resource before the others. Having 10 dedicated servers to serve 10Tb of data is just (1) waste of sysadmin’s time (2) waste of energy.
When I worked at EV1, I had a customer who rented 35 servers. Later I found out he was only using 5! If he bought the extra 30 TB of bandwidth he needed without the hardware, it would have cost much more than servers + bandwidth. Not very logical, is it?
As you pointed out, when you sign up for a hosting package, your usage is subsidized by lower-traffic neighbors. This means if you want just the bandwidth, nobody else would chip in on your costs. And if hosting companies unbundled everything so every separable unit of bandwidth costs the same, they’d end up with Amazon’s business model. I’m not sure if there are other alternatives besides oversold bundles and pay-per-use?
By the way, what you asked for is precisely how Amazon is marketing AWS: different people need different amounts of bandwidth/storage/processing power, so they sell everything separately. You can either pay more for choosing your own combination and being in control. Or let hosting companies decide how and what to oversell.
The bandwidth you get is a little crazy, but the RAM is really quite useful.
I was going to post a comment, but Isabel basically said what I wanted to. =)
Those are two things a hosting package generally could not offer. The cost of human time to us exceeds the cost of machine/bandwidth.
Isabel and Jeff – thanks for the comments. I have since updated this post. Yes, buying arrays of dedicated servers is only cheaper when the cost of administration and management is negligible, which is actually not the case.
That’s not really true: with layeredtech, you can rent a private switch for $10/month. The final bandwidth usage calculation will be based on the traffic that goes through the switch. It doesn’t matter which machine it came from.
That said, yes, there’s something to be said about a hassle free service like S3.
So what is real price for bandwidth? 0.01 / GB?
Nice analysis, I’m using a combination of S3 and a virtual private server, trying to get the best of both. Depending on how your content is distributed, you may want to host the frequently-downloaded stuff locally, and save S3 for the offsite cache.
I put together some numbers here in case you want to play with the numbers above:
http://my.instacalc.com/calc/52b29bbad9ff4abdba83bc76b945e389
Kalid — it is a nice calc app you’ve got there! I can instantly find many use of that. :)
Also thanks to the S3 calc. However from the discussion, the bandwidth + storage cost is not the whole picture, as you need to factor in the cost of managing those storage and pipes. S3 does all that for you. Depending on how costly a sysadmin is, S3 can be cheap or expensive.
Thanks Scotty! I originally made the app for myself but it grew into something useful for others, I’m glad you like it!
I agree about the hidden costs — not having to manage your own server is a huge part of the equation. There’s a “headache factor” that isn’t accounted for in the numbers :)
Thank you for the information on the issue of major topics on the issue,By the way, what you asked for is precisely how Amazon is marketing AWS: different people need different amounts of bandwidth/storage/processing power, so they sell everything separately. You can either pay more for choosing your own combination and being in control. Or let hosting companies decide how and what to oversell.
I would correct the author in that misconception that running services on Amazon versus dedicated assets requires no systems administrators.
On the contrary. The difference is your system administrators don’t have to spend time managing hardware. But that is never greater than 30% of their manhours in any given month usually. Most of the time is spent on the software level (OS, toolchain, applications) and none of that goes away with Amazon. If anything, they will spend more time (especially 1st-year) dealing with whatever idiosyncracies exist with AWS and there are a few — not least of which is the way EC2 instances come up with different addresses everytime and don’t yet offer immutable storage.
The primary advantage is that, especially for startups, SMBs, there is no up-front hardware investment and network commitments. Past 3rd year those start to break even.
But the need for systems administrators is almost equal if slightly less than running your own infrastructure. You do not want developers running systems, that is a recipe for disaster.
“But the need for systems administrators is almost equal if slightly less than running your own infrastructure. You do not want developers running systems, that is a recipe for disaster.”
Never a truer word was spoken, developers don’t give a damn about systems their primary concern, especially in this respect is making some website look pretty, as a result maintainence suffers, bug fixes/patches/whatever don’t get applied, security gets left WIDE open and trojans usually find their way in etc etc etc etc