I enjoyed watching one of the old “Top Gear” episodes recently that discussed the lifetime cost of ownership of the Land Rover as compared to the Prius.  The episode discussed the mining of the materials necessary for the batteries and how that impacted the real cost of each car.  The “Top Gear” hosts stated something to the effect that when you factored in what it costs to manufacture and properly dispose of the lithium batteries, the Land Rover was actually “greener” than the Prius.

While there is debate whether what was said on “Top Gear” was true or not, it did get me to think about some of the new innovations that I’m seeing in data centers.  When I look at factors beyond the obvious operating costs, I find that some of the innovations have hidden costs that affect total cost of ownership.  Let’s look at some designs and explore the possible hidden costs.

Air segregation

One of the new design trends is to enclose the hot aisle, cold aisle, or both aisles.  This design concept can be implemented in a variety of ways from individual cabinet enclosures to entire rows or pods.  Many modular designs incorporate this design feature.  These designs make their systems more thermally efficient, but one of the hidden costs is increased cost for maintenance.

From a facilities perspective, especially in the modular or container designs, maintenance becomes a nightmare due to limited access.  I’ve seen everything from total inaccessibility (we can be left having to cut holes in the walls for maintenance!) to adding an extra 3 minutes each time we service the component because of the design.  While 3 minutes doesn’t seem like much, when you multiply it by the 720 units in the facility, you’ve just added 2,160 minutes or 36 more hours of maintenance per week for your technician.  At the current cost to the company of $80/hr for a technician (in the United States), you’ve added $149,760 per year to your budget.

When compared to the savings from efficient thermal design, this is a rather insignificant cost – but it still needs to be taken into account.  The increased cost for maintenance is just one factor to consider.  There usually are several other design issues which can add significant costs too.

In all fairness, I have seen estimates for thermally efficient retrofits save over a million dollars a year from the previously dated designs.  So the increased cost of maintenance probably won’t make a deciding difference, but it might not allow the facility to reach a specific goal or expected savings.

New electrical designs

Moving the batteries/UPS to the server trays is another trend I’ve seen from Google and Facebook.  Making this change offers several advantages:

1. Isolating the failure of the UPS to individual servers versus entire groups and network gear as in the case of a large UPS failure.
2. Elimination of the associated maintenance and contractor/vendor dependence.
3. Simplified electrical distribution systems (reduced capital cost).  No large static switches, UPS rooms, associated switchgear, battery rooms, et cetera.
4. Increased system apparent efficiency; no double-conversion or associated losses; no friction or conversion losses from kinetic systems.
5. Server UPS/batteries only have to be added to those systems for which you really need that level of protection.  The protection systems are right-sized, no over-built capacity or stranded capacity.

Again, these types of innovations can save a lot of capital cost, but there are other costs that come from this type of setup as well.

Instead of dealing with 640 batteries per MW, you now have over 3300 batteries depending on the design.  Imagine the processes you have to have in place to deal with replacements, failure detection, storage, and recycling – not to mention that these batteries are usually of a lithium variety and require a more complex recycling process than the lead/acid batteries in most industrial sized UPSs.  While it may take a facilities technician at a large facility (over 10 MWs UPS load) full time to stay on top of all the UPS batteries, imagine how many technicians that you would need to handle over 33,000 batteries in the same facility with server-based systems.  For industrial-sized systems, you would have about 6,400 batteries in a 10 MW site, which means you would replace about 1,280 batteries a year with a 5-year average life or about one string (of 40) every other week.  This equates to roughly a day’s work for a couple of facilities technicians.

Now, with the server-sited systems, let’s assume the batteries last about 3 years.  (My laptop battery only lasts about 3 years – feedback on this from those of you who have experience would be appreciated!)  But a 3-year life span means that you have to replace approximately 11,000 batteries per year or about 212 a week.  At 5 minutes apiece, that is 17.7 hours/week, more than twice what it costs to change conventional UPS batteries (approximately 18 man-hours versus 8 man-hours).  This is more than 500 man-hours/year or about $40,000 more a year in labor alone.

Another thing to consider is the infant mortality rate of the different types of batteries.  I don’t know if lithium-based batteries have a higher or lower infant mortality rate than the VRLA batteries that are used in most large UPSs today.  I suspect that the lithium batteries have a higher failure rate, which would drive costs up even further.

These are just two examples of costs that I don’t see addressed in most discussions when these new innovations are put into place.  While I’m sure that there are some great savings calculations with these new designs, I want to know the total cost of ownership (capital, maintenance, operational, waste stream, energy, et cetera).

This week’s blog is simply trying to point out that it’s a good practice to ask the people involved in maintenance and operations to provide their input when new designs are contemplated.  Their insight may show that the new innovation being presented by a vendor or designer might not be as fruitful as you think.  Who knows?  Maybe the maintenance and operations people will offer ideas that will make it work for all concerned and bring you the savings you need.