What regulations govern data center operations?

Posted by on Dec 4, 2014 in Facilities Management, Training | 1 comment

What Regulations Govern Data Center Operations? In the United States, various codes and regulations protect the environment and keep workers safe. There are several types of environmental compliance requirements that pertain specifically to data center operations, each requirement varying based on the size and type of data center involved. But unfortunately, data center managers can become so focused on what it takes to ensure uptime that these environmental and health and safety compliance requirements can get overlooked. There are fines for noncompliance that can amount to hundreds of thousands of dollars and, in certain circumstances, a manager can be held personally liable for a violation. Environmental regulations common to data center operations One environmental regulation applies to diesel fuel storage. If you store diesel fuel for your emergency generators in excess of 1,320 gallons, you are required to perform an evaluation to determine if your data center falls under Spill Prevention, Control and Countermeasure (SPCC) plans. The plan includes spill prevention and cleanup procedures, inspection protocols, recordkeeping, and training and may require that a professional engineer, or PE, sign off on the plan....

Read More

Mission critical training: Are you certified or qualified?

Posted by on Sep 4, 2014 in Training | 1 comment

How do you certify or qualify someone to work in your data center? Do you administer classes, give them a written examination, question them in front of a panel of experts, or review their experience and call it good? I see it on resumes and applications, “qualified data center technician” or “certified data center technician or engineer.”  And when I see that, I always ask, “How did you get qualified? Who certified you?” A combination of experience over the years and a certificate given by the employer (who in reality doesn’t have an accredited program) is usually offered as the answer. I see a definite need for legitimate certification and qualification processes; but before I go further, let me explain how I use the terms “certification” and “qualification.” Certification: A process through which a person’s level of knowledge and skill at a certain level has been validated through accepted, repeatable examination and testing. Qualification: A process through which a person demonstrates repeatable success in the operation and response to a facility’s equipment and systems....

Read More

The most missed component in training — understanding

Posted by on Jan 2, 2014 in Training | 0 comments

Most training programs teach the basic knowledge required to do the job. Students are basically put in a classroom where someone pontificates on the benefits of whatever piece of equipment is being reviewed, the design of the system and what it does, and the basics of how to start it up, shut it down, and make operational adjustments. A good training program provides time to learn the skills to do the job. Students are required to work in a lab-type environment or given on-the-job training. They get to work with and actually operate the system or use a simulator. They are taught procedure, encouraged to ask questions and, more importantly, given time to practice and perfect new skills. A great training program goes to an entirely different level, giving students an opportunity to understand why things operate the way they do. The instruction incorporates an understanding of the design, including the physics or chemistry of the process. Students are taught what makes the equipment or system effective and efficient, how it is affected by changes in the environment, and the concepts behind the operational parameters that affect safety....

Read More

The maintenance we miss

Posted by on Oct 26, 2012 in Facilities Management, Incidents/Downtime, Leadership, Training | 4 comments

It’s not uncommon for data centers to spend hundreds of thousands of dollars a year maintaining vital equipment.  A typical maintenance budget can run 1 percent to 3 percent of your initial capital investment for each year of operation, and this amount goes up as the equipment ages.  It’s a necessary evil if you expect high levels of reliability in your data center operations.  But curiously, in this industry (and many others, by the way) the budget allocated to maintain their most important asset is usually a mere fraction by comparison. Are maintenance budgets misallocated? It’s well known that human-caused downtime is one of the largest reasons for downtime – if not the largest reason.  It’s just an observation, but oftentimes in my personal experience, I see an allocation of budgeted resources that doesn’t align with what we say is important.  Many budgets look like this: Equipment maintenance budget – $600,000/year Training for technicians budget – $10,000/year (if that)...

Read More

Failure is inevitable

Posted by on Apr 23, 2012 in Facilities Management, Leadership, Training | 3 comments

In the mission critical environments industry, we often talk about that “failure is not an option” – and for the most part, we believe that and work toward that goal.  But the stark reality is that failure is inevitable.  At some point in the future, everything will fail.  We do not have unlimited resources, nor do we have perfect engineering or flawless operations.  Whether we look at air travel, nuclear power plants, or even the brakes on our cars, failure occurs.  We build backup systems for those inevitable failures, but even the backup systems will fail.  I have seen quadruple backup processes fail.  The only thing we can do is try to mitigate the results of failure and/or how often it occurs. So how do we cope with inherently dangerous or expensive processes and their inevitable failure?  In some cases, we deem failure an “acceptable risk.”  We calculate the chance of failure as Mean Time Between Failure (MTBF), the likelihood of an event is 1 in 200,000 years, 100-year flood plains, et cetera.  The number of deaths per passenger mile on commercial airlines in the United States between 1995 and 2000 was about three deaths per 10 billion passenger miles.  Not bad, but not perfect either – three people died per 10 billion passenger miles.  Is that an acceptable risk?  Probably not if you’re one of the three.  But even in the highly regulated, inspected and trained world of commercial air travel, failures occur.  What I wonder is, If we doubled the resources that we use in that industry for safety, would we see a reduction to 1.5 deaths per 10 billion passenger miles?  What if we spent ten times what we do now?  Would the statistic be reduced to 0.3 deaths per 10 billion passenger miles?  At what point do we run out of resources, and can we ever get it to zero failures?  The answer is no, there will always be something unforeseen – just ask the management of the Fukushima nuclear power plants....

Read More

The most important 2 seconds of your career

Posted by on Mar 2, 2012 in Facilities Management, Incidents/Downtime, Leadership, Training | 0 comments

The lights flicker, your phone starts making “message received” sounds, and the radio crackles with excited voices.  You recognize that something is not as it should be at the facility and you’re the person on duty with the responsibility to respond.  It becomes apparent that the power system is in distress.  The orders come over the radio to shift the “E” lineup to backup.  You run to the “E” power room and quickly move the switch to the backup power supply position.  You hear the breakers actuate, and then the unthinkable happens – the lights go out.  The ironic thing is that shortly after you turned the switch, your mind actually was pondering the possibility that you could have heard “D” instead of “E.”  And sure enough, the actual order, as it turns out, was to place D into backup and not E.  Your actions caused a loss of power to the facility, compounding the initial problem....

Read More