A day in the life of a data center facilities manager

Posted by on Oct 18, 2012 in Facilities Management | 1 comment

Ever wonder what it’s like to be the facilities manager of a major data center?  Here is a “normal” day – but in retrospect, there really is no such thing as a “normal” day for a facilities manager.

0500 – 22 unread emails

Somewhere close to 5:00 a.m., thoughts begin to penetrate your slumber. Your eyes open and confirm again that, for you, an alarm clock is not a necessity.  A call comes in from Security as you go about your morning routine, something about a contractor who wants to bring in an employee that is not on the access list.  During this call, the contractor calls you.  Switching between calls, you determine this person really is from a sub-contractor of the contractor and that he is needed to help install a piece of equipment integral to their project.  You acquiesce and give the guard your approval, informing him exactly where this individual is permitted to be and for how long.  Two cell phones, radio, keys, computer bag – cold coffee – and you’re out the door.

0545 – 25 unread emails

As you drive to the data center, a sales engineer from the East Coast calls to ask how much available square footage you have that he can sell.  You explain that square footage has very little to do with actual installations and that, more importantly, you need to know what the prospective client’s power requirements are.  He tells you that they’re looking for about 1 mega-watt.

“Really,” as you process the calculation quickly in your head.  You indicate that you believe you can support that but you’ll need to know how many racks they need specifically.  You go through another mental calculation after you’re told that the client is talking about 20 racks – or so. “Uh-huh.  Well, that’s about 50kw per rack” (not really possible that you know of…and you begin to think that someone is off by a factor of about eight to ten.)  You suggest that the sales guy set up a meeting with you and the prospective client because something about the numbers doesn’t make sense, but that you would be happy to help find out the actual specifics with the client.

0625 – 37 unread emails

You’ve been sitting in the parking lot of the data center for the past five minutes finishing the call, more than enough time to notice that the snow removal company left huge piles in places that wouldn’t allow emergency response vehicles into the area if necessary.  You call the contractor as you walk into the data center to have them remedy the situation.  [Mental note as you pass the through the door:  Check on sidewalk de-icer supplies; order more.]

You run by the Network Operations Center (NOC) prior to going to your office.  You glance over the work logs and system parameters on the Building Monitoring System.  You say hi to the night shift and ask about the two alarms that are still on the system.  The team reminds you that they’re for a sensor that died a couple of nights ago.  Your mind drifts to that night and the 1:00 a.m. call.  “Oh, yeah.  I remember that.  It’s the air flow sensor in Number 4 air handling unit.”  You go through the procedures with the night shift that are in place to mitigate the alarm condition.  You check the work request log and see that the night crew has submitted a work procedure for you to review and approve to replace it.  “It’s probably in my queue.”  One of the night shift guys approaches you and wants to talk privately.

You proceed to your office with the person and close the door.  He says that he’s sorry, but his wife’s father’s condition has worsened and he needs to give his notice – well, one week anyway – and that they are moving out of the area.  You express your concern and that you’re sorry to see him leave, but understand.  Your mind refocuses immediately to grapple with how to cover the shift and what the market is like to find another back-shift technician.  [After he leaves, you jot down a note to jog your memory: Send a job description to HR and a req. to your boss to get permission to fill the position.]

0655 – 47 unread emails

You settle down to your desk and log in to your email….47 unread messages.  You glance over them to see if anything is of great importance, and there is one from your boss.  He wants to confirm that all your people will attend the “thank-you” celebration on Friday.  More importantly, he really wants to be assured that preparations for the food, AV, and room are complete.

Back to the NOC to get the turnover from the off-going techs and brief the on-going techs with what is happening.  You get their observations, needs, and complaints.

0700 coordination meeting – 51 unread emails

During the coordination meeting, you provide the data center manager and others information around what work is planned for the day, which procedures need approval, who needs to talk to whom, and when all this is supposed to occur.  You get news that a new installation is planned – well, actually it’s approved (“By whom?”) and happening in three weeks.  It’s only 30 more cabinets.

“Oh, by the way, what do you need to support this?” you’re asked with tacit recognition that you shouldn’t have been left out of the loop.

“How much power do they need?” hoping your reply masks your frustration. “I need to balance where this power is coming from and what to do with the heat load.” You’re given the name of a sales engineer and told to contact him because he knows all about it. You agree, tentatively.

Another item comes up for discussion, the internal client didn’t get the notification that work being done later today would affect them and is saying they can’t support it now. The thing is, they have provided no indication when or if ever they could.  You think this item has been covered in no less than five emails and talked about in at least four to five meetings.  Where has this client been…under a rock?  The data center manager says that he will take it to the VP of the group after this meeting and wants you there to sit in and explain.  The meeting ends – for now.

Back at your office – well, for a minute anyway – as one of your technicians comes in and wants to show you what they found in the chiller area. The technician shows you a pipe that has vibrated loose. “It was never really engineered to take this kind of stress,” the tech says. You call the vendor and ask for one of the engineers to come out and investigate.  They say they can be out later that day, and so you call security to inform them that the vendor is approved for access.

You walk back to the office area where the data center manager calls you into the office and tells you that he has the VP on the line.  After a five-minute discussion, you have your marching orders…the maintenance will be put off until the following week.  You now have to call the three vendors, the equipment rental place, and reschedule the work.  You have the lead technician in charge of the project call the vendors and suppliers.  You remind yourself that you have to review the run data for the chillers…there must be a way to get them to run more efficiently.

0945 – 61 unread emails

Back to your office again, you finally get to the procedure queue and see that there are eight procedures requiring your approval in the work order system.  Closing the door, a sign that you need to be left alone unless it’s really important, you start on the procedures. Having finished two of them, the data center manager knocks and asks you to join in on an important tour of the data center.  You oblige.

A prospective client, two company VPs (sales), and several others are there to tour the facility.  You offer the same information that you do on all the tours: how much, how big, how do you guarantee reliability, and how about those 49ers last night? (You actually didn’t have time to watch the game, but you overheard the technicians talking about it.) The sales team invites you for lunch, but you decline as you have work to get done.

1215 – 115 unread emails

You get a cup of soup out of the machine and a soda, return to your office and close your door. By the time you get the procedures reviewed, approved, or sent back for revisions, you dig in to the emails, all 115 of them.  You actually get the in-box down to 80 or so emails before your next meeting occurs or you have to go out to the field to monitor the work being done.

1800 – 76 unread emails

And so it goes, until you realize that it’s about 6:00 p.m. and you should be thinking about going home.  But you look at the screen and it tells you there are still 76 unread emails. “Okay,” you think. “I’ll just be a little more time to get some of them cleared up.”  It’s nice after everyone has left. You finally have time to get something done.  You open one email and find out that HR doesn’t have the last job descriptions for the replacement position, so you have to write one before you leave.

1930 – 82 unread emails

On the way home, the on-shift technicians call to tell you that the day-shift left an energized panel open (properly roped off and signed, but still open) and are requesting the status of the job. You call the vendor and find out that they are working shifts tonight and just stepped away for dinner, as they had planned. It just was not turned over to the night shift, and so you call them back and inform them as you drive into the driveway.

It’s family time, except that the kids are in bed. You hit the emails again, 82 of them. By 10:30 or so, you’ve had enough and promise yourself that you’ll do it faster and better tomorrow. You call it a day with only 14 unread emails.

For those who are not facilities managers, I hope this glimpse provides some understanding of the vital and pivotal role the data center facilities manager occupies in the operation.  Facilities managers must deal with all levels of the organization, across all functions, and be literally a “jack of all trades.”  To all my facilities manager friends, I wish you all the best and know that you are some of the best and most dedicated of any industry.

One Comment

  1. Very well described

Leave a Reply to Gary Thornton Cancel reply

Your email address will not be published. Required fields are marked *