Roadmap for Routine Administration Manpower Requirements

For the system administrators out there, and the managers of those teams, taking care of an enterprise often boils down to a fiscal year's hardware and software costs and whether those assets are being properly managed. Rarely, are IT teams prepared to justify their own existence to the CIO.In most cases, manpower is lucky to be an afterthought, when it should be part of the equation from the beginning. Immediate supervisors may be aware of this concept, since automation only goes so far. Yet, that information doesn't always translate well to those who are capable of hiring new team members.This is often because the lower-level supervisors are focused on immediate operational challenges, rather than the twelve month manpower horizon. This dilemma isn't easy to solve, but there are ways to create a forum for debate.I believe that identifying the amount of time it takes to maintain you hardware and software through automation and manpower hours is a key to future success, and proper management and retention of talent. By starting with numbers like the ones I'm providing below, not only are you listing the time needed for maintenance on your enterprise, but you can justify additional hours for repairs, training, and possibly even security improvements.It's difficult to perceive how each organization operates its IT team, but my experience tells me that if they wear a security hat, they remain focused on this. Likewise, if their shirt says "operations," their main goal will be to keep everything operating smoothly. This is often lost on outsiders who arrive and ask operations teams to implement security measures, ignorant of the reasons why role delegation is important.This is not a solution for everyone, but it's a guide that may be helpful, allowing each team to justify their manpower needs so they can deliver excellence. The tasks have been broken down into daily, weekly, monthly, quarterly and annual lists. Some of you may believe a task on the weekly list should be checked daily and vice versa. I'd simply remind you that this is a roadmap to customize for your environment. 

Daily tasks (4-50 minutes spent maintaining each server):- Verify server is up/operational and properly providing resources (1 min). This can be as simple as testing a file share, reviewing script output or checking your network monitoring solution.- Daily cleanup tasks (1-5 mins). Are there temp files, logs, etc that can be cleaned up on a daily basis?- Disk space utilization (1-5 mins). You should always know where each of your disks are on a daily basis. Don't get comfortable logging the information and not knowing when you had a sudden jump in utilization. It's those kinds of spikes that can signal issues.- Review event logs (1-10 mins). You need to look at these daily, and for the sake of continuity, keep a tally of the errors and warning events you research and troubleshoot. That way, the next time they show up, you won't have to repeat the work and you'll know you have a repetitive problem.- Maintain a server log (0-30 mins). After reviewing the logs, if you find something, keep a log. If you have hardware maintenance, log it. If you have software issues, log it. This isn't a place for your patches as those fall into the realm of change management, but if you want to log them here, it won't hurt. If you use this log, it will become invaluable to you, mostly because problems usually repeat (if only on other servers).

Weekly tasks (13-120 minutes spent maintaining each server):- Check for standard practice violations (2-30 mins). Are permissions on shares being manipulated? Are there new resources on the server that you don't recognize? Are resources being used incorrectly that may require additional user training or corporate policy re-write? The time spent here will pay off huge, if you take it seriously.- Make configuration backups (1-15 mins). To help identify unauthorized changes, or recover from failures, pull configuration information from your servers and store them in .csv/.txt/.html/.xml/etc... This will assist in configuration management endeavors and once you've figured out what you want, you can script it.- Verify routine internal server maintenance tasks are occurring (5-15 mins). This is often left unchecked and then we wonder what went wrong. Is Exchange running all of the database maintenance tasks successfully? Do you understand all of them? How about Active Directory? Are your drives heavily fragmented? Is antivirus getting definitions?- Talk about your server(s) and the issues you face (5-60 mins). This is valuable. You probably have weekly meetings and no matter how many decades you have in the industry, if you're working an issue, bring it up at a meeting and take suggestions from your peers. Some of the comments may be things you've already tried, but it's a fantastic exercise in troubleshooting/critical-thinking. This also helps to educate the younger minds in the room. If they can learn from a problem you're seeing, it may benefit your customer months down the road.- Don't forget your daily tasks!

Monthly tasks (6-195 minutes spent maintaining each server):- Clean-up (1-15 mins). This is a chance to expand upon the daily cleanup and delete items you're sure aren't needed month-to-month. Clean those old backups from your weekly tasks, and evaluate if there are additional areas where you can improve your efforts (perhaps scripting).- Patching (5-180 mins). Try to automate this with enterprise tools as much as possible, but be prepared to manually patch your highly available farms, and fault-tolerant clusters as needed. Make sure you test if you're able to and follow sound change management procedures in your enterprise. Notify customers and your helpdesk before and after patches are applied and be sure you have functionality checks for each of your servers after patches are complete (pinging isn't a functionality check).- Don't forget your daily tasks!

Quarterly tasks (30-90 minutes spent maintaining each server):- Review (30-90 mins). Verify that your daily/weekly/monthly tasks are getting done. If not, be honest and understand why. If your supervisor is putting too much emphasis on putting out fires, let him/her know that eventually the servers you work on will become fires because you're not giving proper attention to your servers. Verify your configuration and instruction documentation is up-to-date. Is there anything missing that would make your server installation better? Perhaps some new instructions for the helpdesk team to better respond to calls from users? This is the time to knock that out. Honestly, I seriously consider documentation to be a daily task, but sometimes you just have to declare a down day and focus solely on quarterly tasks, which gives you the chance to do your documentation.- Don't forget your daily tasks!

Annual tasks (60-180 minutes spent maintaining each server):- Project requirements (60-120 min). Examine disk space trends and any other performance metrics you've gathered (hopefully, you've noticed an opportunity to exceed plain-old disk space monitoring). Review your server logs and identify recurring errors and any hardware issues. Take into account geographical, political, financial, and technological trends and make recommendations/requests for anything you think you'll NEED in the next two years.- Review new technology (60 mins). If you're only reviewing new technology once a year, you've already failed miserably, but if nothing else, putting this on your annual task list makes it a "must-do" for you to accomplish at work, on the clock. You have to start looking at new solutions and what the industry is planning. For those of you running single-instance stores on Exchange 2003, who weren't looking ahead to multi-instance stores on Exchange 2007/2010, there's a chance you made some bad purchases in the storage world. Knowing that I/O reductions from 2003-2010 we're around 90% would have impacted your decisions greatly. These are the types of things you're looking for during this time, so don't waste it. Gather your information, jot down your notes, and project those requirements. Just make sure you differentiate between NEED and WANT. Your bosses will thank you for it.

---

Again, these are just to get you started, but the numbers are viable. Please consider them if you decide to assemble metrics on how much time it actually takes to maintain servers in your enterprise. There's only eight man-hours in a business day. With proper automation, you can free yourself from some of these tasks, and it's very useful to advertise what you should be doing while you're putting out fires.