Here’s a sobering statistic. According to the National Archives & Records Administration, 93 percent of companies that lost their data center for 10 days or more due to disaster filed for bankruptcy within one year. A full 50 percent of those filed for bankruptcy immediately. Data loss is a business killer. And with more and more ASCs moving to a “paperless” environment each day, the safety and reliability of your data is a critical issue.
There are few events that have a more devastating impact on a facility than the loss of data. If you’ve never experienced a major data loss, or if you were unaware of what it takes to get your systems back online after a loss, you may not realize how vulnerable you are. For instance, did you know that all hard drives fail at some point? That means every piece of information stored on your systems — at some point — will become unreadable. Data loss is a matter of when not if.
The harsh reality for ASCs is that backup systems and strategies are often overlooked until it’s too late. A strong backup strategy protects your interest in three critical ways.
- When data is accidentally lost, a secured backup provides an immediate remedy. All servers and systems crash at some point. When they do, your last backup is your only chance at quickly returning to normal.
- When data becomes corrupted, a good backup is needed. Ever had a virus? Rather than losing your systems for a day or two while IT tries to remove it, you can simply return the system to its last good backup. With nightly backups, your downtime is a few hours instead of a few days.
- A strong backup strategy protects you in case of a purposeful data loss. If you find an employee is deleting or changing data inappropriately, your backups may be your only hope of resurrecting the lost data.
Four principles to developing a backup system
There are four guiding principles to building your backup system. While your IT provider will have a methodology for gathering requirements and analyzing your needs, these principles will ensure your needs are met.
1. Know your data. What are all of the systems and data you keep or access? Think of every system your facility uses. Don’t forget that items like your phone systems, security system and even your time clocks may have critical data that is not currently backed up properly (if at all). A good place to start is to think of everything in your facility that has a power plug, a keyboard, keypad or a mouse. If it has one or more of these items, it may likely contain data that needs backing up.
2. Know your backup needs. You are probably already backing up some or all of your systems. But do you know how often? Or what method is used? Here are some key questions to consider:
- How often does your data need to be backed up? Do your critical systems need hourly backups? How much work are you willing to recreate in case of data loss? Is there data that cannot be recreated if lost?
- Are your backups full or incremental (i.e., do you backup everything each time or only what has changed)?
- Do you keep your backups off-site? If so, are they secured in a HIPAA-compliant location (e.g., not in the trunk of your car).
- What happens if your systems are destroyed (e.g., fire, water damage, etc.)? Do you have a secondary location and equipment to bring your systems up from the backups? Does your IT provider have them available?
3. Know your redundancy. Redundancy is critical for a strong backup strategy. Redundancy starts with the individual systems. Virtually all servers shipped today come with some form of redundant array of independent disks (RAID) drive configuration. RAID is a group of two or more hard drives configured to act as one. Typically, mission-critical servers are configured with a mirrored RAID configuration, which means the same data is stored on multiple drives simultaneously. Additionally, larger facilities and companies use a storage area network (SAN). A SAN a network of hard drives that act as one (essentially a networked RAID), though they may be configured as many drives. In both approaches, the failure of a single drive does not cause an immediate data loss. You have time to replace the defective drive. This transparent approach means users never realize a failure has occurred.
Tip: It’s also a good idea to have redundant backups. Much like wearing suspenders and a belt, a duplicate backup can protect you in the event your pants fall down (metaphorically, of course). You probably have a physical backup solution (such as digital linear tapes (DLTs), USB hard drives or DVDs), but you should also consider a secondary online solution. Your IT provider should be able to help you identify an online solution for off-site data storage that is secure, real-time and transparent. In the event of data loss, you can recover files immediately from anywhere. It’s important to note that most online backup providers encrypt the data before storing it, so that you alone can access your data. Make sure your online backup provider can pass regulatory scrutiny. Ask them specifically about any regulations or guidelines that they will need to follow to keep you compliant.
4. Test your system. The absolute worst time to find out your backups don’t work is right after you notice a problem. A strong backup strategy includes regular testing of critical backups and systems. If you’re not familiar with this technique, it means your IT staff will literally cause your critical systems to fail and then attempt to rebuild them from the backups. While it sounds risky, it is similar to how pilots learn to recover from a stall. You’d rather practice when everything works correctly rather than when a real crisis occurs. Your IT provider should have experience in developing a safe plan for regular testing that meets your needs and budget.
Justifying the expense
In ASCs, where we work to minimize costs related to benefits, IT personnel may struggle to justify the investment in backup systems to those who are not as technically skilled. While it’s easy to understand why backups are necessary, you may have a hard time understanding why the backup system’s complexity and associated costs are justifiable. And while statistics and case studies support a strong backup strategy, your gut may tell you it all seems like too much expense for a “just in case” scenario. However, the first time you find yourself without critical data, you might find yourself willing to hand a blank check over to anyone who could bring your data back!
The costs associated with a backup strategy will be significant but not prohibitive. Not spending money on a backup strategy is akin to not carrying insurance of any kind. A backup strategy should be viewed like your property, errors and omission, malpractice or homeowner’s insurance. And while fires and malpractice are fairly rare, system failures are quite common. Think of this as an investment in your facility’s future.
Creating a strong backup strategy takes thorough planning and good execution. You will need an experienced IT partner to complete this project. Begin by having your IT partner provide you with an analysis of your current backup systems. For a few hours of their time, you can get a good idea of where you may need to improve. Once this is complete, you will work with them to develop a backup strategy based on your needs and budget.
Take a proactive approach to your plan
You will lose data at some point, guaranteed. This is not a “doomsday” prediction that never comes true. It is a sad fact in the information age. All systems fail; the only question is when. Even your backup systems will fail at some point. The best you can do is to make reasonable accommodations for likely failures.
The best approach is to plan for what you think is most likely, then for what is least likely and pick a solution somewhere in the middle based upon your budget and your tolerance for risk. Some companies have a “pound of cure” mentality and will likely never spend much on backups. However, the “ounce of prevention” companies will fare much better when data is damaged, corrupted or lost.
— Stephen Campbell is COO of Mavicor. Harry Cook is chief analyst for Mavicor. Mavicor is a leading ASC technology management company specializing in ASC application services, systems integration and consulting services, as well as the procurement and management of hardware and software. Contact Mr. Campbell at scampbell@mavicor.com. Contact Mr. Cook at hcook@mavicor.com. Learn more about Mavicor at www.mavicor.com.
