Verifying Compliance with Data Backup Retention Requirements

How long should you retain backup data? The answer is not as simple as some would think. It can be kind-of like asking how long to retain your old bank statements. In a lot of situations, there is no rule that says you must keep any backup data or historical records. Maintaining records is often a way of reducing risk. For a simple example; if you are accused of not paying a bill, your records can provide the exact date and amount of payments. However, there are many other situations where the maintenance of records is required by various local, state, and Federal laws. The Sarbanes-Oxley Act of 2002 specifies mandatory practices for public corporations which include certain requirements for retention of records. The
Health Insurance Portability and Accounting Act of 1996 requires that most data and records about patience be maintained with strict privacy. Your business should have it's own records retention policies that are designed to protect you, your business, and your company from lawsuits, downtime, and loss of revenue caused by unforeseen events.

Your data backup and retention policies should be integrated with your records retention policies. Because so many of your important records are maintained in electronic form, your data backup retention plan may be more consequential than a safety net for disaster recovery alone. If your data backups are not maintained according to policy you may be subjected legal action, loss of business, or worse.

Internal audits will help you determine if your data backups are being retained in accordance with policy. In some cases, outside auditors will examine your policies and check to make sure that you are actually retaining your records, including electronic records, in compliance with applicable laws. Most auditors will want to see a plan and also verify that plan is implemented and operational. Auditors or not, it is essential that you verify that your records retention policies are implemented and working properly. In particular, backup data retention tends to be more problematic and requires more detailed inspection.

Manual data backup systems can be difficult and time-consuming to verify. It is often the case that media are not properly or consistently labeled. It is also common to find that media is over-written or otherwise mishandled resulting in loss of data that should be retained. The reviewing of logs only offers limited verification. The only way to verify that historical data backup media actually contain the correct data is to physically mount the media and inspect the contents.

More robust automated backup systems usually maintain a catalog of media and their contents. There is no guarantee that the media actually contains the data that the catalog indicates. Although the better automated backup systems protect against improper overwriting or formatting of backup media, they cannot prevent all human error such as mislabeled media.

Online backup services tend to be the easiest to verify. Online backup services maintain a catalog of backup data similar to the way the aforementioned automated backup systems do, except there is no media involved. Because online backup services are usually entirely automated, there is little opportunity for human error. Without the opportunity for media to be mishandled, the online backup system's catalogs tend to be accurate, which makes it much easier to verify that the data backup and retention policies are being implemented correctly.

Time To Rethink Corporate Backups

There are two technologies that have converged in the last several years to make the traditional approach to corporate backups insufficient to ensure that company data is secured through a backup. The first is the proliferation of broadband connections. The second is the dramatic drop in the price of laptops.

Many corporations are now opting to purchase laptops for their corporate users so that they can connect remotely to the company's network. I have observed this transition myself. In 1999 while managing a data center for Digital Insight an online banking company the only employees that were issued laptops were Executives and System Administrators. All other employees worked on desktop machines. Laptops were much more expensive then the average corporate desktop in 1999, so this policy made good economical sense for businesses. While managing the data center for Digital Insight I also observed many System Administrators openly express their indifference to the data that is stored on the corporate laptops. I was one of them. Even though we were using ARCserve for backing up all the servers, we were not concerned about the other corporate systems. The IT department instructed all the employees to save their critical work to the network drives. The corporate file servers were then backed up nightly. We did not even attempt to backup the desktops, and there simply was not a good solution for backing up the laptops, which were frequently disconnected from the network.

Fast forward to 2006 where laptop prices have almost reached parity with desktop prices (when you factor in the cost of flat panel monitors) that many businesses are not even purchasing desktops. For instance, the last company that I consulted for only issued laptops computers to its employees. Desktop computers were setup for users that were shift-based and who were required to be onsite to complete their work. While equipping employees with laptops does make sense for many businesses, laptops are more likely to be stolen, lost, damaged or suffer hard drive failures. One thing that still has not changed since 1999 is the indifference from many System Administrators to the data that is stored on these corporate laptops.

The rapid adoption of laptops as the standard corporate computer has changed the paradigm of what information needs to be backed up. Remote users simply do not even think about storing their critical information on network drives because the transfer speeds can be very slow over VPN connections. In my experience, remote users do not even bother to upload their data to the network drives. It is not unusual for me to encounter situations where laptops are disconnected from the corporate network for long periods of time and the data on these laptops never gets backed up. As the American workforce continues to become more tele-communicate friendly, the amount of corporate data on remote laptops not being backed up will only increase.

Traditional approaches such as network drives and client-server software do not take into account workers that are telecommuting. Obviously, a new approach is needed. The only thing that the Backup Administrators can depend on is that the remote users will have internet connectivity.

The situation gets much worse. Many System Administrators are asleep at the wheel, relying on the older backup solutions that depend upon the users to knowingly upload their documents to the network drives. Let me say this once and for all... It is not the users responsibility to backup their corporate laptops it is the companies responsibility! This responsibility then falls to the IT staff to insure that corporate data is not lost.

As I work with many System Administrators it is obvious that remotely backing up laptops is one of the more complex requirements that confronts corporate IT staffs. Corporate backup policies have to be updated to accommodate the modern workforce and the new backup requirements of highly mobile laptops. When Backup Administrators are analyzing possible solutions they must take these remote users into consideration and deploy a solution that with accommodate remote laptops without inconveniencing the mobile users.

Internet based backup solutions are the only solution that meet all the of the unique requirements of the modern workforce. Internet based backups are the only solutions that make good since for Administrators to manage these remote users. Many of the more robust solutions allow Administrators to manage the remote backup software configuration via a web interface. Any of the changes that the Administrator makes to the backup configuration will be automatically downloaded to the client software the next time it begins a backup. This centralized management capability is a god-send to Backup Administrators trying to implement and manage remote users.

System Administrators, Backup Administrators, and IT staff need to rethink their approach to corporate backups and integrate the new exciting capabilities that Internet backups offer. While I do not think that any organization should rely solely upon this new technology, it can dramatically simplify the complex issue of securing and backing up corporate data from remote users.

What is a Hard Drive Crash?

Computer disk drives have come a long way since they were invented in the 1950's. The technology has been improved remarkably over the years. However, the fundamental principle behind how hard disks work is basically the same. A platter with a magnetic surface spins while an electronic component called a "head" is positioned over areas of the platter to read and write data. It is very similar to the old phonograph with an arm and a needle picking up music from the tracks on the record. Hard disks spin much faster than a phonograph record and the head does not actually touch the platter. However, the head is positioned very close to the spinning platter. Early hard drives would frequently suffer a mechanical failure that would allow the head to actually touch the surface of the spinning platter, which would cause a horrendous noise as the head was destroyed while it cut grooves into the platter. The term "crash" was a reference to the read/write head physically crashing into the rapidly moving platter.

Hard disk engineers now stack multiple platters together with read/write heads on both sides. Modern hard drives are smaller, they spin faster, the heads are closer, and there are many more heads in each drive. The electronics are also more sophisticated because the data is packed much closer together, which requires much more precision in the mechanism which positions the heads over the data. Modern drives have been engineered for greater reliability; for example, all hard disk devices are now sealed to prevent dust or foreign particles from getting between the heads and the platters, which was found to be the cause of many crashes in the early models.

Even modern hard disk drives occasionally suffer from the traditional crashes as described above, although less frequently on a per-drive basis. Instead of a loud crashing noise, you may hear a grinding or whining sound when a hard drive head actually comes into contact with a platter. While this sound is not as loud and dramatic as the early hard drive crashes, the sound is still considered quite disturbing to most of us. It's not the same disturbance as a car crash, but the sound of permanent data loss is no laughing matter for must of us.

These days, every hard drive failure is called a hard drive crash. There are many reasons why a drive will fail. Most of the reasons are electrical rather than mechanical. Instead of hearing a grinding sound, you may hear a clicking sound. This usually means that the actuator that positions your read/write heads is thrashing back and forth for some reason; perhaps it is malfunctioning or otherwise cannot find the data it is seeking.

Hard disk drives do not always fail suddenly. Sometimes there are signs that your hard drive is failing. You might notice that your computer has become very slow at opening an application that usually opens fast. Or you may find files or entire folders are missing. You may also find that it takes several tries to start your computer. These signals don't definitively indicate a hard drive failure is imminent. There are other causes for the same same symptoms. If you are lucky your computer or hard drive will give you signs that your hard drive is failing and you will have a chance to backup your files or replace the drive before total failure. However, more times than not, your hard drive will die unexpectedly and you will have no opportunity to save anything.

Fortunately, there are services that can recover most data when a hard drive fails. These services use sophisticated equipment and skilled technicians to disassemble your hard drive and extract any usable data remaining on the platters. The recovery operations are performed in clean rooms similar to the operating room in a hospital. These facilities are expensive to setup and operate so you will pay a hefty price to have your data recovered by one of these services. Most of us will consider the data lost and suffer the consequences before we pay for a recovery service.

You are well advised to backup your data frequently so that you will be prepared when your hard drive eventually fails. The cost of a single data recovery can easily exceed the fees of an online backup service for a few years. Backing up your data will not only save you money, but also save you time and heartache.

Laptop Data is Vulnerable

More than 50% of computer users claim that their notebook or laptop computer is their primary computer according to the July 2006 Consumer Behavior Report. Laptop computers are increasing in popularity for several reasons including the availability of wireless connectivity, increased performance and capabilities of laptop computers, and falling prices of quality laptop computer products. The amount of data that the average computer owner accumulates is rising dramatically. Dependency on computers and data to perform daily tasks and activities are also on a rapid increase. These trends may be a prediction of significant heartache for a lot of people.

While laptop computers pack plenty of power and capacity to perform almost all typical workstation computing tasks, they do have a noteworthy disadvantage that everyone should be conscience of. Laptop computers are much more vulnerable to loss or damage than desktop computers. The increased vulnerability is due to several factors; three of the more noteworthy are listed below:

1. Laptop computers are frequently stolen. More than 750,000 were stolen last year, according to some estimates.
   
2. Laptop computers are often dropped or otherwise mishandled. When a hard drive is spinning at 5400 revolutions per minute with read/write heads a few microns away from the surface, a drop or sudden impact can easily and permanently destroy your data.
   
3. Laptop computer components build up heat. Heat is the enemy of electronic parts, including hard drives. Laptop computer components are packed into a very small area and do not have the benefit of larger fans and ventilation system found in desk computers. In addition laptop computers are also more likely to have their ventilation systems blocked when they are used on laps or other soft surfaces. Heat build-up will dramatically reduce the life of a hard drive.

You should make every effort to protect your computer from theft or damage, but regardless of the measures that you take, events outside of your control will occur. While your laptop computer may be insured or covered by an extended warranty or protection contract, your data will not be covered.

The data on computers is often more valuable than the hardware itself. The only way to ensure that your data will not be lost is to perform regular backups. Laptop computers are often neglected in backup plans because they move around and are not consistently connected to backup devices. The best way to ensure that laptop data will be backed up consistently is by using an online backup system that will automatically backup the computer when it is connected to the network.

Online Backup Technology Can Handle Large Amounts of Data

A common concern with using an online backup provider to backup large amounts of data is the time required to transmit the data to the offsite storage facility. If you have ever tried to copy volumes of data over a T1 or Internet connection you know what I am talking about. This is one reason that some people believe that online backup services are primarily used for small amounts of data. Newer technological advances are making it practical to backup substantial amounts of data to online backup facilities.

The simplest of these technologies has been around for many years. Incremental backup methods only copy files that are new or have been updated since the last backup. Traditional media backup schemes typically require periodic full backups, in addition to incremental backups on a more frequent schedule. The periodic full backup is needed for two reasons; 1) the full backup along with all subsequent incremental backups were needed to restore all files in the backup set to their most current state, and 2) the periodic full backup is needed so that the incremental backup media consumed between full backups can be returned to the scratch pool. Online backup services do not require periodic full backups. Online backup services can assemble the most current backups of a set of files without the need to locate and mount media. A entire set of files can be restored at anytime as if a full backup were made as of the last time an incremental backup was run. Online backup services typically store files using online or near-line storage that make media scratch pools obsolete. With online backup services, only new files and modified files are actually transmitted to the offsite storage facility. Online backups typically transmit only a fraction of the amount of data required by a full backup, and yet maintain current backup copies of all files.

Advances in data compression have also contributed to the efficiency of online backups. Online service providers typically use the latest data compression techniques to reduce the bandwidth required to backup files. It is not unusual to see text files, spreadsheets, databases and other documents compressed to 15 - 20% of their original size. Executable files can usually be compressed to about 50% of their original sizes.

The more advanced online backup services utilize delta file technology to backup only the parts of large files that have changed. When a large file is changed, the delta file technology can determine which parts of the file have changed and which parts have not changed. The online backup system can then transmit only the changed parts of the file to the offline storage facility and combine those parts with the sections of the file that were previously backed up and remain current. Your first impression may be that the files may get corrupted since they are being backed-up in pieces. The technology is actually quite solid and the backup files are usually validated with various hashing and checksums to ensure that the file in the backup storage facility that was backed-up in pieces remains identical to the file on the computer that is being backed up.

The combination of incremental backup, data compression, and delta file technology make it practical for home users and businesses to backup significant amounts of data in an expedient manner. Online backup services offer several advantages over traditional backup systems and should be seriously considered. All online backup services are not equipped to handle large volumes of data. If your data backup requirements include large amounts of data, then research online backup providers who provide all three technologies; incremental, data compression and delta file technology.

When Is My Hard Drive Going To Fail?

“When is my hard drive going to fail?” I have been asked this question over and over again by many users. There is not one simple answer to this; however, through years of experience I have developed a few guidelines and recommendations to help in determining if it is time to replace the hard drive. I hope that you find them useful.

Hard drive manufactures do a grave disservice to IT professionals with their Mean Time Between Failures (MTBF) ratings. MTBF is the manufactures best guess based upon historical information as to how long your drive will last. MTBF make the issue of when drives will fail even more complex by providing a theoretical rating of their hard drives. It is common to see consumer hard drives rated in the 500,000 hour range. Now, there is only 8,760 hours in a year; so, a 500,000 hour drive would last me about 57 years! (Yea, right!!) Even if there was some way to simulate 57 years worth of usage on a new hard drive, does anyone actually believe that the technology in 60 years would work with the drive? These ratings from manufactures are so unrealistic that they are rendered almost useless. The only value that you can determine from the MTBF is that a drive that has a higher MTBF number typically is better and will last longer. However, this is not always the case.

So if the MTBF rating is “suspect” what does the consumer rely upon when selecting a hard drive? Look for the manufactures stated “Service Life” and “Warranty Length”. These are indicative of what the manufacture REALLY thinks of their drive. I would recommend that any consumer buy a hard disk with a stated service life of five years and a warranty of three years over one with a service life of three years and warranty of two years, even if the former has an MTBF of 300,000 hours and the latter one of 500,000 hours.

Be a smart consumer, do your research before you order your new hard drive. Hard drive manufactures provide data sheets and other technical information on their web sites. Do your research, make the comparisons, and make a good purchase decision. Your data will thank you!

Now that you know how to buy a good hard drive, you do not want to run that hard drive until it fails. As a matter of fact, you should get into the habit of replacing your hard drive periodically. Just as you have to replace the tires on your car and your filter in your air conditioner, you should also replace your hard drive. You don’t wait for a tire to blow out! You should not wait for your hard drive to fail!

There are three different types of drives that you should consider; laptop, desktop, and server. (Please note that the recommendations are for non-RAID systems. RAID changes this paradigm and is beyond the scope of this article)

Laptop drives should be replaced every 18-24 months.
Desktop drives every 36-40 months.
Server drives should be replaced every 48 months.

Replacing a drive is a very straight forward procedure, and there are numerous software products that will “clone” your old drive so that you do not have to reload all your software. While replacing your drives periodically will save your data, this procedure will not eliminate the need for a good backup system. In the real world, drive failures depend on many factors, including the operating conditions of the drive and how it is used. Unfortunately, luck is also a factor; so, keep those backups current!

Monitoring Your Backups

I hope that most of you are already backing up your data on a regular basis. Automating the backup process is the best way to ensure that you will have a backup copy of your data when loss or disaster occurs. IT professionals have been automating backup tasks for years. Even the simplest backup programs provide some way to schedule them to run on a defined schedule. Unfortunately there is one critical backup task that is not usually automated; the handling of backup media. Mishandled backup media, or neglected procedures for changing media often lead to failed backups and restores. You can't just schedule your data backup system and let it run automatically and then assume that everything is going to be OK.*

Data backup procedures need to be monitored. Even highly automated systems will encounter problems that require human intervention. Assuming that no news is good news could be a disaster waiting to happen. I have personally experienced many cases where backup jobs fail to backup data for weeks at a time unbeknownst to anyone. It can be a devastating experience to find out that your data backup has been failing at the time when you have lost your data. If your backup job has been automated and running quietly for months, you best check and make sure it is doing what you think it is.

An occasional checkup on your backup system is not good enough. It needs to be monitored daily. You can either check your backup logs every morning, or use some kind of monitoring system to alert you when there is a problem. Most of the professional backup systems will send alerts when backups fail or encounter errors. It is the small business and home users who are most vulnerable. Don't just assume your data is backed up.

*There is one type of system that may be fully automated and may even alert you to a problem. That is online backup. Some of the better online backup service providers will send you alerts when problems occur.