Backup Data Retention and File Access Frequency

I want to make you aware of a situation that caused a business to lose some important data. This particular business was using a very competent IT consultant who was using an online backup service to backup their data. While, there is no need to call out names, we can all benefit from their experience. Here is what happened.

The data that was lost was accessed relatively infrequently. At some unknown time, file system corruption occurred on the server. At some point an entire folder on the file system became completely inaccessible as the directory structure was corrupted. The corruption spread slowly until it eventually effected some files that were needed.

The data on the computer was diligently backed up every day. The folder that was missing due to corruption was not backed up because as far as the OS was concerned it didn't exist any longer. Online backup systems generally retain files that are deleted or no longer current on the backed up computer for a predefined period of time. When a user deletes files from disk, the OS typically removes the corresponding information from the directory structure for the volume. So when corruption occurs that causes information to be lost from a directory structure, the corresponding files are deleted as far as the OS or any other programs are concerned.

It just so happened that in this case no one realized that the files were missing for a period of time. The online backup system retained the missing files according to the retention policy, but by the time someone noticed that they were missing, the online backup system had also deleted them from the backup storage.

The lesson that we should all take from this is that our backup retention policies should be longer that the typical frequency of use for the files being backed up. In simple terms; if you only use a particular set of files once a quarter, then your backup data should be retained for more than a quarter.

I typically use more information than the frequency of use to determine how long backup data should be retained. There are lots of other factors that should be considered. One such factor would be compliance with laws and regulations such as the Sarbanes-Oxley Act of 2002. If you haven't checked recently, then I encourage you to review your backup data retention policies to make sure they meet your needs. Don't forget to consider how long it may be before you realize that important files are missing.