Restore Capacity of a Network Backup System

A reader of this blog who is setting up his own online backup system posted a comment about handling multiple large restores simultaneously. In the event of a major disaster, such as Katrina or 9/11, there may be many computers in multiple locations needing to restore data. If the data is backed up to an offsite backup facility over a network or Internet, how do you plan for restore capacity?

The simultaneous restore capacity is mostly a function of the bandwidth availability at the backup site. The offsite data backup facility should be located in a data-center with significant bandwidth capacity. Most commercial online data backup service providers have their equipment located in high-bandwidth co-location facilities. The end-user locations where the data is being restored will typically have less bandwidth than the offsite backup facility. A business office or temporary location may have a T1 connection or possibly DSL or cable modem connection, which will have far less capacity than the offsite backup location.

For example; if the offsite backup location has 100mb/s of bandwidth capacity and the end-user locations have 3 Mb/s of bandwidth each. Then we can roughly estimate that it would take 33 end-users to saturate the bandwidth at the data-center. If all 33 end-users were actively transferring data, then each would be able to transfer 22.5 MB per minute. More than 33 end-users could simultaneously restore data, but the transfer rates would be less than 22.5 MB per minute. This example does not take into consideration transmission overhead and network congestion, which would certainly lower the performance of each end-user's restores, which would make more room for more simultaneous users.

The number and speed of the simultaneous transfers is probably not the most important question to be answered. There are other ways to look at this that might make more sense. After a significant disaster, if multiple users needed to restore data, the fact is that a data center with 100 Mb/s of bandwidth capacity could restore about a terra-byte in 24 hours. The terra-byte in day could be served to one user or 500 users. So if the backup server contained a terra-byte of data that needed to be restored, it would take approximately 24 hours for all of the users to restore all of their data.

The disk storage systems at the data center are rarely the limiting factor in backing up or restoring large amounts of data over a wide area network. A raid 5 array attached to modern servers can easily keep pace with the amount of data being transmitted over a wide area network. It is possible, however, that the server resources such as cpu and memory could be over utilized. Secure Socket Layer (SSL) could significantly raise the CPU requirements needed to process large amounts of data. If the CPU or memory of the servers are over utilized, then data will be served at a slower pace, reducing the total mount of data that can be restore in a given period of time.

When planning an offsite data storage facility that will be used over a network, available bandwidth is the primary resource to plan for. Disk space capacity can be determined by analyzing the amount of data to be backed up and retention requirements. It should be noted, that redundant disk configurations are essential to a good disk based backup systems. The actual configuration of the disk storage system can influence performance. The servers and memory requirements will be a function of the server software, but should not be a limiting factor with modern hardware.

Hide Your Data in Your Mattress

Do you hide your money in your mattress or bury it in mason jars in your backyard? If that is your idea of keepking your money safe, then you must have been in a coma for a couple of hundred years, or you may be a drug dealer. If you make an honest living and pay your bills, then you probably keep your money in the bank, right?? Well, I do know a few people who take their paycheck straight to the check cashing place across the street from Walmart as soon as they get off work, but that's not what this is about. A lot of otherwise smart people are stashing their backup data in a desk drawer or somewhere in the house or office as if that is the smart thing to do. I bet you are saying; "not me, I have a safe..." Okay, I also bet you keep your money in the bank, not in your house or in that safe.

When your money is in the bank, you don't have to worry about it being lost to fire, theft, or other disaster that might make affect your home or business. You might be thinking; who would want to steal my data? and if they did, I would still have a copy... Maybe, what if there was a disaster in your community and looters raided your home or business, and took your computers, your furniture and everything else that was or wasn't tied down. It happens, as I am sure some people in New Orleans can attest to. Your money that was in the bank, is still in the bank and you can still get it. So treat your data with the same respect.

Using an online backup service to backup your data and keep it offsite in a secure data storage facility is roughly equivalent to keeping your money in the bank. Keeping your data in your house or office is about as smart as keeping your money in your mattress. It may have been a good idea at the time, but the times have changed. You might be thinking that using an online backup service is expensive. A typical bank will probably take more of your money in fees and interest than an online backup service will cost you. Online backup services are becoming very mature and robust, most store your data in encrypted form and replicate the data to remote data centers. The technology is solid and most of us have plenty of bandwidth in our Internet connections to backup our most valuable data.

You may not buy into this online backup thing now, but you should, there is no reason not to. In a few years it will be as ubiquitous as cable TV and cell phones, and the thought of backing up your data to a tape and keeping it in your closet will seem like something from the civil war days.

Old Backup Tapes

I was cleaning out the storage room in my basement when I came across a big box of diskettes that I used to backup my PCs in the eighties. There must have been 300 diskettes in this box. Most of them were clearly labelled with something like; "c: drive - 3/10/86". These diskettes were starting to bring back memories of my first computers and how I spent countless hours writing programs and utilities. My first instinct was to put them back on the shelf, and then I remembered that I was cleaning out the basement to get rid of stuff, not to keep stuff I don't need. Since there was no chance that I would ever need these backups, I put them in the trash pile.

A few minutes later, I found a box of about 60 of the old 20MB QIC tapes. Remember those, sloooooow is a good description. As I recall, it would take about two or three hours to backup enough data to fill up one of those tapes. Now, I am thinking to myself, why am I keeping these old tapes. Then I find another box of 2120 tapes with a capacity of about 120MB each, again clearly labelled backups from the late eighties. I recall paying $15 - $20 apiece for these tapes and there must have been about 40 or 50 of them. Next, I run across a box of 4mm backup tapes, nicely labelled as backups from the early nineties. These held a whopping 2GB each. I certainly can't justify taking up space and cluttering my basement with any of these old tapes. I don't even have the tape drives that I could use to restore them. I also found my more recent DLT tapes and the AIT 8mm tapes that I used just a couple of years ago. It might make sense to keep those DLT and AIT tapes because I actually have tape drives that can restore them.

I must be a real data pack-rat for keeping all of those old backup tapes. The trash pile is growing and so is the available space in basement storage room. I should have discarded this stuff a long time ago. I started hauling the trash pile up stairs so I can get it out to the garbage receptacle. Then it dawns on me that I might be making a big mistake. This stuff could be a goldmine for a high-tech thief who wants to steal my identity. Actually it probably wouldn't take a genius to find some old hardware at a flea market that could be used to get my financial records from the past 20 years, not to mention intellectual property, and who knows what else. What about old letters, source code... hmm, maybe I shouldn't just put these tapes and diskettes out in the garbage.

I need to make sure they are destroyed and the contents cannot be read. Maybe, I should smash every one of them with a hammer. I decided against that because of the mess it would make. Besides, I am not convinced that I would destroy them thoroughly enough so that no data can be retrieved from a single one of them. Well, I can burn them. That would do it. Only problem is, I don't have a place to burn things. I need a big barrel like the ones you might see at a construction site where they burn scrap wood to keep warm. I don't have a barrel and even if I did, I doubt that the city would take very kindly to a barrel in front of my house billowing out thick black smoke from burning plastic. Can you imagine the smell of burning tapes?

I guess its back to the basement with these boxes of tapes. Oh, now I remember why I haven't thrown these out before. If you check back here in about five years you will probably see another post just like this one when I decide to clean out my basement again. But next time, I won't have a new box of tapes because I am using Online Backup for most of my backup needs now.

4 Essential Steps to Effective Data Backup

Backing up your computers hard disk drive is really not a complicated task. There are a number of ways to copy your data to a backup device that will provide a basic level of protection against hard disk failure. Ensuring that you will be able to recover from a data loss is an entirely different matter. Most people who rely on a simple backup to a USB drive, CD, or DVD, don't have sufficient protection from common data loss events. It is not a question of IF, but WHEN will your next loss of data occur. Follow the guidelines in this article to ensure that your next data loss will not be permanent, and minimize the loss of time and money.

Plan

One of the first things you must consider is what data needs to be backed up and how often. Think about the data that you would need to recover first if your computer was completely destroyed. Then think about how quickly you would need to recover that data to minimize loss of productivity or money. Be realistic; instant recovery of all data on your computer will require expensive resources and procedures. For example; you may need to recover your accounts receivable data within a few hours to avoid financial losses. On the opposite end of the urgency spectrum; you will probably be happy to get your family photos back within a few weeks.

Consider how often your data needs to be backed-up. That usually depends on how often the data changes. Data that changes daily usually needs to be backed up daily. Data that rarely changes can be backed up less frequently. But keep in mind that even though some data changes infrequently, you may need to back it up soon after it changes. Monthly backups can expose you to loss of up to month's worth of updates. Some data can easily be rebuilt without a backup, e.g. there is no need to backup your Google Desktop index because it can be rebuilt with little consequence.

Automate

Even the best backup plans fail when they are not executed. Any reliance on human effort is an opportunity for problems. Unplanned events and a myriad of other reasons can be attributed to backup tasks being delayed or skipped. Automating the backup processing is the only cost-effective way to ensure that backups will run on a defined schedule.

Many backup plans require at least some minimal human intervention, such as changing a tape or other media. This is often the reason a backup fails; the human did not perform the function as required. Media libraries are available that will eliminate most of the human interaction required to change media for each backup, but these devices are expensive to purchase and maintain. One of the best ways for a small business or home computer owner to fully automate is with a good online backup solution.

Monitor

Even the best backup systems will occasionally fail. Once your backup plan is automated, it should be monitored. Alerts should be sent to responsible people when a backup encounters an error. Even better, alerts should be sent when a backup job does not run as scheduled. In addition to the error alerts, reports should be produced and reviewed with some frequency. Most good backup software includes built in alert monitoring, some will also generate periodic reports. Even with automated alerts and reporting, you should still mark your calendar and periodically review your backup and recovery plans to make sure they are not only working as planned, but also that they are adequate.

Test

The last essential step is to periodically test your ability to recover important files. Your backup system may allow you to restore a file to an alternate location. You may also be able to rename or move an existing file, and then restore it from your backup system. Don't just restore the file and assume all is well, you should also open the file in the application that you normally uses it. For example; if you restore a QuickBooks file, then open the file in QuickBooks and make sure it is usable. Testing of backups by restoring data is an often neglected task. However, it is very possible for a file to be backed up, but cannot be restored for various reasons. You may find that the reason you can't restore a file is due to security or other easily correctable issues. Finding out about these types of restore problems before you have a disaster is much better than finding out about it when you are desperately trying to recover your files. Other problems with restores may be more severe; corrupted files, missing files... find out about these backup and restore problems before you have a disaster and you can correct them before they cause you serious problems.

The backup functionality of backup and recovery software is used frequently and bugs will likely be exposed quickly. The restore functionality may be used much less frequently and bugs can go undetected until you or someone else attempts a restore. Follow this advice and make sure you can restore your backups. Test your restore process periodically. Specifically test your restore process when things change; backup software version, new hardware, operating system upgrades, security software upgrade, or anything else that has a remote chance of affecting your backup and restore.

Everyone is exposed to data loss to some degree. Follow the 4 essential steps in this article and you will be prepared.