As American Eagle Outfitters learned in July, even if you do everything right to ensure you have disaster recovery and business continuity plans in place, Murphy's Law sometimes takes over. And problems can be compounded if you rely on an outsourcer for disaster recovery services.
The multibillion-dollar clothing retailer suffered an eight-day Web site outage because its Oracle backup utility failed -- and then an IBM disaster recovery site wasn't up and running as it should have been, according to a report from StorefrontBacktalk.com.
IBM did not respond to requests for comment on the outage. American Eagle did not dispute StorefrontBacktalk.com's basic account of what happened, though a spokeswoman said a few details about the incident were incorrect.
According to Evan Schuman from StorefrontBacktalk.com, which monitors retail Web sites, the outage began with series of server failures.
Schuman, who said he spoke with an unnamed IT source at American Eagle, said a storage drive failed at an IBM off-site hosting facility. That failure was followed by a secondary backup disk drive failure. Once the drives were replaced, the company attempted a restore of about 400GB of data from backup, but the Oracle backup utility failed, possibly as a result of data corruption. Finally, American Eagle Outfitters attempted to restore its data from its disaster recovery site, only to discover the site wasn't ready and could not get the logs up and running.
"I know they were supposed to have completed it with Oracle Data Guard, but apparently it must have fallen off the priority list in the past few months," the source told Schuman.