March 20, 2007

Format C: /s /v

DOH! From Fox News:
Computer Tech Accidentally Erases Info on Alaska's $38 Billion Oil Fund
Perhaps you know that sinking feeling when a single keystroke accidentally destroy hours of work. Now imagine wiping out a disc drive containing an account worth $38 billion.

A computer technician at the Alaska Department of Revenue deleted applicant information for an oil-funded sales account — one of state residents' biggest perks.

While reformatting the disk drive during a routine maintenance check, the technician mistakenly reformatted the backup drive as well and, suddenly, all the data disappeared.

A third line of defense — backup tapes that are updated nightly — were unreadable.

"Nobody panicked, but we instantly went into planning for the worst-case scenario," said Permanent Fund Dividend Division Director Amy Skow, about the computer foul-up in July that ended up costing the department more than $200,000.

Nine months worth of information concerning the yearly payout from the Alaska Permanent Fund was gone: some 800,000 electronic images that had been painstakingly scanned into the system months earlier, the 2006 paper applications that people had either mailed in or filed over the counter, and supporting documentation such as birth certificates and proof of residence.

The only backup was the paperwork itself — stored in more than 300 cardboard boxes.

"We had to bring that paper back to the scanning room, and send it through again, and quality control it, and then you have to have a way to link that paper to that person's file," said Skow.

Staff working overtime and weekends re-entered the lost data into the system by the end of August.

Last October and November, they met their obligation to the public and a majority of the estimated 600,000 payments for last year's $1,106.96 individual dividends went out on schedule.

Former Revenue Commissioner Bill Corbus said no one was blamed in the incident.

"Everybody felt very bad about it and we all learned a lesson. There was no witch hunt," said Corbus.

According to department staff, they now have a proven and regularly tested backup and restore procedure.
Sheesh -- not testing and re-testing the backup and restore procedure is insanity. Just because a tape runs and just because the backup software said the procedure went alright does not mean that you have a successful backup. Test and re-test -- once a month at the least, weekly if you have the time and hardware... Posted by DaveH at March 20, 2007 12:55 PM
Comments
Post a comment









Remember personal info?