News

posted on 02 June 2008 08:15

A Federal Aviation Agency database was corrupted by a hard drive failure and the bad data written to a backup copy, causing a near day-long outage.

The NOTAM (Notice to Airmen) database is used to provide pilots with pre-flight airport and equipment and security information (sample pictured left¹). On May 22nd FAA staff had a replacement server for the end-of-life server providing NOTAM information but did not install it fast enough.

During a database update a hard drive on the server failed and the database was corrupted. Before the admin staff realized what was happening a backup copy was taken and the database errors propagated into the backup.

The database failed and still did not run properly when the hardware was replaced. That was when the corrupted backup data was detected. Alltogether it took 20 hours for the outage to be dealt with.

While it was ongoing pilots were given NOTAM information by air traffic controllers or through a specially set-up web site.

The FAA says that at no time was passenger safety compromised.

It is apparent that the NOTAM server did not have RAID facilities and that the backup procedures had not been designed to properly cope with a disk drive failure. It is also apparent, with hindsight, that the the replacement hardware for the Sun server involved was not installed fast enough by the FAA.

That's three mistakes by the FAA which, ideally, should not have happened.

^{[Paul Roberts, news editor.]}

 ^{1. Picture from www.av8safe.org/.}