Software limits exposed in air traffic outage

The outage of a high-profile air traffic control system is exposing the difficulties of managing the ever-expanding software networks keeping the country’s critical infrastructure afloat.

On Saturday, a major component of the federal government’s high-tech upgrade to its airplane navigation system went down for several hours, stranding thousands of passengers and causing the cancellation of hundreds of flights.

{mosads}The system, known as En Route Automation Modernization (ERAM), was apparently felled by a software upgrade gone awry. ERAM is part of the Federal Aviation Administration’s (FAA) attempt to convert to a satellite-based flight tracking system known as NextGen.

NextGen will help route flights more efficiently, saving fuel costs and travel time. It will also make the aviation industry reliant on even more complex, Internet-enabled systems.

The same is true at most critical infrastructure industries. Many sectors of the economy — from finance to utilities to defense — are dependent on swelling networks.

Building unassailable software to run these intricate networks is becoming an untenable task, security experts say.

“It is a tremendously complicated problem,” said Jeff Williams, chief technology officer at Contrast Security, which tests for vulnerabilities in networks. “You can’t test everything.”

Software engineers have always conducted extensive testing before rolling out upgrades. For years, those tests could reliably predict the situations the new software would encounter.

“Now that’s essentially impossible,” said Tim Erlin, director of risk strategy at Tripwire, which monitors networks for malicious activity.

The increasing complexity and connectivity of systems means software now “tends to run into more things it didn’t expect,” Erlin said.

At some point, developers just have to put it out there and hope for the best.

“So every once in awhile they make a mistake,” Williams said. “And when it’s critical infrastructure, it can cause real big problems.”

When it’s the air traffic control system, “Then it can have really serious, real-world effects,” he added.

Wall Street experienced a similar problem in July, when a software update malfunction caused a rare trading shutdown on the New York Stock Exchange for roughly four hours.

“When one thing goes awry, the downstream effects, or the cascading effects, often are unexpected and spiral out of control very quickly,” said Jeff Schmidt, a pilot and CEO of JAS Global Advisors, a security consultancy for government and critical infrastructure firms.

Saturday’s “Flypocalypse” also comes as the aviation industry is battling a deluge of cyberattacks.

While there’s no indication that Saturday’s incident was tied to a security breach, in the last few months hackers have infiltrated the U.S. air traffic control system, forced airlines to ground planes and potentially stolen detailed travel records on millions of people.

Security experts explained that software updates can sometimes introduce network flaws, creating one more area of concern for the FAA as it carefully integrates the NextGen system.

Tripwire’s Erlin credited the agency for it’s “very slow and very cautious” rollout thus far, especially of the ERAM program, which debuted this spring.

But Erlin and others cautioned the public likely hasn’t seen the last misstep in the air traffic system upgrade.

“This is going to be a continuing problem,” said Contrast Security’s Williams.