Billion-Dollar Flop: Air Force Stumbles on Software Plan

http://www.nytimes.com/2012/12/09/technology/air-force-stumbles-over-software-modernization-project.html?nl=todaysheadlines&emc=edit_th_20121209&_r=0



NEW YORK TIMES

December 8, 2012

Billion-Dollar Flop: Air Force Stumbles on Software Plan

By RANDALL STROSS

IN policy circles, problems that are mind-bogglingly difficult or impossible to solve, like global warming, are formally termed “wicked.”

For the United States Air Force, installing a new software system has certainly proved to be a wicked problem. Last month, it canceled a six-year-old modernization effort that had eaten up more than $1 billion. When the Air Force realized that it would cost another $1 billion just to achieve one-quarter of the capabilities originally planned — and that even then the system would not be fully ready before 2020 — it decided to decamp.

Silicon Valley sees its share of software projects that end unhappily. The most expensive failures, however, involve acquisitions of entire companies with software assets that turn out to be far less valuable than thought. Those can lead to stunning write-downs in the billions, as Hewlett-Packard has been forced to take recently.

But the Air Force’s software was not some mystery package, nor was it written from scratch. It was commercial off-the-shelf software, or “COTS” (the military can’t seem to resist any chance to use an acronym).

Installing COTS to run an enterprise is not a straightforward matter. The Air Force would have to make myriad adjustments to accommodate its individual needs, and in a military setting that would mean meetings and more meetings, unlike anything ever experienced in a Silicon Valley company. Still, it is hard to understand how the Defense Department blew a billion dollars before the plug was pulled.

The software initiative, called the Expeditionary Combat Support System, was supposed to manage logistics using software from Oracle. In 2006, the Air Force announced that it had awarded a $628 million contract to the Computer Sciences Corporation to serve as lead system integrator; its job would be to “configure, deploy and conduct training and change management activities” before the launch.

Four years later, in 2010, the Air Force said it had pilot programs under way at two bases. In remarks made at the time by Grover Dunn, the Air Force director of transformation, we can see just how unrealistic the project was: “We’ve never tried to change all the processes, tools and languages of all 250,000 people in our business at once, and that’s essentially what we’re about to do.”

Signs that such comprehensive change could not, in fact, be done “at once” were visible last spring. Last April, Jamie M. Morin, assistant secretary of the Air Force, testified before a subcommittee of the Senate’s Armed Services Committee about E.C.S.S.: “The total cost on the system is now over $1 billion,” he said, adding, “I am personally appalled at the limited capabilities that program has produced relative to that amount of investment.”

With the cancellation of the system last month, a spokeswoman said that the Air Force would continue to rely on its legacy logistics systems, some of which have been in use since the 1970s.

THE Defense Department says that the way the system was conceived was flawed. “We started with a Big Bang approach and put every possible requirement into the program, which made it very large and very complex,” says Elizabeth McGrath, the department’s deputy chief management officer.

E.C.S.S. was restructured many times, including three separate times in the last three years, Ms. McGrath says. “Each time, we chunked it down, breaking it into smaller pieces, focusing on specific capabilities.” But this was not enough to save the system, she says, because program managers did not succeed in imposing the short deadlines of 18 to 24 months that the department now requires for similar projects. Tight deadlines will certainly go a long way toward avoiding future billion-dollar fiascos. But much more needs to change before the department’s older software systems can be replaced.

In 2011, the nonprofit Institute for Defense Analyses, which performs independent research for the government on national security issues,issued a lengthy report on Defense Department software initiatives like E.C.S.S. It noted that modernization of the department’s software systems had been a priority for nearly 15 years, and that more than $5.8 billion had been spent through 2009 on large operational software systems, most of which were behind schedule. It recommended that the department suspend deployment of all new systems until reviews were completed.

The report cited many concerns, but the main one was a failure to meet a basic requirement for successful implementation: having “a single accountable leader” who “has the authority and willingness to exercise the authority to enforce all necessary changes to the business required for successful fielding of the software.”

I spoke last week with Paul K. Ketrick and Graeme R. Douglas, two institute researchers who were among the co-authors of the study. They said some small-scale operational systems in the Defense Department had drawn praise for successful unveilings, in the Navy and in the Defense Logistics Agency, for example.

“They got there because they had strong leadership who committed to the program and had the authority to make the changes necessary for success,” said Mr. Douglas. But “it’s rare that a single leader in the Department of Defense has the authority over the span of activities” affected by the systems, he said.

Pat Phelan, a research vice president at Gartner, the information technology research company, also calls attention to the difficult and time-consuming nature of decision-making within the department. She advocates empowering small groups to make necessary decisions, as is done in the private sector, but she does not expect the department to change. “That mind-set, that cultural shift, is not something I expect to happen in my lifetime,” she said.

The record of software modernization at the Department of Defense is discouraging. In Silicon Valley, software projects can run aground, but they don’t fail because of endless meetings and complex bureaucratic requirements, not to mention the constant need to be ready to fight wars. As Mr. Ketrick says, “Replacing the systems that run the Department of Defense is a wicked problem.”

Randall Stross is an author based in Silicon Valley and a professor of business at San Jose State University. E-mail: stross@nytimes.com.



Donations can be sent to the Baltimore Nonviolence Center, 325 E. 25th St., Baltimore, MD 21218. Ph: 410-366-1637; Email: mobuszewski [at] verizon.net. Go to http://baltimorenonviolencecenter.blogspot.com/



"The master class has always declared the wars; the subject class has always fought the battles. The master class has had all to gain and nothing to lose, while the subject class has had nothing to gain and everything to lose--especially their lives." Eugene Victor Debs