Radio Free Agile

One of the striking things about the jaw-dropping cock-up at Heathrow Terminal 5 is that it was preceded by Six Months of Testing.

On the surface, it kind of sounded like they were doing the right thing. For instance, on 22 September 2007, the BBC reported that Heathrow Airport is calling on 15,000 volunteers to help test out facilities at the new terminal five building.

The report goes on to say that “… the trials had been designed using lessons the company had learned from the security and baggage delays faced by passengers at other terminals over the past few months.” And furthermore, “We want to make sure the terminal is safe, secure and works like clockwork before we open, and we need volunteers to help us do that.”

Well, that’s all right then. We’re off to the idyllic dream described by British Airways:

“At London Heathrow Terminal 5 we’ve created a natural, logical journey that’s so calm, you’ll flow through. It shouldn’t take long to get from Check-in to Departures. Transferring and arriving are just as simple and calm. Spend the time you save enjoying the excellent range of shops, cafes and restaurants. Or simply relax and be wowed by the world class architecture. ”

This status report is readily corroborated by satisfied customer reports:

“I will never ever fly in to Heathrow again, rude staff, chaos and confusion abound. Not a good first impression for overseas visitors and depressing for regular travellers.”

“My daughter has travelled home to Newcastle after visiting me in New Zealand, setting off on Weds afternoon – it’s now Friday afternoon here and she is still not home. BA have had to put her in a hotel in London (eventually, after a lot of heartache and tears) as there is no flight to Newcastle after travelling from the other side of the world. What are they doing??”

Hm. It all seems a bit sub-optimal. So how did the idyllic dream become a stressful nightmare?

Is it possible that testing was adequate, they did the best that they possibly could, and it was just rotten bad luck that plagued the opening? Or “teething problems” as the matter has been spun in the press?

The implication of the phrase “teething problems” is that everything was well-prepared and ready to go, but a few unanticipated things went wrong– as they will, you can’t anticipate everything, guv– causing a few minor problems that were quickly sorted out. The phrase seems a bit of an understatement given the impact that the cock-up has had – financial impacts to business and individuals, and public relations impacts for the businesses involved and the country. Our national self-esteem seems to have taken another body-blow.

So was the testing done at Terminal 5 inadequate? Some clues are to be found in the sorts of things that went wrong. It appears from current media reports that there were two main failure points:

1. The parking system – staff didn’t know where to park, wasted time trying to figure it out, and so arrived at their posts late. There also seem to have been staff coordination problems further down the line, with people being unsure where to go when they reached the building.

2. The lack of staff caused (among other things) baggage to be unloaded from the conveyor belts slower than anticipated, which caused the whole system to seize up, apparently because the conveyor belts clogged with unremoved luggage.

The parking problems seem to indicate that peripheral bits of the system were not adequately tested. The approach to the terminal was seemingly neglected in favour of testing the in-terminal systems. In a software project, this could be analogous to a fantastic system hidden behind a disastrous registration process.

It sounds as if many of the staff arrived at T5 for the first time on the day… they had never participated in a dry run. The testing with mock passengers apparently only used a small subset of staff. So it appears that the first time that all components (including the human ones) of the system ran together was the day that Terminal 5 opened for business! On a software project, this would be analogous to leaving integration to go-live day… all the parts work separately, and we’ve tested some of them together, so what can possibly go wrong?

The fact that the baggage system seized up under the circumstances indicates another type of inadequate testing. No doubt they ran a lot of test baggage through the system before the opening date. They may have even cranked it up to see how fast it could go before it broke. However, there were obvious potential points of failure in the system- one of them being the humans unloading at the end– and they don’t seem to have tried tweaking these to see what would happen. In a software system, this is analogous to assuming that the data coming into the system will always be correctly tagged and formatted.

Why didn’t the system slow down when it wasn’t being emptied fast enough? Did staff even know that failure to unload at a certain rate would cause the whole thing to go tango-uniform? Did the managers know?

Perhaps they would have done well to consult classic television footage for an example of what happens when a conveyor belt goes faster than the people can…

As everyone knows, criticising stuff is easy, and fixing stuff is hard. It’s easy to figure out what should have been tested after it’s failed.

But if this had been run as an agile project, would the crisis have happened? It’s impossible to say, of course, but it’s interesting to think what might have been done differently on an agile project.

First, the design of the terminal would have been driven by user stories. There should have been a story, for instance, about a baggage handler arriving for work by car — staff are users too. There should have been a story about dealing with a staff shortage (perhaps due to a strike or flu epidemic,) so that all the Terminal 5 systems would have been able to continue to operate (presumably more slowly) when staffed at 1/2 strength.

Second, acceptance tests would have been designed prior to the systems being built, and they would have centred on the user stories. It’s possible that the crisis wouldn’t have happened if only one of the failure points had been anticipated and tested. If there had been an acceptance test for running the baggage system at 1/2 staffing levels, the non-adaptive property of the system would have been detected sooner. Or perhaps the existence of that acceptance test would have prevented the system from being designed in that non-adaptive way in the first place.

Agile dictates integrating and testing the whole system as early as possible. No doubt there are many obstacles to doing this on a large-scale construction project like this. Expense is almost certainly one reason why the system ran in full for the first time only on go-live day. However, evidence suggests that the cost of not doing integration testing is higher. I haven’t yet heard any estimates of the cost of the Terminal 5 debacle, but they must be staggering.

One of the strengths of agile is to expose problems sooner so that they can be addressed and fixed sooner. It does this by making it less possible to ignore them. The emphasis on how the system will be used (provided by user stories) and what it has to do (acceptance test-driven development) keeps the focus where it belongs.

How many software projects have suffered the same fate as Heathrow Terminal 5? They just tend to be a bit quieter about it.

It’s all well and good to engage in a bit of schadenfreude, but there’s also a big learning opportunity here for those that want to take it.