Radio Free Agile

In our recent MP expenses scandals, I have one thing to say -

Porn films, duck houses, employing relatives and providing your offspring with free luxury housing in Central London may be all well and good, but none of those will aid in fending off the howling mob when they come calling ’round with pitchforks and torches, as they sometimes do under such circumstances.  A well-maintained moat, on the other hand, seems a very sensible investment.

Douglas Hogg gets my vote for Most Practical Abuse of Taxpayer Money.

I am going to be reviving this blog in the coming months.

Commenting is moderated, of necessity because of deluges of spam.  Apologies but the blog would be unusable if I let everything through!  If you want to comment, send me an email with a couple of sentences about why you are interested, and I’ll add you to the allow list.

Talk to you later!

I recently had the opportunity to travel to the US South on business.  The area to which I travelled was a suburb of a well-known American southern city.  I was absolutely appalled by what I found… it was a caricature of itself.  Everything that the critics say about the US is true…

First, let me say that all the people were nice.  In fact, they nice you to death!  In several occasions when dealing with service people I was caught out by their expectation of a bit of chit-chat prior to getting down to business… my London ways expect a bit quicker traversal of the mutual smiles and then straight to what is required.  The friendliness of the American southerners is not in doubt.

BUT… you have to drive absolutely everywhere.  There is no option.  Not only that, but you have to drive the largest vehicle that you can possibly find… both for reasons of prestige and for reasons of survival.  Some comments I made suggesting alternate means of transport – perhaps a bicycle, or even a motor scooter – were met with cries of disbelief – “They will kill me!”

Who are “they?” They are the ones who drive EVEN BIGGER BEHEMOTHS than you do… backed up by the motor companies that are more than happy to feed this need.  There will always be something bigger, something that can keep your precious children safer.  Feed the hysteria.

Air conditioning is absolutely essential.  There is no such thing as fresh air, to which I attribute my getting sicker than a horse on this trip.  Recycled air in closed buildings, that is all you get.

The most astonishing thing was that in the bright, shiny, air-conditioned office park where all this was located, there was no where to get a bite to eat.  Astonishing as it may seem… after driving a behemoth to work, you are not yet finished ravishing the environment!  If you haven’t managed to bring a lunch to work, you have to climb up into said behemoth and DRIVE in order to get the simplest meal…

It satirises itself, really.

My friends, everything that they say is true.  It is a shame that so many people in England aspire to have the same car culture that exists in America.  Look at what has happened there… surely there is a better way for us.

The Software Practice Advancement (SPA) specialty group of the BCS is presenting a series of talks this autumn. The theme is Software Practice – The State of the Art and the talks cover a variety of topics from technical to management.

We’ve chosen a great lineup of dynamic speakers well-recognised in their fields.

The calendar is:

Software Practice: the State of the Art
BCS SPA Autumn Series 2008

See full details on talks and speakers.

And so it came to pass, that the Developer did say to the Project Manager, “I am in need of a new technology, which will make my development proceed more quickly.”

And the Project Manager asked some questions of the Developer, but they were not the right questions. “Dost thou really need this new technology?” asked the Project Manager of the Developer, and the Developer did answer, “Oh, yes, for it will make my development proceed more quickly.”

So the Project Manager did approach the CIO, and did ask to use the new technology, for it would make development proceed more quickly. And the CIO did say unto the Project Manager, “Art thou mad, that thou wouldst further disturb my sleep by introducing a new technology that must be maintained?”

The Project Manager returned to the Developer and said, “Thou mayst not use this new technology, for it will disturb the sleep of our CIO.” And the Developer did grumble and comment upon the intelligence of the Project Manager.

The Project Manager did withdraw in contemplation, and having thought upon the matter returned to the Developer with a new question. “Why dost thou think that this new technology will make your development proceed more quickly?”

And the Developer did reply, “The new technology will make my development proceed more quickly because it maketh it easier to imitate the behaviour of the old system.”

And the Project Manager did begin to catch a clue, and asked of the Developer, “But is the imitation of the behaviour of the old system truly a requirement?”

The Developer did shrug. And the Project Manager and the Developer realised together that they could not answer this question alone. So they went together to their Customer, and they did ask of their Customer, “Is it a requirement that the new system must imitate the behaviour of the old system?”

And the Customer did reply, “No, it is not a requirement, and in fact I’d prefer that it didn’t.”

A few weeks ago I wrote, based on reports in the media, about the problems at Heathrow Terminal 5 and how they seemed to be linked to lack of adequate testing.

The BBC reported today:

[British Airways Chief Executive Willie Walsh] admitted that the airline had compromised on testing the new building and said that BA staff were not given enough training.

“Our staff weren’t as familiar as they should have been and that impacted on their performance,” Mr Walsh said.

“If we were to do it again, we would do things differently,” he added.

[...the Chief Executive of airport operator BAA, Colin Matthews] added that the baggage system had been tested at full load about 20 times.

But admitted that “the totality of the testing regime did not adequately replicate the reality of the first days of operations”.

[...]

BAA’s non-executive chairman, Sir Nigel Rudd, said he was “bitterly disappointed” about the opening of the terminal. [...]

Sir Nigel added: “We have apologised, but my view of the matter is that there were a number of problems that might have been foreseen, but none that would have led to the stopping of the opening of the terminal.”

He added: “In a project as huge as Terminal 5, the problems had a cumulative effect.”

See the full BBC story.

Much of the battle to avoid software development practices that are doomed to failure from the outset stems from the inability of people in positions of responsibility to understand the management of uncertainty and probability.

Many software development practices are geared towards the pretence that uncertainty can be eliminated, if only you think about it hard enough in advance. Surely, if you spend enough time estimating, then uncertainty will be eliminated! What do you mean that you still want contingency? I want control over that contingency, you shouldn’t need it. Ad infinitum, ad nauseum.

As they always seem to put it in school reports: “Must try harder.” Screw up your courage; you’ve already screwed up everything else!

Quantum physics is a bit uncomfortable so we shall pretend it doesn’t exist and stick to the classical. In order to be an effective manager I need more precise information, more control, more transparency.

Never mind that the precision is fiction, control is an illusion, and the transparency in these circumstances is pure fantasy.

I now have the privilege of working in an industry (insurance) which is in the business of managing uncertainty. Our IT organisation leaves much to be desired– which we’ll be tackling over time– but the business is top-notch. (That’s why I joined.) And the business makes money from its ability to manage uncertainty.

So introducing agile to the business is much easier than introducing agile to IT. My business customer gets it. He looks at what we are doing and draws analogies to the techniques that he uses to manage his risk portfolio. In fact, we are learning a trick or two from him.

All the stuff that one usually has to fight battles over – why can’t you tell me the precise day when this project will be finished, why can’t you tell me the hour and the minute? Why isn’t your project plan developed to the minute level of detail that I would like? What percent complete are you? Where are the detailed and unreadable documents specifying the fiction that we believe we are supposed to implement before we really understand it?

All that is gone with this customer. He gets it. His business is managing uncertainty, so when we say that we will initially paint with a broad brush until we have more information, he tells us that this sounds sensible. Contrast with many who would scream and panic that they want more control and more certainty. They can’t have it. They can have an illusion or mirage of it, if they really want it, but it won’t change reality. The reality is that I can’t tell you in advance which horse will win. And if I could, I wouldn’t need this job.

To our customer, we are not doing something extraordinary. We are doing something sensible that is akin to what he does every day to manage his business.

May we live up to his expectations.

After several people mentioned to me that they don’t like having to register and login to comment, I’ve found the configuration option that requires registration to comment and switched it off.

Spammers will be dealt with using methods that I would consider unethical if applied to poisonous spiders.

Please comment, it’s a bit lonely out here with no feedback!!!!

As I gradually adjust to a new job and have new projects, a story from my past sprang to mind.

When I was knee-high to a grasshopper and learning to play clarinet, my teacher asked me what was my favourite piece of music to play. I named a piece (can’t remember what it was now.) She said, “no, it isn’t.” I was a bit taken aback – how dare she say what my favourite piece to play was or wasn’t?

She said, “Whatever you are playing right now is your favourite piece.”

I do believe that she was right.

You hear it everywhere in the Agile and Lean Software Development Community: Embrace Change.  Change is not the Enemy.  Change is inevitable, what matters is how you deal with it.

So how well or poorly do we deal with it?

I recently had the opportunity to play the Product Owner role in an XP/Scrum team.  The product we were making was a consumer-facing mobile information service aimed at commuters in the UK.  There was a big fuzzball of uncertainty around the billing system that we would use.  The UK mobile industry was in the process of introducing a new standard, but vendors weren’t quite there yet with solutions, and the mobile industry was also doing other things that seemed to be contrary to their new standard.  In other words, business as usual.

All of this plus some other business factors led to me making a decision that we were heading down the wrong path, and directing the team to do what could only be described as an about face in what they were building for billing.  They had already spent a fair amount of time going in a direction that, through no fault of theirs, turned out to be the wrong one.

On Planet Perfect, they would have responded to this with: “Oh good!  Now we have a chance to build something closer to what the business actually needs!”  This is what all the Agile gurus say that we should be doing.

Down here on Earth, where I live, that’s not exactly what happened.  

What happened was more akin to what I’ve done when in their shoes many times.  And this is despite walking around intoning “Embrace Change” with the best of them.
I mean no disrespect to the development team- they were (and are) a fantastic team of people who generally can find a solution to anything.

But they reacted with something like: “How dare you mess up our lives?  We’ve spent our valuable time working on doing billing the way you said you wanted it.  Now you want something else.  Why can’t you make up your stupid mind?”

My reaction to this situation was “Oh good! Now I have a chance to understand the customer’s point of view more closely when this sort of thing happens.” 

Or more precisely, “Hell’s bells, is this how I’ve made my customers feel when they have to change something for the sake of the business?!!! Small wonder that they don’t trust IT!”

This led to some reflection on the problem of change in software development.  It is, I believe, a fundamental problem in the IT industry.  I distinguish the IT industry from Computer Science.  Computer Science deals with problems such as “is P=NP?”, the Halting Problem, and how to make more efficient compilers.  The IT industry deals with how to use computing technology to solve problems of business, government and other human activities. 

In short, the IT industry has to deal with Hundred-Million-Year-Old Legacy Liveware, otherwise known as “Humans.”  Hence we are told by our thought leaders to Embrace Change.

I’m reminded of Russell’s Paradox – a problem in set theory that came up at the end of the 19th century that in a nutshell goes like this: Consider the set S that consists of all sets that are not members of themselves.  Is S itself a member of S? 

A minute or two of thought will reveal the paradox.  If S is a member of itself, then it isn’t a member of itself.  If S isn’t a member of itself, then it is. Contradiction.

What’s really interesting about Russell’s Paradox is not the thing itself, but the reaction of the mathematical world to it.  It was almost a classic grieving pattern… Denial, Anger, Guilt, Acceptance, etc. 

(You can study formal logic as much as you want, but you can’t get away from the vagaries of Hundred-Million-Year-Old Legacy Liveware.)

One immediate reaction was to re-write the rules of set theory so that this couldn’t happen – in essence to forbid a set being a member of itself.  Problem solved.  You just can’t do that.  Now go away, and behave yourself for a change.

In old-school IT, we have a similar thing.  It’s called Waterfall.  You deal with change by forbidding it.  Requirements freezes, code freezes, formal sign-offs of horrendously unreadable documents and “you stupid customer, why didn’t you have the sense to tell me what you wanted in the first place?” 

In set theory and formal logic, once people stopped denying the problems that were exposed by Russell’s Paradox, it eventually led to the beautifully mind-blowing Incompleteness Theorems of Kurt Godel. 

This is the equivalent in IT of “Embrace Change.”  Change, or Russell’s Paradox, is inevitable.  What matters is how we deal with it.  Accept it, and see what wonderful things result.