Dr. Bruce Scharlau, of the University of Aberdeen asked,
Clarke, Would you hazard a guess at
A) What the current balance between agile (or lightweight) and waterfall, et al (or heavyweight) methodologies might be, and
B) when the majority of software might be done with agile methods?
If you don't want to do 'B', could you suggest what conditions might be necessary for this to happen?
That's a very hard question to answer - from my point of view - because I mostly talk to people who are doing agile or wanting to do agile. I've seen some research that suggests that most organisations are piloting or considering piloting agile - but I find the numbers meaningless.
But - given that I don't have a clue - here's my answers
A) I think that maybe 5% of organisations (perhaps 1%, perhaps 10%) are predominantly doing agile. I suspect that 70% (perhaps 30%, perhaps 80%) are giving agile a little look-see (i.e. trying it out in a small, sensible sort of way). I suspect the rest have heard of agile ... but think it is either totally daft or that even though it works in other organisations it wouldn't work for them.
B) It depends. I'd say that in 30 years time most organisations will be working incrementally and they'll be doing programming in an XP sort of way. If my (extremely ambitious) EverydayAgile initiative manages to convince more businesses that they can be "agile" by just-enough-agile to make things better, then maybe cut that down to 10 years.
At the moment, I think the biggest hurdle to spreading agile wider is the agile community. I don't mean that in a bad way. It's just that what got us HERE, won't take us THERE.
Too many of us think that it is too hard to "convert" THEM and "if they don't get it already, what's the point?". That's a very sensible opinion to take. It is hard to convert people who don't automatically "GET" the ideas. And most of us in the Agile community don't have the right skills to do so - we are, at core, a bunch of extremely clever and motivated techies ... who don't get along so well with people who wear suits.
That's where we have to start: finding people who believe it is not only worthwhile, but possible, to spread the ideas to the "suits" and everyone else who lives in "everyday land".
I think I've decided that I'm going to spend the rest of my career - 30 years - focusing on the "everyday" problem ... but I hope it only takes 5 - 10 years.