The Cost of Change curve

I'm thinking out loud, metaphorically, here.  Bear with me (I think that's the wrong type of bear but it's late at night and being a Kiwi I pronounce all bear words the same, anyway).

In Extreme Programming Explained Kent Beck claims that XP practices make the cost of change curve shallow and that's why XP works.

However, Scrum doesn't (necessarily) do most of the XP practices. But it still works.

How come?

I have 3 theories:

1. Boehm's original 1981 cost of change study showed that the cost of change ratio (between fixing in production vs. requirements stage) for SMALL projects is 5:1, not 100:1.  So if we're working on small projects we have a shallow curve.
   
   But what if we are working on big projects?  If we break the project into cleanly decoupled teams can we isolate most of the effects of change and also have a shallow curve?
   
2. The original study was done in the late 70's when people were doing (mostly) waterfall projects so the survey shows the cost of change curve for waterfall projects. There were no agile/iterative/incremental/whatever projects so the study can say nothing about agile/iterative/incremental/whatever projects. [If I do a survey of adult males and conclude that the average shoe size is X, I couldn't assume the average shoe size of female children is also X]
   
   In Balancing Agility and Discipline Boehm and Turner say that recent studies confirm the original 100:1 ratio. But, these studies make no distinction between waterfall or agile/iterative/incremental/whatever and if thre were any agile/iterative/incremental/whatever projects in the study, then it is likely they would only be a small proportion of the total, then these studies also tell us nothing about agile/iterative/incremental/whatever projects.
   
   So, perhaps the cost of change curve is based on waterfall projects and, in a vicous circle, was used to justify waterfall projects.
   

3. In the more recent studies the changes were broken down further and it turned out the 80/20 rule was at work: that roughly 20% of the changes caused 80% of the work.
   
   In other words, 80% of the changes on the big projects had a far shallower change curve ratio that 100:1.
   
   AND 20% of them had a much steeper change curve. 
   
   So, in an agile environment, if we had a way of idenfifying those 20% - estimating how long they'll take, perhaps - and if they were important to the customer then we could tackled them first and FAIL EARLY before all the money has been spent. 
   
   And the less important ones?  Perhaps the customer would relegated them to the bottom of the product backlog.

Hmmmmmm ...

 

Comments

[David Anderson]

Clarke,

Your first point gets at "architecture" - something that several of us "agilists" have continued to advocate. My fellow Motorolan, Ron Crocker, being one of the latest examples, with his book "Large Scale Agile Development" which argues that you can't be agile and large without architecture - and its use to partition the domain into smaller independent pieces.

Smaller decoupled pieces reduce the regression effects of change.

The XPers argument that you "just refactor" doesn't hold water on large scale projects and leads to the 100:1 cost ratio that Boehm identified.

The XP argument that analysis, design and architecture don't matter much because you can always refactor - and you let everything just emerge as you go along - is dangerous on larger projects. I don't think Kent Beck or Ward Cunningham ever made such claims. They knew that XP was a solution for the small scale. They argued that there should be more focus on doing things smaller. However, there are domains that will always require large projects, even if those are delivered incrementally.

For me, and the FDD community, it's not about "agile modeling" but "modeling for agilility". The concept of wide but shallow modeling to develop an architecture is all about risk reduction, isolation of concerns and decoupling. This is what contains the cost of change but provides the flexibility to cope with change when it happens.

Regards,

David