Where it all went wrong.

Please forgive all the spelling and grammatical mistakes below, and the general half-arsedness of the writing, but I figured I should share this (currently overlong) chapter from the 4th quarter of my book.  It's the bit where my characters - the hero, Steve, and the guru, Craig - discuss Craig's incompletely-formed theory of when and why software development went wrong.

Just one thought before you read it: my hero isn't a software guy and he's not really a guru in his own mind, he just reads a lot and worked in manufacturing for years.  


Craig (the guru) said, 'Guess what, Steve.'

I played along.  'What?'

He stopped, turned to me and held his hands up.  'When I started working with your software developer colleagues in Wyx-health, I soon discovered that your software gurus - the folk who, I guess, formalised the way you work, way back -  were heavily influenced by the western quality gurus.  What surprised me, though, was how badly they screwed up.'

'Screwed up?'

'You now how, in the 1970s and 80s, everyone in business wanted to emulate the Japanese way of manufacturering?  TQM, Just in Time, Kaizen.  And so on.'

I nodded.

'Well, how do you think most western manufacturers learned what the Japanese did differently?'

That was easy.  'From consultants.  Or books.'

'Exactly.  Now, my guess is that your computer gurus - being inquistive people - read those books too, recognised the books tackled the same symptoms they were trying to prevent - namely too much expensve rework at the end of their processes - so they took the quality techniques from the books and applied them to software development.  Rather clever, but so, so wrong.  Do you have any idea why?'

I shook my head. 

'You, software development people don't do manufacturing.  You do development - the stuff that happens before manufacturing starts.  The hint is in the name - you do software "development", not software "manufacturing".  Right?'

'Umm … '

He frowned.  'I'm going too fast.  I'll slow down.'  

He pointed towards the cafeteria's kitchen.  'In the kitchen, they have big binders full of recipes.  Someone developed those recipes - I'm guessing they took well known domestic recipes then adapted them for use in large scale commercial kitchens.  This might sound strange, at first, but a recipe is a description of a manufacturing process.  The recipe starts with a list of raw materials - the inputs - and follows with the steps to tranform those materials into a final product.  With me?'

'Input-Processing-Output? Gotchya.'  Those words described a generic algorithm and were words familiar to any developer.  

'Now, put yourself in the shoes of the people who developed those recipes.  Do you think they just write the recipe down then publish it, without testing it first?'

I shook my head.  That would be daft.

'Of course they don't.  What they'd do is write version 1 of the recipe down, cook it, then taste the product.  If it doesn't taste good then they'll tweak the recipe, write it down, cook it, then taste it, and so on.  They'll keep coming up with new versions of the product - tasting them and testing them against other criteria - until they're happy.  Do you know what that kind of process is called, Steve?'

I frowned, thought a moment.  'Is it Iterative?' 

'Yes, it's iterative and empirical, in that it's based on evidence - like when they taste the cake - rather than just pure theory.  You might also say it is incremental as they produce version 1, then version 2, then version 3, and so on.  They key word here, for me, though is iterative.  Does that make sense?' 

'I guess'. I then played back his words, to check I understood.  'They iterate, testing their output until they get it right?


I thought a moment.  The output of a recipe development process is the recipe, which, itself, defines the new the manufacturing process.   And the output of the manufacturing process was the product itself.  The development process created the lasagne recipe, the manufacturing process follows that recpie to coook the lasagne.

I said, 'I'd never thought of it that way before.'

'Nor had the folk at Wyx-Health.'  He leant forward. 'Now, put yourself in the shoes of the team which develops new cake recipes.  Would you expect iterating like that - with, lets face it, loads of failure - to be expensive?  '

I instinctively shook my head, 'No. They'd work in a test kitchen where the marginal cost of throwing away food is some lost time and some raw ingrediants.  They need to fail to learn and it'd be much ceaper to fail in the test kitchen than in, say, the canteen's kitchen.'

'Precisely.  Now tell me: you do development so what's your industry's equivalent of the test kitchen, Steve?'

'Our developers work in what we call a development environment.  When we've finished our development work, we move our code into what we call our production environments.  They're the big computers where our real customer data lives.'

'So, your production environments are the equivalent of the factory, or the canteen's kitchen?'


'What, then, is your equivalent of the recipe? 

I blanked.  I really didn't know how to answer him because I didn't really understand his question. I shook my head.  'I have no idea.'  

'Nor did I, at at first.  But your Wyx-health colleagues soon figured it out. They concluded that the recipes you create are the programming code your developers write - both are, after all, the instructions for transforming inputs into outputs.  

'Algorthms', I said.

He nodded.  'Does that sound crazy to you?'

I considered his question for a moment then screwed my face up, awkwardly. 'Actually, yeah, it does.'

His face fell.  'I thought it might.  But, let's pretend your code are recipes and see where it takes us.  Your programmers write recipes in their development environments.  Your testers taste the recipes.  And when the recipes are good enough, they're tranferred to the production environments where they are run with real inputs, producing real outputs.  You still with me?'

I frowned.  His theory sounded plausible, but I'd never heard anything like it before and I was skeptical.  It wasn't so crazy that I didn't want to hear his conclusion, and I didn't want to be rude, so I said, 'Just …'.  

'Okay.  Here's the bit that puzzled me.  Development teams in other industries start iterating - and therefore testing, since testing is an intrinsic part of iterating - very early in their development processes.  They cook version 1 of the cake, test it, tweak the recipe, then cook version 2, and so on.  They test throughout the entire development process, but you computer people don't start testing - or iterating - until mid way through a project.  Why is that?'

Again, I couldn't think of an answer.  I shrugged. 'That's just how we've always built software.'

'Let me tell you what I think', Craig said.  'Like I said earlier, I think your gurus borrowed the quality thinking from the books about quality in manufacturing because that was all that was avaialbe.  If there had been books on quality in product development then your current processes would have been iterative, incremental, and empiracle.  They would have assumed you start with an incorrect recipe then iteratively and cheaply improve it over time.  They would have assumed the output was a recipe fit to run in your production environments.  But they didn't, instead they took the processes applicable to manufacturing which assumes the recipe is right before it starts and, since the recipe is meant to be correct, there's no need to iterate.'

I put my hand up, requesting a moment to think.  'But, since we're doing product development in a manufaucturing environment, you still iterate but not until we start testing in the last half of the project.  That's when change requests happen- at the last half of the project, when they're expensive to do.  Is that it?'

He said, 'Yeah. It's something like that.'


clarke ching2 Comments