Clarke Ching's Rocks and Snowballs

Dynamic and Detailed Complexity, Design and Dinosaurs

1. Dinosaurs

In my first programming job the code was older than I was. 

It was a Credit Card system and it was written in Cobol.    Although it was a good job on a prestigious graduate scheme I was unpleasantly surprised to discover just how hard it was to work on an old COBOL system when I was used to the elegance of more modern languages such as C, Pascal, Modula 2 and (a little bit of) Ada.  I worried that I was turning into a coding dinosaur.

One program – called BZCH10 – was over 25,000 lines long.  The program read in a credit card record, applied that day’s new credit card transactions to the record, and wrote out updated credit card record.  Somewhere in the middle of it there was a single, large, multi-page GOTO statement “GOTO … DEPENDING ON X1” which literally would GOTO another part of the code depending on the value of a variable called X1.  There were around 200 valid values of X1 and, needless to say, the variable X1 was littered throughout the 25000 lines of the program.  X1 was the driver of a finite-state-machine that ran for each transaction.  Try and imagine it.

I remember thinking, as I got more experienced at the job, that my skills were deteriorating and I’d slipped from being an exceptionally good programmer at university to being a not-much-better-than-average COBOL programmer.  But, I’ve come to realize recently that my assessment was wrong: it takes a lot of skill to be an average COBOL maintenance programmer because of the sheer complexity of the code.  The truth was that I was a good developer in a good team working on a very complex system.

2. Complexity

Peter Senge, the Fifth Discipline / System’s thinking guru, talks about two kinds of complexity.  The easiest is detail complexity, where the system is made up of many variables and individual items, and even though it’s the easiest kind of complexity, it’s still damned hard work.  All non-trivial software systems have detail complexity.  Good design makes it easier to understand these complex systems because it breaks the code down into manageable chunks and hides the detail of each chunk unless you need to look at it. 

Good design works on the second, harder type of complexity: dynamic complexity, where the cause and effect at play within the system is subtle, unpredictable and often downright confusing.  For instance, I once spent 6 weeks trying to fix an obscure bug in BZCH10.  It took about 3 weeks to find the problem, about 30 minutes to add the extra 6 lines of code fix it, and another 3 weeks to ensure that I hadn’t broken anything else.  A good bit of the time taken was caused by the detail complexity because it was extremely difficult work to track my way through the mass of code and find the problem.  But, the bulk of the work was due to the dynamic complexity of the code because the convoluted design of the code meant that I couldn’t be confident that my 6 new lines of code wouldn’t cause some very unpleasant side-effects (which you want to avoid when dealing with  hundreds-of-thousands of customers and millions of dollars worth of transactions each day); that is, the design of the program made the cause-and-effect very difficult to un-tangle. 

3. Design

So, good design is important.  It makes us more productive because it simplifies complexity. 

BZCH10 really didn’t stand much of chance because the original COBOL programming it used didn’t lend itself to good design.COBOL was created in 1959, before structured programming or even object oriented programming/design came along.  However, nowadays, modern COBOL (the 2002 standard apparently) does allow OO design and coding.  I wonder what the design (and therefore complexity) of BZCH10 would be like today if object-oriented COBOL was around when it was first built. 

Okay, forget about COBOL, imagine that we’re starting a new project next week using a modern programming language and our team is made up of world class developers. 

How good will the system’s design be in 30 years time? 

What could possibly go wrong?

[
to be continued …
… but, I’d love to
see your answers
]

2005 TOCICO conference presentations available

The TOCICO conference was held in spain this year.  I nearly went, but family things at the last minute prevented me from making it …

Still, here’s the next best thing: DVD's of the presentations (plus you get the 2004 DVD’s thrown in for free if you buy then before January 15th – and if you use the gift certificate deal then it’s another third off).

And, if you want to try before you buy … here’s the next, next  best thing: they’ve just put the 2005 TOCICO presentations on line (and, if you hadn’t already seen them, here are the 2004 presentations).  Happy reading!

Using TOC to Reengineer the Sales Process - Justin Roff-Marsh

Yipee!  There’s a new Theory of Constraints book available at toc-goldratt.com.

Justin Roff-Marsh has released “Re-engineering the Sales Process” which applies TOC thinking to Sales. 

I haven’t read the book yet, but I can tell that Justin’s thinking has been well received in TOC circles. Here’s his 2004 presentation at the TOCICO conference and a white paper.

BTW: if you’re quick then toc-goldratt.com has a special deal whereby you can purchase a gift certificate from their site and they’ll add 50% to the value.  You can the use the gift certificate yourself and buy the book effectively for a third off.

SOFT SELLING in a HARD WORLD - free ebook

Jeffry Vass wrote one of my favourite sales books: "SOFT SELLING in a HARD WORLD - Plain Talk on the Art of Persuasion".  It's short, sweet and straight to the point.  And now it is available as a free ebook download from Vass's website.

From the Preface:

People waste their lives trying to sell. They get confused between persuasion as a belief system (“You have to believe in what you sell.”) and persuasion as a natural talent (“You can’t teach anyone how to sell.”). SOFT SELLING IN A HARD WORLD™ is designed to clear up that confusion.

If you aren’t selling up to your potential, you probably don’t understand that selling is a game. Most people don’t. Those who do make 85% of the money, become executives, or run their own successful businesses. This book is about that game—how to play it, when to play it, where to play it, and with whom.

This book is about fulfilling your potential without resorting to “motivational” and “inspirational” beliefs.  As in sports, you’ll find that certain mechanical moves need to be mastered before your inspiration or genius can shine. A dog can become inspired to chase a car but doesn’t know what to do with the car once it's caught it.

This knowledge is about what you do when you catch the car, that thin slice of face-to-face time with the Buyer when persuasion actually occurs.

This is a handbook. There is very little theory—it is nearly all mechanics. While there is a certain primitive logic in the way this story is told, you can begin to read on any page and profit.

2005 Man of the year: Nigel Roberts

From today's Guardian:

A legal claim against an internet marketing company which has been accused by the recipient of spamming could herald the end to junk-filled inboxes, it emerged yesterday.

Nigel Roberts, 37, who runs an internet business in the Channel Islands, wrote a letter of complaint to Media Logistics UK - a company specialising in "electronic direct marketing" - after it sent him an unsolicited email advertising the services of a contract car company

Agile games

I’ve been researching a few Agile games for an event I’m planning. 

There’s some good stuff out there:

• James Shore has gifted us the excellent Offing the Off-Site Customer simulation / game which demonstrates the difference between having face-to-face onsite collaboration with a customer versus having to work through written documents or intermediaries such as analysts.  I’ve only just discovered this and I’m so impressed that I’ll be asking for volunteers to run this as an Agile Scotland session later this year.
• Jean Tabaka has prepared a lovely little simulation of Scrum that can be run in under an hour.  It’s called the 59 minute Scrum and this is the one I’ve settled on for the event.
• The folks at the XP Belgium user group developed the now classic (and recently improved) XP game.  The folks at exoftware ran this for AgileScotland as part the last Agile SIG and it was a great learning experience.
• Bill Wake has about a dozen games on his website.  Rachel Davies ran the Scrum from Hell for us during an Agile Scotland training event earlier this year and it was a joy to watch!  I suspect Bill’s other games are just as good.

British Medical Journal - Xmas issue

Each year the British Medical Journal takes on a more humours style for it’s Christmas Issue. 

You might find the following articles add a smile:

• White coats and fingerprints: diagnostic reasoning in medicine and investigative methods of fictional detectives suggests that “An ideal clinician would present a harmonic fusion of almost all the investigative methods of fictional detectives and avoid slavish adherence to protocols and procedures”.
• The case of the disappearing teaspoons: longitudinal cohort study of the displacement of teaspoons in an Australian research institute which reveals that of the 70 discretely numbered teaspoons placed aournd a medical institute, only 14 remained at the end of the 5 month study period.
• Cinderella revisited which concludes that the medical use of the Cinderella fable is growing in popularity
• Harry Potter casts a spell on accident prone children reveals that the number of children attending emergency departments dropped significantly during the weekends when 2 Harry Potter books were released.

 

 

looking lean vs. being lean

Toyota Production System guru, Art Small, has a powerful article where he makes the distinction between “looking lean” and “being lean”:

Real TPS is not just about “flow” or “pull production” or “cellular manufacturing” or any of the other catchy phrases or tools you may frequently hear.  For over fifty years TPS in Toyota has been primarily concerned with making a profit, and satisfying the customer with the highest possible quality at the lowest cost in the shortest lead-time, while developing the talents and skills of its workforce through rigorous improvement routines and problem solving disciplines.  In every piece of TPS literature from Toyota, this stated aim is mixed in with the twin production principles of Just in Time (make and deliver the right part, in the right amount, at the right time), and Jidoka (build in quality at the process), as well as the notion of continuous improvement by standardization and elimination of waste in all operations to improve quality, cost, productivity, lead-time, safety, morale and other metrics as needed.  This clear objective has not substantially changed since the first internal TPS training manual was drafted over thirty years ago.

Despite this consistent message on the part of Toyota, the lines have become crossed in many Lean plants.  In the U.S., becoming Lean appears to have gone down a path of implementing tools such as “one piece flow”, “value stream mapping”, “standardized work”, or “kaizen events”, but results have not always followed.   Toyota, by way of contrast, has stayed focused on its principles and a disciplined emphasis on process improvement to obtain results such as “making a profit”, “reducing lead-time”, “improving productivity”, “achieving built in quality”, as well as “respecting human dignity of employees” etc. The difference may sound trivial, but it is actually significant.  In fact, it is likely the main reason why Toyota has continued to see success on so many dimensions, where others struggle. This mishap has not occurred intentionally.  As the title of this article indicates, it is something like the ‘law of unintended consequences’ at work.  If we are to be like Toyota in terms of results, and not just look and sound like Toyota in word, then several things need to change in Lean implementation. 

 

Requirements aren't fixed in stone

[click picture to enlarge]

[apologies to whoever built the beasty website that generates this … I can’t remember the website name, but thanks!]

 

Software development is Product Development

Here’s a wee snippet from an old presentation of mine that shows the difference between Product Development and Manufacturing (powerpoint animation).

Software development, if you didn’t know, is mostly product development, with just a tiny little bit of manufacturing thrown in at the end when the final product is shipped/released into use.

Notice a few things:

• Product development is iterative. 
• It requires TESTING (or tasting, if you’re making cakes), in order to LEARN, because you are trying to figure out what the SPECIFICATION looks like.
• TESTING happens before you finish the specification.  Not after.
• A good definition for Quality for PRODUCT DEVELOPMENT is from Joseph Juran : “Quality is Fitness for Use”.
• A good definition for Quality for MANUFACTURING is from Philip Crosby: “Quality is Conformance to Specificiation”.
• The Waterfall mechanism is based on Crosby’s definition.  That is, figure out the recipe/specification before you build. 
• If you build cakes based on a spec that hasn’t been tested yet then the cakes are gunna taste aweful and you’ll end up fixing them after they’ve been manufactured.  That is, you can’t, in product development, know what the spec is until you’ve done a good bit of testing (tasting) so if you build the product without doing lots of testing for “fitness of usage” first, then you’re product is going to crap.
• And, then you’re going to spend lot’s of time fixing the product until it is “fit for usage”.  But that fixing is going to be very expensive.
• The Agile mechanism is designed to be iterative and involve lots of testing (feedback).  Just as good product development should be.