This article was published in BetterSoftware, September 2006. See http://www.stickyminds.com/BetterSoftware/magazine.asp?fn=cifea&ac=277

Stop Thinking! Drive Your Design with Testing and Refactoring

The fool doth think he is wise, but the wise man knows himself to be a fool. -- William Shakespeare

Are you or your team experiencing excessive excogitation? Have you lost the joy of creating quality software beloved by your users? Have you been reading about Agile Design and haven't figured out how to get started? Read on!

Well-designed software is a joy to work in and a joy to use. If you ask us to describe a good design, our answer will be an unwaivering, "how does it make money?" Users pay money for better software, programmers build better software when they feel good about the design.

Pages of preliminary feature descriptions, diagrams, or even brilliant blueprints do not add to the joy developers or users feel. As it turns out, pausing to reflect on a design, habitually simplifying the code, sharing and critiquing your ideas with others, programming with the big picture in mind, and appreciating good design will bring a smile to everyone's faces. We value design that brings joy to those who create it and those who use it.

We have developed five practices to help you achieve this end:

  • Design Learning
  • Coders Design
  • Enlist Others
  • Refactoring Rhythm
  • Savor Finishing

Design Learning

Think like a man of action, act like a man of thought. -- Henri Bergson, French philosopher (1859-1941)

What constitutes a good design? This is a question with many answers. So many, in fact, we leave it as an exercise for you, gentle reader. Design Learning is the time you spend developing an appreciation for good design. What attracted you to software development? Most likely, it was the nearly limitless possibilities for creating and learning. So, why not:

  • Read open-source code
  • Start a discussion group
  • Share ideas with your co-workers
  • Add titles to your well-worn bookshelf
  • Discover patterns in your code
  • Attend a conference

Zhon was fortunate enough to attend Richard Gabriel's workshop at XP/Agile Universe 2002. Richard asked, “How many of you have written a program more than once?” Proudly, I raised my hand. Ken Auer, sitting next to me, also raised his hand. “Twice?” Hands dropped. “Three times?” Sigh... my hand dropped. “Ten times?” Only Ken Auer's hand remained aloft. “Fifty?” Ken finally relinquished. Our poet, programmer and teacher continued, “poems must be written thousands of times.”

What is your count? Needless to say, my “hello world” count doubled on the flight home.

Coders Design

Think like a wise man but communicate in the language of the people. -- William Butler Yeats

Design is not a separate activity from coding. It is part of coding. Hand-drawn UML diagrams, flow-charted algorithms or barely spell-checked feature descriptions are legitimate ways to express a design. These designs, while more accessible to the lay person, have a high cost, because they lack a very important characteristic possessed by designs in tested source code: ruthless, immediate feedback. Simply put,

Designing without programming costs far more time and effort than driving your design from test-first programming.

We have found that overcoming the urge to design is quite difficult for most developers. Things that help are:

  • Start coding now (the design will follow)
  • Replace thinking with coding and questioning
  • Expect to get it wrong the first time you try
  • Discover your designs, don't force them
  • Make and remake your decisions

Active awareness leads to dynamic designs. It is a myth that agile software developers are unlike the rest of the world that designs everything up-front before beginning a project. We recently spoke with developers involved with the pre-2002 Olympics I-15 reconstruction project, and discovered the entire project from rocks to software only had about 20% of a typical up-front design when construction began.

The most important thing is to break free from the notion that everything has to be known and planned in detail before the project begins. Never allow the word “design” to be prefaced by the phrase “I'll go off and...”. Fight the need to know the “design” of the code before you know the form and the function, as revealed by unit and acceptance tests.

In the spring of 1999, we made a surprising discovery during our first foray into test-first pair-programming: Zhon, wanting to try extreme programming, removed Jeff from from his design meeting, saying, "Lets try test-first programming." Hidden away in an unused office, we wrote our first unit test. Before typing a keystroke, Jeff fled the keyboard, grabbed a marker and started drawing UML. Zhon patiently called him back, "Let's just make our first test pass."

Jeff had to be torn from the white board three more times before we established our test-code-refactor rhythm. Now addicted, Jeff overcame his primal instinct... shared by many developers... to design it all first. Amazingly, the code soon melted into a simple, elegant design.

Enlist Others

A wise man knows everything; a shrewd one, everybody. -- Unknown

Think about the hours you've spent in design meetings or big code reviews... what became of those hours? Did they result in improvements that were ever implemented? If the answer is no, it's time you got out of meetings and really worked with people.

Resist your natural urge to own and defend your ideas. Share your ideas publicly and open them for criticism. We challenge you to:

  • Show off your work
  • Develop a culture of continual learning
  • Code with someone every day
  • Talk design at lunch or the water cooler
  • Ask a guide to help you explore code

A recent study of orangutans in a sumatran swamp published in Scientific American indicated “culture promotes intelligence”. The researchers tracked the use of techniques and simple technologies (tools), and found an acquisition of useful technologies which flowed from the social interaction patterns of the apes. In short, however brilliant, no one individual has a monopoly on “the good ideas”.

The researcher, Carel van Schaik proposes that culture is the key. Primatologists define culture as the ability to learn (by observation) skills invented by others. Culture can unleash ever-increasing accomplishments, even bootstrapping a species toward greater and greater intelligence.

Just as bugs are easier to fix when found early, frequent peer reviews make it easier to recover from bad design decisions... pair programming, design discussions and casual conversation will give you multiple “safety nets”, making your performance on the high wire of software development a safe, rewarding and enjoyable one.

Refactoring Rhythm

Music creates order out of chaos: for rhythm imposes unanimity upon the divergent, melody imposes continuity upon the disjointed, and harmony imposes compatibility upon the incongruous. --Yehudi Menuhin, Violinist, violist, conductor (1916-1999)

Bigger, better, faster... these are the ever-present demands upon modern software. Unchecked, these demands will yield ever-growing quantities of unmaintainable, highly interdependent source code. This entropy can only be stopped by a constant flow of your energy in the opposite direction.

The three charts below represent lines of source code in projects over time. Which graph resembles your project?

Whether you realize it or not, your development has a rhythm. Chances are, your development combines some elements of the three charts above. What does it feel like when you have to re-write the project, like in chart #1? What way of working does chart #3 imply?

Making software requires re-writing it. When was the last time you completely reimplemented your software? A basic tenet of refactoring is that instead taking year(s) to completely redo your project, you can tighten up your rewrite cycle... doing smaller changes more frequently. There is a trend to re-write software products every 5 major versions. Nobody talks about this. Why not acknowledge it, and "pay off" the re-write cost in small increments:

  • Plan to refactor daily, weekly, and monthly
  • Start each task by refactoring
  • Finish each task by refactoring
  • Refactor only with a friend and tests
  • Target complex code with metrics (Sticky Note)

A few years ago while helping our company adopt XP, part of the program we were working on included a sub-system for user authorization modeled after Access Control Lists. This particular code had been very poorly implemented over two years, and was rife with bugs and unused complexity. In the spirit of “the simplest thing that could possibly work” we pleaded with our management to give us the time to simplify it. In the end, we mustered our courage, and quickly implemented only what we needed in a single, well-tested class, removing thousands of lines of code. Nobody noticed, except there were no more bugs.

Everywhere we go, we try to instill in developers an innate need to refactor frequently. Making the refactoring rhythm a part of daily development is a skill which may be hard to learn, but pays big dividends. For example, one of our co-workers, Jared Robinson, was in a tight spot. He had been refactoring for the past two hours and couldn't see how to go either forward or backward... he was trapped. Exasperated, he exclaimed, "How in tarnation do you refactor with out getting stuck?" We helped him get un-stuck, and he codified our conversation into the following Refactoring Rules:

  1. Find the smallest change that could possibly work, and check it in. "If I make this change, will it change nothing else?"
  2. Revert early, and revert often. If you lose a half-day of work because the refactoring change is too large, that's okay. Better to start over than to cost the rest of the team precious time struggling with a broken build.
  3. Do not write new code while refactoring.
  4. Always use cut-and-paste, never copy-and-paste.
  5. Just because the unit tests pass, doesn't mean that the product still works.
  6. Add needed unit and acceptance tests before you refactor

As you continue to test-code-refactor test-code-refactor, you will become better and better at it. It may not feel like you're doing enough or doing it correctly at the beginning. You probably won't begin to routinely ask yourself (or your pair) the right design questions for a while. Be patient, and practice, practice, practice!

Savor Finishing

To finish the moment, to find the journey’s end in every step of the road, to live the greatest number of good hours, is wisdom. -- Ralph Waldo Emerson

Good stories have a rhythm which goes something like this: Introduction, Development, Climax, Resolution. Great stories repeat smaller copies of this pattern which play out in the context of the overall story. Truly awful stories leave out one of these key ingredients, most often the "Resolution". When you tell the story of your project, is it a great story? or does it is sound like this:

"Yeah, we designed it, and then started coding and realized we were way behind, so we killed ourselves for months and eventually gave up and shipped it." Introduction, development, climax... no resolution.

We want you to savor the resolution. We want you to savor it hourly, daily, weekly and quarterly. To find resolution:

  • Follow the check-in checklist (sidebar)
  • Reflect on the session with your programming pair
  • Tell others about your accomplishments
  • Review the documentation
  • Finish every day with checked-in code
  • Celebrate releases

After years of attending Alistair Cockburn's monthly agile round table, Zhon only recently gained an understanding of the value of Alistair's Reflection Workshop. Every completed step is a time for reflection. Alistair insists it's important to ask yourselves two questions when you reflect on a unit of work, large or small:

  • What did we do well, and therefore want to keep doing?
  • What can we try to do differently?

Note the conspicuous absence of "What did we do wrong?" This was Zhon's "aha" moment. Pro golfers don't like to watch bad shots go off into the shrubbery (that's what caddies are for), and it does no good to dwell on the team's failures unless they are impetus for change.

We've been part of release "postmortems" where, when asked what went wrong, the developers could only spew complaints about how miserable things were. Every effort was made to remind them that they had actually accomplished something valuable. They were coaxed to offer positive suggestions for change. Nothing helped. Playing the "bugs approaching zero" game at the end of the release is a losing proposition, leaving a bitter taste in your mouth. All those noble efforts were forgotten and replaced with agony as it all hit the fan. If you're just wanting your work to be over as soon as possible, and never think about it again... there's something wrong.

By contrast, we have also worked on projects without "bug fix" phases, and they were glorious: Our plans sensible, our schedules met, and our software solid. In these cases, we usually run out of paper listing our accomplishments and suggestions for improvement, with a distinctly more pleasant and carefree vibe in the room. These reflections are so enjoyable, we find ourselves doing them after pairing, after designing and after planning as well as after releasing the software. These sessions take only a few minutes, even seconds sometimes... but give us a chance to stick a fork in it and call it well-done.

In Summary

The software we produce is like the neighborhood where we live. We spend a lot of time there (in some cases, more than our real homes). We are each responsible for our neighborhood. If it is cluttered, or it leaks, or is infested with bugs... it detracts from our joy. Even ill-repaired grand, flowing mansions can't compare to quaint, well-maintained cottages. As homeowners, we spend a lot more time worrying about keeping up the landscaping or the interior decorating than we do showing off the blueprints of our new addition to the neighbors. So it should be with our code.

Stop “designing” the way things should be on blueprints, and start enjoying the design of your well-factored code.

Design practices (sidebar)

  • Design Learning
  • Coders Design
  • Enlist Others
  • Refactoring Rhythm
  • Savor Finishing

Check-in check list (sidebar)

  • Is your intent revealed?
  • Is there any duplication?
  • Is the code as simple as possible?
  • Do you have all your tests?
  • Does the code look good from six feet away?

DesignAndRefactoringArticle (last edited 2009-04-30 23:15:24 by localhost)