Tom’s Ten Data Tips – September 2010
Technical Reviews
Technical reviews are management’s measurement of (technical) product quality. You examine suitability of a software product. The main difference between informal reviews (as in pair programming) and formal reviews is that the purpose of the latter is to apprise management of quality and fitness for its intended use. Reviews can pertain to many things like: documentation, code, procedures, specifications, standards, etc.
1. Technical
Although the primary purpose of technical reviews is to provide quality measurement to management, one could argue that their greatest value might lie in a “side effect” they invariably have. When smart peers collaborate to scrutinize software products like documentation, architecture plans, or code, their collective wisdom consistently outperforms what individuals can dream up.
Although there may have been some initial reservations, you often see that after introduction of review processes (everybody has some trepidation having their work “criticized”), this learning is very much valued. When you receive high quality feedback repetitively, patterns emerge that the reviewers themselves may not have spotted (yet). These could be preferred styles of design, “blind spots”, and recurring bug types.
2. Raise Issues, Do Not (Aim To) Resolve Them
It is very important for effective technical reviews that participants focus on pointing out what needs improvement and refrain (altogether) from explaining how issues should be fixed. The latter is exclusively the producer’s responsibility. The desire to “help” is such a deeply ingrained drive that you actively need to suppress it during reviews. Lacopi’s Law: After food and sex, man’s greatest drive is to tell the other fellow how to do his job” (Jerry Weinberg).
There are two fundamental reasons why “helpfulness” that was not (explicitly) requested doesn’t work very well. First, such suggestions, despite the best of intentions, tend to elicit defensiveness from the producer. You want to “open up” communications to tap into all intellectual resources available. Secondly, even though the reviewer’s suggestion may be a good or workable solution, that doesn’t make it the “best” or “simplest” (Occam’s razor) possible solution. “Blindly” accepting any suggestion fails to make best use of all intellectual firepower available within the review committee.
3. Technical Reviews Are Useful Throughout The Development Cycle
Technical reviews are used to assess “software products.” What exactly constitutes a “software product” evolves throughout the development cycle. Technical reviews can apply to specifications, designs, architecture, program code, use cases, test cases, user interfaces, documentation, etc. In short, all more or less tangible products as they are used when building a software solution.
4. Share All Issues With The (Entire) Review Committee
An important element of reviews is bringing issues and (design) considerations out into the open. So all input, from all participants needs to be recorded and presented for everyone to see. That way everybody can respond to everyone’s else’s points, and/or ask for clarification.
Since you do not know in advance how many points will come up, or how verbose some participants might get, your recording media should account for that. A whiteboard, for instance can get full – how do you continue after that? Once you wipe out the previous points, they are more or less “lost”, even if they were transcribed. The same (or worse, actually) with private note pads. Only one person can see what has been written down, while everything might be important. When you use an easel, make sure you have tape ready to hang up pages from your flipchart, so that all points remain visible (to everyone) throughout the session.
5. Begin Reviewing Upstream To Save The Most Costs
Although you will benefit from technical reviews throughout the development cycle, the further upstream you can catch errors, the cheaper total development costs will be. Any time you let a minor misunderstanding during the requirements process “slip through”, you build according to flawed specifications (see also a whitepaper I published: “Three ways to get requirements wrong”). And sooner or later it will come to the surface. The longer that takes, the more expensive this glitch will be.
From requirements follow design and architecture. Then coding, testing, and implementation. You can make a case that one reason why agile (cross-functional!) teams are so effective is that close and informal interaction, combined with small and short iterations gives you the best opportunity to catch little flaws early. Technical (formal) reviews provide unbiased information streams to senior management that embed software development into the broader corporate environment.
6. Technical Reviews Keep Egos In Check
Everybody knows the “lone maverick” programmer, often technically gifted, maybe brilliant, but not very effective in teams. And then you have the “star programmers” who always seem to get it right. Technical review processes tend to lay bare exactly where lone wolves have their weaknesses.
A “side effect” of objective, unbiased reviewing, is that team members who “always” seem to outperform their peers will nonetheless get good constructive feedback about their performance. People gravitate towards methods with which they are familiar, but those aren’t always the best in all circumstances. Leading your “stars” outside their comfort zone requires concerted effort and well-reasoned arguments. For some, this helps them from letting their heads get too big…
7. Evaluate The Product, Not The People
A review session is about a software product. This product has been made, however, by someone with some reputation, which might be favorable or not. These two are necessarily confounded, but the review should focus on the product, and the product alone. Some empathy (without loosing objectivity) with the producer can help reviewers when they realize their colleague has at least some ego involvement. We’ll assume he has tried very hard to make it work. Keep this in mind while you discuss the product and phrase issues.
8. Technical Reviews Require Thorough Preparation (From Everybody)
When you plan a review session, you want to invite as many (or little) participants as are necessary to do a good job. When you invite too many people, not only will the meeting take longer, but you have also tapped into additional resources. When you invite too few people, you will not be able to cover the whole product, and all of its facets. So the trick is to strike a balance here.
Once you have found “the right”, hence minimal number of participants, it is obvious you can’t afford to loose anybody. If someone needs to cancel, the review should be rescheduled. Likewise, everybody needs to do a thorough job preparing, because everybody is required for the review. If not, cancel or abort the review, and reschedule. The fundamental purpose of reviews is to provide (senior) management with reliable information about the quality of a software product. This quality is compromised if only one of the participants has not been able to prepare properly. In that case you might just as well not do a formal technical review at all.
A walkthrough, on the other hand, benefits but does not necessarily require exhaustive preparation. In a walkthrough, the producer leads a committee through his product, and explains crucial features and key considerations. Dialogue may ensue, and is in fact welcomed.
9. Avoid Hierarchy In The Review Committee
Composition of the review committee should be driven by your review objective. You want an accurate assessment of quality, which you will play back to management. Members should be selected on their technical merit: will they be able to judge, will each member be able to fully qualify the product?
However, there is one more consideration for the manager who is responsible for composing the review committee. Can they do a good job together? For this reason, it is strongly recommended to never ever include managers in the committee. That is, managers whose subordinates are also in the review team. In those cases, this particular manager cannot avoid evaluating committee members, his subordinate(s), besides the product at hand. This interference stands in the way of product assessment, and should therefore be avoided at all costs.
10. Reviewing Drives Test Adoption
Everybody makes mistakes. To err is human. After reviewing has become customary, professionals quickly learn that despite utmost care and the best of intentions, mistakes still happen. You will catch some of your own errors, but others will catch more.
That’s why organizations that have embraced technical reviews quickly establish sound testing practices. They’re quite aware of the risks that design errors or bugs still occur. Those who have not developed this awareness, yet, may continue live in blissful (and happy) ignorance.
Further reading
Two excellent books on Technical Reviews:
Handbook of Walkthroughs, Inspections, and Technical Reviews.
Daniel
Freedman & Gerald Weinberg (1990)
ISBN#
0932633196
Software Inspection.
Tom Gilb & Dorothy Graham (1994)
ISBN#
0201631814
