Rex Black

Only honest data yields worthwhile information.

One development engineer told me that he “felt very depressed” after reading the risk analysis documents my test staff had prepared. I find it more depressing, however, to ignore the possibility of failure during development, only to live with the reality of preventable failure after release.

Skip. You've decided to skip the test for this pass, typically because it's a relatively low priority. Again, the Comments column of the test case summary worksheet should reflect why the test is not run. One way to think of the Skip state is “I could have run the test, but I didn't want to.”

You also might want to think twice about certain types of metrics—in particular, those that address questions such as “Who found the most bugs overall?”, “Who spent the most time testing?”, and “Who found the most bugs per hour spent?” I find these numbers dangerous, and I don't generate them. As soon as you begin analyzing employee performance data, you take risks. If people know you are measuring them this way, they will soon figure out how to manipulate the numbers.

The non sequitur interview involves asking some question that seems to be analogous or related to testing. The classic example is, “Tell me how you would test a salt shaker.” I find this silly. I call this a non sequitur interview style because it does not follow that, just because someone can craft a clever tale about testing simple real-world objects, they can test complex software and systems. You are hiring a test professional, not a raconteur.

I sometimes hire people whom I think will grow into a job, but I never hire people I think are trying to con their way into a job.

People, regardless of where they are, are not widgets. They want responsibility. They want respect. They want to be valued. They want us to be interested in developing their skills and their careers, even if they work for a third-party contractor. They want feedback.

The most dangerous kind of wrong is the kind of wrong that sounds reasonable, as I've said elsewhere in this book.

Had he tried his hand at computer testing, I suspect that Hemingway would have written great bug reports (provided he was sober). I doubt that Faulkner would have. It's not simply a matter of style, though. Even if you write like Hemingway, it takes concentration, practice, and discipline to describe bugs well.

When I write reports, I document the minimal number of reproduction steps, and I include only isolation information that yielded an interesting data point. If I run one test and see two independent bugs, I write two reports.