Contents

5.5 The Cost of Testing

It is well known that most of an iceberg is hidden under water. The same is true of software. The size of a software project is often described by the number of lines of code in the product, but that's just the tip of the iceberg. The amount of software that needs to be written to test the product can be staggering. Often the number of lines of code of test software will exceed the lines of code in the product software.

Test software can get out of hand in a hurry. Suppose you have to write three lines of test software for every new line of software you deliver. (That's probably what I average.) Then suppose management says that all your test software must be fully tested. (You've got to know your test suite works, don't you?) So for every line of test software you write, you need to write 3 lines of software that tests the software that tests your product. If X is the number of lines in your product, you will have to write 3X lines of test software, and 9X lines of code that tests the test software. Your job has just increased by a factor of 12! If management then insists you fully test the software that tests the software that tests your product, your job (and cost and schedule) has increased by a factor of 39. Sooner or later you have to call an end to the madness. (Fortunately, the managers I've worked for have required formally testing the test software, and no more.)

What generally happens is that people write as much test software as they can in the time left over at the end of the project. (Is there ever time left over at the end of the project?) The test software is inadequate, so the product gets shipped with some bugs in it.

Maybe you don't believe you have to write more test software than product software. Well, just look at size of the Cos_Test (Listing 74) and Cos_Diff (Listing 75) compared to the fraction of the TRIG package they are testing. I wanted to include complete test programs for all the source code in this book, but there's just not enough room for it all. That's why I've only described Cos_Test.

Testing is a big job. You have to plan money, people, and most of all, TIME for it. If you write three lines of test code for every line of product code, it will take you three times as long to write the test code. Don't expect to write and debug all your test code in the last month before the critical design review. Even if you could, it wouldn't do you much good. By that time the design is cast in concrete (and probably behind schedule), so nobody is going to change anything unless your test programs find catastrophic errors. If you find moderate errors, people will say, "That's a shame, but we can't do anything about it now. Why didn't you catch these errors sooner?" You have to code and test early in the development cycle while there is still time to take corrective action. Some people believe it is possible to do such good planning during the design phase that flawless code can be written in a few weeks at the end of the project. They think testing is a mere formality to show that the design is correct. That's nonsense. ÿ


Contents | Next ...