Why Software Doesn't Work and What Educators Can Do About It

Design Procedures. When Michelangelo designed the dome over St Peter'ss, domed buildings were still an experimental form. He didn’t quite get it right, and not long after the dome was completed, lateral forces cracked it. Architects repaired the defect in the design by adding step rings to redirect the crack-causing forces. In the four centuries since the building of St Peter’s, structural engineers have created a collection of design rules, principles, and procedures that avoid most building defects of this kind. The field of software engineering has not progressed that far.

Software engineering lacks the rules of systematic analysis and design that most engineering disciplines have developed. As a consequence, trial and error predominates over analysis and design. This must change if software engineering is to make significant strides towards maturity.

Test and Debug. Nearly all software is constructed using the test-and-debug procedure. Software designers piece together a software product, run some tests, fix the errors they observe, and hand it over to test specialists, who test the product every which way they can think of. They cannot possibly think of enough ways.

Effectiveness. The procedure doesn't work, and anyone who has used a computer for more than a few hours knows it. Almost all software is riddled with defects, and it is demonstrable fact that defects will remain, regardless of how many tests the software passes. Why? Because any significant piece of software has so many potential states it can get into that even if you ran a trillion tests a second from now until the sun runs out of fuel, most of the potential states would remain untested. The test-and-debug method cannot ensure defect-free software.

The Solution. Prominent thinkers in the software field, people such as John McCarthy, Edsger Dijkstra, Antony Hoare, and John Backus, recognized this fact several decades ago, and all of them suggested the same solution: mathematical logic. Apply the principles of mathematical logic to specify requirements for software products, and use those same principles to verify that products meet their requirements. Every serious investigation of software engineering methods has confirmed that reasoning about software is many times more cost effective in preventing and removing defects than testing and debugging. That is, the solution to the frequently bemoaned software crisis is known and has been known, almost from the start.

Then Why Aren't They Doing That? In the early days of computing, computing machinery was hundreds of times more expensive than software engineers. Machine resources had to be utilized as efficiently as people could figure out how to use them. Software that is clearly specified and amenable to logical analysis often makes less efficient use of machine resources than software that has been hacked together with a primary focus on resource utilization details. So, things started off in the wrong direction. The entire intellectual enterprise of software development has concerned itself primarily with machine resource utilization and not with software defects. This point of view permeates the field, and predominates even among computing educators.

By now, several generations of engineers have grown up with a set of values that favors test-and-debug over mathematical logic as a basis for software development. The people attracted by those values do not have the same collection of talents and interests as the people who might have been attracted by values based in mathematical logic and focused on producing defect-free software. Software engineers are not using the solution because they don't know how, and they're not inclined to learn the kinds of things they need to know to use such a solution.

What Educators Can Do. A juggernaut like that is hard to turn. Turning it will require lots of effort on many fronts. What educators can do is to design courses of study that focus on defect-free software. They can design software projects that require the use of mathematical reasoning to ensure software correctness. They can refuse to accept the inevitability of the test-and-debug method. They can expose students to the serious use of mechanized logics, such as J Strother Moore's ACL2. They can demonstrate the use mathematical logic for software development in every software course.

This requires rethinking and reorienting all of the conventional coursework in the computing field. Course titles and course sequence can remain about the same, but the software methods studied in these courses must be completely revamped. Computing educators must reeducate themselves, and must change the way they think. This won't happen overnight. It will take a generation or more.

Getting Started. The Beseme Project, an educational experiment funded by the National Science Foundation and the University of Oklahoma, seeks evidence that better software engineering can come from improved and expanded use of mathematical logic in computing education. If the Beseme Project successfully demonstrates a connection between coursework that requires the use of mathematical logic in analyzing software and the subsequent effectiveness of students in software design, the experiment could be one of the multitude of small forces necessary to begin to turn the juggernaut in the direction it must go if people using computers are ever to gain the benefits of defect-free software.

Rex Page
University of Oklahoma