Handbasket model

Several definitions will be necessary for the enjoyment of this post:

IT: Information Technologies. A department/division/branch of a U.S. corporation responsible for accomplishing things with computers and, as I will elaborate, generally failing. Corporations outside the United States probably have IT departments as well, but I don't know anything about them so I'm going to exclude them from discussion right up front.

C: A programming language. C has been said to combine all the power of assembler with all the readability of assembler, but that is not true. C can, in the hands of a skilled practitioner, be either much more readable or much less readable than assembler.

C++: A bizarre combination of punctuation. It is also an enhancement to C in which all the popular ideas between the conception of C and the conception of C++ were crammed into C, leaving a mush. Most significantly, C++ is object-oriented. However, C++ also has a number of other exciting features, such as being able to encode compile-time compilation, objects outside its own type system, etc.

Java: Another object oriented programming language with C-like syntax. It is much cleaner, from a design perspective, than C++ and programs written in Java run correspondingly slower.

C#: The result of Sun telling Microsoft they were taking Java and going home. These days, Microsoft has its own Java.

Visual Basic: a crawling horror. Whereas it is possible, although with varying degrees of difficulty, to write readable code in each of the aforementioned object-oriented languages, we contend that it is, in fact, not possible in Visual Basic. This is an extremely rare accomplishment in programming language design, although it has been accomplished before; see Perl, APL, etc.

Haskell: a functional programming language. Every place that I refer to Haskell in the remainder of this article, you may substitute the functional programming language of your choice. (If you substitute Scheme, you must assume that anything I saw will be "easy" will actually be "moderately difficult." This extends to such things as "tying your shoes.") If you specifically do not prefer any functional language, you may close your web browser at this time and return to hopefully throwing bones at monoliths. You are lost already.

Finally, to truly appreciate this post, you must care about the state of software engineering and IT in the United States. I figure that should weed out the rest of you.

It is a well known fact that for every promise of how information technology

(which is a particularly loathsome term. Book publishing is "information technology." A bullhorn is "information technology." "Technology" is a word which conjures up simultaneous images of ivory towers and pocket protectors and needs to go away completely.)

is going to improve American corporations and the lives of their workers, twenty people wait around for a computer system to come back online, two IT workers curse impotently at the idiotic tasks which fill their days, five office workers play games of solitaire on their computer, and eight factory workers are put out of jobs by robots. To the outside observer (or me) this does not seem like the unqualified improvement that was originally promised.

For every IT-related promise, there is a failure. For every failure, there is an explanation and a promise that things will be better next time. This is known in some circles as a recursive relationship.

I used to believe that this was primarily because most software developers were not trained in software development. Just as chemistry and chemical engineering are difference disciplines, and their practitioners are not considered interchangeable, computer science and software engineering shouldn't be considered the same disciplines. Sadly, at least most places, software engineering is lumped into computer science, taught by (generally uninterested) computer science professors, and it's hard to be tremendously surprised when the graduates of these programs, no matter how personally interested they may have been in software development, are not engineers.

Then I saw a software engineering textbook,

(It is certainly true that there are plenty of lousy textbooks in the world; however, it is my sincere hope that most of them are not used. This book is used at Northern Illinois University which, while it may not be considered at the same level as my particular alma mater in most ranking schemes, does produce ten graduates for every Dartmouth grad and thus probably has a much greater educational impact on most people (and most software engineering projects) than Dartmouth could ever hope to. Following this line of logic, it would seem significantly more important that the quality of education be high at NIU than at Dartmouth, which tends to be the opposite of how it works out.)

and I won't go into details on who wrote this particular book (George M. Marakas) or where you could get more information on said book if you were interested (but here's a start) or whether the author doesn't seem to have noticed any changes in mainstream programming languages since the early 1980s (he devotes one poorly written appendix to a demonstration that he doesn't understand object-oriented design) or changes in mainstream software development techniques since about the same time (he doesn't mention agile programming or any of its variants, and his discussion of rapid application development is anemic) or his tendency to use straw-man arguments (for example, he suggests that rapid application development techniques are inadequate for applications that use cutting edge technology, while proposing a model that takes a minimum of 36 months to produce any results and thus by definition can't use cutting edge anything) or even his inability to produce diagrams that correlate with his text (his discussion of terrariums as closed systems is accompanied by a picture of a tank filled with water and fish) because those are all incidental to whether his core point has any value. It was his core point that crushed all my beliefs about software development education.

I shall summarize his core point as the idea that serious software requires serious design. This is elaborated to the Software Development Life Cycle, or, in more common usage, the Waterfall model. The waterfall model of software development, so called because it consists of a series of stages, and sometimes requires moving from one stage back to a previous stage as a result of newly discovered problem details or research, is a serious design process. Of its seven stages, the first five do not involve writing code or buying hardware. Instead, the first five stages involve discovering problems, researching industry best practices, modeling existing systems, interviewing users and managers about their expectations and experiences with the existing solutions, prioritizing elements of the proposed solution, and performing a variety of other four-syllable tasks. After you have finished all of that up, Marakas seems to claim, you get someone to go write some code and buy some boxen and slap it together and pass maintenance off to someone else, because the important things are done.

I should start out by saying that I fully support, in the abstract, industry best practices, users, models, existing systems, expectations, experiences, and while I have skepticism about managers I suppose I should allow them for the time being. However, I believe his combination of these factors is completely wrong-headed. Primarily, Marakas (and other proponents of the waterfall model, I suspect) ignores all the realities of software. Just as it is not possible to design a building without taking into account the details of the materials you will use, the construction techniques available, the ground it will be built on, etc., it is impossible to prepare a useful analysis or design of a software project without considering the physical and mental resources available and the language and coding techniques that will be used. Most programmers would recognize the ridiculous nature of claiming that a single design document proposes the best solution to a given problem in C, C++, Java and Haskell, but Marakas suggests that the majority of software development should be devoted to producing just such design documents. It is a necessary corollary that design at such a high level cannot produce any sort of verification except at a similarly high level. Unfortunately, such high level verification is completely useless. Observing that a finished inventory system does not produce the right inventory reports does nothing to suggest why it does not or what should be done to fix it, and tests that come from a design that is independent of any particular physical resources or coding techniques cannot be any more specific.

In short, the waterfall model, as Marakas proposes it, is not a model for software development. It might be a management process that could keep managers amused while programmers wrote software, but the more it was allowed to affect the software the worse the resulting product would be. The real tragedy is not, I think, that wrong-headed ideas exist. Wrong-headed ideas are an indispensable part of intellectual progress. The problem is that with stunning evidence to the contrary clearly available, Marakas is touting a phlogiston-based model of software development and that people are listening to him.