We Do Need More Programmers

Edd Dumbill writes that we don't need more programmers. Besides a few because that is what I believe kind of rationales, he doesn't really provide any evidence for his assertion. But, let's play along.

Note: This is not an attack on Mr. Dumbill's ruminations. Instead, this is a thoughtful response to his post in the spirit of furthering a discussion.

Mr. Dumbill asserts that since "It's the job of the computer to serve people" there is no need for everyone to know how computers work. He uses the analogy that all drivers don't need to be mechanics.

That analogy misses a couple of points. First, computers have no job. A computer can do whatever job its user gives it to do. Cars, for the most part, do only one thing -- move passengers and goods from one point to another. Computers can do an infinite number of different tasks.

Second, while every computer user doesn't need to know how computers work, computer programming teaches a systematic way to think, to analyze, and to solve problems. Abelson and Sussman(s) write in the preface to the classic Structure and Interpretation of Computer Programs, "A computer language is… a novel formal medium for expressing ideas about methodology."

I agree with Mr. Dumbill that "We need better software." But, he again misses the point when he implies that "The best minds of our generation are focused on making us click more ads," and laments that it is a "Profoundly misdirected effort." First, the ads that make us click them are great software because they achieve what they set out to do. That what they set out to do may not gel with our social objectives, well, that is a different problem, and does not obviate the need for more programmers. There are remarkable initiatives such as Code for America that are trying to harness the creative energies of young and not-so-young coders toward more socially higher goals, but everyone to herself. The beauty is that there is enough need to go around.

Mr. Dumbill says that "The computer already knows how it executes, much better than a human can." He wishes that "For many common purposes, a computer ought to know how to program itself." I ask how on earth can a computer know all this? I assert that, if the the computer knows, it knows because a smart programmer programmed it to know. Many commons tasks are still too difficult to perform on personal computers and even on smart phones, many of which could be smarter still. I, for one, am more interested in making scientific data access, search, analysis and viz tasks easy so domain scientists may focus on their science.

I agree with Mr. Dumbill that there is a lot of exciting innovation happening in the cloud, and voice interfaces are going to get better (although, I fear, ten years from now we will still look silly talking to an inanimate device). But, much innovation is happening and continues to be needed in hardcore computing. One area I believe in which more innovation is needed is that of metatools. We need better tools to make better tools to solve domain science problems. In fact, I personally find creating software that helps create software to be most satisfying.

In the domain of data management, and when advocating free and open access to scientific data, which entails, long term archiving and natural language mining and retrieval technologies, analytics and visualization, and all manner of computer hardware and software wizardry to deal with massive amounts of data, I see a huge disconnect between those who create and use data and those who formulate data policy.

Free is very difficult; an incredible amount of complexity and wizardry is required to make things simple. Choosing what to include is almost always easier than choosing what to leave out. I see nothing but a need for more computer programmers, really enlightened computer programmers who bridge the gap between those who need to use data and the data itself. In fact, I see a need for policy makers who are also practicing programmers and data managers. It is not enough that they programmed ten years ago, because ten years, in the world of digital information lifecycle is last century. We need those who make policy based on first-hand practice.

The only reason we collect and store data is so we can take it out again and analyze it. And, we need ever more better ways to do this. To say that the programmer's job is done subtracts from the value of this discipline. As is, programmers are generally considered lower than managers in the organizational hierarchy, and that distresses me. I take courage and strength in the words of one of the great programmers the world has ever seen. Ken Thompson's pride is evident in his statement that:

I am a programmer. On my 1040 form, that is what I put down as my occupation. As a programmer, I write programs.

Saying "We don't need more steenkin' programmers" is like saying we don't need any more masons or rail workers or carpenters. Our work has merely started. Like Mr. Dumbill, I will go out on a limb and make an unscientific assertion: teaching the following to all students as a part of their core curriculum would be immensely beneficial -- computer programming, sketching, visualization, and music appreciation. I believe today's interdisciplinary and cross-national scientific problems utilizing massive data need really holistic masons and carpenters for the information age.

I will end with quoting significantly from the authors of SICM who say eloquently:

These skills are by no means unique to computer programming. The techniques we teach and draw upon are common to all of engineering design. We control complexity by building abstractions that hide details when appropriate. We control complexity by establishing conventional interfaces that enable us to construct systems by combining standard, well-understood pieces in a "mix and match" way. We control complexity by establishing new languages for describing a design, each of which emphasizes particular aspects of the design and deemphasizes others.

Underlying our approach to this subject is our conviction that "computer science" is not a science and that its significance has little to do with computers. The computer revolution is a revolution in the way we think and in the way we express what we think. The essence of this change is the emergence of what might best be called procedural epistemology -- the study of the structure of knowledge from an imperative point of view, as opposed to the more declarative point of view taken by classical mathematical subjects. Mathematics provides a framework for dealing precisely with notions of "what is." Computation provides a framework for dealing precisely with notions of "how to."