Steve Yegge has posted a followup to his earlier rant about Agile. Now it seems that those of us who take agile approaches to development are superstitious and unscientific. We’ve swallowed the snake oil and don’t realize that we’ve been blinded by religion. Only Steve sees things clearly. That’s okay, he says, we can do what we want as long as we don’t try to corrupt others with our ideas.
Somehow, the possibility that the set of tools and techniques the agile community have created may actually have been developed out of observation, analysis and understanding of a very real crisis that exists in software development never seems to cross his mind. I’ll go into that in a minute, but first I want to digress into a commentary on one of the sources he cites in support of his rant.
Steve references an ACM article by Jef Raskin that talks about technical superstition. There are two key points I’d like to make about this article. Let’s start with Jef’s anecdote about the time he set up two hi-fi systems and told people that a big switch would change the sound from the the cheap amp to the expensive ones. Unsurprisingly, a lot of the people he tested it on said the expensive amp sounded better. The punch-line is that the switch actually didn’t do anything at all so people’s perception of the difference must have been based on their expectations. Jef puts this forward as part of a discussion of debunking hi-fi myths so the implication is that clearly there is no difference between cheap amps and expensive amps. However, the experiment does not demonstrate this in any way. All it demonstrates is that there is a thing called a placebo effect where people’s expectation colours their perception. There is nothing new about this. It’s why science uses double-blind tests to test things. The experiment says nothing at all about the difference in quality of amplifiers, or about any other experiment which one might do to test people’s perception of the quality of amplifiers, any more than would a setup where the switch actually did work.
The second point I want to bring up from this article is Jef’s discussion about CD players and the “bogus” claims by makers of such things as weights to stop the “wow and flutter” of the CD from affecting the sound. Jef asserts that because the data on the CD is digital and because bits are just 1 and 0 then the data coming off the CD must be perfect and any addition to the mechanism driving the CD could not possibly affect the sound quality. I, too, used to counter such claims with this argument until one day someone pointed out to me that, in fact, the data coming of an average CD in an average CD drive actually has an astonishingly high bit error-rate. One of the problems with a CD is that because it is reading in real-time there is no opportunity to go back and re-read the data. CD audio data has quite strong error detection built in but limited error correction. The upshot is that when the player detects that the data coming off the disc has errors, it attempts to compensate for that by interpolating the signal. In other words, it makes it up. Although modern CD drives are better at dealing with these situations they still have a lot of errors. That is why iTunes, or any other MP3 CD ripper, has an option to re-read on errors when ripping so that the resulting MP3 file doesn’t contain skips or noise. Try switching it off sometime and ripping an even mildly scratched CD and see how it sounds.
Once you factor in this detail, i.e. that the data coming off the CD may be 1’s and 0’s but it isn’t necessarily the same 1’s and 0’s that were written to it, the wholesale debunking of such things as the rim-wieghts on the basis that you can’t improve on perfect, goes away. Note, that I am not making any claims as to whether any specific devices or techniques might have a material affect on the sound, merely pointing out that the primary “proof” against them having any effect is not valid. It also provides for a convenient and easy experiment to test the claims. Simply create a setup which allows you to record the bit error-rate from a CD and see if there is any change when you apply the ’snake oil’.
Science has always operated on this basis. Theories always contain axioms, which are really just assumptions that are considered so obvious that no-one challenges them. Occasionally, someone comes along and successfully challenges an axiom and science undergoes a revolution. Einstein did it to Newton. Newton’s laws worked perfectly well for hundreds of years but they contained the axiom (assumption) that there was an absolute background geometry to space (and time) that didn’t change. All the experiments we were capable of doing found that to be true. However, Einstein pointed out that at one extreme (the very large, very heavy and very fast), this assumption broke down and we got the General Theory of Relativity.
So what has all this got to do with agile software development? Well, very little really. I just wanted to get that off my chest and to demonstrate how people’s smug assurance that something is obviously and unquestionably scientifically false can be punctured by adding a previously unrecognized, but highly salient fact to the equation.
The reason it has nothing to do with software development is that software development is not science. To some it is engineering, to some, a craft, for others an art and often considered a mixture of all of those and more. When developing software, or, indeed, software methods, we do not create theories, subject them to rigourous analysis and then devise experiments to confirm or deny them (although test-driven development, a key feature of many agile methods, is as close as most programmers have ever got to it). Our development processes are tested in the marketplace in the real world and can succeed or fail for so many different reasons that no current scientific method could possibly hope to make sense of it. Of course we can try and measure things like line counts, defect rates, etc. but as has been pointed out many times, programmers are smart people and the act of measurement usually results in them re-optimising their development activities around the metric being measured. Additionally, development projects are chaotic so there is no practical way to set up controlled, repeatable experiments to rigourously test one method over another.
There is, however, one glaring metric which stands out and points to a very real crisis in software development. Here in the UK, the figure typically quoted for government software development projects is that 65% of them end in failure. You can argue over the precise figure. Perhaps it’s 55%, 45% or maybe 75%. The fact is that a lot of very large, very expensive projects conclude without delivering what they originally set out to do, spending far more than budgeted and often being cancelled completely. Clearly something is wrong with the way these projects are being carried out. Private sector project failure rates are lower, but still unacceptably high, usually being quoted as 45%.
This is the background against which agile methods have been developed. Not as an alternative way for small startups to operate, or companies like Google, but as a reaction to the astoundingly bad success rates for real projects. The various methods have evolved thought taking a good, hard look at the conventional wisdom (primarily waterfall methods) and the practical problems associated with those methods and making changes to address those problems. Probably the single most important understanding is how most software development processes try to nail down requirements at the start and prevent change happening in the later stages of the project where the approach to development made it expensive. For me, the most important thing that Kent Beck, et al, did was to point out that it is possible to flatten the cost-change curve by taking a different approach. Here was an axiom of software development (it is more expensive to make changes the later you are in a project) which didn’t have to be true, and XP put forward a set of interrelated practices that said “here’s one way of making that true”.
For Steve, perhaps that isn’t such a big issue. He probably already works in a way that allows for change in later stages of a project and, anyway, what does ‘later stage’ mean when there’s no delivery schedule? When you have deep pockets, the cost-change curve doesn’t apply. Here in the real world, there are external forces that impose deadlines. When you’re building a processor to parse tax return documents being submitted over the internet, then it has to be ready for the filing deadline. I can’t tell my client “it’ll ship when it’s ready”. If I miss that deadline, we lose the contract and ministers have to stand up in parliament and explain what went wrong.
On the other hand, perhaps the same forces do apply, but just on a time-scale which means Google have yet to experience the pain of their approach (though Sergey has hinted that this may already be the happening). As deep as Google’s pockets are, they are not infinite and they can only operate as a charitable institute for programmers for so long on the back of their advertising revenue. Again, Steve should reflect that the pay he takes home each month is not related to his programming activities, but simply the result of the founders having a good idea and attracting enough eyeballs to make advertising revenues not just a viable revenue stream, but a veritable torrent of income. I’m reminded of another company that made one good deal and has been living off it ever since.
The rest of us have to test our ideas in the most scientific way that software development has yet devised - the commercial marketplace. It’s a poor substitute for a particle accelerator, but does its best to keep us real.
Is there a lot of hype and bullshit surrounding Agile? Of course. The same is true of every idea that has ever come along in history. I’ve been programming long enough to remember, for example, the Open System wars, or the advent of object oriented programming. Those, and other revolutions driven by great ideas, attracted their own share of companies who traded on buzzwords only, but that didn’t invalidate the real potential of the ideas. My biggest objection to Steve’s posts is that he thinks we should not try to communicate our ideas and our experiences with agile. For someone working in an industry and a company that have done more than any other in human history to promote the communication and accessibility of ideas, I find this very perverse.
There are always unscrupulous people and organizations that will try and dupe unsuspecting people by perverting good ideas into snake oil, but don’t let that blind you to things that might actually make a difference to the crisis facing real-world software development.
Technorati Tags: agile, programming
