This week alone I have coded in Java, Javascript, HTML, CSS, PHP, Flex, Actionscript, Antlr and a domain-specific-language of my own that describes optimizations in ASTs, not to mention a variety of XML document formats.
The syntax gets very confused after a while :-)
I’ve been reading “Dreaming in Code” recently. This is written by a journalist, Scott Rosenberg, who embedded himself in an open source start-up company a few years ago to follow the progress of the development project, in much the same spirit as “Soul of a New Machine”.
In an early chapter, Scott explores the traditional ideas about software development evolving towards a ‘lego’ model where software is created by snapping together pre-formed units to form an application, and then ponders the reasons why this has not happened. This has got me to thinking. Do we really want to aspire to a ‘lego’ model for software? Are we really suggesting that software should be compared to a child’s toy? After all, look at any lego structure and tell me you would want to use a software equivalent for anything. In fact, look at the world around you? Tell me, is any of your furniture made of lego? Your household appliances? Your house? Lego structures may look impressive, but only in a sort of dancing-dog kind of way. The key features of any lego house I’ve ever seen, for instance, are that all the top surfaces are covered in small, round, protuberances and everything is squared-off and all the corners are sharp. Oh yes, and it’s a toy. Even if you built a full-size house out of lego, I doubt anyone would want to live in it, except perhaps as a post-modernist art exhibit.
The usual comparison that is made at this point is with electronics. After all, aren’t there lots of standard electronic components that you can plug together to make, say, an amplifier? It doesn’t matter which electronics company you get them from, you just take standard transistors, op-amps, resistors, capacitors, stick them on a circuit board according to established patterns and, hey presto, an amplifier. What are you going to do with it then? Well, at that point you’d write ‘Matsushita’ on the outside of the case and put it on the shelf of Dixons for £49.95 to sell to people who don’t give a damn about audio quality. The fact is that anyone who wants to make an amplifier that doesn’t sound like a couple of tin cans tied together with string takes care to select high-quality components from specific manufacturers; builds their own power circuits; develops creative new patterns for circuitry that make the best of those components and minimises interference between them. They test them and then make changes and test them again. Then you can put Quad or Linn on the outside and sell them to people who care about sound. BTW, there is a “standard component” way of developing software. It’s called Visual Basic. You can very quickly knock up a useful, database-backed GUI in VB with all the usual bits and pieces such as text fields, combo boxes, radio buttons, etc. as well as much more complex things such as spreadsheet components in a very short time. But when you see one of these apps you’ll notice that the main defining characteristics are the software equivalent of there being small round protuberances on all the top surfaces, everything is squared-off and the corners are sharp. In a lot of cases, these apps are no more than toys (though, contrary to real lego, it is possible to build non-toys as well.)
Take another example: the car industry. This is a very mature engineering industry, but do they have a lego model? Look outside at all the different cars you see. Even the cheap ones. Can you take a door off a Nissan and stick it on a Ford? Even within the same brand, the main visible components are not interchangeable, they’ve been tailored to the design. There is an interesting parallel between car manufacturers and software development, however. That is the concept of the ‘Platform’. In the past couple of decades car manufacturers have realised that developing a new car from the ground up is just too expensive. Now, several different manufacturers will get together and design a chassis together and use that as the basis for a whole range of different cars of different makes. A lot of variation can be built on top of a good platform for a fraction of the cost of ground-up. But that is exactly what the software industry has been doing over much the same time period. In the early days if you wanted to make a computer you build the hardware and put your own bespoke software on top. This was the approach taken for mainframes and mini-computers and the early micro-computers. But then in the 80’s people customers demanding cross-compatibility and since then the industry has standardised on a few, common platforms. There are the Unix-based platforms (including Linux and OS-X), the Windows platform and a few mobile phone platforms. In addition, the hardware platforms have become more standardised, too, with the IBM PC architecture being the most common. IBM soon lost control of that and advances were driven by a wide consensus between manufacturers. No-one builds a new computer from the ground-up anymore. Even Apple have moved to much more standard Intel-based components. They just take more care in putting them together than most companies. The variety of systems you can build on these few platforms is immense.
But it is interesting to note that the computer industry reached this point only a few decades after it’s inception. The motor industry took close to a century to get there. Perhaps we’re not so far behind in our thinking and practices as some people would have us believe.
I’ve updated the theme on my blog to introduce that quintessential web 2.0 feature, rounded corners and I thought I’d share some of my thoughts on this.
For guidance I referred to chapter 3 of Andy Budd’s excellent book, CSS Mastery. I first tried the “flexible rounded-corner box” approach. This wraps three layers of div around the content and uses four images to provide the corners by placing them appropriately as background images. The basic technique is to create images with the correct background colour and use a “sliding window” technique so that as the content size grows or shrinks (say by changing the font size), more or less of the left top and left bottom images are exposed. Since these are of a constant colour the user sees it as the corners moving.
The concept is straightforward so I fired up Photoshop and dived in. My first realisation was how annoying it is to generate the right images with photoshop/image ready (remember, I’m a programmer not a designer). After figuring out that I needed to use Image Ready instead of Photoshop to get the rounded rectangle marquee tool, and searching for several minutes before finding the option to change the radius of the corners, I finally created a nice rounded-corner rectangle. Then I had to chop it up into gifs for each of the four corners, a tedious process.
My assumption had been that I would just need the corners and I could rely on the background colour of the content to fill in the rectangle but after installing the images and CSS I realised this isn’t the case. The top left and bottom left images need to be large enough to fill out the content no matter how tall or wide it gets. I don’t like this as it puts a maximum size on the content box unless I use very large images. I don’t like that kind of constraint. Also, this approach requires that I make different corner image sets for each colour box I might have so I moved on to the next section of the book.
This is entitled “Mountaintop Corners” and takes the “opposite” approach. Instead of filling in the corner, this technique uses gifs to mask off the corner. Basically, I created four images where the outside of the corner is white and the inside is transparent so that the content colour can show through. This is much more flexible in that it can be used on any box on the site regardless of the content colour. The limitation is that I would need to create a different set of corners for each background colour I might be using. In my case, the background is always white so I can get away with one set of corners.
Here’s the result:
Okay, not the most amazing of boxes I’ve seen (hey, no gradients, drop shadows or even anti-aliasing) but somewhat nicer than the rectangular look I had before, I think. Plus, it only took the most minor of tweaking to get it all looking good in FF 1.5, IE 6, Opera 9 and Safari, which is a bonus. All in all it took about an hour to do.
But all this raises the question in my mind as to when CSS will start to include things like rounded corners as native features. It seems bizarre to always be having to jump through these kinds of hoops to get visual tricks like this up and running.
I was reading Jeremy Keith’s journal entry about his upcoming presentation at Flash on the Beach in which he mentions that Neville Brody is speaking at the conference. I hadn’t heard of Neville (I’m a programmer, not a designer after all) but reading his profile entry led me to Research Studios, which is his design company.
I spotted the Guardian on the project list and, having worked on the Guardian Unlimited sites in 1998, I clicked through. What do you know. Research Studios developed the block-layout design that I helped architect and implement using Vignette StoryServer. At the time I was much more of a heads-down programmer and never realized what illustrious company I had kept.
I latched on to dojo early in my Ajax career (that would have been, oh, say 4 months ago). The approach the dojo developers take to modularity, classes and so forth appealed to the Java programmer in me. It made things familiar enough to get a handle on easily. Plus, I went to see Dylan Schiemann present a workshop and was impressed by the depth of his knowledge.
However, as I started to work seriously on our application, I began to get the feeling that I was missing something fundamental about JavaScript by hiding it behind the familiar constructs of classes, inheritance and encapsulation in the form that I am used to. Because real work (you know, the stuff that pays the bills) came up I needed to pause what I was doing with JavaScript, but in the meantime, I began to take a close look at HTML and CSS design. In my spare minutes I re-developed our company website to use the ideas I was learning from Andy Budd and others about separating content from presentation, making the design flexible to handle different screen sizes and printers, and things like using CSS for rollovers rather than JavaScript actions.
Then, last month, I came across jQuery for the first time. The first thing that impressed me was the documentation. It’s not 100% comprehensive, but it’s a lot better than dojo’s documentation and good enough to get going straight away. The other thing that struck me, however, was the different focus it has. Using dojo, I felt as if I was using a programming language first and DOM stuff second. I’m not saying that that is the fault of dojo, but its efforts to make things more Java-like meant I didn’t need to break out of my mental model. Also, jQuery has the feel of a second wave of toolkit, building on the lessons learned from things like dojo and protoype.
jQuery most definitely focuses on the DOM first. It provides a lean, but well-formed set of constructs to search and manipulate DOM trees. The searching bit uses CSS selector strings (or XPath, but I like the CSS) which are now more familiar to me. My own focus has now shifted and I’m seeing the client-side application as primarily visual elements with behaviour attached, rather than application behaviour with views attached as I had been. I can see pretty clearly now how this fits in with an MVC-style and how I can connect it up with the messaging system we wrote earlier to reflect changes in our business domain objects.
A bonus is the large number of plugins that exist for jQuery, written by third parties. I am making use of the drag and drop features of the Interface plugin and already I am seeing a big reduction in the amount of code I am having to write to get things done.
Throughout the 25 years or so that I have been writing software I’ve looked at a lot of different languages. I’ve always believed that even if a language was not practical to use for whatever sort of problem I was trying to solve at the time, it is still valuable to be pushed into thinking about things in a different way. I’m starting to feel that JavaScript and jQuery are causing me to think in new ways
Technorati Tags: jquery, programming, javascript, Web2.0
Steve Yegge has posted a followup to his earlier rant about Agile. Now it seems that those of us who take agile approaches to development are superstitious and unscientific. We’ve swallowed the snake oil and don’t realize that we’ve been blinded by religion. Only Steve sees things clearly. That’s okay, he says, we can do what we want as long as we don’t try to corrupt others with our ideas.
Somehow, the possibility that the set of tools and techniques the agile community have created may actually have been developed out of observation, analysis and understanding of a very real crisis that exists in software development never seems to cross his mind. I’ll go into that in a minute, but first I want to digress into a commentary on one of the sources he cites in support of his rant.
Steve references an ACM article by Jef Raskin that talks about technical superstition. There are two key points I’d like to make about this article. Let’s start with Jef’s anecdote about the time he set up two hi-fi systems and told people that a big switch would change the sound from the the cheap amp to the expensive ones. Unsurprisingly, a lot of the people he tested it on said the expensive amp sounded better. The punch-line is that the switch actually didn’t do anything at all so people’s perception of the difference must have been based on their expectations. Jef puts this forward as part of a discussion of debunking hi-fi myths so the implication is that clearly there is no difference between cheap amps and expensive amps. However, the experiment does not demonstrate this in any way. All it demonstrates is that there is a thing called a placebo effect where people’s expectation colours their perception. There is nothing new about this. It’s why science uses double-blind tests to test things. The experiment says nothing at all about the difference in quality of amplifiers, or about any other experiment which one might do to test people’s perception of the quality of amplifiers, any more than would a setup where the switch actually did work.
The second point I want to bring up from this article is Jef’s discussion about CD players and the “bogus” claims by makers of such things as weights to stop the “wow and flutter” of the CD from affecting the sound. Jef asserts that because the data on the CD is digital and because bits are just 1 and 0 then the data coming off the CD must be perfect and any addition to the mechanism driving the CD could not possibly affect the sound quality. I, too, used to counter such claims with this argument until one day someone pointed out to me that, in fact, the data coming of an average CD in an average CD drive actually has an astonishingly high bit error-rate. One of the problems with a CD is that because it is reading in real-time there is no opportunity to go back and re-read the data. CD audio data has quite strong error detection built in but limited error correction. The upshot is that when the player detects that the data coming off the disc has errors, it attempts to compensate for that by interpolating the signal. In other words, it makes it up. Although modern CD drives are better at dealing with these situations they still have a lot of errors. That is why iTunes, or any other MP3 CD ripper, has an option to re-read on errors when ripping so that the resulting MP3 file doesn’t contain skips or noise. Try switching it off sometime and ripping an even mildly scratched CD and see how it sounds.
Once you factor in this detail, i.e. that the data coming off the CD may be 1’s and 0’s but it isn’t necessarily the same 1’s and 0’s that were written to it, the wholesale debunking of such things as the rim-wieghts on the basis that you can’t improve on perfect, goes away. Note, that I am not making any claims as to whether any specific devices or techniques might have a material affect on the sound, merely pointing out that the primary “proof” against them having any effect is not valid. It also provides for a convenient and easy experiment to test the claims. Simply create a setup which allows you to record the bit error-rate from a CD and see if there is any change when you apply the ’snake oil’.
Science has always operated on this basis. Theories always contain axioms, which are really just assumptions that are considered so obvious that no-one challenges them. Occasionally, someone comes along and successfully challenges an axiom and science undergoes a revolution. Einstein did it to Newton. Newton’s laws worked perfectly well for hundreds of years but they contained the axiom (assumption) that there was an absolute background geometry to space (and time) that didn’t change. All the experiments we were capable of doing found that to be true. However, Einstein pointed out that at one extreme (the very large, very heavy and very fast), this assumption broke down and we got the General Theory of Relativity.
So what has all this got to do with agile software development? Well, very little really. I just wanted to get that off my chest and to demonstrate how people’s smug assurance that something is obviously and unquestionably scientifically false can be punctured by adding a previously unrecognized, but highly salient fact to the equation.
The reason it has nothing to do with software development is that software development is not science. To some it is engineering, to some, a craft, for others an art and often considered a mixture of all of those and more. When developing software, or, indeed, software methods, we do not create theories, subject them to rigourous analysis and then devise experiments to confirm or deny them (although test-driven development, a key feature of many agile methods, is as close as most programmers have ever got to it). Our development processes are tested in the marketplace in the real world and can succeed or fail for so many different reasons that no current scientific method could possibly hope to make sense of it. Of course we can try and measure things like line counts, defect rates, etc. but as has been pointed out many times, programmers are smart people and the act of measurement usually results in them re-optimising their development activities around the metric being measured. Additionally, development projects are chaotic so there is no practical way to set up controlled, repeatable experiments to rigourously test one method over another.
There is, however, one glaring metric which stands out and points to a very real crisis in software development. Here in the UK, the figure typically quoted for government software development projects is that 65% of them end in failure. You can argue over the precise figure. Perhaps it’s 55%, 45% or maybe 75%. The fact is that a lot of very large, very expensive projects conclude without delivering what they originally set out to do, spending far more than budgeted and often being cancelled completely. Clearly something is wrong with the way these projects are being carried out. Private sector project failure rates are lower, but still unacceptably high, usually being quoted as 45%.
This is the background against which agile methods have been developed. Not as an alternative way for small startups to operate, or companies like Google, but as a reaction to the astoundingly bad success rates for real projects. The various methods have evolved thought taking a good, hard look at the conventional wisdom (primarily waterfall methods) and the practical problems associated with those methods and making changes to address those problems. Probably the single most important understanding is how most software development processes try to nail down requirements at the start and prevent change happening in the later stages of the project where the approach to development made it expensive. For me, the most important thing that Kent Beck, et al, did was to point out that it is possible to flatten the cost-change curve by taking a different approach. Here was an axiom of software development (it is more expensive to make changes the later you are in a project) which didn’t have to be true, and XP put forward a set of interrelated practices that said “here’s one way of making that true”.
For Steve, perhaps that isn’t such a big issue. He probably already works in a way that allows for change in later stages of a project and, anyway, what does ‘later stage’ mean when there’s no delivery schedule? When you have deep pockets, the cost-change curve doesn’t apply. Here in the real world, there are external forces that impose deadlines. When you’re building a processor to parse tax return documents being submitted over the internet, then it has to be ready for the filing deadline. I can’t tell my client “it’ll ship when it’s ready”. If I miss that deadline, we lose the contract and ministers have to stand up in parliament and explain what went wrong.
On the other hand, perhaps the same forces do apply, but just on a time-scale which means Google have yet to experience the pain of their approach (though Sergey has hinted that this may already be the happening). As deep as Google’s pockets are, they are not infinite and they can only operate as a charitable institute for programmers for so long on the back of their advertising revenue. Again, Steve should reflect that the pay he takes home each month is not related to his programming activities, but simply the result of the founders having a good idea and attracting enough eyeballs to make advertising revenues not just a viable revenue stream, but a veritable torrent of income. I’m reminded of another company that made one good deal and has been living off it ever since.
The rest of us have to test our ideas in the most scientific way that software development has yet devised - the commercial marketplace. It’s a poor substitute for a particle accelerator, but does its best to keep us real.
Is there a lot of hype and bullshit surrounding Agile? Of course. The same is true of every idea that has ever come along in history. I’ve been programming long enough to remember, for example, the Open System wars, or the advent of object oriented programming. Those, and other revolutions driven by great ideas, attracted their own share of companies who traded on buzzwords only, but that didn’t invalidate the real potential of the ideas. My biggest objection to Steve’s posts is that he thinks we should not try to communicate our ideas and our experiences with agile. For someone working in an industry and a company that have done more than any other in human history to promote the communication and accessibility of ideas, I find this very perverse.
There are always unscrupulous people and organizations that will try and dupe unsuspecting people by perverting good ideas into snake oil, but don’t let that blind you to things that might actually make a difference to the crisis facing real-world software development.
Technorati Tags: agile, programming
Steve Yegge doesn’t like Agile. All that hype, marketing, consultancy. Except that recently he decided he likes a bit of it; ‘agile’ with a small ‘a’.
Agile proponents think estimating is hard so they try and make the estimates very short and check their assumptions frequently rather than try and estimate out weeks or months (or even years) into the future. Steve thinks estimating is hard so he thinks you shouldn’t do it at all. Apart from that, everything he talks about is actually very close to the essence of agile. He wants project managers who keep things running smoothly, eliminating obstacles, rather than managers who tell people what to do next. He wants people to be co-located so that they can communicate when they need to. The list goes on but, for the most part, what he describes actually is what agile is about. Of course Agile (big ‘A’) is also a fad which is used to sell consultancy, books and proprietary processes. That’s because really understanding the essence of agile is hard for a lot of people and they are much more comfortable buying things which make them feel like they’re doing the latest thing when in fact they’re just repackaging the same ideas about software development that have been around for decades.
What I take exception to in Steve’s post is when he describes Google’s approach to software development and puts it forward as if it is some kind of viable model for other companies to adopt. Things like never telling developers what to work on, allowing developers to switch teams whenever they like, encouraging them to spend 20% of their office time on projects unrelated to their main project. Sounds great, but let’s take a reality check here. Google is a company that:
In other words, google can run their software development in the way that Steve describes simply because they do not need to make any money from those activities to survive (at least in the medium term). It is interesting to see this week Sergey Brin announcing that he is asking google developers to stop making new products and actually concentrate on finishing the ones that are already there and get them to interoperate with each other. Perhaps allowing developers to jump to new projects when they want, never setting deadlines or release dates, and encouraging them to work on things other than their main project is not such a great boon to software development after all.
The one key thing about agile software development that Steve seems to miss is that it is, in my opinion, the most successful approach to date of actually getting working software out the door. Companies other than Google rely on that because they actually have to sell their products to stay in business, not keep clever ideas in beta for years and years. Most of us don’t have the luxury of the deep pockets which allow Google to operate the way they do.
Don’t get me wrong. I’m glad that Google exists. I’m glad that there exists an environment in which very smart software developers can explore ideas. I think Google mail changed the world by showing that rich browser apps were a viable way of serving millions of users, for example. I just don’t think that anyone at Google is in a position to be lecturing the ‘real world’ on how software development should be done.
Technorati Tags: agile, programming
Or at least it is what programming is becoming. If anything defines Web2.0 for me then it is the idea of moving the software into the web itself (hey, the network is the computer!) Web 1.0 was about getting users to interact with websites. The websites, by and large, were (and mostly still are, for that matter) islands. The user needed to hop from island to island to do several things, sometimes copying the data by hand to get something done, e.g. taking a post-code from an address on ons site and copying it into the search box on streetmap or similar. If you were lucky, the website designer had done this simple sort of stuff for you.
Now, however, it is becoming possible to use the APIs which a lot of sites, such as Flickr, Technorati, Amazon, Yahoo!, Google and so on, are providing to produce actual applications. Those sites are building core services that others can create richer applications on. The basic stuff is being pushed down the stack just as with any programming language libraries and frameworks. The system is evolving.
This is heralding a fundamental shift in the nature of programming. These APIs will become standardized over time so I won’t be tied to Flickr to hold my photos, or Google for my maps. Standards groups will form to create abstract APIs and these web service provider will either conform to them or wither. This is the network effect in action.
These APIs are programming language independent. They are mostly specified in XML or JSON and libraries exist for every mainstream language to make calls in those formats. The new applications will be on the web themselves, or be rich, desktop, connected applications. The ‘connected’ app is a powerful idea. Let me use the power of my computer running a proper desktop application talking to a variety of data sources on the web. The iTunes Store is like this. The blog software (ecto) that I’m using to write this does this. It’s nice.
The biggest immediate problem such an application face is what to do when a service that it relies on is unavailable. In theory, any application is like this, but we have become used to the reliability and availability of desktop machines, local databases, etc, that such a failure can be taken as fatal and a small amount of downtime tolerated until the problem is rectified. Web services are not so highly available at the moment. It could be that the service itself is down, or that the network has failed somewhere between us and them. The standardization of APIs might mean that I can fall back to a different mapping service if my main service is down, but what about if I want to get to my photos?
Another reason I like the idea of connected desktop apps is that there is the opportunity to cache data locally so that if I am totally disconnected from the net then I can still work and things will synchronize when I re-connect. (This sounds great, but something I’ve learned over the years is that synchronization is hard to do and few implementations do it well. Maybe there is the opportunity for a web service to do it right and make it easy?) Alternatively, because my ’stuff’ is all on the web as well, if I don’t have access to my desktop or laptop then I can still get to the data and functionality I need via a web browser.
We’re still in the early stages of all of this. The Web2.0 crowd are as small (but growing) band of pioneers and it will be down to us to experiment, innovate and resolve these issues before the whole idea can become as mainstream and embedded as, say, object orientation is now that even languages like Basic support it. It is, as always, an exciting time to be a programmer.
Technorati Tags: programming, Web2.0
When I’m developing in JavaScript I primarily use Firefox to display the results as it has really good tool support for debugging code and inspecting the resulting page. Every now and then I cross-check in IE 6 to make sure things are still working.
I was very surprised the other day to find that when I took all the style settings I was applying to DOM elements created in JavaScript out of the JS code and put them in a separate style sheet, that all the style formatting disappeared when viewing things in IE. It took a good half-hour of hair-pulling, google searches and referencing the dozen or so books I have available on web development before I finally stumbled across a fragment of code which pointed me in the right direction. I had been setting the class name of the new elements using:
domNode.setAttribute("class", "classname");
which works perfectly well in Firefox. However, IE seems to completely ignore this (although the DOM inspector from the IE web development toolbar showed the class apparently being set). The code I came across showed this instead:
domNode.className = "classname";
After converting my JavaScript to use this form, lo and behold my styles were correctly applied in both browsers. This is the kind of thing which I have come across a lot since starting to work with JS - it is really hard to find answers to some really basic things like this. One thing that is clear is that most of the books and other literature is approaching JavaScript from the point of view of HTML and CSS designers, not people coming from a background of software development in languges like Java, C++ and C.
Powered by WordPress
