In 2004 I started writing an article related to some musings on the effects I was seeing from the agile development practices. I eventually got the article finished in 2005. However, even today I am hearing from different companies the noises that inspired me to write my article in the first place, so it seems as relevant today as it was back in 2004. I also have an internet friend that seems to have lots of spare time, so hopefully he’ll like this. Here it is.

In the world of software construction, amongst the many new technologies and brilliant ideas that emerge to shape our industry, there’s a revolution going on. This is not another rebuke of the software industry for its poor practices and worrisome graveyard of failed software projects costing billions of dollars every year. Projects fail for many reasons and in most non-trivial endeavours, the failures will be a complex of technical and project management errors, coupled with defective business decisions that all conspire against success. Interestingly we know pretty well what causes much of the problems with large software developments, but little seems to improve. This is why this article is about something positive – a positive step being made by the software development industry to embrace a new wave of thinking and practices, which are proving to increase the quality of the software that is getting built. Of the neophyte movements in the industry, agile development is demonstrating that with disciplined use of its practices, software can be delivered that is what the customer wants and needs, and it is of a higher quality with lower defect incidents. The element of focus for this article relates to a revolution that the author deems as important as the work on modularity by David Parnas in the 1970’s, and its subtext is equally pivoted on the dedication to improving the quality of the software we write – programmer testing.

A Little History

Ever since the dawn of the computer, when the most rudimentary programs were written, it is likely such programs experienced errors, even for the likes of Von Neumann; quality has been a priority right from the earliest days of programming. Over time, as computers became more widely available, the development of better programming languages has offered measurable benefits to the quality of the software that can be produced. As far back as the 1950’s, writing computer programs was tedious and time consuming, often involving the use of switches and dials as a means of input of the machine code; a difficult endeavour fraught with problems. However, progress in this field was rapid, and by the 1960’s computers started to become more “personal” and accessible to those that could afford them. Out of the many driving forces that saw lots of research into new programming languages, moving away from 1’s and 0’s was crucial if computers were going to become mainstream. By 1957 a language called Fortran appeared on the scene, and it was deemed the first of the major programming languages. Even with a language made up of only three constructs (IF, DO and GOTO) this was a massive step forward for the computer programmers of the time.

Today the world of programming is well endowed with a wide selection of programming languages, with new concepts and developments happening every year. Crucial to this article is one of the reasons why better programming languages were needed – quality. Writing programs in raw machine code clearly wasn’t conducive to building any business critical enterprise system; it is difficult to understand and easy to create bugs – staring at a bunch of 1’s and 0’s for too long cant be good. Even in second generation assembly languages that didn’t support the sub-routine call, duplication of instructions was often the easiest way to keep a program simple enough to understand; of course today duplication is seen as one of the largest culprits behind difficult to find bugs. With the subroutine, a set of program instructions could be conceptually packaged together and referred to by name. While the subroutine is an integral part of the modern programmer’s dominion, when the concept was first introduced its impact was profound. This is not only one of the first steps towards cohesion, encapsulation and reuse, but it is also the construct that obviated the need to duplicate code, and this allows programs to be smaller and ultimately leads to less errors.

As programming languages and their compilers have become more sophisticated, a fundamental objective of the “language” is to not only provide more appropriate ways of representing concepts, but to also do as much as is possible in preventing errors to occur in the code (that is, as much as is deterministically possible with a compiler). One of the truisms of software is that no matter how well we perfect the practice, in the end humans write the programs, and humans make mistakes. Not only do we make mistakes, but also software design and construction is hard, and like all disciplines there are different levels of skill available that result in varying degrees of quality in the software that is produced. Better ways for how programs are conceptualised, modelled and transmuted into source code has been the focus of academia and industry for over half a century. There are many important milestones along the history timeline that have seen great advancements in the way programmers distil their cerebral programs into source code, with much impetus towards improving the dissonance between that which was intended, with that which is finally produced.

And so as the sophistication of the languages improved, so to did the systems that were created with them. In the youth of software engineering, the practices and principles used to create programs was rudimentary and coarse. As the movement evolved and systems became more complex, with age comes more experience, and with this new knowledge came new principles and heuristics to reason about the structure and quality of software. While object oriented programming languages had already been appreciated and well researched by the early 1970’s, the work of David L Parnas presented to the industry a set of criteria for decomposing software into modules, which is today referred to as information hiding. Parnas was rebuked by some in academia over his pivotal paper on modularity, stating that all he was doing was explaining what everybody else had been doing all along. It may be true that many already used information hiding techniques before his paper, but what Parnas did was make conscious that which was unconscious, and in so doing offered insight into the best practices of the time that the rest of the software community surely needed. There were people constructing software with code that was clean, robust, readable, maintainable and reusable (mantras of old, mantras of today). At the same time there were people who were building software that was hard to understand, difficult to read and a nightmare to maintain. In the school of best practices, necessary changes to source code could be actualised with minimal or no affect on the rest of the system. In the school of poor practices, making any change was like playing a game of Jenga – a minor change in one piece of code would cause the well-known ripple effect where remote modules would be surprisingly affected by the change; in such systems change walks hand-in-hand with fear. The ripple effect alone has a massive impact on the quality of software, its robustness and its ability to weather change in a world where every moment is woven from the fibres of the whims and needs of a problem space that is less static than academics and engineers would like to believe. Till this day, whether Parnas first conceptualised information hiding or not, the concepts he made conscious have been as pervasive and important to software quality as electricity has been to the improvement of the lives of those who harness it.

Some Realities of Testing

Improving the quality of software has been one of the objectives since programs were constructed many decades ago, and quality is a notion that includes much more than just the form and structure of code. In its own mission to develop better ways for the industry at large, academia has offered numerous techniques and methods that go a long way in proving a software system, well before a single line of code is written. There is no denying the benefits such approaches deliver, but unfortunately formal methods and other techniques that rely heavily on predicate logic have never become mainstream beyond academia or niche research and development arenas. Of course, even predicate logic or Z-based specifications rely on another aspect of quality that assumes the scientist has understood correctly the requirements of the users of the system. It doesn’t matter how elegant a specification or design is unless it is actually what the customer wants and needs. While this highlights yet another facet to the complex of quality, the purpose here is to focus on the measures that can be taken to improve the quality of code. Assuming all other variables on a project are in good order, the degree to which the software is crafted using best practices and principles plays a fundamental role in whether that software will eventually sink of float. Walking alongside the programmer’s practices in developing the software is the testing infrastructure that is put in place to aid and support the reduction of defects in the system. It will be interesting to briefly consider the levels of testing that take place in the production of enterprise software today, because as it turns out, not everybody does it the same, nor with the same amount of success or commitment. Furthermore, the role of testing in the lifecycle has different meanings to developers as it does to testers.

In today’s modern landscape of million dollar software projects constructed by the careful orchestration of legions of business analysts, architects, programmers, testers and many more, why is there still a crisis in software quality? Again, the answer is not simple. What is interesting is that there are successful projects that do well by many metrics they are gauged against. And there are those that don’t do so well…

In my 13 years of commercial experience I have witnessed more differences in the testing processes than I have seen in the way we build the software that is to be tested. What seems true from my own window onto the world is that large companies typically have bigger budgets that allow the housing of big test teams that have their own departments purely for testing. Equally true is that similar large corporations have little or no testing infrastructure, even though the money spent on in-house software development is as comparable to the other more test-conscious companies. Why is that? I have been in small, focused companies harnessing software for their core business strategy, that have budgets for maintaining software quality that is greater than the million-dollar corporations. I have been in software houses that have rigorous processes to enforce quality, and I have been in those that have little or no process, flying by the seat of their pants. There is a large improvement in the way software is built by much of the industry today, but there is still a major sector of the business world that engages in building its own software, constructing it using in-house development teams, even though software is not their core business. Some do well with this approach yet many do badly, and it often comes down to poor skill sets across the team, ineffective custom process, and inappropriate testing infrastructures

Building software is complex and it is costly. Big business is “fortunate” because it can afford to bring in external software consultancies to aid the construction process, presumably on the supposition that the expense guarantees a level of skill and experience that will map appropriately to quality and success for the project. So far in my experience this hasn’t always been true. I have had the fortune of working with a number of external consultancies with varying degrees of calibre, and to my horror I have seen software produced by these consultancies submitted to the client with absolutely no testing performed. Often the question has also arisen, “Where are the unit tests?”, only to hear from the emptiness around, “What unit tests?” So I am wrong, paying for external consultancies to write the expensive software doesn’t always map to quality or bottom-line cost savings. I feel confident in further asserting that in such cases, it really is better to bring software development in-house, for at least there is a chance of gaining control of the quality of the software.

Large testing infrastructures are a luxury, and in some shops the test phase is the one that is typically cut short, or not done at all. Testing is often seen as a necessary evil, a secondary citizen amongst the prestigious aristocracy of the architecture, design and construction society within a development team. But it is wrong to view testing in such negative light, and the psychology that is the bedrock behind this perspective is most probably in part the reason why programmers grimace at the thought of having to test their own code; oh, those bourgeois! Testing needs to rise up the social ladder of software construction, taking its rightful place within government where it can play a pivotal role in the success and quality of the software it watches over. And so, it seems there’s a revolution going on, and it’s changing the psychology of programmers the world over…

In software environments that suffer from a lack of skill or experience somewhere within its lifecycle, which don’t have the investment or ability to identify and manage this fact, will in the end pay the price on any non-trivial endeavours. The issues may come at different stages of the project, from an inability to elucidate the true user requirements and map them to a solution, through to poor architectures and less than ideal software construction practices. Teams unaware of the need to remove duplication, to increase encapsulation and reduce complexity will often build code that eventually turns into the big ball of mud many programmers will have experienced at some time in their career. Many things conspire against any initiative to build software, no matter how well intentioned its brainchild’s may be. Unfortunately part of the problem is in the minds of the programmers that build the software, insidiously playing havoc from the realms of their unconsciousness, as they go about their daily chores. The problem is simple, and can be summed up quickly – many programmers think testing is not part of their job description, and so they don’t do it, and hate it when they have to. This is probably why at different clients I have seen software submitted by both internal and external practitioners that has no accompanying tests of any description – its need remains outside the conscious awareness of a large part of the development community, and the result is that the quality of software that is produced is affected.

The Disciplined Software Practitioner, the Disciplined Practices

Of course, it is unfair to tarnish the whole of the development community with the same blithe to quality. However what is clear is that there does exist a dichotomy between software developers – those who see testing as part of their daily role, and those that do not. If we consider the waterfall model, testing comes after coding, and it is unfortunate such a process has been taken with such literal meaning. The reality is that there are development teams that see testing as the next stage once they hit their release date; their code is built, integrated, possibly smoke tested, and then the system is given to the tester(s). Worse still, and I have worked on teams that have done this, code known to have problems is passed over to the test team just so it looks like the developers hit their deadline, and that the code is ready to be released – I don’t feel proud! To some management, passing the code over the fence by the release date is acceptable, as much as it is to have the code returned shortly after with a myriad of problems. “Oh well team…at least you hit the release date! If it hadn’t been for those testers! …. ” – I have heard a team lead say this once in my career.

Even in the cases where deadlines are met with ease, the amount of testing performed by the developers varies greatly from company to company. In some cases no unit tests are written at all, whether it is an inexperienced junior programmer who can be excused, or a veteran consultant with a CV as long as his nose. There is also an interesting syndrome I have witnessed in my career, which I refer to as the “It works in my test harness” syndrome; a developer writes a mediocre test harness without consideration of the dependency problems that might occur when his code is deployed to a clean box, submits his code to the code repository, only to realise later his code fails, lamenting in his defence, “It works in my test harness.” Also worth noting here is that it is incorrect to assume that a programmer can write sufficient tests, and that the skills of a tester are a mere subset of those of a developer.

These may seem like harsh words, and unfortunately they are a reality experienced by the author, both in the UK and America. However all is not lost. Just as Parnas brought to the consciousness principles and best practices many disciplined practitioners had already been using, today there is a revolution taking place that is as important as information hiding, brought into awareness by well disciplined practitioners of today – it is programmer testing. For many programmers, writing tests as a major step in the production of their code has been a disciplined ritual they have practiced since they first learnt to program. For many more programmers, writing tests has been deemed unnecessary or annoying. So now, the tides are changing, and we have the agile community to thank for this.

Many craftsmen have always done rigorous testing just as Parnas’ peers had always been using information hiding principles, and what is now happening is that these craftsmen have shown that it is a fundamental responsibility of the programmer to consider “testing” (therefore Quality) as an integral part of his whole as a developer, that it is a matter of professionalism, and nothing less.

The Revolution: Programmer Testing

And so we arrive at the threshold of the revolution. As each moment blends into the next, programmers around the globe are being assimilated into the dogma of a new breed of software development. An invisible wave is flowing through the minds of the software development community, new and old, and slowly but surely the psychology of those who succumb to the tantalising vogue of programmer testing, is causing a beautiful ripple effect we would normally want to abate were it within our code. Better still, its inside our minds, and whether you believe in agile development and its principles or not, nay-sayers to programmer testing are socially slain as the revolution takes no prisoners; a final coup de grace?

Today it is cool to know what JUnit or NUnit is, and it is not cool if you don’t know what these tools are, regardless of whether you are an agile subscriber or not. It is an exciting time for us all, for more and more programmers are learning about programmer testing and the testing tools that are used to run such tests. You may be forgiven if you don’t know what these tools are, but your career rests on making sure you change this.

The xUnit framework was borne within the agile/XP and Smalltalk communities, with Kent Beck being the original author of SUnit, the programmer-testing framework for Smalltalk. Since the early days, SUnit has become the forefather of many different implementations for the multitude of programming languages there are in common use today. The xUnit framework provides a mechanism for the programmer to write tests using his normal programming language, and then run the tests in a simple GUI to check they all pass and the code works as expected; a stupidly simple concept that has far reaching implications to the quality of the code that can be produced. And a sacred gift that comes from the programmers writing a suite of tests as part of the coding practice is that they can now make changes to their code for reasons of improving its quality! – read this sentence again. It is so fast and easy to re-run their tests in the xUnit GUI, with each change made to the code-face can be verified to have not broken any functionality across the system simply by re-running all the tests. Simple. Profound.

As a programmer, it doesn’t matter if you do agile, and it doesn’t matter if you are even practising test-driven development with your xUnit test tool – at this stage all that matters is that you are diligent in writing your tests to accompany your code, for this in itself will help the quality of your software to reach new heights. Agilers go one step further than just writing tests and they have created a whole method known as test driven development (TDD), which is a set of principles and patterns that guide the developer in writing their programmer tests before they write a single line of production code. Furthermore, TDD is less about writing tests and more about specifying the expected behaviour of the production code that is being constructed. The tests are specifications, and the specifications act as a form of executable documentation, demonstrating whether the system is behaving as expected simply by running the specification through an xUnit derivatives GUI. A subtle yet powerful distraction that states that programmer tests are much more than tests, may well be the oil that lubricates one of the major the cogs that drive the agile machine of today.

Having a suite of test specifications to verify the system is behaving as expected opens up the software construction process to a new era. Code sculpting, which has at its heart the practice of refactoring, is yet another technique offered by the agile/XP community. Refactoring in its original form is an evolving set of commonly referred to structural code changes that are used to increase the quality-envelope of object-oriented code. As a fundamental part of the TDD process, code is shaped through a refactoring step, always in the name of improving the quality of the code. There is even a mantra that is whispered by the lips of those who TDD, “Red, Green, Refactor.” That is, write a test, see the GUI go red to indicate that the test fails, then write the actual production code the test is asserting against, and watch the GUI go green as the test passes. After a green bar has been achieved, the next step in the TDD process is to safely refactor the code, confident the GUI will turn red if anything goes wrong during this micro-stage. Refactoring is the step in software construction that the industry has always wanted to do, but often felt it was too unsafe to do so, certainly in situations that saw the code-base begin to turn into a big ball of mud. Managements misunderstanding of this crucial step in the process, paradoxically risks the very software qualia they rest their promotions on. The fact is that refactoring in some fashion is something programmers have always done (to varying degrees), and it is a step in the software process that is so fundamental to maintaining a high level of software quality, that management needs to take heed and get educated.

The reality today is that best practices and principles used for a long time by many, are now becoming part of the collective unconscious of the programmer community, and we do have the agile community to thank for such exposure. Echoing down the corridors of the corporate world today are the words of a new vocabulary, new verbs and nouns that are the manifestations of the changes in mindset the programmer world is experiencing as this revolution travels to the ends of the globe.

Theres a revolution going on. Bring on the revolution.

References:
1. http://www.blinkenlights.com/pc.shtml
2. David L Parnas, On The Criteria Used In Decomposing Systems Into Modules