On my knees, in pain, a broken man with nothing left in me than the last breaths of a dieing geek, I reached over and, with my last utterance, said, “Bloody Microsoft Press” and I fell unconscious on the floor.

Well I lived to tell the tale, and the nurses are pretty nice here in A&E too. I’ve just seen the doctor, and it has been explained to me that I had attempted to lift this : dead-weight. Apparently, this is not a book, but is in fact nothing other than a heavy weight one would benchpress in the gym. Furthermore, I’m the third person in the hospital this week! Some other happless techy attempted to pick up this Its gotta hurt, at 2704 pages! The doctor said it was like the guy had attempted to lift a small goat, without the appropriate warm-up.

So, it seems Microsoft has realized the health implications of us geeks spending all of these hours sat in front of our computers “burning the midnight oil”, and has gone into producing books the size of small farm animals, with which we can actually use to keep ourselves fit. If, when the cold chill disturbs you from the screen in the dark of night, when everyone else is in bed and you’re still on your computer, dont think its time to go to bed yet. No. Pick up your latest Microsoft Press “book”, get down on the deck, and start to benchpress! In fact I’m sure you could manage a whole workout from just two Microsoft books, but do make sure you work your way up with the lighter weights, first, and dont end up in A&E like me!

Nurse??!!

In the world of software construction, amongst the many new technologies and brilliant ideas that emerge to shape our industry, there’s a revolution going on. This is not another rebuke of the software industry for its poor practices and worrisome graveyard of failed software projects costing billions of dollars every year. Projects fail for many reasons and in most non-trivial endeavours, the failures will be a complex of technical and project management errors, coupled with defective business decisions that all conspire against success. Interestingly we know pretty well what causes much of the problems with large software developments, but little seems to improve. This is why this article is about something positive – a positive step being made by the software development industry to embrace a new wave of thinking and practices, which are proving to increase the quality of the software that is getting built. Of the neophyte movements in the industry, agile development is demonstrating that with disciplined use of its practices, software can be delivered that is what the customer wants and needs, and it is of a higher quality with lower defect incidents. The element of focus for this article relates to a revolution that the author deems as important as the work on modularity by David Parnas in the 1970’s, and its subtext is equally pivoted on the dedication to improving the quality of the software we write – programmer testing.

A Little History

Ever since the dawn of the computer, when the most rudimentary programs were written, it is likely such programs experienced errors, even for the likes of Von Neumann; quality has been a priority right from the earliest days of programming. Over time, as computers became more widely available, the development of better programming languages has offered measurable benefits to the quality of the software that can be produced. As far back as the 1950’s, writing computer programs was tedious and time consuming, often involving the use of switches and dials as a means of input of the machine code; a difficult endeavour fraught with problems. However, progress in this field was rapid, and by the 1960’s computers started to become more “personal” and accessible to those that could afford them. Out of the many driving forces that saw lots of research into new programming languages, moving away from 1’s and 0’s was crucial if computers were going to become mainstream. By 1957 a language called Fortran appeared on the scene, and it was deemed the first of the major programming languages. Even with a language made up of only three constructs (IF, DO and GOTO) this was a massive step forward for the computer programmers of the time.

Today the world of programming is well endowed with a wide selection of programming languages, with new concepts and developments happening every year. Crucial to this article is one of the reasons why better programming languages were needed – quality. Writing programs in raw machine code clearly wasn’t conducive to building any business critical enterprise system; it is difficult to understand and easy to create bugs – staring at a bunch of 1’s and 0’s for too long cant be good. Even in second generation assembly languages that didn’t support the sub-routine call, duplication of instructions was often the easiest way to keep a program simple enough to understand; of course today duplication is seen as one of the largest culprits behind difficult to find bugs. With the subroutine, a set of program instructions could be conceptually packaged together and referred to by name. While the subroutine is an integral part of the modern programmer’s dominion, when the concept was first introduced its impact was profound. This is not only one of the first steps towards cohesion, encapsulation and reuse, but it is also the construct that obviated the need to duplicate code, and this allows programs to be smaller and ultimately leads to less errors.

As programming languages and their compilers have become more sophisticated, a fundamental objective of the “language” is to not only provide more appropriate ways of representing concepts, but to also do as much as is possible in preventing errors to occur in the code (that is, as much as is deterministically possible with a compiler). One of the truisms of software is that no matter how well we perfect the practice, in the end humans write the programs, and humans make mistakes. Not only do we make mistakes, but also software design and construction is hard, and like all disciplines there are different levels of skill available that result in varying degrees of quality in the software that is produced. Better ways for how programs are conceptualised, modelled and transmuted into source code has been the focus of academia and industry for over half a century. There are many important milestones along the history timeline that have seen great advancements in the way programmers distil their cerebral programs into source code, with much impetus towards improving the dissonance between that which was intended, with that which is finally produced.

And so as the sophistication of the languages improved, so to did the systems that were created with them. In the youth of software engineering, the practices and principles used to create programs was rudimentary and coarse. As the movement evolved and systems became more complex, with age comes more experience, and with this new knowledge came new principles and heuristics to reason about the structure and quality of software. While object oriented programming languages had already been appreciated and well researched by the early 1970’s, the work of David L Parnas presented to the industry a set of criteria for decomposing software into modules, which is today referred to as information hiding. Parnas was rebuked by some in academia over his pivotal paper on modularity, stating that all he was doing was explaining what everybody else had been doing all along. It may be true that many already used information hiding techniques before his paper, but what Parnas did was make conscious that which was unconscious, and in so doing offered insight into the best practices of the time that the rest of the software community surely needed. There were people constructing software with code that was clean, robust, readable, maintainable and reusable (mantras of old, mantras of today). At the same time there were people who were building software that was hard to understand, difficult to read and a nightmare to maintain. In the school of best practices, necessary changes to source code could be actualised with minimal or no affect on the rest of the system. In the school of poor practices, making any change was like playing a game of Jenga – a minor change in one piece of code would cause the well-known ripple effect where remote modules would be surprisingly affected by the change; in such systems change walks hand-in-hand with fear. The ripple effect alone has a massive impact on the quality of software, its robustness and its ability to weather change in a world where every moment is woven from the fibres of the whims and needs of a problem space that is less static than academics and engineers would like to believe. Till this day, whether Parnas first conceptualised information hiding or not, the concepts he made conscious have been as pervasive and important to software quality as electricity has been to the improvement of the lives of those who harness it.

Some Realities of Testing

Improving the quality of software has been one of the objectives since programs were constructed many decades ago, and quality is a notion that includes much more than just the form and structure of code. In its own mission to develop better ways for the industry at large, academia has offered numerous techniques and methods that go a long way in proving a software system, well before a single line of code is written. There is no denying the benefits such approaches deliver, but unfortunately formal methods and other techniques that rely heavily on predicate logic have never become mainstream beyond academia or niche research and development arenas. Of course, even predicate logic or Z-based specifications rely on another aspect of quality that assumes the scientist has understood correctly the requirements of the users of the system. It doesn’t matter how elegant a specification or design is unless it is actually what the customer wants and needs. While this highlights yet another facet to the complex of quality, the purpose here is to focus on the measures that can be taken to improve the quality of code. Assuming all other variables on a project are in good order, the degree to which the software is crafted using best practices and principles plays a fundamental role in whether that software will eventually sink of float. Walking alongside the programmer’s practices in developing the software is the testing infrastructure that is put in place to aid and support the reduction of defects in the system. It will be interesting to briefly consider the levels of testing that take place in the production of enterprise software today, because as it turns out, not everybody does it the same, nor with the same amount of success or commitment. Furthermore, the role of testing in the lifecycle has different meanings to developers as it does to testers.

In today’s modern landscape of million dollar software projects constructed by the careful orchestration of legions of business analysts, architects, programmers, testers and many more, why is there still a crisis in software quality? Again, the answer is not simple. What is interesting is that there are successful projects that do well by many metrics they are gauged against. And there are those that don’t do so well…

In my 13 years of commercial experience I have witnessed more differences in the testing processes than I have seen in the way we build the software that is to be tested. What seems true from my own window onto the world is that large companies typically have bigger budgets that allow the housing of big test teams that have their own departments purely for testing. Equally true is that similar large corporations have little or no testing infrastructure, even though the money spent on in-house software development is as comparable to the other more test-conscious companies. Why is that? I have been in small, focused companies harnessing software for their core business strategy, that have budgets for maintaining software quality that is greater than the million-dollar corporations. I have been in software houses that have rigorous processes to enforce quality, and I have been in those that have little or no process, flying by the seat of their pants. There is a large improvement in the way software is built by much of the industry today, but there is still a major sector of the business world that engages in building its own software, constructing it using in-house development teams, even though software is not their core business. Some do well with this approach yet many do badly, and it often comes down to poor skill sets across the team, ineffective custom process, and inappropriate testing infrastructures

Building software is complex and it is costly. Big business is “fortunate” because it can afford to bring in external software consultancies to aid the construction process, presumably on the supposition that the expense guarantees a level of skill and experience that will map appropriately to quality and success for the project. So far in my experience this hasn’t always been true. I have had the fortune of working with a number of external consultancies with varying degrees of calibre, and to my horror I have seen software produced by these consultancies submitted to the client with absolutely no testing performed. Often the question has also arisen, “Where are the unit tests?”, only to hear from the emptiness around, “What unit tests?” So I am wrong, paying for external consultancies to write the expensive software doesn’t always map to quality or bottom-line cost savings. I feel confident in further asserting that in such cases, it really is better to bring software development in-house, for at least there is a chance of gaining control of the quality of the software.

Large testing infrastructures are a luxury, and in some shops the test phase is the one that is typically cut short, or not done at all. Testing is often seen as a necessary evil, a secondary citizen amongst the prestigious aristocracy of the architecture, design and construction society within a development team. But it is wrong to view testing in such negative light, and the psychology that is the bedrock behind this perspective is most probably in part the reason why programmers grimace at the thought of having to test their own code; oh, those bourgeois! Testing needs to rise up the social ladder of software construction, taking its rightful place within government where it can play a pivotal role in the success and quality of the software it watches over. And so, it seems there’s a revolution going on, and it’s changing the psychology of programmers the world over…

In software environments that suffer from a lack of skill or experience somewhere within its lifecycle, which don’t have the investment or ability to identify and manage this fact, will in the end pay the price on any non-trivial endeavours. The issues may come at different stages of the project, from an inability to elucidate the true user requirements and map them to a solution, through to poor architectures and less than ideal software construction practices. Teams unaware of the need to remove duplication, to increase encapsulation and reduce complexity will often build code that eventually turns into the big ball of mud many programmers will have experienced at some time in their career. Many things conspire against any initiative to build software, no matter how well intentioned its brainchild’s may be. Unfortunately part of the problem is in the minds of the programmers that build the software, insidiously playing havoc from the realms of their unconsciousness, as they go about their daily chores. The problem is simple, and can be summed up quickly – many programmers think testing is not part of their job description, and so they don’t do it, and hate it when they have to. This is probably why at different clients I have seen software submitted by both internal and external practitioners that has no accompanying tests of any description – its need remains outside the conscious awareness of a large part of the development community, and the result is that the quality of software that is produced is affected.

The Disciplined Software Practitioner, the Disciplined Practices

Of course, it is unfair to tarnish the whole of the development community with the same blithe to quality. However what is clear is that there does exist a dichotomy between software developers – those who see testing as part of their daily role, and those that do not. If we consider the waterfall model, testing comes after coding, and it is unfortunate such a process has been taken with such literal meaning. The reality is that there are development teams that see testing as the next stage once they hit their release date; their code is built, integrated, possibly smoke tested, and then the system is given to the tester(s). Worse still, and I have worked on teams that have done this, code known to have problems is passed over to the test team just so it looks like the developers hit their deadline, and that the code is ready to be released – I don’t feel proud! To some management, passing the code over the fence by the release date is acceptable, as much as it is to have the code returned shortly after with a myriad of problems. “Oh well team…at least you hit the release date! If it hadn’t been for those testers! …. ” – I have heard a team lead say this once in my career.

Even in the cases where deadlines are met with ease, the amount of testing performed by the developers varies greatly from company to company. In some cases no unit tests are written at all, whether it is an inexperienced junior programmer who can be excused, or a veteran consultant with a CV as long as his nose. There is also an interesting syndrome I have witnessed in my career, which I refer to as the “It works in my test harness” syndrome; a developer writes a mediocre test harness without consideration of the dependency problems that might occur when his code is deployed to a clean box, submits his code to the code repository, only to realise later his code fails, lamenting in his defence, “It works in my test harness.” Also worth noting here is that it is incorrect to assume that a programmer can write sufficient tests, and that the skills of a tester are a mere subset of those of a developer.

These may seem like harsh words, and unfortunately they are a reality experienced by the author, both in the UK and America. However all is not lost. Just as Parnas brought to the consciousness principles and best practices many disciplined practitioners had already been using, today there is a revolution taking place that is as important as information hiding, brought into awareness by well disciplined practitioners of today – it is programmer testing. For many programmers, writing tests as a major step in the production of their code has been a disciplined ritual they have practiced since they first learnt to program. For many more programmers, writing tests has been deemed unnecessary or annoying. So now, the tides are changing, and we have the agile community to thank for this.

Many craftsmen have always done rigorous testing just as Parnas’ peers had always been using information hiding principles, and what is now happening is that these craftsmen have shown that it is a fundamental responsibility of the programmer to consider “testing” (therefore Quality) as an integral part of his whole as a developer, that it is a matter of professionalism, and nothing less.

The Revolution: Programmer Testing

And so we arrive at the threshold of the revolution. As each moment blends into the next, programmers around the globe are being assimilated into the dogma of a new breed of software development. An invisible wave is flowing through the minds of the software development community, new and old, and slowly but surely the psychology of those who succumb to the tantalising vogue of programmer testing, is causing a beautiful ripple effect we would normally want to abate were it within our code. Better still, its inside our minds, and whether you believe in agile development and its principles or not, nay-sayers to programmer testing are socially slain as the revolution takes no prisoners; a final coup de grace?

Today it is cool to know what JUnit or NUnit is, and it is not cool if you don’t know what these tools are, regardless of whether you are an agile subscriber or not. It is an exciting time for us all, for more and more programmers are learning about programmer testing and the testing tools that are used to run such tests. You may be forgiven if you don’t know what these tools are, but your career rests on making sure you change this.

The xUnit framework was borne within the agile/XP and Smalltalk communities, with Kent Beck being the original author of SUnit, the programmer-testing framework for Smalltalk. Since the early days, SUnit has become the forefather of many different implementations for the multitude of programming languages there are in common use today. The xUnit framework provides a mechanism for the programmer to write tests using his normal programming language, and then run the tests in a simple GUI to check they all pass and the code works as expected; a stupidly simple concept that has far reaching implications to the quality of the code that can be produced. And a sacred gift that comes from the programmers writing a suite of tests as part of the coding practice is that they can now make changes to their code for reasons of improving its quality! – read this sentence again. It is so fast and easy to re-run their tests in the xUnit GUI, with each change made to the code-face can be verified to have not broken any functionality across the system simply by re-running all the tests. Simple. Profound.

As a programmer, it doesn’t matter if you do agile, and it doesn’t matter if you are even practising test-driven development with your xUnit test tool – at this stage all that matters is that you are diligent in writing your tests to accompany your code, for this in itself will help the quality of your software to reach new heights. Agilers go one step further than just writing tests and they have created a whole method known as test driven development (TDD), which is a set of principles and patterns that guide the developer in writing their programmer tests before they write a single line of production code. Furthermore, TDD is less about writing tests and more about specifying the expected behaviour of the production code that is being constructed. The tests are specifications, and the specifications act as a form of executable documentation, demonstrating whether the system is behaving as expected simply by running the specification through an xUnit derivatives GUI. A subtle yet powerful distraction that states that programmer tests are much more than tests, may well be the oil that lubricates one of the major the cogs that drive the agile machine of today.

Having a suite of test specifications to verify the system is behaving as expected opens up the software construction process to a new era. Code sculpting, which has at its heart the practice of refactoring, is yet another technique offered by the agile/XP community. Refactoring in its original form is an evolving set of commonly referred to structural code changes that are used to increase the quality-envelope of object-oriented code. As a fundamental part of the TDD process, code is shaped through a refactoring step, always in the name of improving the quality of the code. There is even a mantra that is whispered by the lips of those who TDD, “Red, Green, Refactor.” That is, write a test, see the GUI go red to indicate that the test fails, then write the actual production code the test is asserting against, and watch the GUI go green as the test passes. After a green bar has been achieved, the next step in the TDD process is to safely refactor the code, confident the GUI will turn red if anything goes wrong during this micro-stage. Refactoring is the step in software construction that the industry has always wanted to do, but often felt it was too unsafe to do so, certainly in situations that saw the code-base begin to turn into a big ball of mud. Managements misunderstanding of this crucial step in the process, paradoxically risks the very software qualia they rest their promotions on. The fact is that refactoring in some fashion is something programmers have always done (to varying degrees), and it is a step in the software process that is so fundamental to maintaining a high level of software quality, that management needs to take heed and get educated.

The reality today is that best practices and principles used for a long time by many, are now becoming part of the collective unconscious of the programmer community, and we do have the agile community to thank for such exposure. Echoing down the corridors of the corporate world today are the words of a new vocabulary, new verbs and nouns that are the manifestations of the changes in mindset the programmer world is experiencing as this revolution travels to the ends of the globe.

Theres a revolution going on. Bring on the revolution.

References:
1. http://www.blinkenlights.com/pc.shtml
2. David L Parnas, On The Criteria Used In Decomposing Systems Into Modules

Just as relationships are productive when there challenges, the same is true for any team getting acquainted with a new process. That which does not kill us makes us stronger. To say that the adoption of a new process should be a bed of roses would be naïve, so while the past three iterations have been challenging for all, the positive results of this experience is palatable to all. I am looking forward to the new dawn; one in which we will all begin to feel agility, rather than just do it.

I started off thinking it wouldn’t be easy, and today I still have a healthy respect for the difficulty of snowboarding and the dangers that come with it. My progress in the end involved making small steps, gaining experience with the manoeuvres, and slowly but surely progressing further up the mountain. The new skill I have today is that I can do pretty well down the mountains on a decent board, but I think my progress was stumped by trying to go to far at the beginning. Once fear set in after injury, my progress was much slower, to the point of cessation, while I slowly regained my confidence. Fortunately, not all new skills development are so physically painful.

With agile development, the practitioner must garner a lot of new skills, especially since it involves getting involved with the whole process stack, from requirements down to implementation. Teams which have rigorous traditionalistic team structures, where the programmers may never even see the real customer, will require a different approach to a team that already engages actively with the business and users. Some of these new skills will be in learning how to capture the customers requirements so that they are suitable enough to drive the agile process. Other skills will be in learning how to think less horizontally about technical solutions, focusing more on the immediate needs of the customer and iteration.

To start working with the principles without first building a foundation of experience, will be a lot like starting to snowboard by taking your first ride down an advanced black run. If you don’t learn about edging a snowboard, with practice you can still learn to turn your board reasonably well – I hadn’t realised I side-stepped edging until my second holiday, when the instructor couldn’t work out why I kept falling over so drastically, until he realised I was clipping my edge. I managed to get up the slopes and down again, but there was always a tumble at some point. I’d been naïve about what I needed to know, thinking that since I was turning in some fashion, then that was good enough. It was good enough in so much as I enjoyed my holiday, felt I had made progress, and got some return on my investment of time, money and effort. It wasn’t until the second holiday, when all I did was fall over and I didn’t know why, did I realise I needed some help; I’d gotten ahead of myself.

While agile development is not the same as snowboarding, it does have a lot of similarity. For example, it is very easy to pick up the snowboard of agile principles and start skiing down the mountain, as it were. But without the basic foundations in place, the inevitable will happen – either someone gets hurt, or the principles are deemed at fault when we take a tumble, just like throwing the snowboard down on the ground and shouting “its no good, I need a new one!” Of course, if the expectations are loose and uncommitted, then the benefits gained from just getting on the slopes may actually be enough. At the same time, to safely get up the slopes and start gliding down the advanced red or black runs, some basic effort is needed in learning the basics, understanding the theory, and taking small steps.

Consider test driven development, which is a technique that one can start using within minutes. Download Nunit, install it, create a fixture and off you go. This can be a great way to immerse oneself in the practice, and the benefits gained form this process of learning may well be good enough – if your defect count decreases due to better test coverage, then that might be all you seek. With a sincere respect, this to me would belike buying the best snowboarding equipment money could buy, travelling to an expensive skiing resort, and spending your whole time on the nursery slopes.

Test driven development, or TDD, is a process in and of itself, that at one level is a simplistic way of writing a test, but in another way, is a vehicle for creating smaller, simpler code that is aligned with other principles of agility. A naïve appreciation of TDD would completely overlook this point, and would result in software that is non aligned with the ethos of simple design. Learning the foundations involves pushing yourself further and further up the mountain, in small steps. And, as you gain experience you will learn to let go of the hand-rail of BDUF design, and slowly but surely you’ll begin to feel the elegant skill of edging, turning and gliding with emergent design.

Perseverance and a healthy non-egoistic approach to learning something new will open up a greater world of potential. If your interest in agility is a hobby and nothing more, then your level of practise and approach needs to be chosen appropriately. With that said, if you wish to become a solid agile practitioner working as a functioning and supportive team member, then your approach will need to be more thorough. And with practise, things will come together for you, and your software will be a true testament to your success, with smaller, cleaner designs that flex and bend in all the right places.

When I went on my second snowboarding holiday, I naively believed I knew enough to start on the intermediate blue runs, since I had managed to turn and stay stood up reasonable well on my first holiday. But on the first day of my second holiday, I made a fundamental error going down a blue run, and headed off down a turnpike towards a more advanced red run – I panicked and bailed out. Having picked up so much speed, my wipe-out was disastrous, and I nearly snapped the lower part of my back as I snowballed down the slope. Needless to say I spent most of the holiday high on ibuprofen, nursing a sore ego. What I hadn’t realised was that once you pick up some serious speed on a steep slope, there are other forces and rules in play that I needed to understand so I could control my board safely, and wipe-out without killing myself. For two days the catastrophe was blamed on a defective snowboard, until the lights came on and I realised my own ignorance; I still had a lot to learn. When I got an instructor and got back on the slopes, I quickly understood that I lacked knowledge in the fundamentals of turning and edging the board. My instructor enlightened me on the theory and practise, and I suddenly realised the errors I had been making that had cost me so dearly. I learned a lot that holiday.

So, there is indeed a learning process required with the principles of agility, so don’t believe anyone who tells you otherwise. Only get on the advanced double-black diamond runs if you don’t care about the consequences.

Is this good?

While some concepts in our universe cannot be explained, only observed, in our discipline of software engineering the phenomena we face are not so esoteric, and are often explainable and observable. We are fortunate in that we can develop the right level of understanding to grow with new concepts as we discover them; the greater challenge for many is in the commitment to learning in a busy world. Yet, exposure to new models of thought, new paradigms of practice, ultimately requires deeper knowledge which has to be attained through experience, intellectual inquiry and time. Of course, the first step is to realise that a new concept or idea is different to that which we compare it against, and to appreciate that we cannot successfully project existing mental models onto the new idea and hope to completely understand; at some point we need to allow the formation of new models, which as a process could potentially shatter previous belief systems and whole ways of thinking – these shifts may be too great for some, yet for others, be they individuals or organisations, it can mean progress.

Understanding the Chasm

One can be forgiven for thinking that choosing an approach such as agile development might be as simple as learning a few new simple rules and practices. In some respects, at the superficial level, this may be true; but it’s not so simple. Any organisation that makes the decision to learn agile development, is not just stating the forgoing of traditionalistic values, principles and practices, but is committing to the endeavour of education and discovery of new paradigms and mental models that jeopardise the very survival of its present identity. In some respects, at a deeper level, this is exactly what happens to those that cross the chasm.

A step onto the road of agility can mean the want to make progress, to seek the pastures of greater success where the software endeavour itself has been civilised and socialised into the organisations culture and way of being. To an establishment rooted in hierarchy, deterministic projections and control and command management cultures, the land of agility can often seem like a nice sunny place one might visit for a short holiday, but would not wish to spend much time there. “Those people over there…they’re not like us”. Through diplomacy and noble struggles the two can conjoin on a foundation of understanding, on the bedrock of recognition that differences abound, the cause is common; to achieve success with software development, to discover the key behind the software code. This is Right Understanding.

To my mind, it is important to remember that there will be great times to be experienced by many new teams that discover agile development. And of course, there will be times when difficulty arises, when the team is faced with a problem that rocks deep to the heart of their being, when fear itself manifests and challenges them to step back, to return to what it knows, to old habits that are predictable and safe.

Agility means going slower than we used to, rattling off feature after feature like a helicopter machine gun, ripping through the heart and soul of the victim that is the very code our project and life depends on; thunk, thunk, thunk…in go the features. Stand back – do we have a flatliner? The focus is quality, not quantity. Agility means stepping into a world of uncertainty, a world where a plan starts to fade as soon as the ink reaches the parchment, a place where both chaos and order are allowed to exist as respected adults. Command and control is substituted for a healthy respect for the people, no longer are they seen as cogs in the machine of corporate strategy. The people are left to self-organize, to act and react to the forever existing impetus of uncertainty and change; adapting, reacting, embracing. Through shared vision, shared goals and passions, customer, developer, manager, all come together in a collaborative, respectful honour of the reality that software construction is hard, but together through united understanding, a difference can be made; but there must be change.

Commitment 

To have longevity with agile a commitment is needed to develop enough understanding to know what to do and how to behave when the journey brings adversity.  Agility is not just about programming, its about creating a foundation of values and ethics from which to interact with our customers, our employees and other teams.  Its about understanding that while there is uncertainty, this breeds innovation from within (if cultivated), further allowing that organisation to act, adapt and ultimately create new differentiation in a dynamic business environment.   Courage and strength come from a commitment to developing the right knowledge and theory behind the principles (surely its an obligation?), so that the team can be supported by their leaders in the face of adversity.  Agililty is a new paradigm, and it does require new understandings – right understandings.

I attended a New Parents evening last week over here in Woodley, in which new parents got to meet each other and share experiences. During the evening we were asked to pair up in fours and write down on a piece of paper what we thought a baby/child needs. A simple statement which resulted in the typical responses of love, happiness, warmth, comfort, joy. After a few minutes we all regrouped and shared what we’d written. Once the sharing was over, the facilitator offered us a word we’d forgotten – Respect. I couldnt believe I hadnt thought of it. The facilitator explained that just because we are parents and we have to instruct and guide our children, it doesnt mean we dont give them respect – they are people too.

It goes without saying, respect in the workplace is paramount, but sometimes it can be easy to forget the simple things, and respect is one of them. Relationships are based on trust and respect, and software development is all about lurve…only kidding, its all about feeling respected and trusted wihtin the relationships we engage in during 9-5. When theres a healthy level of respect, great things can be achieved. Naturally, the inverse can happen and a lack of respect can breakdown relationships and communication, and agiles software is built from communication.

I will try to remember the simple things in life are often the wisest.

Keeping a design simple requires that we first appreciate what we mean by non-simple, or complex. And one mans complex is another mans simple, for the map is not the territory. So often I hear that adding new classes or new interfaces is to be avoided, but these are two of the very fundamental carving tools we have to work with as carpenters of the digital oak. As methods bloat and classes expand in size and responsibility, we refactor, we carve, we constantly shape and mould our software design; vanguards in the pursuit of quality.

From principles to values, from patterns to rules, new techniques abound, but timeless are the ever relevant mountains of quality, the object oriented software construction practices. Simple design is about managing complexity, but it doesn’t mean as practitioners we must throw down our skills. To truly be agile, the healthiest of all of our parts has to be the code itself – if the code is in poor shape, it can quickly turn into an anvil that will quickly sink your team. Simple design doesn’t say forget cohesion and coupling, it suggests being mindful of these things with our design, but not to overreact too soon. Neither is simple design about accepting duplication, bad names, bad class responsibility allocation, bad abstraction or classification, or even just bad assumptions. It is simply about making sure the design does exactly what it needs to do today, and nothing more, while being mindfully built from the very best engineering practices we know. Indeed to accept anything less is to misunderstand the essence of being an agile practitioner – to be relentless in our pursuit to seek out a rotting design, using only the very best in experience and best practices for building software with object technology. To ignore this on an XP project, will be the Achilles heel that will bring it down.

Great post. I read somewhere that a company had 99 lever arch files full of documentation so that they could be assessed and accredited by an external CMM auditor. CMMI is an acceptance by SEI that maybe its a bit too heavy weight for practical use; though many have moved up the levels of CMM with success. But agile is a bit more than iterative, as iterative is still around today in the forms of RUP and other BDUF methodologies that accept the canonical waterfall dictum is not practical. Iterative to me is a step towards realising that maybe the problems we are trying to solve are too big for us, and that embracing divide and conquer might be more wise; a positive realisation. Agile however embraces something different (to me), and that is Rittel’s notion of the wicked problem, that software is not only very complex, but also that by trying to solve some of the problem, we can learn more about what it is we are trying to solve in the first place – this new knowledge being feedback into understanding again what the problem really is. Of course, there is much more to the agile movement than just problem definition, but its still no use building for the wrong problem.

The more I work using agile practices, the more I develop an appreciation of the less agile/non-agile processes. People give BDUF a real bad-wrap, and I have to say that when I hear it, it makes me embarrassed to be in the sector. It wasn’t too long ago I used to hear that people dissed project management, that it was the root cause of failing projects, and that that this is why XP gives a royal finger to the PM world (not that I agree). Now, XP is definitely seen as being better complemented with Scrum, a project management framework; time to kiss and make up. RUP on the surface is as daunting and overwhelming as anyone looking at the ISO or CMM/I directions, and this is one reason for its bad press. The sad thing is is that most of the rhetoric is wrong – RUP is meant to be appropriātus, it is meant to be shaped and customized to meet the needs of the organization. Because of this fact, RUP isnt half as heavyweight as is believed by the misunderstood methodology-bourgeoisie.

There are also examples coming out that show that “agile” in the form of XP and Scrum are delivering on their promise, thats for sure (ACM have some good articles to this effect). At the same time there are also clear examples that unless there are Level 2 or Level 3 Cockburn style practitioners predominantly in an XP style team, then the rhetoric is fluff and nothing else, as these projects spin their wheels refactoring, rearchitecting and simply getting stuck in the mud of advanced engineering practices they have little or no skill in. Interestingly a project I worked on not too long ago feels today like such a project, and the introduction of agile practices will have offered some respite to the wheel spinning for sure, but I have doubts whether the team will achieve the levels of success attainable by other XP teams – simply because of the people on the team. In this case, some level of direction and rigour in applying less agile, more plan driven practices might have offered a better level of risk mitigation than just hoping quick deliveries will suffice – that is ofcourse if quick deliveries could ever be achieved.

Yegge’s comments are valid, but they are also naive in my view, for their lack of appreciation of the deeper truths behind the practice. Agile, in the sense of less heavyweight processes, where the team are constantly planning throughout the process to adapt to new knowledge, this type of agile is as much a candidate of choice as is any less agile option, but its based on context. I get miffed because Agile seems to be associated with things like XP, but I know of companies doing “Agile” who just do lightweight planning, use NUnit/JUnit and do TDD. I have a friend who is working for a company that is practicing Agile, and he sent me some code to review that had been crafted using test driven development. The code wasn’t good. Anyone can build crap code, but the very fibres that weave the elemental parts of eXtreme Programming into the methodology that it is are based in solid, quality driven software engineering practices, and this is one of the reasons for its ability to deliver. Not getting bogged down in project management can be more agile than a tanker PM process that cannot deviate quickly from its trajectory, if at all, but this type of agile is aeons away from XP. Granted, at the same time all are attempts at improvement in some form or another, yet I feel the industry knows pretty much most of the people issues and the planning issues. What the industry needs to do is push the envelope in the skillsets of the practitioners that extol the discipline, because I personally feel this is a place where the greatest travesties can occur.

As a closing anecdote, I started working with a team a few years ago on a web application. I started pairing with a guy who was ten years old than me, has been in the industry since I was a thought in my dads head, and apparently had been technical in every part of the IT practioner stack, from programmer to enterprise architect. Our first problem was to fire some data down to an instrument – the user pressed a button to say “send”, and this would trickle down and our Instrument Layer would be invoked to deal with generating the necessary data for the instrument. Within only a few moments my partner suggested that we needed to know some extra details, such as who is the user firing the send request, that type of thing. So I asked if he had any suggestions, and he told me he did – we should pop up a dialog to ask for the details. I first mentioned that we were building a low-level layer, and asked if he felt working with the UI from this level was appropriate – he didnt have a problem. I then asked him about the more pressing worry that we were building a web application, and popping up the MessageDlg from down here, if we could do that, would probably not be what we wanted to do anyway. Long story short, I failed to convince my partner, and he merrily went ahead building the pieces he felt were needed. Eventually he realised his error, but the deeper issue for me was “oh no!”, this guy has got over 20 years experience – he’s meant to be one of the good guys. There are indeed good guys out there, but you gotta look hard enough.

Nick Robinson
Lab49, Inc
http://www.lab49.com
http://blog.lab49.com

From: Bruce Fancher
Sent: Sat 21/10/2006 9:13 PM
To: Nick Robinson; Luke Flemmer; Ashley Whitney; tech
Subject: RE: “Good Agile, Bad Agile”
First of all, what the hell does Agile even mean at this point anyway? The original manifesto was pretty vague and the dude who wrote that blog post [yegge] is right that the term has become another buzzword for the usual suspects to pile onto. This is unfortunate because it obscures the original point of the so-called “Agile” methodologies (or what used to be called “Iterative Development” until they found out that no one could pronounce “Iterative”). And the result is that when guys like Steve whatsisname decides to rant about whatever it is he’s frustrated about at the moment (I couldn’t read far enough into the post to completely figure out that part) he blames it on “Agile.”

Software has always been difficult to write, and hard to deliver on time (or sometimes at all). For a long time the industry tried to address this by creating more and more elaborate systems and methodologies for upfront analysis, design, planning, scheduling, testing, measurement, etc. The thinking seemed to always be, well the last project was late because the customer didn’t tell us they needed feature X, and we didn’t think of technical problem Y, or anticipate bug Z. So, this time we’ll specify everything out in advance in even more detail so that our estimates will be more precise and the product will be finished on schedule. Oh, and while we’re at it, we’ll have one or a few architects at the top, who will tell the rest of the developers who everything should be designed (possibly with a few more layers of management in between).

And the amazing thing is, after all these years, and none of that stuff ever working, they’re *still* at it. Remember the CMM? I just checked and the Software Engineering Institute, which is apparently the North Korea of heavyweight software methodologies, has now replaced the CMM with the CMMI. Just one of the documents on their site is 573-pages of nonsense about how define your own ridiculously detailed set of policies and procedures. Here’s a random excerpt, taken from the “Process and Product Quality Assurance (PPQA)” section of the CMMI Product Team Technical Report (CPTTR):

Provide Objective Insight
Noncompliance issues are objectively tracked and communicated, and resolution is ensured.
SP 2.1 Communicate and Ensure Resolution of Noncompliance Issues
Communicate quality issues and ensure resolution of noncompliance issues with the staff and managers.
Noncompliance issues are problems identified in evaluations that reflect a lack of adherence to applicable standards, process descriptions, or procedures. The status of noncompliance issues provides an indication of quality trends. Quality issues include noncompliance issues and results of trend analysis.
When local resolution of noncompliance issues cannot be obtained, use established escalation mechanisms to ensure that the appropriate level of management can resolve the issue. Track noncompliance issues to resolution.
Typical Work Products
1. Corrective action reports
2. Evaluation reports
3. Quality trends
What the %*@*(#$ does that have to do with actually building working software?
Not surprisingly it turns out that trying to plan anything as complex as computer software, with instruments as limited as human brains, doesn’t work at a technical company any better than it did in the Soviet Union. If you check out a few books and articles by economists like Fredriech Hayek, Thomas Sowell, etc. you’ll find that just about everything they say about the limits of central planning, the importance of trial and error, having a feedback mechanisms, etc. is just as relevant to waterfall methodologies as to the socialist planning they were writing about. Planning anything too complex ahead of time doesn’t work, simply because we don’t know enough to do so, and usually the only way to accumulate the knowledge we need to do such planning is to actually attempt to build whatever it is we’re trying to build and refine it as we go along.
To me, that’s the main point of “Agile.” Simply acknowledging that you’re going to get a much better result if you distribute responsibility and authority to those closer to the problem (e.g. the folks writing the code, rather than a remote architect imposing his vision of how things should be), and build a feedback mechanism into your process. Accepting this insight doesn’t imply that Scrum, XP, or any other process or methodology is perfect. In fact, it almost means that it isn’t, since like everything else it should be subject to improvement and refinement based on ongoing experience. And whether or not the C3 project was a success or not doesn’t really have any bearing either. The question isn’t whether “Agile” (again, whatever the hell that means) is perfect, but how it compares to the alternatives. Given the absolutely appalling track record of the software industry, it’d be hard to do any worse.
Bruce

It was clear one of the conversation partners had a little exposure to agile concepts, but the other had a lot more. Whenever I hear someone talking about interesting computing topics I always want to join in, but I’m not so good with strangers – must be due to all of those Stranger Danger Police presentations at primary school that I still cant shake. Content to join in the conversation silently from my table, I continued to listen and make my own comments and suggestions in my head.

Agile development has come a long way, and so has the meaning of the name. For this reason I am sometimes sentimentally saddened when I meet a new team that claims to be “agile”, when in reality they have little or process at all left from their stripping-down of their chosen flavour; becoming agile isn’t about deciding to move from a heavyweight, documentation-burdened process to simply no process at all. So I was a little dismayed as I continued to listen to the conversation, because it seemed the only practice the more experienced agile engineer had was test driven development (TDD). The profusion of terms he uttered placed him as clearly well read in the practice of TDD, this could not be denied. When little else from agile development came forth, I reasoned that maybe while the discussion had started off clearly using the term as the basis to their confab, the focus had simply moved to just TDD.

Nevertheless I zoned out. So I’m back to writing on my computer drinking my caffe misto when I catch the words “…well the reality is we don’t need to do analysis any more..pick a feature and go straight to TDD’ing – TDD is a really just a unit testing and design technique….” I nearly spat my coffee out all over my Dell. At this point I felt compelled to ask if I could join in on the conversation, but in the end I chose not to…

It wasn’t long before I packed up and left, and I tried to give little time to what I’d overheard. Even though what I heard had errors and was a fluffy showcase of agile development at best, I put it down to a normal fact of life, that when people are learning new things they will often get things wrong; its only natural. How many times have you given counsel to someone who has had a strong conviction in what they are talking about, but from what they say they have clearly misunderstood an important detail that throws their argument into a different perspective? For this reason I didn’t give the conversation much more thought.

Part of the journey to knowing, is in not knowing and using learning to develop knowledge to know. Sometimes people will think they know the essence of what they are learning, but they don’t, and they don’t know that either. They’ll have bright ideas and create new intellectual associations from the old place of not knowing to the new place of being knowledgeable. Then someone will challenge their propositions, firing torpedoes of reason that violate the deductions behind their new found knowledge. These invalidations cause the usurped to search and learn more, to question and reason more deeply, and as they do, they learn more and their journey towards knowing continues. Maybe after sometime on this path they’ll look back and wonder how much they now know, and how much they really didn’t know when they thought they knew so much! Humility helps. But not knowing is neither right nor wrong, it just is – it’s the human way. An interesting lack of humility can be seen here : http://steve-yegge.blogspot.com/2006/09/good-agile-bad-agile_27.html, along with an unknowing of how much he really doesn’t know about that which he talks – poetic irony? Probably just Googler arrogance. But one journeyman shouldn’t treat his fellow journeymen with contempt – its neither respectful nor professional.

The fact is I have heard developers new to agile principles express beliefs about analysis and similar practices that are fundamentally wrong – and I’d be surprised if I didn’t. At the same time, this phenomena of not knowing is the natural step to learning the truth, and it would be foolish to claim this is unique to computer science or software engineering – it’s the process of knowledge. Of course when an engineer claims that in agile development analysis is no longer practiced, it is for those further along the learning curve to help correct the misunderstandings, yet at the same time accept them as a natural aspect of learning; that’s what mentors and coaches are for, in a formal capacity or not.

To the claim that analysis is not performed on agile projects, analysis is indeed performed for sure – do we really think the agile protagonists are so arrogant as to discard the years of research that has gone into software engineering? As the discipline has matured it has become ever clearer to the industry that analysis is a very important aspect to building the right software for our customer. Omission of analysis on any non-trivial problem should therefore only be practiced if you wish your software development to fail.

With reference to the earlier blog reference, ridicule or excursions into the inflammatory will not help the Starbuck agilers nor anybody else or the pioneering movement they are a member of that is in the process of learning and evolution. Agile development is nothing more than a manifestation of evolution in the field of software engineering, just as was structured analysis, functional decomposition, object oriented analysis and design or the many methodologies and approaches that have arisen over the years in the pursuit to build better software. All are attempts at improving the discipline, but none are the silver bullet; the bullet glides beyond the frontiers of today’s pioneering endeavors to create a greater practice.

Rather than rebuke, maybe its better to think about why there are misunderstandings about analysis? The once software zeitgeist was Waterfall development, which broke down the process into unique and distinct phases. Simply, the established project involved a phase of requirements gathering, followed by analysis, then design which preceded coding and implementation. If you were one of the lucky ones, you also had a phase for testing. So it wasn’t too long ago that job advertisements requested systems analysts and systems designers, since the industry at that time felt that the roles were distinct enough to require specialists, that maybe a programmer lacks the skills of analysis or design. These days most roles are described as developers, object oriented developers, technical leads or architects. Irrespective of process, any embryonic career in an undisciplined software environment could lead the less experienced to think even less of analysis, if indeed they thought about it at all.

Of course the paradox is that the decline in emphasis on the discrete phases and roles is not because they are seen as offering little value, but because it is realised each phase is an interleaved continuum with the other phases in the process of building software.

Analysis is the process of understanding the problem domain, and it is impossible to build non-trivial software without it. At the same time, due to changing requirements, rapid change in business intentions, analysis rarely ceases throughout the lifecycle. Hand in hand with a requirement change is the need to understand the problem that drives the change, and this requires analysis. This has nothing to do with agile development, this is just reality – you need to understand the soil that seeds new changes if you are to fruitfully grow a solution to meet its needs.

So the real irony is that agile development teams will specify initial requirements in the form of user stories, and since they are no more than a few sentences at most, only the misguided would assume they could implement the feature from a few informal words. User stories are reminders for conversations, and those conversations are steps in allowing the engineers to understand the need behind the story. And with a level of expertise, these conversations unravel the realities of the problem domain as part of the natural inquiry into the needs of the customer and her stories.

Each story is broken down into tasks, and this is normally through the expression of intentions of the customer for her story, followed by a torrent of questions from the engineers to unravel the real requirement. From this inquiry, this analysis, comes understanding of the problem domain, and ultimately what is required to build the solution domain – the boundary between analysis and design is present, if not difficult to pinpoint; it’s a continuum. At the same time during the search to understand a feature it would not be uncommon for the engineers to build models, static or dynamic representations of either the problem domain or their evolving ideas on how the solution might look. All of this is intertwined, and continues throughout the lifecycle as feedback and knowledge enters the process and spurs new insights. Designing and modelling with code is also both a process of understanding the solution domain, but as it progresses it will uncover shortcomings in knowledge about the problem domain, and the delicate dance of analysis, design and coding play their part in crafting an evolving solution. A developer is never far from analysis, design, or implementation, as they are the fibres that weave together moment to moment of the modern engineer.

Any statement that analysis is not performed in an agile process, whether it is extreme programming or not, is simply wrong. The only true basis to such a claim is not stupidity but lack of experience, and I wish those people luck in their endeavours to uncover the deeper truths of the practice, as I know I once walked the same path as they, and continue to discover how much I too must learn new concepts and disciplines as I mature as an engineer. To me just as Lab49 is built on people, intellect and experience, Agile Development is built on the deeper truths behind the practice of software engineering, implied or otherwise, and it is therefore no surprise that the agile manifesto contains the maxim Individuals and interactions over processes and Tools.

After the phone call Dez walks over to the coffee vending machine, contemplating his bleeding ear and increasing workload, wondering how he’s going to make it to his daughters first year birthday party that night. Sitting back at his desk, Dez downs his coffee in one gulp, feels the burning fluid singe the first layer of his throat away as he punches into the telephone his wifes number. “Princess, I wont be able to make it tonight”

Dez has been blamed for the performance problem in his user interface, even though he uses components built by other team members. Not only has Dez got to complete his “day work”, but he must also fix this “bug” because the boss doesn’t want the bug list to go up – all of this must not knock the release schedule out of line, so it looks like even more longer hours for Dez.

This is hypothetical of course, but is it unreal? Not at all, even in todays software development climate.. The reality is that using an agile process such as eXtreme Programming or similar, this scenario wouldn’t happen. The fact is that without the most rigorous and formal definition and creation of software, the best we can do is accept bugs will come along, and embrace that fact rather than creating an environment that frowns at the thought of bugs at all. Bugs will come along, and the customer has the choice to make a decision of more features over removing bugs. If a bug seems longer than a quick fix, a story is created for it, and the customer gets to choose between the stories remaining that he feels will give him the highest business value, over other stories which might be recorded defects. Nobodies balls are broken, families aren’t destroyed.

With high-grade engineering practices we can do our best to craft code with minimal defects, and eXtreme Programming focuses greatly on achieving this objective. On most up-and-running eXtreme teams, due to acceptance testing and a good suite of programmer tests, defects will be minimal, but they will exist. The customer is involved with dealing with any non-trivial problems and engages with the team in a positive manner to work together to solve the issues. Creating stories out of these types of eventualities simply integrates them into the fluid and ever changing requirements plan. Simple. Not hard.

Acceptance is a wonderful life tool.

I was recently reminded of a team who engaged with XP style development, but sadly struggled with its fundamentals such as TDD, Refactoring, Collective Code Ownership and the dreaded Pair Programming. I sat down with this team more than once, and I often felt that they were too hard on themselves. I now think that while they could have relaxed a little, what was truly overlooked was the fact that there will be pain. Had they accepted this at the outset, I feel the outcome may have been different. As it turns out, they no longer do “XP” though they do “Agile”. I recently found out that they no longer push TDD, its optional – if you like it, good and if you dont, DONT DO IT!

To me this is unfortunate. Learning anything new takes time, and something overlooked with techniques such as test driven development, is that its a paradigm shift. In software, such shifts come with some pain. The reality is that making the quality of the practice a core objective, results in the quality of the practice becoming a reality. Focusing on learning first principles and accepting things might not work out quite right is a trait I use a lot in furthering my personal endeavours. Courage is a useful value for a practitioner of XP, and having the courage to screw things up actually cultivates more opportunity for growth. I only wish this team had seen this, for they started off with great reasons for wanting to develop with XP – because they valued the quality of the work they produce and deliver to their customer.

“If you want to garden, you have to bend down and touch the soil. Gardening is a practice, not an idea.” — Thich Nhat Hanh

Software development is no different.

High-grade software cannot be built just by “doing agile”, or any other methodology. Just because a developer knows the complete list of Martin Fowlers Refactorings, up/down/left/right and inside out, doesnt make them a high-grade developer either. Nor does knowing how to write a TDD test, or dare I say it, a BDD spec, make a developer top-notch. Theres a lot more that goes into the pot of experience that puts a mindful practitioner on the road to becoming high-grade. Theres also a lot more that goes into delivering high-grade software than refactoring, test-driven development or 2-week iterations and a 40 hour week. Prefactoring distills some of the principles that complement and support a development team, agile or not, to move towards being successful in their software endeavours. Theres a lot more to “doing agile” than the neo-jargon that proliferates the web, and if you are a developer serious about furthering your career, that pursuit will be achieved through ignoring the uninformed distractions that form the “information” on the internet, and remain steadfast in your pursuit for gaining experience in the profession, from hands-on development to the study and critiqueing of the work and ideas of other practitioners in the field. Prefactoring is one such book I would recommend to anyone who is serious about software.

On a well-oiled, well-seasoned agile development team, the above example is profoundly inverted. The team comes together in a way that is nothing but good. Should someone slip, they will be graciously caught, and together the pursuit of delivering top quality software continues. Machiavellian’s need not apply.

So what is it that supports the more positive team experience over the negative? I believe the answer is a complex of reasons, some conscious, and some unconscious. However one such reason I have witnessed, is the open acceptance that every member of the team is human and fallible. Quality of work is non-negotiable, but the team accepts that as time progresses, the dynamic forces of life will at some point affect every member of the team in its own unique way. The key for the team is in building a level of trust and support so that when things do slip, the response is helpful and positive, rather than negative and chastising.

Sometimes my family and I have to drive across the country to see relatives, and the journey can often be long and monotonous. When I driving and I’m tired, sometimes if its dark, the glaring lights, the drone of the wheels on the tarmac, the powerful desire to just get home so intense, I forget to check-in with myself to see if I am still good to drive. So I have my wife Sally sat beside me, equally tired, equally fatigued and yearning to be home in a cosy bed. But we have a job to do: my task is to drive us home safely, that’s what the family relies on me for. Sally’s role is the same – to make sure we get home safely. She keeps an eye on me to check that I am not falling asleep or nodding, that’s her priority. At the same time, when I get a burst energy, I glance over to Sally to check she’s still awake – she’s my wing-woman, I need to be able to rely on her. When I’ve had enough, we pull over, grab a coffee, and swap places so that Sally will drive and I will be her wing-man. Throughout the journey, we accept that tiredness will come – it will come to both of us, so we support each other, and that’s how we achieve our safe journey home.

To me this is one of the elements that exist in a well-functioning, well-disciplined development team. Each member has a role to play, and part of that role involves looking out for the wider team and its overall objectives. Every member must keep an eye on quality, and no matter what, if they even suspect there might be a transgression in the level of work being produced, they must raise their hand and speak up for the good of the team and the project. We all transgress, and we all want to get home to a cosy warm bed, safe and sound, but sometimes its so easy to let the eyes close shut. I rely on the open eyes of my team members, and they rely on me – this is what makes a team work well. While the team doesnt necessarily have to be doing agile to implement or benefit from this teamwork, I often see it exhibited more on successful, value-delivering agile teams than on non-agile teams.

The discussion was very interesting when diagnosing the produced IL, but what caught my attention more was the use of the new modifier – I admit that while it does have a purpose, I am cautious with its use. I have found the modifier not only causes confusion, but it has an even worse by-product – it destroys ones ability to reason about a program based on abstract types, which is one of the key elements to building solid object code.

Before I show a (contrived) example, it will be worth agreeing on a design invariant that is: Object oriented software should strive to be type-agnostic in all of the relevant places, favouring abstract rather than concrete dependencies where possible.

In my example there are two companies, CompanyFoo and CompanyBar. CompanyFoo is a tax specialist and has created an assembly with some tax calculation classes. CompanyBar uses the assembly from CompanyFoo to perform its own tax-related calculations as part of its business. Take a look below:

To keep things simple, there’s an abstract type called TaxStrategy which models a discrete Taxing decision in the application. Strategies are small, so a typical strategy here would check if the price falls between its programmed range, and if it does returns the tax, e.g.:

By default CompanyFoo provide a number of TaxStrategies, but for this example there’s only one – NormalTaxStrategy:

Don’t be too distracted behind the numbers because remember this is a contrived example. CompanyFoo, rightly or wrongly, has decided that strategies are the right way to design this part of the problem domain, and they also provide a class to capture the full tax-decision algorithm:

TaxCalculator is the type used by an application, and it is initialised within the using application during application-bootstrap. Notice it takes a list of strategies, which it uses to workout the tax for a priced item.

So far…so good? Company B enter the stage…

CompanyBar begins to use the tax assembly provided by CompanyFoo, and one of the programmers is tasked to create a new TaxStrategy. For whatever reason, the programmer thinks about the problem, and decides that he doesn’t believe his TaxStrategy is semantically the same as those provided by CompanyFoo. Aware that other applications use the CompanyFoo assembly, and that he cannot modify the tax assembly, he makes a decision (work with me) – he will create a new TaxStrategy (because it is a tax strategy) but he will use the new modifier on the Calculate method:

After writing TDD tests against the StaggeredTaxStrategy it seems everything is working ok, so the programmer integrates it into the build, and he can clock off early conscious he’s done a good days work…

But theres a problem. The design of the TaxCalculator class is pretty simple, and pretty clean. It is loaded with TaxStrategy instances, and when asked, it uses these objects to return a valid tax figure. Better still, TaxCalculator is type-agnostic – it depends on an interface (abstract class in this instance), and doesnt need to know of any new types. New types can be added without affecting this module at all. Where’s the problem? It seems the programmer still wants the StaggeredTaxStrategy to work within the system, and to all intents and purposes it honours the contract of TaxStrategy. This code does not work. If the StaggeredTaxStrategy is added to the TaxCalculator, the results that are expected will not materialize…

The program below demonstrates what happens; the console output is shown immediately after:

Console output:

Expected Tax for 5.00 = 0.00, Actual Tax = 0.00,
 Expected Tax Rate = 0.00%, Actual Tax Rate = 0.00%, Correct? = True

Expected Tax for 25.00 = 4.25, Actual Tax = 4.25,
 Expected Tax Rate = 17.00%, Actual Tax Rate = 17.00%, Correct? = True

Expected Tax for 101.00 = 16.67, Actual Tax = 17.17,
 Expected Tax Rate = 16.50%, Actual Tax Rate = 17.00%, Correct? = False

Expected Tax for 750.00 = 116.25, Actual Tax = 127.50,
 Expected Tax Rate = 15.50%, Actual Tax Rate = 17.00%, Correct? = False

In the last two price evaluations, the StaggeredTaxStrategy was used, but the Calculate method on StaggeredTaxStrategy is never called. The new modifier is partly to blame (the programmer is to also to blame) because it basically says to the compiler – “break the link between this method and its parent, and dont dispatch polymorphic invocations on this method in a base class to this class”. And what it is implicitly saying is, “I’m happy with this because I know that this class will be used directly rather than through a base class.” – because thats the only way to get access to the method. If there are subclasses created from the StaggeredTaxStrategy class, the Calculate method is seen as nothing other than a normal method that had never be virtual(!).

Obviously this is the incorrect use of the new operator, and clearly wasn’t the right approach to be taken here in the class design, and of course this was contrived. While one might use the new modifier to help with class versioning problems, it is still easy to see how in a large application code that has been built to depend on an abstract type(s) as in this case (TaxCalculator), the use of the new modifier can have disastrous affects.

I mention this because I have seen the new modifier used in such a way that it was impossible to reason about the code – the cyclomatic complexity was such that we had to create a code-magnet to slowly move code away from the miasma. What I took from this experience was that the new modifier certainly does have its uses, but its use should be sparing and considered. Polymorphic message dispatch and type-agnostic dependencies are the foundation to good object software, and the new modifier helps the wary to surgically interact with this execution process, while to the unwary, the modifier can be exceptionally useful in introducing bizarre and difficult bugs into a large and established system.

Many aspects of life are governed by the yearly orbit of the earth around the sun. The rotation of our planet gives us the 24 hour day, and it is known that plants open and close, lift and droop in rhythms of approximately 24 hours, even when they are not exposed to the sun. These findings suggest the rhythms for the plant are within the plant itself, and have nothing to do with sunlight per se. Apparently in 1985 researchers found that there is in fact a genetic element to plants that provide the source of the rhythms. However sunlight acts as a reset on the plants genetic clock, allowing it to speed up or slow down to match the changing seasons. Fascinating.

Humans are also affected by Mother Nature and our own genetic clock are influenced by the Earths seasonal rhythms. More importantly, our moods, well-being, stress-levels and mental capacity vary throughout the day in well-known predictable rhythms, known rhythms. However the rhythms which are important for improved mental capacity and well-being, especially for those who work hard and play hard, are whats known as Ultradian Rhythms. Around 16+ times a day our bodies experience the effects of these 90-120 minute rhythms followed by a 15-20 minute restoration period where the body naturally purges the stress from the body to cleanse the body and mind. This rhythm is so important it even has a scientific name for it – Basic Rest Activity Cycle

In simplistic terms what this means is that after every 90-120 minutes our body has a natural inner cleansing process that takes place and lasts for around 15-20 minutes. The effects of this cleansing process if it works well are an improved clarity of mind, emotional well-being, and a new sense of alertness. However, theres a catch. When the ultradian rhythm comes to its end and the 15-20 restoration period kicks in, if you are busy writing tomorrows report or hurriedly running across city and country for the next big meeting, your body has to focus on the main task at hand of aiding you in writing the report, or whatever it is you are doing. So the resoration window comes, and it goes. If however you recognise you are moving into one of these restoration windows at the end of a ultradian cycle, you can help your body by taking timeout and resting your eyes for about 15-20 minutes. The most effective restoration period is when you dont actually fall asleep but go into a light sleep, because as your natural rhythm comes out of the 15-20 minute window you’ll naturally open your eyes and feeel greeeat!

Theres lots more to Ultradian Rhythms and I strongly suggest anyone with a career reads “The 20-Minute Break by Professor E.L.Rossi” for a more thorough explanation.

If you’ve been doing a task for a reasonable amount of time and you suddenly find yourself forgetful, trying to remember what you were just abouts to do next, this could be your body moving into the 15-20 minute window (or it could be you’re forgetful). If you help your body out, you will be refreshed and restored and profoundly more productive than if you just keep on hammering at the grind…