Software development dogmata – good practices gone bad

Many practices in the software development world are widely accepted at some moment in time as the “true way”. The trouble starts when the “true way” is forced down the throats of people who need a “new way”. Usually popular practices follow a similar path between the moment somebody discovers/creates them and until they get massively popular.

The common path is this:

  1. Some programmer has to solve a problem. The problem is very specific and he tries to tackle it using the tools at hand and the methods all his peers used successfully until then.
  2. The existing tools and methods prove to be of no help. Even worse, they start to stand in the way.
  3. Our programmer is a smart cookie and imagines new ways that solve his particular problem.
  4. He uses it or at least he tries, but he collides with the established processes, methods or tools.
  5. All hell breaks lose and the “old ways” fight back. Management is not very fond of new risky ways.
  6. Since there is no other solution, the freshly found “new way” is masked under some “old” clothes and it is used.
  7. But the shiny gem cannot be fully covered by dirt. The success becomes known to other people who become interested. Our programmer has now supporters. People start to spread the good news. The “new way” starts to be known in small circles as the “true way”. Our programmer is encouraged to write a book. A bible is born!
  8. More and more people hear about the underground “true way”. Some bother to try understand it and the context which lead to its creation. Some manage to generalize it and apply it in new contexts. Some reject it because it is not familiar. Some reject it because it does not apply in their context.
  9. The change wave is now strong. Most people start to hear about the “new true way”. It sounds good. People start to see the daily problems they have with the “old ways” in a new light. Wishful thinking makes them see all solutions coming from the “new way”, the “true way”. Nobody knows anymore where and how the “true way” was born. Our programmer might decide to cash in on the new found celebrity or he might get scared and start to fight the madness. If he does that he becomes a fallen prophet. The “true way” is the deity, our programmer only the messenger.
  10. The “true way” is adopted as the “only way”. Manuals are written. Processes are created. People are trained and certified. The “only way” is enforced in some places with an iron fist. It starts to look bad if you don’t use it for all your problems. People still using “old ways” are seen like dinosaurs. A dogma was born!
  11. Some other programmer tries to solve a new very specific problem. He tries to tackle it using the tools at hand…

Dogma (plural is either dogmata or dogmas) is the established belief or doctrine held by a religion, ideology or any kind of organization, thought to be authoritative and not to be disputed, doubted or diverged from.

How often do you see this pattern of dogma creation in software development? More often than you think. How to recognize a newly born dogma? Every time when you have to do something that doesn’t make sense, only to satisfy some principle of X where X is a popular and established way of doing things, you just discovered that X is a dogma.

Let’s see a list of practices that are applied unfortunately most of the time as dogmata even if they certainly have value in specific contexts. Keep in mind that I am describing dogmatic applications of the concepts I am listing.

1. Agile software development

Agile ain’t agile no more when adopted as the official enterprise software development process. The problem with agile is “agile != flexible”.
Agile was born in the world of contractors who have to show something for the money at every status meeting. For a while clients let themselves fooled with use cases written on many hundreds of pages but this doesn’t work no more. They smartened up and demanded to see something working, for a change. Agile came in handy because it allows to show a prototype early on. As long as you keep adding features to it and you are able to demo them there is a good chance your contract will be extended.
While in theory the Agile methodologies tell you to be flexible, the cruel reality is that most people that apply them don’t think a lot before taking the manual and enforcing every bullet point with a thick stick.
There are lots of things that can go horribly wrong when forcing agile in the wrong context. Let’s pick on two:

  • Short development cycles or scrum at every phase of a project
    The problem here is that sometimes it is impossible to have a short development cycle and demo features at the end of it. You cannot demo a car engine until it is fully built. It works for projects that have a shallow infrastructure and lots of small features. It doesn’t work at all for projects where the infrastructure is 90% of the effort.
  • Pair programming
    I had two successful experiences with pair programming many years ago. In both situations it worked well because me and my pair had matching personalities, we were actually good friends, the problems were well defined and we didn’t routinely program in pairs for very long periods of time. Twist any of these conditions and you may end up with people killing each other with the keyboards.

Point to take home: not all projects and not all phases of a project are suited for Agile. For another opinion about Agile as dogma read here.

2. Test driven development

TDD has a noble goal but, as we all know, the road to hell is paved with those. I see a bunch of problems with TDD. First of all not all people think and create in the same way. Differences in personality, imagination, IQ, mood all lead to different ways of writing software. Some people naturally feel TDD is the best thing for them, some cannot stand it because it collides with their way of creating value.
Also in my opinion TDD makes people focus on the wrong thing. I think quality comes from well understood problems and good structure. Test should only verify this. They should not replace design and architecture. My prediction is that TDD leads to architectural disintegration because of the wrong focus at every step in the development.
And on top of everything, some complex code might not even be testable (think multi-threading) without major headaches.

I prefer Thought Driven Development.

3. Design patterns

The problem with design patterns can be summarized this way: “when you have a hammer all problems start to look like nails”. Many programmers consider them solutions even before understanding the problems they face. I was once accused by a fellow programmer that I messed up with the singleton principle when I wrote a class that managed more than one instance of itself. When I responded that I didn’t care about the mentioned principle he wanted to beat me with the GoF book.

4. UML as programming language

A couple of months back I wrote a full post about how I feel about UML – 13 reasons for UML’s descent into darkness. The value of UML, in my mind, was always to facilitate discussions between designers. But please, pretty please, don’t force me to generate empty code shells from UML diagrams created with insanely expensive tools just to get hundreds of pages of useless pseudo-documentation.

5. Open concept

I know of companies who force the so called “open concept” at work. This means everybody works out in the open. No private offices, no cubes, sometimes not even fixed locations. Managers who do this to their programmer have never been programmers themselves. Distractions kill a programmer’s productivity. Programmers are often described as very intelligent people with ADD. They can concentrate as long as there is no distraction. Repeated distractions make them zero productive. Everything is well explained in Tom DeMarco’s book Peopleware.

6. Exception handling in Java

In Java world there is a debate between two camps. Some favor checked exceptions while others consider them a failed experiment and propose a migration to unchecked exceptions. Again the focus here is on the wrong thing. The important thing to remember is that your code has to deal with failure scenarios. Checked exceptions are just a way to remind you about this annoying fact. Checked or unchecked you still have to handle exceptions or your applications will fall flat on its face sooner or later.

For my full opinion on Java exceptions start reading here: Exceptional Java – Thoughts on Java exceptions

7. Code reviews

Code reviews as in Fagan style code reviews are wishful thinking for lots of organizations. My opinion is they focus on the wrong goal – finding defects. As a dogma this practice forces you to fake code reviews because there is no way in hell you can inspect all the code by the book. Once I even got this lecture from a code review Nazi: “Code reviews should happen before testing the code at all, just after a successful compilation. This way we can find more defect during the reviews and the effectiveness of the review is greater!”. Sadly, I am not joking.

8. Code comments

Here we have multiple dogmas. Some people want no stinking comments in the code. Just write clear code and use good naming conventions. Others are very pedantic and want ample narrative comments all over the place (see useless comments for getters and setters in Java: “getX() gets the x, setX(x) sets the x”). Both are reasons for terrorizing programmers from a position of power. Both are wrong – one should comment no more and no less than what is needed to make the code maintainable.

There are lots of other examples but I think I made my point: There is no silver bullet in software development!
There might be TAO in every method you encounter in the software industry, but not when applied blindly.

15 thoughts on “Software development dogmata – good practices gone bad”

  1. It’s probably just a sign of a young and immature industry; everybody is frantically searching for things to grab onto to keep from drowning. Of course, once they’ve found something to prop them up, it has to be the very best they reason, allowing them to avoid having to continue the search. People are strange, programmers are …

    Paul.
    http://theprogrammersparadox.blogspot.com

  2. This reminds me of a poem by the Greek modern poet Constantine Cavafy: “Do Truth and Falsehood really exist? or only New and Old do exist, and Falsehood is but a truth grown old?”

  3. Excellent post, though it comes across as a little angry, and contains some rewritten history and some very judgmental statements, but overall the theme is spot-on.

    Some additional examples:

    Waterfall, initially conceived as something that you would iterate quickly with, was adopted as dogma and turned into grand phases of a project. The idea’s still fine, in its informal interpretation, but you just can’t slam it down as a grand template for success on every project.

    CMM was a good idea too, as something to informally check yourself against. But the whole formalized CMM certification thing was a disaster. The CIMM guys took it in the right direction (not the CMMI guys), but the damage has been done.

    I’d like to emphasize the topic of distractions, because I’ve heard managers extolling the “quiet” workspace, but it seems to me as a programmer that distractions are the key concept. For example, juicy gossip is extremely distracting — *especially* when whispered quietly!

    Cubicle environments have been uniformly bad in my experience, whereas open workspace has been both the worst, and the best. When open workspace works well, it separates you from those who aren’t collaborating with you, and your collaborators respect your need to focus. It is, more properly, a “project” workspace, not “open” to outsiders and distractions. I love that immersed-in-the-project feeling of a great project workspace. In a cube farm, there are no sound-proof walls to stop a distraction. You WILL hear your neighbor’s phone ringing. Usually there’s nothing you can do except put on headphones or take your laptop to the nearest coffee shop, which is ironically seems to be a far more productive environment than a cube farm. If you’re really lucky, you can establish a project workspace that works well. It is a bit tricky. Or you could go work for a company that believes dogmatically in private offices ;-) Well, it’s better than cubicles.

  4. @Paul W. Homer
    I guess that is just human nature. What you know is always better than what you don’t. Why people prefer to grab an idea and just apply it instead of reasoning about it? I think it is the education system that emphasizes knowledge over critical thinking.

    @Eli Burmin
    You are welcome :)

    @Federico Grilli
    Very nice :). Maybe falsehood is a truth well understood.. or not understood… now I am confused :)

  5. @awh
    I only knew CMM from the “being certified” perspective and it wasn’t pleasant. Certifications on process and methodologies in software development look more and more like a consulting scam. People who have no clue how to write a line of code certify your way of doing it.
    Most cubicle lands don’t separate people on projects so it is really hard to concentrate. This is not very important during implementation but during design and architecture phases it is. So I prefer to isolate myself when doing design and trying to solve complicated problems. I actually prefer private offices to open space. An open space should exist also for people to meet and discuss or work together.

  6. I am glad there are other people who are coming to the same conclusion. That all of these project management process, or the software patterns. These are all variations on a theme, and one is not necessarily better than others. That trying to generalize all of these different practices as a “true way” is a great way to make sure you are the next dinosaur. On the other hand you keep following the next big thing, you will never get anywhere, because you are chasing windmills.

    My experience leads me to believe that being a methodology zealot is very dangerous, because it can blind you to a solution that may be the best fit. If you choose to follow just one methodology you lose your flexibility, no matter the name, or the rules.

  7. Hello. I like the summary, “don’t apply badly”, but I have a few items to nitpick with your post:

    You pick on Agile in #1, which I think is a mistake. Agile is the least dogmatic, hmm, I don’t want to say the word ‘methodology’, let’s say ‘set of principles and practices. It’s the least dogmatic. You are correct to assert that Agile is in a hype cycle, but you’re incorrect to assert that Agile is essentially the same as any other, um again I don’t want to use the term, methodology.

    Don’t equate two unlike things. Agile is not JUST the new hot software engineering fad like all the rest.

    Further.

    Your common path listed at the top is a good dramatization, but isn’t necessarily factual; you could have easily substituted the South Park common path instead:
    1. Create new ‘true way’
    2. ???
    3. Profit!!!

    What I’m getting at is, there’s no point in including it.

    Further. I’ll point out that, if you’re bashing TDD, at least try it first. The difference between informed criticism and uninformed criticism is vast. From the post above, I can’t tell if you’ve tried TDD or if you’ve just seen it from afar.

    Let me just summarize by saying: if you’re going to attack someone who is dogmatic, do so factually, not by dismissing their ideas. Don’t wave your hands and say “oh it’s all the same, what difference does it make.” Agile and TDD specifically are two things that (if done properly!) still involve pain up-front in the learning curve. The same can be said for pair programming, though again, if you explain to the dogmatic person that “the emotional cost/loss of morale involved with sitting me next to Farting Mike all day is just too high,” maybe you’ll get through. Maybe not.

    Also crazy people are just crazy, like your code reviews guy. That’s not dogmatic, that’s just nuts.

  8. his article is spot-on. Software development is a craft, not a science, nor is it a manufacturing process. Software ‘Engineering’ is a misleading term. Building software is not like building a bridge. There are no silver bullets. No single methodology is ever going to cover all the software development cases. There are simply too many different environments for which software is built.

  9. Thanks for this post but i have believe that being a methodology zealot is very dangerous, because it can blind you to a solution that may be the best fit. If you choose to follow just one methodology you lose your flexibility, no matter the name, or the rules. so guys go ahead..

  10. Interesting read, i liked it.

    I agree that agile is all about being agile and flexible. But on the other hand these rules are there for a reason. From my experience you have to be guided by some rules especially at the beginning to fully appreciate scrum.

    I totally agree that you cant adopt everything. But thats the part of scrum. You analyze your sprint, pick one or two things you want to try different and apply change. With time you find a sweet balance between all the forces pushing on the team and project.

    I think its all about people who believe they know the best and dont want to change their mind. If you have manager who forces you to do it as it states in the book you are screwed ;- ) If your managers are clever they will let you (the team) to do the work the best way possible only asking you to try a new way.

    You will see that people will object to rules like sticky boards and standups (actually standing up). but once you work this way for a while you will like it. and you will appreciate how big of a difference it makes.

    Scrum is the great fun and it actually works so i just would not like anyone to get the wrong idea. It is good, you just need to have common sense.

    art

  11. > people are strange, programmers are…

    What? Even stranger? No, we’re just lazy that’s all but yes, we do think up new ways to blow things up, so amongst the population on our Earth, you could say we are the elite.

    Smug? You bet.

  12. Good to see such articles. It highlights pitfalls to avoid and reminds us that the end goal is to be successful in developing the right software, where the correct approach is judicial tailoring and mixing of all knowledge.
    The intention behind invention of each methodology needs to be understood for proper application without becoming dogmatic. For example, the key element of all design patterns is isolation of parts that changes from other, so that code/design can be extended/modified with minimum risk to existing code. As such knowing all the patterns in textbooks is good but less important than understanding the principles underlying them.
    Similarly objective methodologies such as Agile is to reduce risk by tackling key elements first; Pair Programming reduces error escape rate via real time review by the other partner and so on for other practices.
    I believe dogmatism is sign of organization/person looking for legitimacy/approval. CMMI does not guarantee project success but may be required to win project.

Comments are closed.