What's wrong with software development in large corporations

How much time does it take to become a knowledgeable professional? 2 years, 4 years, 10 years or half a lifetime? How much time does it take to become a good artist such as painter, musician or composer? What about the saying that good artists are born? That somehow implicitly means that either you are a good artist by birth or you aren't and you can try as hard as you wish, you'll never become a good artist even if you had an eternity.

What has that all to do with software development? If work in the IT department of a larger corporation or as a consultant helping them, you'll certainly can tell by now what I'm about to write next. It doesn't matter where you look, you'll get to know over time hundreds of people working as programmers for those corporations and it's very, very rare to encounter someone who really has a clue about what he's doing. Think about something as common as object-oriented programming. How many people do you know who really got it and are able to come up instantly with a good OOP design with good separation of concern, all those little collaborator objects talking to each other and so forth? I guess you will have to think hard. Sure there has been that one guy or gal but that's about it - isn't it? But you certainly can easily remember a fair number of people who call themselves software architect and the only contribution to the project you were working on where a lot of blockage and repeating stuff that sounded awfully similar to some vendor's sales talk to praise their rigid waterfall'ish tools.

But why is that? I think it has something to do with education and the expected behavior good employees should show. Let's start with the latter.

Today's corporations are still created after the organizational model of the military. Any military organization has a clear chain of command where orders flow down from high rank to lower rank and the lower rank is expected to execute the order given by the higher rank without questioning it. That works great in a battle situation - at least that's what you usually hear about that. But let's ask for a moment what that battle situation actually is. It's quite simple and cruel at the same time. Any battle ends with a large number of dead people and low ranking soldiers are disposable "units". Nobody really cares if they die. The less the know about the overall picture and the more they believe in what you want them to believe the easier it is to make them accept the risk, which nobody would ever accept under normal circumstances. But let's not slip into talking too much about that topic. The point is that in the military it is not common to not inform lower ranks about the overall picture, they get some strong motivation to do a stupid thing (put themselves up for being shot) and they are made to follow orders without questioning - and of course there are sanctions put in place for those who do question and not follow an order.

Now let's look at any corporation. The chief in command is the CEO. He has a number of officers (CFO, CIO, etc)., which command other officers at the departamental level and so on until it comes to the ranks of sergeant, corporal leading groups of privates. The sergeants and corporals are the low ranking line managers and the privates are the common workers. So in the world of corporate software development we can identify the programmer and tester as privates and their team lead as some kind of corporal. The project manager might be some kind of sergeant. As you can see that all so important project team is not so important after all. At least from the vantage point of the commander in chief, the CEO commanding the corporation.

How much time does it take to become a good soldier? Most armies of the world train their lowest ranking members for something betwen a 12 or 18 months. In the beginning they perform some tests to determine whether a person is better suited for less complex work, that is usually Army or manufacturing in most corporations, or for more complex stuff, which is usually Navy/Air Force or administration/support/development. Then basic training starts and the most important part of that basic training is to learn the rules. That is to make people understand that they have to obey orders or suffer consequences. In the corporate world the common consequences are reassignment or you simply get fired. And as people's well-being depends usually on their job, everybody understands all too well that it's a bad idea to show too much of an independent mind.

So... What is software development in its core? Is it some other form of manufacturing? Or is it research or art or what? Can you construct software or is it more that you create software?

It depends on how you answer these questions whether you want corporate soldiers (sergeants, corporals and private) to work on projects or not.

In my opinion software development is something between research and art. After all the word development implicates that something gets created for a purpose and before you develop you will have to know what the purpose is (the problem you want to solve; what you want to improve), what the environment looks like, who will use it, etc. So that includes a very large research component as well. I think one can describe software development as some kind of exploration with the goal to create something with a certain level of quality to solve a problem that is well understood.

That doesn't sound very much like a battle situation - does it? One may think that a good part of software development is about learning, reflection, experimentation and other activities usually associated with scientists. And when it comes to write the code then skills and attitudes, which musicians are known to possess, will be helpful. It needs an awful amount of time to practice test-driven development and be disciplined enough to implemented only what's needed to solve a problem described by a user story.

What's the situation of most corporate programmers? They get hired based on their resume and then put to work on a project. Typically the HR department decides upon hiring or not and they do some kind of pattern matching by looking at the keywords. They get a shopping list from the department and if it says Java, Struts and Junior Programmer, they will easily find someone with less than 5 years experience. Now that poor soul arrives and gets assigned some tasks by the team lead (sent into battle) and there we go. Will that person be creative? Will she be part of a group analyzing the business problem the team wants to solve? Usually not, because everything has already been done and now it's about to write the code based on a specification, which is kind of the battle plan.

Now here is a suggestion. Wouldn't it be great to employ the master/apprentice model known since the middle ages? Why do we allow inexperienced people to mess around with the most important thing in software, which is the code? I think a well motivated apprentice working alongside with a good master will evolve into a true master himself over the years. He will take more and more load off his master as his skills evolve. He will understand why the master does employ certain techniques and why he doesn't use X or Y in different situations. That takes time. But the quality will be higher and so will be the capacity of the whole team, because the team will be comprised of more masters as time goes on. That's an investment and corporation leaders would be wise to invest into the future of the corporation they serve instead of only stare at the short-term shareholder value created in any given quarter.

How long does it take to become a master? In my home country Germany plumpers, carpenters, painters, electricians and others spend three years as apprentices and easily up to eight years as journeymen before they are allowed to lead their own shop as masters. So that old saying "learn how to program in ten years" isn't that wrong after all. Between 4 - 6 years formal study at the university followed by an apprenticeship of maybe 2 years means one learns for at least 6 - 8 years. The next few years are then to hone one's skills and after 10 years one will know the whys and hows of software development.

This article has been posted to social media sites. There might be comments. Just follow the links: