Software as an industry cannot claim quality metrics acceptable to almost anyone on the aggregate and that has characterized the history of our profession. We must ‘finally’ experience the fundamental reset of our conceptual meta-model for processing software engineering akin to what occurred in the industrial revolution. Stepwise refinement is not enough.
But how? People have been saying this for decades.
While I am not so delusional to believe I have all the answers, I think the direction is clear, and I have collected my experience, research and practice into this ‘practitioners’ guide to get beyond the hype.
A ‘culture of assumed failure’ has emerged in many organizations around our work.For me the term ‘Software Factory’ has always been very appropriate as it is so firmly rooted in the fundamentals of global business.
Also think of the change the industrial revolution brought.
That is akin to what software will face now and well into 2020. It will demand so much in terms of the personal change required of each and ever person already entrenched in all too often dire conditions people have acclimated to today.
I am often asked if by ’software factory’ in some way it advocates replacing highly skilled (and paid) ‘craftsmen’ with far lower cost staff. The amusing aspect of that question is it is happening now in the outsourcing industry. Outsourcing is failing as it ignores the fundamental dynamics we learned from Agile practices.
To directly answer the question, it does the exact opposite (there are sill dire consequences for some). The ‘worker’ mentality will finally not be acceptable in software. Everyone will simply be recognized for what they are, and a career as a ‘developer’ only, in how we think of it today, will likely not exist.
In other words, the developer who succeeds will be a business domain expert, architect, designer, and more with the literal ‘code/tech’ skills being as assumed as speaking the native language is. The arbitrary lines mandated by the size of a team go away as you can do far more with far less people, but they must be the right people. So now the average is proven to be around 3,000% less productive then the best. We are shifting the hiring mandate into just the upper performers and executing on the proven facts in the practice and art of software development.
Damon Carr
July 2008
New York, NY
Introduction
Imagine if we were forced to make the same transformation that the craftsmen had to make when their ‘lives’ became automated on factory assembly lines. Before their world changed you had 
highly skilled craftsmen for almost all stages of manufacturing if not one craftsmen who arduously did everything.
The ‘craftsmen centric‘ model simply does not scale, nor does it have the characteristics needed in highly demanding markets where the cold facts of quarterly reports drive decisions. We are failing our stakeholders today more then ever I argue, as much of what is experienced in the negative is now avoidable. We are nearly out of excuses for or many failings. This is why software factories become a near imperative as I argue in this book. Their posses a vastly under-realized ability to simplify the adoption of most of the practices, tools, cultural shifts, job description changes, etc. that are required immediately. They encapsulate and package in a digestible way what is a very large amount of change. Also, it was only recently (again in this author’s opinion) that it was even feasible (in spite of many notable attempts) to implement software factories as I and many others describe and implement them now.
Software Engineering, much like the artist who will not be rushed, has proven that ‘predictability is not to be expected’ as I like to say and only a statistician would likely find humorous. Great progress has been made in areas such as quality, security and developer productivity but there are still too many fundamental and avoidable flaws in our industry.
It is quite hard to manage something they behaves in erratic and random ways. If a more predictable but slightly lower quality solution is presented, most American business executives (depending on the specifics) will choose the predictable solution. We know that the characteristic of being ‘the best’ if poorly correlated to overall success in a highly competitive market (see Beta vs. VCR or OS/2 vs. Windows).
To be honest I have been doing this long enough to know what to expect, however my statements are completely based on the many studies performed which attempted to measure our performance (see the appendix for some of the most notable). Logically, unless the goal is to produce true ‘works of art’ which are valued on some quality which requires the arduous work of hand-crafting, why would an organization consume more resources while loosing time to market advantage all for a result which is of lower quality and a lower ‘fitness’ compared to what they could of had?
As I will discuss, software factories can be applied to areas related to deeply technical domains such as SOA with great success. However the true power I believe comes when we reach a critical mass where technical challenge are no longer at the core of our efforts. When this happens (and for some it already has) everyone’s focus becomes adding business value with a far great direct line to revenue. Also the lines blur to a far greater degree between a ‘technologist’ and a ‘business domain expert’ to the point they are one and the same.
So the problem becomes how do we transform Software Engineering such that what I argue will be the core ‘manufacturing’ of the next 100 years in a similar fashion while finding a model that maintains the deeply intellectual foundation of the ‘art’ of software engineering and doesn’t trivialize it to the point that it becomes an undesirable, low-pay/low-skill profession.
After well over two decades of trying it is fair to say that it remains elusive for organizations to become highly effective at truly injecting reuse into their development efforts. I’m frankly not surprised as the reasons in my experience have noting to do with ant technology limitation. Software Factories are so compelling as one of their key benefits is the astounding level of intelligence that is captured, evolved, and leverage while also being fully customizable where needed.
The answers are complex and simple but rather obvious, just not in the software engineering domain. Many have tried before to achieve what we can achieve now, however there were problems not yet solved that we have the luxury of ignoring.
Said in a different way, this book covers precisely how we can achieve a high degree of predictability and confidence for our stakeholders. Almost everything that I or others discuss when speaking of factories owes a large debt to the work of the Agile community, which I will always claim to be a member of. In fact as the title should hint Agile is arguably even more relevant in the work of factories. We know for example that on the whole, your developers need to be much closer to your business, not farther way. Even at the same expert level as your best domain experts.
The fact is that today many view our work as nothing more then a burdensome expense that is require for continued existence. In far to many cases they are 100% correct.
So this book then is also about strategies for moving the nature of our work into a higher layer. It’s about the fundamental shifts in our thinking and the mechanics of how problems are solved. It’s about completely changing what is expected of us and where our limits reside in the value we can deliver.
From an upcoming review of ‘Agile Software Factories’
…the author makes some very strong statements in this book, and unfortunately I could find few that were not called for. This is far more then ‘beyond Agile’ or an attempt to capitalize on some buzz around these topics as I immediately though upon reading the title. The author pulls no punches in redressing the entire industry (mostly for companies where writing code has nothing to do with their business).
Although Agile is a core component of this book, and mastery of its practices and concepts is assumed, Agile ends up being a relatively small but critical aspect. What is described is an ambitious holistic set of practices crossing almost every organizational boundary, especially calling for a reboot of thinking of people and groups as ‘non-technical’ or ‘technical’. This is not to say developers are expected to disappear. Instead, the necessity of the massive divide which now exists between these group will simply go away.
Overall this book is not for your ‘9 to 5′ developer, although they probably could benefit more then any other group. Instead (and much to the author’s chagrin I would imagine) this work might become another ‘high bill rate/low result’ domain of the consultants. As pointed out, it is not even their fault, as often the very people who hire them are not REALLY willing to really change all that much when they realize that is what is required. Unless forces exist to mandate the changes described (and the author does point out what those forces would likely be are what they are now) they will likely not happen in spite of everyone knowing it would be a major step forward for our industry. It’s really in the hands of the decision makers and we know one thing will happen without any doubt : change.
….
Although the book covers what is now a very unsure future, the field is actually far more evolved then many know, and I or one finished the book a little upset at just how little things have really changed (although I knew this already from my work, I was not aware of the scale).I was encouraged however by the innovative people making ‘bet the business’ type investments already in factory driven product and just how much is out there now.
The author shows his experiences with applying Agile practices to creating quite real and in production software using precisely what is described. And for many the concepts will not be completely new, simply evolutionary (there is a strong sub-plot of the book around evolution and how lessons learns are so relevant here).
Commonalities between factories and broader ‘Lean Manufacturing’ concepts are also covered in depth, however the author mostly references the excellent work already done in this area and how is has been mostly accepted into the Agile community.
Also explored (in addition to much more) is the idea that Software Factories for the Agile-enabled company today will have a smaller learning curve then others in adopting factories. For those who couldn’t stomach the changes required to ‘embrace change’ it is obvious they will need to reconsider their position if the author is even remotely correct in his (and many other’s) enthusiasm for what is described.
NOTE: Although the author’s stated goal was a vendor agnostic work, one cannot come away without seeing the Microsoft bias.
Also, the early edition of the book I read was based on products that have not been delivered yet from Microsoft or others in most cases (although in almost all cases betas are out).
