This is probably the most important blog entry I’ve ever posted.

The video is the longest video I’ve ever posted on the blog, and for that reason, I’ll keep the text content to a minimum. 

Here’s why you should watch the video:  CMMI may be entirely wrong for you, and you may not know it!

The video explains an epically crucial reality about CMMI that many agile (and other) teams are not aware of, leading them unknowingly down a path of self-defeat and damage.  All of which could be avoided with this one super-critical piece of knowledge.

You’ll thank me later.

Backstory:

The lure of seemingly limitless opportunities can be quite strong, obviously.  And, especially in tough economic times, succumbing to that lure can cause even the best of businesses to act unwisely.  Such is the lure of CMMI ratings.

Well, anything that’s very alluring can cause unwise behavior, I suppose.  Whether it’s as apparently harmless as indulging in a luscious dessert, spending money on unnecessary luxuries, or any of equally limitless opportunities to make bad choices, doing what we want instead of doing what’s right shows up even when working with CMMI.

This blog is full of examples of such bad CMMI choices, but there’s one bad choice I haven’t mentioned much about.  That’s the choice to even try to use CMMI.

When working with a knowledgeable, concerned, trustworthy CMMI consultant, an organization should be steered away from CMMI when their circumstance doesn’t align well with model-based improvement using CMMI.  In some cases, it may be a matter of steering towards the right CMMI constellation (e.g., for Development, or, for Services).  However, just as whether or not CMMI is right for an organization ought to be discovered before too much energy is put into it, so should the decision about a particular maturity level within the constellation.

No CMMI constellation should be attempted if/when the organization doesn’t control the work that it does.  Namely, that the work it does is controlled by another organization, such as a customer.  Or, put the other way, CMMI should only be used if/when the processes used by the people doing the work are controlled by the same organization using CMMI to improve them.

At Maturity Level 2 (ML2), almost any type of work can use the practices in that level to improve its performance and to demonstrate that the practices are in place.  However, at Maturity Level 3 (ML3), you have to be doing the type of work in the particular constellation in order to be able to use the practices in it.  If you’re not doing that type of work, the practices will be irrelevant.  Attempting to use the practices when there’s no such work being done will only cause the practices to get in the way and add nothing but frustration.

In particular, if you’re not doing work that involves structured engineering analysis, CMMI for Development at ML3 will be truly unwieldy.

Adding practices for work you’re not doing is an example of the bad behavior many organization exhibit when they’re chasing a level rating rather than hot on the trail of performance improvements.  It’s these sorts of behaviors that are somehow rationalized as being beneficial when, in fact, they are unequivocally, diametrically, and everything but beneficial.  They are a colossal waste of time and money and detrimental to morale and productivity.

You really need carve out about 11 minutes to watch the video.

A more appropriate title would have been "counting changes" but it would have hardly been as interesting. 

Change happens.  And often.

In particular, when a product is in its operation and maintenance ("O&M") phase, changes are constant.  (Note: O&M is frequently called "production", and this simple choice of words may also be part of the issue.)  But, too often, changes to products are handled as afterthoughts.  When handled as "afterthoughts", product features and functions receive far less discipline and attention than warranted by the the magnitude of the change were the new or different feature/functionality have been introduced during the original product development phase.

In other words, treating real development as one would treat a simple update just because the development is happening while the product is in production is a mistake.  However, it’s a mistake that can be easily diffused and reversed.

O&M work has technical effort involved.  Just because you’re "only" making changes to existing products that have already been designed, does not mean that there aren’t design-related tasks necessary to make the changes in the O&M requests.  Ignoring the engineering perspective of changes just because you didn’t do the original design or because the original (lion’s share of) design, integration and verification work were done a while back doesn’t mean you don’t have engineering tasks ahead of you.

In O&M, analysis is still needed to ensure there really aren’t more serious changes or impacts resulting from the changes.  In O&M, technical information needs to be updated so that they are current with the product.  In business process software, much of the O&M has to do with forms and reports.  Even when creating/modifying forms, while there may not be any technical work, per se, there is design work in the UI.  The form or report itself.  And even if you didn’t do that UI design work, you still need to ensure that the new form can accept the data being rendered to it (or vice-versa: the data can be fit into the report).

It’s frightening, when you think about it, how much of the products we use every day — and many more products that we don’t know about that are used by government and industry 24/7 — are actually "developed" while in the "O&M" phase of the product life cycle when the disciplines of new product development are often tossed out the door with the packing material the original product came in.  Get that?  Many products are developed while in the "official" O&M phase, but when that happens they’re not created with the same technical acumen as when the product is initially developed.

(I have more on this topic, and how to deal with business operations for products in the O&M phase, in this Advisor article from the Cutter consortium.)

In a sadly high number of operations I’ve encountered, once a product is put into production, i.e., is in O&M, the developers assigned to work on it aren’t top-notch.  Even in those organizations where such deleterious decision-paths aren’t chosen, the common experience in many organizations is that the developers are relied-upon even more for their intimate knowledge of the product and the product’s documented functionality — as would have otherwise been captured in designs, specifications, tests and similar work artifacts of new product development.  In these organizations, the only way to know the current state of the product is to know the product.  And, the only way to fix things when they go wrong is to pull together enough people who retain knowledge of the product and sift through their collective memories.  The common work artifacts of new product development are frequently left to rot once the product is in O&M, and what’s worse is that the people working on the new/changed features and functionality don’t do the same level of review or analysis that would have been done were the functionality or other changes been in-work when the product was originally developed.  Of course, it’s rather challenging to conduct reviews or analysis when the product definition only exists as distributed among people’s heads.  Can you begin to see the compounding technical debt this is causing?

I’ve actually heard developers working on legacy products question the benefits of technical analysis and reviews for their product!  As though their work is any more immune to defect-inducing mistakes than the work of the new product developers.  What’s worse is that without the reviews and analyses, defect data to support such a rose-colored view seldom exists!  It’s entirely likely, instead, that were such data about in-process defects (e.g., mistakes in logic, design, failing to account for other consequences) to be collected and analyzed, it would uncover a strong concentration of defects resulting from insufficient analysis that should have happened before the O&M changes were being made.

Except in cases where O&M activities are fundamentally not making any changes to form, fit, feature, appearance, organization, integrity, performance, complexity, usability  or function of the product, there should be engineering analysis.  For that matter, what the heck are people doing in O&M if they’re not making any changes to form, fit, feature, appearance, organization, integrity, performance, complexity, usability or function of the product?!

If anyone still believes O&M work doesn’t involve engineering, then they might need to check their definition of O&M.  Changes to product are happening and they’d better be counted because if not, such thinking fools organizations into believing their field failures aren’t related to this.  Changes count as technical work and should be treated as such.

(I have more on this topic, including how to help treat O&M and development with more consistent technical acumen in this Advisor article from the Cutter consortium.)

Apple, Inc. learned the hard way what happens when engineering isn’t complete.  In particular, when verification and/or validation aren’t performed thoroughly.

Verification is ensuring that what you’re up to meets requirements.  “ON PAPER.”  BEFORE you commit to making the product.  It’s that part where you do some analysis to figure out whether what you think will work, will actually do what you expect it to do.  Such as, walking through an algorithm or an equation by hand to make sure the logic is right or that the math is right.  Or, stepping through some code to see what’s going on before you assume that it is behaving.  Just because something you built passes tests, doesn’t mean it is verified.  All passing tests means is just that: you passed tests.  Passing tests assumes the tests are correct.  If you’re going to rely on tests, then the tests need to be verified if you’re not going to verify the requirements or the design, etc.  Another problem with tests is that too many organizations only test at the end.  Verification looks a lot more like incremental testing.  Hey wait!  Where’ve we seen that sort of stuff before?

Had Apple’s verification efforts been more robust, they would have caught the algorithm error that incorrectly displays the signal strength (a.k.a., “number of bars”) on the iPhone4.  This is why peer review is so central to most verification steps.  The purpose of peer review, and of verification, is to catch defective thinking.  OK, that’s a bit crude and rude… it’s not that people’s thinking is defective, per se, but that thinking alone didn’t catch everything, which is why we like to have other people looking at our thinking.  Even Albert Einstein submitted his work for peer review.

Validation is ensuring the product will work as intended when placed in the users’ environments.  In other words, it’s as simple as asking, “when real users use our product, how will they use it, and will our product work like we/they expect it to work?”  Sometimes this is not something that can be done on paper, and you need some sort of “real” product, so you build a prototype.  Just as often it’s not something that can be done “for real” because you don’t get an opportunity (yet) to take your product into orbit before it has to go into orbit to work.  Sometimes you only get one shot, and so you do what you can to best approximate the real working environment.  But neither of these extreme conditions can be used by Apple as excuses for not validating whether or not the phone will work as expected while being held by the user to make calls.

Had Apple’s validation been operating on all bars, they likely would have caught this while in the lab.  When sitting in its sterile, padded vice, in some small anechoic chamber, after taking great care to ensure there are no unintended signals and nothing metallic touching the case, someone might’ve noticed, “gee, do you think our users might actually make calls this way?”  And, instead of responding, “that’s not what we’re testing here”, someone might’ve stepped up and said, “hey, does our test plan have anything in it where we’re running this test while someone’s actually using the phone?”

Again, testing isn’t enough.  Why not!?  After all, isn’t putting it in a lab with or without someone holding the phone a test?   True…  However, I go back to the same issue we saw when using testing as the primary means of performing verification… Testing is too often at the end.  Validating at the end is too late.  You need to validate along the way.  In fact, it’s entirely possible that Apple *did* do validation “tests” of the case separately from the complete system, and, in *those* tests — where the case/antenna were mere components being tested in the lab — performed fine, and, then only when the unit was assembled and tested as a complete system would the issue have been found.  In such a scenario we learn that component (elsewhere known as “unit testing”) is not enough.  We also need system testing (in the lab) and user testing (in real life).  Back we go to iterative and incremental…

So you see… we have a lot we can apply from ordinary engineering, from agile, and from performance improvement.  Not only does this… uh… validate(?) that “agile” and “CMMI” can work together but that for some situations, others can learn from applying both.

In full disclosure, as a new owner of an iPhone 4, I am very pleased with the device.  I can really see why people love it and become devotees of Apple’s products.  Honestly, it kicks the snot out of my prior “smart” phone in every measurable and qualitative way.  And, just so I’m not leaving anything out, the two devices are pretty much equally balanced in functionality (web, email, social, wifi, etc.)  – even with the strange behaviors that are promised to be fixed.  For a few years, this iPhone will rule the market and I’ll be happy to use it.

Besides embarrassing, this will be an expensive couple of engineering oversights for Apple to fix.  And, they were entirely avoidable for an up-front investment in engineering at an infinitesimal fraction of the cost/time it will take to fix.  For even less than one day of their engineering and deployment team’s salary, AgileCMMI can make this never happen again.

Apple, look me up.  I’m easy to find.

The Spectacular Crash and Burn (mine)

My morning talk on Top 10 Clues You’re Probably Not Doing Engineering was a spectacular bust!  Oh, but the lessons I learned!

In the immediate after-action analysis I realized what had happened.  (Any other excuses I might’ve made at the time to the contrary.)  Here’s what actually happened (at least the most likely scenario):

I was tired.

On the evening (read: early morning) when I was working on the final final of the presentation, instead of merging the presentation’s pictures with the slides right there, I chose to procrastinate that task to the next day (or later).  Welllllll, being as tired as I was, by the morning I’d forgotten that I had not completed that task.

OK, so that explains why my slides didn’t have their pictures.  So, moving on, my next idea was to just present without the pictures.  That idea was met by the audience with a resounding moan of disappointment.  (I guess a lot of folks were at prior presentations and they liked my pictures.)  So, off I navigated to grab the source files from their folder.  The folder where all the pictures were supposed to be.  And they weren’t there.  WTF?  How do you/ I explain that!?!

I was tired.

So, back to the night (read: early morning) of the great non-merging event.  What must’ve happened (at least the most likely scenario) is that some files were saved to some folder other than the one with all the source materials, and I was completely oblivious to it.  How?  Of course!  I was too tired to notice.

Always quick to find the silver lining, my tremendously inspiring wife, Jeanne, (she’s a veritable silver-lining-finder) pointed out, "You’ve got great material for future presentations!  Just talk about how you can’t take care of business if you don’t take care of yourself!" 

Dangit!

Caught.  Red-handed.  Pants down.  Wedgie.

The same applies to your team, work group and your company.  If you don’t take care of them, they can’t take care of the business.  That’s a People CMM presentation if I ever heard one!

The rest of my day was spent licking my wounds. 

Thanks to everyone who said nice things about it nonetheless.

I keep answering the same question over and over again.  On one hand, it’s good that interest in CMMI (and Agile) is growing.  On the other, I’d really like the development world to be able to move onto more interesting challenges.  I know it’s foolish to believe that answering a question on my blog will stop the questions from coming in, but at least once it’s answered, I’ll just point people here instead of repeating myself in emails and phone calls.

Whether the desire is to implement CMMI in an agile environment or a traditional environment, the key to making CMMI “work”, and by “work” I don’t mean “get you a rating”, or as people incorrectly call it, “get you CMMI certified”.  By making CMMI “work” I mean, incorporating CMMI into your organization in such a way that the resulting processes:

• Add value to your organization and to your customers,
• Are things people want to do,
• Foster continuous communication and improvement,
• Measure what matters, and still
• Result in a desired rating (as a by-product, not the goal).

Here it is.  The “key” to making CMMI work is what I call, an “Engineering Mindset”.   Engineers realize that when they fail to be honest, precise, and thorough, things fail to work and many times this failure means someone is either disappointed, or worse, hurt or killed.  To avoid the undesirable outcomes, engineers have a sense of integrity. Not in the moral or religious sense, but in the sense that things are whole and complete.  An Engineering Mindset includes seeing things as problems or opportunities and their corresponding solutions or exploits.  So an Engineering Mindset has two elements: Integrity and Solutions.

For using CMMI, these two elements align nicely with what I’ve been saying for years.  These elements are not magic or glamorous, but people find them to be challenging, nonetheless:

1. Be clear and honest with CMMI and how it works and what it really is and isn’t.  If you’re following in the path of Agile, you MUST also be honest with the Agile Manifesto. The “things on the right” *do* have value, and their value must be justified.  If you’re not all that concerned about “agile”, you MUST be concerned about value.  And for that you must get a grip on your work flow and be committed to knowing the value stream of your work flow and be committed to continuously and aggressively pursuing the elimination of non-value-added activities.  This often takes shape in investment in activities and a culture that appreciates and recognizes awareness, tools (methods, technology, assets, systems, etc.), LEARNING, pluralism, empowerment, and excellence.
2. Recognize that CMMI PA practices are not processes but that each practice avoids some pitfall. Identify how your organization avoids that pitfall and you will have accomplished the intent of the practice. If how you avoid it is weak or absent, then it’s something you should build into your work.

Please pay attention to this next point: In my work, the organizations that embraced key point #1 had no trouble with key point #2, and, in many cases, required very little new or different to take them to any maturity level they wanted.

When we work with clients, we spend most of our effort to help them internalize #1 (if they need it).  When they’re able to do #1, they generally “get” #2 on their own AND they figure out where the value lies in the practices in CMMI they’re not already doing AND they figure out where and how to incorporate those practices into their own.

If your organization can’t figure out the value to your organization in CMMI practices, you’re not allowing yourselves to be creative enough and you’re not thinking like engineers.  An engineering mindset must exist first.  When it doesn’t, it’s a long and challenging road ahead.  Choose wisely.

An article in this month’s CrossTalk periodical, is now out.

See it here.

Download it here .

Enjoy!

P.S.  There are other great articles in the issue as well.  I’m in great company with an article by my friend, colleague and client, Jeff Dutton.  And, don’t miss out what’s coming next in v1.3 from my buds Mike Philips and Sandy Schrum!

Your people with prior CMM/CMMI experience are probably worse than worthless, they’ll probably cause you to fail.

Why?

Because what they (or you) think they (or you) know is probably wrong and the advice you’re getting, the expectations being generated are entirely off base.

It all goes back to the many ways in which CMMI can be done poorly and the few, simple, but hard work ways in which it can be done correctly.

Every time I meet with a new prospect I’m confronted with reams of inaccurate assumptions and assertions about what it will take to implement CMMI and how am I expected to “do all that” and still claim to be “agile”.

My simple answer: I’m not going to do all that.  And, you shouldn’t be doing it either.

Seriously, you’ve got to wonder about executives who will force their company into doing stupid things for the sake of a rating instead of doing their homework to learn about CMMI before they head out on an implementation journey.

A recent client didn’t know any better.  They hired a consultant and an appraiser to evaluate their work against CMMI and to help them prepare for a SCAMPI appraisal.  Unfortunately, they got as far as the appraisal only to realize they weren’t going to get the target Maturity Level.  (I won’t get into some of the inappropriate behavior of the firm they hired.)

However, when this client was confronted with:

1. Do something stupid, or
2. Find a better way to do something smart.

They took option B and found a consultant and an appraiser who understood their context and found how to both be on a disciplined improvement path while also remaining true to their own business.

Fortunately for them, this client had a strong engineering backbone and knew what they did worked and were confident in their processes.  Many companies have a while before they can claim that much.

Next week:

Picking a Lead Appraiser:  “Dammit, Jim!  I’m a doctor not a bricklayer.”

I’ve been noticing an interesting "convergence" going on in this corner of the universe:  "engineering" is being re-introduced to the idea of software.  It’s fascinating how no sooner do I have the idea to write this blog (while entirely not connected to the Internet), then I re-connected to get an article that’s entirely speaking of the same root issue.

Of course, I’m not saying that engineering had completely left the software universe, though, a strong argument can be made that for the last decade and then some, software has allowed engineering to escape from the premises.  In particular, architecture, analysis, systems thinking, design, hardware and other integration issues, and planned, deliberate, methodical testing have largely been allowed to merely "emerge" from the work that was completed.

Again, I’m not advocating for Big ____ Up-Front as a solution, what I’m pointing out is that many people who embrace agile methods incorporate engineering practices into how they organize and perform their work, but enough don’t that it raises issues with agile scalability.

And here’s where I *am* expecting to annoy some people…

Programming and Development are NOT the same.

Development is an engineering function.  Developers ought to be using engineering practices in what they do.  Just look at the word "development".  The connotation is that something is "grown" or "evolved".  The denotation of ‘development’ in the technical sense is that it is done deliberately, not by happenstance.

This idea is where I believe software, in general, not limited to agile practices have short-changed themselves.  Too often, activities that amount to nothing more than programming are called development when no actual engineering is happening.  In other words, programming is allowed to take place without any (or at best without enough) engineering, and therefore what’s really happening is the building of something without any/enough forethought about the thing itself that is to be built.  Instead, what happens too often is all the focus is on "staying busy" (albeit on ostensibly priority work), but what is worked on is absent sufficient technical rhyme or reason.

Is this true of all agile development?  NO.  But, it is what happens in many organizations when they don’t have sufficient technical leadership.  For what it’s worth, many development projects don’t need much engineering, and product development is sufficiently described in tasks defined by few people.  So the jump from development to programming is small and fast. 

However, there are projects (or tasks) of sufficient technical complexity that skipping the engineering and handling such projects/tasks as programming alone is where I believe a space is created for the unfair reputation for agile and its scalability, as well as some of the anti-process bias among agile proponents.  When I read (and sometimes contribute) to agile and non-agile software groups, I’m often struck by the same thought: where’s this person coming from?  This is basic engineering!

But that’s the matter, isn’t it?  Programming isn’t development without engineering and too many programmers aren’t engineers (not should they be) but are being told to "develop" without given the time or resources or something to do the engineering.  And so what they’re really being told to do is "program", not "develop".  Someone, somewhere doesn’t see and/or understand that what many projects need are to be engineered.

I think what this points to is a persistent phenomenon plaguing software: it’s not being taken seriously as an engineering discipline.  Sometimes by leadership in organizations where software is being worked on, sometimes by programmers and sometimes by customers.  I’m sure there’s plenty of blame to spread around, and spreading the blame is both a waste of time and not the point at all.

Programming is to software as assembly is to construction.  Not everyone swinging the hammer needs to be the civil engineer nor the architect, and not everyone with a nail gun can be the foreman (and no, I’m not likening the skill set of programmers to those of construction workers, and no, I’m not saying construction workers aren’t smart….geez).  There has to have been engineering taking place before software can be actually developed, and as evinced by the kinds of challenges I encounter regularly, enough software shops are going about their work absent acknowledgement or awareness or consideration for the engineering that has taken place or has yet to take place (or should have taken place but didn’t[!]).

Process stuff generally finds its roots in engineering.  Especially process stuff as found in CMMI for Development.  Excepting processes that are over-engineered, are themselves lacking in engineering, or are odious even alone in a room, I’m beginning to piece together that resistance to processes in general, and CMMI in particular, is actually from a lack of engineering discipline in the software practice and not from anything intrinsic to process as a topic.

It’s no wonder CMMI is so hard to use by so many, it assumes people are not only experts in process improvement, it also assumes everyone using it is an engineer.  Some people are nail-gun swingers, worried about getting enough done that day to avoid having to work on the weekend.  Meanwhile, someone else already worried about in what order to build piece the trusses together and someone before that worried about the right number of trusses and their thickness and someone before that worried about its shape.

It’s becoming fairly clear that anyone fooling themselves into believing that agile advocates not doing an architecture at all, or a design at all, or other engineering activities at all are doing themselves a disservice.  In fact, I’d go so far as to say that once an architecture has been settled upon and once a design becomes clear, that agile practices can happen more freely and effectively.  More so, I’d assert that the future of agile "scalability" depends on these.

What I and a colleague are setting out to do over the next few months is help agile scale by re-introducing engineering to software, and while we can’t fix the software universe, we hope to help agile out by giving it some engineering practices that software (as a whole) lacks — not everywhere, just in too many corners — but that we believe agile can really take and run with.

Let me know if you want to play with us.