I’m not sure, but I’m told some famous guy back in Biblical liturgy was once asked to explain the point of the Pentateuch (aka, the Torah, aka, The Five Books of Moses) while "standing on one leg".  

I now undertake a task, possibly no less daunting, regarding CMMI.  And, if there ever were anyone more appropriate to try it, I doubt I’ve met them.

Seriously though, much has been written here and many other places (not to mention eons of conference and user group content) about a number of "universal truths" about CMMI.  Let’s get these out there first, but without dwelling on them:

• There are no "processes" in CMMI, only practices, and there’s a difference.
• The practices in CMMI are "what" but not "how".
• These practices are use to improve your processes, not to define them.
• The CMMI does not require the SCAMPI appraisal to be effective.  You can use CMMI to improve your operation without ever using the SCAMPI to appraise your use of CMMI.
• 42.  OK.  Not really.

However, not a single one of these "truths" explain the point of CMMI, or,  how to actually use CMMI.  So, here it goes:

Each one of the practices in CMMI improves some aspect of your organization’s performance resulting from how you do your work.  It doesn’t matter whether it’s providing a service or developing a product.  And, it doesn’t matter whether you do so using so-called traditional development methods or Agile approaches.  If you have performance issues in an area of your operation (called, "Process Areas" in CMMI), Check each of the practices in that area for activities in your operation that might be causing those performance issues. 

It’s assumed, then, if you don’t have any issues covered by a practice then you don’t need to do anything about a practice, because you’re already doing it.  This says nothing of how well you do it, why you do it, how you do it, whether you recognize that you do it, or whether the fact that you do it is a complete coincidental freak of nature, but, if you read a practice, understand the risk it avoids, and you don’t encounter that risk, you’re somehow performing that practice.  Pretty simple.

I’ll repeat and summarize that two-step thought experiment:

1. Look in the process areas for practices that address performance issues you’re experiencing with the operation of your work.  When you encounter a practice (or more than one), the absence of which can explain why you’re seeing those issues, make appropriate changes to your operation so that you incorporate that/those practice(s) into your operation.  Rinse and repeat.
2. Practices that don’t represent risks or issues you’re not seeing are (pretty much, by definition) practices you’re somehow managing to accomplish.  Don’t bother with them — unless you notice that you don’t like something about how you do it, but that’s a different priority/matter.

Keep in mind, this says nothing of

• whether what you do/don’t do will suffice as "evidence" for an appraisal
• how well you perform the practices (regardless of whether or not you perform them or believe you can use them to improve),
• what it takes to incorporate practices or make change, in general, happen in your operation,
• whether an appraisal team will concur with whether you do/don’t perform practices, or
• you interpret practices in constructive ways.

Nonetheless, if you internalize the significance of the above 2 steps, you can (I dare say, "will") save yourselves a lot of time and grief when using CMMI.  This approach can certainly help you prioritize the practices for which to focus on, appraisal or not.  And, if you do take this approach towards preparation for an appraisal, keep in mind the bulleted caveats and don’t try this alone.

Call for papers to the Cutter IT Journal for which I’m the edition’s guest editor.

Topic: Agile and CMMI: Why Isn’t this Conversation Dead Yet?

Proposals/abstracts due 17 August 2012, so act quickly!

See Cutter’s blog and web site for more information, including specific areas of interest, editorial guidelines and how to submit your proposal/abstract.

(If you reference CMMI experience, please ensure that your experience with CMMI is from using v1.2 or later.)

OK, OK, so it’s Mothers’ Day in the US… but we all miss out on family matters all the time for things far less awesome than this.

Lean Camp New England is a one day open space event led by Jim Benson, author of Personal Kanban. It is an opportunity to share and learn about Lean and Kanban in software development, IT operations and services, and other knowledge work fields.

Lean Camp New England is an all day event at the World Trade Center Boston, on May 13th (Sunday).
Registration is open now at a cost of $300 – catering is included.
Register at http://lssc12.leanssc.org/.

Boston is the center of gravity for lean thinking in the US. However, much of that thinking has been in fields outside of IT and Software and Systems Engineering/Development.

If you live in New England, or people you know live in New England, and are interested in getting up-to-speed among the leading thinkers and practitioners in Lean in IT and Software and Systems Engineering/Development, I highly encourage you and them to register for Lean Camp New England.

This is a rare opportunity to enjoy a regional 1-day open space / unconference in conjunction with a large international conference, Lean Software & Systems 2012 – as a result a significant number of international experts will be present and participating.

Where else can you get direct coaching from the experts for only $300?

For more than the last 10 years I’ve been thinking a lot about CMMI.  Many of these thoughts have been ruminating on the ideas of how to incorporate CMMI in ways that add value, demonstrate effectiveness, and don’t disrupt the operation.  I’ve even opined much in this blog (in too many ways) on the need to know what your processes are before you can use CMMI to improve them, and that for many operations, CMMI isn’t even appropriate.

A few recent discussions and experiences put a particularly fine point on the extent to which CMMI is really the ‘cart before the horse’ when applied within an operation that has yet to clearly discern its process, and here’s why:

Most operations I’ve encountered are not ready to use CMMI because they are unclear on exactly how they make money.

Obviously, I’m not talking about the accounting process of billing out invoices and depositing checks.  And, I’m not even talking about the voodoo around figuring out how to ensure that internal costs (salaries, equipment, etc.) are less than what they charge clients for the work they do.

So, I must be talking about something more subtle.  I wish I were.  And, this is what’s both frightening and sad.  I’m merely talking about the relationship between capacity and demand.  For that matter, I’m not even worried much about demand, which is another matter.  I’m mostly talking about capacity.

What is capacity?

According to some, capacity is a measure of volume of work.  Throughput, for instance.  According to others, it’s the wherewithal to do the work.  Either way, too many operations don’t know what their capacity is. 

What does it actually take to get work done?  And, along with that, can it be reasonably expected of the operation to reliably and predictably continue to run how it runs (not knowing exactly how it runs) and to have any right to greater-than-zero confidence that they will continue to run as it does?

For one thing, many operations run outside of reasonable tolerances. In particular, people put in many many hours of unpaid over-time [in the US this is common for salaried employees].  This is an "out-of-tolerance" condition.  It is unreasonable to expect an operation’s greatest source of working knowledge to continue to work nights and weekends.  Furthermore, it is risky to do so.  One client said he couldn’t be away from the office for 5 minutes before product would stop shipping.  Eventually he will get married or his wife will have a child, or, heaven-forbid, he may take vacation! 

What’s worse is the extra time he and his team put in is entirely unaccounted-for.  His employer merely estimates, contracts, and bills enough to cover his and the team’s salaries, not what it actually takes to get work done.

Another reason true capacity is obscured is because more work going into the operation than there’s product (or services) coming out.  The most common cause of this is the misperception that work started = work completed, but this is an incomplete equation.  But a better equation to work with is work worked-on = work completed.  The key mistakes is the assumption that started = in-work.  That’s true for maybe 50% of the actual started work (often less).

This next reason for the lack of insight into true capacity (or capability, really) is that so many operations don’t account (either in their estimates — which sort-of makes sense — or in their capture of time-spent — which is unforgivable) for the time to correct defects, time to perform rework, time for paying-down technical debt, or time and effort to tracking-down the causes of defects and rework to avoid defects, rework, and technical debt in the first place!

I’ll end with one of the toughest, most sensitive observations of the last few months.  Some of my best clients (from the perspective of having their act together) have strong confidence in functional competence and, admittedly, weaker confidence in their programmatic credibility.  And, to put it plainly, by "programmatic" I mean their ability to have the same confidence in the rationale for their estimates and plans as they have in their ability to produce what their clients want. 

In these operations, I’ve found fundamental disconnects between how work is estimated and why clients should trust the technical competence of the operation.  In other words, they build trust with their prospects and clients on their ability to do the work and build the products, but in order to get the work in the first place they have to use a lot of hand-waving and breath-holding when it comes to their estimates.

A more-or-less summary way to describe all of this is as follows:

Most operations have Built a way of working that they’ve managed to Capitalize.  Over time, they’ve found that their Build*Capitalize approach is tough to Sustain.  Try that they might, whether it’s CMMI or something else, they’re looking to "fix" the equation on the "Sustain" side.  The problem is that CMMI *does* operate on the Sustain side, but the problems with the operation aren’t in the "Sustain", it’s that their approach to Capitalizing on what they Built is no longer Sustainable.  What needs to change is on the Build side.  Occasionally, there’s a need to revisit the Capitalize component, but most often it’s the Build that needs refactoring.

Hence, applying CMMI to an operation whose "build" is broken is putting the cart before the horse.  While it’s possible to build CMMI practices into the operation’s way of working, this is an activity of the "build" side of the equation, the sort I noted above contrasting from applying CMMI to the sustain side.  If CMMI is to be truly about improving the processes of the operation in a "sustaining" sort of way, and not defining them, the operation must understand what’s going on, and that means it must know its capacity.  Because unless it knows its capacity, it doesn’t really know what’s going on.

For more than the last 10 years I’ve been thinking a lot about CMMI. Many of these thoughts have been ruminating on the ideas of how to incorporate CMMI in ways that add value, demonstrate effectiveness, and don’t disrupt the operation. I’ve even opined much in this blog (in too many ways) on the need to know what your processes are before you can use CMMI to improve them, and that for many operations, CMMI isn’t even appropriate.

A few recent discussions and experiences put a particularly fine point on the extent to which CMMI is really the ‘cart before the horse’ when applied within an operation that has yet to clearly discern its process, and here’s why:

Most operations I’ve encountered are not ready to use CMMI because they are unclear on exactly how they make money.

Obviously, I’m not talking about the accounting process of billing out invoices and depositing checks. And, I’m not even talking about the voodoo around figuring out how to ensure that internal costs (salaries, equipment, etc.) are less than what they charge clients for the work they do.

So, I must be talking about something more subtle. I wish I were. And, this is what’s both frightening and sad. I’m merely talking about the relationship between capacity and demand. For that matter, I’m not even worried much about demand, which is another matter. I’m mostly talking about capacity.

What is capacity?

According to some, capacity is a measure of volume of work. Throughput, for instance. According to others, it’s the wherewithal to do the work. Either way, too many operations don’t know what their capacity is.

What does it actually take to get work done? And, along with that, can it be reasonably expected of the operation to reliably and predictably continue to run how it runs (not knowing exactly how it runs) and to have any right to greater-than-zero confidence that they will continue to run as it does?

For one thing, many operations run outside of reasonable tolerances. In particular, people put in many many hours of unpaid over-time [in the US this is common for salaried employees]. This is an “out-of-tolerance” condition. It is unreasonable to expect an operation’s greatest source of working knowledge to continue to work nights and weekends. Furthermore, it is risky to do so. One client said he couldn’t be away from the office for 5 minutes before product would stop shipping. Eventually he will get married or his wife will have a child, or, heaven-forbid, he may take vacation!

What’s worse is the extra time he and his team put in is entirely unaccounted-for. His employer merely estimates, contracts, and bills enough to cover his and the team’s salaries, not what it actually takes to get work done.

Another reason true capacity is obscured is because more work going into the operation than there’s product (or services) coming out. The most common cause of this is the misperception that work started = work completed, but this is an incomplete equation. But a better equation to work with is work worked-on = work completed. The key mistakes is the assumption that started = in-work. That’s true for maybe 50% of the actual started work (often less).

This next reason for the lack of insight into true capacity (or capability, really) is that so many operations don’t account (either in their estimates — which sort-of makes sense — or in their capture of time-spent — which is unforgivable) for the time to correct defects, time to perform rework, time for paying-down technical debt, or time and effort to tracking-down the causes of defects and rework to avoid defects, rework, and technical debt in the first place!

I’ll end with one of the toughest, most sensitive observations of the last few months. Some of my best clients (from the perspective of having their act together) have strong confidence in functional competence and, admittedly, weaker confidence in their programmatic credibility. And, to put it plainly, by “programmatic” I mean their ability to have the same confidence in the rationale for their estimates and plans as they have in their ability to produce what their clients want.

In these operations, I’ve found fundamental disconnects between how work is estimated and why clients should trust the technical competence of the operation. In other words, they build trust with their prospects and clients on their ability to do the work and build the products, but in order to get the work in the first place they have to use a lot of hand-waving and breath-holding when it comes to their estimates.

A more-or-less summary way to describe all of this is as follows:

Most operations have Built a way of working that they’ve managed to Capitalize. Over time, they’ve found that their Build*Capitalize approach is tough to Sustain. Try that they might, whether it’s CMMI or something else, they’re looking to “fix” the equation on the “Sustain” side. The problem is that CMMI *does* operate on the Sustain side, but the problems with the operation aren’t in the “Sustain”, it’s that their approach to Capitalizing on what they Built is no longer Sustainable. What needs to change is on the Build side. Occasionally, there’s a need to revisit the Capitalize component, but most often it’s the Build that needs refactoring.

Hence, applying CMMI to an operation whose “build” is broken is putting the cart before the horse. While it’s possible to build CMMI practices into the operation’s way of working, this is an activity of the “build” side of the equation, the sort I noted above contrasting from applying CMMI to the sustain side. If CMMI is to be truly about improving the processes of the operation in a “sustaining” sort of way, and not defining them, the operation must understand what’s going on, and that means it must know its capacity. Because unless it knows its capacity, it doesn’t really know what’s going on.

So much about software development (in particular, and product development in general) as a business has less to do with technology than it has to do with keeping customers happy.  What do customers really care about?  While they say they want their product on time, on budget and doing what they asked of it to do, most of the time, managing their expectations has little to do with time, cost, features, functions, or quality.  What they experience is more about how the developer treats them as a customer.  In other words, what they perceive as the developer’s business as a service is what customers react to.

Of course, customers aren’t typically happy when the product is late, doesn’t do what they need it to do, and/or costs more than they were expecting to pay – scope creep notwithstanding.  Be that as it may, agile development and management practices recognize the importance of customer involvement (and all stakeholders, in general).  In fact, while the “traditional” development and management world has long espoused the importance of an integrated team for product and process development, it’s the agile development and management movement that actually made it work more smoothly with more regularity.

(Before anyone from the “traditional” development camp jumps down my throat, keep in mind: I came from the traditional camp first and saw attempts at IPPD and saw how difficult it was to get it going, keep it working, and eliminate the competition and other organizational stress that IPPD continues to experience in the traditional market.  And, I’m also not saying it doesn’t work in traditional settings, just that it worked much better, much faster, and with much more regularity in the agile settings.)

From the beginning, agile practices understood the importance of the customer and of being a service to the customer.  Kanban (more recently) even refers to different types of work as “classes of service”.  In fact, if we look at the most common pains in development work (e.g., staffing, time, agreement on priorities and expectations), we see that it’s seldom technology or engineering issues.  They’re issues more aligned with the developers’ abilities to provide their services.

Let’s now look at another importance facet of all development, agile notwithstanding.  Much of it happens after the initial product is released!  Once the product is released, there is precious little actual development going on.  The ongoing support of the product includes enhancements and other updates, but very little of that work requires any engineering!  Furthermore, what is worked-on comes in through a flow of requests, fixes, and other (very-often unrelated) tasks. 

After a product has been released, the operation of a development shop resembles a high-end restaurant far more closely than it appears as a production floor.  Once the menu has been “developed”, from that point forward, patrons merely ask for items from the menu and for modifications to items on the menu.  Even were there to be a “special order” of something not at all on the menu, the amount of “development” necessary to "serve” it is minimal.  And, when something truly off-the-wall is requested, the chef knows enough to respond with an appropriately apologetic, “Sorry, we can’t make that for you right now.  Please let us know in advance and we’d be happy to work something up for you.”  At which point, they would set about developing the new product.

Meanwhile, the vast majority of the work is actually just plugging away at the service.  In the service context, development is often not the majority of the work.  In that context, engineering plays an important role much less often than the ability to deliver services, manage transition of services, ensure continuity of service, handle incidents, manage resources, and so on.

What does this mean for agile teams, and, what does this have to do with CMMI?

Well, maybe much of the perceived incompatibility between CMMI (for Development) and agile practices are not due to incompatibilities in CMMI and agile, but incompatibilities in the business of agile and the improvement of development.  In other words, maybe the perceived incompatibilities between CMMI and agile are because CMMI for Development (CMMI-DEV) is meant to improve development and many agile teams aren’t doing as much development as they are providing a service.  Perhaps it’s just that the business models presumed by the two approaches are not aimed at making progress in the same way.

When agile teams are doing actual development, CMMI-DEV should work well and can help improve their development activities.  But, agile teams are often not doing development as much as they are providing a service.  They establish themselves and operate as service providers.  Most of the agile approaches to development are far more aptly modeled as services.

CMMI for Services defines services as follows*:

• A product that is intangible and non-storable.
• Services are delivered through the use of service systems that have been designed to satisfy service requirements.
• Many service providers deliver combinations of services and goods. A single service system can deliver both types of products.
• Services may be delivered through combinations of manual and automated processes.

*Glossary CMMI® for Services, Version 1.3, CMMI-SVC, V1.3, CMU/SEI-2010-TR-034

Many requests made of many agile teams have more to do with supporting the product than developing a product.  While the product is still under development, then, by all means, CMMI for Development is apropos.  But after the initial development (where more product-oriented money is spent), the development is hard to see and harder to pin down.

Maybe, improving development is not the right thing to develop.  Perhaps agile teams could look at how they handle “development as a service” for their improvement targets.  Maybe CMMI for Services is a much better fit for agile teams. 

Could a switch from CMMI-DEV to CMMI-SVC benefit agile teams?  Could a switch from CMMI-DEV to CMMI-SVC make achieving CMMI ratings easier and more meaningful?

I believe the answer to both is a resounding: ABSOLUTELY!

ATTENTION AGILE TEAMS: You need a CMMI rating?  Look at CMMI for Services.  It might just make your lives easier and actually deliver more value right now!

[NOTE: I have an essay, Are Services Agile?, in this book on this topic.  Since you can “look inside” you might be able to read it without buying it.  Furthermore, the essay has been published online in some places.  You might be able to find it out there.]

I’m speaking @ the Lean Software and Systems Conference 2011.

The program is amazing!

I highly encourage attendance.

There’s an entire day in cooperation with the SEI with 3 unique tracks on it including a track on CMMI and Multi-Modal Processes (which I’m chairing).

Take a look at my talk… it’s from my upcoming book: High Performance Operations.

Register quickly and make your hotel reservations! Block rooms are nearly gone!

I’ve been given several opportunities lately to be thinking about the relationship among product development, agility, and services.  In a recent conversation regarding (of all things) how to sample work for artifacts in a CMMI for Services appraisal, it became clear that taking a services view of development actually makes a lot of things more obvious when it comes to where and how to make performance improvements.

Furthermore, the idea that product development can be modeled as the organization of particular services – such that the culmination of all the services results in a product – not only enhances the understanding and performance of the development flow, but it also creates a strong affinity to agile management and development values, principles and practices.  In fact, a service-oriented development flow is how Kanban views and manages development, and even shares many parallels with traditional services such as “cumulative” work and flow.  And, seeing development as a flow of services simplifies if not eliminates the endless catch-22 of dealing with planning, resource allocation and work volume.

In the video, I was at the tail end of a week-long exposure to a very demanding product development and services delivery context: aboard a pleasure cruise ship.  At this stage of our family’s development, pleasure cruising has emerged as our vacation of choice so this was my sixth cruise in over 10 years.  The first three cruises were with three different cruise line companies and the most recent three were with the same line.  What struck me most about the ship’s (and this cruise company’s) operations were its flexibility and responsiveness to change.

Despite many constraints, within those constraints the ship was autonomous, and, the various departments within the ship had degrees of autonomy.  Beyond autonomy, there were clear components run centrally and just as clearly there were components that were decentralized.  But it all worked as a single service: the ship.  Within nearly every service were products to be developed, whether produced from scratch or recreated afresh over and over again.  Yet again, the massive, highly complex service system operated in an agile way by nearly any measure of ‘agility’ in nearly every facet of how it ran.

A few days after my return from the ship I had the opportunity to teach Introduction to CMMI.  This offering was to one of my clients and a guest.  All participants were sharp and involved – which isn’t always the case with such classes.  The class was special in that I was experimenting with new course material for the SEI in which I was delivering content from the CMMI for Development constellation following content from the CMMI for Services constellation.  This experience reinforced for me and exposed the participants to the strong relationship between Services and Development, the strong benefits of viewing development as a service (from both operational and improvement perspectives), and, helped my client (who uses Scrum, Kanban, and traditional development in various parts throughout the company) see common threads to help improve performance irrespective of how they approach management and development.

The learning for agile and CMMI cooperation may very well be found in services.  Think about it.  Now, class, discuss.

Apple, Inc. learned the hard way what happens when engineering isn’t complete.  In particular, when verification and/or validation aren’t performed thoroughly.

Verification is ensuring that what you’re up to meets requirements.  “ON PAPER.”  BEFORE you commit to making the product.  It’s that part where you do some analysis to figure out whether what you think will work, will actually do what you expect it to do.  Such as, walking through an algorithm or an equation by hand to make sure the logic is right or that the math is right.  Or, stepping through some code to see what’s going on before you assume that it is behaving.  Just because something you built passes tests, doesn’t mean it is verified.  All passing tests means is just that: you passed tests.  Passing tests assumes the tests are correct.  If you’re going to rely on tests, then the tests need to be verified if you’re not going to verify the requirements or the design, etc.  Another problem with tests is that too many organizations only test at the end.  Verification looks a lot more like incremental testing.  Hey wait!  Where’ve we seen that sort of stuff before?

Had Apple’s verification efforts been more robust, they would have caught the algorithm error that incorrectly displays the signal strength (a.k.a., “number of bars”) on the iPhone4.  This is why peer review is so central to most verification steps.  The purpose of peer review, and of verification, is to catch defective thinking.  OK, that’s a bit crude and rude… it’s not that people’s thinking is defective, per se, but that thinking alone didn’t catch everything, which is why we like to have other people looking at our thinking.  Even Albert Einstein submitted his work for peer review.

Validation is ensuring the product will work as intended when placed in the users’ environments.  In other words, it’s as simple as asking, “when real users use our product, how will they use it, and will our product work like we/they expect it to work?”  Sometimes this is not something that can be done on paper, and you need some sort of “real” product, so you build a prototype.  Just as often it’s not something that can be done “for real” because you don’t get an opportunity (yet) to take your product into orbit before it has to go into orbit to work.  Sometimes you only get one shot, and so you do what you can to best approximate the real working environment.  But neither of these extreme conditions can be used by Apple as excuses for not validating whether or not the phone will work as expected while being held by the user to make calls.

Had Apple’s validation been operating on all bars, they likely would have caught this while in the lab.  When sitting in its sterile, padded vice, in some small anechoic chamber, after taking great care to ensure there are no unintended signals and nothing metallic touching the case, someone might’ve noticed, “gee, do you think our users might actually make calls this way?”  And, instead of responding, “that’s not what we’re testing here”, someone might’ve stepped up and said, “hey, does our test plan have anything in it where we’re running this test while someone’s actually using the phone?”

Again, testing isn’t enough.  Why not!?  After all, isn’t putting it in a lab with or without someone holding the phone a test?   True…  However, I go back to the same issue we saw when using testing as the primary means of performing verification… Testing is too often at the end.  Validating at the end is too late.  You need to validate along the way.  In fact, it’s entirely possible that Apple *did* do validation “tests” of the case separately from the complete system, and, in *those* tests — where the case/antenna were mere components being tested in the lab — performed fine, and, then only when the unit was assembled and tested as a complete system would the issue have been found.  In such a scenario we learn that component (elsewhere known as “unit testing”) is not enough.  We also need system testing (in the lab) and user testing (in real life).  Back we go to iterative and incremental…

So you see… we have a lot we can apply from ordinary engineering, from agile, and from performance improvement.  Not only does this… uh… validate(?) that “agile” and “CMMI” can work together but that for some situations, others can learn from applying both.

In full disclosure, as a new owner of an iPhone 4, I am very pleased with the device.  I can really see why people love it and become devotees of Apple’s products.  Honestly, it kicks the snot out of my prior “smart” phone in every measurable and qualitative way.  And, just so I’m not leaving anything out, the two devices are pretty much equally balanced in functionality (web, email, social, wifi, etc.)  – even with the strange behaviors that are promised to be fixed.  For a few years, this iPhone will rule the market and I’ll be happy to use it.

Besides embarrassing, this will be an expensive couple of engineering oversights for Apple to fix.  And, they were entirely avoidable for an up-front investment in engineering at an infinitesimal fraction of the cost/time it will take to fix.  For even less than one day of their engineering and deployment team’s salary, AgileCMMI can make this never happen again.

Apple, look me up.  I’m easy to find.