Say you’re in a truly disciplined, lean and agile operation and your processes are so deeply ingrained in what you do that putting your finger on tangible evidence is a challenge, and not for lack of process. Just lack of being able to step back far enough from the canvas to see the whole picture. What do you when it comes to demonstrating your practices for a CMMI appraisal, for example?
Well… the best advice I can give companies in such situations is to work early and closely with a consultant and/or lead appraiser to elicit the best evidence for the appraisal long before the appraisal event itself is planned or carried out. It’s important to be clear about what the evidence is, and, you want the appraiser and the appraisal team on board with how the evidence will “show up”. This is not something you want to surprise anyone with come appraisal time.
Working early and closely with a lead appraiser will not only help everyone understand the context, and not only will it provide an opportunity to strengthen practices and identify operational risks, but it will give you a good idea about whether or not the lead appraiser has the wherewithal to think broadly about practices and to assemble the contextual picture for how practices would “show up” in the context of your operation.
Sadly, not all appraisers have this skill set. In fact, in my experience, the great majority do not have the skills to make contextually relevant model interpretation such that actual, naturally-occurring evidence from an operation can take its most natural form and still be recognized as implementation of CMMI practices. In my experience, most lead appraisers expect evidence to come in very specific shapes, sizes, and colors and they don’t recognize the evidence when it doesn’t meet their pre-conceived notions of what particular evidence should look like.
That being said, this does not give carte blanche for not having evidence. I’m not saying that the evidence isn’t there, I’m just saying that the evidence may not be what’s traditionally thought-of as evidence from larger or more traditional development operations.
Process evidence from operations whose processes are deeply ingrained can often show up as very clear, obvious artifacts. Especially from traditional development operations. However, in small, lean, and agile operations, the evidence can be much less obvious. It is a special skill set to be able to recognize the outputs of such operations as evidence of CMMI practices and organizations are served well to work with the lead appraiser early to determine whether or not their operation produces evidence as well as whether or not the appraiser can see more broadly than the evidence they’re used to seeing from traditional operations.
Since few organizations know how to pick a lead appraiser, perhaps this “litmus test” for a lead appraiser can serve to help them through the process. The alternative could be a disastrous paper-chase to create evidence on top of the evidence that’s already there.
So much about software development (in particular, and product development in general) as a business has less to do with technology than it has to do with keeping customers happy. What do customers really care about? While they say they want their product on time, on budget and doing what they asked of it to do, most of the time, managing their expectations has little to do with time, cost, features, functions, or quality. What they experience is more about how the developer treats them as a customer. In other words, what they perceive as the developer’s business as a service is what customers react to.
Of course, customers aren’t typically happy when the product is late, doesn’t do what they need it to do, and/or costs more than they were expecting to pay – scope creep notwithstanding. Be that as it may, agile development and management practices recognize the importance of customer involvement (and all stakeholders, in general). In fact, while the “traditional” development and management world has long espoused the importance of an integrated team for product and process development, it’s the agile development and management movement that actually made it work more smoothly with more regularity.
(Before anyone from the “traditional” development camp jumps down my throat, keep in mind: I came from the traditional camp first and saw attempts at IPPD and saw how difficult it was to get it going, keep it working, and eliminate the competition and other organizational stress that IPPD continues to experience in the traditional market. And, I’m also not saying it doesn’t work in traditional settings, just that it worked much better, much faster, and with much more regularity in the agile settings.)
From the beginning, agile practices understood the importance of the customer and of being a service to the customer. Kanban (more recently) even refers to different types of work as “classes of service”. In fact, if we look at the most common pains in development work (e.g., staffing, time, agreement on priorities and expectations), we see that it’s seldom technology or engineering issues. They’re issues more aligned with the developers’ abilities to provide their services.
[NOTE: For the remainder of this post, I’m going to assume the development operation actually knows its technology and knows what real engineering development looks like. This is a big assumption, because we all know that there are development operations a-plenty whose technical and engineering acumen leave much to be desired.]
Let’s now look at another importance facet of all development, agile notwithstanding. Much of it happens after the initial product is released! Once the product is released, there is precious little actual development going on. The ongoing support of the product includes enhancements and other updates, but very little of that work requires any engineering! Furthermore, what is worked-on comes in through a flow of requests, fixes, and other (very-often unrelated) tasks.
After a product has been released, the operation of a development shop resembles a high-end restaurant far more closely than it appears as a production floor. Once the menu has been “developed”, from that point forward, patrons merely ask for items from the menu and for modifications to items on the menu. Even were there to be a “special order” of something not at all on the menu, the amount of “development” necessary to "serve” it is minimal. And, when something truly off-the-wall is requested, the chef knows enough to respond with an appropriately apologetic, “Sorry, we can’t make that for you right now. Please let us know in advance and we’d be happy to work something up for you.” At which point, they would set about developing the new product.
Meanwhile, the vast majority of the work is actually just plugging away at the service. In the service context, development is often not the majority of the work. In that context, engineering plays an important role much less often than the ability to deliver services, manage transition of services, ensure continuity of service, handle incidents, manage resources, and so on.
What does this mean for agile teams, and, what does this have to do with CMMI?
Well, maybe much of the perceived incompatibility between CMMI (for Development) and agile practices are not due to incompatibilities in CMMI and agile, but incompatibilities in the business of agile and the improvement of development. In other words, maybe the perceived incompatibilities between CMMI and agile are because CMMI for Development (CMMI-DEV) is meant to improve development and many agile teams aren’t doing as much development as they are providing a service. Perhaps it’s just that the business models presumed by the two approaches are not aimed at making progress in the same way.
When agile teams are doing actual development, CMMI-DEV should work well and can help improve their development activities. But, agile teams are often not doing development as much as they are providing a service. They establish themselves and operate as service providers. Most of the agile approaches to development are far more aptly modeled as services.
CMMI for Services defines services as follows*:
*Glossary CMMI® for Services, Version 1.3, CMMI-SVC, V1.3, CMU/SEI-2010-TR-034
Many requests made of many agile teams have more to do with supporting the product than developing a product. While the product is still under development, then, by all means, CMMI for Development is apropos. But after the initial development (where more product-oriented money is spent), the development is hard to see and harder to pin down.
Maybe, improving development is not the right thing to develop. Perhaps agile teams could look at how they handle “development as a service” for their improvement targets. Maybe CMMI for Services is a much better fit for agile teams.
Could a switch from CMMI-DEV to CMMI-SVC benefit agile teams? Could a switch from CMMI-DEV to CMMI-SVC make achieving CMMI ratings easier and more meaningful?
I believe the answer to both is a resounding: ABSOLUTELY!
ATTENTION AGILE TEAMS: You need a CMMI rating? Look at CMMI for Services. It might just make your lives easier and actually deliver more value right now!
[NOTE: I have an essay, Are Services Agile?, in this book on this topic. Since you can “look inside” you might be able to read it without buying it. Furthermore, the essay has been published online in some places. You might be able to find it out there.]