Say you’re in a truly disciplined, lean and agile operation and your processes are so deeply ingrained in what you do that putting your finger on tangible evidence is a challenge, and not for lack of process.  Just lack of being able to step back far enough from the canvas to see the whole picture.  What do you when it comes to demonstrating your practices for a CMMI appraisal, for example?

Well… the best advice I can give companies in such situations is to work early and closely with a consultant and/or lead appraiser to elicit the best evidence for the appraisal long before the appraisal event itself is planned or carried out.  It’s important to be clear about what the evidence is, and, you want the appraiser and the appraisal team on board with how the evidence will “show up”.  This is not something you want to surprise anyone with come appraisal time.

Working early and closely with a lead appraiser will not only help everyone understand the context, and not only will it provide an opportunity to strengthen practices and identify operational risks, but it will give you a good idea about whether or not the lead appraiser has the wherewithal to think broadly about practices and to assemble the contextual picture for how practices would “show up” in the context of your operation.

Sadly, not all appraisers have this skill set.  In fact, in my experience, the great majority do not have the skills to make contextually relevant model interpretation such that actual, naturally-occurring evidence from an operation can take its most natural form and still be recognized as implementation of CMMI practices.  In my experience, most lead appraisers expect evidence to come in very specific shapes, sizes, and colors and they don’t recognize the evidence when it doesn’t meet their pre-conceived notions of what particular evidence should look like. 

That being said, this does not give carte blanche for not having evidence.  I’m not saying that the evidence isn’t there, I’m just saying that the evidence may not be what’s traditionally thought-of as evidence from larger or more traditional development operations.

Process evidence from operations whose processes are deeply ingrained can often show up as very clear, obvious artifacts.  Especially from traditional development operations.  However, in small, lean, and agile operations, the evidence can be much less obvious.  It is a special skill set to be able to recognize the outputs of such operations as evidence of CMMI practices and organizations are served well to work with the lead appraiser early to determine whether or not their operation produces evidence as well as whether or not the appraiser can see more broadly than the evidence they’re used to seeing from traditional operations.

Since few organizations know how to pick a lead appraiser, perhaps this “litmus test” for a lead appraiser can serve to help them through the process.  The alternative could be a disastrous paper-chase to create evidence on top of the evidence that’s already there.