Say you’re in a truly disciplined, lean and agile operation and your processes are so deeply ingrained in what you do that putting your finger on tangible evidence is a challenge, and not for lack of process.  Just lack of being able to step back far enough from the canvas to see the whole picture.  What do you when it comes to demonstrating your practices for a CMMI appraisal, for example?

Well… the best advice I can give companies in such situations is to work early and closely with a consultant and/or lead appraiser to elicit the best evidence for the appraisal long before the appraisal event itself is planned or carried out.  It’s important to be clear about what the evidence is, and, you want the appraiser and the appraisal team on board with how the evidence will “show up”.  This is not something you want to surprise anyone with come appraisal time.

Working early and closely with a lead appraiser will not only help everyone understand the context, and not only will it provide an opportunity to strengthen practices and identify operational risks, but it will give you a good idea about whether or not the lead appraiser has the wherewithal to think broadly about practices and to assemble the contextual picture for how practices would “show up” in the context of your operation.

Sadly, not all appraisers have this skill set.  In fact, in my experience, the great majority do not have the skills to make contextually relevant model interpretation such that actual, naturally-occurring evidence from an operation can take its most natural form and still be recognized as implementation of CMMI practices.  In my experience, most lead appraisers expect evidence to come in very specific shapes, sizes, and colors and they don’t recognize the evidence when it doesn’t meet their pre-conceived notions of what particular evidence should look like. 

That being said, this does not give carte blanche for not having evidence.  I’m not saying that the evidence isn’t there, I’m just saying that the evidence may not be what’s traditionally thought-of as evidence from larger or more traditional development operations.

Process evidence from operations whose processes are deeply ingrained can often show up as very clear, obvious artifacts.  Especially from traditional development operations.  However, in small, lean, and agile operations, the evidence can be much less obvious.  It is a special skill set to be able to recognize the outputs of such operations as evidence of CMMI practices and organizations are served well to work with the lead appraiser early to determine whether or not their operation produces evidence as well as whether or not the appraiser can see more broadly than the evidence they’re used to seeing from traditional operations.

Since few organizations know how to pick a lead appraiser, perhaps this “litmus test” for a lead appraiser can serve to help them through the process.  The alternative could be a disastrous paper-chase to create evidence on top of the evidence that’s already there.

Imagine what would happen if you went to a doctor (or any specialist) who had no experience in your specificcondition or situation.  Has this every happened to you?  It has to my family when I was young.  Let me tell you, it wasn’t pleasant.  What was frightening was that the “professional” didn’t know that they didn’t have the right experience.  What was just as bad was that my family didn’t have the knowledge or experience to know that the person we went to was not qualified.

This is a situation encountered by many organizations when seeking advice and/or appraisal services from a CMMI consultant / appraiser.  However, in business, you should at least know enough about your organization and ways of operating to do your homework before picking someone to help you with CMMI.

What you may not have known is that CMMI and the appraisal method are not as clear and obvious as other means of performance evaluation and that you must choose your consulting firm and appraiser very carefully, and among other factors, consider their contextually relevant experience. . . .

In this quote, CAPT Kirk wants Dr. Bones McCoy to do something he feels he’s not-qualified to do because he doesn’t know how to treat the species.

I’m using it to explain that organizations looking for a lead appraiser to work with them towards an appraisal and/or to perform an appraisal ought to think of what we do as they would think of a doctor, not a laborer or vendor.

Do you really want the lowest price doctor?

For that matter, is the highest price doctor necessarily the best in town?

When reaching out and interviewing for a lead appraiser or CMMI consultant, you:

• Want the person who is the right person for the job.
• Want someone who is qualified (definitely not under-, but preferably not over- either).
• Not the lowest bid.

Seriously, whoever you hire for this effort has in their power the ability to make or break your future.  They literally have the health and well-being of your organization in their hands.  They can put you in the dump just as easily as they can take you to the next level.

They should see themselves that way as well.

Unfortunately I’ve got too many sad stories of appraisers/consultants who definitely see that they can make or break you, but they don’t feel like they personally own the responsibility for what happens to you when they’re done.

If it costs too much?  So what?
If you get no value?  Not their problem.
Didn’t see any benefit?  Didn’t learn anything?  Things take longer and cost more and you’re not seeing internal efficiencies improve?
YOU must be doing something wrong, not them.

In an AgileCMMI approach, your CMMI consultant and/or lead appraiser would see themselves as and act like a coach, and would put lean processes and business value ahead of anything else.  And, an AgileCMMI approach would know that when the processes work, they add value; when they add value people like them and use them; when people like and use them, the next “level” is a big no-brainer-nothing.  You get it in your sleep.

Let me know if you want help finding the right lead appraiser or consultant.

A recent post to the Agile Thoughts blog caused me to have a serious case of déjà vu. 

First, I will start by saying that I’m not going to take a position on the content of the post.  Namely, I’m not going to weigh in on whether or not Scrum is valid, whether or not Mary Poppendieck’s points or approach are appropriate.

The purpose of this post is to make a suggestion.

Go ahead and (re)read that post. 

Replace

• "Scrum" with "CMMI",
• "CSM" or "Scrum Master" with "Lead Appraiser", and
• "Lean" with "Agile". 

My favorite line in the entire post is this one:

"… spent 90% of her time cleaning up after bad Scrum implementations…"

And an associated comment that noted:

"…the difference between the good and the bad ones depends mainly on who’s doing it…"

I don’t feel like taking the time right now to ponder what it means (I’ll probably do it anyway after posting), but what I find fascinating is that people are now debating various agile/lean concepts in the way the debate continues to fester about CMMI/agile.  And, those in the agile/lean debate are recognizing that it’s not enough to have a named method or model, and it’s not enough to be "certified" to do something to really "get it", but that there is real need for understanding the underlying concepts and intentions and for implementing from that basis otherwise there is risk of "bad implementations".

What every perspective in these discussions is (hopefully) saying is that there is no one "silver bullet".  That addressing the issue of great products, ecstatic customers and happy teams requires more than superficial application of someone else’s ideas.  Requires more than one set of principles, when hiring an "expert" requires serious due diligence and interviewing skills, and requires a lot of hard work and soul-searching to reach the "comfort zone" of every project and team.

Again, I’m not pointing fingers and I don’t want to accuse one person of saying something they’re not, nor do I want to label an entire field of people with any one person’s perspective.  With that said, the following is drawn from my own experience and I’m merely reminded of it thanks to Tobias Mayer’s post.

Many people now finding themselves defending Scrum — against bad implementations and other abuses — are saying that it’s not anything inherent in Scrum that’s bad.  My guess is that many of these people are (or were) also among those who vilify (vilified?) CMMI.  Accusing CMMI of evils that were perpetrated by too many goobers inappropriately implementing and appraising it.  Vilifying CMMI (can be read: Scrum) by juxtaposing implementation with content.  These evils are just as much not CMMI’s "fault" as bad Scrum implementations are Scrum’s "fault".

In fact, our recent SEI Technical Note, spoke to this very issue.  I guess the point to this post is to say to those folks in the Scrum and Lean communities: Welcome Aboard!  Let’s start some constructive discussion together on defeating "silver-bullet-ism" in software development.

Well, I’d like to say it was "nothing", but really, I was sweating bullets!

Since I consider readers of this blog to be among my more friendly professional acquaintances, I thought I’d let you in on something that has not yet been released to the general public.

See here for an as-yet to be released announcement article/press-release-style about my becoming an SEI-Certified High Maturity Lead Appraiser.

The actual release might change slightly as the PR folks work on it, but you’ll get the gist.

One fun thing that had to be left on the editing floor was a quote from a technical member of one of my client’s staff when he asked, "… how did you get into this field?  I mean, you actually have a personality!"

The techie was worried I might be offended, but I thought it was pretty funny, and so did PR… but it didn’t make the cut for the article.

Here’s some insight into what it was like:

The oral exam gives high marks for noting very specific terms, and is structured in some way (like the CMMI model itself) that lends itself to recursiveness (not exactly redundancy).  Towards the last half hour, my seasonal cold, my lack of sleep, and my indigestion had all caught-up with me.  My brain was empty, I couldn’t tell what I’d said previously and what I still didn’t say and I couldn’t pull certain ideas from my brain to save my life.

Last impressions are often just as memorable as first impressions.  In my case, I recall that I nailed the earlier parts of the exam to the wall.  Not just hitting the target but obliterating it.  But when the end drew near, I felt like I’d lost that clarity of thought, that I was going in circles.  Not knowing when I’d said enough or when I was digging my own grave, I started searching for ideas on the blank neutral hotel walls hoping my eye-movement would stimulate new recollections to re-open my log-jammed head to what I knew I knew.

At best, I felt I was squarely on the fence.  I’m grateful to the guys who administered the exam for translating my verbal spaghetti into thoughts that must’ve communicated my intentions even though at the time I felt my 22-month old had more effective speech patterns.

A new Q &A have been added to the CMMIFAQ.
Answered is the not so trivial question: How do we pick a consultant or lead appraiser?

This question has come up a number of times. Not usually in such a straight-forward way. Actually, it often comes up in the form of something like:

Doesn’t the variation from lead appraiser to lead appraiser in model interpretation and what will be considered acceptable appraisal evidence make the model bereft of meaning and value?

It’s a legitimate question. How can a model really provide any guidance when there’s so much room for interpretation of the practices and what would be accepted as evidence by an appraiser during an appraisal?

Well, let’s go back to a basic and fundamental root: it’s a m o d e l not a standard or a process. (See this post, please, for more on that.)

And, as such, models are both incomplete and only representative of a particular reality, at best. Some people are just plain better at working with models than others. I’m beginning to believe it’s a talent. And, I’m not sure whether it’s a skill that can be taught — at least not quickly.

I’m still working on how to assess/interview a prospective consultant/lead appraiser for their “talent” at working with the CMMI model. The challenge for most organizations in need of a CMMI consultant/lead appraiser is that their knowledge and comfort level with the (CMMI) model may not be sufficient to recognize “good” answers from “poor” answers — in the context of the given organization, of course.

If you’ve got ideas, please let me know.

(Though… come to think of it… maybe that’s a handbook I ought to write!?)