Archive for the ‘Appraisal’ Category

You’ve got processes, but . . .

Friday, September 23rd, 2011

A friend who consults in program, project and risk management (typically to parachute-in and save wayward technology projects) is working with a client whose project is dreadfully behind schedule and over budget, and, not surprisingly, has yet to deliver anything the end-client or their users can put their hands on.  It doesn’t help that his client isn’t actually known for being technology heroes.  In fact, this is not the first time his client has tried to get this project off the ground.

Looking everywhere but in the mirror, my buddy’s client decided to have the developer put under a microscope.  After all, reasoned the client, they hired the developer on, among other attributes, touts that they were rated at CMMI Maturity Level 3!  So, they had the developer and the product undergo a series of evaluations (read: witch hunts) including a SCAMPI (CMMI) appraisal.  Sadly, this tactic isn’t unusual.

Afterwards, trying to help his client make sense of the results, my pal asked me to review the report of the appraisal which was fairly and (more or less) accurately performed by someone else (not us).  The appraisal was quite detailed and revealed something very interesting.

Lo-and-behold, the company had processes!

However, the development organization nonetheless failed to demonstrate the necessary performance of the Maturity Level 3 (ML3) practices they were claiming they operated with!  In other words, they had processes, but they were still not ML3!  In fact, they weren’t even Maturity Level 2 (ML2)!

How could this be?

While the details bore some very acute issues, what was more interesting were the general observations easily discernable from far away and with little additional digging.  The appraisal company created a colorful chart depicting the performance of each of the practices in all of ML3.  And, as I noted, there were important practices in particular areas with issues that would have precluded the achievement of ML2 or ML3; but, what was more interesting were the practices that were consistently poor, in all areas as well as the practices that were consistently strong in all areas.

One thing was very obvious: the organization, did, in fact, have many processes.  Most of the processes one would expect to see from a CMMI ML3 operation.  And, according to the report, they even had tangible examples of planning and using their practices.

What could possibly be going on here?

Seems awfully much like the development group had and used processes.  How could they not rate better than Maturity Level 1 (ML1)?!  Setting aside the specific gaps in some practices that would have sunk their ability to demonstrate anything higher than ML1 – because this isn’t where the interesting stuff shows up, and, because even were these practices performed, they still would have rated under ML2 – what the report’s colorful depiction communicated was something far harder to address than specific gaps.  The developers’ organization was using CMMI incorrectly.  A topic I cover at least in the following posts: here and here.

In particular, they were using CMMI to “comply” with their processes but not to improve their processes.  And, *that* is what caused them to fall far short of their acclaimed CMMI ML3.

How could I tell?

Because of where the practices were consistently good and where they were consistently gap-worthy.

I was reviewing the report with my friend on the phone.  As I was doing so he commented, “Wow!  You’re reading that table like a radiologist reads an X-ray!  That’s very cool!”  The story the chart told me was that despite having processes, and policies, and managing requirements and so on, the company habitually failed to:

track and measure the execution of their processes to ensure that the processes actually were followed-through as expected from a time and resource perspective,

objectively evaluate that the processes were being followed, were working, and were producing the expected results, and

perform retrospectives on what they could learn from the measurements (they weren’t taking) and evaluations (they weren’t doing) of the processes they used.

It was quite clear.

So, here’s the point of today’s post… it’s a crystal clear example of why CMMI is not about process compliance and how it shows up.  There are practices in CMMI that definitely help an organization perform better.  But, if the practices that are there to ensure that the processes are working and the lessons are being learned aren’t performed, then the entire point to following a process has been lost.  In Shewart’s cycle, this would be akin to doing P & D without C & A.

The only chance of anything that way is compliance.  There’s no chance for improvement that way except by accident. 

CMMI is not about “improvement by accident”.  (Neither is Agile for that matter.)

Interestingly enough, while there were clearly issues with the developer’s commitment to improvement, there may not necessarily have been any clear issues with either the product or the results of their processes.  While the experience may not have been pleasant for the developer, I don’t know that by buddy’s client can say to have found a smoking gun in their supplier’s hands.  Maybe what the client needs is a dose of improving how they buy technology services – which they might find in CMMI for Acquisition.

Posted in Appraisal, Generic Practices, Improvement, PDCA, SCAMPI, compliance | No Comments »

SEPG North America – Tutorial Day

Monday, March 22nd, 2010

So today started out with a bus ride from the hotel to the Savannah International Trade and Convention Center rather than the expected ferry ride over the river.  A container ship in the port managed to get damaged and leaked fuel into the Savannah River on Sunday immediately closing the river to non-clean-up traffic, including the otherwise convenient cross-river ferry.

Be that as it may, the bus ride gave me an opportunity to connect with Michele Moss from Booz-Allen, Hamilton.  A kindred spirit in things related to "the future of process".  She and I had plans to meet anyway some time today to discuss ideas about "bringing ‘younger people’ into the field" and a related topic, addressing modern-day issues such as cyber, agile and value as these concerns are manifested in processes and process improvement.

First order of the day after registration was to co-create what I perceived as a rather successful (and well-attended) tutorial with Judah Mogilensky on a tailoring for SCAMPI appraisals that increases efficiency, collaboration, and reduces time and cost, we called "One-Stop Shopping".  Immediately following, Michele and I met with Bob Rosenstein, the events and conferences manager at SEI.  David Anderson, just arriving to the venue, was a very beneficial addition to the discussion, conveying his experience with creating communities and conferences specific to a community such as his LSSC.  Dana Hanzlik and Danny Pipitone from SEI’s PR group also sat in on the conversation.  About the only definitive expectation to come out of this meeting (other than our commitment to come to the retrospective with with data from the Peer-to-Peer), was that SEI will be open to more closely tying into other gatherings.  Not bad since we had no expectations going in, and, even if we had, it wouldn’t have been reasonable to have expected any commitments.

Much came up in just under an hour with Bob.  We’re planning to include bits of this topic in our end-of-conference committee retrospective on Thursday.  Part of what will feed into that retrospective will be a Peer-to-Peer session on Wednesday afternoon that Michele and I will be co-creating and was planned with David’s help.  Our Peer-to-Peer is being billed as, "Where do we go from here? Value, Agile, Cyber, and all things Future Processes."

The mind-map of the problem-space was really intriguing.  This will not be an easy matter.

After a conference lunch with David and Michele, we split up and I attended the invitation-only advanced overview of the changes to "high maturity" to CMMI v1.3.  Good stuff, really.  Way too geek for here.

After getting as much as I cared to get from the high maturity campfire (which coincided with the moment I sensed my lunch moved far enough down my digestive tract to make room (literally) for a run) I decided to go back to my hotel to squeeze a run in before the evening gorge-fest that includes the opening of the trade-show floor, a board meeting, and later, a surprise opportunity to attend a special reception, all of which were to include food (and in order of continually improving quality at that).

Before I could get back across the river, I nabbed an opportunity to comment on a frequent occurrence here, on the Savannah River:

Several lovely hours later of socializing (albeit, mostly work-related) I’m back at the room planning my day ahead.

Posted in Appraisal, Collaboration, Conference, David Anderson, Efficiency, Jeff Dalton, Judah Mogilensky, LSSC, Michele Moss, Process Improvement, SEI, SEI Partner, SEPG, SEPG Conference, Speaking, Training, value, waste | 1 Comment »

Picking a Lead Appraiser: "Dammit, Jim! I’m a doctor not a bricklayer."

Sunday, December 27th, 2009

In this quote, CAPT Kirk wants Dr. Bones McCoy to do something he feels he’s not-qualified to do because he doesn’t know how to treat the species.

I’m using it to explain that organizations looking for a lead appraiser to work with them towards an appraisal and/or to perform an appraisal ought to think of what we do as they would think of a doctor, not a laborer or vendor.

Do you really want the lowest price doctor?

For that matter, is the highest price doctor necessarily the best in town?

When reaching out and interviewing for a lead appraiser or CMMI consultant, you:

  • Want the person who is the right person for the job.
  • Want someone who is qualified (definitely not under-, but preferably not over- either).
  • Not the lowest bid.

Seriously, whoever you hire for this effort has in their power the ability to make or break your future.  They literally have the health and well-being of your organization in their hands.  They can put you in the dump just as easily as they can take you to the next level.

They should see themselves that way as well.

Unfortunately I’ve got too many sad stories of appraisers/consultants who definitely see that they can make or break you, but they don’t feel like they personally own the responsibility for what happens to you when they’re done.

If it costs too much?  So what?
If you get no value?  Not their problem.
Didn’t see any benefit?  Didn’t learn anything?  Things take longer and cost more and you’re not seeing internal efficiencies improve?
YOU must be doing something wrong, not them.

In an AgileCMMI approach, your CMMI consultant and/or lead appraiser would see themselves as and act like a coach, and would put lean processes and business value ahead of anything else.  And, an AgileCMMI approach would know that when the processes work, they add value; when they add value people like them and use them; when people like and use them, the next “level” is a big no-brainer-nothing.  You get it in your sleep.

Let me know if you want help finding the right lead appraiser or consultant.

Posted in Appraisal, Business Benefit, CMMI, Cost, SCAMPI, Time, consultant, lead appraiser, value | 2 Comments »