A starting point for a discussion on marrying Agile methods and CMMI.Verification, Validation, & the iPhone 4
Posted by Hillel on Jul 7, 2010 in Agile, Agile+CMMI, Apple, CMMI, Engineering, Validation, Verification | 1 comment
Apple, Inc. learned the hard way what happens when engineering isn’t complete. In particular, when verification and/or validation aren’t performed thoroughly.
Verification is ensuring that what you’re up to meets requirements. “ON PAPER.” BEFORE you commit to making the product. It’s that part where you do some analysis to figure out whether what you think will work, will actually do what you expect it to do. Such as, walking through an algorithm or an equation by hand to make sure the logic is right or that the math is right. Or, stepping through some code to see what’s going on before you assume that it is behaving. Just because something you built passes tests, doesn’t mean it is verified. All passing tests means is just that: you passed tests. Passing tests assumes the tests are correct. If you’re going to rely on tests, then the tests need to be verified if you’re not going to verify the requirements or the design, etc. Another problem with tests is that too many organizations only test at the end. Verification looks a lot more like incremental testing. Hey wait! Where’ve we seen that sort of stuff before?
Had Apple’s verification efforts been more robust, they would have caught the algorithm error that incorrectly displays the signal strength (a.k.a., “number of bars”) on the iPhone4. This is why peer review is so central to most verification steps. The purpose of peer review, and of verification, is to catch defective thinking. OK, that’s a bit crude and rude… it’s not that people’s thinking is defective, per se, but that thinking alone didn’t catch everything, which is why we like to have other people looking at our thinking. Even Albert Einstein submitted his work for peer review.
Validation is ensuring the product will work as intended when placed in the users’ environments. In other words, it’s as simple as asking, “when real users use our product, how will they use it, and will our product work like we/they expect it to work?” Sometimes this is not something that can be done on paper, and you need some sort of “real” product, so you build a prototype. Just as often it’s not something that can be done “for real” because you don’t get an opportunity (yet) to take your product into orbit before it has to go into orbit to work. Sometimes you only get one shot, and so you do what you can to best approximate the real working environment. But neither of these extreme conditions can be used by Apple as excuses for not validating whether or not the phone will work as expected while being held by the user to make calls.
Had Apple’s validation been operating on all bars, they likely would have caught this while in the lab. When sitting in its sterile, padded vice, in some small anechoic chamber, after taking great care to ensure there are no unintended signals and nothing metallic touching the case, someone might’ve noticed, “gee, do you think our users might actually make calls this way?” And, instead of responding, “that’s not what we’re testing here”, someone might’ve stepped up and said, “hey, does our test plan have anything in it where we’re running this test while someone’s actually using the phone?”
Again, testing isn’t enough. Why not!? After all, isn’t putting it in a lab with or without someone holding the phone a test? True… However, I go back to the same issue we saw when using testing as the primary means of performing verification… Testing is too often at the end. Validating at the end is too late. You need to validate along the way. In fact, it’s entirely possible that Apple *did* do validation “tests” of the case separately from the complete system, and, in *those* tests — where the case/antenna were mere components being tested in the lab — performed fine, and, then only when the unit was assembled and tested as a complete system would the issue have been found. In such a scenario we learn that component (elsewhere known as “unit testing”) is not enough. We also need system testing (in the lab) and user testing (in real life). Back we go to iterative and incremental…
So you see… we have a lot we can apply from ordinary engineering, from agile, and from performance improvement. Not only does this… uh… validate(?) that “agile” and “CMMI” can work together but that for some situations, others can learn from applying both.
In full disclosure, as a new owner of an iPhone 4, I am very pleased with the device. I can really see why people love it and become devotees of Apple’s products. Honestly, it kicks the snot out of my prior “smart” phone in every measurable and qualitative way. And, just so I’m not leaving anything out, the two devices are pretty much equally balanced in functionality (web, email, social, wifi, etc.) – even with the strange behaviors that are promised to be fixed. For a few years, this iPhone will rule the market and I’ll be happy to use it.
Besides embarrassing, this will be an expensive couple of engineering oversights for Apple to fix. And, they were entirely avoidable for an up-front investment in engineering at an infinitesimal fraction of the cost/time it will take to fix. For even less than one day of their engineering and deployment team’s salary, AgileCMMI can make this never happen again.
Apple, look me up. I’m easy to find.
New ideas emerging at SEPG Europe
Posted by agilecmmi on Jul 1, 2010 in Improvement, Performance, SEI, SEPG Europe, v1.3 | Comments Off
Regardless of venue, country, time of year, or language, SEPG Europe continues to demonstrate itself as a valuable event for exchanging ideas and making progress in the field of performance excellence. It’s a clear indicator of the value of SEPG Europe that attendance at this year’s event both doubled from last year’s event and exceded all headcount-based logistics planned for the event. This, despite the sputtering global economy, in particular Portugal’s current banking challenges.
Conference-related activities for SEPG Europe 2010 began with pre-conference activities and tutorials on Monday, official tutorials on Tuesday, then keynotes, mini-tutorials and sessions on Wednesday. This entry comes on the morning of the last formal conference day, Thursday, after experiencing Wednesday’s keynotes, a full day of sessions and mini-tutorials, and the event’s gala dinner.
In particular, I want to focus on common threads heard throughout the week, what they mean to those of us in the field, and why it’s only at SEPG events where these ideas can reach critical mass.
The common threads
CMMI, appraisals, and the focus on “process” are, together, insufficient to meet the needs of today’s businesses and still relevant.
Insufficient because, alone, they can miss attributes important to business, and can inadvertently place too little emphasis on performance and results. Still relevant because, without them there would be no robust, complete product set of performance improvement tools in the marketplace.
What these threads tie into is the experience that the market for performance excellence is ready for the “next evolution” of CMMI and SCAMPI and other process-oriented models and tools. The market is ready for a way of looking at performance excellence that is appropriately applied in ultra-large systems as well as small and/or agile systems/organizations. An approach that emphasises results rather than compliance, and an approach that looks at the entire business, including its market, culture, social economics, leadership, management, customers, relationships and other behavioral sciences.
By no means is this to imply that CMMI and SCAMPI are wrong. They are widely acknowledged and credited as a necessary step in the evolutionary path of performance excellence thinking. And, some flavor of CMMI and SCAMPI will most certainly persist as a necessary component of a broader focus on improvements. All this is actually saying is that the market has absorbed the lessons of CMMI and SCAMPI and they’re ready for more. They’re ready for what’s coming next, and they want to be part of shaping it.
A lot of the hallway conversations I’ve had have been about just this. They’re about “what’s next?” What’s after version 1.3? It’s not clear what’s coming after v1.3, but what is clear is that whatever v.NEXT looks like, the ideas for what will be in it (by any name or version) will have roots at events like SEPG-Europe.
People here are clearly thinking ahead. They’re thirsty for making progress.
What the common threads mean to those of us in the field
Those of us who provide consulting, instruction and appraisals in CMMI and SCAMPI wares, or who are internal to companies implementing improvements will be impacted by these threads in a number of ways. Including, a potential wholesale change in what will be a “model” for improvement and its related appraisal approach. Another impact would be the possibly broader reach of areas of improvement into aspects of business currently unfamiliar to organizations or professionsals in the field.
Furthermore, the business impact of the v.NEXT model could be a body of work that raises the stakes and the perview of where the model seeks to have an impact. In other words, it could be a model that’s much more business-oriented and “systemic” than it currently and would require skills and aptitudes for implementation not demanded by the current frameworks. It could become a model for which it’s not enough to be a model subject-matter expert, but also requires that users be equally versed in business as they are in performance improvement.
The core concepts in CMMI today are not likely to disappear, rather, they’re more likely to be absorbed into a more broadly-minded view of causing performance excellence.
Why SEPG events are where the critical mass is reached
Unlike any other type of events, at SEPG events there are presentations, discussions, new ideas and the direct interaction among users, practitioners, business leaders, government, academia, many industries, and the SEI. Face-to-face, high-bandwidth communication and incidental interactions made possible by SEPG events are unlike any other events because SEPGs are focused exclusively on improvement. It’s a conversation at a higher level. The ideas for such a higher level of thinking in v.NEXT will be where the SEI gets its ideas. These are the types of conversations taking place at SEPG Europe which is why I attend. I attend so I can return to my office and my clients with new ideas and a glimpse of where things are going.
Until v.NEXT is reality, we can muse philosophically over what will be in it over glasses of the fine Porto port wines.
Happiness is not an edict.
Posted by agilecmmi on Jun 7, 2010 in Book Review, Change, Culture, Employees, Happiness, Leadership, Morale, Performance, Results, Service, Tony Hsieh | 1 comment
I make a living helping companies improve their performance. Bar none, the absolute hardest challenge is dealing with outdated, defunct, and proven-failed management and leadership attitudes and techniques. It’s even harder when leaders won’t let go of them.
Attitudes that don’t account for the emotional needs of people.
Attitudes that don’t start with the fundamental requirement that tough or large changes require the top-down example and behavior to lead the way.
Attitudes that don’t stand up for what’s right and acquiesce to blatant cognitive dissonance.
Attitudes that fail to realize you can’t lay-out the company’s supposed mission or vision, or the staff’s objectives and goals, and not provide them with the tools or empowerment to make anything happen — or worse, to undermine efforts with behaviors and decisions that conflict with the messages.
Attitudes that perpetually fail to create sustainable results, superior performance, and legendary relationships among the company staff, and between the company and its suppliers and customers; yet they are the same attitudes that failure-destined leaders continue to retread in some insane hope or expectation that everything will work out on its own.
Well, here’s some not-so-profound news (which I’m pretty sure you didn’t need me to tell you): these attitudes are nothing short of leadership’s inability to own up to their responsibilities and accountability for actually doing what they’re there to do: lead.
Instead of leading you’ll often find such cowardly executives hiding behind backbone-belying excuses such as "I lead through delegation" (which, when said by these leaders actually means, "I really don’t know what to do to help you get your job done"), or "I trust my people to do the right thing" (which, when said by such leaders, actually means, "I’m lucky my people know what they’re doing because I don’t"), or "I’m serious about quality, I’ve made Joe, here, the company quality sheriff" (which is poser-leader-speak for, "I haven’t the foggiest idea what drives quality in our company and but it’s a problem and I don’t know how to fix it, but getting someone else to blame is a lot easier than getting mixed-up with all that touchy-feely stuff.").
Such leaders are nothing more than overpaid, over-indulged and over-honored paper-pushers whose lack of involvement in day-to-day operations is probably a blessing. Because, when they do get involved (especially considering their only natural inclination is to micro-manage if/when they bother), they typically don’t make things better.
In a profound way, most companies’ leaders simply don’t "get it", they won’t admit that they don’t get it, and they’re not open to having anyone help them get it.
One the basic "its" they don’t *get* is that what’s going on in their company is a culture, and because they don’t see what’s going on as a culture, or appreciate the role culture plays in making a business successful, they not only don’t see that what they need is to change the culture, but that in order to do so they must lead the change themselves.
Furthermore, they don’t see that what it means to change the culture is not to pass down new rules or to disseminate a verbal memo through subordinates, it means to get out and lead. To behave or simply *BE* the new culture. How flat does a new, great-sounding idea fall when it’s delivered in the same mechanisms as all the old, not-as-great (dare I say "dumb") ideas?
Do you agree?
Perhaps you need validation of your way of thinking.
Perhaps you need an example to follow.
Perhaps you need to crystallize your thoughts so you can pave a path.
Or perhaps you need something to believe in, something to help you find a better way to work, place to work, or how to start making a difference in your work life and work-life balance.
Maybe you’re someone who’s been burned trying such non-traditional ideas and you’re seeking what might have been missing, or a simple reflection on a point you didn’t see or know to look for.
In any case, if you are seeking how to understand and positively influence culture as a primary business driver of positive change and astounding accomplishments, then Tony’ Hseih’s Delivering Happiness is a must read. Scratch that. If you’re reading this review this far you MUST read. Delivering Happiness. Even if you’re currently not big on culture as a business driver, or, if you think you know about culture and business but secretly, deep down, you really have no clue what it is, how it looks, or how to create the one you want, you must read this book. If you *know* your business has a culture problem but don’t know what to do about it, read this book.
The amazing things Tony and his team have accomplished by putting _happiness_ at the center of their existence and by making it unmistakably simple to lead by example will leave you wondering why it’s taken anyone this long to put this sort of content together. There is nothing new in leading by example. It’s talked about, written about, lectured about in every credible business and management venue anywhere at any time in history. But, one thing (of many others) unique in Tony’s book is that he actually conveys _what_that_looks_like_. It’s not just words to him and his team. It’s how they *are*. It’s in everything they do and say and consider. They wanted a certain culture? They *became* that culture and started every action from there. It’s in how they treat each other, their customers, their suppliers, their partners and their extended relationships. They want to sustain that culture? They *built* and *rebuilt* their existence to continually refine what it takes to generate the culture they want. Delivering Happiness is like being the ultimate "fly on the wall" to get the insight into answering the question most business books and theories frustratingly fail to answer, "OK, I get what you /describe/, but what did you *do*?!" This book lays it out.
Delivering Happiness will probably make you wonder why bother with any other business approach, and even why bother with any other business book! It’s as though all you ever wanted to know about running a business and making people happy (either as one, the other, or both) are in his 272-page first offering. I’m not saying other business books don’t matter, and that other business books don’t have valuable insights and concepts. What I’m saying is that Delivering Happiness is so fundamental that it almost makes no sense to start elsewhere. If business leaders don’t understand the fundamentals of the role culture plays in their success and what to do to generate the culture they want, then little else can help.
Tony’s done great research. Thanks (or no thanks?) to Delivering Happiness, my reading list has not only grown, but has sprouted an entirely new branch of exploration. It’s fascinating stuff. I was so intrigued I rescheduled several days’ plans to finish-up some of the reading I had already started so I could clear my reading time and my mind for the new stuff Tony introduced me to.
Get it here.
Not only does Tony make an eloquent and convincing case for the influence and effects of business culture, but his amazing story about Zappos.com and the insights from places he worked or owned before — as they all relate to _happiness_ http://www.deliveringhappinessbook.com/ — are inspiring and instructive — for anyone with an open mind.
And… not the kind of "open mind" that rationalizes nonsense, but the kind of open mind that’s open to questioning tradition; asking where a tradition came from and why it continues to exist despite its time having come and gone. The kind of open mind that’s continually seeking better ideas built on progress and results, not on reactionary fears, theory, or short-term thinking. And, the kind of open mind that’s ready to make a really big difference in their work, life, or in the world, and not cling to stale thinking born of a time when those old ideas were once new.
It’s time to replace those ideas with ideas built in and for the 21st Century. It’s time for ideas that deliver results. It’s time to start Delivering Happiness.
P.S. I received a free copy of the book so that I could read it and write an honest review about it in time for the launch date. I received my free copy on nothing but the promise that I would write an honest review, and, all I received was the free book. Other than my own integrity, I had no compelling reason to keep my promise. I didn’t have to read the book, or write a review, or even write a positive review. In fact, I could just as easily have written a review (irrespective of whether or not I’d read it) trashing the book — if it were trash. But it’s not. I have received plenty of free books to review and even requests to write forewords, summaries, afterwards, etc. I have written (and had published) unflattering reviews (not something I relish) and I’ve also chosen not to write a review or contribute to a work when, as in the case of reviews, there was little nice to say, or in the case of contributions, I didn’t want to associate with the work. All this is just to say: this is a very honest, unencumbered review. There are no strings attached in any way for having written it. Go read Delivering Happiness. You’ll be glad you did.
P.P.S. There’s also a Tweeter (@dhbook) to follow, and a community to join, and a Facebook crowd to like. If this is your thing, I encourage you do so.
This site.
The opinions expressed here are the authors' and contributors' and do not express a position on the subject from the Software Engineering Institute (SEI) or any company or SEI Partner affiliated with the SEI.
Your Host
Hillel Glazer
Principal & CEO Entinex, Inc.
Email me.-
