Archive for the ‘Defects’ Category

Counting Change

Monday, May 2nd, 2011

A more appropriate title would have been "counting changes" but it would have hardly been as interesting.  :-)

Change happens.  And often.

In particular, when a product is in its operation and maintenance ("O&M") phase, changes are constant.  (Note: O&M is frequently called "production", and this simple choice of words may also be part of the issue.)  But, too often, changes to products are handled as afterthoughts.  When handled as "afterthoughts", product features and functions receive far less discipline and attention than warranted by the the magnitude of the change were the new or different feature/functionality have been introduced during the original product development phase.

In other words, treating real development as one would treat a simple update just because the development is happening while the product is in production is a mistake.  However, it’s a mistake that can be easily diffused and reversed.

O&M work has technical effort involved.  Just because you’re "only" making changes to existing products that have already been designed, does not mean that there aren’t design-related tasks necessary to make the changes in the O&M requests.  Ignoring the engineering perspective of changes just because you didn’t do the original design or because the original (lion’s share of) design, integration and verification work were done a while back doesn’t mean you don’t have engineering tasks ahead of you.

In O&M, analysis is still needed to ensure there really aren’t more serious changes or impacts resulting from the changes.  In O&M, technical information needs to be updated so that they are current with the product.  In business process software, much of the O&M has to do with forms and reports.  Even when creating/modifying forms, while there may not be any technical work, per se, there is design work in the UI.  The form or report itself.  And even if you didn’t do that UI design work, you still need to ensure that the new form can accept the data being rendered to it (or vice-versa: the data can be fit into the report).

It’s frightening, when you think about it, how much of the products we use every day — and many more products that we don’t know about that are used by government and industry 24/7 — are actually "developed" while in the "O&M" phase of the product life cycle when the disciplines of new product development are often tossed out the door with the packing material the original product came in.  Get that?  Many products are developed while in the "official" O&M phase, but when that happens they’re not created with the same technical acumen as when the product is initially developed.

(I have more on this topic, and how to deal with business operations for products in the O&M phase, in this Advisor article from the Cutter consortium.)

In a sadly high number of operations I’ve encountered, once a product is put into production, i.e., is in O&M, the developers assigned to work on it aren’t top-notch.  Even in those organizations where such deleterious decision-paths aren’t chosen, the common experience in many organizations is that the developers are relied-upon even more for their intimate knowledge of the product and the product’s documented functionality — as would have otherwise been captured in designs, specifications, tests and similar work artifacts of new product development.  In these organizations, the only way to know the current state of the product is to know the product.  And, the only way to fix things when they go wrong is to pull together enough people who retain knowledge of the product and sift through their collective memories.  The common work artifacts of new product development are frequently left to rot once the product is in O&M, and what’s worse is that the people working on the new/changed features and functionality don’t do the same level of review or analysis that would have been done were the functionality or other changes been in-work when the product was originally developed.  Of course, it’s rather challenging to conduct reviews or analysis when the product definition only exists as distributed among people’s heads.  Can you begin to see the compounding technical debt this is causing?

I’ve actually heard developers working on legacy products question the benefits of technical analysis and reviews for their product!  As though their work is any more immune to defect-inducing mistakes than the work of the new product developers.  What’s worse is that without the reviews and analyses, defect data to support such a rose-colored view seldom exists!  It’s entirely likely, instead, that were such data about in-process defects (e.g., mistakes in logic, design, failing to account for other consequences) to be collected and analyzed, it would uncover a strong concentration of defects resulting from insufficient analysis that should have happened before the O&M changes were being made.

Except in cases where O&M activities are fundamentally not making any changes to form, fit, feature, appearance, organization, integrity, performance, complexity, usability  or function of the product, there should be engineering analysis.  For that matter, what the heck are people doing in O&M if they’re not making any changes to form, fit, feature, appearance, organization, integrity, performance, complexity, usability or function of the product?!

If anyone still believes O&M work doesn’t involve engineering, then they might need to check their definition of O&M.  Changes to product are happening and they’d better be counted because if not, such thinking fools organizations into believing their field failures aren’t related to this.  Changes count as technical work and should be treated as such.

(I have more on this topic, including how to help treat O&M and development with more consistent technical acumen in this Advisor article from the Cutter consortium.)

Happy 2011!! Don’t let mediocrity be a “goal”!

Sunday, January 2nd, 2011

With many people and business executives making New Year’s resolutions, today’s topic is about goals and how setting the wrong goals can often undermine becoming high performance.

For example, a business *goal* of +/-10% budget/schedule? What’s wrong with this picture?  What’s it saying about an organization who makes a business *goal* out of being within 10% of their budget and schedule?

Does it give customers a warm fuzzy that a business knows what it’s doing when *their* *GOAL* is to come within 10% of what they said they’d do?  *THAT’S* supposed to make you feel good?

Shouldn’t goals be something to aspire to?  A challenge?  And, if getting within 10% of the budget or schedule is an aspiration or a challenge, that’s supposed to be *goodness*?

Such goals are nothing more than an aspiration to be mediocreAn admission that the organization actually has little confidence in their ability to deliver on commitments, to hit targets.

That’s one way to look at it.

Another is to say (what’s probably more accurate) that their estimates are a joke, and that when the “estimate” becomes the allocated budget, what they’re saying is that they’re praying the estimate won’t screw them.  Furthermore, it’s a likely reflection that they really don’t know their organization’s true capability in a “show me the data” kind of way.  They don’t have data on lead time, cycle time/takt time, touch time, productivity, throughput, defect/muda or other performance-revealing measures.

And so, without real data to instill confidence in capabilities, setting lame goals to hit targets is like many other things such organizations do: they go about business without a clear understanding of what they need to do or what it’s going to take to get the job done.  That way, when they don’t hit their targets they can just blame the innocent or find some other excuse for remaining mediocre.  After all, how exactly would such an organization expect or plan to hit their targets?  Come on!  Let’s be real.  They have no idea! 

Either way, making it a *goal* to do something we *expect* them to do is rather lame!

This year, don’t make lame resolutions, instead, come up with a strategy and a plan to to attain *confidence* in being able to hit specific SMART targets.  Then, grow that confidence and narrow the spread of the targets.

Cut, Paste, and Why Requirements are Tricky

Saturday, July 26th, 2008

imageI’ve been using PDAs for about 10 years now.  So, while I wasn’t the among the fist Apple Newton users,  and I didn’t buy the first version of the PalmPilot®, I did follow shortly thereafter with another Palm device and was a Palm fan until it came time for me to move to a "crackberry" because I could no longer wait for Palm to come out with an email-over-air device.

Rest assured, when Palm came out with its Treo 650, it couldn’t have been a moment too soon, as I lusted for it upon its announcement and needed only the smallest excuse to plunk down about 500US$ for it at the time.  That excuse came in the form of hardware failure on my RIM device.  (It was getting up in age — not to mention that I bought it 2nd-hand.)

Well… my life with the Treo was bitter-sweet.  Software and synchronization/integration issues brought my productivity to a grinding, frustrating, halt.  By then I was keeping tabs on the latest Windows Mobile® devices purveyed by my wireless provider and saw that a device was due with *everything*.  A good phone, Email, text, browsing, broadband-over-wireless, WiFi, and simultaneous phone and Internet access all built-in.  I practically counted the hours until it was finally released after being tantalized mercilessly by the press releases and ads.

Life was good.  I really had no complaints.  Until a few days ago.  It died.  Well… mostly died.  The touch screen stopped responding, to be specific.  Despite a built-in keyboard, the device relies on its touch screen.  And, in the middle of a business trip, the touch screen stopped working.  Back at the hotel, and a while at my laptop, I learned that this seems to be common for that device.  In fact, I found myself considering myself to be lucky it lasted the roughly 20 months it did.

I put together my list of requirements, prioritized them, and a bit of research later I’d settled on my next device.  Being due an upgrade anyway, I decided to stop in at a local store to get one on my way to the airport for my flights home.  I decided to go with a device from another manufacturer that didn’t have a touch screen since the previous day’s experience taught me that relying on a particular feature for functionality was risky.  I was even willing to sacrifice the WiFi requirement to gain a form-factor benefit, so I had a bit of technical/performance trading-off to do.

A few hours of messing around with the new device and I was both pleased and perplexed.  Everything seemed to be what I had read about it.  Oddly enough, I couldn’t figure out how to cut or copy and paste.  Furthermore, I could only find one alarm.  Which, for anyone power-using their PDA knows, alarms are critical.  So, back in my home, I downloaded the complete user’s guide (since all they put in the box these days with new phones is the "quick start" guide).

Reading through the guide I learned that, indeed, the device should have Copy/Cut & Paste, as well as two alarms (which is still lame, but ‘fixable’ with 3rd party software).  So I called the manufacturer — a world-wide company renowned for being early adopters of what’s known in the industry today as "6-Sigma" — and spoke to a customer technical representative. 

The conversation went something like this:

Me: I can’t seem to cut and paste, but the user guide says I should.  I mean, it’s not that it’s not working, it’s not even in the menu.

Rep: No, my technical notes [a.k.a. "release notes"] don’t say anything about being able to cut and paste unless you’re in [application].

Me: Yes, I can do it in [application] but I should also be able to do it elsewhere.

Rep: Why do you think so?

Me: Because it’s in the user guide and it’s a basic function.  Why wouldn’t you be able to cut and paste?

Rep: What user guide are you looking at?

After guiding the representative to the exact user guide I was looking at, she apologized and said that the guide is wrong, but the phone doesn’t actually have that functionality.  So I moved on the why the phone doesn’t have more than one alarm.

Me: OK, so can you tell me where I might find the functionality to set more than one alarm?

Rep: Sure, just go to [......]

Me: Yes, I’m there, just like in the user guide.

Rep: [Sounding surprised] Oh, that’s strange, I’ve got a phone like yours right here and you’re right, it only has one alarm.

Me: Would that be another user guide defect?

Rep: Yes, it seems so, I’m very sorry.

Me: "Sorry" doesn’t change the fact that your defect rate is way too high here.  What happened to 6-sigma over there?

Rep: I’m very sorry.  I don’t have any other information for you.

. . .

I was rather quite disgusted and was halfway out the door to my local store to return the device by the time the phone was back on hook.

After all, I’d become accustomed to copy/cut and paste on a PDA for over 10 years.  What sort of PDA doesn’t have cut and paste?  How is that a business device as it’s intended to be sold and used?  No copy/cut and paste from email, text messages, or the web?  Furthermore, it seemed that the manufacturer *did*, in fact, expect that basic function to be shipped.  So what happened?

Clearly, there were errors in the user guide, in the representative’s information, and inconsistencies with the device.  In CMMI, we’d find these defects avoided in strong practices in traceability, work product verification against requirements, technical data packages, releases/baselines, and configuration audits, and validation, among others.

Yet, I realized that none of my research included checking whether I could cut and paste, or even how many alarms the devices have.  In fact, they weren’t among my requirements at all.  They were just assumed to be "basic" functions!

So, in the store I look at the device I ranked #2 in my previous research.  Lo-and-behold!  It has the exact same lack of functionality as the earlier device despite being from different manufacturers!  What does that indicate? 

It’s a supplier issue!  Both manufacturers must get the software from the same supplier!  Hence, the one company’s expectation of functionality to the point of including it in their user guide and release notes only to later take shipment of the supplied wares without all expected features and functions!  What I then put together (in about a nanosecond) was that someone (sometimes someone tied with marketing?) promised to ship the product to the consumers but the product wasn’t ready!  In fact, they expected certain features and functions but for some other reason the supplier of the software couldn’t keep to the pre-defined delivery schedule!My new PDA

(In any case, I ended up with my #3 choice, the one with the touch-screen I was trying to avoid.  I’m quite pleased with the improvements they’ve made since the previous version.  They’ve also improved on the form-factor.  And, as a bonus, I’ve already got plenty of accessories for it.  So I can save a little money there.)

Were there defects in the first attempted purchase?  Were these defects excusable?  Those aren’t really the point to this post, are they?

The point is this: Requirements are tricky.  For lack of complete requirements a company has earned themselves a tarnished reputation, and I spent several hours being frustrated and having to take unscheduled time to deal with the error.  However, at least from my perspective, I could recover easily.  The iteration time and the correction were both fast.  In other words, I failed early and adjusted.

Were my requirements unreasonable?  Probably not.  But the episode highlights the challenge with requirements and the benefits to being flexible, iterative, and able to correct quickly.  Customers or developers who believe anyone will have all the requirements right and complete — including all the assumed and derived ones — are fooling themselves.  Witness the supposedly 6-Sigma manufacturer who "froze" the requirements based on what they thought they’d get and now they’ve got a black eye.  Whereas for myself, I iterated, and got the right product after all.