What You Measure is What You Get?

"Perhaps what you measure is what you get. More likely, what you measure is all you'll get. What you don't (or can't) measure is lost" - H. Thomas Johnson

Those of you who are Deming fans may liken this quote to Deming's admonition that "the most important figures that one needs for management are unknown or unknowable, but successful management must nevertheless take account of them." (from Out of the Crisis, p121).

I came across this quote recently, which was quite apt as I was completing the end of year wrap-up required of all employees at my firm. Like many firms that run on making the metrics, making them look good, having airtight explanations for variances, plans to move the needle, and so on, the powerpoint decks generally tell a tremendous story. (If only Wall St. could see them - perhaps stock prices would be better?).

Conversely, if you don't have numbers to back up your story, come back and talk to me when you do.

So back to my year end wrap-up, which, by the way, is a key component of performance evaluation, merit increases, and future career path within the firm. I was advised to include as many "quantifiable accomplishments" as possible. Having joined the firm only mid-year, and assigned to work on a quality issue that has plagued the industry for 10 or 20 years, it is probably too soon to declare victory and post a dramatic improvement to the things our customers care about most.

In fact, the bulk of my time has been spent trying to develop facts and data about the process performance, and impress upon the producing organizations the voice of the internal and external customers, so that we can focus our measurements and improvement efforts on the right things. Very basic questions - who are the customers? What do they need? How well are we meeting those needs?, basic questions that we are still trying to answer. All the while, the quality of this particular product hasn't changed at all since I launched the effort several months ago.

Our team has also been highly aware of the dark side of metrics - to Dr. Johnson's point, what you measure may be all you get. And to quote another favorite thinker of mine, The Lean Thinker, "you get what you measure, but don't be surprised of people are ingenious in destructive ways in how they get there" (full post here.) So we strive for 100% on-time performance, only to see our first time yields plummet. Or we strive to measure revisions, only to have needed corrections go un-made in order to show a reduction. As one of my colleagues put it, "tell me what number you want to move, and we'll make sure that we do".

Now, this is not a plea to remove metrics. Only to measure the right things, and measure them correctly, so that we account for the "dark side". So, present on-time performance figures, if that's what's important, but make sure the FTY % is always presented next to it. It's also a plea to keep taking account of the unknown or unmeasurable things that matter - just because we can't measure it, doesn't mean we get to ignore it.

Last week, a senior leader and sponsor of this effort jokingly asked whether I had the problem figured out. Not quite. But I was able to tell him that department X has several talented black belts on it, is now focused on measuring defects the right way, and is starting to really understand the needs of their internal and external customers. To which he replied, "That's a major accomplishment!".

I asked if I could quote him in my wrap-up - it's probably the best piece of data I have going for me thus far.

Whatever holiday you and yours celebrate, I hope it is a good one.