The one metric that muddled customer service

Opinion editor’s note: Strib Voices publishes a mix of guest commentaries online and in print each day. To contribute, click here.

•••

Like David Banks (“You call that customer service?” Jan. 12, and “ ‘You call that customer service?’ — part two,” Jan. 24), I lament the loss of real customer service. Over my lifetime, which is about to enter its eighth decade, I have seen customer service go from spotty during my youth, to a serious focus during the W. Edwards Deming-inspired quality movement of the 1980s, to a digital afterthought today. While it would be easy to blame cost-cutting for this latest trend, I like to channel my inner 4-year-old and ask "why" a few times. This chain of questions inevitably leads me to the system of bonuses paid to senior leadership being at the root of our corporate problems in the U.S., and perhaps many of our societal problems as well.

One perverse side effect of the bonus system, and the admirable desire to base bonuses on performance, is that the people who design the bonuses have to come up with metrics. Often inventing them out of whole cloth. Unless, as I am about to show, a well-respected business publication presents them with a gift.

One such gift contributing to the impression that senior leadership in U.S. corporations is tone-deaf to the suffering of their customers and employees is NPS, or the Net Promoter System. If you have ever had a customer-service experience where you were politely asked to rate the interaction with a “9 or 10” score, you have encountered NPS.

Under the system, “Promoters” — who would respond to a question such as “How likely are you to recommend our product/company/service to a friend or colleague?” with a score of 9 or 10, on a scale of 0 to 10 — were loyal customers who actively promote the brand. “Passives” (7-8) were satisfied but unenthusiastic customers. “Detractors” (0-6) were unhappy customers who may discourage others from engaging with the brand. The NPS score calculation is derived by subtracting the percentage of detractors from the percentage of promoters.

According to my new best friend, ChatGPT, NPS was introduced to the business world in a Harvard Business Review article (“The one number you need to grow”) in 2003. (I wonder how much damage has been done over the years by articles such as this one being taken as gospel by the business community. Another one that comes to mind suggested that corporations segment themselves into Strategic Business Units, or SBUs, that would then compete against one another, guaranteeing sub-optimization, to the delight of our overseas competitors.)

I landed a Six Sigma trainer job at UnitedHealth Group’s Optum division — “Six Sigma” being a data-driven methodology intended to improve business processes — in 2017. It was obvious from day one that the Six Sigma program there had been hijacked by managers to be focused solely on cutting costs, an all-too-common mistake that had also befallen GE and 3M, and that it had lost any connection to Deming’s “chain reaction” (i.e., improve quality, lower cost due to subsequent savings, grab market share, provide jobs and more jobs, etc.), which had as its focus the improvement of quality. Also, no one really understood the value of practical, statistical thinking.

There is a huge difference between the statistics taught at most colleges and universities (i.e., enumerative statistics — the science of the significant difference) and what actually works in business (i.e., analytical statistics — the science of understanding what happens over time). Deming spent the last few years of his life, into his early 90s, trying to get his peers at the college teaching level to wake up to the importance of this distinction and to transition to teaching analytical statistics. I believe he failed. And the results were evident at UnitedHealth Group, and I believe this led to its warm embrace of NPS.

The big problem with NPS is evident to any well-trained Six Sigma green belt. You never “dumb down” a continuous measurement, or one that approximates a continuous measurement, to make it discrete. In a primer on NPS, Step 2 says to convert the customer scale-score responses (0-10, which is another problem for another day as to why the creator of this tool felt humans could differentiate among 11 levels of performance, when five to seven levels are generally felt to result in more reliable results) into those three buckets: Promoter, Passive, Detractor. So now a measurement at 11 levels has only three. You have destroyed information in the process, which in my world is a very, very bad thing to do willingly.

While teaching Six Sigma to UHG employees, I was privileged on many occasions to assist them with projects related to their particular area of the company. Often, I would learn that they were focused on improving their NPS scores, trying to achieve a corporate goal of “70% in Seven Years.” My first suggestion would be to get their hands on the original, scale-score data so they could look at the original frequency distribution for hints about possible root causes. Without exception, they were prohibited from seeing that data. That in itself is a problem. Why were the owners of this data so reluctant to share it with the very people who might be able to improve the processes that influence the results?

Fast-forward a few years, and I was in a role where I was able to get my hands on the NPS scale-score data. Below is a simulation I have created, one that produces a NPS score typical to health care organizations, to make my point based on this one time I actually was able to analyze the scale-score data:

(NPS scale score and number of customers giving it)

Promoters

10 — 4,425

9 — 3,179

Passives

8 — 1,153

7 — 727

Detractors

6 — 217

5 — 313

4 — 78

3 — 135

2 — 167

1 — 479

0 — 1,500

(With promoters comprising 61.46% and detractors 23.35%, the Net Promoter Score in this case is 38.1%.)

If this were data from UHG, the fact the NPS score was well below the corporate goal would be cause for concern. But without the raw data, you wouldn’t be inclined to ask the obvious question, “Why do we have a mode at zero?” Have you ever had an experience where you rated a supplier of a product or service with a zero? If yes, you were likely pissed off.

They had an expression at UHG that applied in these circumstances — “customer abrasion.” My dad was a lifer at 3M, so I always associated abrasion with sandpaper, but at UHG this term was used when discussing how we often irritated our customers.

In the situation above, an obvious next step would be to randomly sample the surveys from the “0” respondents, to look for comments that might shed some light on what issue or issues they were so upset about. But if you only have the 38.1% value, you are “deaf” to some of your most angry customers!

While I share the sentiments Banks had about the senseless murder of a devoted husband, father and business leader, this question did occur to me: “Do you think some of the people offering online support for the shooter of Brian Thompson ever rated their health insurer with a zero?”

As I mentioned earlier, the NPS metric literally destroyed the information contained in those zeros (or, for that matter, the fact that a lower mode exists and starts building around a scale score of 2). If true, people like Thompson probably never knew they existed. He would have been blind to their plight. All because the good folks who created NPS, in a well-intentioned attempt to build customer feedback into the corporate decision-making process, failed to understand the consequence of placing scale-score data into buckets! They were, as I like to say, data-illiterate.

NPS as a metric is almost worse than not asking the question in the first place. But if organizations that use NPS will switch to using the scale-score data, setting goals designed to increase the average, lower the standard deviation and/or decrease the proportion of low scores, then they may indeed have a helpful metric. Then they only have to eliminate shenanigans.

•••

To close out the NPS discussion, I for one have had many instances where at the end of a business transaction a willing worker (a term often used by Deming to describe the well-intentioned workers who were just “doing their best,” when often that was not good enough because the root causes of poor performance were in the system, and that system was owned by management) would say something to the tune of “You will be receiving a survey in the mail/email/text. I hope you will find it in your heart to rate me a 9 or 10 … .” This is a clear signal that this enterprise is using NPS as a tool to measure employee performance and that there is bonus money or, perhaps worse, employment security involved. This employee knows he or she will receive a reward if you claim to be a Promoter. This kind of shenanigan is likely to skew results at a minimum, and perhaps contribute to customer abrasion.

If Deming was correct, and 95% of performance problems are the fault of the system, and that system is owned by the management and not the willing worker, then creating incentive programs trying to get the willing worker to improve performance will result in either no change or, as is often the case, outright cheating in an attempt to “win” the bonus. I have stopped counting how many times in my career I have seen incentive programs fail or backfire. Instead, senior leadership needs to study the teachings of masters like Deming and learn how to grow their enterprise in a sustainable manner.

Deming was fond of saying, “You can hire a monkey to cut costs!” His point was it takes no particular genius to pull cost out of a system. But as they say in the military, “If you want to cut the fat, be careful not to hit the muscle.” Deming taught — and more than 37 years after first hearing him say this I still believe he was correct — that if you improve quality, costs will decrease because of fewer mistakes, delays, less waste and rework, etc. So start by first learning, then doing the hard work necessary to improve quality in all aspects of business.

Kevin Church, of Hugo, worked for 37 years in business process improvement, including at Optum in Eden Prairie from 2017 to 2021.