Since it was created, the net promoter score (NPS), which measures how likely a consumer is to recommend a given retailer or brand to a friend, has been a cornerstone of measuring brand strength.

But despite its popularity, NPS is not with­out its detractors. Academics and market researchers have questioned whether it is a good predictor of growth and whether it really captures a brand’s relative trajectory when taken in isolation. Some have also challenged the basic math behind net promoter because different sample sizes of positive, neutral and negative consumer sentiment can all generate the same score even though the underlying dynamics may be dramatically different.

Forever 21 presents an interesting case example on why it is potentially misleading to put too much weight on just one metric, even when that metric is NPS. In several recent studies, Forever 21 had fallen to near the bottom of the range of net promoter scores for women’s apparel retailers.

This is, quite frankly, a stunning result, given that Forever 21 is obviously a wildly successful retailer. It has grown its store base by a combined annual growth rate (CAGR) of 7% from 2007 to 2012, far above the average of 2%. Its brick-and-mortar sales grew 29% during the same time period and its e-com­merce sales increased at a CAGR of 21%. Plus, during our ethnographic studies—one-on-one detailed interviews with consumers—many consumers gushed about the brand, shopping experience and value. How then is it possible that the net promoter scores have frequently shown up near the bottom of recent surveys?

Problem 1: Sampling and getting the right mix of consumers in the survey

Classic net promoter methodology would suggest that by taking a sample of the broad population aware of the brand, you can get a reasonable and reflective score. Obviously, this creates major sampling problems because the score will be driven by the size of different cohorts within the survey. We have consistently seen that those who have interacted with any brand more recently will score it higher than those who have not. The recent election showed the problem of polls with the wrong mix of likely voters—as Nate Silver will attest. Even when the survey focuses just on “ever” or “current” purchasers, this problem still exists if the comparative brands have different sample sizes of recent or more distant shop­pers. One obvious rule is to never trust surveys unless the sampling is in the thousands.

Problem 2: Cult followings

In many studies, we have found brands with a “love it” or “hate it” following. In a broad net promoter score approach, the brand may well score poorly, but within that score there is a cult following that adores it. Historically, brands like Juicy Couture and Abercrombie & Fitch have experienced this, even during their boom times. Getting the right segmentation is critical, as we discuss below.

Problem 3: “I buy it, but I would never recommend it.”

This problem has shown up for multiple retailers including Walmart and Forever 21. Here consumers find incredible value, but when asked if they would recommend the store to friends, they often give it a mid­dling score because there are aspects of the shopping experience that they don’t want to recommend to their friends. For example, in one-on-one discussions with shoppers at Forever 21, we often heard comments like, “I love them, but I would not recommend them strongly to my friends because some­times it’s hard to find the right fit or get any help and they might not like that.” In this case, the very nature of the question triggers a con­sumer score that is lower than how they really behave—in part because of the distinct and highly differentiated nature of the retailer.

So what are the solutions to these problems?

We have used NPS for years and believe that it is valuable, but have always advocated looking at brand strength and consumer sentiment through many different lenses and using segmented NPS scores in conjunction with several other evaluation metrics to get a truly accurate picture.

Clearly, the way a survey is written as well as how the data is analyzed can have significant impact on the value of the NPS. There are four key dimensions to consider when deciding how much weight to place on a brand’s NPS.

1. Competitive set. Whether they realize it or not, consumers’ answers often change relative to the set of retailers they are asked to rate. 55

For example, in a set that included mostly higher-end women’s apparel brands like Coach, 7 For All Mankind and Ralph Lauren, Forever 21 ranked second to last with 35% NPS. But in a set that included mostly teen and discount retailers like H&M, American Eagle and TJ Maxx, Forever 21 ranked first with NPS of 45%. (See Exhibit 1.)

zoom icon

Focusing on current purchasers and, specifically, frequent purchasers will provide a more accurate picture of how the brand’s most important and profitable customers see it.

2. Brand interaction. Measuring NPS for everyone aware of the brand doesn’t get you much because it includes many people who have never truly interacted with the brand, but examining NPS among “ever-purchasers” (current and lapsed consumers) can give a broad snapshot of sentiment among those who have actually experienced the brand. Focusing on current purchasers and, specifi­cally, frequent purchasers will provide a more accurate picture of how the brand’s most important and profitable customers see it. Of course, it’s critical to have a large enough sample to ensure that the different cohorts reflect the general population mix.

3. Demographic and psychographic segmen­tation. Forever 21’s core customers tend to skew younger and to be female and lower to middle income. So it follows that these segments report higher NPS for the retailer. For example, in one survey, Forever 21 scored 41% NPS among those earning under $50,000 annually, but only 16% among those earning over $100,000, highlighting the importance of testing NPS against the brand’s target consumers. Adding psychographics into the mix helps clarify the results even further. For example, women we identified as “fashionis­tas” and “price-conscious” rated Forever 21 twice as high as those who said they were “basic shoppers.”

4. Usage occasions and product categories. Layering on product and occasion details can shed light on a brand’s strengths and weaknesses when it comes to assortment. For example, Forever 21 is more highly ranked for accessories (51%) than shirts (32%), owing to its massive, prominently displayed accessory section. And, as would be expected, Forever 21’s NPS was lower for work (22%) and casual (26%) occasions than for going out (34%).

While these four filters help ensure NPS is focused and a more accurate reflection of a brand, we like to think more holistically about measuring brand strength through a mix of consumer surveys, digital chatter assess­ments, ethnographic studies, shop-alongs and trade interviews. Together, these tools can help uncover a more complete picture of a brand’s strength.

  • Overall awareness of the brand
  • Overall conversion rate and by occasion and category
  • Psychographics such as whether the brand is favored by fashion-lovers or frugal consumers
  • Focused NPS
  • Head-to-head comparisons between a brand and its primary competitors 57
  • Whether consumers identify a brand as “for them”
  • Likelihood that a consumer would repurchase from that brand
  • Level of positive buzz about the brand in the market
  • Strength in certain key attributes like fit and quality
  • Social media chatter about the brand

The brand strength gauge (see Exhibit 2) shows that Forever 21 is clearly a hot brand, which is much more on par with its historical performance and growth trajectory.

zoom icon

So, when evaluating a retailer or brand, go beyond a basic NPS or risk missing out on the next Forever 21.

8 March 2013