“How likely are you to recommend our product to a member of family, friend or a colleague?”

We often see these surveys in emails or at the end of a service contract almost always. These questions and surveys are asking for future behaviors of the user and these scores heavily contribute to the overall NPS.

As indicated above these questions are asking for a future behavior from the user, they are lagging indicators as the action and event has already ended and there are almost zero remedies to correct at this stage. More importantly the answers are graded over a scale. With a score on the left end of the spectrum showing a lesser propensity than a score to its right, but not quantitative of by how much?

One of the most important and crucial tasks at the beginning of any project or action is to understand how exactly is its success to be measured? In the case you don’t do it already; establishing clear criteria for success, i.e. variables and metrics to evaluate and stand as a bar should be prioritized as it can help determine not just the most important factors and KPIs of a project but also understand the relevance of the scope and the estimated impact of the same.

User Happiness metrics therefore make the most sense when they can be retrieved real time. Usually in traditional methods these metrics are lagging indicators.

NPS are usually composed of these “overall happiness” scores that don’t pinpoint of exactly which component of the product worked best for the user, what moment and what behavior did it cause in the user?

A question like “In the past week that you have used our product how many people did you recommend it to?” and the answer, a binary Yes or No, making the quantitative data from such surveys a lot more actionable and insightful.

In a recent talk by Tomer Sharon of Goldman Sachs User Research team, he advised how eventually metrics like NPS and overall happiness and overall success metrics are looked at by the business end of operations anyway when discussing profitability and future goal setting; however that on a day-to-day basis for pinpointing and reflecting success of the product, these metrics and questions must be asked at every important key action stage and have real time actioned behavior questions.

Most product teams come up with some variation of their own custom made real-time actual NPS related to the following:

  1. Beginning a new user journey (Adoption)
  2. For existing users Engagement during the active usage
  3. Determining the use case product was intended for versus actual capacity in which it is used
  4. Retention (are more users sticking with you, making the action a habit?)

Scores that are recorded at individual stages for desired action/ event and then a composite score normalized over all the categories will still show an accurate proxy of the overall NPS but with the capability of diving down, helping teams and managers identify which feature of the product is really moving the needle in the direction of the goal.

And more than often in me experience, the real-time metric has shown a higher correlation to actual loyalty than the NPS irrespective of the time-frame in question.

If you have trouble coming up with what specific metrics should you be looking at, just ask yourself if the metric measures actual behavior of the user or just their opinion? A data-informed decision versus a data-driven decision.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.