Webinar: Product KPIs & Metrics That Every PM Should Know by Zalando Sr PM, Zafeer Rais

hello everyone uh good morning good evening and welcome to the uh welcome to this webinar by product school on product kpis and metrics that every product manager should know uh before we jump into it a quick background about myself uh well my name is the field race I'm a senior product manager at zalando uh which is uh the largest and leading fashion e-commerce platform in Europe I'm based out of Berlin and before zolando I've been associated with different domains as a product manager in edtech I worked with future skills Prime which is uh an initiative

sponsored by the government of India for the ID upskilling uh in India I've also worked for companies like Cornerstone in the past uh before we jump into the uh the discussion on kpis and metrics uh let's talk about the synergies between data and product managers uh well if you're building a product without data uh I wouldn't call it a product it's only an opinion because most of the things would start with I think XYZ feature would work I think uh this XYZ feature would not work uh well as product managers we should not be talking

in terms of I think we should instead let data do the talking uh if data is talking on your behalf it makes your life easier uh to convince the stakeholders to get buy-in from stakeholders uh and uh you know it it just leads to Fever arguments uh of product managers between different stakeholders uh we all might have heard about uh this quote which says data is the oil of 21st century but Peter sonengard has taken it a step further and he calls analytics as the combustion engine ah this is beautiful uh because just like a

combustion engine cannot work without oil and oil is is more or less useless without a combustion engine uh product manager should not be working without it or rather can not work without data oh that's how critical data is for product managers uh always strive to be data driven product manager rather than an opinion driven product manager and I think that's what uh that's what the difference between create and good product managers lies will data will talk uh data will definitely talk uh uh if you're if you're willing to listen to it uh and uh uh

if you toss your data long enough it will tell you three secrets of how to make your product successful as I said earlier uh uh uh you know building a product without uh data uh you're only building uh you're only making an opinion and continuing to build the product without data is the biggest crime in product management uh data and finally data is a product manager's best friend as I see it uh what are the tools to capture I think this is not really important uh don't get me wrong uh these are great tools Google

analytics amplitude mixed panel I've used them myself these are great tools and it will present you with a ton of data however uh it is useful uh only if you measure uh only if you use them properly only if you measure the data that is coming out of it uh hence everything comes later I think we first need to be very clear as to what we are doing with data uh make use of kpis and metrics to decide if the hypothesis that you're working on uh is going to create the right Reef the wave that

you intend it to create to have okay uh now coming to kpis and metrics but what are these kpis and what are these metrics that we keep talking about uh well kPa stands for key performance indicator uh and you have three very important terms that come in uh uh over here which is the first one is key which signifies how important uh you know a kpi is it so it shows the importance of this thing that we are trying to measure the second one is performance uh so it's measuring performance probably over a given period

of time uh because performance needs to be measured over a time period so hence it measures your performance uh an indicator which signifies that it's quantitative in nature which also means that there are a lot of numbers uh that you you need to work with when you're talking about kpis and metrics uh so it's a quantifiable measure of performance that demonstrates how effectively you are achieving your main strategic objectives on the other hand uh metrics are uh metrics of measurements of specific activities or processes they're more tactical in nature uh there's something that that are

measured probably on a daily basis and uh there's some sorry there's something that are measured on a daily basis uh and they are more tactical in nature this is important every kpi is a metric however every metric might not be a kpi uh it's again it's it's it's uh it's a superset and a subset kind of an analogy where in a metric is a superset and a kpi can be a subset of this metric superset uh I'll take a good exam example of uh you know which might help us I understand the difference between a

kpi and a metric uh let's take an example of uh Netflix at this point of time now we all know Netflix is struggling with uh with uh with a low subscriber it many of its existing subscribers are dropping off uh and that's where the problem of customer retention comes in so customer retention is probably one of the most important kpis that uh that Netflix is measuring uh currently and how is it measuring uh it's making use of the return rate uh metric uh or probably The NPS metrics to measure the customer attention uh now hoping

uh this uh different is clear uh and now uh we should be good to jump into it and look at the 10 product metrics and kpis that are extremely important for product success uh now these metrics could be across products uh uh it is not necessary that all of them apply to your product it also it is also possible that none of them apply to your product uh it is by far by no means this is an existing exhaustive list of kpis and metrics you know you can have n number of metrics that your your

measuring uh why the number 10 I don't know probably because I like the number 10 uh and according to me these are 10 important uh metrics that probably you should be measuring at your risk uh it also gives you a good uh Head Start or uh you know or an entry point into exploring these uh kpis and metrics and how you should be working with them cool having said that uh coming to daily active users and monthly active users uh uh well uh you know you know this this should actually actually be active users uh

and and we'll see uh what that means now the end goal of any product is to generate money uh products are owned by companies and and companies out our year with the intention of generating profits and earning money and being more profitable uh however to generate money the primary focus is to get users on the platform you may not be able to generate the expected uh revenue from your product uh or your company if the users are not coming on the platform hence that means that is important to hear and that is why the active

users uh metric uh however we do not want any user on the platform uh there's a very specific definition of what you call as an active user active users are users that perform some valuable activity on the platform right now this valuable activity would be closely linked uh to the to the DNA of the product for example watching a video might be uh uh you know the quality of an active you saw on YouTube sharing pictures is something that an active user would do on Instagram similarly reading a Blog is something that an active user

on medium.com would do foreign like monthly active users or weekly active users depending on the product and the requirement that it has bue Dau generally is used uh for mobile applications uh because mobile application is something which which is a very personal device you carry it on a daily basis there are multiple times when when you have the mobile in your hands and you're accessing different applications hence uh Dau is is mostly used to measure uh uh a mobile application products online games and social networking sites because their usage is more on a daily basis

however this is not limited and it can be extended to other products as well uh the ratio of daily active users upon multi uh uh monthly active users is useful in identifying the stickiness of the product so how do you define stickiness of the product it's the ratio of Dau divided by uh the Mau uh uh us uh a stickiness of 20 is is normally uh considered to be a good stick in a stickiness score uh however a stickiness of more than 20 percent is generally considered to be a very successful product uh stickiness also

allows uh uh to track growth uh it's an important measure to track growth or also to track the decline of the product uh if it's going up or down in either case uh not every product uh should be measured on uh on the basis of these metrics uh because not every product needs to be used on a daily basis to be successful a good example of that could be Uber uh I may not use Uber every day but I use Uber uh or let's say on on a Friday night when I'm out drinking and I

don't want to drive my car that's when Uber comes in Airbnb uh it's a great product but I don't use it on a daily basis I would be using it um maybe once a month or once a quarter when I'm when I'm off to vacation or where I want to book uh you know a state so not every product needs to be measured on a daily basis uh what is daily active users well it's it's a number uh so there's no formula but it's the number of unique users visiting your application during a predefined one

day period uh the next one is uh monthly recurring Revenue uh monthly Reckoning the recurring Revenue simply put is the income that a company can expect to generate every single month now it measures an organization's Financial Health uh it's it's really important for SAS business models uh which are based on on subscription scheme uh so you know Amazon Prime Spotify uh Netflix Etc which work on on a subscription model uh this becomes an important metric to be tracked uh however however on a standalone basis it might not make a lot of sense to a product

manager uh it needs further analysis right uh it needs further analysis because this can help you identify where other customers dropping off and eventually it will lead you to also understand the why of it why is this happening so uh in order to increase the mrr of a product uh you know it's something that should uh reside with the product managers uh the different types of Emira mrrs are a new mrr expansion mrr John mrl and a net MRI uh what's the formula the formula for MRI is monthly average revenue per user into the total

number of users uh in case of B2B companies users can also be replaced with businesses or accounts uh that you are catering to as a product the next one is customer lifetime value or CLTV CLTV it also uh it allows you to determine the amount of Revenue that can be generated from a user or account in the long run right uh in the long run would mean the lifetime of the user uh again let's take an example of Netflix CLTV displays or defines the average profit generated from one user before they cancel their Netflix subscription

now how does Netflix Define the lifetime of uh of of a user uh take an example of me uh if I'm paying 10 euros per month for my Netflix subscription and I keep the subscription on for a period of uh 12 months that is one year uh the my my total customer lifetime value would be 120 euros in case of uh for Netflix similarly uh it is also calculated uh interesting thing to note is that uh Netflix has one month as uh as a free trial uh the first month so that also needs to be

taken into consideration where I'm calculating the lifetime of the Euro so in this case my lifetime is 13 months and not 12 months uh as I said average lifetime is till the time the user is a paying customer of the platform uh well it aims to identify how much you should spend to attract a new customer uh at an uh you know at an early stage uh based on the profit generated from the user uh this could also help companies like Amazon Prime identify that what should be the cost of the Prime Membership considering that

they are offering the first month as free uh so uh so the first industry could also be taken as a customer acquisition cost which we will look at later but it's a it's a good measure to identify uh how much you should be spending to attract a new customer based on the uh on the profit or uh the value that the customer is generating at the end of the period uh again it helps you identify the right customer acquisition channels uh where should you be spending the money uh is it on social media campaigns or

uh maybe television at business or print media Etc uh or should you be giving marketing schemes and discounts and offers in this case uh what are the purchasing channels that you should be using should you be spending money on Google ads uh and what other retention strategies that that could be used I remember for that matter when I when I canceled my Netflix subscription for the first time they came up to me with with an offer that hey we're giving you two months free please take back uh with that platform so that's one of the

retention strategies uh the formula for CLTV uh well it's the average revenue per user into the average lifetime uh of users on the platform the next one is uh customer acquisition cost uh now what is customer acquisition cost well it's something which is mainly used uh uh by uh uh product marketing teams uh what it does is it computes the cost of acquiring new customers including uh marketing spends uh advertisements sales expenditures Etc so if you if you look at its formula it's it's it's everything all the expenditures combined the salary is the overheads the

marketing spends Etc uh upon the number of new customers that you've managed to acquire enough of PMS for product managers it is important to know the values as it complements other metrics uh how does it complement other metrics well you know it can also help you uh identify what's the uh what's what should be the free period in case of a 15 model should you have the first month free uh how should you how should you be taking in the money uh is it that I do not provide any credit card details uh and you

know you enjoy the first one for free or you you take my credit card details because I will not be charged the first month but for the next 12 months I will be charged all I will be charged for the entire 13 months out of it the first month would be free so uh you know in this way it it helps product managers to understand how it complements other metrics and how can I improve the other metrics will an ideal CLTV to CAC ratio is 3 is to 1 that means uh uh the the the

amount of value that a particular customer is generating should be three times the cost that is spent in acquiring uh the customer uh less means you're spending too much so if the ratio is less that means if the denominator is is higher uh so let's say in this case it's three uh uh the ratio is three is to two this basically means that you're spending too much money in order to acquire new customers and those new customers are not generating the expected value uh that we thought they would derive uh whereas spending more means you're

spending too little and you're missing out on new business opportunities uh let's take us here let's take a cltb to CSC ratio of five is to one this basically means that uh you know for for every you every dollar spent uh in order to get the customer you are generating five dollars worth of business uh it also means that you if you spend more money in acquiring new customers uh the the total revenue generation could be higher in this case because you know by just spending one dollar you're you're making a business of five dollars

so you know you might want to uh increase the denominator so that uh ultimately it increases the numerator in this case as well formula it is the sum of all costs involved in customer acquisition efforts upon the total number of customers acquired as a result the next one is uh the session session duration uh well it's one of the easiest metric to track for digital product usage and what is it uh it calculates the session duration of a product of a group sorry of a group of bounds or churned users and this helps me uh

identify how to improve user direction right uh user user interaction my bad uh uh it leads you to it helps you to understand what made the users turn uh now this could be a certain page uh this could be something very annoying on a particular page so the average amount of time that the user is spending on the platform in one session uh will give you insights on on on uh these metrics and how to fix this uh the formula well it's the total time spent by users on the product divided by number of users

bounce rate uh bounce rate is again uh it's a very easy metric to uh Target uh to track uh Google analytics personally is is one of my favorite and best ways to measure bounce rate it also happens to be the cheapest since Google analytics is free now it helps product managers identify any page where the users bounce off without spending a lot of time uh in my experience uh bounce rate really helped me tackle one problem where my sign up form was really long and and I saw that uh users are bouncing off from that

page and that helped me shorten the form optimize it and reduce the overall time required to fill the form uh that that's the way in which bounce rate has helped me improve uh one of my uh product apis uh it's also a hint that something is wrong uh well if you have different entry channels into uh into your product uh let's say some of them on the home page some of them some of them land on the about us page or some of them land on the sign up sign in or sign up page right

if you have different entry channels into the product uh it gives you a hint that something is wrong with a particular entry channel uh because the product is the same the functionalities are same why is this happening uh that users are bouncing uh from this particular page uh this could be a hint that something is wrong and need a fix it it needs an urgent fix uh it actually impacts other metrics like uh Dau or mrr as uh if the use if the users are bouncing off from a platform uh well you you will not

really have a good Dau or you would not be you would not be able to generate the revenue out of that so it's easy but it's a very important metric that needs to be solved uh that needs to be tracked what is a bounce rate well it's the ratio of User it's a ratio or a percentage of users who visited uh your your product or your page only uh you who visited only one page of the application before the leaving the application so out of 100 users who come to your platform if 40 users uh

just drop off after visiting one page your bounce rate is 40 percent uh the next one is customer retention uh this is a rather important one uh it is the ability of your product uh to retain customers after a certain period of time uh let's take an example of Netflix again uh it's been off hybrid while taking examples but uh in case of Netflix uh how many users uh remain or are retained on the platform after the one month of free subscription that ends now that number would would give Netflix an insight as to what

is their customer retention rate uh it's it's actually a parameter to measure the stickiness of your product uh according to big spannel uh average crr for most software products is below 20 uh over a period of eight weeks so that's that's pretty low uh what is it it's the percentage of users who prefer to stay with the platform uh it could be you know for whatever time period if Netflix wants to see uh what's the custom acquisition cost it would take the time period as one month when you have the free subscription to the platform

uh it defines uh now what is important over here is you need to Define what a returning user is and what your desired time period is for example is a mere sign-in enough uh to consider a retained user uh well um wouldn't be the case in most applications but uh that's something that you you really need to identify what do you define as a returning user just just the way uh you know we Define for active users a returning user is really important uh is there any user doing any activity or just uh do you

call that user retain just because the user is still there on the platform he you know you might be logging in one month uh once in a month once in a year uh that's something that would differ from product to product uh complicated formula what is crr right so let's say if I'm calculating the crr for a particular month uh what I would do is number of users at the end of the given period so number of users let's say at the end uh of uh July so 31st July what's the number of users on

the platform uh that can be 120 let's say the 120 users minus the new customers gained in the time period so in in the in the month of July I gained let's say 20 users right and what were the number of users at the start of the time period uh well let's say in this case this is 90. so I have 120 users right now 20 users came in uh uh in this one month uh and what were the number of uses at the start of the time period that was 90. so uh this gives

you a a decent a good enough customer attention it and and that's how a customer retention rate can be calculated uh it's almost a 90 customer retention rate in the previous example okay moving on to the next one that is churn rate uh now churn rate is is is the exact opposite uh of uh uh the previous one that is customer retention rate uh it measures the users that left your platform right not retained but left the platform uh it measures the number of users lost in a given time period uh what are the types

of churn uh now the types of churns could be user churn or a revolution when you're losing out on Revenue uh or you're losing out on users in this case uh Revenue churn is obviously uh Revenue churn is an effect of user churn uh if you're losing users on the platform there would obviously be uh uh a churn in the revenue as well uh it makes sense to pay attention to revenue churn rather than customer churn uh because customer churn uh although although customer chin gives you gives you insight into customer satisfaction so if you

really want to understand uh what the customer pain points are how satisfied the customer is with the product you should be looking at a user churn however uh Revenue churn is is is important right because that's where you're losing out on money you're losing out on on your Revenue so I think that that uh deserves uh more Focus uh the average churn rates are less than 10 percent of mrr uh in in an ideal case uh churn end at the low low end of two percent would be considered good so if if you're losing uh

if your if your Revenue turn is is around at the lower end of two percent uh of your mrl that should be a good uh figure generate user churn rate would be the total customers lost upon the total customers uh similarly in case of Revenue it would be the revenue lost upon uh the total revenue that you generate uh the next one uh is The NPS net promoter score uh what is a net promoter score so a net promoter score is uh uh is a survey which is used to measure Customer Loyalty or user loyalty

and satisfaction by asking the users How likely they are to recommend a product on a scale of one to ten so uh to put it short there's a scale of one to ten and I will ask the user how uh did you like my product if yes How likely are you to recommend uh this product to uh to your friends and family I'm sure a lot of you have filled this for um for some product or the other I mean what the what is the product trying to do with this they are trying to measure

the net promoter score uh now net promo uh the respondents are usually classified into three categories uh they are uh promoters they are passives and they are retractors promoters are a loyal product users who will use the product service and actively refer the platform right so if you if you really like something um uh you would you obviously tell your friends and family that hey this is a great product I I highly recommend you to use it uh the fuel organic growth right so you do not really have to spend money to grow your product

in this case it's it's kind of like free marketing honest on the scale they would rate the product in the survey between nine and ten uh passive passives read the product 628 and like the product however they are prone to shift their loyalties to a competitor if the competitor is able to pull them towards uh itself uh whereas detractors read the product between 0 and 6 and in general are not satisfied with the product you know a good example could in this case could be of Apple probably has a very high in net promoter score

because there are certain users who are hard for Apple Loyalists and who would you know who would uh you know be the first in line whenever there's a new iPhone launch or any new Apple product which is blown so Apple for example has a very good net promoter score uh uh detractors can do negative publicity as well uh and this will eventually hamper the image of the product uh again a good example of this could be uh you know the case of Samsung where where phones uh I think it was the note uh where phone

started uh going up in flames and that caused a lot of negative publicity and your passives uh quickly turned into detractors in this case uh NPS generally lies between minus hundred and hundred uh and negative NPS Force means retractors are more than promoters obviously minus 100 would mean all the people that you surveyed uh are retractors and 100 would mean all the people that you serve it are promoters of product uh some stats in 2018 Netflix had an NPS of 64. uh PayPal had scored uh 63 uh and uh 54 53 and 49 for Amazon

Google and apple respectively uh be polite uh uh be polite in timing then uh the the survey uh so it's really important to ask them uh the to ask the survey at the right time right so for example you do not want to ask the user to fill a survey when they're in a really uh uh you know when when you are in when something has gone wrong right so ask the user if you would want to uh refer when they've successfully placed in order right uh do not ask them when they when when the

order order placement has failed that would lead to a bad score also you want to ensure that you're getting genuine responses so try to keep the uh the survey shorter because the longer the survey the the more annoying it gets for the users to fill a long form keep it short keep it short and sweet uh and place it strategically the net promoter score is the percentage of promoters minus the percentage of distracted practice uh now the last one and probably one of the most important metrics uh uh that companies and products would uh right

uh not star metric the not star metric uh is what first of all was not star at the North Star is the anchor of the northern sky it's a landmark or a sky marker that helps those who follow uh mostly Sailors who follow determined Direction uh as it glows brightly to guide and Lead so it helps those who follow uh determine the direction as the star glows brightly uh to guide and Lead uh it is the most important metric of the product true and what's uh how did it start so taking a cue from the

definition of uh not start or entrepreneurs on Ellis uh coined the term not star metric for for the first time in his book uh hacking growth now this was done to reduce the administration around it uh and to have a singular goal for the entire uh company or for a department uh it defines the success of the product or the company so what does success mean for a product or a company uh it's linked to the vision of the company so uh so uh for example Facebook has a vision of uh connecting people and and

making it a more social place so uh it would be interesting to know what is not star metric of Facebook is uh again we'll take an example of Netflix I promise this is the last time you'll be hearing Netflix in this webinar but uh Netflix's vision is becoming the best Global entertainment distribution service thus we can safely say that the North Star metric for uh for Netflix would be session duration as it denotes it users are spending more time on the uh on Netflix because the entertainment portion of Netflix is really high there is no

defined formula for this it's small for strategic uh uh metric which which the team needs to come together and uh have this metric so that the rest of the team can follow it finally uh We've we have gone through 10 important metrics in kpis but uh just wanted to leave you with some thoughts uh this is just an overview uh this these 10 metrics are uh these are these are not an exhaustive list as I mentioned before the purpose of this was to introduce you to these metrics right uh hence uh it's more to help

you inculcate a much more data driven attitude it's not exhaustive there are many other metrics that that might be important for your product but I think this is a good starting point uh again to become a more data driven product manager uh dive into the details of each uh each one of them uh look for metrics that might be relevant to your product and to your organization uh well we cannot do a deep dive in this webinar because each metric would then take a take a separate webinar in itself uh what are the next steps

for you uh so the next steps for you could be identify the metrics for your product and start measuring it uh also you can also think of some popular products uh and think of what they are uh uh not star metric could uh oh sorry what they are uh uh what their meant uh you know the different metrics that you that they measure a good example would be a not star metric and this is something that can also be used in cases of interviews for product management wherein you can ask the interviewer hey what's your

not star metric or according to me this is the North Star metric uh you know is my thinking in alignment so I think that can lead to a good conversation in your uh interview as well finally don't just track uh metrics as important it is to measure and track uh it is also important to make use of the data that is coming out of this and remember uh build and build a product not an opinion uh finally uh I want you to leave the I want you to leave uh leave you with this uh quote

from Mark Twain and he says data is like garbage you'd better know what you're going to do with it before you collect it and that's uh and and that's why the product rules come in later you you need to know your metrics you need to know what you want to measure and why you want to measure uh and that will make your life easier thank you so much for spending time uh if if you want to stay connected you can reach out to me on LinkedIn I'd be happy to answer more questions on this topic

or any other topic in general thank you so much see you later [Music]