Product manager metrics question with Google PM (YouTube success metrics)

Hi everyone welcome to this Google product manager mock interview, my name is José I used to be a product manager at Google where I interviewed many candidates and i'll play the interviewer role today, with me I have mark, mark why don't you introduce yourself. Hi everyone my name is mark and I used to be a senior product manager at Google for many years I worked on software I worked at hardware and we are thrilled to help you on your journey here José back to you. Thanks Mark, so the question is you are the pm for YouTube analytics, what would you pick as the three key metrics and why?

okay so I am the pm product manager at YouTube analytics and you want me to pick the top three metrics that I would want to look at on a regular basis okay got it Couple questions so one is am I a global product manager or am I like a local regional project manager what kind of product manager is this for YouTube analytics? yeah I think you can pick if you think it makes a big difference for the answer you can pick a vision or you can think global i'll leave it up to you. Okay well I like to think globally so i'll go with globally.

okay great second question in terms of three metrics that I want to look at are we looking at three metrics that I want to report up to the executive team or we want three metrics that I want to record to like the product health. Yeah I want to focus mainly on your team and the success of your your product. okay okay great okay great so let me think about this for a moment oh i'll get back to you in just a second not how I want to approach this problem okay so when I think about questions like this I always like to start with the mission and then break it down into components and so what i'm going to do is i'm going to talk about Google's mission i'm going to talk about how YouTube sits within that and i'm talking about how YouTube analytics sits within YouTube and then i'm going to go through a model that talks about our goals our actions metrics uh and then do an evaluation and come back with a summary of the answer so let's let's start with actually the mission because I think that's always a great place to start so Google's mission is to organize the world's information and make it universally accessible and useful uh that's important context for how we're thinking about this problem and then when I think about YouTube as a product within Google and a major product it's a know billion plus user product when I think about YouTube i'm not sure of their mission but I i think their mission is probably something like to you know to share the world's videos and show them the world or something to that effect so let's just say that the mission is for for YouTube is to uh you know allow the world to share videos with the world and if I kind of think about that mission that's a very broad mission but then if I think about YouTube itself it's really a two-sided market and it's a market of consumers right these are things that we're all familiar with on a daily basis we go and watch cat videos and videos on how to fix your sync and things like that or music videos so there's a consumer side of the market and there's also a creator side of the market which is the people that are actually producing the videos and so for YouTube to have a healthy ecosystem for Google there needs to be a really healthy ecosystem of consumers and creators and then since i'm the product manager of YouTube analytics right this is a product within YouTube really the the market or I should say the segment or the customer that this product is interested in consumers kind of interesting yes but it's really focused on the creators so really what i'm looking at is I want to evaluate the main tool that creators are going to use on a daily basis to do their videos and so so first let's talk about goals a little bit so overall goals goals behind this product are one we want to create a healthy ecosystem of of of uh of developers who are or i'm sorry of creators who are creating uh videos two we want to have a a rich set of videos that consumers can choose from that are fresh and constant and so forth and then three it's interesting for the creator marketplace it reminds me a little bit of the airlines right you know when you get on the airline and they say hey we know you have lots of choices when you fly today and thank you for choosing southwest right so it's in a way it's very very similar which is that creators actually have a number of platforms now right they have tick tock they have instagram they have snap facebook they have twitter you know it's in a way it's like uber right there's drivers that do uber and lyft and it's non-exclusive and so I think the opportunity for Google is to is to have one all of these videos but second to you know really be like the first stop shop for for these creators so i'd say the third goal is to create preference for creators uh so that they choose uh so they choose so that they choose uh YouTube.

In questions that focus on specific product areas it's important to start with overall mission of the company you can then connect the dots to get to the goals of your particular product area mark did this by defining three clear goals for YouTube as a company this allowed him to then focus on the metrics that are relevant to these goals. Okay so there's our three goals now let's take a look at actions so what kind of actions are we looking for for our creators as they're going through this process I think there's about four or five actions here and i'll just kind of walk through them i'm just thinking about the creator flow right first you've got to sign up right step one step two after signing up right on a habitual basis they've got to sign in and I imagine users are going to want to sign in on a regular basis three they're probably going to be uploading videos and getting content live to their customers four they're probably going to be using things like insights and analytics right this is the you know YouTube analytics product so they're probably going to be wanting using insights and analytics about how consumers are using their videos how they're viewing their videos where they're seeing the videos what geographic regions are the videos too long are they too short and lots of things like that and then the fifth element is is monetization I think this is an important part here too because YouTube has created a two-sided market and as part of that two-sided market uh creators love creating videos i'm sure but you know a lot of this is it's big business and so they want to make sure that they can monetize their users so they're not creating content for free and so they've got to be able to monetize them as well okay so we've got our goals we've got our actions let's talk a little bit about metrics so on on metrics what i'm going to do is i'm going to walk through each of these actions and talk about a couple of metrics which I think are appropriate that i'd want to watch in a regular basis and then what we'll do is we'll prioritize these metrics according to our our goal and our evaluation process okay so signing up our first action what kind of metrics are we going to look at well one we're going to look at new users right probably look at new users on a daily basis monthly basis this is probably a chart we just want to look at just to see that new users are coming out of the platform and we want to we want an evergreen source of new users in general the second metric I want to look at is the onboarding funnel right and this is a classic funnel of if there's four steps to the process you know one two three four where are people falling off in the funnel and how do we optimize that so that's going to be a metric i'm going to look at signing in uh this is the next metric signing in I think is a really useful metric because it it really tells me like how many people are engaging with the product at the highest possible level right and see these are metrics like dau uh daily active users weekly active users monthly active users this gives me a good idea as you know like are people logging in and using the product on a daily basis and sign in as a great proxy for that next is uploading a video some metrics that I want to look at around this is a number of new videos right so I think YouTube as a platform is as long as there's uh there needs to be a constant stream of new videos so it's probably just a raw number of new videos which I think is an interesting metric but I also think there's another interesting metric of videos per creator right so if you have you know to draw an absurd example if you've got one creator making a thousand videos and then a thousand creators making one video that's not going to make for a very rich ecosystem so i'm going to probably want to take a look at some kind of percentile distribution on videos per creator to make sure that i'm developing a rich ecosystem and encouraging people who aren't creating the videos to make more and so forth next piece is about insights right so again the insights are a big part I imagine would be a big part of the YouTube analytics framework and so this is helping creators you know discover users see what users are watching videos how long they're watching videos and things like this and so the way I think i'd look at metrics here are probably on uh like inside views right so if i've got different pages with show analytics i'm probably looking at inside views you know and again you're you know DAUs, MAUs metrics like that and then also probably interesting is kind of like what's my most popular insights right so if i've got you know a dozen or two dozen insights uh what i'm going to want to do is see which ones are the most popular and put some continue to put weight on those things and make sure that they're relevant and then the last piece I want to look at actually second to last is monetization and the key metrics here i'd look at are average monetization right you've got uh making a billion dollars a year what's our average monetization per user of course averages don't always tell you a lot I want to look at total monetization there's a health in my business that's probably more on the YouTube side but I think what's more interesting here for the health of the business is to come up with a kind of a histogram kind of a p-value monetization and what i'm looking for is probably like a classic 80 20 model where you've got probably 20 of my creators creating 80 of the revenue but I also want to make sure that my other 2 my other 80 of my creators are also making some revenue too right uh and I want to make sure that that's a a healthy ecosystem if you're giving too much money to the uh to the uh you know strong creators that can that can basically uh remove incentives from the smaller creators and vice versa and so you want to create a really healthy ecosystem there and then the last thing I want to look at I think is your nps score i've been a big fan of nps scores and I think for a richer product like this I think nps score is good and nps is your net promoter score it really tells you you know would you recommend this product to a friend and I think this is really good for for creators because for a couple of reasons one is it gives me a good score whether I know if it's useful or not the the second thing is that on at least the nps surveys i've done I always ask for comments and I get a lot of great attractors and detractors and that ends up being content for features I want to add or maybe features that I want to improve uh in the future. Framing the user journey as actions is a good way to remain user focused when choosing metrics like mark did here he described what creators care about in each step and then connected reactions with specific metrics with a great balance across quantitative and qualitative metrics.

Okay so i've just listed a whole bunch of uh uh a whole bunch of metrics let's go into an evaluation but yeah you have a question yeah one question on the 8020 yeah why do you think it's so important for you to that 20 is successful right like what is the your rationale for that? well it's a classic pareto curve and and all businesses tend to shape to this and really what i'm looking for is a healthy ecosystem right and even in evolutionary biology you have healthy ecosystems and what i'm looking for is that what i'm looking for is that to make sure that I have a healthy ecosystem and here are some signals of what would be an unhealthy ecosystem right so again if I had an ecosystem where you had let's say even 10 creators that were making 99 of their revenue right and I had 10 000 creators that were making one percent of the revenue that's actually not a healthy ecosystem why because the other 10 000 creators are gonna be like this this is terrible i'm not making any money i'm not going to put my videos here because it doesn't matter right so that's one extreme and then you could have another extreme where you have lots of creators making a little bit of money and they're great but you notice that your large creators start to attract right and maybe they're not making enough money and maybe they're like yeah I used to post on YouTube but uh the fact is I get a lot more money from TikTok or I get a lot more money from Snapchat or Instagram so i'm just you know YouTube whatever uh you know it's not my choice of preference so I i think this this kind of paredo curve this histogram of monetization distribution and usage in general is is a good signal for the health of the overall ecosystem and I think it's really important okay thanks. Answering probing questions is part of the process you should be open to change while defending your point of view here mark defended this point about the importance of creative diversity by giving good examples to illustrate it.

Okay you bet so let's go back to our let's go back to our goals here so we talked about goals about one you know having a healthy ecosystem and two making sure that developers or creators really like this platform and use it and so if I think of all the the metrics we talked about I i think that the the three metrics that i'm going to look at that give me probably the best indications of health are one probably just signing in right I think kind of your uh your your DAUs, weekly active users monthly active users I think that's going to be a barometer that i'm going to look in day in day out week in week out I think that's a key metric the second thing i'm going to look at is monetization because I think this is going to give me a good idea of health in the ecosystem and again this is where we talked about this this histogram and so I want to create this healthy ecosystem where we know that creators are making the right amount of money across the ecosystem and are broadly satisfied and we're creating a healthy business environment and then the third piece i'd look at I think are that is that nps score so this is the more qualitative side although it can be quantitative but this is a qualitative side that I want to make sure that we have a rich product that creators use and love and then also we have a good input stream for all the things that we know that they like and all the things that we know that they don't like so that we're constantly adding new features that are relevant and we're constantly improving features that may not be working to their expectations okay and so there we have it. It's important to finish strongly by summarizing how your shows and metrics will help you measure progress toward their goals mark picked a comprehensive set which allows him to track growth monetization and user satisfaction. Let's imagine this scenario like where the monetization is actually going up and the sign ups or the daily active users and I assume that the active users yeah the relatives is going down what what what do you think is is potentially happening there.

oh interesting so daily active users is going down but monthly actives is not. Right our weekly active is not so daily is going down. Yep your dollar yeah your your revenue dollar is actually going up so your your ads total dollar is going up but the daily activities is going down.

Got it got it so that could be so that's why the health of the ecosystem is really important that could be a symptom of basically your of your large creator right by large what I mean is creators that create lots of volume that most people like so your large creators are probably continuing to create their content which people love which is why you're seeing your revenue uh increase right but I think but they're they're a smaller set of users right and then you've got a larger set of users which is your long tail creator uh and perhaps they're not making the the revenue that they want or perhaps they're not engaged and so perhaps they're disengaging from the platform right so that would account for your revenue going up your DAU going down uh and this could represent a sickness in the overall health of the ecosystem so to counter balance that you may want to look at strategies like hey what can we do to incentivize more small creators right be either joining the plot new new creators joining the platform so you have your creator creator creation engine so you want to have new creators joining the system or going back to creators that maybe show have a traded or show signs of attrition to try and attract them back and maybe you want to offer them a little bit more monetization or an incentive to upload a video or something like that to uh to bring that back. Yeah I think that that's that's a very valid hypothesis let's say that that your actually your health of the ecosystem is not your period to analyse actually it's quite healthy so the that increase in revenue is happening across the board but still you are seeing DAUs like the AUS and that when I say DAUs it’s DAUs mostly of the consumers uh watching the videos going down but the revenue overall revenue of the business going up. Got it that could be interesting so in this scenario you're looking at basically your revenue is going up your your creators are engaged but you're seeing that you're that the overall content viewership is declining right what do you do right and this is probably something that uh there's probably two forks to this path right uh so one is i'm the product manager of YouTube analytics tool right so i'm gonna observe this the first thing i'm gonna do is probably pick up the phone and talk to the product manager who is the product lead for the core YouTube app right or somewhere in that somewhere in that group i'm sure it's a large group and say hey i've noticed these details is there anything wrong in the consumer generation space right and understand like is there anything wrong in the visitor space so that's one that's one fork I would take just to just to drive that just to drive some understanding there the second thing i'd look at is the ins the insights right so uh we have insights within YouTube analytics and so there might be some observations and learnings to be made there maybe the decline is geographic right maybe like you know maybe all other markets are doing great but brazil is tanking right and brazil is tanking because right maybe you know maybe there's a a new uh a new player in brazil or something like that and people are just you know you know uh flocking to that platform instead right or you know maybe there's a war in uh maybe there's a war in ukraine and all of a sudden people in ukraine can no longer view videos and it's you know 40 million less users you're seeing in a day and so you know your things are going down so it could be geographic it could be and also could be insights right it could be like look people are no longer interested in cat videos dog videos are the thing now it's all dog videos puppy videos right so that also could be a you know a trend you know that an individual creator might not see but as YouTube analytics we could see and maybe letting users know that like hey the hot new content now is uh you know dancing puppies uh unless unless the cats so maybe make some videos there maybe maybe that could maybe could be a content problem.

In metric exchange questions finding the right reason for the change isn't really the point the key thing is to show that you can think of multiple internal and external scenarios and be structured in your thinking be prepared to adapt your approach like mark did when it switched from creator-facing problems to user-facing problems as I gave him more information okay mark that that was the end of the interview well done uh thanks for for the input you did a great job. Mark used the GAME framework to good effect in this interview it helped him stay structured as he developed an answer that was user-centric and very comprehensive he also dealt well with follow-up questions showing himself to be open and curious to different perspectives while using good examples to defend this point of view. I hope this interview was was helpful for people watching, if you are preparing for the upcoming Google interview both mark and I are coaches on IGotAnOffer so feel free to book a session with us there.

all right thanks José for the interview and thoughtful questions and also thank you everyone for joining I hope you learned some useful skills today today and again uh please feel free to reach out and contact one of us for help thank you.