How Netflix builds a culture of excellence | Elizabeth Stone (CTO)

we can't really have any of the other aspects of the culture including cander learning seeking excellence and Improvement freedom and responsibility if you don't start with high Talent density and in some ways it's very reflective of Reed Hastings as founder of Netflix so when he founded Netflix and grew the company over time it was with a belief that there could be a different approach to build building a company that would make it a place that people thrived in and loved being and would feel different than other places both in the quality of that Talent density

but even more importantly like the Excellence and the outcomes and that that's where people would derive a lot of sense of fulfillment so it it's very deeply seated at Netflix from its original days and in order to do that you have to really hold yourself to a lot of stuff that doesn't feel like natural human behavior today my guest is Elizabeth Stone Elizabeth is Chief technology officer at Netflix and as far as I can tell the first Economist to ever be named CTO at a Fortune 500 company prior to this role Elizabeth was vice president

of data and insights before Netflix she was vice president of science at Lyft coo at Nuna a Trader at Marl Lynch and an economist at analyst group in our conversation we cover a lot of ground we talk about how an economics background has helped Elizabeth in her career and why she expects to see more economists rise in the ranks of tech companies she shares some of her secret sauce for Rising so quickly at so many companies so consistently we delve into Netflix's very unique culture of high Talent density radical cander and freedom and responsibility we

also talk about the structure that Netflix has for their data and user research teams which she believes is a part of Netflix's secret to success we also get into what biking and triathlons have taught Elizabeth about life and how she brings that into her work and so much more a huge thank you to Ali Ral for introducing me to Elizabeth if you enjoy this podcast don't forget to subscribe and follow this podcast in your favorite podcasting app or YouTube this helps tremendously and I really appreciate it with that I bring you Elizabeth Stone after a

short word from our sponsors this episode is brought to you by vanta when it comes to insur in your company has top-notch security practices things get complicated fast now you can assess risk secure the trust of your customers and automate compliance for sock 2 ISO 2701 Hippa and more with a single platform vanta V's Market leading trust management platform helps you continuously monitor compliance alongside reporting and tracking risk plus you can save hours by completing security questionnaires with Vana AI join thousands of global companies that use V to automate evidence collection unify risk management and

streamline Security reviews get $1,000 off vanta when you go to v.com Lenny that's vana.com SL Lenny let me tell you about a product called sber the all-in-one Communications API platform designed for both web and mobile apps in a world saturated with multi-channel communication product teams are just discovering the effectiveness of inapp communication with sen bird businesses can Elevate their inapp experience with decluttered and branded communication featuring AI powered chat Bots one-way messages chat video calls and live stream capabilities all tailored for Commerce marketing and top tier support forward-thinking companies such as hinge patreon Yahoo Accolade

and more use senir to build inapp communication experiences that drive engagement conversion and retention Ina communication has the highest conversion highest engagement and highest satisfaction of any communication Channel and when it comes to investing in this channel trust sber to take your inep communication experience to the next level start today with sen Bird's free plan and as a listener of Lenny's podcast you'll get an additional 2 months of unlimited usage and access to all premium features including creating your very own generative AI chatbot visit send bird.com Lenny to begin your your free Journey that's sg.com

Lenny Elizabeth thank you so much for being here and welcome to the podcast thank you thank you for having me so when we booked this conversation you were VP of data and Netflix and since then you got a promotion you're now the CTO of Netflix which feels significantly more fancy question for you what is life like now that you are CTO versus vpf data how is it most different I'm imagining more meetings I would say the biggest thing that changed is probably the amount of context switching and the degree to which I feel behind or

I have a lot to learn and I felt like I had a lot to learn in the vpa data and insights role that I was in before in park because we cover a lot of different areas of the business and there's always people to learn from but the engineering organization just takes that to a 100 basically so more people to get to know more problem spaces aspects of technical expertise that I'm just not as deeply familiar with um and yeah a lot more meetings I imagine many higher Stakes meetings as well yes so thankfully not

a lot of meetings at Netflix feel like now you're really in this scary room but it does feel like the role has more consequence which is actually an exciting thing kind of along the lines of what you just talked about being a CTO your background is actually unusual you're a trained Economist you got a PhD in economics and from what I can tell you were the first CTO of a Fortune 500 company that is an economist trained in economics first of all is that true I don't know if that's I think that's true but you

tell me I have not checked the list that was one of the things I did not do after getting the title I might be unusual I've heard a lot of feedback on that so I don't know if I'm the one and only but I will definitely it's probably unusual yeah so I guess the question is just do you think this is an anomaly and going to continue to be really rare do you think this is some we're going to see more at tech companies and generally do you think tech companies should hire more economists yes

to the last question that's the easiest of it but one of the things I observed even with the focus on data science where I've been deeper for a period of time is that economics is a flavor of data science so it it's a set of tech technical skills for sure it's a way of framing certain problems or solving challenges and so when I was first switching from economics into Tech it was before there was a lot of sort of the frenzy around data science that we've seen more recently and it was harder to make that

argument that economics is a flavor of data science and maybe complimentary to other versions of data science and I feel more strongly about that now that I've seen it up close and so maybe by extension I would say just like I think data science can be helpful for a lot of different problems that you might not immediately think oh this is something that we should bring data to I think that economics is generally valuable for a lot of different challenges and it's a useful perspective to add to things especially in a business context and especially

in how we want to simplify problems in a way that makes them feel tractable so I feel like that's been a benefit to me to have had that type of formal training and then bring that perspective or way of thinking to different roles and so I I don't know that many people at Netflix think of me as an economist but I I find it comes out in the way I think about things so to the extent that that's true generally I think it's useful in a lot of companies and I feel like even since I

made the switch towards Tech I've seen it become much much more common to think about the value of having Economist on teams just to pull on that threat a little bit more is there something very tactical or conrete that you can share that you find helpful with that background that you found helpful in your career other than the Dismal dry science of it all uh so so one example would probably be in an understanding incentives and thinking about unintended consequences and I think that that is true both in terms of internal leadership so being part

of a management team that's thinking about how do we clarify priorities or motivate a company or Define the problems we want to solve and then part of it is more externally oriented how do we want to think about what Netflix is to Consumers and how we want to think about competition and there can be a rational way of thought which is one version of Economics of like shouldn't rational intelligent people behave in the following way and then there's the well if given certain incentives what might you see that we didn't think was optimal or we

weren't expecting to happen but could be a consequence or repercussion here and so I think that that type of framing I don't know if it's Unique to economics in a way you know because it has elements of psychology to it as well um and kind of planning ahead has become really useful for thinking through kind of cause and effect so that has come up in a lot of different spaces in Netflix and in other roles I've been in I was looking at your LinkedIn and looking at your career over the years and it seems like

you've had a meteoric rise at four different companies and I'll just walk through them briefly so your first job you went from associate to vice president in three years at the next company Nuna you went from manager of data science to coo in two years at Netflix you went from VP to CTO in three years I think that's really rare I'm curious what is your secret sauce to being so successful at so many places and especially in the context of what advice can you share with folks earlier in their career this is one of those

questions that Sparks the reflection that I wouldn't normally do so that's great um I really don't think of it as a secret sauce but maybe I can walk through some of the things that I think Have Been instrumental as you listed that out it's like it sounds like the two to three year point is the real sweet spot um so then maybe there's something about that timeline but I I think some things almost feel trit in how I would say them which is I'm very dedicated to the work and to the teams I'm part of

it's been part of who I am for a long time that I give everything I've got to the job that I'm in and I think that dedication and that I get joy out of that has mattered it matters because I enjoy what I'm doing I do the best work I possibly can less so for myself and my own ambition and more so because I I think of myself as being part of a team and so I really need to deliver for that team and I think that framing in my mind and that motivation has helped

me in a few fr fronts which is the way in which I build Partnerships with people I work with that I really care about setting other people up for success and being someone that people want to work with so I learn from them they learn from me and we get better outcomes for the business together I have found that part of that is being someone that people can leverage to translate from technical to non-technical and non-technical to technical so that I do think has been a relative advantage in my role so while I was often

sitting in more technically oriented teams a lot of the advancement in my career was to roles that required that type of communication fluency and it grew out of being able to partner with people across the businesses who didn't necessarily have the same background but where we needed to really connect spaces so that we could be more effective and and that was something that I I think really the training for that came from analysis group where it was a very quantitative set of work that we had to find a way to communicate to judges and juries

for economic cases so that was something that was trained in other roles and I think I've been able to leverage I'm a relatively introverted only child so I observe a lot uh which means that I learn from other people uh and in each of these roles I have tried really hard to watch what other people are doing think about like how could I learn something from them whether it's the thing that I want to be able to do myself or it's the thing that I think oh that doesn't quite fit or feel authentic with my

style and I do a lot of that introspection so I I have been surrounded by amazing people in all these roles and I have a feeling that I learned a lot by osmosis and observation and then have been able to leverage that to be stronger in the roles I was sitting in so I took a few notes here so a few things you mentioned is just like dedication essentially working really hard and taking your work seriously being part of a team and setting other people up for Success translating complex Tech language and problems to non-

tech people and then being really good at observing and learning from other people around you is there an example or two that you could share of some of these to make it even more concrete for people like dedication is that just like working many hours uh being part of a team anything along those Lin to make to share a story maybe to help people put this into practice no and it's a good clarification because the dedication piece really isn't about long working hours it's more about how much I care about Excellence I guess so giving

it my best in those situations and that might not mean that I work really wild hours or I work weekends or I'm the one who's willing to sacrifice the vacation I've actually tried to avoid setting that as an expectation but more that I hold myself to a very high standard so an example would be especially as I've gotten more senior in roles there can be an expectation that it's okay for other people to wait on me so whether it's like the timing for a meeting or providing input on something or reviewing a document or following

through on something I said that I was going to do and I really try to avoid that which means that if someone sends me something I try to be very responsive about it if I know that I said I'm going to do something I follow through on it in the timeline that I said I was going to do if I have a meeting I try to be on time to that meeting and those are all flavors of dedication to the work that show up and oh it seems like Elizabeth works really hard but the motivation

factor is other people are relying on me and I want to show up for them and so that's when I say dedication and it's related to the second Point around showing up well for the team those would all be examples of I feel urgency in responding to people and doing high quality work for the other parts of technical to non-technical I think a great example is actually a very timely one at Netflix which is we are making strides to offer live content types so Live Events live TV shows we announced this week that we're going

to be hosting WWE starting later this year and early next year 2025 that is easier said than done done I know that there's a lot of entertainment companies that have live content but Netflix has really been in the streaming content business so live content is something new for us and it's something that's going to require a really close partnership between our content organization and our products and Technology organization because there's a Content strategy to it there's a business strategy there's a technology strategy to it and a big part of my role is can I explain

how we're going to approach those technical problems in a way that builds confidence with the content team and can I try to understand their content strategy in a way that sets the technical teams up for success and we understand what we need to be able to deliver on here in terms of requirements and I don't think I'd be able to do my current role well if I wasn't able to do that type of translation for something that's going to be a big bet for the business and something we want to invest in jointly and then

to set my partners up for success in that so I like I'm going to do everything I can to make sure we deliver well for my content partners because I feel like that's what's best for Netflix in the business amazing examples in terms of Life content I think about uh the Love is Blind I think it was uh for Premiere whatever reunion that my we we got sucked into that show so good job and I think there were some issues with that right that reunion stream yeah that was about a a little less than a

year ago now um so the amazing thing about failure is you learn a lot we learned a lot we've taken notes on it and we had a couple successful events after that including the Netflix Cup last October and we've got some exciting events coming up so I think that's something that strengthened us but did reveal that we're tackling a hard problem yeah the Twitter feeds during that love is blind Premier hilarious people are pissed okay and then in terms of the keeping a high bar for yourself I love that I think about a quote that

um there's a VC and Muro at Floodgate and she did this interview with Tim Ferris and she shared that her dad always asked her are you doing a world class job with this like are you doing a world-class job with your homework are you doing a world class job with your piano recital and that's the bar that he always had for her and I feel like that's a really good way to think about work and you know life in general if you can yeah uh my mother used to describe to me probably still does though

it required more repeating when I was younger that the last 5% is the 5% that really mattered and so it it is that framing of the thing the extra effort you put into something to make it world class or to make it excellent and so I do like to push myself that way and I I hope it sets a good example for other people too and it's very consistent with especially the company cultures that I tend to thrive in where that's the general expectation of the culture so you don't feel like you're doing it alone

because then I think you can start to feel frustrated by that I know that there this is a big part of Netflix culture and I want to get into it but before that I'm curious just what that looks like with people that report to you how do you help them level up in this skill of having a really high bar and an example I'll give as you think about maybe an example is the way I describ this to my PMS was you want to have this aura that you've got this that if you give Lenny

something he's got this he's gonna follow up he's gonna close the loop he's gonna get it done if he can't get it done he'll tell me I feel like this thre red will not disappear you won't drop this ball is there anything that you've learned as a good way to help someone learn this kind of skill and understand why this is so important it shows up for the people who report to me is one part example setting so if I don't do it why would they do it and I treat that very seriously that we

should all be held to the same standards and as a second thing I give feedback when it's not up to the standard so I think some one of the things I've observed especially with people on my teams is that the expectations aren't always clear and you can't assume that they're clear if you don't share them and when something's not meeting expectations or really showing up as Excellence I think it's a combination of both giving the feedback on that and being direct about it and being specific about what would it take to get this to the

bar that I'm expecting or to show up in the way that I'm expecting and then the third and probably most important thing is help them fill that Gap so that would mean like let's take an example it certainly has happened frequently in many jobs like a document is okay it's not great it's not going to be easy for people to follow it's not as crisp as it could be there's things that would strengthen it I can both give the feedback on that to make sure like yes it's going to take another round of iteration yes

we're going to have to work another week on this and not yeah not be done with it but pushing people to get there and then jumping into the document and helping so I I feel very strongly about and that's kind of what I mean by setting an example of like let's work on this together and then through that help people uplevel themselves so the next time around they kind of they know the expectations and they've had help kind of getting there in the past so that's probably happened a thousand times in my career where I

I jump in with both feet because something needs to be better and I think the teams are better for it right afterwards or I hope they are I think that's such a good framework just to kind of mirror back what you said set expectations that the bar is going to be really high and here's what I'm expecting from you give them very specific feedback on where the Gap is and then help them fild that Gap and I think a lot of people may feel this and hear this and are like oh man I I don't

want a manager that's like this high of an expectation person and it just feels really stressful but I've had these managers and I feel like they're that's when I've learned the most and leveled up the most is having someone that gave me had really high expectations and then helped me understand here's where you're not doing as well as you can I know you could do better go back and work on this like I know that sounds annoying but I think in practice it ends up helping you most in your career I imagine you've seen a

similar uh result I think so I mean you'd have to ask some of the people on my team so I might look at it differently than look at it it it's a hard skill because it's not always easy to give feedback especially if if you feel like you know someone's put a lot of effort into something and so I give a lot of thought to how I deliver that feedback so it feels like we're on the same team and I'm trying to help them be successful not to help you know encourage failure and that's where

I think that third piece of the framework of jumping into help can make people feel like I'm in a safe space my manager wants me to be successful my manager's helping me here and I do often do that behind the scenes so maybe that's another flavor of this which is I don't do it live in the big meeting in front of all the people where the presentation doesn't go very well I do it afterwards where it feels like a safer space to say here's a way this could have gone better let's think about this differently

next time so it gives people you know a little Grace you know and a little bit bit of an ability to absorb that feedback without feeling like it's kind of on a on a stage another thing someone may be feeling when they're hearing this is like oh my God this going to be take me so many hours to just get it to a place Elizabeth is happy with and I know you said that this doesn't mean necessarily many hours you any advice or thoughts on just how to avoid like burnout and working all the time

but also keeping this really high bar and high expectations it truly is not about time I even found myself in a meeting earlier today saying if we're clearing the objectives of something it might be that the last 20% of polish on the document is a really bad use of time so if we're going to come together to talk through like quarterly Business Review was the example what were the highlights what were the low lights what were the learnings from the quarter where are there places of misalignment the reason we're doing the quarterly Business Review is

to have a really candid conversation about how we think things are going to have a debate about things where maybe we're stuck it's not to have a perfectly Polished document for that conversation so my feedback in that instance would be I would much rather have someone spend the time thinking about what's the conversation we really want to have how do I tee that up not could I spend another 20 hours to make it look like everything's perfect in this document and so I think in that sense it's it's not just Excellence like you wrote the

perfect document I should probably be careful to not use that as the only example but instead we really got to the outcome we wanted to get to because we were thoughtful about it and we put a lot of energy and time and iteration into making sure we got to that outcome and is this examples an example where you gave someone feedback because they spent too much time on the Polish or is this earlier and thinking of the Pyramid of this framework you kind of shareed set expectations give specific feedback help them fill the Gap and

then do it in private is this like the expectation setting in this example example or is this like feedback you spend too much time on this this is expectation setting so one of the things in my new role is that there's some practices that the team has had where they're trying to understand are we still going to have those practices what's going to be the same versus different about those things and get an understanding of my expectations so it's great that people ask that question so that I can be clear about oh wait if you're

on the last 20% of this polishing the dock I'd rather spend time over here and here's how I would like the conversation to go so we all get something out of it instead of it feeling like it's just a leadership reviewer on my behalf so this in this specific situation it was setting expectations ahead of time so that we can set everyone up for success awesome Okay so we've kind of been talking around this but this is an important part of the Netflix culture so just broadly Netflix has a really special and unique culture even

though it's been around for I think over 25 years now it feels like the culture has come up many times there's that initial culture deck that came out that kind of blew everyone's Minds there's a recent book no rules rules think it's called and it feels like Netflix has done a great job at maintaining its culture and it feels to me there's kind of these three important elements and maybe there's more one is uh very high Talent density and a focus on high performers two is cander and being really direct and then three is giving

people freedom and responsibility and getting rid of useless processes like time and things like that so maybe just to dive into that first one of high Talent density which and high and this focus on high performance I guess the question there is just like how does this actually look what does this look like in Netflix and imagine part of it as hiring part of it is performance reviews and then just why is it so important why is this such a focus at Netflix what is the what is what happens when you have such a high

Talent density it's just so intrinsic to who Netflix is as a company and in some ways it's very very reflective of Reed Hastings as founder of Netflix so when he founded Netflix and grew the company over time it was with a belief that there could be a different approach to building a company that would make it a place that people thrived in and loved being and would feel different than other places both in the quality of that Talent density but even more importantly like the Excellence and the outcomes and that that's where people would derive

a lot of sense of fulfillment so it it's very deeply seated at Netflix from its original days and a big piece of that Talent density is definitely hiring so who are the people coming in and joining the team but a lot of it is we can't really have any of the other aspects of the culture including cander learning seeking excellence and Improvement freedom and responsibility if you don't start with high Talent density and so in some ways that's a it's not the end it's the means to the end in what Reed and the rest of

the leadership team has been trying to build and so in order to do that you have to really hold yourself to a lot of stuff that doesn't feel like natural human behavior and what I mean by that is giving the feedback this gets into the second bucket so giving feedback being candid around your expectations when they're not being met what could be better in helping people improve and be able to receive that type of feedback yourself in order to keep Talent density High because no one comes to Netflix as a perfect human being and stays

a perfect human being the whole time we all have ways that we could grow and improve and and so in order to keep that bar High you have to be willing to have those types of very uncomfortable conversations it's an uncomfortable amount of cander and feedback in order to keep that bar high and then the other piece of it is another thing that doesn't come naturally to humans which is making a call in pretty timely fashion if someone's not able to meet the bar to say either I don't think the role you're sitting in is

the right role or I don't think that Netflix is the right place for you and to make that something that is part of best practice to get to a point where you could make that decision and that's where we refer to the keeper test which is really just a mental framing to make sure we hold ourselves accountable for this where if I'm asking myself the question if if this person on my team came to me and said I'm leaving today I have a different opportunity and I would like to take it would I do everything

I could to keep them at Netflix if not then I should be having that tough conversation about should you really be here are you in the right role if I might be a little bit relieved if you said you were leaving and the reason the keeper test and that question is useful is because no one wants to think that way it's very hard to say to someone I think this isn't the right fit I think you should move on from the company so we have to introduce some of those Reflections in order to encourage the

behavior and we also then want to get to a place where when you're having that tough conversation people aren't surprised by it that is easier said than done but you can only get to that conversation around I don't think Netflix and you are the right fit for one another if you've been giving feedback along the way and so it feels like you know in it in its most ideal State it's a mutual observation in practice it's not always that smooth obviously we are humans but it that all feeds on itself in order to make sure

that we're really holding ourselves to what we say our our behavioral Norms as part of the culture how how's that operationalized is that just like a mental model that you should have in mind or is it like every quarter you should go through this exercise is a part of the performance review process how does it actually operationalize the network uh it's definitely a mental model so when we talk to managers about what does it mean to be a manager at Netflix it would mean you should be with some frequency asking yourself this about the people

on your team people ask me frequently am I passing your keeper test so it becomes part of a regular manager direct report oneon-one and it is just another way of saying am I meeting your expectations what's going well what's not going well how are you thinking about things and that can sometimes be a very awkward conversation to have so in the middle of a lot of like we have to talk about this project or that deliverable or this thing that's happening to take the time to step back and just say how am I doing can

feel loaded sometimes and the keeper test while it feels like a very heavy concept creates a lightness around being able to have that conversation regularly so it it we do operationalize it a point that you made I'll just clarify we don't have performance reviews oh wow so we don't have a practice that a lot of other companies do where we would think about reflecting on like a rating of how things are going we do have an annual cycle of 360 feedback where you request and receive feedback from a lot of people but it's not an

input to some output it's just for the value of the feedback and to make sure we're keeping that muscle and we have an annual compensation cycle where we reflect on how are people doing and so you think about performance as part of both promotions and as part of compensation but in that way it's it has to be part of the day-to-day and part of the operating Rhythm because we don't create a process where that would come to the surface interesting I didn't know that so the idea there is just ongoing like the whole it's what

many people dream of no performance reviews we will give you on going realtime feedback we don't have to wait 6 months I feel like people talk about this but very rarely do this but that's how you guys operate in the ideal in ideal practice it's like that you have to keep reminding yourself this is our ideal because it's really easy to rely on the annual 360 cycle and all of a sudden I might get I do get about 300 pieces of feedback and some of those things are things that happened six months ago and I

think oh I wish you had told me this at the time that would have been more living the Netflix culture so we have to push ourselves to do it that way but yeah that is if working well it's very timely direct feedback the 360 cycle is sort of the annual check-in on let me get the full picture let me be able to distill some themes let me Tee It Up For a conversation with my manager and then it it does remove the the sort of crutch of an every six-month performance review or something like that

when you talked about this example of someone asking you often am I am I meeting your keeper test it makes me feel like someone's just like super nervous they're like am I passing your keeper test and it makes me feel like it could create a culture of just like a lot of stress and worry and this Hunger Games mentality of like I got to compete and worry and I might die or get fired any day I'm guessing the solution to that is partly cultural this is just the way we work you you don't need to

stress all the time but you may be let go if you're not meeting this keeper test how do you avoid this just like constant worry that you might be fired any day and that you may not be hitting the bar in my personal experience I have felt a lot more at ease by having these conversations than by not having them so in many roles I've had I haven't been sure how I was doing or things I could be doing better on and I didn't quite know how to get that information and that made me feel

much more stressed or nervous or at risk than having it be part of the culture to have those conversations so the thing that I think can be nerve-wracking and I feel it myself is the high bar for excellence at Netflix and you're surrounded by if we're doing this well you're surrounded by amazing people and that can feed a sense of am I doing well enough compared to how everyone else is doing and I I know the bars high for the most part that can drive people and in a good way but in some ways it

makes people nervous and that's where I think it's helpful to know we expect to have these conversations so you can just kind of you know let your shoulders relax a little bit of yes the expectations are high but my manager says I'm doing a great job or my manager says I'm not doing a great job but they gave me concrete things that I could do better and so I I think knowing is better than not and so in that sense it's the culture combined with the conversations around performance I hope take a little bit of

that stress out of it but I've certainly heard it a lot that without that conversation people can be nervous that's such a good point and such a good example that I feel like every company wants to have a high bar and have only High performers and keep the bar really high for every person they hire I'm curious I know this could be its own podcast and book but just in terms of hiring people at that are amazing and keeping this bar of Excellence is there anything you can point out that might be helpful to other

companies hiring to help identify amazing people and make sure that bar stays High one thing I know is you guys pay top market for salary I think that's one unique thing about Netflix is we just pay pay people uh so maybe that's an answer part of this answer but just what what advice do you have for people to keep a really high bar in their talent yeah I mean on the compensation point we pay what we call personal top of Market meaning we want to be highly competitive in the pay but we don't want pay

to be like the golden handcuffs of Netflix sets Market rather than paying people a strongly competitive compensation so I think that that's important for attracting and ret retaining talent and has been a big part of the culture but almost more importantly we hope we don't have to rely on that to want people to want to be at Netflix or for us to be able to assess whether people are going to thrive at Netflix and the way that I've thought about hiring with that context is we know we're going to offer you very highly competitive compensation

but are you going to come to Netflix and help us identify the right problems to solve or new ways to solve existing problems and that's a different way of hiring then you might think about especially at scale where you're saying does this person have this skill this skill this skill check they're going to fit in this box and they're going to deliver this work that I need them to do I'm being intentionally simplistic I recognize a lot of people don't hire actually that way but at Netflix we try really hard to say we're looking from

the new perspective or the person who's actually going to make us stronger as a team so thinking about additive skills additive perspectives people who are going to push our thinking on something and that tends to help us with thinking about Talent density because you're constantly introducing people to the team who up level so then the questions you have to ask in an interview might be different because yes we're trying to assess do you have the Baseline skills to be successful here but we're also looking for the things that make people exceptional or even stronger than

the team we've got and then you think about making the magical teams comprised of all those amazing minds and what can you get out of that and that feels like more the the talent density and practice got it so the advice there essentially is don't look for someone just simply great look for someone that raises the bar for for the whole team brings in a whole new perspective yeah I think what's great about this idea of just maintaining Excellence consistently is that the best people want to work with the best people and as soon as

there's one person that sucks and the company allows for that it just brings everyone down because they know hey we can we can be okay we'll sticker no one's goingon to do anything about it and when you make it clear we only have want the best and only hire the best only keep the best it keeps the best there right I that's part of this strategy it's definitely the goal and I I think understanding that having gaps in the team and people's skill sets or their behavior can be really toxic for other people on the

team so it's a cost this episode is brought to you by explo a game changer for customer facing analytics and data reporting are your users craving more dashboards reports and analytics within your product are you tired of trying to build it yourself as a a product leader you probably have these requests in your road map but the struggle to prioritize them is real building analytics from scratch can be time-consuming expensive and a really challenging process enter explo Expo is a fully wh labeled embedded analytic solution designed entirely with your user in mind getting started is

easy explow connects to any relational database or warehouse and with its low code functionality you can build and style dashboards in minutes once you're ready simp embed the dashboard or report into your application with a tiny code snippet the best part your end users can use explos AI features for their own report and dashboard generation eliminating customer data requests for your support team build and embed a fully white labeled analytics experience in days try it for free at expo. co/ Lenny that's ex. Lenny so asked on Twitter what questions to ask you and there's a

great question that came in from nonu he's the head of product for linear and he asks what practices does Netflix do that other companies should not attempt to do because their talent level is so much higher than other companies that is freedom and responsibility in a nutshell so let me explain that um it's a good question and it it's kind of related to what I was saying earlier that Talent density is a prerequisite for a lot of the other ways we operate so if if we want to create a work environment where we are not

prescriptive about how people solve problems or the scope of problems that they could tackle assuming they're highly impactful for the business and we don't have a lot of process around that work so think about being able to make large Innovations to our Engineering Systems or introducing new ways to think about metrics and experimentation we get a lot of those things because we give people the freedom and the space to explore and question things and experiment in a way with Solutions and I I think that that would be very hard if not dangerous if we didn't

have a high Talent density it's really not a top down do a then B then C even in how we go through some of our planning processes or thinking about how we think about priorities there is a lot of room for contribution across all levels of the team and that requires Talent density and then there's things like you know you have to have amazing people if you're not going to have really strict guard rails that would influence the consumer experience or business stakeholders experience and we do give people a lot of responsibility on those things

so I think the the lack of process and prescriptiveness is all hinging on we've got amazing people who are smart but even better have strong judgment this is kind of what you always hear from people giving Founders advice is just hire amazing people get out of their way and let them do their job which is often not a successful experience what are examples that come either things that came out of this freedom I don't know products or features or ideas that came out of this or policies or processes you don't have that everyone else might

have so I know there's no vacation time there's like unlimited vacation time if that I assume is that still a thing unlimited vacation time okay cool yeah all right uh great uh and I know performance reviews you talked about that so I guess in either direction like is an example of something that came out of this Freedom or some process that would surprise people that you don't have or a framework or system we've been able to deliver you know I'll speak to my own team around Innovations in our content delivery Network or Innovations in encoding

or Innovations in Discovery and personalization were not driven by some leader saying I think this is a priority they were driven in many es by individual contributors who had great ideas for Innovation so a lot of the stuff that Netflix has succeeded in came from creating space for people on the team so there's probably thousands of examples of product features and things like that that came out of creating this space and right now the trick is finding the sweet spot so that we can operate efficiently at this type of scale with out snuffing out some

of that what was kind of the core beauty of the culture maybe a last question around just the culture we talked on about cander a bit I'm just curious if there's an example that comes to mind of of an example of cander that you recently saw or had to be the candid person that might be interesting to share uh where it's like oh wow that's what you mean when you say a culture of candid of cander there's a couple things that come to mind I am generally a transparent leader meaning I share information freely and

open it's part of the culture to context not control which means part of my job is to make sure that people have the context they need to do their jobs well and in practice that means I take notes in leadership meetings and I share those notes with the whole organization and that is sometimes it it includes cander around Reflections on things that aren't going well or problems we need to solve sometimes it's just letting people know here's what leadership's talking about so that they have a sense of what's top of mind but it's it's a

version of transparency that I feel strongly about doesn't exist a lot of other places and I I think it's a version of cander too in being able to share I can't we share every detail of everything that we're talking about but I do try to share things that probably push the boundary a little bit in the team feeling like they understand what's happening across the company and what I'm thinking about and then there's a second example that comes to mind which is until two years ago individual contributors didn't have levels at Netflix so all Engineers

were just senior Engineers all data scientists were senior data scientists and we did not have a leveling system we introduced IC levels two years ago almost exactly and it was a big big big shift because it it was seen as something that was sort of sacred of it a lot of people came to Netflix because we didn't have it we didn't have process around promotions we didn't this is probably part of why we never had performance reviews because promotions really weren't at play and it gave people a sense of freedom of not having to worry

about that type of structure but when you get to a St scale of an organization we needed some type of scaffolding to say we want to talk about how we compose teams like when do we need the person who has 30 years of experience when do we want to have a new grad because that's what the work requires we didn't have a language for it so I introduced levels a couple years ago and it we had quite a change roller coaster is the only way I can describe how it went good phrase uh and I

yeah it was sort of like being like the tumble dry machine for a few months and so really talking through it with the team that's just context and backdrop for an example of cander recently which is we had kind of a postmortem or retro on how has it gone with IC levels so it's kind of like Wells not Wells and I would normally think in a lot of cultures it would be like well we got past that change we're living that change like don't don't reflect on it because that kind of opens some of that

early debate and I felt differently about it I think it's a good example of being candid about this was a big change for us it hasn't all gone perfectly there's a lot that we can do better how we Implement levels at Netflix and I would rather share that information than pretend it's all gone swimmingly and we achieved every objective so I think that I try to build examples like that because I I do think that level of cander and reflection helps build a sense of community and Trust across the team it's an awesome example kind

of along the lines of that but also this category of freedom and responsibility something Netflix innovated long ago and I'm curious if this is is still a thing is this idea of chaos monkeys which essentially are a program that runs on your infrastructure that just kills random processes and things and just to see what breaks and make sure things are stable when things actually start falling apart so uh is chaos monkeys is that what it's still was that what it's called and then is that still a thing is there still some chaos monkeys running around

the the servers not unbridled chaos monkeys no okay contained chaos no car we carry too much responsibility speaking of freedom and responsibility for the member experience to inject pain though we do do a lot of experiments to test resilience and that does probably mean injecting things that we're not quite sure whether a is better than b and so that happens across Engineering Systems really at scale but it it's not for Pure chaos it's for intentional learning and so we can avoid making bigger mistakes and then as we go into new efforts like Cloud games we

have a beta that's out now live would be another example we do try to come up with intentionally lowprofile examples where we can test the bals of our systems in a way that's unlikely to damage the member experience but uh that's less Randomness more by Design and so we're doing that in a few places that feels mostly like good engineering practice so we can understand when when it's really showtime and we're going to really test our systems will they be able to perform like we want them to rip chaos monkey yeah so kind of along

these lines of data something that so data itself has always been at the heart of Netflix and my understanding is the way the data team and the insights team is structured has been one of the reasons Netflix has been so successful and that's the team you led before you moved into this new role can you just talk about how these teams are structured and why the structure is so effective yeah I certainly like to think of it as being special it's unusual I can explain why so at the scale of company that Netflix now is

is very often data oriented teams are embedded in other parts of the business so it could either be they're embedded in a business line like ads or games or they are organized more functionally separating data Engineers from data scientists from analytics Engineers from consumer researchers and we've resisted that and kept a centralized team that is both functionally diverse so across all those types of functions that I just described and works on nearly every area of the business from within the team and I sort of understand why a lot of companies move away from this because

it really does require basically extraordinary partnership that we would have people work on data problems that don't report into the teams that are relying on them but the benefit we get is we get to think about our functional expertise so are we really world's best data Engineers world's best data scientists and how do we continue to be ever better from a functional and Technical perspective it gives people better career paths because there's more Mobility across the teams it feels like a team that has a functional expertise with a lot of different problems to solve and

so I think it enables more cross-pollination of ideas in a way and it also allows us to be really objective that is probably the most important thing that our job is not to tell the story that someone wants to hear with the data or to solve the problem that someone thinks is most important it's for us to have our own perspective about things and I think that that up levels the whole organiz a because it means that we're able to be truth tellers or to be curious in a way that might not fit if we

had a different organizational structure we have to balance that with be a good partner deliver on the things we agreed were priorities be flexible without we're spending our time but it gives us agency and responsibility beyond that and I feel like the team takes that very seriously so I've seen examples of that in how we bring data to a lot of spaces including how we partner with engineering on data related topics or how we partner with content that I'm not sure we would have gotten to if not for having that kernel that's sort of a

center of excellence around it and uh it's data and insights that was the team that you ran insights is that describing user research or what is that a function actually so part of data and insights is a consumer insights team that includes a lot of different flavors of research really so in some ways consumer is even a misnomer because there's parts of the team that do internal research for example on tools and products for our studio Productions so that's more of a a user research oriented versus consumer and then the parts of that team that

are consumer oriented do things all the way from content screenings to make titles the best version of themselves before they're on the service to more traditional ux research search to think about how can we deliver the best title Discovery experience or how can we think about things that improve accessibility and then that team has a global remit so there's also teams that are more local or Regional expertise in understanding consumer needs and entertainment so consumer insights and a team formerly known as still kind of known as as a shorthand data science and and Engineering combined

together to create data and insight it's probably about two years ago that's another piece that's unusual becomes truly a full stack data and research expertise and so we could tackle a problem like what's the right way to think about recommendations and how best to surface them in a way that combines attitudinal research qualitative and quantitative with behavioral research on more of the data science data engineering and analytics side it's super cool because I think it's really rare that the what people think of of us user research is within the data or and I think that

might be a solution to some of the backlash a lot of user research teams get where they're like I don't know what are you guys doing all this anecdotal evidence if it's under the same org uh I feel like that leads to a lot more credibility and uh avoids this like oh data is telling me this thing this user research team is telling me this thing what should we yeah I I mean consumer insights was one of the newer teams for me you know it wasn't in my background to lead a team like that and

not in my individual training but they they are critical for making sure we keep a consumer orientation a member orientation on things and I have loved to watch the teams collaborate on problems because it it's we talk about it as a superpower internally in combining those skill sets so I think the consumer insights team at Netflix has had a lot of credibility in a certain area of expertise and we took it to the next level by combining it with other of the fun functional expertise so it's it doesn't it's not required in every problem space

so we try not to overdo it and say we need to be collaborating everywhere because that just feels like a the wrong expectation but we try to make the most of it in spaces where we really benefit so yeah it's worked out really well awesome okay I'm GNA ask two more questions before we get to our very exciting lightning round and they're both uh skills that little birdies have told me you're very good at one is that you are very very intentional and thoughtful about staying close to individual teams and individuals within the company even

though you're higher and higher in the org I'm curious how you do that how you actually practice this skill of staying really close to teams kind of at the bottom of the ladder and individuals that are working on things in on the ground basically a lot of it is how I spend my time and fighting to preserve opportunities to connect with people so examp examples would be I still have bi-weekly office hours people sign up for slots um and I I get it can feel a little like speed dating for 20 minute slots but I

get to meet a whole bunch of people and hear about work here what's top of mind and people book them out many months ahead and it's just an opportunity to stay in touch and then I do as me anything sessions with team of different sizes depending on how intimate we want it to feel but truly anything is fair game as a way to get to know me as a person for me to hear questions to try to be candid about what I can answer can't answer and so those things have helped me maintain connection but

both of those examples are about making the time for it so what I have found as my role has changed is that it just wouldn't happen if I didn't make it a priority and then through those types of sessions I I do think I become or I hope to become more approachable so people know you can send me a slack message you can send me an email like I mentioned earlier I'm going to respond to you as quickly as I can because I want to hold myself to that bar and so that builds a flow

of communication between me and the team that I really value I don't think I would want to do my job if I didn't have those points of connection so that helps to you also send that email to everyone after every leadership meeting so they're like oh yeah Elizabeth yeah they hear from me yeah kind of related to this um so we have a mutual friend that's how we got connected Ali Ral uh she was a data scientist at herb B now she's at Uber and she had a question that she wanted me to ask and

it's about uh how good you are at being present so her question is something you she's noticed about something I've noticed about her is how 100% present she is no matter who she talks to do you have any advice for people to get better at this because it's so hard in the day of email and iPhones and slack her question is like like when does she respond to stuff if not sometimes in meetings I actually think I'm the most present when I'm having conversations like this one and I I do preserve a lot of time

to have one-on-one conversations where I'm genuinely curious about how someone's doing how I can help them what they're excited about that's authentic and so while my EA would probably cringe at saying I I like to spend time doing a lot of those one-on ones it's it is relatively easier for me to say like the human connection is part of what I enjoy about this I think that's true for a lot of people in what we get out of work in life but I try to live that in those meetings I'm probably not as good when

we're talking about meetings of 30 people and I'm multitasking um so I will admit to doing that for sure but I I think the one-on-one conversations I treat as being pretty sacred and one of the things I've noticed that helps me continue to invest in that and maybe is helpful for other people is some of my greatest friends and connections including people like Ali are people I met along the way professionally so I worked very closely with alli's husband Keith enwood at multiple places um both an alysis group and at Lyft and that means that

it's created opportunities and it's been points of connection and so you get back what you give basically um there are people in my life who are part of my life because I worked with them or because I crossed paths and I like to think that if I can make a like a positive mark on them you know it'll come back and be a benefit at some point too so I think to distill that is that I truly enjoy it it's what I get out of especially work and then it it's my community and that's served

me really well over time and so I have given people advice of this is a small community think about what you're investing in other people because that will matter down the line for yourself too and try to live that myself that such good advice there's kind of two things that come to mind there when is treat people the way you want to be treated someone once said that maybe and I think you come back to this a couple times this idea of just pay attention to what gives you energy and that you're good at and

just almost double down on that just like make that more and more of a superpower yeah that last part resonates it's been a big part of my personal and professional practice to reflect on how I'm feeling what what I'm excited about what I'm enjoying and I I do think it helps me be more grounded which maybe helps me be more present or helps me be a better manager or leader that might be part of the secret sauce too but it it's part of my practice I can't help but ask is this like an actual practice

do you do this on a regular basis or is this just something you think about like I should reflect back I wish I was so Advanced to say I meditate and I I create all this structure it's more that I I think I mentioned maybe I'm an introvert so I do spend some time alone that's how I recharge and early mornings especially people who know me sometimes are horrified at the time of day I send emails but early mornings are a quiet time for me where I do try to have a a daily check-in of

just how are things going why am I feeling anxious why am I feeling excited and it's kind of a muscle you build so while I don't I don't write in a journal I don't have a meditation practice I do have a time of day when I I try to keep it protected from other things so that I can think for a second uh what I think about there is Jeff Bezos has this approach in the morning he just calls it he Putters around he has no meetings until I think 10 or something he just wants

to putt around read the newspaper see what's going on in email which I'm trying to do I really like that that feels really good I'm just GNA putt around I have no responsibilities in the morning I like I've never heard that I'm gonna adopt that language buttering puttering um Elizabeth is there anything else you wanted to touch on or leave listeners with before we get to our very exciting lightning R I'm ready for the exciting lightning round well with that we reached our very exciting lightning round first question what are two or three books that

you recommended most to other people it's probably a little recency bias but I've been recommending what I talk about when I talk about running um by muracami which is talk about introspection about the the similarities between running and writing as sort of flow States and very meditative things so I I had read some of his fiction books and the autobiographical reflection on these types of either professions or Hobbies I think is very insightful so that's one and then one of my long favorite books is a fine balance by mystery um and that is just a

great story of human complexity and Challenge and relationships so I'm drawn to both books and TV and film that are about humans speaking of TV and film this is a maybe a high stakes question for someone that works in Netflix do you have a favorite recent movie or TV show I'm not going to name all n nli that feels too much like an advertisement um film triangle of sadness is phenomenal if you haven't seen it and then I'll go Netflix for TV beef was I thought hysterical I'm an Ali Wong fan but also just a

pretty unique storyline and I think they just won a bunch of emmies they did amazing good picks next question question do you have a favorite interview question that you like to ask candidates that you are interviewing High Talent density I'm usually looking for the person who would be better in my role than I am in my role so I often ask people what would their priorities be what would they do differently if they had my job next question do you have a favorite product that you recently discovered that you really like so while I carry

the CTO title I live a pretty analog life um so my most recent product is a fellow pourover coffee maker which is actually part of my morning ritual which I'll Now call puttering around where I take Great Lengths in my coffee making process because I find it calming and then it's not a recent find but I have to shout out that my pelaton is probably the favorite product I own the bike or the treadmill bike I'm a recovering outdoor cyclist so it's also kind of questionable if I can even admit to this but that's why

I will admit to I love the pelaton despite being ideally an outdoor cyclist I have questions about your cycling but before that question do you have a favorite life motto that you often come back to or share with friends or family that you find useful either in work or in life my mom said something to me that has stuck with me I don't know if I live it very well but her the phrase was something good happens every day and the reason she said it was because she was encouraging me to be more mindful about

enjoying the small things in the daytoday rather than letting myself get caught up in the busyness beautiful final question you're a big biker and triathlete I am curious what that Sport and time has given you in your career or in life what what benefits have you found from spending so much time and energy running biking being an athlete certainly by mental resilience so while those sound like physical strength I found especially Endurance Sports are much more mental and how you go through the highs and lows and sustain and then coming back from challenge so those

Sports have had their highs and lows and from the lows I've really learned how to kind of recover and bounce back so those feel like universally applicable skills you have such an interesting mix of athleticism and then Netflix what a good balance for life this is going to give me permission to go watch some Netflix repor recording this on Friday afternoon Elizabeth you're awesome thank you so much for being here two final questions where can folks finding online if they want to reach out and maybe follow up on things and how can listeners be useful

to you um you can always find me on LinkedIn um so definitely reach out or ping me if you have questions or comments and I think the way the listeners can be useful to me is being maybe curious about how they can show up even better in their lives now that we've done this reflection on Netflix culture and how we show up for other people and I would like to ask listeners to pay that forward to people that they're working with and how they show up for them amazing I love that if you end up

doing this and you're listening uh maybe leave a comment on YouTube or in substack with something that you all covered about yourself Elizabeth thank you so much for being here thank you Lenny I hope you have a great weekend same bye everyone thank you so much for listening if you found this valuable you can subscribe to the show on Apple podcast Spotify or your favorite podcast app also please consider giving us a rating or leaving a review as that really helps other listeners find the podcast you can find all past episodes or learn more about

the show at Lenny's podcast.com see you in the next episode