Why great AI products are all about the data | Shaun Clowes (CPO at Confluent)

I love that you have very strong opinion about this which is just the state of the product management career and how it most PMS are not that great why is it that product management is still such a relatively undeveloped discipline like we're like 15 to 20 years into this and so there's something about the current state of product management that isn't getting at the truly important things the truly value added things if we were doctors you'd be like that's totally unacceptable what's the answer Sean how do we solve this problem in everything always talk from

the customer perspective from the Market's perspective from the compend perspective a very small number of PMS do that they get dragged into internal politics they get dragged into scrum management or scrum execution or product delivery and you just can't win that way you kind of have this hot take that the way AI will most impact product management is data management well you've got this synthesis machine which is this llm thing that's going to help you do synthesis but if it hasn't got all that data to do synthesis on top of it's got nothing and so

that means that LM can only be as good as the data they are given and how recent that data is in the future if you can easily CL a B2B SAS app like Salesforce or at lassan what happens to these businesses long term do they just become are they all in trouble people really underestimate where the value is created in these applications and they just kind of get it completely wrong today my guest is Shan klous sha is Chief product officer at confluent previously he was Chief product officer at moft which is a billion dooll

business within Salesforce before that he was Chief product officer of metromile a a public auto insurance technology company and prior to that he spent 6 years at atlassian where he ran the jir agile and also built the first ever B2B growth team he also created two of the most popular reforge courses one on retention and engagement and one on data for product managers sea is awesome because he is both very tactical and execution oriented while also being very philosophical and insightful about the craft of product and growth in our conversation Sean Shar why most PMS

are not good what it takes to become a good or great product manager how he thinks about his career like a bingo card and why he indexes towards finding very different roles for every new job that he takes why good data is the most important ingredient in AI tools and for product managers working with AI also how to build a great B2B growth team what he's learned about doing B2B growth and his really interesting take on how AI will and won't disrupt SAS tools out in the wild if you enjoy this podcast don't forget to

subscribe and follow it in your favorite podcasting app or YouTube it's the best way to avoid missing future episodes and helps the podcast tremendously with that I bring you Sean klouse this episode is brought to you by interpret interpret unifies all your customer interactions from gong calls to zenes tickets to Twitter threads to App Store reviews and makes it available for analysis it's trusted by Leading product ORS like canva notion Loom linear Monday and straa to bring the voice of the customer into the product development process helping you build best-in-class products faster what makes interpret

special is its ability to build and update customer specific AI models that provide the most granular and accurate insights into your business Connect customer insights to revenue and operational data in your CRM or data warehouse to map the business impact of each customer need and prioritize confidently and Empower your entire team to easily take action on use cases like win loss analysis critical detection and identifying drivers of turn with interprets AI assistant wisdom looking to automate your feedback loops and prioritize your road map with confidence like notion canva and linear visit n t p.com Lenny

to connect with the team and to get two free months when you sign up for an annual plan this is a limited time offer that's interpret. comom Lenny this episode is brought to you by build better. back in 2020 when I was just a toy build better bet that it could cut down on a product teams's operational BS fast forward to today 23,000 product teams use purpose-built AI in build better every day first build better uses custom models to turn unstructured data like product and sales calls support tickets internal Communications and surveys into structured insights

it's like having a dedicated data science team second build better runs those structured insights into workflows like weekly report reports about customer issues context aware prds and user research documents with citations it even turns standups into action items that automatically get assigned and shared into your tools plus with Unlimited seat pricing on all plans built better ensures everyone at your company has access to this knowledge truly no data silos in a world of AI demos over promising and underd delivering seey build better has a 93% subscription retention get a personalized demo and use code Lenny

for $100 credit if you sign up now at build better. a/ Lenny Sean thank you so much for being here and welcome to the podcast thank you Lenny it's really awesome to be here I've had you on my radar for a long time and I am really excited to finally have you here and big bonus points for having a very beautiful sultry Australian accent that always helps with the ratings I think I don't know if it's causal but it's correlative I'm I'm glad to be a bit of a curios [Laughter] so I want to start

with something I totally believe and I love that you have very strong opinion about this which is just the state of the product management uh career and how it's most PMS are not that great and how there's a big opportunity to level up you just talk about what you've seen there and you're just like thinking here yeah it's honestly like a big conundrum for me I think it's actually part of I would it's grandio to say so bit of my life's work like why is it that um product management is still such a relatively undeveloped

discipline like we're like 15 to 20 years into this thing right you would have thought that it would be less random than it is like the outcomes are random the behaviors are random individual performance is random you know seemingly right and so there's something about the current state of product management that isn't getting at the truly important things the truly valued things the the right way to think about problems the right way to Think Through problems the abstract reasoning that's needed something that isn't working about it I spent a long time trying to put my

finger on it and then be like how do you reproducibly produce that like reproducibly produce people who can really be really great product managers the thing is that if you think all the way back to it like I spend a long time as an engineer and people always talk about 10 times Engineers right and I wanted to be a 10 times engineer you know I'll leave it to others to decide to tell you whether or not I was or I wasn't but certainly I wanted to be and I tried to be a really great engineer

and it must be true that if there's 10 times engineers and I would argue they definitely are there must be 10 times fr managers too but at the same time those 10 times product managers because product management is ultimately about leverage so it's about helping other people have dramatically more impact than they would you know if they were unorganized that they didn't have somebody to kind of organize the goals and what we're trying to achieve then that means that a 10 times product manager has 100 times return or more because because they're 10 timesing the

return on 10 times resources right so the outcomes are so wild like wildly distributed and the benefits are so good that you would have thought that that kind of it would have behooved us there would have been a way that this had evolved and improved and really gotten way crisper than it has but here we are you know I'm not saying that we haven't gotten better we 100% have but I I think I think it we could all say that we're not reliably producing you know 10 times product managers every day every day of the

week I love this point and it's especially painful that when someone works with a PM that's not great there's just this like meme of why do I need PMS PMS are useless PM suck and it just creates like no one's ever like Engineers are useless or designers are useless but ever there's so many people are like I don't need product managers on our team never hire pm and it just sets the whole profession back when I first started out in PM somebody you know it's a it's obviously a chestnut but he pointed out that like

realistically when you're a product manager your job is to say no to 90% of things that that get brought your away and so that kind of makes you the bad person pretty much from the St and so you're saying no to 90% so you can say yes to 10% and and that kind of puts you behind the eightball right at the very beginning and so you have to kind of very quickly get runs on the board you have to prove to be to have the right insights to have the right data to make the right

decisions or you don't get another go you don't get another swing you don't get another swing at it so it makes sense that you know product managers are the easiest to kind of single out and uh and kind of criticize but that is also what makes it the funnest thing like if you think about like why do we do this you know somebody once asked me like you know would you retire like what why do people do what they do uh cuz certainly at some point it isn't just about the money and at the end

of the day product management is so damn fun because it's about trying to figure out an edge it's like trying to look at the world find the portion of the of the chess board that isn't occupied but that is valuable and find a way to get into it invade it and destroy it like it's a it's a it's a really fun like it's decisions under uncertainty and that makes it unbelievably fun like really really painful and very frustrating and very hard to convince people but very very fun um so you know in equal measures basically

what's the answer Sean how do we solve this problem I know you said it's your life's work what what do you find actually helps most in helping PMS level up and become say 10x PMS I think the most important thing and the kind of the chestnut that I repeat to everybody is that at the end of the day the time you spend looking inside the building doesn't really benefit you very much at all right and you know Steve blank and people used to talk about you should be spending 80% of your time thinking about things

going on outside the building you might not be outside the building but you should spend 80% of your time thinking outside the building and I would say a very small number of PMS do that they get dragged into internal politics they get dragged into scrum management or scrum execution or product delivery like elements of the delivery thing uh and you just can't win that way like you just you just can't win that way you can never get an A because because you're fundamentally not solving the job like the job is not about execution or anything

it's about finding reliable different value right that you that you can uniquely deliver into the market so I would say like if there's one thing you know two things I would say actually that I generally guide product managers to do one is to like always starts from the point of view outside the building in every document in everything always talk from the customers perspective from the Market's perspective from the competitor's perspective and the people who listen to me on that I would say get better almost immediately because they're starting from a place that's easier to

understand and then secondarily be data informed like like kind of use use all of that view of the world but don't just make up a bunch of statements like support that statement with you know anecdotes and bits of data doesn't have to be a Treatise but like kind of bring into bring kind of convince everybody of what the world really looks like and what the opportunities ahead of the company looks like and good good things happen to you and all of a sudden you go from a world where nobody wants to help you get anything

done to where everybody is wants you to win they want you to win and they they may not give you everything you want but they certainly will try because they're like well of all the better could make this is a good one I imagine many people listening to this are thinking oh I am that person I talk to customers all the time I'm always interacting looking at research putting data together and I what you're saying is you're probably not doing that enough is there anything that you could help someone recognize of no you're actually not

doing this enough and you think you are but you're not it's one thing to say you're spending a lot of time looking outside the building it's a whole other thing to like hear from the places you don't normally hear from so like so avoid of avoid availability or confirmation buas like most of the time people go talk to the people they always talk to and they learn nothing particularly new they don't synthesize the results that they got from that conversation they don't seek out the counterfactual they don't seek out the proof that they're wrong they

they they don't analyze what their competitors are doing and figure out what that must tell you about the market they don't bring back the data of how their product is actually being used versus how people say it's being used it's it's like you know kind of all data no analysis is not very useful like all kind of like you know everyone can bring back an Omnibus edition of like you know random stuff I heard on a Tuesday but the but the competitive Advantage is is extracted out of figuring out what other people don't see figuring

out what you know where we're wrong figuring out where a well-placed bat could have dramatically um you know outlandish returns and so people you know I think firstly people often say that they do a lot of this stuff but they actually don't right because they don't have any structured way of doing it so what they really mean is like every now and then I get in I get in a customer call or if now and then I get stuck into an escalation and so they're kind of conveniently bucketing it so firstly they they don't do

it in a very structured way then they don't bring back analysis like true insights from that thing so they don't really gain very much at all it's just it's just a more more activity no outcomes activi is not people people do far too much activity with not enough outcomes um and there just isn't enough time in the day to do that to be successful you as a product leader is at the vend diagram Center of The Sweet Spot of where this podcast has been going recently which is product and growth and how AI helps you

with all these things and so to follow a thread there with synthesizing and understanding what people are saying C user research and surveys and all these things have you found any tools that you and your team have found really useful to help you do this more efficiently versus you know traditionally just manually going through all the stuff and finding patterns yeah so firstly like stepping back a little bit just into like the motherhood apple pie portion of like um qualitative research or whatever like I find that most people don't even understand or don't start with

a rigorous foundation in what they what they're going to need to do to get the answers that they want so for example your listeners have probably heard about the Neilon number before but basically the idea is that once you interview between seven and 14 people you stop learning new things less than seven you don't learn enough more than 14 you stop learning anything new and so if you interview two people you probably don't have enough data if you interviewed 22 you probably had too much so like they don't even rightsize their efforts so that's a

problem so they don't start that way then they go into these conversations asking leading questions which really are designed to get the customer to say what they already want to be true which is think so so they haven't done enough research or they've done too much and they've blown up all of the results before they've even heard anything like so you know if you don't if you don't rightsize your research and you don't kind of set this up to learn then you're going to lose like no amount of no amount of applying llms or any

type of kind of structured reasoning is going to help help you because you're just basically you're reading back what you want to hear or some weird summarized version of what you want to hear but you know stepping back from all of that like like what I like to do specifically getting to llms is like I think that we live in just the most amazing time for product managers right now in terms of being able to analyze vast quantities of information and see the common threads so give let me give you a few a few examples

of that right one might be you can do a bunch of uh C interviews you can put a bunch of customer interviews into chat GPT and you can say hey chat GPT this is my strategy tell me where my strategy does not fit what these customers talked about it's all about the not not where it does where it does not like people spend far too much time looking for what they're hoping to see not for what they're not looking to see so you can you can literally ask chat duty to help you find where the

customer is probing at the edges of what you're trying to do where it's where it's wrong where what you're saying is not what they believe um and you can ask a questions like that you can ask it where you what what your customers are saying would better fit what your competitors are saying so you can basically say Hey you can copy and paste what of your competitors positioning documents into chat DPT and say is this a better fit for what they have said than my thing which is which is you can summarize your your own

strategy like uh you can take your competitor's public documents and you can ask it to summarize what their strategy probably is and it's actually surprisingly good at that because mostly your public documents are actually a summary or at least a derivative of what your strategy is so it will give you crazy insights into what other people's literally their product strategy at times creepy like oh they will probably do this they will probably do that it's more likely they would do this that than they would do that and so like normally that type of insight was

hard one like you know it's like it took a lot of sweat work you basically had to read a lot of stuff you kind of had to use your brain as like this big kind of summarization machine and eventually you knew what you felt about all the things you had read you couldn't summarize why lm's let let you get to that really really really quickly in a very structured way but only if you push at the edges provoke the provoke the answers you don't want to hear provoke the problems like try and try and you

know prove to yourself that you're wrong I think is the easiest way to start um trying to use some of these tools I love that and it sounds like in your experience you're just using Straight Up open AI J gbt Claud not like any specific tool for use a research for this specific use case uh no mostly I find that like the the straight up llm and are good enough um and uh and we do have some internal toiling that we built um around you know I don't know if you've ever had um Sachin Rey

on on the show you may have uh he was a product leader pretty well known in the growth community and he was a leader at LinkedIn for a long time and he kind of uh he used to he used to call this concept a feedback rer and he basically said they really smart product managers are constantly swimming in a feedback River they set out to surround themselves by a feedback River and uh and and I really deeply believe in that it's like okay how can I surround myself with you know user interview data with Direct

customer feedback with NPS data with competitor information like I'm always kind trying to wash myself over with information and where I'm going with this is that uh llms and tooling based on it can be exceptionally good for this so for example we get a ton of at conform we get a ton of inbound customer requests as you can imagine coming from the field or directly from customers we use llms to take in th those those asks to summarize what they're about to find other asks that are like that one like really in a compelling way

like a real way like a semantic way not a not of other words the exact exactly the same is the same concept so that we can look across all of the inbound Demand on us and say well the most popular idea is this one and is getting more popular the least popular idea is this one it is getting less popular in a in a really deep rich way even across hundreds or thousands of pieces of inbound feedback I think you know it's a really great time to be a product manager if you can put these

types of tools to work but they would they don't do the job for you they just help you do these things that are you know intricate in that job of finding the finding the gaps finding the opportunities finding the common threads without you know necessarily having to do all of it just inside your ww just inside your brain I'm going to stay in this AI river that we're in right now and ask a couple more aish related questions and this may be what you just said but I'm curious if there's more here you kind of

have this hot take that the way AI will most impact product management is is data management and data versus like models You're Building or anything else can you talk about what you've seen there yeah I mean I think there's two implications for people as they're building products based on AI and as they're thinking about um like Ai and their workflow so let's start with the first one because that's how product managers do product management things you just ask this question of like should it be specific tools built for you know to to make AI easier

for product managers to use or is it in fact like more General models being put to work at the end of the day like these models are very very very smart but they're also like insanely dumb like and everyone knows that right insanely dumb in other words they really only know what they were trained on or what you bring to them right at that moment like in that millisecond and then they will forget it immediately and so and it's very easy to con convince yourself that that isn't true but it's actually what really matters and

let me add one extra piece that makes that really important at the end of the day information has a Decay rate so think about customer feedback it has a Decay rate or what your competitors are doing has a Decay rate so any new piece of data decays in its value to your decision- making very very quickly very very quickly you can PL your own Decay chart if you want to but the answer is very very quickly and so when you think about the job which is synthesizing all of this very complicated information to make good

decisions what is that mean well you've got this synthesis machine which is this llm thing that's going to help you help you do synthesis but if it hasn't got all that data to do synthesis on top of it's got nothing and so that means that like L&M can only be as good as the the the data they are given and how recent that data is they're ultimately like information shredders they're like they they they are they are you know Limitless information eaters like they just can't be you can never have enough information to give to

an llm to truly gain its value the more things you give it the better it gets broadly speaking that's you know kind of just not perfect but that's close enough and so what that means is a as an internal product leader or you know using putting llms to work you need to figure out how to bring as much information about customers or their asks or your competitors all of it how can you find all of it and bring it together and give it to the llm either in your tooling or even in just copying pasting

or whatever your flow is going to be that's one thing but then if you take it beyond that and you go okay well now I'm a product leader and I'm building an app and I want to put AI in my app what will make my AI experience really great it's definitely not going to be the models because like these models are mostly going to be somewhat replaceable and you could say okay well is it going to be the prompts maybe but you know certainly good prompts are better than others and you certain like that's kind

of an ongoing investment You' probably want to make to ask better questions to get the llm to deliver better answers but it's obvious that the real answer is the context like all the context you're going to give it all the data you're going to copy and paste and so if you think about let's say I'm I'm building a you know I I have no relationship to this but let's say I was trying to build a human capital like a HCM a bot like an AI bot let's say I was working at workday and I was

trying to bring an AI bot it's pretty obvious that the smarts of the bot would really be related to all of the employee information but not just that it would be the benefits information it would be the legal um situation in the country where that person is currently working it would be the company's um policies and procedures that apply to the so you get what I mean by about like these these kind of like the jumps of logic and the jumps of data and the way data is all linked together if you want to have

a smart AI experience you'll convince yourself that all I really need to do is get a model and wire it in and I'll build a little pipeline that will suck some data in and it will whacked into the LM and if you think that way you're going to be very sad very very sad for a very long time because you're constantly going to be wrestling with how do I get data to this thing how do I get good data to this thing how do I get timely data to this thing how do I get well

structured data to this thing and so you know it's a data management problem like it's getting access to good data getting access to high quality data getting access the timely data and getting it to the llm to get the llm to make a smart decision that's where 90% of the calories go maybe it's a bit like Einstein's thing you know it's 10% inspiration 90% perspiration nobody wants to hear it everybody wants to just think about what these really cool models and how smart they are and the next one will be even smarter but but really

it's just the hard work of getting really good data data to the llms to get them to do good things it sounds really obvious as you make this case it makes me think about at the at the lenning friend Summit Mikey creger talked about how he had kind of the two types of PM groups within anthropic one was focusing on user experience product and the other was working on the model research side and they realized that all of the success came from the model research work like making the model and and the data they provided

the model was where all the value came from not just like optimizing the user experience and they're just putting more and more of their product team on just that versus like tweaking ux and buttons and things like that yeah exactly right something sort of related I'm just going to ask one more AI question I don't want every talk to end up being just all AI but something that's kind of been a meme recently and I know you have a perspective on this is that AI makes it really easy to build products so in the future

if you can easily clone say a B2B SAS app like Salesforce or atlassian or whatever whatever your favorite B2B SAS app what happens to these businesses long term do they just become are they all in trouble they going to be hundred Sal Force competitors what's your sense and prediction on what might happen there yeah I think it's really weird um I think people really underestimate where the value is created in these applications and they just kind of get it completely wrong and I'm not sure like why that is so like if you think about so

I spend long time at atasan so I worked a lot on jro which many people know and I spent a long time at Salesforce so I spent a lot of time in the CRM ecosystem the the marketing ecosystem and all the rest of it if you wanted to be like not charitable you'd step back and You' look at all those applications and you'd say they're all just forms on databases you'd say the jira is a form on a database you know workday is a form on a database so sales force they all forms on databases

like all all vertical SAS or business SAS is ultimately forms on databases and you be like well how hard can that be to replicate and the answer is like unbelievably hard like unbelievably hard and people just like totally get it wrong because it's not actually just about you know the data model so if you think about the if it's form some databases it's these beautiful user experiences that sit on top of data models right so whatever the object is it might be a customer object or a you know a campaign object or some or employee

object right you could say that well there's some elements of lock in in the object like the object itself like the fields of the object I'm like pretty boring right that's not very interesting but sure maybe certainly there's some value in being the system of record like the default that everybody uses there's definitely some value in the uxx like the the well you know I want to be the best um H you know HR facing application for working emplo data yeah there there's some value there but the real thing just staring everybody in the face

is it's all about the business rules like that is what drives the lock in because like what why do you buy workday you don't buy workday for it's out of the box configuration you buy workday because you want to configure it to be you know Lenny inks HR processes like it becomes Lenny inks workday it's not it's not Shan inks workday it's l inks workday and actually as you the longer you have the software the more it becomes that the more it becomes less and less like work day more and more like your specific company

which makes sense because it was built to be configured to meet the needs of any specific company and every company's their own press Snowflake and as that happens those configuration pieces that bit that makes the application um Native and a fit for your organization makes it a fit for nobody else's organization and also makes it a black box to the point that you don't even understand how it works anymore like if you went to for example sales force and you said hey could you define all of the you know processes by which software was sold

inside Salesforce they couldn't tell you that without reading the code of their Salesforce instance right I'm not that's not a proprietary secret that's obviously true because over time that's that literally how sales happen there's no other way to do a sale other than through their internal tooling and so what that means is that is that it's not the UI that matters and it's not the uh the data model that matters although those are both very useful it's the years and years and years of evolution of the underlying workflows of the product to support the customers

but also the customers evolving those workflows to make them work the way they do and so how does that impact ai ai companies right you could say it's easier than ever to build and a forms on forms on a database application and so I'm like yeah okay that presumably drives the incremental value of every new one of those to zero right so probably leads to more power to the to the existing winning systems of record because it would just be a gazillion competitors who would just more forms on databases so like how would you ever

choose between them you may as well just go with the winner you know nobody ever gets fired for buying Salesforce or whatever you may as well start from from the kind of the pre Premier vendor that's one element you could go the other way and you could say I've heard a few people Mount this argument which I think is really interesting that um at the end of the day agents are going to take away most of the the use of that user interface so let's say for example your your Salesforce with service Cloud I've heard

people say well you know a lot of those Service agents might end up getting be being replaced with agentic workflows that will mean that you know there is no person operating the UI if the UI doesn't even exist anymore then why do you even need sales force you mean we just have raw database tables and who even needs forms on databases who can literally just have databases but that also doesn't make any sense either because the agents have to operate against the rules of the system and the rules are defined by the business processes so

think about Salesforce without a head imagine Salesforce had no UI it would still have those business rules that I was talking about and those business rules are what Define what the agent should do they're always telling the agent what what it what it should do and how the world can operate what is possible what is allowed and so from my perspective like this idea that this just completely destroys like the the differentiation of of these kind of business business process SAS applications just seems like a fantasy crazy fantasy the only way I could really believe

it is if you said well like you know you could have a new startup that introspected all of the rules that are configured into a Salesforce to try and reverse engineer what your actual business processes are and then kind of operate on top of that but the best place people to do that would be forc themselves or or alasan in elan's case or workday in workday's case you know I don't I just can't see a world in which this like I think one of two things could happen all this move into AI makes makes those

applications even better like even more even more um unable like they basically get stronger it makes the strong stronger or it could enable some new level of like applications that come from a more platform based thing so less a domain specific spe thing like you know HCM or Erp or um you know you know engineering or you know less less of the domain specific stuff it could enable a more platform like play where you have more business objects and business objects have rules and you could imagine a world in which like the there's kind of

a whole evolution of new more platform like sass applications that do more than one business function worth of worth of the business rules and the way things move around in the Enterprise but that doesn't exist today so you could say that that that could exist and you could say it could be way better than it than we've ever thought of because of AI or you could say that the Richer going to get richer like the most likely outcome is that the is that the currently dominant companies are going to get more dominant but I don't

think this idea that it would just cause a spring up of a whole bunch of new new apps that will more easily CH challenge the incumbents makes any particularly it's not straightforward to me how that would happen basically wow that was uh extremely fascinating and there's so much there I can go in so many directions one is uh I thought you would actually go this direction which is distribution advantages become even more important if it's easy to like today I could sit there hire team clone Salesforce might take a while but I could copy it

but by the time I'm done they've evolved their moving their adding features they're ahead right you're skating to where the puck was and so if that's the case one of the advantages one of the ways to get anywhere is to have some kind of distribution Advantage like it's one thing to have Salesforce as a product clone another to get anyone to know about it to adopt it to sell it procurement all that stuff so feel do do you have a sense of like distribution advantages being even more valuable in that world yeah I mean it

certainly makes sense like ultimately at the end of the day distribution is always an advantage because the hardest problem is to even be in the consideration set for any given problem like the world is full of problems it's just when people have that problem they firstly don't think they're going to solve it at all and when they do think of solving it they don't think of you so distribution is always like an incredible Advantage but again like in the world of AI it seems like distribution is more likely to get hard than easy so so

they uh you know if you think about for example diminishing returns on cold email because cold email is getting easier and easier to send even worse spam like it sounds better but it's you know effectively causing everybody to become desensitized everything you know I don't know if you've noticed like half the LinkedIn reach outs now are all basically clearly llm generated spam I mean like to some degree it's actually uh worsening the signal to noise ratio and so I think that a lot of the kind of break break through distribution mechanisms that startups often use

seem to be getting Crowder more crowded just in general and more expensive that doesn't bode well for kind of you know I'm I'm the not as good Salesforce I'm the not as good Salesforce but I'm cheaper it kind of has to be something different you have to there has to be some angle upon which you are materially better and what I saw happening and what I've been seeing happening and I think it's been really interesting is a lot of modern you know nextg applications are bringing data as a first class citizen into the workflow and

I think that that's pretty compelling right so like if you look at the next generation of um you know applicant management um products that deal with you know inbound job applicants a lot of them now like the the latest cool ones they include you know um your time to fill data like uh uh they include outcome data of like who's got the best hiring outcomes who who over what period of time has the worst attrition you know like literally all the way back to the interviewers and where the interviewers were in the interview cycle like

so it basically embeds data into the whole life cycle and I think so I think that there are kind of these ways in which startups can bring these experience benefits by just bringing a kind of different approach to the world that does enable them to capitalize on traditional disruptive innovation like at the end of the at the end of the day this is just disruptive innovation it means that most most companies have overshot the utility like the average utility so you can win by meeting the average utility and being different you know meet the bar

and be different meet the bar and be different is is the way to cut through so that makes sounds like that's a half decent Playbook but if but if you know even for those companies now they're going to have all these AI competitors who are using AI to engineer faster to build a competitor just like them as quickly as possible and start jamming it into the into the channel you it's going to be interesting to see how this whole thing evolves um you know kind of got R to the bottom uh you know characteristics around

it you're probably right that distribution is still the hardest part software when you're getting started right so if you have some kind of uh clever Fair Advantage it feels like that becomes even more powerful say have a platform of of an audience or something like that um you mentioned this ATS product they you really like is there one you want to give some love to you that you think is really cool that you like or you want to keep it Anonymous yeah it's Ashby it's the one all the cool kids are talking about now and

it's funny because like people literally talk about it in comparison to all the even the the you know the last generation of of modern SAS ats's or whatever and they talk about it in glowing ways because of the way they put data in inside the actual workflow so the the actions and its outcomes are directly tieable to each other in the application you're doing to work in I think that's a pretty compelling user experience so just to maybe close this thread before I move in a different direction this point you're making about how valuable data

is and how that's like at the core of being successful and differentiating in the future especially with AI tooling and products any advice you'd give to someone that wants to do that is it just make sure you have a is it like half proprietary data is it like make it a first class citizen like what's the advice you'd give to Founders who are trying to do do this what you're suggesting yeah I mean I think at the end of the day it's kind of all of those things isn't it like if you have first party

data but you can't bring it to Bear then it's not very much use if you have third party data and you bring it to bear in interesting ways like the problem with data is like we're all surrounded by data all the time because there data is everywhere right what really matters is the right data at the right time in the right place because we're all humans right and so and so to me like there are obviously data advantages and there even Data Network effects if you can end up in a situation where you have very

valuable first party data but but you know uh in any case it's still about being able to bring the right data at the right place at the right time for those users to for them to be able to get Advantage from it you know you know like a little um kind of segue I guess on on that one is um is I spent um I know I spent a lot of my career like weirdly acity me I've been a product person for a long time but weirdly I've ended up inheriting data teams I've actually run

data teams at a lot of different companies which is weird because because product managers don't normally own own data teams I think it's because I have just like a really massive affinity for data I've always been really um data I used to call myself data driven it was kind of my kind of my jam and uh because I and and in hindsight I look back and I think like data is like data is the opposite data is more like a compass than a GPS right like if you look at data as a way of like

giving you the answer you're always wrong you're always wrong or you're slow wrong or slow or sometimes both because mostly data doesn't give you the answer it just tells you if what you just said is like ridiculous or you know this potentially something there so it's more like about disproving what whatever you think and you end up being slow because if you try and use data for everything your brain is ultimately a data sifter or whatever so the reason your intuition tells you something is because you've seen a ton of data that tells you that

this the most likely answer and so and so being like data driven being data obsessed is like something you can easily overdo very very very easily overdo so it's about right sizing data having the right data at your fingertips having the right kind of view on data rather than kind of like trying to expect data to give you the answer or trying to use data as a weapon or trying to use data as a way to kind of force people to believe you or to go to go in your direction but data is kind of

at the center of everything and about how to influence and be successful in products you're building and arguments you're mounting internally and everything else all I love that you went there this is I definitely want to spend time on here you it's it's interesting you say that they used to be uh data driven like I was I'm Mr Data driven you created the reforge course data for product managers and also retention engagement course on reforge and by the way we'll link to these you're still you're still helping with these courses by the way they're still

running they're awesome people love them yeah great so we'll Point people to those uh I love that you're also saying you're like I think the way you described it to me before this is your reform data driven PM a lot of people say this they're like don't you know don't just tell don't just do what data tells you to do use your intuition use it as a guide It's hard like on the ground to operationalize that advice what's your s say like you know to your PMs and your teams when they have data telling them

hey this exper this experiment is a huge success or there's a huge onboarding opportun uh conversion opportunity I guess just like what's your Tactical advice to folks that have data telling them one thing and and maybe something else telling them something else I I think the first thing I always encourage people to do is to like look at a piece of data if you're looking at a piece of data and the result tells you something that your intuition tells you is like insanely wrong like that you're probably not right first believe your intuition and go

and prove yourself right like don't just take it at first glance because most of the time it's like aam's Razer the most likely explanation something that is insanely not intuitive is that it's just wrong that there's there's a problem somewhere now occasionally sometimes you actually will be right like now those will be PID moments those are the moments that make it all worth it like there are times when you do find the the the Nugget of goal like you're like you're staring at it and like this is it this was the problem this was the

thing we were looking for this whole time but you have to be very diligent about like following it through like really understanding what you're looking at is this data representative is this data like a good sample of the audience we care about is it already um subject to some sort of selection bias like often times when I see analysis from different product leaders or even data teams you can drive a truck through it like literally drive a truck through it like and and if you if you present data with authority and that data is like

ridiculous or the analysis is just full of holes you don't just not get benefit for that like you lose a whole bunch of brownie points like it would be better not to show up with an analysis that isn't clear then it would be to show up with an analysis that's dumb and I see people self emulate like on this actually relatively regularly uh because they just bring a a knife to a gun fight or whatever they just bring in an analysis that is just not it doesn't hold water and they present it and then get

shot down live which is you know no nobody's idea of a good time um so so kind of if I give you a little bit of additional tactical thing things about that it would be okay uh if I'm looking at a piece of data what was Upstream of this piece of data and does that look normal like so so this thing happened or whatever which you're very very excited about what happened before that and does that match what you think should have been right right so what what happened before this moment a situation and then

okay for that thing that you're looking at what happened after like you if you have an idea of what happened before and after that gives you some idea of whether or not this thing is at all worth interesting interesting to talk about and then go one click above the um above this data that you're looking at so it's like okay these things let's say it's it's um you know I'm looking at onboarding success let's say I'm looking at onboarding success to second week retention or something like that I'm like I have found this thing that

totally crushes it this intervention crushes it if you go upstream and you find out that this intervention only applies to 2% of the inbound onboarding stream it's meaningless it's most likely just a random aberration but even if it was not a random abortion it's not a useful tool right and so you and so you got to go up and then you might go Downstream and you might find yep they last for two in second week but in the third week they all churn they're basically pointless why are we even talking about this or or then

then you might step all the way back and go okay yes those those people do get retained for longer but their average ASP is smaller because what we really care about we do care about engagement and we care about more customers but we want to keep the customers at a high ASP to reach a certain Revenue goal like the final goal is Happy customers paying us money so that's what I mean about like getting going a click up if you go a click to the left a click to the right so before and after and

then a click up and you still see the the the thing that tells you the story that you want to tell then now you've got something that's very compelling because people want to hear about that they want to hear what did happen before what did happen after and why is that why is that outcome happening but you have to really do your homework and really be rigorous about it to to avoid F fighting false gold I love that advice uh ASP what does that stand for by the way oh average sale press aage think Mr

or or some other like Revenue metric got it this point you made about how a lot of times experiments show positive and then they end up not have being anything uh I had the head of growth from Shopify on the podcast and they do this really cool thing where they keep holdouts for years of cohorts and then and then it auto emails them I think a year or two later hey check this and see if this these cohorts are still uh this is still higher or not and 40% of the time turns out neutral after

a positive experiment long term interesting it's really funny because at last we did something similar we had a global holdout group actually that was held out of all experiments there was a and so the experiment platform couldn't Target that group at all so 10% of all people never saw anything ever turns that to be really really helpful because you can always compare them against whatever the experience was for any of the same vintage of of cohort I agree with you but the other thing is I I don't really love some of that thinking process just

in general it's like hey you know um let's say an experiment does show a temporary benefit if an experiment shows a temporary benefit but that benefit does not persist forever does that mean the temporary benefit was never worth it or does that just mean the temporary benefit was an opportunity to reach another level you just didn't capitalize on there's no I don't think there's a perfect answer is what I'm trying to say it's a I don't think that the fact that a benefit doesn't last forever means that you failed but I agree with you that

like not trying to understand well what what has the net benefit been what has the net lift been is also really important too that's why growth is so hard like growth is as a part of product is so especially hard marketers I know that you love tldrs so let me get right to the point Wick Studio gives you everything you need to cater to any client at any scale scale all in one place here's how your workflow could look scale content with Dynamic pages and reusable assets effortlessly FastTrack projects with built-in marketing Integrations like meta

Capi zapier Google ads and more AB test landing pages in days not weeks with intuitive design tools connect the tracking and analytics tools like Google analytics and semrush and capture key business events without the hassle of manual setup manage all your client social media and Communications from a unified dashboard then create schedule and post content across all their channels if you're working on content Rich sites Wick Studio no code CMS lets you build and manage without touching the design and when you're ready for more Wix Studio grows with you add your own code create custom

Integrations with Wix made apis or leverage robust native Business Solutions drive real client growth with Wix Studio go to Wix studio.com so you built the first B2B growth team when you're at lassan correct yes yeah yeah it was makes me feel like an old person but yes was a very long time ago slasher maybe you know it's a new thing we it's yeah it's either a long time ago or it's just we've just recently figured out this is a thing that you could do in a B2B is like focus on growth yeah it is it's

like it's like when I so it was around about 2012 um and at that time kind of grow hacking was a thing I didn't people don't really use that term anymore but in in um B Toc it was a very big deal because people could see Facebook doing their 10 friends in seven days and they could see this kind of thing that was working for people and they're like man that's amazing and at at last time we set out to go okay well do those techniques work in B2B and honestly they you know it's kind

of obvious now that a lot of them do and that it's worth doing but at the time it wasn't that obvious because it for a lot of B2B companies I mean you summarized it earlier then distribution covers all faults like almost all ills can be you know filled in by really great distribution like if you have a really good ground game really good marketing a really good ground game and kind of jamming your product into the channel like you're jamming your product in front of people and you're papering over the ugly Parts with you know

customer success people and services and Consulting and whatever the people will buy almost any software um or you can certainly be successful with with a lot of different software but back in 2012 it wasn't clear of like okay which If instead you wanted this differently and you tried them make software that sell itself is the juice worth the squeeze right and you know now I would say that it's pretty clear that the juice is worth the squeeze to the point that people lots of people think about this all the time um but it was a

bit of an interesting kind of time at that time and that was essentially the beginnings of product like growth is that a simple way to think about it no basically it's now called plg but yeah at that time we didn't even know what to call it exactly just growth right so kind of based on that experience a lot of B2B companies now have growth teams they're investing in growth what makes a great growth team in B2B any things any pitfalls you often find folks fall into that you think they should try to avoid ultimately a

lot of these types of Endeavors are a matter of balance so what I mean by that is um is growth teams tend to go through a set of phases right their first phase is proving their value at all right so so that they call that the Gold Rush phase that's the that's the this thing's probably not worth even doing why are we doing this merry band of people out there trying to prove that there's some growth growth factor somewhere right so so that's the pro it phase and so you know the advantage of that phase

is is like life good because there's usually a lot of gross to be found because nobody's gone looking before so Life's good uh but it's pretty random because you're just literally searching across a random search Spas going have we tried X have we tried y have we tried Z then then kind of once you get that that model going then it starts to be okay um you know how do we scale this thing like is this just a flash in the pan do we just find a little bit of you know low hanging fruit and

there's nothing else here there like was this just a project we should have done rather than rather than an ongoing thing so you have to kind of make it um a system like you have to prove that that it can be repeated and then you have to scale it like it has to become a thing it has to become part of your DNA you have to be taking a p lens to everything you do all the way from you know paid acquisition to um activation retention engagement cross product expansion upsell I mean you name it

like all the different ways you can grow a product by revenue or engagement there's many different ways ways to go about that and so you end up having to scale out and be able to do all of those different things and then you have to figure out how you fit in with the rest of the organization because there's other people who build products all day every day there's other people who sell that product all day all day there's other people who Mark Market that product all day all day and so you know growth organizations are

in this interesting space they're in between everybody else they're kind of in everybody else's sandpit in a in a little bit in a in a little way and they're kind of at the edge of everybody's kind of full-time job and they are very valuable but they can be you know complicated because of all those relationships and because of the way they sit amongst amongst all of the other parts of the organization so so many organizations fail because they don't they don't really find much the wins or when they do find WIS it just seems totally

random or they do find a lot of Wis but they all can't understand them because they just they seem like they're just a random walk through a bunch of the a bunch of potential opportunities like there's many different ways to kind of fail to fit as you go through your growth phase from trying the ideas to success to scaling to operationalizing one of the biggest memes along these lines is a lot of companies claim there's like just plg rarely ever works you always event either you try it and it just doesn't work or it eventually

just Peters out I guess any thoughts on just like what are signs that your product has a chance to work peel product Le growth versus you're just just go straight to sales immediately and don't even worry about this first let's examine the counterfactual right so let's start with the opposite of your question and say Hey you know how would the world be sad sadder if we all just gave up on blg like if we just said hey just there's no point in doing it in in B2B says the problem is that there is not a

natural uh force that pulls companies towards thinking about um the end users enjoyment and success earlier in the early in their Journey there there is no natural force there's no natural countervailing force why is that I mean 101 the buyer is the most important person the economic buyer is the most economic person their their needs are the number one thing they're usually the person driving the RFP they're usually the person dealing with the sales organization so the needs of the person who you hear are usually all feature driven and they're not from the end users

and so you're kind of sewing a seed of your own demise if you don't think about that end user but it's one thing to say that you should think about the end User it's a whole other thing to have a system by which you do that because people people pay lip service to all sorts of things but you know my uh you know I'm sure you've heard this one before but in economics like people only do what Their incentives told them to do like broadly speaking that is what they do that is what happens get

what you set out to measure you get what you give people incentives to do if there is nobody in the organization whose true incentive is to measure that success the the end of success their enjoyment their happiness their retention their engagement early on it will not happen or or at best it will be a hobby right and so then by extension if I start from there then I say okay let's say it doesn't exist PG doesn't exist and therefore it's a hobby and therefore there will be a bunch of hobby people who care about this

then you ask yourself okay will that mean that there will be many products for which like those experiences really suck and does that mean that that will be an opportunity for competitors of those products to be better at that and is that a differentiation differentiate differentiated competitive Advantage yeah i' say it is I'd say it is and so they're kind of working I just work my way backwards and and I go okay you could say that your P investment might be too high you could be like well if I invest more I won't get any

more juice like this is not like I can't spend my life just experimenting in the onboarding like that's not the only thing that matters and that's very very true but it's very hard to argue it should be turned to zero and so so to me therefore it's about the balance it's about okay how does PG fit with the other different ways that I grow in my business so a consant for example we have a PG function we do grow um with um you know self- serve signups people who sign up literally their credit card like

lots of them sign up and they're very successful never speak to us you know we also have like a Enterprise sales team that sells you know directly to very big companies you know some of the biggest banks in the world you know people you would definitely know of I don't think it has to be one or the other I think that you know it's about a balance it's about getting the Motions to work and for really sophisticated companies the people who really nail this it's about making both motions work together like if you can if

you can get a plg motion work to feed your sales team and a sales team motion work to feed your plg funnel when the sales leads AR aren't ready yet and kind of you can get those motions into playing with each other you can make a lot of money it can be an extremely successful way to go to to build a very resilient business why because you get a lot of customers and you get a lot of Revenue like you can't be that successful as a company if you have a lot of Revenue but a

small number of customers because you're captive everyone everyone knows that you can't be that successful as a company if you have a lot of customers but not enough Revenue because you just don't have enough money to sustain operations so the magic is in having both a very large number of customers and a very large amount of Revenue it's very hard to knock over a company like that you know if if I look back on my time at at Asen and I think they they shared their most recent numbers I can't remember what it was but

it was in the public data or whatever something 80,000 or 100,000 customers something like that like that's a lot of customers that's a lot of customers you you let's say you're going up against jir and you're like yeah man I'm going to pick off a thousand customers right from from abas that's a lot right that's a lot obviously a th000 customers is a lot you only have 19 sorry it's it's going to be like U you know 89,000 to go or 79,000 to go however many it is to to go I can't remember their exact

number of customers but like it's very to a sale a company which has a very large number of customers and a very large amount of Revenue uh and so that's why I think that PG as a as a mechanism is incredibly important for almost any type of company if you can make the motion work like obviously there there are companies for whom the motion just is irrelevant but for those where it does matter it seems like the juice is worth to squeeze that was an awesome answer uh I looked up last year they have 300,000

customers oh man I'm so far off when when I left must 000 customers they've done good work since then also you're talking about incentives and how the the power of incentives Charlie Monger has this great quote I looked up just to make sure I get it right show me the incentive and I will show you the outcome yeah exactly right exactly right you know I've seen I've seen like um cases where like a sales team was people trying to get a sales team to do like a p motion and you can beat them over the

head as much as you like you can get into a meeting and tell them that you really really want them to do this but at the end of the day like they're not going to do it and the same is true for every other kind of like function it's just the nature of things I have some newsletter posts around the stuff if folks want to dig deeper also um Elena Verna had an awesome podcast episode talking about product let sales and kind of the combination of these two things that we'll point to this like a

whole other topic we can go deep deep on but we're not going to do that in this episode maybe just one more question so you mentioned all the companies you worked at so you've been at Salesforce G product officer U moft specifically within Salesforce uh metromile glassan confluent now a lot of really interesting and different roles how do you choose where to go work and how do you choose which opportunities to take I imagine you have many options I have to think of M only in hindsight looking at it this way Lenny so like I'm

just I don't know if forward looking it was obvious to me this way but looking back my career's been a little bit like a bingo card like I've always been looking for to fill in boxes I didn't have filled because I felt like that would make me a better professional it's it's like if I didn't know anything about that specific type of sales model or that type of marketing or that type of product management or that type of product or that layer in the stack or that kind of thing is like well if I learn

about that thing I will become more versatile so actually two things I will it's fun it's fun to learn something new it's fun to prove to yourself that you can do those new things and then it makes you more versatile because it means that any given problem you go up against you've seen something that that patent matches to it like it kind of feels like you end up bringing a gun to a knife fight in a way because every problem you look at you're like oh I have seen this from the other side like I've

seen this from some other angle and so I know that this is likely to work and this is unlikely to work and so you know when I when I joined you know early early on in my career I was working for a big Enterprise software company sorry small Enterprise software company that sold to the Fortune 100 and I joined at lassan and like I shared with you we had no Salesforce at all actually at all literally nobody to sell the software it sold itself or didn't get sold at all and we grew to have 880,000

customers like it was just pure product that growth and just an incredible company then it was at metral which was a consumer company uh that got acquired made an insurance product for end consumers so like got nothing to do with technology products like literally a complicated um Internet of Things device you installed in your car but ultimately it's an insurance product that you'd sell to grandmothers in in Florida as much as you would Urban millennial so and then also have totally backend software that's used by it organizations and a consant you know infrastructure that's used

by um developers everywhere to build really interesting data driven applications data powered applications to do all sorts of things in real time and you look across all that and you go it's all a bit random right um but like I didn't see it that way because like I learned you know I actually was in sales for a bit so so I was I ran a pre-sales Engineering Group went around the world selling software so when I joined at Lan I wanted to kind of understand what it was to sell s for a massive scale with

no sales team like can it even be done right and so I learned a lot in my time at and when I went to metral I'm like well I've never built a consumer product before like I can say that I've actually built a product that's touched many millions of people because J has so I felt pretty good about that but I'd never built one that I could say yep a consumer your average consumer can use this thing it's so simple even my grandma can use it I'd never built a product like that uh so I

got that experience at metrom which is really fun I'd never worked inside an organization as big a Salesforce or an organization with with as good a sales motion like the you know you talked about distribution earlier Salesforce is an absolutely insane distribution machine like just an incredible company with just an amazing distribution uh Network and a fantastic marketing like approach that like it's like a PhD in marketing you know when when you spend your time uh at Salesforce you're like this company is just oneof a kind it's a one-of kind and it's so outlandishly good

at one specific thing and so looking back you know all of these jobs have been when I say bingo card like I've just got an outlandish education in these areas that you know are not obvious at all and once you've seen them they're like superpowers they're superpowers to be able to bring that to bring that same experience to be to bear on things and so one thing that I really I'm trying to figure out as why often people don't do that and often times people um stay in a very specific domain like they prefer to

stay in a domain or they prefer to stay in a specific kind of type of company or very or a role that works in a certain way like companies that have the same operating model or they plan the same way or they kind of they try to stay with things that are pretty similar but it seems obvious that the most likely way to to kind of really grow is the opposite it's to constantly be choosing things that are that are outside that not totally outside the lines like don't don't jump out of a plane if

you've never parachuted before like obviously you want them to be in some way an adjacency you know that you want them to have something in common with with what you know but you want them to stretch you and change you you know I uh I had like a really kind of a transformative experience many many years ago when I was at um when I was at alasan and a guy called Tom Kennedy he was our general counsel so like Chief legal officer basically and a lifelong lawyer very smart guy I liked him very very much

but like just a lawyer a lawyer you know corporate lawyer corporate Council I'm sure you know what they're like and really great guy and I remember so mostly in our meetings like our meetings he didn't talk that much except about legal things right but I remember um in one meeting we were having this vigorous debate about a product strategy question about like what we should do should we go should we go left or should we go right and like as usual he's there and he's mostly just staying silent and then eventually the conversation's been going

on for 15 minutes and he's like hey everybody like a year ago we talked about X Y and Z and he proceedes to lay out our product strategy at that time and he's like just recently we said the following things and that was a product strategy whatever now you're saying this isn't it obvious that that isn't this like what you guys are saying is not congruent with that and if you really meant what you said back then we should be doing X and again like the room went silent everybody kind of turned to him kind

of nodded and then everyone yeah okay I guess we probably should be doing it differently and so like the meeting stopped like when the GC randomly mentioned that he like deeply understood our product strategy and he knew enough to like be able to contribute in that way and so like the the life-changing part for me about that was just this realization that if I'm going to be a really great professional if I the type of professional I want to be is is that type of person the type of person who can contribute to the whole

company in all sorts of ways like doesn't spend all of their time in everybody else's business but understands the business and has the you know mental horsepower and the experience to be dangerous in all sorts of and I mean that in a compliment way I don't mean that in the negative way but to be dangerous in all sorts of situations I think that when you have kind of like leaders like that behind you and with you then you're just Unstoppable you're an Unstoppable force in business when you kind of have that that motion happening wow

that was an awesome story and an awesome uh perspective it's similar to the advice I always give PMS of people always wondering should I like go deep on a specific subject should I just try different things and I find just variety especially earlier in your career is really powerful not just to help you discover the thing you like but also to your point just using insights from all these different parts of the product and like internal tools and trust and safety and platform and consumer product side and growth and just core stuff like the more

of that you have the stronger you get and I think I feel like another benefit of your approach is if you if you work at just B2B SAS companies you're never gonna like if you have too many of that on your resume it's very hard to get hired a consumer company and so just having it creates a huge optionality for you if you do what you did yeah it's interesting people used to talk about people who were t-shaped or whatever and I've never really loved the analogy because it's more like people are scribbl shaped like

I mean like there's the really best people you've worked with they're more like scribbles than they are um t-shaped because of course you want to be horizontally capable so you want to be be Broad and you do want to be deep but you actually want to be deep in way more than one thing now obviously when I say deep I don't mean like like I'm not able to do the job of like you know our finance function all day every day but I'm 100% good enough to go like three clicks below like the simple financial

analysis like like I can go reasonably deep in our financials because I want to and because it's partly like it matters like it's important to be able to do that and so so maybe a different way to think about that bingo card is like I've really regretted going deep in something that isn't quite my job like I've really regretted it like the worst case scenario is I've learned something new that I willon never use which you know I guess at least that made my brain slightly more agile like I don't know like there must be

some potential benefit of that but the very best case scenario is that when I least suspect it at some point in the future it will turn out to be the thing that matters like it will be it will be the tool that I need when I'm facing some important problem and I will be like oh my God this was worth every scent and so like if you think about it on an Roi basis doing things that aren't in your wheelhouse like that aren't the things directly in front of you the ROI can really be outlandish

like it can be off the chart great but I guess it's speculative because it's you know you don't know you're going to need it tomorrow you don't know if it's going to be something that's going to be a regular tool you use it's interesting use the bingo card as the analogy what are you trying to is there a bingo moment at the end of this is it retirement is there oh you you mean like you've got everything you've gotle Pokemon collect Pokemon collect them all yeah I was I was working with um you know somebody

at Salesforce and you know he was like a really he'd been there a long time very you know um very very very successful person like honestly uh you know didn't need to work anymore and and he said to me that I found really useful he's like well now I'm at the point of my life where I want to work at the intersection of things that I am good at and things that will be valuable to the company to do so basically like it feels like the reward of completing your bingo card is actually to just

get to spend more time doing things that are leverage that you enjoy and that are high leverage uh and so that seems like a good outcome to me like if if you it's not as though going to I don't think most people are going to like work and hopefully have some sort of great financial outcome and then go well that's it I'm picking up stumps I'm retiring uh I think for most people achieving some sort of financial outcome or some sort of um you know uh Independence or whatever is really just another stage it'll be

at that point it will be okay well now what do I do like what do I do with my life like why and so that was why I said earlier that at the end of the day product management is like at times the worst job in the world and at times easily the best like and and it's both and it can be both and so you know it's hard for me to think about what you know if I think about the things that are insection of what I'm good at and are valuable to the world

product management is a pretty fun one to do and it's different every day so I think we're pretty privileged for those of you who listen I mean obviously your podcast reaches a lot of product people like I think we're pretty privileged to be able to operate at that um intersection but it's not easy because um you know you got to show value you know it's like it's not it's a very complicated job to show value in and to demonstrate value to the world and it's constantly being attacked like you mentioned but it's still amazing like

when when it all goes right you know when when a product is very successful in the market it's hard to describe the joy you get from from that kind of along those lines to close out our conversation before a very exciting lighning round I want to take us to failure Corner people hear listen to these podcast episodes and everyone's always just sharing all these wins everything's always going great the CPO of this CPO of that just moving on up and they people really want to hear times when things didn't go right because that's those are

stories people don't share as often can you share a story when something didn't go right when you maybe had a failure in the course of your career and if you learned something from that experience what you learned I mean there's a lot of things that didn't go exactly the plan Lenny um like very early on in my career I uh you know I was a s developer and I accidentally um deleted like one of the core systems of the of the company that I was working at um so that's that's going to go down in

INF for me but luckily that one's far in the review mirror like that wasn't it last year no no no that was pre that was far prean but very bad uh yeah you know the one I like to talk about I wasn't I wasn't directly responsible for it but I feel like responsible for it I was at a company and we launched a product that was one of those products that um you know in hindsight should have been really obvious it was going to fail but for some reason we were all blinded by the potential

it was a it was a product that um that was about it was basically for to measure the environmental impact of your company and to help you reduce the environmental impact of your company by doing think about like power management building power management managing the power drawer of computers managing the power drawer of you know AC and all of that stuff that was division basically it's like a kind of a manage your environmental impact of your business kind of the idea was pretty cool at the time and also it was the right time for that

and it's still a thing it's still an area of active research and investment or whatever but it was like one of those things talk about the wrong company wrong players wrong time wrong distribution like we had literally no right to win no right to play like just absolutely no business in hindsight being in that in that business and I feel really bad because like again good idea wrong wrong company and and at the end of the day uh we launched the product we actually kept the product in in market for 2 years and the and

the final the final straw was weird the final straw was actually when a when a customer finally wanted to pay for it like it it had been in market for two years and we found ourselves with a customer who wanted to pay millions of dollars for it they were ready to sign on the dotted line and that was actually the moment we decided to kill the product because well if anybody if this per person signs this piece of paper we are stuck with this forever like this this one this One customer will be bound by

contracts for her for long or whatever like so we actually ended up killing it the at the moment after two years of of um of failure when kind of somebody wanted to pay his money for it and I I looked back on that and I'm just like man like that was a really big I feel really bad because like it should have been obvious it should have been it was obvious and we should have been able to call a spade a spade and I guess speak truth to power um but instead it kind of got

through to the keeper and turns out to be a real accidental drain on resources for years and just a big mistake so is the lesson there uh just be real with yourself just yeah like I like that you had this forcing function of like okay this get for real now is it like I wish we had a earlier forcing function to force us to make a decision yeah yeah I think if if I could do it differently like I probably I might not have necessarily been able to 100% change the decision but I should have

tried like I mean it was pretty obvious after 6 months like this thing was like a bit of a zombie product and it would you know it would the least I could have done is said like this thing is dead could have called it dead way earlier but instead we proceeded for another year and a half uh investing in it and so that that's the bit that makes me kind of feel like a real bummer about it it reminds me of recent episode with Ros who is the CMO at whiz and she joined us the

first pm and a few weeks into it with doing tons of calls with customer she's like I I think I this I need to quit because I don't really understand what we were building I don't get it and everyone's like you know like I don't I don't either and it just yeah the com the founders just had a vague idea what they were doing but they didn't really have an idea and that just sparked okay wait you're no one actually does let's actually get more concrete and it helped them pivot and now I don't know

if you know about whiz but they ended up being the fastest growing startup in history you see isn't that amazing right you know like it doesn't it's not doesn't mean it's permanently fatal but asking that question and going to go through that Reckoning turns out they came out came out stronger scary but turns out it's for the best often before we get to very exciting lightning around is there anything else that you want to mention or leave listeners with maybe a last nugget something that you think might be helpful before we rrap maybe a couple

of different things that that that I think uh sometimes well understood but just repeating them I guess because they're very valuable to me one is um that uh like if you let your calendar rule you then nothing good will happen like you know I know people talk about that a lot but it's surprisingly common in product management in particular that people end up ruled by their calendar and so it's related to that whole look at spend 80% of your time thinking about things going on outside the buide the business easy said very hard to do

and if you don't do it no one's going to do it for you and so like it's really hard to be successful unless you find a way to force that to happen and so to repeat that I kind of also like somebody said this to me I never actually looked up the quote but apparently pal said that if you're making a decision with less than 30% of the available decision 30% of the available data you're making a big mistake if you're making a decision only after you have 70% it was either 70% or 77% I

can't remember the exact number when you have 77% of all the available data you have waited far too more right and that's really I've always found that like very insightful and it bit relates a little bit to what we're talking about about data earlier but at the end of the day they we get paid in product management to make decisions good decisions paid to make good decisions that will deliver business benefit and a decision with too little data is fatal a decision that takes too long and collects too much data is also fatal so like

everything it's about trying to find the balance of all of these different things to try and deliver business Advantage a great way to Circle back to all the things we've been talking about with that we' reached our very exciting lightning round are you ready yes let's do it let's do it what are two or three books that you have recommended most to other people yeah um they they all but goodies is probably going to be uh the Lan startup that I still find actually really good and the kind of key lessons in there I still

think are very applicable to a lot of people particularly the cohort analysis bit which for some reason I still don't see people do anywhere near enough cohort analysis so there you go that's my little tip and then uh inspired how to how to build products that people love by Marty Kagan and the Silicon Valley product group that's an oldie B goodie think you know it's got a lot of the key lessons of product management in it even though it's been around for a long time those are some Classics very cool do you have a favorite

recent movie or TV show you really enjoyed I'm watching a program like just I don't get to watch very much TV mostly at night I like to watch things that are extremely light that like just don't at all Inspire any element of stress and that are very short so basically short and funny is basically my thing and there's a new program on Netflix I think it's called detroiters I've been watching that yeah it's really funny I really like that it's so ridiculous but very funny so like that that Main guy he's so funny I forget

his name Tim Sweeny or something like that yeah he's so good good one I've been watching that I'm loving it it's like very quirky I think the New York Times quote on there like very weird the quote so weird like in the first episode I'm like what is this show it's not even clear what time it set in and like it's very weird it's really cool yes good way to describe it uh next question do you have a favorite product you've recently discovered that you really love uh yeah this one's like some of your some

of your listeners might be using it but glean like it's a pretty well-known startup now they recently raised a ton of money we've been using um Gina confluent for a long time uh and it's just amazing like it's just amazing okay I can't I can't describe how good it is and I don't say this lightly because you know I think search like a business search is probably one of the hardest problems in Computing actually getting it right is one of the hardest problems in Computing amazing it's not often I use a product and I'm like

this thing is like 10 times better than anything that's come before it it's one of those for me what's the simplest way to understand what it does for you it searches all of our organizations knowledge so like the the thing you were just saying before you're like um you know what does uh ASP mean right if I had that in the meeting I just open my my new tab it automatically take over my new tab or just be like what does ASP mean and it will summarize back to me what ASP means and it will

give me a link to all the documents inside our company that describe what ASP means and then it will tell me who the expert in ASP at our company is like it's like just it's like having a second brain it's like an insanely cool um kind of organization search thing great tip okay two more questions do you have a favorite life motto that you come back to share with folks find useful in work their in life uh I think about this one a lot um you know when I started off in my career I was

a very I was a you know an engineer's engineer I used to very much about like technical correctness and what computers were capable of and kind of technical righteousness you know the right answer rather than you know there is only one right answer and whatever it's a long-winded way of saying that I often think about this phrase which is um people don't care what you know until they know that you care and uh and so I've realized that really being able to influence people it doesn't matter about whether you're not you're right or whether you're

not you're wrong and at the end of the day it's first about trust and about relationships and caring about what each other's out outcomes are what their incentives are and all good things sit on top of that like you once you have those kind of Foundations then you can build like really good Partnerships and that's where you know good progress comes from wow that is so good it connects with like radical cander is similar like in theory of just carrying they need people need to feel like you care deeply about them before they take your

advice and it also connects with this parenting book I'm reading called listen that a previous guest recommended which is all about how your kids C have problems when they feel like your connection to them is weak and so the solution is to build a stronger connection for them to know that you care deeply about them so this is really connects with so much what I've been reading yeah exactly great one final question you're born in Sydney folks can may be guessed by your accent if someone were to visit Sydney any tips anything they think you

think they should check out favorite thing in Sydney yeah Sydney is a really beautiful city and like it's kind of famous for its beaches and it's a basically a metropolitan City people probably be very surprised when you visit it it's a very big city very Metropolitan a little bit like New York but New York with really beautiful beaches if you want to think about it that way it's kind of crazy uh but there's actually like a ton of really cool nature and beautiful things all around Sydney and so if you want to do something like

off the beaten path you can actually go to there's an area called the Blue Mountains which is like an hour and a half drive from Sydney and you can have sail down a waterfall which is well actually firstly you go canyoning through through a through a canyon full of water and then you have sale of for waterfall at the end and if you're looking for like just a really beautiful fun kind of Adventure like thing an hour and a bit away from a massive Metropolitan City that's my sort of happy place like really beautiful Outdoors

stuff while also next to a beautiful city and you said you sail what sort of sale off of waterfall sale you might think of it as repelling repelling I think yeah lowering yourself down on a on a rope or got it okay because when I hear sail I'm like thinking a boat just jumps through over the waterfall oh no no no ab ab sailing which is also I think in the States you guys called it repelling repelling yeah wow very cool Sean you're awesome this was extremely cool thank you so much for being here two

final questions working folks finding online if they want to reach out also Point folks to your reforge courses that you created and uh final question how can listeners be useful to you sure yeah so my reforge courses you can check them all out at reforge tocom as you mentioned the retention engagement course and the um data for product managers course so you know love to see folks get some value from that lots of people have been through those courses already and I really get a lot of value from it because like I said one of

my goals is to like help all of us be better product people I think our leverage could be massive um where you can get in touch with me obviously LinkedIn but also Sean M clouds on X if you want to get in touch uh and in terms of you know being useful to me I mean broadly speaking like I'm always open to new ideas like if people have ideas about how to do better B2B um PG better B2B um you know product Le sales for example better better ways of going about distribution and product Le

sales and product gross inside Enterprise companies hey I'm I'm open to learn myself we're all we're all in one big Journey learning how to do this better so true Sean thank you so much for being here awesome thank you very much the name was great bye everyone thank you so much for listening if you found this valuable you can subscribe to the show on Apple podcast Spotify or your favorite podcast app also please consider giving us a rating or leaving a review as that really helps other listeners find the podcast you can you find all

past episodes or learn more about the show at Lenny podcast.com see you in the next episode