Introduction to Modern Product Discovery - Teresa Torres

[Music] hi everybody I want to clear up a misunderstanding don't worry it's not about brainstorming a number of people have asked me why I have a Portuguese name so let's just talk about that for a minute as you can tell probably from how I speak English I'm an American in the United States a lot of us are mutts I don't know if you're familiar with that term but we come from multiple ethnicities my father is both of Spanish descent and Mexican descent hence I say my name is Teresa Taurus which is maybe the most American

way to pronounce this name so if you come up to me and speak Portuguese I might try to understand what you're saying but mostly I'll say I'm sorry so now that we've gotten that cleared up let's talk about modern product discovery so I work as a disc a product discovery coach and so naturally that means the most common question I get is what's that how many of you guys have are familiar with the term product discovery some of you okay good so don't worry no knowledge is required we're gonna start at the very beginning we're

gonna do a little walk through history and we're going to take a look at the very end what's coming next so that as these things start to come into your world you have a way of making sense of all of the rapid changes that are happening in the world of developing products so the best way to think about product discovery is to think about it in relation to product delivery now if you build products you do both of these things it's not possible to build a product without doing both discovery and delivery so discovery encompasses

all the activities that we do to decide what to build right it's all the decisions around what should we build next whereas delivery is all the activities that we're doing to you know write code ship package things into releases ship product it's how we're delivering the value that we're creating to our customers now the benefit distinguishing discovery from delivery is that most companies overestimate overemphasize delivery and under emphasized discovery and when we separate these two things that allows us to say how are we doing in each are we any good at discovery are we any

good at delivery and if we look over the last 15 or so years especially on the delivery side we've seen a lot of a lot of progress and we have a really clear yardstick probably every single one of us in this room would agree that on the delivery side our goal is to ship value as quickly as possible so once we've decided what to build our goal is a product team is to get it out of the door as fast as possible and we've seen this in our delivery practices they've evolved rapidly over the last

decade and a half and we've seen teams go from releasing once a year to releasing every quarter to releasing every month to releasing every week and many teams are now releasing software as soon as their software developers are done writing in it they're releasing many many many times a day and they're getting value to their customers as fast as is humanly possible now it's great no matter some of you might be looking at this and say wow there's no way we'll ever release every day and that's fun right what's good about this clear yardstick of

how can we ship faster is that no matter where we are as a company we know what good looks like we can look to the future and say eventually we want a ship we want to get faster and faster and as we're evaluating our delivery practices we can evaluate them based on how fast are our release cycles and we all know what good looks like so when I started to think about this I naturally asked what's the equivalent for product discovery what's our goal and how do we know if we're any good at this and

given that so many of us are dude product discovery differently how do we know if we're doing a good job so when I started to ask this question I looked back over time and I was actually pleasantly surprised to see that product discovery has actually had a very similar evolution to product delivery and around the same time period so I want to walk through the fifteen years briefly and talk about what have we seen in the world of product discovery and then I want to use that to help you put all of those trends into

context and to know what to use when so that we're not just guessing about what tool in our toolbox to use at any moment in time so when I go all the way back to the year 2000 when Windows launched Windows me now I have no idea if this product even made it to Europe because it was a complete flop this was one of Microsoft's worst products it was their follow up to Windows 98 it was launched in June of 2000 by 2004 when Microsoft was already announcing they didn't want to support it anymore and

by 2006 they ended support for it now what led to this failure let's think about what product management looked like during this time period now I've never worked with Microsoft I've never been a client of mine I have no inside knowledge about how this product was built so at this point I'm going to describe how many companies worked during this time period and I'm going to assume this is what things look like at Microsoft a product manager or a number of product managers probably spent months gathering requirements from internal business stakeholders they documented it in

a really long product requirements document they sent it off to their engineering team who spent months if not years building to that document when they were done they stamped their software onto a CD they packaged it in a shrink wrap box that looked a lot like this they sent it to stores and only when it still sat on the store shelves later did they realize nobody wanted their product so if we look at this model of how do we decide what to build it's really clear now 16 years later to say wow this is not

a very effective model maybe we should learn before we ship our products to stores that nobody wants it how can we avoid flops long before we finish building the whole thing and it's this type of question is really driving the evolution of product discovery so fortunately even at the time that this product was launched there was a lot of PEEP in the software community who were already frustrated with this way of building products they a lot of nobody wants to spend years working on a product only to learn at the end that nobody wants it

and so a bunch of people came together they started sharing what they were doing they started asking what could we do differently and a lot of this frustration culminated in 2001 with the release of the agile manifesto now the agile mindset gives us a lot of value it's there's a whole lot of things that agile helps us with I want to highlight one thing in particular on this road to the evolution of discovery that I think is a really important part of the story so in the waterfall method we often spent months if not years

building a product and we would periodically tell our business stakeholders how things were going we would often learn that we were on the wrong track we weren't building what they wanted because product requirements documents are a pretty terrible way to communicate language is very vague we misunderstood the requirements and we got further and further away from what our business stakeholders wanted and for those of us that have switched to agile or even early in our agile transformation what we start to see is one of the great things that agile encourages us to do is two

things to one develop in much smaller batch sizes so don't write code for months and months and months write code for a couple of weeks and then show it to people and see if you're on the right track now for many of us when we first started with agile show it to people meant our internal business stakeholders so we started to ask the question earlier in the process are we building what our business wants this was a good step forward but it around the same exact time period we saw another trend a trend that was

pushing on another dimension asking a different question and that question was not do are we building what our stakeholders want but are we building products that our customers and know how to use so right around the same time that people started pushing on agile we started seeing the rise of user experience design and design thinking and we started to see so if we think back to the early tooth thousands we started to see teams were starting to hire interaction designers we the design world was debating about information architecture versus interaction design versus user experience design

good times and we also saw the popularity of Design Thinking and we started talking about building an empathy for our customers and it was no longer just about building for our stakeholders and we've got a lot more focused on building for customers this was phenomenal this these two trends together I think helped us take a giant step forward in informing how we decide what to build recently about five years ago we saw another giant step forward and this step forward actually a number of people contributed to this movement but I think one the release of

a specific book really propelled us forward many of you may guess what it was but about five years ago Eric Ries launched the Lean Startup and the Lean Startup was really informed by Steve blanks work so I want to make sure that we give him credit as well and this is where we didn't just ask are we building something that customers can use but we started to ask are we building things that our customers want now user experience designers of course also started asking this question I'm going to give a simple narrative to walk through

our history here but this is where we started to ask an even deeper question do customers want our solutions and what's great is both Steve Blank and Eric Ries started to give us a lot of tools for how to answer those questions right we started to learn about MVPs we've Ryan through the build measure learn cycle really quickly we're learning earlier in the process and then more recently we're starting to see the popularity of yet another tool and that's the jobs to be done framework and again the jobs to be done framework was popularized recently

by Clayton Christensen but it's based on a lot of work that was done by Anthony Novick and basically what we're seeing now is we're starting to ask a different question we're not just asking do customers want our solutions but are we solving the right problems for our customers so I can create a solution to a problem that you have but it's not as compelling as a solution to a much bigger problem that you have right so we've moved from just focusing on solutions to starting to ask are we solving important problems now this evolution is

not stopping right we're still gonna see new tools new frameworks in fact even in the last 12 months we've seen a giant push in popularity of two new methodologies one is the design sprint so this book came out earlier this year it's a five-day process we're on Monday a team starts with a big meaty challenge they have no idea how to solve and by Friday they've prototyped a solution and tested it with real customers and it's a fantastic way of introducing human-centered experiment driven product management to a team and I suspect many of us are

just starting to think about how do we use this tool the second trends that is really risen in popularity this year thanks to Google building on work by Intel is the use of objectives and key results or okrs and if you aren't familiar with them that's fine the details are what matters right now is not the details of all of these methodologies it's just that our practices continue to get better and ok ours that are a way for us to think about how do we focus on outcomes and how do we align everybody in our

organization around the same outcomes and the world of product management this is really important because we get obsessed with features and we shouldn't be obsessed with features we should be obsessed with what outcomes are we creating for our customers and for our businesses and this is where something like okay ours are really helping product teams focus on what's that outcome that I'm driving and so I started to look at all these trends over the last 15 or 16 years and I thought wow it's been a good couple of decades for product discovery this is awesome

but here's the thing most teams that I talk to have no idea what product discovery is they know these tools but they don't know what to use when they don't know people are dogmatic they so the only way to build products but the lien is with Lean Startup let's say the only way to undercover users needs is to use the jobs to be done in framework this is doing us a disservice a lot of these methods come from the exact same underlying principles and it's not about one's the best and another one is no good

it's not about design thinking versus lean it's not about agile versus well it is kind of about agile versus waterfall but really here's the idea like art we have all these tools and frameworks because all of us are pushing to get better and we're pushing together really fast and so we're seeing a lot of different methodologies that have a lot of overlap and so when I look across all these methodologies the question I ask is how do we know if these are helping us and how do we know what to use when and is a

product team being constantly inundated with all these methodologies how do I decide which ones are right for my team and how do I know what to use one so when I go back to the original question which is for product discovery what's our goal our goal is to learn fast instead of learning after we ship our products that nobody wants it we want to learn as quickly as possible if what we think we should build is the right thing we to build and every single time we learn about a new tool or a new technique

we want to ask ourselves what does this help me learn quicker than what I was doing before if we look at this same history we can see that with each technology with each methodology we're starting to answer different questions earlier in the process so one of the big questions we start to answer when going from waterfall to agile that we can answer much quicker is are we meeting our stakeholders needs right we don't wait six months before we say here's what I built we say every two weeks I built a little bit more what do

you think so this is great it really helps us move that learning earlier in the process when we get from user experience design as a different question if we build any usability testing early in our development cycle we're able to answer ten customers use it so we can answer this long before we ship any product which gives us time to fix it if the answer is no now this in 2016 a lot of this now sounds obvious but for those of us that we're building products in the late 90s and early 2000s we used to

have to fight to run usability studies right so being able to answer this question really early is phenomenal and then thanks to people like Steve Blank and Eric Ries we're now asking an even more important question much earlier in the process which is do customers want our solution right we're not just assuming this is the best thing in the world we're leaving some room for humility and doubt and we're saying we might be wrong how can we learn that earlier in the process and then again with jobs to be done and with other opportunity finding

methods we're now asking much earlier in the process in the process are we solving a problem customers care about so in jobs to be done they're asking what job is your customer hiring your product to do what problem is so important to them what problem or challenge or opportunity is so important to them they're willing to hire your product and how do you make sure your product really meets that need so this is a great deep question that we're now asking much earlier in the process and if I look across all of these questions and

if I mix and match these methods and I really learn all of this early on long before I spent months building something really what I'm able to do is I'm able to answer the most important question which is are we driving towards a desired outcome as a product team my goal is to increase engagement to reduce churn to increase revenue to increase customer satisfaction to make my NPS score go up right it's not to just build feature a B and C and what's great is if I look over this exact same progression of 15 years

what I see in these questions is we're we're shifting from a focus on output and my meeting my stakeholders needs to a focus on outcomes am i having an impact on my customers lives am i improving the value of my business and what's great about making this shift is now we all can look at what good luck we now have a yardstick for what good looks like no matter where you are in this process if you're just new to usability testing if you're just new to experimenting you know that your goal is how do I

learn fast how do I learn faster than I did last time and so no matter where you are in this list of questions even if you're only answering some of them or you're answering all of them you can take stock and look at where you are and say ok I may not be I do not have confidence that I'm solving the right problems but at least I know that people want my solution and that's good and I now know what the next step is so no matter where you are on your product discovery process you

now have a yardstick for what good looks like and this is really powerful I think we've been missing this for a really long time but now as an industry we're starting to realize learning and learning quickly is the most important thing now I work as a product coach and so what that means is I work with a product team usually between for 3 to 5 months and I'm working with them week over week in the context of their own work when I first started doing this what I was teaching teams was how to conduct good

customer interviews how to run sound experiments how to draw good conclusions from those research activities and I would get really good feedback but I would always also get the same question over and over again what my team's would say to me is Teresa we love that you're helping us do interviews and we love that you're helping us run experiments but we still don't know what to use when you always have to tell us what to do next how are you doing that what how do you know when to talk to a customer versus when to

run an experiment versus when you're ready to go ahead and move forward with your product and I started to think about this question and it seemed really intuitive to me and I said well if I'm doing this into it how am I supposed to teach another team how to do this if fortunately at the same time that I was asking this question I was reading a book called peek peek was written by Anders Ericsson he's a researcher who for most of his academic career has studied expertise and he looked at he looked at what makes

experts stand out from novices and a big part of his research has found that experts rely on stronger mental representations that novices don't have so a mental representation is what's a structure that you have internally in your head that's helping you organize information that's helping you make sense of the world around you and helping and helping you make decisions quicker and easier and so I started to ask am I using a mental representation to guide my teams through this messy land of product discovery and it turns out I do and I started to play with

it I started to whiteboard it this became my design challenge and for the last five or six months I've been using it with my product teams and of all the things I've ever taught this this visual structure has at a bigger impact on improving the quality of product decisions anything else that I've ever seen and it's because it's a critical thinking aid it helps people see their thinking examine the connections and really question are we building the right thing and so I want to share it with you today it's called the opportunity solution tree this

is not rocket science if you're familiar with decision trees it's really just a decision tree that helps you make sense of this messy world of product discovery and when I started to play with this visual structure I realized there's a huge gap in the discovery world that we're just now starting to fill so at the top of the tree is the blue box and the blue box is a clear desire to outcome now this sounds obvious how are we supposed to build a product that drives the outcome if we don't know what the outcome is

but I can't tell you how many product teams I meet they have no idea what their desired outcome is they say things like we're just trying to make the user experience it's better how are we measuring that how are we going to know when we've done that so one of the things I love about the popularity of okay ours and okay are is a qualitative objective combined with quantitative key results so my objective might be to have the easiest diagramming tool but my key results forced me to say how will I know if I'm making

progress towards that objective and I have to come up with a quantitative measure now this is really important the root of this tree needs to be a quantitative measure because by the time we get to the very bottom and we're running experiments we're going to evaluate our experiments based on that quantitative measure so if our goal is to increase engagement by 10% every experiment we run while searching for what will drive that desired outcome is going to be measured by how much it impacts engagement okay so first step one define a really clear desired outcome

it sounds simple we all know we should do it sometimes I spend weeks with teams on this because if you don't have a metrics driven culture you're not going to agree and it's going to take some time but if you don't have a clear desired outcome you're not going to drive a desired outcome the second step is the one that I see missing on 98% of product teams and I think jobs to be done is going to help us fix this but the second step is we need to discover the opportunities that are going to

drive that desired outcome and so what I mean by opportunities it's a little bit jargon it's because I'm trying to appease people who have problems with the word problem right so if we think about this from a problem solving lens we have to define the problem before we can solve it if you're familiar with the Einstein quote if I had an hour to solve a problem I would spend 55 minutes first defining the problem in the last five minutes solving it that's this idea I mentioned it earlier in the panel how we frame the problem

has a really big influence on the types and quality of solutions we can generate but as product teams we often skip this step somebody says to us your goal is to go increase engagement and we start brainstorming solutions I promised I wouldn't talk about brainstorming we start generating solutions right and the challenge here is we have no way of evaluating solutions if we don't know what problem we're trying to solve now the reason why I call them opportunities and not problems is because not problems encourages us to fix things and sometimes things aren't broken we

can just make them better right so op think of an opportunity is a pain point it's a need it's sometimes it's just a want or a desire right like Elon Musk really wants to go to Mars that's not really a pain point I mean he might say it is because he thinks earth is done but right like there's these aspirational things that we want and need there aren't necessarily problems we just want them so the opportunity is just a little bit more inclusive and it's really getting clarity around what's gonna drive our desired outcome so

if my desired outcome is to increase engagement I want to know two things one what prevents people from engaging today this is the problem mindset what are the obstacles what are the barriers I have opportunities to remove them then the other side of it is the really positive sort of appreciative inquiry and question which is for my customers who are engaging today why are they doing it what problems am i solving for them we're in the jobs to be done language what job does my service or product do for them and if I can uncover

that I can reach out to everybody else in the world and say hey the job that you should use my product for is this and I can use what I learn here to go find more customers so what makes me sad about the fact that most people skip this step is I believe that the opportunity space is where product strategy happens the opportunities we choose to go after is what differentiates us in our market to companies in the exact same space are going to pick different opportunities so I really encourage teams to take time to

explore the opportunity space to assess which opportunities are most likely going to drive their desired outcome and to use their company's mission vision and strategy as a filter because Google's going to choose one opportunity apples gonna choose a different opportunity it's not because one opportunity is bigger than the other it's that those companies have completely different DNA's and so they're gonna filter opportunities differently this is exact this is where I think the heart of the product strategy lives and most of us are skipping it and then finally and once we discover opportunities we need to

make sure that we discover solutions that deliver on those opportunities and we don't have time to talk about today is the links between all the links on this visual it's the links that help us as a critical thinking aid we really want to ask this question in fact our experiments should help us ask one is the solution viable but then it should help us test the link does this solution deliver on the opportunity and again that sounds so obvious but we build all sorts of solutions that don't actually deliver on the opportunity we're targeting and

then finally we have to ask does the solution deliver on the opportunity in a way that drives our desired outcome because even if we solve the problem for the customer but it doesn't increase their engagement we didn't actually create value for our business so what this structure does is it helps teams remember what's our goal what's that outcome we're driving and as they do all these research activities they can track it this becomes their discovery roadmap this is how they communicate to the rest of their company I have no idea I'm gonna increase engagement that's

scary I know but here's the opportunities that I see these are the solutions that I'm exploring it will deliver on those opportunities and these are the sets of experiments that I'm running that will tell me if I'm gonna reach my desired outcome and what's great about this is it helps us put all of our tools into context so if I think about this okay ours are helping me set a desired outcome now my whole team knows what's our end goal what are we trying to accomplish together where it's job is to be done and even

design thinking design thinking is really good at helping us with opportunity finding because we're doing empathy interviews and we're doing observations and we're co-creating with our customers we're learning how to live in their world and that's helping us for example why are they engaging or what's keeping them from engaging so these tools they saw at ik about it and we argue about them but they're helping us do the exact same thing they're helping us discover the opportunities that will drive our desired outcome and then if we go down one level when we look at usability

testing and we look at the Lean Startup and we look at MVPs what is it what is this helping us do it's helping us test do our solutions deliver on our desired outcomes so here's the thing I know I went through this history pretty quickly it doesn't matter these specific tools are awesome if you're not familiar with them I recommend you learn about them but here's the more important message five years from now there's gonna be ten new books that I could have included here our world is changing really fast right now we're talking about

design sprints and jobs to be done five years from now we're gonna be talking about two completely new methods it can feel really overwhelming but here's the thing to remember our goal is not to write good user stories or to do good user story mapping our goal is to drive a desired outcome for our customers that creates value for our business and the the structure of this tree five years from now I don't think it's going to look any different we're still gonna be trying to drive desired outcomes we're still gonna have to discover opportunities

that deliver on that outcome and we're still gonna have to discover solutions that deliver on those opportunities so our tools might change but what we're trying to do I don't think it's going to change and the reason why I don't think that is that we have a hundred years of research on good decision making good problem-solving and good critical thinking and it's all consistent with this idea of start with an outcome explore the problem space and use the problems place as a way to expand the solution space and as you explore solutions it feeds back

into your understanding of the problem this is the heart of problem solving and decision making so this to me is the stable part of what we do and I find that it's real helpful for putting all of our tools into context so when you read that next article about whatever comes next all I would ask is that you take a step back and you say okay here's what I need to ask myself to know if I should use this tool and when I should use my tool use this tool first does it help me learn

something faster the answer is yes you want to adopt it second what does it help me learn faster does it help me set a desired outcome does it help me discover opportunities or does it help me discover solutions and that helps you understand what to use when now I want to end by looking towards the future and one thing that I love is a quote that I love by William Gibson a science fiction author who basically said the future is already here it's just unevenly distributed and I see that with the teams that I work

with so I want to talk about a concept that may sound really foreign to you but teams are already doing this it's happening today and I believe this is the future of product management and it's this idea of continuous product discovery so a lot of product discovery especially these recent trends grew up in this big project mindset let's go off an interview a dozen customers we're gonna synthesize what we learn we're gonna put it in a research report and we're going to give it to our product team and hope they use it to inform their

product decisions and what we find is that doesn't always happen because the people doing the work are too far away from the research and so what I teach teams to do I work with a product manager a user experience designer and a tech lead and I coach them on how to do continuous discovery so what that means is instead of doing big research they're talking to a customer every single week they're doing a prototype test every single week they're running a big experiment every single week and by big experiment it could be a landing page

test from the lean startup it could be a simulation of experience in person with the customer but here's the goal the goal is to do smaller research activities every single week by the team building the product so that at any point in time when they need to make a product decision they can stop and take stock and have multiple data points from multiple research activities that very week and say based on what I know right now what's the best decision I can make this to me is really important because in the product world we're running

experiments like 18th century scientists we run one experiment we decide its truth and we make a decision based on it the example that I use with my product teams is I asked them how many how many of you have read an article in the newspaper that tells you that coffee is good for you and everybody's read one of those articles then I say how many of you read an article that says coffee is bad for you and almost everybody has read one of those articles the reason why is because a journalist is finding a one

study that says coffee is good for you and then they're finding another and then a different journalist is finding another study that says coffee is bad for you and they're each writing their own articles but that's not how we learn that's not how research works what we do the way to know if coffee is good for you we have to look at the whole set of studies that have ever been done on coffee and do a meta-analysis and say based on everything that we know today what's the best decision we can make about drinking coffee

and here's the thing a year from now that answer might change because during that year more studies are going to be done and the data might start to look a little bit different the same is true with product decisions so we don't want to make a decision based on one ad test we don't want to make a decision based on one customer interview we want to make our product decisions based on sets of data sets of research activities so in order to do that continuously we need to make sure that we're continuously adding to our

bank of interviews and do our bank of experiments and so part of this is we have to develop good knowledge management techniques we have to be able to document all the experiments that we're running we have to be able to capture interviews snap shops and archive them so that when we remember I interviewed this person who had this exact problem I want to go back and find all the interviews where this came up we have a way of doing that and this may sound like it's impossible to do but I can tell you I work

with some really big companies in the United States in old industries like banking and insurance and they are doing this and they're doing it really well so I just want to leave all of you with we've made a ton of progress in the last 16 years and I really believe that five years from now most of us are gonna be talking about smaller sized research done week over week by the team building the product and our products are going to get significantly better so here's what I want to leave you with I am online I'm

very easy to find please if you have questions if you want to keep the conversation going I think about this literally all day every day and I would love to talk to you about it thank you very much you