How to Build a Product Roadmap by Walmart Senior Product Manager

um so yeah my name is Hudson I am currently a senior product manager at Walmart labs in San Bruno with sunny valve does anybody use the Walmart app or the Walmart com website or just just Amazon in here okay alright so I so I own the the pharmacy health and wellness experience and so so basically that's cross-platform so iOS Android and and responsive web so it allows people to to manage their refills do do repos to the store do competitive transfers medical expense summaries and things like that and so happy to answer any questions more

specifically about that afterwards but today I thought that it would be or tonight I thought it would be cool to kind of walk through my approach in thinking about like how to create a product roadmap what considerations I kind of pull in stakeholders to align with and things along those lines and so it's gonna be more more generic than just than just Walmart here so basically you know the first thing is like you know a road map is is kind of a compilation of all of the ideas that would go into future iterations of the

product right and so where do those ideas come from I think one of the biggest misconceptions that I've heard in the valley is that like the product manager is responsible for coming up with all the ideas and all of the solutions to all the problems and that if if there is you know a solution to be found within the product the product manager should kind of like work to work to find that in fact you know that's partly true but really you know the good ideas will come from all over the place they'll come from

your customers you know every every day I get into to work you know I I get a constant feed of customer feedback so within the app and within the website we have kind of a built in feedback mechanism for customers to write in tell us what they really love about the experience and what they don't love so much about the experience but most of the time it's what they don't love but oftentimes you'll hear some ideas like I'll be really cool if you guys did this so certainly the customer is a very very valuable source

of ideas and feedback for how the product is engine Aires also will come with with with their ideas particularly as you're going through you know product specs for a certain initiative the engineer may bring to light certain technical aspects that that you may not have immediately thought of and say hey would it be cool if we you know incorporated this piece of technology into this design so that's that's part of it as well marketing sales and marketing obviously there's no closer closer place to the customer than sales and marketing and so they'll certainly bring back

an office it depends on the business model but they'll bring back a lot of key customer insights that you as a product manager or as an engineer wouldn't have and even you know could be just Sally from Starbucks you know if you talk to anybody about what you do when you're out and about they will always have ideas for you what you should do with your product or where you should take your product so there's never a shortage of ideas I think you know the important job of a product manager is really not to come

up with the ideas but it's to say no and it's really kind of to understand which ideas are most valuable to pursue if that makes sense and so you know in a lot of ways the product manager is is is kind of the gatekeeper there's a lot of ideas a lot of noise that is going to be floating around and it's it's the product managers job to understand the core value of the product where they want to take the product what their vision is and how the different ideas that they hear or be they're being

tossed around aligned with that with that core value and aligned with with the vision of the product and it's not always obvious and it's it's it's it's definitely not always obvious particularly if you've got a short term win that could potentially acquire a lot of customers but might not align perfectly with a long term strategic direction so some balance is there any questions so far I know that we have some Q&A session at the end but I also want to encourage people to ask questions as we as we go along here because sometimes it just

flows better if you're asking questions about a certain subject that we're talking about so has anybody have any questions so far about anything don't be afraid at all we're gonna talk about we're gonna talk a little bit more about that for sure it's a great question how do you kind of how do you validate and and think about the ideas yeah yeah yeah yes good question also let's talk let's talk about that on the alignment area so let me yeah let me let me move forward a little bit here I'm sorry but there was another

hand too yeah how do you filter good question let's talk about that we'll talk about that in a couple slides so you know the product roadmap as I said is kind of a an outline of all of your different ideas that come in to come into play when you're talking about building a roadmap and so you know product roadmaps should do a few different things for starters it should it should reflect your product vision it should be kinda it should tell the story about where your product is right now and where you want to get

to how you're getting there second it should be a communication device I don't think there's many documents or tools that I refer to more often internally than the product roadmap and I'll talk to you more about why that is as we go through this presentation but it's such a powerful communication tool as you socialize the the vision of the product internally should be very simple no one reads a product roadmap that's difficult to understand I've seen product roadmaps that are super difficult to understand I don't read them and certainly as you work to socialize your

your product roadmap up to the to the stakeholders and leaders they don't have time to to learn about how you may be representing different initiatives either and so stay consistent with a simple practical manner and it should be the source of truth you know we'll talk more about this but once you have alignment on the priorities once you once you kind of come together and understand this is the goal this is the direction that we want to take the product that should be the source of truth and and and it actually would will prevent a

lot of unnecessary priority and alignment meetings through the year if you're able to solidify and come together around that from the start lastly delivery dates it's always kind of an interesting subject or not I mean it's basically kind of up to you I think it depends on how detailed you want your roadmap to be I've sinned and also I think it depends on the audience who you're kind of talking about the roadmap with so for instance developers oftentimes want to know roughly what delivery dates you have in mind for certain features whereas you know if

you're talking to to marketing maybe or somebody more outside of the development zone maybe the exact delivery dates aren't as important but certainly want to think about how you want to how you want to bring delivery dates in whether it be a week a month or a quarter I've seen all pretty effective for road maps so what inputs and considerations should you be should you kind of be thinking about here I think I think when when building a road map you kind of want to start with the problem and develop the solution around the problem

and I I do start with the problem there because I've seen starting with the solution is very dangerous for product managers I've seen situations where product managers get so anchored to a solution or other people get really anchored to a solution that they really kind of forget about what it is they're solving for and so I think as a product manager it's it's really important to understand what problem you're solving for and it doesn't have to have to be a problem with the product you know what I mean it's it's it's a product it's a

problem that the product is solving but the problem doesn't have to be within the product okay so so it could be a problem that your customers facing that you want to improve the product and this way to solve for important distinction there but but understanding kind of what it is that the product is solving and what benefits there are that are being derived from the product I think that in addition to to that every product will have like a set of goals or okrs objectives and key results that should also tie back to a higher

level mission of the company and and that's important as well because it you know if you're you know if your solution doesn't have you know clear goals that tie back to a higher level mission of the company then really it's that you might be distracted from from from what the core values of the product are and so you want to make sure that that they all map back correctly and and a product may have you know three or four or five multiple goals and objectives that that all tie back into into the mission of the

company take a look at a company like Facebook right whose mission is to connect the world make it more open got a lot of products within the Facebook platform that support that mission and each one of those products have a set of ok ours that map perfectly into that mission right and so that's kind of how you want to think about the the solutions that you're putting in and building into your roadmap for your particular product to solve a particular problem and an opportunity does that make sense any questions so far yeah yeah so so

for Walmart you know I'll talk about the pharmacy health and wellness experience you know Walmart's Walmart's mission as a company is to kind of you know provide an everyday low price so that consumers can just save more and live better and and in doing so I have a set of products that that I manage to help to help with that to help fulfill that mission statement right and so for me one of those products might be a medication item page so providing information that a customer can arrive at a Thailand or not a title of

any any medication rx item page and understand you know read some more details about the medication and understand what price that they'll be paying for if it's a four dollar prescription for instance or not and I'll and and that will be maybe the objective and goal there is to expose the price price transparency expose how we're delivering medication to the customer at a lower price than the Walgreens or CVS the problem that I'm solving for us customers are unsure what prices for prescriptions they would pay at Walmart the solution being the rx item page the

goals and objectives to increase price transparency and that feeds into Walmart's everyday low price mission and and thing that they're trying to solve okay so now the question is you kind of know what a road map does you know what considerations to kind of bring in when building a road map now how do I actually build it and there's a lot of a lot of people a lot of product managers out there that like I said they they kind of take it upon themselves to think of all of the solutions and to prioritize all of

the solutions and then socialize it and then get alignment across across the organization that that could work that hasn't really worked best for me what really works well for me is to to kick things off by having kind of collaborative road mapping sessions with your with your stakeholders and so you know understanding that you know you may have stakeholders from engineering stakeholders from sales and marketing stakeholders from the business strategy team all stakeholders in Walmart's case from the from the store side as well as the dot-com side so understanding you've got a lot of of

stakeholders here that you're map will impact let's bring them in when you're when you're kind of in the ideation and formulation stage of the roadmap so that it's it's kind of like a team effort and even even with your you know vision it's a lot easier to get buy-in early on as opposed to kind of having more of a dick toriel this is the road map let's get behind it after the fact and so get consists us understand like what the main goals and objectives are and make sure those are aligned because if the main

goals and objectives are not aligned you know you're gonna have trouble lining on the solutions and you know to the problems that feed that feed that so make sure that that's aligned on and as I said and get those stakeholders engaged early you want you do want to foster a healthy team debate if you're gonna have a debate this is the time to have it you don't want to have it mid development so have it have it up front and and trust me more often than not you're gonna be dealing with reasonable people and reasonable

people tend to you know be open minded about you know what did it what it is that that the other person is saying and why they why they might be saying that and so listen to them and and you know I think I think having a healthy team debate is all part of tossing out ideas tossing out solutions for problems next you're gonna have a lot of problems or I'm sorry you're gonna have a lot of ideas with this collaborative session start big and narrow down the ideas what you'll see when you get a lot

of ideas out there is that you'll start to see themes pop up for the ideas for instance the medication item pages right you know one of the themes one of the themes is you know kind of creating a knowledge portal of medication information so that you know customers see walmart.com as kind of a center of knowledge for healthcare an are excited page would fit into that theme perfectly right and you'll you'll start to see that you'll start to see these these solutions feed into broader themes really well and and those those broader themes honestly if

you can map those solutions back into those broader themes it really kind of gives gives you the answer as to why you're doing something again for internal socialization and priority challenges it really kind of helps you tell the story that you want to tell is the product manager yeah so you're you're referencing mostly like a business-to-business model yeah I think that I I'm hesitant to I'm again I think that there's always there's always exceptions but generally speaking when you're talking about building a product roadmap in prioritizing features I think it's always dangerous to bring customers

into that process I think that the most important job is to bring customer feedback into that process and aligning as a company internally on what is a priority you know what I mean taking into consideration that feedback you never want to feel like you're favoring certain customers and things like that and so I think it's dangerous to bring customers into too early before before achieving internal alignment yeah yeah yeah good question so the question the question is when you're having kind of an open collaborative session how do you keep the focus really and and and

you know I think when everybody may have different coming from different vantage points may have different ideas as to what the themes or problems are how do you kind of focus in on what the core themes should be and you know for me I think it's always important first of all as a product manager to have a point of view on us to where the product is going and what your vision is of the product and so more than anybody else you should really understand what the core value of the product is all right and

and so for me the product manager would would set the context for this meeting and say this is this is what I'm thinking right now the use of the main goals and objectives that we want to accomplish in 2018 these are the themes that would that would kind of help us get there and and these are the solutions kind of they're going to help complete complete those themes and deliver on those themes people aren't going to always agree out of the gate but I think at least you're kind of setting the framework for them to

either way in positively or negatively or add additional insight to it but I don't I don't want you to see collaborative remapping sessions as kind of a free for all and everybody tossing in ideas I think it's very important that the product manager sets sets the context where everybody comes in with a point of view and is open to feedback from the other people with this session but it's definitely a dance it's definitely a dance not a science any other questions that was a good question yeah yeah another another good question is how often do

you really want to have these sessions and for that I think it depends on the industry how how dynamic the industry is but for but for us I typically would have this session annually to set a to set an annual context uh and and and kind of like lay out an annual roadmap and so understanding that this is what we want to accomplish over the course of the year how do we go about doing that and then from there once you're able to kind of align at a high level you can then be able to

and you can begin to prioritize initiatives by core and break that break down those those deliveries and initiatives into into different quarter tour the reason we can talk we'll see more about that but you don't want to do this too often because this is almost like a reset you know what I mean I mean every time you bring two people to the table and say this is what we're thinking you know you you kind of want to stay you want to stay focused on on what you set out to do at the beginning of the

year and I think it's important not to do it too frequently but also not wait too long so it's it would depend on the team but I do it once a year okay great questions so far so all right so this is always a fun process so you know you have your list of ideas let's say you've got ten different ten different solutions tossed around that feed into two or three different themes how do you go then and prioritize those ideas and those and those deliverables features to the customer and so what works for me

is kind of just a benefit effort matrix where you can kind of layout and estimate what the benefit is against the effort and and I think what I forgot to mention on the last slide was kind of the last thing you want to do once you once you prioritize your your features you want to work with your developers and certainly your tech lead to understand you know what is the technical effort or technical complexity to deliver this this feature and tell you kind of and it doesn't need to be a lot of work I think

for me I at this at this point of the game you're often doing like a t-shirt sizing sort of approach so small medium large extra-large sort of effort you don't want the developers to spend a lot of time doing this effort because if they're sizing up effort they're not coding and so it's it's a balance there as well but um but at this stage when you've got say a list of I'll say ten ten different features you also want to have a rough idea after after partnering with your with your tech lead a rough idea

for what sort of effort would be required to deliver each of those ten features all right and then you can begin looking at this benefit effort matrix and understanding how those different features may plot against this matrix and so you may have one over there two down there not a lot of benefit but really high effort for engineers to deliver another third one even more effort with about the same value and and you can just kind of get an idea for let's say you've got seven seven initiatives now plotted so looking at this what would

be kind of a couple two of the the biggest opportunities here do you do you think right one in four looks pretty good right not a huge effort very high customer benefit and yeah that's exactly right I think and you won't you won't be able to plot this without first understanding what the customer value of the of the feature is and also understanding what the level of technical effort is to bring that feature to life so you need both of those things to really plot that yeah yeah great question great question so benefit question is

how do you how do you kind of look at benefit given that benefit could mean a lot of different things and and you're exactly right so there could be customer benefit or there could also be like business benefit not always the same thing for instance bringing on new customers is a huge benefit to the business but not a huge benefit necessarily to the current customer base if that makes sense and so understanding kind of like you know how your how your defining benefit and I think to be fair to the question benefit may be different

for different features if you're talking about a growth-oriented feature the benefit may be we could probably acquire a lot of customers by bringing this to market if you're talking about a core feature for instance in the pharmacy health and wellness a feature that allows parents to link their children to their walmart.com accounts to manage their children's pharmacy refills isn't necessarily going to generate a lot of new accounts but it could it will have benefit that customer a lot because now they're able to manage their entire family's prescriptions from their one account if that makes sense

yeah monitize well yeah and I think that that's that's that's part of what goes into the business value so you're looking at benefit in terms of customer value business value and and all of that feeds into and to where those features would be ranked on this axis I think you're exactly right and you're kind of balancing that with with the engineering effort yeah I think that's right on so what you'll see here is there's like a low effort a low effort zone there which is almost like a quick win and in an agile environment agile

methodology which which a lot of companies here are Walmart's a two-week sprint cadence this to me would would mean we could probably knock this out in one sprint and just get it to market yeah [Music] great question so the question is if you're doing sort of a high-level t-shirt sizing sort of level of effort exercise you know there are some things that are gonna be missed and to what extent do you want to have accountability or to what extent can you kind of a design for almost or plan for certain things being missed like that

and I mean you're right on I mean I think that I think that we're talking about there's this kind of when you're talking about level of effort to me there's kind of two stages one is the initials t-shirt sizing stage that happens very early on and then as you get closer to actually developing the product that's when also from a product manager standpoint you're drilling down into what the product specs will be and what the what the true product requirements are going to be with this with this deliverable certain sets of acceptance criteria might not

have been communicated before product requirements may not have been communicated in the level of detail that you will have right before you start you know building and so then there's that second second layer of assessment that the engineers would then come back with once they have had a chance to look at that level of detail that I think that's pretty standard and so hopefully it doesn't it doesn't you know kind of like straddle the medium and then get into the extra-large when that when that kind of second second pass happens I think that some sometimes

it could go from medium to large or from large to medium when you're talking about a lot of projects hopefully that balances out so that you're able to deliver about what you set out to do at the beginning of the year but it doesn't always happen that way and I think that it's it's it's part of the product managers job to be able to plan and adapt as those technical challenges become more complex than originally thought well yeah I think that the team's velocity is is super important that's going to play though more into that

the detailed estimations I think what happens more it's not necessarily not knowing a velocity what it is is the engineers diving in a little bit deeper and saying oh we actually didn't realize this would be required to deliver this we thought that we could maybe use a common component turns out we're gonna have to customize this and it's gonna take a little bit longer things like that will will add scope but I think that it's important to set those requirements before development happens certainly you don't want to scope creep to to happen mid development I

think it's important to happen before it starts and sometimes you will adjust your your delivery estimates based on that I think that's okay though yeah yeah so are you basically referencing sort of like back-end service layer work that would that would facilitate yeah yep I think you absolutely do I mean I basically anything that is going to require bandwidth from your developing developing team needs to be on the roadmap and it's not just for you but it's for visibility so that people know what your team is working on and they know what's coming down but

you're right there are a lot of things that the engineers will work on that don't that you know customers don't see but that doesn't need to be called out so as soon as you kind of like you know have this had this scatterplot you can then also understand where you might want to focus your focus your time on here and as you can see kind of you've got you've got not just a quick win section but you really kind of will see this clear area let's do it like these are these are initiatives that offer

a huge value to either the business or the customer and when plotted against the level of effort that the engineers came back with it's it's it's obvious let's go ahead and do it and at that point thing you're just just just talking about okay when do we want to do it in how soon yeah how do you assess like from the company level so you're saying the benefit touches one area of the company but doesn't extend up to maybe the higher mission of it yeah yeah yeah I think so that's a good question not not

not all of the benefits will transfer to other areas of the company certain certain benefits are going to be kind of more isolated and to the extent that key stakeholders are impacted so for instance let's say initiative for here doesn't have anything to do with bringing on new customers marketing is sitting here saying look I'm measured against how many new customers I can bring on board so marketing doesn't really want to do number four a lot they might want to do number seven which let's suppose that's like a supportive growth campaign or an onboarding funnel

it's a balance it's it's it's a discussion with those stakeholders that aren't necessarily going to be benefiting from different different features ultimately and it depends on your organization but ultimately this is a this is a conversation that needs to happen with product business strategy and aligned on from the start usually product is is the owner of the vision and so product should have a strong point of view and and but it's it's it's never want to feel like you're shutting anybody out for sure does that answer the question oh okay yeah I think if if

that's the question and you've got different stakeholders buying for different initiatives because they both bring unique value to that group then that's that's a debate I don't I mean you you have a conversation about how those initiatives map back to the okay ours so the objectives and key results that you want to deliver for that year and if if you know and and then also as I said before how how they also map back to the broader higher level mission of the company is also super important as well hopefully there are themes that have kind

of a priority distinction between the other themes and if for for maps back to a higher priority theme than seven I think that's kind of how you make those judgment calls but you can't always make everybody happy with every road map decision and so there are going to be times where it's just a difficult decision to make so yeah so I think a I think of farik I think a fair example that I can give you well let me wait until later in the presentation we talk about conflict we're talking about conflict right now so

after this we'll have a couple conflicts cell slides and those will be fun I think everyone liked those but so with this prioritization you'll also get some kind of area where it's like why bother this just isn't gonna this is not going to be worthwhile it's a lot of work for the dev team not a lot of benefit for the customer and then there's this area in the middle where you'll see a lot of the discussion actually happening and I think this is where more of the discussion between okay it's benefiting sales it's it's not

benefiting the core user or something like that and that's where some of the debate might happen particularly as you have something like six and five where both have high five has like more more benefit but also more effort you'll have a little bit more discussion around that so before we get into the conflicts which I saved for last we'll talk about different roadmap templates and these are the different styles that I've seen used in the past different levels of effectiveness but the first kind of roadmap theme that I've seen or template that I've seen as

kind of this laddered roadmap which is kind of interesting I don't I don't you know understand it entirely I mean it goes you know up low to high left to right which is good that's kind of what you want profits to do and so it might be good for your product room have to do that but other than that there's like a lot of white space on there I'm not exactly sure how I think that there are more effective roadmaps that I've seen for example this one I think I actually like this one a lot

I use this personally I think a quarterly release roadmap shares the right level of detail across the company and I think it's very easy to kind of like consume as a reader as well and so you'll you'll have all of your features listed out here and and kind of like just you know by quarter that it's delivered on and you'll also see down here the certainty and confidence will drop dramatically like as you go through the year as well and that needs to be understood within the organization to things that are slated for q4 are

not going to be as firm as things that are going to be delivered in q1 if you're building this roadmap in December of the of the year before right and so things later in the year do tend to shift around priorities do tend to shift around but within the next quarter or two it should be pretty firm and this is this is something else oh yeah and this is this is you know pretty easy to to comprehend any questions on on this template I'm sorry so this so you can you release this doesn't necessarily tie

back directly to product to incremental product releases and so for instance at Walmart will release every two weeks for the app right so within you know a given quarter we'll have several several of those releases right well six of those six of those releases within a given quarter but you don't necessarily need to tie into one of those you just tie it at a high level what quarter are you coming down with this also works very well for marketing as to because you know marketing they're interested in what's coming down the pipe they want to

know what product releases they would potentially be developing marketing campaigns for to get the customers excited about the features so they're gonna want to know kind of on a quarterly basis like how they're going to be allocating their time over the course of the year as well and as you get into that quarter then you can kind of get into specific dates and and strategies and tactics but at a high level this works really well on an annual road map yeah okay alright so that's an interesting question are you are you basically talking about products

that require a high level of effort with an unsure level of benefit and you want to test them before okay yeah yeah I think you I think you you would I think you whenever you're testing a product you need to get the product to a stage of development that is worthy of being tested right and so that in and of itself is going to require some work and you know typically you might call those like you know MVP test ready product releases in which case you know you'll test it you you'll always want to test

and get early feedback on how it's doing and you're right if it's just not doing as well then you don't want to throw you know good bandwidth after bad after a bad product and so you might kill it but you would want to absolutely call call that plan out on the roadmap yeah yep so yeah we can talk about that a little bit right now let me drink some water to get that one mm-hmm so the question is let's suppose you're halfway through the year you've laid out this roadmap and due to competitive forces let's

say Amazon does something leadership at Walmart wants to react and that reaction is going to impact the roadmap you know how do you how do you kind of plan for that or incorporate that right and it's definitely I think that it's hopefully doesn't happen too often I think you know to start with just has some context hopefully like I said leadership is aligned with the roadmap and the priorities and initiatives and solutions that are on the roadmap all right so they need to be aligned with that at the beginning of the year or at least

a few quarters in advance if something comes down from leadership that says Amazon's doing this we want to do this to react they should have a fairly good understanding for what consequences that are going to be with the current roadmap that's in place so there's there's always a trade-off there's always sacrifices anytime you want to do that I think that what it is at that point is it's a conversation of okay we can do that but this is what it means this this means family account linking is going to get pushed back a quarter this

means that your Rx item pages are going to get pushed out to quarters so it really kind of is an exercise in communicating the impacts making sure that the impacts are clear and hopefully you agree with it if you disagree with it that's another story but hopefully you kind of agree with it and can have an open dialogue with them about about it yeah yeah so there are different kind of like collaborative tools that companies could use we use we use like confluence there's also Zera which mat you know which which which kind of talks

to confluence as well with Atlassian but so JIRA and confluence are both you know shared tools that are shared across teams internally and and so what I do is I typically would have like an open access confluence page with this shared on it so that people can click the link and the other good thing about that is if you do update it obviously it would always be up up today yeah I think that's a that's a very high level of detail that that not might not be appropriate for all teams I think that and also

it depends on how dynamically you want your roadmap updated usually it's updated on a quarterly basis and so if you're if you're marking a JIRA epoch say has done it may or may not adjust the roadmap in real time like you know what I mean maybe maybe you'll write I think it the answer question I it's a good question the question yeah it's a good question I think it I think the answer is it depends on the level of detail and it also depends on the size of the initiative because there could be a lot

of epics in JIRA that feed a release and just because one's done it may not impact what you would show show here right because it's still in open development and so I think it depends on the level of detail that you want to communicate who you're talking to and the size of the size of the release something completely dynamically let's say for example walmart.com integrated to the e-commerce aspect the earnings were amazing the revenue growth was amazing quarter after quarter and then all of a sudden boom revenue growth wasn't as expected the growth decelerated and

the traction has been reduced significantly and you guys are still planning on great but at the same time do you continue with your roadmap or do you actually assess what went wrong maybe the roadmap you're taking is with me wrong because it's not delivering what is it supposed to be delivering yep because it's like a fake out in a way because it looked good and then it stopped yeah so what do you do at that point you reassess the room maggot and and the tricky part is tell the design solutions you can't you know complicate

it that mean you may have to go back and redo some of the architecture I mean how do you design it's an interesting hypothetical question so the question was you know let's suppose you build out a roadmap and with with certain with a certain level of expectations of a certain level of performance expectations that come with that roadmap and let's suppose you're you're going down the year and you hit a quarter that fails to deliver on those expectations to what degree do you adjust right I mean that's it's kind of the genesis and I think

that I think I I think you you want to have you want to have metrics in place that are that are good indicators for whether your roadmap deliverables thus far have been successful or not and if and if the indicators are telling you that they haven't been successful then you may want to rethink building on top of those of those features right but at the end of the day if if you're talking about a company-wide reporting event that failed to deliver you really have to take a close look at at what features are driving that

and what features aren't driving that and and so I think what what I'm trying to say is not all roadmap features are going to drive profitability for the company and if you're if your vision of a product manager extends out a year or two you you really want to think hard before reacting and scrapping that vision because of a quarter or two of that performance if that's the case if you're gonna if you're gonna react that quickly and be that sensitive you probably didn't have a super strong vision for what you wanted to do with

your product and so I guess my feedback to that question is you can't be too quick to react you want to have the metrics in place that give you indicators for what may be causing disappointing reporting and and dive in closer to that but but don't you know react too quickly and change your roadmap that has previously been aligned on previously been prioritized people thought this was a great idea two quarters ago so it's unlikely that now they don't so you really you really got to think hard about about reacting like that yeah no I

didn't I didn't I didn't fill this in don't don't okay sorry yeah don't read that yeah yeah absolutely absolutely you want to you will you will absolutely talk to your stakeholders at least quarterly probably weekly maybe daily and so your stakeholders should all be aware what's coming down if you're if you're not going to hit a release by a certain quarter and it's gonna move to the next quarter you want to be socializing honestly as a product manager you can not be too transparent you cannot share too much information to your stakeholders nothing should ever

come as a surprise to your stakeholders and so absolutely as roadmaps adjust and as roadmaps change your stakeholders should always be aware yeah ya know I think I think that's a good question the question is what if you roll out a feature and you're not seeing the benefits that you thought you would see and I and that speaks to kind of the question earlier where you really want to make sure that you have the proper analytics and and and metrics in place to gauge the success of a launch and if and if those analytics are

coming back and saying hey customers aren't using this feature or customers are dropping out of the funnel at this stage you really want to be able to either react and make it better or in sometimes if it's if it gets bad enough let's say you you reallocate bandwidth to a project you think might be more successful given this new information yeah yeah question question is you know how do you know when you got enough data to make a decision based on based on post release analytics that are coming in and you know it obviously depends

on the numbers if you're a smaller company with not a lot of customers it's going to take a while if you're a large company it could be very quick I think you still want to make sure you know and it also you know it also has to do with like comparables so if you're launching a product where the comparables are a little bit fuzzier um you may want to give it a little bit longer but if the comparables are clear then you may see you may know within a couple of weeks it's just yeah you

I think you also want to be careful though not to come to conclusions too quickly that's the that's the one warning yeah having the metrics defined great question so the question is I've been mentioning these metrics at what point do you bring in like an analytics team to to partner with and build out those metrics to make sure that they're there when the product launches for for us you know it's kind of it's definitely you know not the first thing you do it's it's it's done actually later in the development stage to where you know

you have the the builds basically ready and then the analytics team can go in and tag the proper places within the builds with with the with the tagging that will give you those analytics and metrics and feedback that being said you certainly have to engage an analytics team early on to let them know hey this is coming I'm gonna need some resources say in about four weeks or six weeks to do this work so if you could just like slot that for me I'll come back at you in a few weeks and give you the

build so that you can dive in and do the work the product manager usually should understand what exactly they're looking to extract from from the metrics so that would be the product managers one of the product managers key jobs understanding what metrics would indicate success or failure what they want to track and also the metrics isn't just like a success or failure sort of read it's also like where are the areas within the experience that I can make it bigger improvements on and so it's it'll also give you an indication for where you want to

focus more energy in the in the future as opposed to just killer or continue a release yeah yes sir so that wouldn't that wouldn't be on the on the product roadmap I so the question was if you're going to also be releasing a lot of like emails or flyers or media or whatever would that also be on this roadmap that particular thing went beyond my roadmap that would be on like a marketing roadmap or a sales and marketing roadmap but that sales and marketing effort would come from this roadmap right and so they would see

hey in q4 we've got snapshot coming that's a huge feature we're gonna want to put on our roadmap maybe in q3 building out a snapshot media campaign sort of thing and so the roadmaps would be obviously related but I wouldn't necessarily be exposing sales and marketing effort on my product roadmap well some small startup then I would probably have more I think so let's we have to we have to make sure like we're understanding like what value the roadmap brings to the company for me the roadmap has served as a really valuable social tool of

socializing what I'm doing with my product and when I'm bringing that to market when you're talking about success of a feature and that's when you're talking about metrics and analytics and understanding for instance users and yeah you're right if marketing runs a giant campaign on on May that's gonna impact my users when you're talking about April versus May and so when I'm measuring success and I measure measuring the the metrics I absolutely want to account for various peripheral things that could be influencing those numbers not just the product and that would be like a marketing

campaign but that doesn't mean that I'm putting my marketing campaign on my product roadmap yeah product pipeline and backlog - yeah yes so when I'm thinking about a backlog and again I'm just coming from the Walmart labs perspective here when I'm thinking of a backlog I'm thinking of more of a Jewish story backlog that may consist of stories that feed these items or bugs that are in production right now that I need to tackle but that level of detail wouldn't wouldn't make its way to a road map like this I think it's a good question

yeah yeah yeah my mother knew yeah exactly right exactly right I mean this this is much firmer than like a JIRA backlog would be or some other thing and and don't get me wrong I think that there's there it's almost valuable to kind of have a running collection of ideas and things that that you kind of collect as you go through the year doesn't necessarily mean you're gonna throw them on the roadmap I think that there's a lot of consequence in in having a roadmap that's too agile if that makes sense okay great questions I

think that there is one thing that is is an issue with this type of roadmap can anybody think of like what that what that might be I mean this roadmap is great for like a quarterly release sort of thing can anybody think of something that they might also be interested in seeing that's not that's not obvious yeah yeah I think it was that yeah exactly I think dependencies who's doing what and that is where kind of this more Gantt style roadmap I think lends a lot of value with this you can really see um you

know and again don't don't read this but you can you can kind of you can kind of understand more the the effort by the teams that are involved and so if you've got a back-end development team that is going to be working on this for three months that's highlighted in this roadmap and totally beneath the surface on the other roadmap if you've got native developers that are required to work on a certain initiative from February to July you know that the native that the app developers are going to be tied up during that time and

you can't assign that bandwidth to other projects that require app work during that time doesn't mean you have to pause everything on the web but it allows you to kind of understand how your resources are being allocated and and to what extent they're tied up that's that's the the value that this that this Gantt style roadmap brings I think with that adds a level of kind of like complexity though that may be too much detail for some people for instance marketing probably doesn't need to know you know that it's gonna take the engineers four months

to build they just want to know when it's coming when the customers are going to see it so something - something to think about there any questions on this sure there are yeah Japan it depends I think that um you know if if you're understanding if this moves out then obviously it could impact the other roadmap to where you know if something moves out on this then you know maybe the api's are now going to move to q3 and so you would want to definitely have them consistent for sure and to your to your question

about the JIRA epics this is where I've seen JIRA epics Drive again again style roadmap to where you have kind of the epics kind of listed out the start and done dates of the various JIRA epics listed out and that would kind of give you kind of that Gantt bar to say okay this epochs supposed to be done here that's that's that's more where I've seen that as opposed to the quarterly release okay yeah yeah quality so qualitative research yeah so definitely a lot of you know qualitative in and quantitative research really that go into

to making you know roadmap decisions I think that you know for for us there's a lot of qualitative decisions that will go into specifically UX aspects of a roadmap if you're doing a site redesign or an experience redesign being able to come up with a with a couple of different designs and maybe put those through a qualitative a/b or qualitative focus group sort of studying where you're getting voice of customer feedback about the different designs making sure customers are kind of traversing down the journey that they that you're expecting them to and and things like

that and so a lot of times that would that would kind of be a study that would be conducted ahead of release but not necessarily ahead of development if that makes sense yeah how do you bounce what tech debt sounds like a conflict well yeah we'll get into that yeah and each of our team has their own engineering team yeah and say you were the PM yeah and for some reason say you are yeah yeah yeah right I think that's a that's that's a common issue is and to repeat the question you know every product

manager owns a road map and is and is supposed to be accountable for delivering you know on the road map what happens if do to turn over a couple of Engineers leave on another team and to balance out engineers get pulled off of your team and to support the other team right or just yeah we'll just leave so I think that you know what you're I think that that's just a risk you need to call out a lot to be honest with you it happens everybody knows that it does happen there's no magic bullet there

you know there are strategies to to kind of mitigate that risk but you know consultants things like that but there's there's definitely no no way to kind of avoid yeah the options are to either burst up bandwidth by bringing on consultants the the options would also be to to bring over engineers that are like kind of under serving or not exactly needed there also are periods on other people's roadmaps where for instance let's suppose a dedicated pool of Engineers full-stack engineers on somebody else's roadmap they might be doing very back-end heavy work at that point

and their front-end engineers are more flexible maybe bringing in some front-end engineering talents we'll work on what you need so based on what the engineers are working on currently where they're needed hopefully the engineering team is large enough to be able to account that's the third option I know right I yeah I mean something's gotta give right it's either the timeline of the project or the scope of the work or the bandwidth that's working on one of those you know it's one of those things have has to give and and it's just a discussion with

the engineering team to say hey can you do this or it's or if not it's a discussion with business team say hey we're not gonna meet this timeline or it's a compromise as a product owner to say all right you know what I really wanted this feature in this release I guess I'll take it out to de-risk the project and get it out on time with the limited bandwidth that I have no idea option there if there's a real conflict between two product efforts you do you absolutely do you absolutely do and particularly large companies

will have a list of priorities that they've all aligned on the leadership word has aligned on to say hey our number one priority as a company is is this our second priority is this and you may have a list of 20 different priorities right and so where does your need or gap stack rank against those priorities is certainly certainly falls in but at the same time it's it's tough to pull people off of dedicated engineering teams and so it's it's definitely a discussion but you're right I mean all initiatives at a company are right it

doesn't necessarily mean that every that the people at the top have have a clear path to though yeah yeah so so that's part of agile methodology is that the end of every sprint will have a retrospective where the product team the engineering team the design team will come together and talk about what worked really well during that sprint and what didn't work really well or what could we improve on for the next sprint so that that's part of like a continuous feedback loop on to help the team with their with the velocity there there isn't

kind of a more well I mean there is but I don't kind of Oh bye-bye a more formal longer term thing with the team at the I mean at the end of projects we'll have you know release release parties and things like that where we'll informally be like oh it would have been great if we could have done this or that but there's there's there's nothing that's formally captured that that could be an opportunity to be honest with you to say at the end of every big release would you want to document like I'll have

a release retrospective I think it's done more on a casual casual level right now for me personally okay yeah how do you decide what what metrics do you use for privatization I mean you could use you know an investment delivery business requirement yeah yeah I mean that's basically the leadership at the company would would align on those priorities based on kind of the strategic direction that they want the company to be in at the same time leadership mean I know the initiatives but maybe sometimes have been undertaken that might be you to that result that's

exactly right the the question is how are those when you've got when you got like say 15 or 20 huge priorities at a company that are ranked how does that ranking kind of occur definitely product managers senior product managers directors are involved in in providing data to support their area but at the end of the day usually that priority and ranking happens above that above that pay grade all right complex oh I think we're out of time No so conflicts will emerge and I think that you know you're gonna have a lot of conflicts one

of the one of the set of conflicts that you'll see is kind of balancing short term versus long term you know short term product iterations versus more long term more disruptive sort of opportunities and that's always fun so you know for instance from my world there could be a short term opportunity to add a pickup scheduler component to the app refill experience but if we do that we're taking away potentially for something that's more disruptive like an Rx item page that's a lot heavier lift for the developing team but probably strategically a more powerful delivery

when you're talking about becoming a center of health knowledge destination for customers and so it's always a balance and being able to as a product manager apply some judgment to understand we'll still be able to disrupt we'll still be able to make an impact and be where I want to take the product in a year or two and also do these these shorter term product iterations at the same time because if you're you can't just pause the product iterations and you can't just pause long term it's definitely a balance it's a delicate dance the other

the other issue that was mentioned tech debt like bugs versus specific features that are listed on a roadmap and this is tricky because you know bugs don't stop happening you know bugs old bugs will continuously find their way into a backlog and and you will you can't ignore them and so you know what I do as a product manager when the bugs come in I'm able to prioritize them on a scale of one to four so four different increments of priority where I know if this is a priority-one it needs to you know it needs

to be tackled immediately we need to drop everything we're doing and tackle it if it's a priority too it's just something that can't can't go on and can't persist for more than a few days if it's a priority three our priority four there's more flexibility there just to give some context I won't release a product if there's any priority one or priority two bugs in the build priority three and priority for bugs I'm okay with priority four bugs are usually like you X related bugs or something like that but yeah p1 or p2 s always

need to be always need to be done before we release a new product the problem is after you release a product sometimes those bugs find their way back into production and then you're talking about potentially pulling resources from a firm of q3 feature to work on a q1 release production bug and that's where things get a little bit murky and and could potentially delay or push back a roadmap but it's it's just something I think as a product manager you're cognizant of you want to try to build some buffer into the timeline to to account

for that and you know and in the buffer would very I mean usually a tech debt buffer could be in here from ten to thirty percent and that'll vary as you go through a release cycle and then last you you know we kind of have been circling around it but you've got you know core product features that you want to build but then you may have you know marketing that comes comes and says hey I really want you to build like a landing page to support a campaign that I'm doing or or you know optimize

part of this conversion funnel that's that's not performing very well and particularly of smaller companies where you're sharing on engineering pool across you know marketing development and product development sometimes that could be a conflict that needs to be aligned on between product and marketing and sales I'm sure there's a lot of other ones that you guys want to toss out right now right yeah earlier a CEOs a leadership so yeah I'm brings in and wants something how do you accommodate their interest as well yeah I think um we we briefly touched on it earlier I

think if a CEO or or VP comes in and they have a strong opinion that you know we should add this to the roadmap you know I think it's important to to kind of set the context that hey this isn't a this is a road map that has been aligned on and agreed on by leadership across the company across all stakeholders marketing business product engineering UX everybody is aligned on this if we want to drop this future in and wedge it into the roadmap we can do that but this is the consequence this is the

sacrifice that we're going to be making to the customer by not delivering these features now they're gonna get pushed back and so I think it's always tricky when when talking about you know the CEO or VPS of companies and accommodating their requests because they're usually requesting it for a very good reason it's it's it's important that everybody involved understand the consequences of making kind of quick roadmap adjustments on-the-fly like that yep yeah so when you say standardized levers published what what give me a couple of examples okay got it okay got it yeah [Music] yeah

I think I think what that touches on also is going back to the beginning of the presentation where you're in that collaboration stage of forming a roadmap and people are tossing around ideas and initiatives and features that could be released you'll start to see these themes emerge and you know for instance some of the one that one of the themes might be grow customer growth okay you know acquire double our customers another theme might be enhanced the customers customer experience not necessarily related different themes right when these themes emerge you need to as a product

manager understand what the priority of those themes are and how they how they weigh against each other for sure and I think that having a clear point of view on that and where those themes prioritize against each other is does exactly what what you're what you're kind of alluding to later later in execution stage is conflicts arise you're able to lean on on that previously aligned on priority weight to say hey we have agreed that acquiring new customers is our top priority this year under which will also enhance the customer or will enhance the conversion

you know or refill experience or whatnot I think you know a second layer there is though even though you had these priorities ranked how much are the priorities being impacted by this initiative and so if you're balancing one initiative that barely moves the needle on your top priority versus another initiative that makes a huge impact on your second or third priority how does that decision made I think is is is oftentimes kind of an additional layer of complication yeah what advice would you give to an aspiring party manager one in terms of navigating how to

tell apart entities that landscape is changing yeah at least if you see that right MBA or weather products board you know the above and then you know what do you look for when you're looking for tangible qualities characteristics of backgrounds as well yeah sure well sure I mean I think I think this school thing is awesome I you know I looked into it a little bit when when they reached out to me and it seems like they're just doing an amazing providing an amazing service here hopefully you're learning a little bit about read maps this

evening but I you know what I personally look for hasn't changed a whole lot over the last few years I think that you know you typically it's it's important to understand you know what sort of product you're looking to get into usually it's usually you have a passion for the product a passion for what the product is doing and how it's like improving Society or bettering whatever I'd only be too old for a suit but like you know usually there's a passion there's a passionate area for you to serve and you want to get into

that area because at the end of the day the product manager you know the product is your life basically and so that's why it's important to care about the product that you're managing other than that like there's obviously going to be a lot of product passion I think it's very important to also have kind of a balance of appreciation for technology you don't necessarily have to dive into code and anything I don't I don't code myself but you have to have a kind of a baseline of understanding for what technical dependencies may be required to

deliver certain initiatives and that's not something that you're going to have right away but something that you're open to learning over time to be able to say okay I kind of roughly could t-shirt size this myself if I have to knowing what I know about what we've delivered in the past and and then I think in addition to that it's it's just having a passion for the customer or the user of your product and being able to really understand you know what value the person is getting out of your product which helps you immensely prioritize

when you're talking about 20 different features that you can bring to market I think those are kind of the three things that that stand out as a super-important all right so in summary the roadmap is your vision it's where you're going and why you're going there it's the path forward so it's when you nail down your roadmap you can sleep better you know what I mean because alignment is set you know that priority is clear and all the sacrifices to your point all the sacrifices are understood once you get into execution phase if somebody comes

at you with an idea the sacrifices should be understood if everybody's aligned with that roadmap and the priorities on that roadmap and lastly it's important don't over promise things with your roadmap don't speculate if things are unclear or if there's a level of effort that's unclear don't be you know just don't speculate that's all I have to say you want to keep it as realistic as possible and always remember to build in a little bit of room for air and buffer for for hiccups along the way if you can but that is what I know

about Road mapping thank you very much [Applause]