Product Team Meeting - 2019-07-09

it's like my family I am definitely I definitely have no subjects hey Daniel hair hello welcome welcome back hurt thanks so much yeah it's great to be here hi Virginia right let's let's kick off I wanted to start out with some reminders first we have a book club coming up on inspired in four weeks on August 7th I just reread it myself it's a good read it's highly aligned with how I think about product management and does a good job of explaining why some of these things are important but that I also believe to be important so it's nice to have another voice explaining all of that so please do read that I think I'm gonna update the new hire onboarding doc and ask all new hires to read this as well so that we all everybody in the team is on the same page with respect to this book let's see reminder be remember there's this interview spreadsheet see essence sales have populated that with the number of customer contacts for meetings please do follow up on that I want to ensure goodwill with that team and follow up promptly with meetings with these customers so that that team can see that we're taking advantage of it third reminder we've got a little engagement survey I'm gonna run this once a month in q3 just to take a pulse given all the change going on please do take a minute to fill it out it's five you know quick questions and then one free forum where you can share whatever feedback you have Fabia didn't receive it I'm pretty sure I went through my emails some maybe it's on my end but I'm happy to fill it out but I you didn't get it need to get it to you alright I will I'll ask Jessica Therese that to you anybody else in the same condition when you did not receive it I don't recall but is there way to put the link to the survey in the agenda well it's it's personal it's uh it's tied back to your user ID so we can track which team are on and that kind of thing I do believe it's anonymous but nevertheless everyone has their own custom ID so I'll ask Jessica descended to fabian and Carina anybody else oh I just I hadn't seen it Scott but I searched my email real quick and it looks like that's the title of the email so if you just search for that in your in your Gmail you should be able to find it if you got it pulse survey culture area okay anybody else didn't get it please ping me all right next reminder we we need we have a goal of at least three customer interviews per p. m. there's an okay our issue out there if you have it updated it lately please do so and remember we have three weeks until q2 to hit our goal so please please do invest the time to get those set up and get at least three done if you haven't already next one category maturity page last week we talked about this Josh did a great job of creating some new views one of which is sort of this flow chart showing how mature we're gonna be at a given point in time which raised questions about whether we were forecasting that accurately if you haven't already please go in and either confirm that it's accurate or update it thanks to Kenny for creating that issue somebody had a direction maturity page won't talk about that well I would just mean you're up instead size adding a link there ads all over folks I've seen the updates there for the charts I just check it out so you've got it's a good way you can get a sense for like you know it's alright it's hard in tabular form but when it's charted it's much easier to see like if it's achievable or not based on some of the trends and there's also if you scroll down these little trends as well so you can see how your stage in particular is trending or set to be trending so great thank suggestion all right some team updates we hired a couple more p.

m. so we got a good rhythm going on hiring we hired Gabe Weaver he originally came through that growth funnel but we have a really strong candidate for that fourth slot so we're gonna target Gabe for our third manage PM the Charter that team is to be defined but bottom line we're gonna have a third group in the managed area and Gabe will lead that and then Dov Hershkovitz we just hired him as the API monitoring he's got a great background in monitoring and has most recently been at elastic so thank you to everyone who's been involved in the hiring loop I know it's taking a lot of energy from everybody but I think our hiring processes continues to pick up speed b2b worked with with Christy and David Sakamoto to change in the language around customer results just wanted to make sure you all saw that so there's the mr it's got on that one I just the there's a diff highlights what is new content I believe and there's one section that is great I can totally understand why I would add that about prioritized ruthlessly but then the rest is I guess a bunch of formatting changes and I don't know if there's new content in any of the dogfooding I guess the TLDR the addition of that prioritize ruthlessly or is there some other point we were trying to make in this change so it's been a little while I think there were a number of changes but before the handbook basically read that internal feedback is worth ten times more than external feedback and I understand why we want intern feedback because of dogfooding and using our own product it's it's a it's a great channel for feedback but I think it was sending the message that customers weren't nearly as important as internal opinion and both Christy and I want to move off of that position like we should be customer first and treat our own teams as a customer but let's I don't I don't want people to interpret that our own internal opinion is worth 10 times more than a customer's opinion if that makes sense so it was mostly language wherever that showed up in the handbook catch hey okay one come with a head and this is that some of the text seems like we should focus on core competencies as opposed to a new scope and like as in we should focus what we're best at so and I might anyway that's one thought I had on this I don't remember that being the point of it maybe it reads that way I don't know feel free to continue to suggest tweaks the point was let's prioritize and do what matters most first just it's kind of what I've been preaching the whole time like let's in your area wherever that is do what matters first don't try to do it all at once we're gonna have to work our way through that was the point yeah you know I don't know if this is a follow up issue in the way you described it it doesn't seem controversial but I will say there was a big discussion and the recent initiative you know from Sid and other leaders that we should heavily prioritize dogfooding because there were parts there are teams within the company that were not utilizing our features and we wanted to make sure that the product team was responsive to requests from them yeah it's a little bit different than saying it's about our internal opinion like that we should we had always said we should validate it so that clarification is good that it we want to make sure it's about us saying this is in line with where we want to take the product and where we're hearing customers but if internal customer once said we should we should original thinking was that we should emphasize it I just wanna like if the intent was to make sure we weird clarifying that same position but if we're saying actually we should kind of pull back from the push for more dog fitting no I would please don't conflate the two okay we very much still want a dog food I think the point is when you're thinking of customers for your thing think of our internal teams early like you can get great feedback from them they have an incentive to work with you there's very little risk in rolling out things early to them so treat them like a customer and think of our internal teams early as you're rolling something out that's still very much the message but let's not over rotate on an internal feedback or internal opinion but still seek external feedback - because that's just one customer of many cool ok does alright to see customer training discovery training coming soon Sarah O'Donnell and her team we're gonna do a bunch of sort of quick videos on a variety of customer discovery topics it was super excited for that they should start dropping any day now I think starting this week and so will will release those to you as they come out will embed them in the how we work description on our team page as well alright number 312 to kick off feedback Josh thanks for leading the charge I thought you did a good job of emceeing and sort of adding color commentary in between I thought the screenshots definitely helped there were a bunch that did not have them I was wondering why is it just because we're not there yet on many of these yeah ok yeah I mean some the commentary I don't know or Nicole added that yeah many of the issues we're saying we're gonna do UX front end and back end in the same iteration so it hasn't started and in some they're like I can think of a number where there just aren't appropriate screenshots or at least there weren't screenshots or mock-ups created in advance okay for the purposes of front-end working on it cuz front end was going to work on it without Amaka okay I'd love to get to where we're a bit ahead so that we'll have more of these earlier and hopefully the customer discovery flow will get us further ahead on that in my case some of the features also just have no you X component yeah there's no UI component that could be spin shorter that's understood yeah I don't expect to everyone I mean use your judgment if it doesn't need it fine but where we do need design it'd be great to get at least a month ahead so them as we roll into dev we have that to offer them Scott just a quick question to you how do you feel about presenting like balsamic or super lo-fi mock-ups on the kickoff call fine with that okay cuz that could be an option to for PM's that are waiting for UX work in the same sprint and I know that plans done a pretty good job at at least in the past of kind of running ahead of UX and saying okay this is kind of what I think I want this to look like before spinning UX cycles on I'm making a more hi-fi mock-up so just if you think it does a better job of describing it then the issue itself then use it I think I think in some cases like a picture can be worth a thousand words I mean no matter how many words you throw at something it's like you know for example one of my things that I request or I reported on for the release the kickoff meeting was expanding the epic view in the roadmap and like those are basically just a bunch of buzzwords put together that you're like okay what does that mean expand epic and I'm just I literally thought on that one for like 20 minutes saying how do I make this epic how do I make this issue title like more descriptive for customer value and just came down to like that is the functionality we're adding what does that mean oh here's the screenshot you can see that we're gonna add a drop down you can see the issues and children epics that are attached that epic and in that case like I was like I'm so thankful I have a screenshot even though that one is actually not a hi-fi a mock-up it's more it's more lo-fi there's a little bit pieced together so yeah I think like in general there's a lot more value if we can show something like that so you know product managers you can you can consider that you should feel free that you know you're empowered to take a tool that you're comfortable with even if it might even be just like Google slides and make something that gets you at least a part of the way there in terms of what you want to experience it look like yep perfect 3c I thought the top track shifted I was definitely more a problem focused I noticed a number of speakers really trying to zero in on that which is perfect some of them could have been more problem focused I thought so just keep keep considering that as you as you yeah it's important to be able to pitch these things in ways that people that aren't close to it can understand and so just think about that how do I explain this as someone who's cold who doesn't know a darn thing about this why should they care that getting that crystal in your and you're thinking is going to be important no matter what so it's time well spent hey this is Kareem I just said add to that if you don't mind um I think this has always been a challenge in product even before I joined get web for many people as how to how to get there on some of this terminology when those of us have deep technical background um so my thought would be is there a way that you can start sharing you know or applauding good examples of this so that the product team can start to kind of ruminate on this and develop that skill if we're not there yet yeah I thought luca's were very well framed up those those two popped out at me as yeah that's the problem we're trying to solve check those out I'll look through for some other examples thank you for this suggestion all right three we went long we just have a ton of speakers which I love that ever lots of people get a chance to speak so I'm I'm good with that but we're gonna have to we're gonna have to limit the number of items probably so it looks like there's some other ideas in here perhaps themes yeah I mean if there are some that relate to each other you could tell a story hey we're trying to prove this and then a a B and C tie back to it I think it's okay to be pretty brief in your description as long as you're hitting what it is and if somebody's really interested they can dive deep team attic is a good idea recorded video if you really want to go deep maybe it's technically complex that's a great idea and then you can just cover the customer value at a high level and leave the detail to the video watch statistics I think Josh looked this up last time he I think he said there were a thousand geo there we go Kenny's putting him in so somewhere between 500,000 and check add to the time just a feedback I was timing myself this time and I had two features listed and I hit three minutes and 14 seconds obviously has shortened that so when we talk about you know I think somebody mentioned doing 2 or coupling it down it's interesting that I landed there with the two that I chose yep that feels about average but we've had how many speakers will probably have to be a couple minutes max per person I mean Eric pointed this out and the next line I do think we are due for a rethink of Howard yeah doing the kick off because we're gonna have next month we're gonna have 25 people trying to give content and yeah you've been at 2 minutes already gone so yeah we expand it I I will give a shout out for Jason I know because he's on paternity leave created a video but I think the original intent of the kickoff was actually just as a company we had a retrospective and a kickoff retrospective immediately followed by a kickoff and we just decided to post that on YouTube we now post a whole bunch of content on YouTube so just just having what you would normally do for your kind of like grooming or kick-off within your individual group posted to YouTube and that's maybe having a specific channel for people who wanted to follow it and we should discuss it in an issue and come up with something I do you think prior to next release kickoff just to evaluate alternatives to the format yeah I mean I don't even if we said every person has one minute I feel like we're doing a disservice because we're now highlighting much less because we feel like we have a time constraint and need to keep it into one synchronous 30 minute block huh need to do that okay yeah plus one de la revamping it I think I think we're trying to it's like got so many jobs right now that we're not doing a good job at I mean we're one of them by that I I think that feels the most important customers are internal and it's just like getting internally because like people have send that thing then we had like 50 people on the zoom call alone not even considering YouTube people were asking about whether it will happen to youtube link and things like that so it's it's well attended internally I think they're just for alignment alone you know marketing value of like sort of like a release for correctional customers it kind of feels like that better off having that go webinar or livestream on the release day right yeah maybe the externally focused one would be more about what we just shipped there was a webinar that used to happen I called release writer and I think I participated in a couple of those like three or three of them back to back and they were pretty poorly attended from what my experience was in I think I actually got ended by the product marketing team for that reason I'm sure someone from that team could actually give feedback but I think one thing about the time limit is it's really hard to motivate problems particularly like in a short amount of time particularly when they're very technical like has product categories growing maturity and sophistication like the problems become more and more specific that we're solving and so motivating those specific reasons and why we're going after like this specific tiny piece of a very mature category it's hard to do with 30 seconds in a way then and so because we want agree do that better that's going to put more pressure on like communicating a reasonable number of items I think okay thank you all for the feedback I like the idea of creating an issue and perhaps tweaking the format before next month I also like the idea of asking internal and external constituents what they like or don't like about the format yeah just one final thought on that like I love that it's a half an hour might almost even like take pic to particular categories over over over linking the time as an example just because I feel the feeling I have a feeling that if you want to watch it consistently it's gonna be in that block but that's just me so like if you know other customers or you know saying simply would like the larger block then then that's the right way to go so that's that's where I'd love to get feedback in some fashion they'll get say okay you know here's how we should change it but we clearly have gone breadth wise we've gone so much harder that it's gonna be hard to cover all those topics and quick them down okay thanks Kenny for starting the issue James over to you for number four yeah I just thought I'd share this but many I think many on this call haven't heard knock comes back speaking about product discovery spirits that he advocated for this quite a number of times previously from his experience wrapping these does a prior company so the idea is kind of different to it I guess a UX discovery sprint I think Fabien linked one of the books about that where it's really focused on UX iteration and research the product discovery sprint is more focused on kind of like actually building something iterating on something that's built and trying to get to some sort of MVC really quickly by trying to make the process more synchronous so the source code group is going to try and do that around file-by-file diff navigation to solve performance and usability problems in 12. 3 and I thought it'd be interesting to share that because internally we've been wrestling with like how to make this work well in async such remote environment so we're looking at trying to confine the participants in a specific time zone so that we can all be available with a significant amount of overlap but that's also difficult because we have it kind of excludes automatically 50% of the team who are just geographically remote from any of their peers we only have one UX designer that's only available in the European time zone and so some interesting challenges there if it goes well we're gonna try and replicate it - later on a different problem that is also really complicating hard I'm gonna make progress on quickly but I'll share any findings we have and if anyone's interested in discussing that with me more and put a meeting my calendar or drop me a message this is great James by the way I think the UX team is gonna run well let me just say we had the option to run one with Google Ventures who's one of our investors in that sprint book that Fabien link to was written by a guy from GV they did hundreds of these things for their clients they know what they're doing so if we get a chance to do one with them we should we're gonna have to figure out how to do it within our basic model though so whatever you learn from yours James please feed that back super interesting topic I think if we could get good at this a synchronously that would be a breakthrough yeah I think one other interesting challenge is that the Sprint sort of terminology is kind of challenging and like it's not sustainable to be doing design sprints or discovery sprints on a daily basis whether or not we were in person or not it's not scalable to actually sprint all the time so choosing the right tasks cuz choosing the right time is I think one of the other challenges I agree yeah you don't want to do this for everything because well if you follow the to the letter it takes a whole week and you're totally dedicated to it which is amazing for focus sake but you can't get anything else done so depending on how we structure this it would need to be done for things that are really big unknowns we're dedicating a big chunk of time like that is worth it and not everything clears that bar I think it's also most relevant for four stages that they're very kind of that Tyron was like one of their biggest example for a little ventures when they obviously like solving clinical trials for the world is like super complex problem so they just figure out what diseases thing that we can do so that we can start getting there and I think these are problems that teachers we used it pretty successfully at my last company around pricing and packaging stuff and ran a bunch of interviews with customers on that so I've seen it work all right okay Christopher number five yeah just want to call out we over the past month we pair that is significant number of outages really calm and that effect that at least one customer revenue potential and because of that you know we've had some some focus from an exec leadership perspective so I encourage every to look at that document you kind of look through and particularly there's couple things from an engineering perspective make you aware of one is is we started an infrastructure to development board where we're gonna start matching issues up and trying to make sure that if those get prioritized highly where appropriate particularly for anything that you know affects performance around these issues the other issue that I put in there was one around that's listed specifically which is around the fact of prioritizing P performance availability work so one of the significant features of this particular recent outage last week was is that the register server apparently can't handle the load anymore we started digging until we found a bunch of stuff that we hadn't checked like for instance it's an example RJ unit tests were basically going and getting cashed and there was no limit on the number of unit tests that could actually be cached so they're getting these like blocks of like several megabytes of data that had to basically be transferred around and read us and that's really what's affecting its performance overall from a caching service perspective so consequently it's kinda I sent it to you I hope that's okay yeah because it feels like it was like you need to help out in regards to the fact of you know how do we best make sure that we get is this kind of systematically going and I just want to make sure that everybody was aware and just kind of for discussion if there were any questions or or any feedback or early feedback on it from that restrictive I added some comments to it Christopher okay I haven't had a chance to look I apologize about that from can I ask - we and maybe Mac this is a question for you do we categorize performance issues as bugs they should be under under bugs okay okay yes this is an example where oftentimes the way we treat performance is a reactionary this is try anything about a more in a proactive way so like as an example I'll give a horrible example but my work that amazon tags originally when Amazon's created tags were were they were expect them just to label you know certain instances and that was it and it turns out that all customers started using like 20 and 30 or 50 tags and they're like what the heck's going on and they realize tags were being used to basically assure environmental informations the VMS could they could put the same drop a code on two different VMs and they could behave differently based on the tag which was a total novel way for customers to use it so then they had that basically limit the number of tags they could use because it wasn't scaling with the system effectively so like this is kind of another example where like I think we got to start thinking in terms of you know like when we create something new a new feature of piece of functionality like what's the cost associated with that right because like it does cost some something internal and I'm not asking product managers to necessarily think in terms of the exact bytes but I am concern to think in terms of like you know what are the expectations around because like as an example if you went back and looked at a unit test and reporting you know if we said unlimited that's you know that's a tough engineering call right clearly guess it's free right now for customer service is my understanding we also don't have a number of repos mirror we don't have a limit on that and that seems dangerous yeah so I guess I would comment you know I think the product team is expected to prioritize all things and to understand them deeply whether they're a security issue or a performance concern I think what you're highlighting is in order to be proactive I don't know if the product team would immediately know the impact of a proposed change but maybe that's an opportunity for our infrastructure or sre stable counterparts to be involved in vetting and looking at issues early in the pipeline to decide whether or not they would yeah or or let's say we're implementing a feature like let's say we were in blank mirror and from scratch like the first question we should be asking is is like how many how many mirrors as a customer expected to be able to support and what I want to start charging for it if they get above a certain limit and you know and right now we don't and you could argue that a scaling is just as much a reason for customers to start paying us as its feature sets that's that that's kind of the argument I would be making because those things cost money like whether we like to admit it or not yeah Krister I would I would agree with you and what you're trying to sort of shape up and call out here and the sunset you know going through pages for example performance of getting those page loads loaded is not great and I don't know if we set out originally to track some of those performance things but I think that performance and in to your point Kenny I think performance should be somewhere incorporated as we move forward and something we should be thinking about for scalability across the board because it's just as important as bringing forth that really cool thing to them is that that really cool thing works and people will stay there to use it I think just as a side note I think we have something that product end book today but like a couple days ago on performance something like fast applications are like always you know like more usable and I think that's that's definitely important and I also think that github.

com is massive right I think we have 4 million users and for example 4G you I know that only by actually like interacting with the infrastructure we are getting feedback on some of the performance bottlenecks that we are just not seeing otherwise and so I think that's actually also really valuable and in that regard maybe also like again you know dogfooding these things helps and I think with the combination of CD we may hit both of those things at the moment yeah and the dogfooding thing on that front is a little confusing me I met with Maron to talk about that and you know there's sort of this mentality of looking at calm first are leading with calm for scalability and I just it's not really Chris to me where we're going from making sure that we're you know how we approach making sure that we in tact scalability for calm if we're sharing the calm or restarting somewhere else from a doctorate in perspective I'm pretty sure the handbook says that when meant to release the guidelines used to be that for new features they were meant to be available on get like calm and self hosted at the same time and that there used to be a production room ready checklist that I think the engineering team was responsible for I know that for when he launched in geo there was a production readiness process that we had to go through and certainly we giddily we can see these things on the source code front were regularly considering scale like moving terabytes and data from the database into object storage and considering all these sorts of things performance is very much a feature and should be considered that and I think particularly in categories where adoption is still growing and in early stages of maturity performance by understandably is less of a concern because there's lower usage so like solving scale at like an enormous level doesn't make sense commercially like necessarily when usage is small so there is a bit of a juggling act here because we don't want to build a product for billions of users if there's only I'm like 20,000 users experimenting with our newest feature and so there's a there's an iterative approach that needs to be taken but I I would agree that particularly coming from a team that's digging out a lot of technical debt and solving a lot of performance problems all the time we've probably historically not been very good at picking the right moment to pay off technical debt and address performance problems until they become fires so yeah so - to that point just real quick James and sorry Scott and I think some things are obvious like when we look at our progressive deliveries strategy I think that we like if you look at something like feature flags there's no that like that's something that I think is going to be like I wouldn't imagine that that's not going to be a key feature that we're gonna bring forward so I feel like that that should be a gimme on whether adoption has yet struck or not but the second thing that is not clear to me like again when I was interviewing there and about dogfooding is that I noticed that meirin's like we don't this isn't we weren't they didn't come to us first and so this is not scalable or this is not usable for us internally and so it's like the approach and process moving forward to dog food in the right spots is not clear to me or you know what the best practices event or if anybody's you know crack that yeah I can give a concrete example is I did a call with Myron a few months back around confidential merge requests so we knew that customers wanted to resolve them we knew that we wanted to do that and we're trying to get rid of Jeff document org so I had a video call with him and a bunch of async conversations with like I've got these ideas for the first iteration looks like and then we did a few calls and work through them and worked out which were the things that they would happen and so we're shipping the first iteration of that and 12.