Testing Business Ideas

hi this is elizabeth crossnickel and i'm filling in for john secrets today super happy to have you at our seattle underscore angel meetup we're really about increasing the ecosystem for investors and for founders in the startup community and john's been doing this for quite a number of years our guest speaker david bland is here tonight and david's going to introduce some stuff in a few minutes but david and john have been working together for a number of years and they have been working in with the lean startup focus and so that's going to be our

talk tonight yeah thank you thank you for uh having me this evening i'm out here in northern california so not too far from you all but also also very warm apparently it's almost as hot as where it is you are so um i am going to not be using slides today so i'm going to be using this mural board and then i will share a link in chat in a bit and you all can join me for a little exercise that we do yeah i'm david uh again out here in northern california uh my career

has been all over the place as most entrepreneurs uh tend to say that i went to school for design ended up in dot com craze for about 10 years uh was one for three with startups there so uh the one that was successful i was there for almost eight years and we pivoted we went from uh we thought we were b2c company end up being b2b you caught a pivot back then we just kind of freaked out and it was almost out of money and realized that banks were our customers i'd like to say we

pivoted based on data and insights but it was kind of like nobody wants our product except these banks maybe they're really our customers and that's how we took off uh we required for 16 million and uh some of the other side was that you know we just kept persevering no matter what it was like well um people don't know what they want we'll just keep building this we'll make it better and uh didn't didn't pan out so well um then i switched coasts uh and so i've been in the bay area for like about 10

years in silicon valley and around uh helping companies there and startups there and accelerators work with some vcs there and then recently during the pandemic relocated outside sacramento so kind of all over the place my career is kind of a weird mix of startups and also uh coaching and so working with big companies and with startups and so mostly the single thread i could say that's that's common throughout the mall is that it's based on kind of this zero to product market fit area so usually even though i've helped skill companies where i really find

that i enjoy working and helping people is if you have an idea there's a messiness to when you find something that works and you you scale it i usually uh back away when it's time to scale but but i do work with some vcs down here in the valley um i just listened to some of your ideas and uh you know i work with some web 3 cohorts for tim draper down at draper university and working a little bit in food and hospitality um a little bit in chemicals a little bit in health tech i

don't really focus on a specific industry so hopefully you all get something out of this session uh tonight all right so um the goal today and and i try to have a learning objective for every session i do it's basically like to help you de-risk what you're working on so uh some of you are probably a little further along in your journey some of you already have you know customers on your platform selling things so you know you're a little further along some of you sounds like you you might be ready to create a company

in three years and so you're just exploring some of you had side hustles that's fine but basically whatever you're working on no matter where you're at in your journey i want to help you kind of de-risk that and basically feel a little more confident if you're on the right track or not i also have a code of conduct for everything i do which is pretty much the bill and ted code of conduct which is be excellent to one another not too worried about this group but basically you know when we're sharing out and and uh

please hold your questions to the end um of after the exercise we do but if you if you're sharing and giving feedback just be constructive in tone and in language okay so i'm going to give you kind of a high level a lot of this is going to be based off of my book with alexander osterwalder um it's over two years old now i think yeah it's over two years old now almost three just follow b3 and i think it's just getting started you know like it's it's um it's amazing with business books it feels

like i think the stat my publisher gave me was almost like um it's really rare for a business book to even a non-fiction business book to sell even 10 000 copies right and we're well on our way to to six figures in ours and so basically i think it's in 20 different languages so i think why it resonated so well is we tested the book along the way it's pretty much based on like how i coach coach companies you know so i tried to put on my like coaching hat when i was writing it and

tried to write in that tone and i think that's why it landed really well but one of the three themes you know the three themes that we keep talking about in the book and i want to share with you a little bit and you're probably familiar with these but i'll cover them really briefly which is this idea of desirability viability feasibility it's kind of i first learned it in design thinking and design school but you pre you pull this thread it goes back through decades of human centered design but essentially this idea of like hey

do they want this so when you think about do people want this is there a value prop uh is it like the value prop resonate with the customer do we understand their jobs to be done like what risk do we have around the desirability of of the initiative we're trying to do a viability which is more kind of should we granted you should have a moral compass but basically in addition to that should we like do this is it something we can make money at is it something that we can least sustain like even non-profits

i work with i work with some like huge nonprofits around the world and uh you know they have to show impact you know for the funding they've raised and and so uh even even they have to be viable in some sense and then feasibility is more about the can we and the can we is technical but it's also regulatory governance you know some of you working in food and other things uh in medical devices those worlds are also heavily regulated so just because you can do it technically it doesn't mean that uh the regulations will

allow you to do it and you'll be shut down if you violate them or at least usually shut down i've noticed some startups in the valley just don't get shut just they just keep going i don't know um uber's a good one um anyway so when you look at this um the thing i want you to take away from this framing and i'm going to jump to the canvas in a minute is if you only focus on let's say desirability and viability so you focus on hey people want this and we think we can make

money doing this they'll pay for it but you don't focus on feasibility you're going to fail in a big way because you're going to fail the execution side of things so think of like a lot of kickstarters that didn't make the you know light of day they they promised to like great explainer video and concept video and a bunch of perks and people threw money at it but then if you couldn't figure out how to do it right that's still a big risk the backstage the other way is if you look at this if you

say okay well something's desirable [Music] and it's uh feasible so people want it and i can do it but they won't pay enough or i can't create it at a low enough cost to keep it going and that's going to fail too you know i did a big keynote before the pandemic in san francisco and mine the product and i used clearly canadian which i loved uh back in the day in the 90s as like my product you know that was desirable um and feasible but it wasn't viable because it cost so much and then

other players came in the market like pepsi clear and all these things and kind of disrupted it it is back now because vc's bought it and kind of like did an indiegogo and brought it back but it's not the distribution it was before i think when they when they bought the ipv literally in a cardboard box it was like not a lot left of the company by the time they got it but but you can fail that way too where somebody really loves it uh and you could do it but if you can't make the

numbers work you're gonna fail and no one that's really frustrating because you can have a product everyone loves and you still fail because you just can't make it work the numbers work and then the last one here that unfortunately still happens here in the valley and other places too which is you know you can do it and it looks like there's a model there which is nobody cares and you're not solving a real need um it doesn't mean everything has to be needs based but it's like just nobody desires it there's no there's no like

job to be done behind what you just built and a lot of these i see you know i went to so many minutes i used to run lean startup circle in san francisco and i members vividly having uh this this guy just just stand up there and give us this really long dissertation about this very like complex algorithm that he'd been working on for like six years and he had no idea what job it actually solved for people or what what the use case was what the value prop was or who the customer segment or

how much he would charge for it and so why he was very passionate about the technology you know he was very frustrated because i think that's why he ended up in our meetup he was very frustrated because it was just like just i keep working out so i've like spent my life years of my life on this and and i don't know what's wrong and it's just like if you miss desirability it kind of doesn't matter how awesome your thing is like if it doesn't really solve for a problem it doesn't resonate with people it's

going to fail so ideally right like any you know any good venn diagram right you're going to want to focus on all three now you don't have to focus on all three at once um usually and i'll kind of share this in the in the next segment here but usually what i see is desirability is typically the biggest risk early on because you are brilliant you can probably you know do anything but if you miss the customer segment and you don't have a value prop and it's not connected to a job to be done or

some kind of pain or experience and gain they're looking for you're going to fail in a big way early on so so what we've done is uh alex was the first one to do this and i was kind of like around him when he was doing it um you see and i were writing together and i think the testing business ideas book is the first book where we officially you know use this framing on the canvas but basically we're just like layering that thinking over the canvas so the canvas has been around at least 10

years at this point um when i first met alex um it was right when business model generation launched that book i was like oh this is brilliant i'm already doing lean startup stuff and this stuff like fits because i was working with eric greece back then and i was like this stuff fits together really well and that's how we became kind of friends over time and we realized that people can't take this check the box mentality and i'll zoom in a bit here for you um it's really not that useful when you think of something

like this is like i'm just gonna check the boxes and yeah i'm gonna go execute on this one there's a ton of assumptions you have in your model um and two while you get some value out of like checking the boxes it's gonna be the relationships like how things connect that's really gonna help you like tell a story to investors um you know i do i work with vcs and they you know i work with them because they want their portfolio to be able to tell a coherent story or people that want you know they

might invest in tell a coherent story and so you may not like show this canvas to an investor or investor might not request this canvas but there's a good chance they're going to ask about all the stuff that's here so when you think about it you know the kind of the flow i use is okay well who's your customer you know like who who are you serving here and then i jump over to okay so what's your value proposition value proposition sorry like my messy drawing here this evening there we go and then uh what

are your channels okay let me uh go back to my thicker one here so what are your channels so think acquisition distribution and then okay so what are the types of relationships you have with your customer are they automated in person are they ongoing are they one time so think through what kind what's the nature of the relationship and you'll notice that's all desirability risk pretty much it's all like this interconnection of your value prop and your customer it's a lot of desirability risk early on when you're filling out your strategy you're like oh maybe

i've never talked to my customer or maybe i don't know how to reach them and and so just be mindful that early on this is a lot of desirability risk and then we drop down to okay so how are you going to make money so this is all front stage right and then on the back stage we quickly go into okay what are the verbs so think key verbs activities that you need to do to make this work not every activity in the world but just like the main ones what are the resources you need

so think uh physical and digital you know what do you need to make this happen what's this going to cost and then who are you going to partner with potentially and be careful with startups because you know we we we partner and i see this other startups too nowadays like they want to partner with big companies right away like i was just talking to somebody who was like i'm partnering with a large and national bank and like when you just said those words together i just i felt uh nervous for this founder because um in

in in this grand scheme of things for big companies you know six months 12 months night 18 months it's like whatever that's fast for us you know but the startup years that's like an eternity right so just be careful when you're thinking about partnerships but one way you can think about a partnership is hey we're only going to partner with people who bring an activity that we can't do or need help with or they bring a resource that we don't have and we need to make this happen or maybe even they create a channel for

us to reach our customer now when you think of partnerships that's that's a a really strong way to think about it i mean you could you could you know if depending on how uh disciplined you are you could create you know kpis for all this too and measure and say okay what's the the health of this partnership can they do x y and z another thing to think about is well we need to deliver our value prop to our customers and reach them and we want to build a relationship with them right and then we

also like need to generate revenue somehow so that should come from somewhere and oh by the way you know our costs should be less than you know what we're charging here and so when you start thinking about it and you can draw your costs from activities and resources you know and you could say oh those are our costs if we have to do these things and have these things the verbs and the nouns this is kind of backing our way into cost so i don't really teach this as a checklist um the way i teach

is is this is like your story basically and so where my book kind of picks up is um you know there's this great series and strategizer right and alex osterwalder um he's based out in lasad in switzerland and the designers that make these are in toronto uh alan and um and trish and team and they already had great books like we had with business model generation value prop design and we say it's a stack it's like literally a stack at this point it's like a stack of books and i was like we need something to

plug in here we need something to help people test you know and that's kind of my background right there's a lot of like lean startup early adopter design thinking agile all that stuff mixed together and so what we ended up doing okay let's write down our assumptions around you know this the strategy that we have our business model because it's not fact especially early on it's just a bunch of guesses and then let's map them out and try to figure out okay which ones should we focus on so i have this little two by two

i helped create called assumptions mapping it's used by google and governments and all kinds of things now but why it works i think is that you just kind of have well how important is this to my success and then how much evidence do i have that supports the statement and so what happens you start bringing one over and you say okay well we have some big assumptions around like um the jobs the customer has right like we don't even know if they have these jobs or trying to do these tasks when you pull your next

one over you can say well is this more or less important than that one and do i have more or less evidence and then once you do that you know it comes together pretty quickly and it's really relative i'm not looking for pixel perfect you know as placement on this thing it's more of a focusing mechanism but what should probably happen early on is that you don't necessarily have like a lot of blue that's why i like colors you don't have a lot of blue in the top right you should have probably some green and

a lot of orange um let me zoom in for you all so basically you know when you look at um when you look like what are the riskiest assumptions like that's how eric could frame it or what are your leap of faith assumptions which is kind of even like pulling from like kirk guard but basically i'm doing the same thing with this exercise except i'm just using a little different words but i'm trying to get you here like i'm trying to get you to focus here right so if we say this is a leap of

faith right or we say this is the riskiest right that's what i'm trying to get you to i'm trying to get you to just let me uh screw this over to focus on okay if this is wrong it kind of doesn't matter what else we do it really doesn't matter so um what we tend to do is we have people kind of sketch these out if i'm working with founders sometimes they do them by themselves sometimes they have their advisors give feedback on it if it's a founding team you know and it's cross-functional so if

you have people that can talk to the desirability viability feasibility risk even better we kind of try to map these out and say okay this is probably wrong but based on what we know today this is what we think are the riskiest assumptions so if we're going to experiment let's do this first uh because if we skip this it kind of doesn't matter what else we do like yeah we can make it look prettier we can make it do like more integrations and all this stuff and we're all these different things you could do in

software or even hardware or services but it kind of doesn't matter if you get these wrong so one of the reasons i wrote the book was a lot of the the companies i worked with i felt like they would do interviews and surveys and that's kind of it and then like build the whole thing or maybe a landing page and i realized there's like so many things that you can do in addition to that and the way we frame them is kind of building off of steve blank's work in four steps epiphany which kind of

informed eric reese's leading startup book because eric was in steve's class back in the day but we kind of frame things discovery and validation so discovery it's like i have kind of i don't have a lot of evidence at all that i i'm on the right track here so it's more exploratory so if you think from an evidence point of view you know we're trying to get you to go from some actually actually even before that we're trying to get you to go from none to some okay so we just want you to gather in

like directional evidence that this is actually something that i could possibly build right we want you to defer a building as long as possible with validation it's a bit different you're usually a little further along in your journey and so you've already gathered some directional evidence through um you know a lot of like ethnographic research or search trend analysis or really scrappy testing with customers just to see do i understand their jobs pains and gains do they understand my value proposition are there's any willingness to pay for for this opportunity have they paid for this

in the past in other ways and hired other products all kinds of stuff you could do when you get to validation there's more of a value exchange and so you're trying to go from an evidence point of view you're trying to go from some to strong so basically when we talk about like mvps this is where i personally would put mvps and i tried to write a book without saying mvp and i didn't quite succeed but if you look at it from like an evidence point of view you could say well a concierge like maybe

i can just do this manually with almost no tech you know a lot of travel companies do that like if i want to do ai with travel there's probably no ai first right i'm just booking travel for people and understanding patterns and looking at data and then automating things right and i've seen a lot of that advice especially for even bootstrap people recently which is pretty much like do things that don't scale do it manually and when you become overwhelmed you automate the things that you need to automate and i like that i like that

approach rather than raising too much money early on you know you own more of your company too that way um with wizard of oz it's kind of like concierge except basically um it's not obvious there's a person involved and so that is another thing that uh i have a funny story i can share with you in a bit on that one but basically i did a cohort of ai startups in the valley i was a couple months ago and it was it was very little ai and a lot of spreadsheets and people so um there's

some there's some of that going on as well when you think about other things there are like a single feature which is you know i did one thing and it just does one thing and it does it really well and then mashups are becoming more common especially in software today where you can um mash up different services and end up being a product that you can actually uh it might last you one to two years you know it's pretty incredible the maturity of the tech space now um but basically there are a series of things

here and i'll share the link to this board here in a minute with you all so you can join me but there we can't cover all 44 of these experiments today but there is a nice collection that you can choose from to say okay here's the kind of risk i have then what is like my next best test i could run to get a little more evidence on that before i go and rush to build because i've seen some crazy stuff and i'm sure you off too where it's like well i talked to five customers

then i spilled 600 i spent 600k on an app and launched it it's like no there's there's like so many things between that um so what we did and this is all again based off my coaching with teams and stuff is we kind of applied a taxonomy to this to help people so i felt like there wasn't there's plenty of lists of stuff out there but what we did is we said all right let's kind of break it down it's not perfect by the way and if i do a second edition of this book there's

a good chance these dots are going to change based on what i've seen even the last couple years but it's pretty good um what we did is we said okay how much do these cost so think how much does it cost to do this is it free is it going to cost me like tens of thousands of dollars do it i'm just going to cost to run this experiment how long is it going to set up so how long like much prep time am i going to have to have to do this and then how

long am i going to run it for so it's a runtime when i first wrote the book i had these combined as like just time but then i realized some of these like you you can set up really quickly but they run a while and others take a long time to set up but then you can run them very quickly so i broke it out into two and then uh what kind of evidence strength did they generate is it light evidence is it strong evidence you know customer interviews surveys they're great pretty light evidence it's

like what people say or what they type it's not necessarily what they'll do and can you get closer to real world where it's people doing things or people paying for things or giving you their time and attention in a real way those are much stronger evidence we called out the capabilities you'll need at a high level either if you don't have those capabilities maybe you have access to tools that can help you with those capabilities like if you're not a designer but you're trying to spin up a landing page they're playing like tools now that

have great templates that are good enough that you can use instead of being a designer and then probably one of the biggest things here is remember these themes i talked about desirability feasibility viability we tagged those for every experiment so you could start to understand hey i have a lot of let's say desirability risk what are some experiments i can run that address desirability or i have a lot of feasibility risks a lot of backstage what are the things i could do to address that or i have like viability risk around cost and pricing what

are the things that i can do to address that and so if you look at this one concierge is like always a fun one because you're manually doing it all basically if you look at it you know if you're back here creating all this stuff by hand that's a lot of feasibility you're trying to address like even though it doesn't scale you're trying to address the feasibility aspect of it with desirability right you're starting to understand do people do people really value this thing that i created and if they're paying for it you know you

are also testing viability you can also you know do back of the napkin math back here oh my thing's still drawing but um there we go you could also look back here and say like what's it cost to do this but if you look at the themes you can start pecking this apart and say okay this is what's going to take to do this manually do people value it will they pay for it and start backing your way into some of this and getting like really really strong evidence for it so what i want to

do is kind of throw a link in here all right so you do not have to create an account by the way i mean um actually full disclosure i do advise this company so i have to say that like the more i do these sessions it's like oh yeah i actually help that company so i actually do advise mural however you can create a free account or you can just click and you'll just appear as anonymous snail so you do not have to create account even to hang out with me today so full disclosure though

but i do advise them all right so what i did for here for you all um i know we're trying to get books for people too and i appreciate all that but in the meantime um i have a cheat sheet two book in this board and i actually gave this presentation to my editor at wiley and they were kind of cool with it so i'm just going to run with it until they tell me to stop so basically these uh cards these are like little cards now they don't go into the same um depth as

the book but it's it's a pretty amazing cheat sheet like i'll leave this board up for 30 days for you all but it's a pretty good cheat sheet for you all to have for a bit and so uh let's say you weren't sure about like um what's a good one to talk about with this group um let's do biofeature so let's say you're trying to figure out what customers really want and what they value well buy a feature is a technique that i actually learned from luke holman who wrote innovation games who's based out of

san jose and it's it's a thing where you give people essentially play money and they um and then some features and the features all have dollar amounts and they have to like rank them they have to spend money on the features they really care about because there's never enough money to buy all the features a lot of my b2b clients do something like this you'll notice here like it doesn't take long it's not that expensive you can do it pretty quickly right and so these are all pdfs so if you just double click this guy

you can get a higher res thing all i ask is you don't share this board publicly like i said it'll be up for 30 days but you can get a nice like high res view of what these are each one's a pdf but basically these are sorted by um discovery so again when we talked about none the sum up here at the top so let's say uh what's another one day in the life so intuit does this a bunch still i know the folks that run their day in the life program they call follow me

home which is kind of creepy but they do get permission they'll just randomly follow people but you can basically observe people and understand better understand jobs pains and gains they do that to basically understand how to solve for needs of their customers so so that's one that's really popular as well but there are a bunch in here and i can't cover them all today down here in validation we have a bunch of mvps we have if you're familiar with lean startup you know a lot of these things i would say one that's becoming really popular

recently with my teams has been letter of intent so what we'll do is we'll write especially b2b startups we'll do a one-page uh letter of intent which is a non-legally binding contract and so we're doing interviews we'll say okay this sounds they'll say oh this sounds great i want to buy in number of things right we'll say okay well before we do a purchase order can we just do an loi letter of intent to get this in writing and some of them won't put it in writing you know so then you start to understand okay

you said this but you're not willing to put it in writing so like this is why i talk about light evidence stronger and evidence right letter of intent is a little stronger evidence because they're putting in writing and some people won't when i'm working in health tech we'll go into a hospital nurses will say i love this device i want to put this in everywhere like great can you write a letter of intent um to like or a recommendation a letter recommendation even to your your administrator recommending us right and like oh i'm not willing

to do that like i love it but i'm not willing to actually recommend you to the person with purchasing in uh like purchasing power and decision making in the organization so there's all different ways you can do this but basically what i wanted to do is give you like a taste and this is a real story by the way a taste of kind of like how companies um how companies do this and how startups do this so what i'm going to do is i'm going to do this on the fly with you all and if

you've seen this one before i've actually shared this with a few other meetups recently so uh just pretend to be surprised all right so uh one of the ones that i worked on in the past i worked around in the past was and i love this story it's um it's basically a men's a dating advice chat bot okay so this is the team out of new york uh pretty amazing team the the the founder is now she has a different startup where it's date it's it's not dating advice it's like advice for young parents on

learning milestones and how to shop for their their uh their their children and their toddlers right so it's it's really interesting i caught up with her the other day but basically um this is this is a um this is a a real thing that basically would um how could i say this basically you text it and it would give you advice back on dating okay that's pretty much what it was but instead of building it right away they did customer interviews all right so that was one of some of the first experiments they ran they

interviewed men uh they ended like what are the pains with dating getting dating advice is it overwhelming where do you get it from they also did some search trend analysis after that so they basically went in and said okay um what [Laughter] um based on like you could interview 10 to 15 people but that doesn't scale right so you can go online you can use different tools to see are people searching on this uh this topic online or related topics online in region per uh seasonality and all that um they also did uh some paper

prototyping so basically they were sketching out what the bot could do again and testing with people not necessarily building anything yet they did they created a landing page okay and then they also basically did a wizard of oz and there's some other things he did but i'm just trying to simplify it for you all today but so when you look at this and the wizard of oz is um it's basically faking the bot experience so what they did is they had a team of women um and every uh request that came in they just pretended

like they were a bot and they gave advice back to the people and they thought they were talking to a bot but really they were just talking to women giving them advice through text okay so if you want to test out chat bot instead of building the bot right away you can kind of just do it yourself it doesn't scale which we'll talk about here in a minute but it's a good way to test like the different themes so what i liked was they tested you know with customers they started to understand qualitatively the insights

and then they used that to inform the search to analysis they used that to help inform what the paper prototype of the thing could be and test it with people who use that to help inform you know the landing page and landing page design and then they did a wizard of oz where they basically had um you know three women right and three women and they were texting back uh men who were putting in real requests and i have some screenshots of the actual requests and by chance i actually know one of the people that

sent a request in which is kind of funny very small world but um here's where it went awry okay so again not every story has a happy ending and this one does not why this one does not was that their landing page had a thousand signups in a few hours okay so when people say i say keep your experiments small this is one of the reasons i say that because that completely overwhelmed you know these women are just like how am i going to respond to all these texts like basically they had people complaining on

forums saying i'm so helpless the bot is ignoring me like dating my dating life is a mess you know so they couldn't even keep up with demand so when you talk about wizard of oz keep in mind that you have to keep it small usually you usually have a small team usually doing it manually and you can't have a thousand sign-ups like you can't even get back to these people right so they had to change that to a wait list and that helps stem the tide and then they could start to you know go through

and process these uh requests now interestingly enough we talked about these three circles right we talked about desirability we talked about viability we talked about feasibility so desirability i thought that was a pretty good demand you know it was like people love the advice um i still hear stories about some of the questions they asked which were pretty hilarious but they loved that that they could do that they could get the like advice on demand that was personalized to them okay feasibility the can we do it was kind of a question mark because it's like

well you could take all these requests from people and then you can make a bot and you could try to make it happen but that's really tough like you could do it it's going to take time to make it like really um flow like in a text conversation and viability was a huge no like people did not want to pay for this bot now you probably could figure out of maybe some weird sponsorship or affiliate or thing i don't know there's probably a way to make it work but this team decided to to shelf this

one so not every experiment uh is going to be successful okay and this one you know trying to test out desirability viability and feasibility people loved it like almost too much okay to be fair the feasibility is kind of like well we could probably get data scientists to work on this and make it a real bot and then viability was like well people don't pay for it so what what is the end game like what can we do and so this one was shut down so what i thought we could do and this is a

real story uh of one out of new york um way better to learn that early than if then can continue to work on this for a while right yeah for sure like i like i said i've met founders that would talk to a couple customers or think there's a problem and just build the bot right away you know and spend hundreds of thousands of dollars sometimes and then realize that nobody cares so what i want to do uh if you are up for it and i see a few people here in the board is i

want to give you some time to check over these experiments so you basically use like the little hand in the corner here uh that'll help you drag around if you're having trouble navigating but basically all these experiments are here for you to per uh to basically just explore and what i thought is like some of you are working on some really cool stuff and if you're willing to share um think about what kind of experiments might help you you know like this one we went from interviews to search trend to paper prototyping to landing page

to wizard device that's just one flow that's not necessarily a flow that you should follow maybe there's a different flow but if you want to create stickies all you need to do is just double click and type like it's pretty much like using powerpoint or keynotes pretty easy if you can't create a sticky there's a good chance you have the hand icon on it won't let you edit when you have that on okay so i'm going to set a timer for a few minutes here i'm going to set it for five and here's the link

again and i'm a place of music and then i'm let you check out these experiments here to the left here the discovery and the validation experiments and then if you have something you want to share great i'm happy to give feedback if not we can jump into some q a okay so i'm going to go and mute five minutes and go do [Music] oh [Music] so about three minutes left [Music] do [Music] [Music] [Music] have about one minute left [Music] [Music] [Music] [Music] all right try to finish up this that's seatbelt all right so um

does anybody want to share uh what they came up with and after that we could jump into some q a you can just unmute use uh this one with the paper prototype local prototype data experiment one it looks pretty cool if anyone wanted oh it's me yeah do you wanna explain what you're working on there sure um so basically uh we were trying to figure out whether having um a a portal with information i'm trying to be careful of what i'm saying here given i'm working on a law firm that's super sensitive um so we

were trying to work out whether having a portal um with information in was going to be useful um to our clients so the first thing we did was develop a bit of a paper prototype of what the information would contain shared that back with the client and said if we had this as a system and you were accessing information was that would this be useful usable and they sort of said yes sort of gave us a bit of a direction of what it might look like and so we built a clickable powerpoint that would show

them how it would actually have the right information in um and that was able to sort of give them a view of actually seeing it would they click on it would they use it um and then we basically made it a bit more of a high fidelity prototype so we actually built um something sim very simple with the right data in um to show them and then we've literally done an experiment and said okay we'll give you this data we'll give you this prototype use it for a month and then we'll ask you some questions

and then now we're going to meet with them next week and the the biggest question we're going to ask is you know if this was real and you could have access to this would you use it would you pay for it how often would you use it what would you use it for when would you not use it and get more information to figure out before we actually enable it whether it's valuable very cool thank you for sharing and i know it's hard to talk about things sometimes um i'm insulated by ndas so hopefully that

gave you a bit of an example of how we've you know got from idea into you know before we build anything and these have been quite quick turnarounds between each stage yeah i think um no this is cool a couple of things uh maybe just like just points feedback and i don't understand the complexity of everything you're working on of course did you notice a change between paper prototyping which is just kind of like it's fun but you're just drawing right and you're showing them drawings and everything maybe they're pressing a piece on the paper

or something but did you notice a change in like perception or how they gave feedback between paper and clickable or you know paper and data when they were interacting with something even if it wasn't live and fully baked did you notice any difference between you know how they responded how they spoke about uh the potential thing you were going to build yeah definitely so when you you know the first bit is more of a show and so they sort of understand the concept but when you're actually physically clicking and looking at something they're a lot

more engaged and then they're trying to figure out you know what information is most useful and so and then they're asking more questions about oh i'd use this more if we could then drill down here or if we could get this or if that was at the click of a button so they were able to explain a lot more how useful or usable you know the idea was by clicking and actually doing something rather than just hearing about it or seeing it yeah thank you i've noticed that with some of my other teams as well

um some of my teams love like explainer videos and i like explaining their videos too but but explainer video is a very passive experience right you're just like it's almost like a presentation you're watching it uh hopefully i have little templates for them too you know it's like here's the job pain and gain and what we do and all that there's a flow but it's just a different kind of feedback you get from that watch watching it's like a very passive experience to clicking even even if it doesn't completely work right it just might be

hot zone buttons here here here and there's dummied up data or just screens that move or something you know and i've noticed that the quality of feedback and just like their excitement level and how that changes by just clicking on something that could literally just be a hacked together powerpoint or keynote or something is really interesting another point of feedback and i just want to bring this up because you're talking about what you're going to say to them when you go back after they've been using it i think be careful with the would you future

hypothetical situations i know it's tough because they've been using it but i think i would just challenge you to think through can you frame the question to have them talk about the past of like when in the past would this have been useful for you when did you have a situation that this would have benefited you had it been like fully you know implemented and all that or have them talk about how much time and effort and money they spent doing something other ways right and and and i think um you're going to get some

more interesting insights out of that because i'm afraid what might happen in the future hypothetical scenarios is they're kind of going to give you it tells me like yeah of course i would pay for that or of course i would use it but sometimes they just end up telling you what you want to hear in that situation and i think one way to get past that is have them talk about the past and events where i'm doing this with a medical company right now where we're having them say well when in the past would this

have been valuable like it's still a would you question but they can anchor on the past so that's just something you might want to try if you've yet so helpful yeah that's really helpful thank you so much cool and so yeah and this and the reason i do this exercise is that this whole process it's it's tough right it's like applying a scientific method to business and so it's not just like so many i was watching this thing the other day was like so many inventions have happened by accident and and innovations have happened by

accident over the years and you're not going to get it right the first time and it's going to take some trial and error so don't think just you're going to run one experiment and then it's going to be enough chances are you know if i zoom out a bit when you think about this two by two up here right you have a collection of things you're worried about that if wrong could cause your whole business to fail so it's going to probably take more than one experiment to really learn about those and so that's something

i've been trying to drill in with my teams is that this isn't a one-time event like i checked the box i ran experiment now i'm going to move on and build the thing it's going to take some time and so if you're looking for time boxes you know i might spend something like 12 weeks on this before i make a big decision because i don't want people to get too excited or too depressed after the first experiment because i've unlocked like tens of millions of dollars with companies and quite often the first couple experiments they

were ready to give up they're like this is hopeless nobody wants this and then we're like well maybe we tried this instead like one of the big famous ones i can't really talk about publicly but it was with um it was with mouth pain right it was with pain in your mouth it was a product and uh all these people were like yeah whatever i don't care and then you know what what worked for us like the team was about ready to give up and what worked for us is we went online and we said

how many people are searching for mouth pain right now and they're all these search terms they're like thousands and thousands of search trends uh like search queries about mouth pain and solving mouth pain they're like what if we'd use the same ad in the same landing page and just put it there in the results when they're searching and we went from like almost a zero percent conversion on our page to a 40 conversion so that's like ridiculous 40 is really high by the way uh for conversion on a page and so you know that team

almost gave up they were almost like no nobody cares this is pointless why are we even working on this it's like well what if we just get in front of the people who are actively seeking a solution and see if they like it you know so it takes time it takes time and so don't feel like oh my gosh i'm so bad at this it's like back in science class you know in high school or something really you just you try to isolate like what are the things i'm really worried about and then what are

the things i can do to learn about those and can i do it quickly and keep challenging yourself to do it quickly and so hopefully you know this gives you some opportunities to do that i know we're over time especially my time anyway but i have a couple minutes um before i have to to go to a kids karate class but um what kind of questions do you all have for me anything that like was confusing or anything that you'd want to ask me why you have my time i have i have a question from

uh i've heard you use these uh phrases twice and i'm not exactly sure what you mean by them because um it it's almost like you read this i mean it's amazing so um the questions the question is what do you mean by verbs adverbs are activities you need to do you use that verbs and nouns thing yeah that's a great question so people get confused by this canvas part especially the back so um everything here in orange is pretty much like front stage so think of like you're at a play and if you're at a

play everything you're watching is like happening on stage or interacting with the audience like it's it's like this interaction ever it's pretty visible right yeah backstage there's a lot of stuff that has to happen to to make a play work like my wife used to help do like set design i had some friends that do set design for like theater in san francisco right there's a lot of stuff that has to happen like props and everybody in the right place and all that that's a lot of the blue so when you think about it that

way think about the verbs activities so what are the key things you need to do is it marketing is it creating content is it manufacturing is it software development is it sales like what are the what are the big things that you need to do to make this business work when you think about resources those are the nouns you need to have so think physical digital could be uh you need a platform you need a team you need a patent you know there's some kind of ip that you need so i like separating those activities

and resources to think about what are the key verbs or key activities that you need to do to make this work and then with resources think about what are the key nouns what are the what are the physical and digital things i need and there might be hundreds of those but think about like you're trying to explain this to an investor or you're trying to explain this to a co-founder you want to bring on it should be high level it shouldn't be down like this isn't project management i mean not that the project manager is

very much needed but this level it's more like i'm telling a story at a high level at a stakeholder level level at an investor level and so think about the key things that you need to have and you need to do to make the business work and when you tell a story i've noticed that if you follow even these numbers you said here's my customer this is the value i create for them here's how we reach them this is our relationship here's how we make money and then when you think backstage you're like here's the

thing we need to do things we need to do here things we need have here's what costs and who's who we're partnering with if if that story doesn't make sense along the way somewhere it's probably an issue with your strategy like um i work a lot of startups that they'll just like load up the revenue stream box with a bunch of stuff and then i'm like oh so where did that revenue come from it was like well i don't know it just sounded good it was like no no we need to sort through the revenue

right like and so the storytelling i i keep coming back to the storytelling it is so important um i mentioned this on a video actually with draper university working with tim draper in um san mateo and i had a startup founder in there that said this is the first time anyone's ever understood my startup and i was like that's great like now you can tell a story about it you know and so everyone's so deep like subject matter expert domain expert and we talk in words that we're comfortable with yeah we need to be able

to tell stories about our company about our business especially the business model and i feel like if if you have a problem in your story there's a good chance there's like a problem with your model and so i've been really trying to emphasize storytelling and all this okay thank you so much i'm i'm very excited about this so cool to be with you live and to be able to ask a question wait till i tell cameron dan because they said oh can you give me that because i asked them what that meant in your video

and they said can you give us the time stamps yeah i'll have to talk actually i'm talking to them soon so i'll let them know yeah i think i mean i've been doing this for a while um but i think um a lot of this stuff i feel is taught but it's not taught at a very actionable way and so what i'm trying to do with this work and hanging out with you all and writing this book is trying to make it a bit more actionable for people and less like this is a thought exercise

right because what i want to do is help you run experiments to de-risk what you're working on like the goal isn't even to run experiments right the goal is to de-risk what you're working on and so do you have a hobby or do you have a startup you know like some people just have really expensive hobbies and they call them startups and they don't make any money and they keep throwing money at it and so being able to kind of work through this risk is really really important so it was a great question though thank

you so much thank you so i know um i want to be sensitive because david did tell us beforehand that he actually had to sign off for us and we still have the opportunity to hang together but i wanted to make sure that you are aware that david is actually running a few workshops over the next little bit and i'm just i just put one and that registration is tomorrow is that correct it finishes tomorrow yeah the timing this is interesting uh i don't want to put pressure on you all i didn't plan it this

way but um so i'm doing a small cohort so i don't know if you're familiar with cohorts think like if you're familiar with seth godin and that kind of stuff it's pretty much like a small collection of people that are individually working on their own ideas and so i'm running my first one uh the cutoffs tomorrow but it starts next week and it runs for 10 days and basically i take you through this whole process and you're also doing it with other people and so they're all working on their own stuff but then we can

share out together so the reason i'm doing this is because um i do a lot of really giant workshops like conferences but people can't work on their own stuff it's always like a case study and it's fun and we have live music and it's amazing but i wanted something more like intimate and kind of down to earth and so that's why i'm doing these so that's the july cohort that's based on the book that i'm and i created new content for which is starting next week the other thing that's uh really tried and tested and

been out with i don't know how many we've done this we've probably done we do two or three a year since the pandemic started we used to do them in person now we do them remote is the master class and i'll just like literally double click on it for you here but basically the master class is what i do with alex and um those are three day events and they're pretty like high production quality like you can see what we're doing here so basically we have live music uh it's pretty crazy and uh people love

them like uh we keep it really high paced uh we've actually extended the time on these we used to do them they were a bit shorter and we realized people just wanted more so we extended an hour each day now um and that's happening in september so that's a more big experience if you want to like network with other entrepreneurs innovators and stuff around the world that's a great one to come to but if you're looking for something a little more intimate um the july thing hopefully i do more of these i'm kind of testing

it out right now but it's very promising i like this idea of having like a small group of people together it's kind of where i shine so um so yeah that's stuff that i'm working on if you're curious like how else to get value out of this stuff if you're really into it thank you thank you thank you yeah i have some some comments to say and of course i have some background in system building and so it starts from the design and then you build the system and you do verification and then go to

evaluation of the system and then verification and the validation then the evaluation where verification meant to build the system right and validation to build the right system and evaluation is like the proof of voting is in its eating an example of this is a very simple example example is building a house so you give you you you choose the design of the house you give the the the the design to the to the engineer to build the house and you go visit the house and to see if it is as you like or not it

is as you like so it is a validated or evaluated but it happens that the engineer built the house in the neighbor land not in the in in your land so it's you loot the everything so it's a a cycle that of course it's academic point of view there is a lot of details in in every step you know or during the system uh the the model is is nice and uh from uh and i like it i can enjoy the the the lecture very much and it was very useful but this is the comments

that you want to share with the audience yeah thank you um you know we often uh some of my workshops we show videos from frank gehry you know and how he worked and he would a really famous architect and he would talk about how he makes really rough models very quickly and with the clients and just like basically going wide so he would go for very many different models and then they narrow into something they go wide and narrow so this double diamond kind of thing from design thinking and one of the case studies i

have my book is actually it's a micro apartment so the guy built it's based in europe but he built um a prototype apartment that was very modular and then he had people test it so he would uh it was for business travelers and he would have business travelers come and stay in it and then he would test it with cleaning people and he would say is it hard to clean you know because he could have something that people like to stay in but then if people couldn't clean it it would be really expensive you know

and i really loved um um how he tested it like even though it was literally an apartment you know he kept just taking rounds and rounds of people through it and and leading people so i feel like um while while this is really uh common in software i am noticing other industries i'll obviously work this way and um yeah there are many different ways to approach it i think we're really trying to boil it down is this kind of like scientific method applied to the business basically i need to go david um thank you so

much for your opportunity for speaking with everyone