How to Find Product Market Fit - Stanford CS183F: Startup School

Today we have Peter Reinhardt, he's the CEO of Segment, went through YC when? >> 2011. >> 2011, and now Segment is doing extremely well. Peter is going to talk about product market fit, sort of this magic concept or word people use. I don't think they really understand it, certainly, the first job to get right in any startup. Peter, thank you very much for coming to talk to us, and I look forward to hearing this. >> My pleasure, all right, so I'm going to talk about finding product market fit today. Segment is a B2B company, so

that's probably why none of you have ever heard of us. But just to give you a little sense, we're about 150 people, and grew from about 4 people, 3 and a half years ago. So growing quickly, and hopefully I can shed some light on finding product market fit for you. I thought I'd just start by sort of looking back to Alan K's lecture last week. I think he had some pretty incredible advice around how to do really amazing research by sort of looking into the future and imagining what that future might be like. And in

particular, he had this diagram of sort of exploring the pink plane, and sort of testing out different ideas there. And then at some point, having a breakthrough and realizing that, actually, there's a blue plane, right, and you can sort of go explore an entirely new space. And that's a key breakthrough for sort of understanding how you can really invent that future. And in particular, he had this idea of sort of going into the future and imaging yourself there. So he had this Wayne Gretzky idea of skating to where the puck should be or will be,

and sort of going there, grabbing that future, and then pulling it back to the present. And the way that he talks about doing this is, I think, amazing for research. He talks about basically going and sort of imagining that future world. But I'd actually wager that not a single successful company has actually been founded by doing that. I think that's, again, awesome for research, but when it comes to founding companies, it actually has nothing to do with sort of imagining your vision for the future and what that future could be. It's not about sort of

how you wish the world was, but it's actually about what customers want. And I think a lot of companies get marketed post facto, of hey, we have this great idea, and here's how we deductively logic out what customers were going to want, and then we built the thing. But in reality, that's not how it works at all, right, in reality, it's a very inductive process. You're going back and forth with customers, and you eventually find something that works. And so I wanted to talk a little bit about that process today, and just how critical it

is, and actually, how difficult it is, too. So one of the most common mistakes that startups make is to build something that no one wants, right, or solve problems that no one has. And so that Wayne Gretzky method for sort of going into the future and imagining what that future might look like, again, is a good idea for research. It might even be a good idea for venture capitalists to sort of imagine what the future is like and sort of put startups into different buckets of what building that future might look like. But actually, I

think it's a really bad strategy for founders. And the reason is that most founders can actually pretty easily build something that looks like it's from the future, or is from the future, to some extent. It's easy to solve the technology problem there, but it's actually really difficult. That's not really what you need to do, right, you don't need to build something that is your vision of the future. You actually need to solve a specific problem that customers have today, here, now, and most startups actually fail to do that. Most startups actually build something that looks

vaguely futuristic, but is not, in fact, a real problem that people have today. And it will kill the company every time, right, the market always wins, so we're going to dig into that a little bit more. And this sort of problem of finding product market fit is that 80% of all founders fail to find product market fit. So four out of five attempts to found a company just fail at the sort of earliest stage of even finding a problem that you can solve in a unique way. And the last one out of five are going

to sort of struggle through the remainder of actually building that company out from there, but that's just the stats. I'll share our own story of finding product market fit. But the short version is that when you're not finding product market fit, it feels like sort of a bottomless emotional free fall. Not to get too dark, but when you're failing to find it, it sort of becomes a strange obsession, and it could really make you sick, actually. To get, maybe, too personal for a second, in 2012, in the midst of our search for product market fit,

I lost ten pounds in three weeks, and then went to the hospital twice for panic attacks. I haven't had a panic attack before or since then, and I think it's not just me, actually. I think Sam, I don't know if I should show this or not, but when he was founding Loot, I think he basically forgot to eat properly and ended up with scurvy, if I remember correctly. Sam is back there somewhere. So basically, be warned, finding product market fit is, very emotionally, a grind. And the reason it is is that you really, really desperately

want to find that fit, right, you really want to figure out what is going to work. What is something that customers want? We're all wired to want to succeed and help people like that. And so you're so intent on finding this thing that you sort of convince yourself into seeing mirages of it. And basically, you convince yourself, this is the thing, we're almost there, it's like this is something that people really want. And so this is a picture of myself with my three co-founders, Elliot, Calvin and Ian, in the summer of 2012, I think. And

this is in the midst of sort of our deepest false hopes that we had found something. So I'll dive into that story, and obviously, I'm here today because it worked out. I wouldn't be speaking if we had just disappeared into the darkness. But yeah, today we're about 150 people, and growing rapidly, yeah? >> How can you make sure you're finding a big market, and not just a small niche [INAUDIBLE]? >> Yeah, that's a good question, so [COUGH] I think PD's advice on this is good and valid, which is that it's actually harder to find the

first problem, and solve any problem well, than it is to find a route out. So finding a route out from some initial foothold into a broad market is actually not that hard. It sounds hard, but once you find any foothold, you can pretty much find something, and I can cover that a little bit in a bit, but yeah. >> Is that because you have more leverage with the customer? >> Yeah, it turns out that once you solve a problem for a customer, they'll keep bringing you more problems. All right, the most efficient thing for them

to do is just say, well, I have this other problem that's also adjacent to it, can you solve that? And you're like, sure, and you solve that, and you just keep solving adjacent problems. So once you find one thing, it's actually almost trivial to find the next thing, and the next thing, and the next thing, especially in selling to other businesses, where you have someone who can just tell you more stuff. Much harder in consumer, I'm the wrong guy to talk to you about consumer stuff, I don't get it. Cool, but back then, we were

just four dudes in an apartment, we were failing to find product market fit. We were convincing ourselves that maybe we had it, or we were on the cusp of it, and we were writing a lot of code. I saw someone back there who was writing code earlier. We were writing a lot of code, and we had no customers. That's the wrong order of operations, and so we were tricking ourselves into this. In reality, the good thing that was going on here, the reality was that we were a bunch of cockroaches. And the reason is that

it's very important, pre-product market fit, to basically save as much cash as possible, spend as little as possible and extend your runway as long as possible. So a lot of founders make a mistake of spending a bunch on sales, or marketing, or other things as soon as they have an idea. But in reality, until you find product market fit, until you find something that people want, you shouldn't be spending any time on that. You should just be spending time talking with customers and iterating. And you really want to constrain your runway, or constrain your burn

so that you can have a really long runway. So that's what we did, we kept our burn really low. I think we paid ourselves like, the minimum allowable amount by law, which was 20 or 30k a year. We lived in our apartment/office and like the company paid for part of it as an office, as it was the minimal thing. And we stretched out our runway through as many ideas as we could. And so to sort of make this process a little bit more explicit, step one is that you build, launch, and sort of iterate on

several different ideas. And this is where it's really important to be a cockroach, where you're sort of conserving everything you have. Then suddenly something magical happens, we'll talk about that. And this is product/market fit and it really does feel magical. Hopefully, I can illustrate to you and help you feel that. And then, three product/market fit, sort of, suddenly turns you into a unicorn/cockroach. And rather than just surviving, after product/market fit, everything gets suddenly easier, right? You can, not easy but easier. Customer show up and buy your thing, people want to join your team. You're still

pushing the boulder uphill if you will, but you're not totally constrained on progress, right? You can actually feel forward momentum for the first time, it's not like one step forward at the product idea, full step back when a customer doesn't care, you actually can keep iterating on a product that's starting to work. So, let's take that middle portion and deconstruct it a little bit. First we are going to deconstruct what we call category leaders. These are really large companies that have been very successful. Then we'll get to the heart of why product market fit is

so important at the very beginning. And we'll talk about what bad fit feels like. Bad product market fit. And then we'll talk about what good product market fit feels like. And we'll try to do that through stories because the goal here is to for you to walk away and be able to more easily identify which is which, with your own ideas. So category leaders. The reason we care about category leaders is that they're much, much larger than basically the rest of the companies out there. So I'll give you an example. Amazon and Facebook are sort

of consumer companies that you might be familiar with. Salesforce is a company that sells to other businesses that provide sales software basically. And the reason that we're interested in them is that they're huge, right. They're often ten or a thousand times larger than the next competitor in the space. So, Salesforce for example you've probably heard of SugarCRM and Zoho, are probably companies you've never heard of. They all do roughly the same thing. And so, we want to dig into how can you build a category leader, what's the layer underneath that. And basically what it comes

down to is building a platform, where it's not just a product that you're selling, but where the data inside of your product is actually useful for other businesses to build their business on top of yours. And that's what Salesforce has done, that's what Amazon has done, that's what Facebook has done. If you look at Salesforce, they have something called the appexchange, and the appexchange sort of reveals all of the data inside of Salesforce and allows other companies to build marketing and sales products on top of that. Similarly, Amazon has the reseller program, right? Where now

there's ton of other businesses, built on top of Amazon. And so, the key here is having some sort of platform, eventually, when you get the scale having some sort of platform that other companies are building on. Selling, and by nature that pulls people into your ecosystem. But, how do you build a platform like that? I know Peter Thiel may not be the most popular right now, but his opinion is to build a sustainable and compelling platform. You really need to get to 100 million in revenue. And the reason is just that you need to be

at a scale, where someone actually can capture a couple percentage points of your customer base and build a real business themselves. So if they can capture a couple percentage points, say, 2 or 3 million in revenue, that's a real business. You can build something on top of that. So the sequence I weigh here, you want to build a category leader, to do that you need a platform, to do that you need a 100 million in revenue. So let's keep digging. 100 million in revenue is tough, right? So, let's keep digging down. I think you met

Jason Lumpkin a couple of weeks ago, am I right? Maybe he is coming soon. He's written some amazing stuff about building software as a service businesses, would highly recommend reading all of his answers on Quora. And he basically breaks down the path to 100 million in revenue in sort of three different steps. He says zero to one million is impossible $1-10M is improbable and $10-100M is inevitable. And he says the $10-100M is inevitable, because at that point there is so much momentum. You have customers out there who are singing your praises, they're buying, their companies

are going to buy more. And it might take you a while, but eventually you're going to get from $10-100M. One to ten million I think is basically always a brutal grind for the founders, and the reason is that you're running a real business. You have real customers. You have some scale. Your customers have reasonable demands about sort of how, the quality of service that they expect from the product. But the problem is that pre-ten million in revenue, you have a really tough time attracting a world class management team. because you can't hire a great exec

team yet because the large enough market isn't there. And so, what you have is basically all the early crew sort of like holding the ship together and trying to make sure that it's going to work. So, that's a very tough period, I think, for most founders to go through. And then, zero to one million isn't possible because this is finding market product fit and 80% or more of people fail at that first step. And so the question is, how can you become not and how can you actually make it through that impossible section and become

one of the ones that succeeds? And one of the things that people talk about as investors or founders is how important it is to learn from failure. And I recently read a pretty good study talking about the stats of, sort of, the actual analysis of failure and success in startups and finding product market fit. And basically the research shows that you're actually no more likely to succeed the second time around if you failed the first time. So if you fail in finding product market fit the first time, your odds of success are still 20 or

22% the second time around. But, if you succeed in finding product market fit, the chances of your success the second time around go from 22% to 34% which is still miserable. But, it's at least 50% better. And so what I think that means is there's actually not that much information encoded in failing to find product market fit. Like you don't actually worry that much. But it's actually quite a bit encoded in feeling the success of actually understanding what did work. And so when we were struggling to find products market fit in 2011 and 2012, I

felt this really acutely. We had failed multiple times at this point, but I still didn't feel like I really knew what I was looking for. What is this mystical product market fit thing? So frustrating. We kept seeing glimmers of hope. We kept convincing ourselves that this, that or the other thing, like a visitor chatting with us on our site whatever was a big deal, or a vaguely interested sales prospect was product market fit. And so without any positive product market fit examples, we didn't really know what it looked like. And so we could sort of

convince ourselves of everything. And the way I think about it now is I really desperately if you imagine like a machine learning model, I had a bunch of negative training set examples but I had no positive training set examples. And so of course my machine learning algorithm was like I don't know. And so I'm not going to walk through three stories. But I'm going to show you two examples of failure, actually five stories, two examples of failure, and three examples on the positive side of times that we actually did feel product market fit. And again,

my goal is for you to actually feel what this feels like so that you can identify it in your own product. So today's segment is a customer data platform, but, we actually started as an education tool, and it was actually designed exactly for lectures like this. So, this is us coding in our Mountain View apartment, on a summer of 2011. And the idea was that, as a professor standing up talking up to a class, you have no idea if anyone in the audience actually understands what you're saying. And so we are students at that time

at MITA renounce school design. And we said we really want to do this it gives students a button to push where they can say, I'm confuse, right? Or I get it, either one. And the professor would see this graph overtime of how confuse the students were, might be helpful to me right now. And so, we built this. We were at hundreds of thousands of lines of code it had like commenting and notes and all sorts of crazy stuff, and we actually came to Stanford's campus, we've convinced them, it might have even been in this hall

Convinced some professors. We would like run up to them after class. This is a picture from Berkley, we pounced on this professor right after class, and we were testing for product market fit, right. We were trying to convince. Hey professor like did you get any feedback from your class during this class? No. Okay well we have a solution for you. So, we were hustling to try to get people to actually use this tool. But we we're mostly sort of ignoring any test of real product market fit there. And so professors would agree to test it

out for a few lectures sort of out of pity, maybe, for some students from MIT who were trying to help. And so basically we thought this was product market fit but it really wasn't. And I'll show you why. Because if you stand in the back of the classroom and look at what people actually had on their screens, none of them were using the product. Right, like people were using all these different things. This is that same class at Berkley the next week by the way. It was horrifying, and basically the students, as soon as students

opened their laptops they all went and did other things. And so basically putting a laptop into the classroom was the most distracting thing you could conceivably do. So as you can imagine, this was pretty horrifying. One of the more embarrassing things that could have happened. We had just raised 600k coming out of why Commonator Demo Day, and we had sold this vision of like, this is how the future of classrooms is going to work, right? Like, it's going to be digital, it's going to be online, much as this is a moot, etc. And it was

a great vision, but again the market wins every time. It doesn't matter where vision is, it matters what the market actually wants. And in this case, the students didn't care. All right, the students didn't actually get that much value out of using the tool. And actually if you go back, we should have had an even earlier warning sign which was that the professors didn't really want to use the tool either. Right? I mean you go and talk to the professors, they would sort of out of pity agree to test it for a few lectures. But

that is not the same thing as product market fit where they're like holy crap that solves this problem that I have. Bullying customers into using your product is not anything close to product market fit, even if they reticently agree to do it. I think that being dismissive of users and having your clear vision of the future that isn't necessarily solving a problem for your customers is a pretty stunning failure on our part, and is a key thing that founders do again and again and again in their search for product market fit. So then we had

to the awkward thing. Which by the way is the right thing of calling back all the investors and saying. This was like four weeks after they sent the checks right. By the way, turns out this is a terrible idea we are going to do something else. Do you want your money back? And in most of the cases the investors did take. Or sorry, did not take the money back. Instead, we invested for the team, like go find another idea. We believe in you guys. Go find something else. So he said, okay, let's do it. And

we were all very committed to working together as team or four founders. So we shut down the lecture tool. We went and sort of shut down all the classrooms, and then we went back to the whiteboard and we said, what is something that is sort of interesting here and we had always felt like we should have been able to determine that we didn't have product market fit, that the product usage wasn't there from our actual data. The way that we actually figured this out, right? Was we went and we stood in the classroom, back where

Sam is. And looked at what was on all the laptop screens. And that was how we figured out whether we had product market fit or not. But we should have been able to do that with the data. We should have been able to just look at the analytics and figure out not only are people using it or not, but are anthropology classes using it different than computer science classes? And so we decided to build basically an analytics tool, which it turns out as bad idea, in case anyone was considering that. And so the way that

we approach this is, how many of you have read the book Lean Start Up? Wow, that's actually awesome. You all are way far ahead. So as talks about you want to get out of the building and actually talk to customers, right. So we read that right around this moment and we're like, okay, we're going to go out and talk to customers. So we did that, and we tried to validate our ideas. So we'd take people out for coffee and we'd pull people from companies and try to figure out if they were interested in analytics products.

And I'd say again and again, they were vaguely interested, right? And they had willingness to meet and chat with us. And they said they wanted product updates as things came out. And so we thought this must be it, right? This must be product-market fit. People are interested in what we're doing. Again, this is not product-market fit. This is idle interest. Very, very big difference and so based on this sort of we got very excited and we spend about the next six months just writing codes and now this is in a new office, that's my co-founder

as you can see there's lots of code on our screen again, wrong order of operations. I had gone on one sales trip to visit potential customers. They were all pretty happy with mixed panel and Google Analytics, but that had these edge case features that they were hoping that we could solve. So, I sort of tricked myself into believing that these little edge cases that we might be able to solve were actually a really wide gap we could fill as a product. And so I came back, and I would sort of pitch my cofounders, and we

would keep sort of believing that we were almost there. If we just ship one more feature, one more thing, we're going to get to product market fit. Again, bad idea. Give you another one, we used lots of little positive interactions like this. You can also read this not as a positive interaction, but every once in a while, some stray person would visit our website, and they'd open a chat. This is idle interest. Hey, what segment? And there's a complete transcript of that website chat. You can see it's 3:00AM, so we're up late coding. And, yeah,

this person is interested. So maybe this is product market fit. Again this is not what it looks like this is ideal interest. So it's now December 2012, we've been at this for a year and a half and we decided that something was wrong. Right, clearly this was not working. And so we went back to YC, we emailed. A Paul Graham. And we said. Hey, we think we should catch up and sort of explain what's happened over the last year and a half. And so, okay, great. So we go. This is us in front of the

old YC building, and we're walking around the little cul de sac by YC on Pioneer Way. We bring him up to speed. Okay, we've spent half a million dollars. Here's everything we've been through over the last couple of years. And as we walk around, he comes to a stop. Aand looks at us, and he says, "so, just to be clear, we spent half a million dollars and you have nothing to show for it." totally accurate. And super fair and sort of like shocked us into, ****, yea, we gotta do something. And that was the moment

where we realized we had hit rock bottom. But we still have a 100k left so we still got one more shot, right? Alright, so that was all the product market failure cases, now you're going to see some successes. But let's rewind. Let's go all way back to the first week of YC. And in that first week, we have been like well, we have this class and lecture tool and we should have analytics on it, right? So, we looked at the different analytics tools, guest metrics, Google analytics, mixed panel. And we were looking at what's similar

here and we saw basically they have different graphs and different APIs. But it's actually the same data going into all these tools it's just that they give you different stuff out the other end. And we were like, well, we don't really want to make a business decision here about which tool we want to use. So we'll just solve the engineering problem because we're engineers and we'll just build some code that sends data to all three and does this automatic translation. So we put one data point in, gets translated into three API calls that go out

to all three services. Cool, this was like 100 lines of code in the hundreds of thousands that we wrote, right? Set that aside. And then now four months later, it gets improved a little bit, four months later, it gets improved a little bit. At that point we are trying to sell our own analytics tool, right? Akin to Mixpanel and Google Analytics. And we keep encountering the sales objection when we're trying to sell it, which is I already have Mixpanel installed and I don't really want to install your tools it seems like a lot of work.

So my co-founder Ilya has this great idea, he says, what if we take that little library we wrote a year ago that we've totally forgotten about and we add ourselves as the fourth service that it can send data to. And then every time someone hits us with that objection, we hit them back with the open source library and we say, okay, great now you can try both in parallel. And we use this like a growth hack, basically, to get customers to start adopting our tool. Okay great, we start doing that, start setting it up, people

start replying. This is awesome. I love the library, I'm definitely going to use it. A few weeks later we follow up. Hey, we saw you're not still sending any data to segment.io, what's going on? And they're like well, the library is fantastic. I just don't really want to use your analytics service. Should have taken note right then. But a few months go by, people start storing this on GitHub. Maybe it gets up to, it was a big deal for us at the time, like 30 stars. And I think there was maybe one pull request issued.

And then, fast forward some more, people keep sort of paying attention. It's the first time we'd ever felt pull. People were just finding this thing and doing something with it. And so fast forward, we have this conversation with PG, and the next day we sit down and we're like all right, we need a new idea, right? And so my co-founder Ian, he's like, you know what? I have an idea, you remember that analytics JS library that has been sort of idling on GitHub. I think that could be a big business. And I was like, you've

got to be kidding me, that's the worst idea I've ever heard. First of all, it's open source, and second of all, it's 580 lines of code so who the heck's going to pay for that? Right, how do you build a business around that, it makes no sense. And so we were like fighting and fighting and fighting and I went home and I was wracking my brains so like, how can I kill this idea? It's really bad. And it's going to sink us, we only have one more shot. And then, so I came in the next

day and I was like, all right, guys, here's what we're going to do. We're going to build a landing page. It's going to be awesome landing page. It's going to be beautiful. We're going to put it up on Hacker News. It's going to push the product and we'll have an Email sign up form at the bottom. And we'll use this to just test whether it's a good idea or not. They agree. Okay, great. Some I'm like, all right, totally done. We get ready to launch on Hacker News. I'm starting to think about other ideas and

it goes straight to the top. So it gets about 300 up votes on Hacker News, gets a few thousand stars on GitHub. We have people reaching out to us on LinkedIn demanding access to the beta. Like this guys says, what does a brother have to do to get bumped up on your beta list? And there were others like this, right? Like holy crap, so full stop, right? Compare this to everything previously, everything changed. This is what product market fail looks like, where it's not just a single metric, slowly starts moving, it's not just a few

random conversations where people express vague interest, right? Literally every single metric went totally haywire. And with our lecture tool and our analytics tool, we've been sort of searching in the dark for what features to build next. We did not have that problem any more, right? There were thousands of people who had signed up. And they're like, your seven integrations are good, but I need these ten more. I'm like, I"m like deploying it tomorrow, I'm like blah, blah, blah, we're like, holy crap. Like, okay, slow down. And that's one of the key things. One of the

key things is it flips from being something that you're like pushing against the customer to all of a sudden the customer's running away with it. And you're like, but like hold on, wait it's not quite ready yet. And so another example, with our analytics tool, we had this sort of sad unanswered questions and chats. No one really seemed to care about what we had built. But now all of a sudden we had thousands of stars, people were issuing pull requests. We got like ten pull requests in the first 48 hours or something like that. And

I guess the other key thins is with the lecture tool and our analytics tool, we had had this huge vision, right? We have a vision of like here's how the classroom should operate early. Here's company should do analytics. And then we went about trying to build a product that fit that vision. But this was the total opposite, right? This is like a little tiny library that we built for ourselves that solved the real problem. We had no vision associated with it whatsoever at the beginning. Now it does because we have something that we really want

to go accomplish. But at the beginning, it literally solves the tiniest of tiny problems and so, to your question earlier, right? Like, this is that tiny little foothold. And again, it's an open source library with 580 lines of code. That's a foothold, right? And since then now we've expanded greatly into doing all sorts of things and solving adjacent problems for customers. But the key is that, again, the market doesn't care at all what your vision of the world is. The market wants what it wants, and it will win every time. So if you walk away

with sort of one thing today, I think it's, let's be incredibly clear that basically product market fit doesn't feel like vague idle interest. It doesn't feel like sort of a glimmer of hope from some early conversation. It doesn't feel like a trickle of people signing up. It really feels like sort of everything in your business has gone totally haywire. There's just this big rush of adrenaline from customers starting to adopt it, and sort of ripping it out of your hands. And it really feels like the market is dragging you forward. I think that Dropbox founders

said this best actually, that product market fit feels like stepping on a landmine. And you really, you can't mistake the two. So if you are at all questioning whether you have product market fit or not, you don't. So, obviously that didn't stop us from making this mistake. So just to make sure that this is [INAUDIBLE], no, all right. So, I thought ClassMetric for sure had product market fit, right? Big vision. Market said no. Like, market doesn't care what you think. I thought Segment.io for sure had great product market fit. Again market said no, market doesn't

care, market wins. And even on our third attempt when we did find a product market fit, I thought for sure they were like, this is too tiny to matter, right? But actually it solved the real problem, and the market demanded it and sort of ripped it out of our hands. Either that goes to show sort of how obtuse I am or how hard it is to actually find product market fit. >> So how come people for something that was open source and only a couple lines of code? >> That's a good question. So it turns

out that the open source library by- >> [INAUDIBLE] Repeat the question. >> Sorry, the question is why would someone pay anything for an open source library that's 580 lines of code? It's a good question. It turns out that the open source library by itself doesn't totally solve the problem. So the actual problem that we found out that we were solving after we launched it was that marketing teams keep coming to engineering departments and saying I just signed a contract with a exact target in marketing tool or I just a contract with W Analytics and here's

the docs. And engineering team says, what the heck, I have a road map. Like, I need to be executing this whole thing. I can't do this analytics implementation. And so what really needed to happen and sort of what analytics suggest could solve the problem, that we could solve the hosted version it was allowing engineers to do a single implementation of collecting that data once. And then letting marketers just push buttons in our interface and send the data wherever they need it. But the open source library doesn't quite solve that problem. Engineers saw that it was

the right abstraction, but if a marketer needed a new tool and there is an open source version, the engineer still has to go compile it. So, in some ways we got lucky, that the open source version doesn't fully solve the problem. Cool, so I'm going to steal gratuitously from Allen K's slides again. And I think finding product market fit feels like finding the sort of blue plane, when you're in the pink plane. And I want to give some other examples of finding product market fit, after finding that initial product market fit. And so the key

difference from that initial breakthrough from what Alan Kay was talking about, is that rather than removing yourself from the world and trying to imagine what the future is like, you actually need to go out into the world and research what problems people really have. But then once you find that product, once you find that sort of blue plain, then things actually get enormously easier. That blue plane is basically a foothold into a totally new perspective. And I think Allen K talked about the value of perspective and context. sort of get this foothold into a whole

new way of thinking about the world, and new way of solving this problem for someone. And so, not only do you know what good fit feels, like but you are now operating in sort of green pastures. And you are approaching problems in ways that all the encumbrance and the pink plan don't think about in the world. So I wanted to give you two examples of finding product market fit, since that initial a win for us. A short intro to what Segment does, we basically help you collect data from mobile apps and websites, we pull it

up to Segment and we fan it out to all the different tools that you need. And, that's all you really need to know. One of the places we could send data was Amazon S3, so this is basically just a place to put all of your log files. And we started to notice that all of our business to customers, were using this one integration. They were all sending their data to S3, like you have to do something with the data. You don't just like collect log files, do something with the log file and you're like what

the heck is going on here. And so we went and visited five of our largest customers in New York about three years ago, and we said okay, you're using this integration, but what the heck are using it for? And for five out of five customers in a row they said "well, we have a data engineering team" that's taking data from the S3 bucket, converting it into CSV files and managing all the schema translation, and then uploading it into a data warehouse. Red shift. And the first time I heard that from a customer I was like,

okay, that's interesting and I took a note. Went to the second meeting, customer said exactly the same thing. I was like, that's weird, okay, take a note. Third conversation I was like, all right, this is getting ridiculous. Did you guys talk ahead of time or what happened here? And by the fifth time it's like, okay, well obviously I know what we need to build, we need to build a connection from this straight to red shift. So then the question is, we went and built that. And just to show you now, now we have a company

that axis here in millions of dollars in revenue. And so we have a real product that real people are using. And so what does product market fit look like at that point? So we're introducing a second product basically. Well, it looks like this. So you can tell when we introduced red shift. And basically again, almost every metric in the business goes nuts. So, it's very, very clear whether or not you have something, that is really transformative for your customers or not. One more story and then I will open it up for Q and A. So,

this is maybe about five months ago, we had five ideas for products that we thought might be exciting to our customers. So I went to visit the customer up a large company, up in the Pacific Northwest. And, I sat down with a data architect there and I said okay, here are the five ideas that I want to run through and see if they're interesting to you. So I went through the first idea and he was like, yeah, you know what? I totally get it. That sounds super valuable. And I went through the second one and

he's like, that's really cool. That's cool that you can do that. And like, that makes a lot of sense. And then third idea, fourth idea, same kind of thing. And so this is how I would summarize that, that's great, I totally understand what the value is. It means, yeah, that would be great. Doesn't care is what that is code for, that's someone being nice. And then on the fifth idea I said, hey, here's what we can do, blah, blah, blah, blah. And he said "wait sorry, you can do what"? >> [LAUGH] >> And I re-explained

it and he said, " interesting". He turns to his friend and says can you set up a follow up meeting with this team, this team and this team. We also need to tell Joe about this, because it could affect this other thing. That is the feeling of product market fit, which is like, you're like wait, wait, wait, no, no, no it doesn't exist yet, these are ideas we might like all of a sudden the customer is just going to rip it out of you. And so now you're on a tight timeline because the customer expects

that it already exist. So again, that's what product market fit actually feels like. And I think if you want to find product market fit and build one of these category leading companies, become one of the one out of five founders that actually does succeed in finding this. I think you just need to be really honest with yourself. That the sort of glimmers of false hope that you have are not the same, as customers actually ripping something out of your hands. And so yeah, you just need to be honest with yourself, that's the message. So, questions?

Yeah, Sam. >> I certainly agree, and it's been my experience that that is what product market fit is like. But when you're trying to find it, [COUGH] how do you even like have for what kind of ideas to try? When you're just sort of casting around looking for ideas. >> I do think that the bigger problem is not necessarily having the ideas. I think that everyone has lots of interesting ideas. I think the bigger problem is not killing the bad ideas fast enough. I think, actually I have the most respect for the Codecademy founders in

this respect. I think they tried 12 ideas in 7 weeks or something like that in the summer of YC, something totally absurd. But they legitimately tried them and they killed them so fast, and the four days before demo day, they started a new idea which was Codecademy. And then it worked. And on Codecademy on launch day, on demo day they had 300,000 users or something like that. Where like again, that's the landmine effect of nothing, nothing, nothing, nothing, nothing and they were so good about killing the ideas that weren't, they had no problem throwing out

something that wasn't going to work. So yeah, I'm not sure yet. I don't think there's any magic to finding the right idea. If you just kill the bad ideas fast enough, you'll probably find something. Yeah. >> Talk a little bit about so this seems like a [INAUDIBLE] lines of code. And it's solving a significant but relatively small, significant but small problem, I don't know if that's the way to describe it. How do you go about pricing something like that, when you're doing that customer discovery? >> Yeah so we didn't, the question is how do you

go about pricing something like this where it's just 500 lines of code, it seems like it solves a small problem for people. We thought it was a small problem for people, and we under priced hugely for a long time. It turns out that that problem that we, the 500 lines of code solves is actually really valuable. There are, we have lots of customers now that pay over $100,000 a year to solve that problem. And there's more to the product now than the 500 lines of code, a lot more. But, it's nevertheless is, the size of

the business problem has almost nothing to do with the amount of code written. And I think it did take us a little while to revalue things in that respect. But to answer your question, most directly we spent the first year just accumulating customers. So we just had lots of people adopting it, we got to maybe 1,000 or 2,000 companies sending data through Segment. And then we tried experimenting with pricing. And then I'd say the biggest kick in the **** we had to actually get to the right point here is we had a sales advisor. Who

said, there were reasonably sized companies using us like Live Nation and Party O and stuff like that. And so we would go to sales meeting at these companies, and the sales advisor would basically get me pumped up ahead of time. And he'd be like, Peter you have to ask for $120,000 a year. And I'm like, that's crazy, what are you talking about? And he's like, Peter, if you don't ask for $120,000 a year, I'm quitting as your sales advisor. It's like, well all right, here we go, right? And so I would ask, I'd turn beet

red and then they would negotiate it down to 18k a year. But if you keep doing it, eventually someone is like, well that seems reasonable, and so. >> [LAUGH] >> And you have to keep testing the value like that, right? because you don't know how much value you're delivering until you start asking for money. So I think if I had to do it over again I would start asking for money earlier, and I'd be a lot more comfortable with it. If you're solving a real business problem people are going to be happy to pay for

it. I'm not sure I totally answered your question but, yeah >> How do you know you're asking the right people about your product? What if there should maybe be a just a small shift in who you should be asking that you're not realizing, how did you? >> Yeah, so the question is, how do you know whether you're asking the right people? And how do you know whether you should slightly shift who you're talking to versus shift the product? Yeah, this is actually why it's so hard to find product market fit the first time. And why

it's so much easier the second time, once you have some product market fit. Because once you do have a defined customer set that you sell to, you'll pretty easily figure out by going back to those same people whether or not you're solving an adjacent problem for them. But its really hard, the thing that makes it super hard at the beginning is its like you have two things. And you can either slide the product or you can slide the market by talking to different people. And I'm not sure that there's a magic rule to know when

to slide which. But I think founders get slippery with themselves, they aren't totally honest with themselves about which thing they're doing when. So I think it's most important just to recognize that okay, if we make this shift, we are shifting the audience versus shifting the product. because I think otherwise you can shift the product and the audience simultaneously, and think you're doing well, but then realize you're not. So I'm not sure there's a slam-dunk solution to that, other than just being really honest with yourselves about which thing you're moving when. Any other questions? Last chance,

yeah. >> So you mentioned that you guys were trying to find proper markets. That before you guys found it there was a lot of moments that like kill this product, kill this product. How do you encourage people on your team to be like let's keep focusing on discovering the next thing? What do you self-talk, and what do you talk about as a product? >> Yeah, so I think there's a pretty big difference between finding that first product market fit, where you can kill a product, immediately move to a totally new audience. Codecademy tried a whole

bunch of different stuff, with a whole bunch of, speaking of switching audiences, they went from S and B, to programmers, to [SOUND] restaurants, all over the place. Later on, when you actually are building a company around it, and you have product managers, and each product manager is sort of searching for a big breakthrough. It's a little bit easier in that you can't shift the audience anymore, the audience is fixed. And so now it's a slightly more straightforward problem for sort of helping a product manager understand how to go talk to customers. And sort of reveal

what problems those customers have, and then actually solve them. So it's most training in that loop. And it comes from talking to either larger customers, or your best customers, or your worst customers. And sort of trying to push the boundaries around the core of very happy people have, to what other problems you can solve for them. Yeah? >> When you talk to your customer, what do you ask them? What's your script? >> You basically just, the question is, when you're talking to customers and trying to sort of understand what their problems are, what's the script?

I think the biggest mistake is, trying to pitch your existing ideas, there's a place for that. Right, when you have some product market in fit, and you want to test some of this things. But I think most founders, and certainly we made this mistake early on, of pitching and trying to sell. Rather that actually trying to understand what their problems were. So now when I talk to a customer, I just start by asking what their business problems are, right? Hey, person at retail company, do you spend a lot on Facebook? They're like yeah, yeah, yeah,

we spend a lot of Facebook. I'm like how do you measure whether that spend is efficient or not? And they're like well, we don't really know. Okay, well is that a problem? They're like yeah, yeah that's a big problem. Well okay, tell me why you haven't solved it. And they're like well, we haven't solved it because blah, blah, blah. And you just start digging in to these problems, and then you're actually really set up to do a sale then, right? You're like well, I have just the thing, right? And then they're like, holy crap. But

if you don't have a product yet, now you know exactly what problem you can fix, right? So I think it's more about listening and digging than it is about pitching an idea. Yeah? >> How do you find these customer? How do you be like hey, this is my customer, this company or is company? >> Yeah, so the question is how do you find these customers? At the beginning, it's hustle, it's just emailing people cold, it's getting interest through everyone you know. It's scratching and clawing your way through your social network and through introductions. After you

have customers in an area, then it gets easier, because you have people coming to you all the time, mostly if you're inbound. And so then there's a steady stream of people to talk to. But the initial piece, that's why people talk about hustle being such an important founder quality, is because no one's going to help you. You've just gotta go find them, right? LinkedIn, investors, friends, it's whatever you can claw together. Yeah? >> So at the beginning, when you lectured for these customer, when you're cold emailing them, how many percentage of people actually come back

to you? The question is, when you cold emailed customers, what percentage of people come back to you? I don't remember, that was awhile ago, very low, single digits percentages maybe. You're going to send a lot emails that are unanswered. But that's the nature of sales. Honestly, if you're in a business to business kind of environment, you're going to send a lot of emails that go unanswered. Sales is basically the exercise of getting the door slammed in your face nine times out of ten, right? Yeah? >> What sort of second, where do you think would be

in five years? >> Go back to diagram. So what we have today is basically sort of stream processing. So we have both the ability to collect data across some broad section of stuff here. As well as the ability to fan it out to a bunch of different places. And so what we want to do is basically provide a platform that all of these tools can build on top of. That's the super short version. So you can imagine basically every team in a company needs access to customer data. And we want to become the platform on

which those tools are built, whether that's email marketing, or push notifications, or analytics, or help desks, or CRMs or payment systems, fraud detection, etc. All of those things operate on a core set of customer data, and that's the customer data that already flows through Segment by nature of us being that integration connection layer. And so we just want to expose that data to partners to build on top of it. So when I talked about building a platform, at the beginning, that's the next step for us. >> Peter, we are out of time, thank you very

much. >> [APPLAUSE]